An improved target detection algorithm based on a single-stage network model

Abstract

The present invention discloses an improved target detection algorithm based on a single-stage network model. The method uses a deep residual network Resnet-50 to replace the basic network VGG-16 in the original algorithm, and uses 7 feature layers for detecting targets; combines deconvolution with feature fusion, fuses the extracted high-level feature layer with the underlying feature layer, and the fused new feature layer also contains rich context information to improve the detection capability; designs a denser detection frame for detecting targets, and adds batch normalization operations after each convolution layer in the algorithm to increase the model training speed, prevent overfitting, and achieve the effect of improving the detection capability of the algorithm. It can improve the problems of false detection and missed detection, and meet the demand for improving the detection capability.

Description

Technical Field

[0001] The invention belongs to the field of digital image processing, relates to target detection, and in particular to an improved target detection algorithm based on a single-stage network model. Background Art

[0002] Object detection is a very important research problem in the field of computer vision. Its tasks can be divided into two parts. The first is to distinguish the category and confidence of the input object; the second is to detect the position of the input object, that is, "recognition" and "positioning". It is also the premise for advanced visual tasks such as scene content understanding, and is applied to infrared detection technology, intelligent video surveillance, remote sensing image target detection and medical diagnosis. Object detection algorithms have been developed to date and can be divided into two categories: one is the traditional object detection algorithm, and the other is the object detection algorithm based on deep learning. Typical algorithms of traditional object detection algorithms include SIFT algorithm and VJ detection algorithm, but the candidate region extraction method of sliding window is not well targeted, has high time complexity and generates many redundant frames; at the same time, the extracted features are mostly around the underlying features, and the robustness is poor. Object detection algorithms based on deep learning are divided into two-stage object detection algorithms and single-stage object detection algorithms according to whether there is a mechanism to generate candidate regions. Its typical algorithms include R-CNN algorithm, Faster R-CNN algorithm, YOLO algorithm, SSD algorithm, etc. Although there are many excellent target detection algorithms at this stage, the detection performance still has many shortcomings, which leads to problems such as missed detection and false detection. Summary of the invention

[0003] In view of the defects or shortcomings of the above-mentioned prior art, the object of the present invention is to provide an improved target detection algorithm based on a single-stage network model.

[0004] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0005] An improved target detection algorithm based on a single-stage network model, characterized in that it includes the following steps:

[0006] Step 1) Use the deep residual network Resnet-50 to replace the base network VGG-16 in the original SSD algorithm. After the replacement, remove the fully connected layer of Resnet-50, and add three convolutional layers to the Resnet-50 network after removing the fully connected layer; process the input image through convolution and pooling operations to obtain multi-scale feature layers with successively smaller sizes, where:

[0007] The large-size feature layer is the bottom feature layer, which is used to detect small-scale objects in the input image;

[0008] The feature layer with small size is a high-level feature layer, which is used to detect medium and large-scale objects in the input image;

[0009] The basic network of the original SSD algorithm is modified to a deep residual network Resnet-50, which uses 2 low-level feature layers and 5 high-level feature layers for detection;

[0010] Step 2), through the deconvolution operation, the high-level feature layer used for target detection in step 1) is enlarged and fused with the bottom feature layer. The fused feature layer contains both semantic information and detail information;

[0011] In step 3), a denser detection box is designed for target detection. At the same time, a batch normalization operation is added after each convolutional layer in the algorithm to increase the model training speed and prevent overfitting.

[0012] According to the present invention, the input image size in step 1) is 224×224; the sizes of the two bottom feature layers are 56×56 and 28×28 respectively; the sizes of the five high-level feature layers are 14×14, 7×7, 5×5, 3×3, and 1×1 respectively; a deconvolution operation is performed on the smaller high-level feature layer to enlarge the size of the high-level feature layer, and the high-level feature layer is fused with the larger bottom-level feature layer through feature fusion.

[0013] Furthermore, in step 3), a total of 17,340 frames are extracted from the bottom feature layer and the high-level feature layer extracted in step 1) and step 2) for detection.

[0014] Preferably, the specific detection algorithm process is as follows:

[0015] a) Using the idea of ​​regression, we directly regress the category and bounding box of the target on the input image through a convolutional neural network. First, we use the deep residual network Resnet-50 to replace the basic network VGG-16 in the original algorithm, remove the original fully connected layer of Resnet-50, and add three convolutional layers after the Resnet-50 network after removing the fully connected layer.

[0016] Since the bottom feature layer in the algorithm has not undergone a large number of convolutional pooling operations and has rich detail information, while the high-level feature layer has richer semantic information, the bottom feature map is used to detect small targets, and the high-level feature map is used to detect medium and large targets. The algorithm extracts 4 feature layers in the deep residual network Resnet-50, and the newly added 3 convolutional layers are also used as the extracted feature layers for target detection;

[0017] b) Combining deconvolution with feature fusion, the feature layers at different levels are fused. The deconvolution formula is:

[0018] d=s(i-1)+k-2p (1)

[0019] Where d is the deconvolution output feature layer, s is the step size, i is the input feature layer, k is the convolution kernel size, and p is padding;

[0020] Through deconvolution operation, the rich semantic information in the high-level feature layer is mapped to the underlying feature layer, so that the underlying feature layer has sufficient semantic information while having a large amount of detailed information, thus enhancing the recognition of the network;

[0021] c) The 7 feature layers extracted for target detection set detection boxes of different sizes and aspect ratios in each unit. The scale of the candidate box is calculated according to the following formula (2):

[0022]

[0023] In the formula, m represents the number of feature layers; s k Indicates the ratio of the candidate box to the image; s max and mi n represents the maximum and minimum value of the ratio, s max The value is 0.9, s min The value is 0.2; the scale of each candidate box is obtained using the above formula (2);

[0024] For the aspect ratio, the general value is And the width of the candidate box is calculated according to the following formula (3): and height Perform the calculation:

[0025]

[0026] For the candidate box with an aspect ratio of 1, a scale is added. The candidate box has the center coordinates of:

[0027] where |f k | represents the size of the feature layer;

[0028] d) Use a 3×3 convolution kernel to detect the category and confidence of the feature layer through convolution operation, and train the detection model. When the detection model is trained, the loss function is defined as the weighted sum of the position loss (localization loss, loc) and the confidence loss (conf). The calculation formula is shown in the following formula (4):

[0029]

[0030] Where N is the number of matched candidate boxes; x∈{1,0} indicates whether the candidate box matches the real box. If it matches, x=1, otherwise x=0; c is the category confidence prediction value; g is the position parameter of the real box; l is the position prediction value of the predicted box; α is the weight coefficient, which is set to 1;

[0031] For the position loss function in SSD, Smooth L1 loss is used to regress the center (cx, cy) and width (w) and height (h) offsets of the candidate box. The formula is as follows:

[0032]

[0033]

[0034]

[0035] For the confidence loss function in SSD, the typical softmax loss is used, and its formula is:

[0036]

[0037] The improved target detection algorithm based on the single-stage network model of the present invention is based on the SSD algorithm. When extracting the feature layer of the input image, the size of the feature map resolution and the influence of the feature extraction network depth on the detection performance are taken into account, and the basic network of the algorithm is replaced. The deep residual network Resnet-50 is used to replace the basic network VGG-16 in the original algorithm, and 7 feature layers are selected to detect the target. Combined with deconvolution and feature fusion, the extracted high-level feature layer is fused with the bottom-level feature layer and a denser detection frame is designed to improve the detection capability. At the same time, a batch normalization operation is added after the convolution layer in the algorithm, which speeds up the training model and effectively prevents overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a schematic diagram of the extracted network structure after replacing the base network VGG-16 in the original SSD algorithm with the deep residual network Resnet-50;

[0039] Figure 2 Figure 1 is a comparison of the detection effects of the original SSD algorithm and the improved target detection algorithm, where (a), (c), (e), (g), (i), and (k) are the detection images of the original SSD algorithm; (b), (d), (f), (h), (j), and (l) are the detection images of the target detection algorithm;

[0040] The present invention is further described in detail below with reference to the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0041] The improved target detection algorithm based on the single-stage network model of the present invention adopts the technical idea of ​​analyzing the deficiencies in the algorithm based on the single-stage target detection algorithm SSD, and proposing an improved SSD target detection algorithm. Use the deep residual network Resnet-50 to replace the basic network and extract better feature maps; analyze the role of each layer of feature maps, and add a layer of underlying feature layer for detecting small targets; combine deconvolution and feature fusion to fuse the extracted high-level feature layer with the underlying feature layer, so that the new feature layer has richer contextual information; design a denser detection frame for detection; add batch normalization operations after each convolution layer to increase the model training speed and prevent overfitting. By improving the algorithm, the detection capability of the target detection algorithm can be improved, and problems such as false detection and missed detection can be improved.

[0042] This embodiment provides an improved target detection algorithm based on a single-stage network model, including the following steps:

[0043] Step 1) Use the deep residual network Resnet-50 to replace the base network VGG-16 in the original SSD algorithm. After the replacement, remove the fully connected layer of Resnet-50, and add three convolutional layers after the Resnet-50 network after removing the fully connected layer, to obtain the extraction network after replacing the base network VGG-16 in the original SSD algorithm with the deep residual network Resnet-50 (see Figure 1 ). The input image is processed through convolution and pooling operations to obtain multi-scale feature layers with successively decreasing sizes. The feature layer with large size is the bottom feature layer, which is used to detect small-scale targets in the input image, and the feature layer with small size is the high-level feature layer, which is used to detect medium- and large-scale targets in the input image.

[0044] In the original SSD algorithm, 1 bottom feature layer with a size of 38×38 and 5 high-level feature layers with sizes of 19×19, 10×10, 5×5, 3×3, and 1×1 are used, for a total of 6 feature layers for object detection.

[0045] In this embodiment, the basic network of the original SSD algorithm is modified into a deep residual network Resnet-50, which uses 2 layers of bottom feature layers with sizes of 56×56 and 28×28, and 5 layers of high-level feature layers with sizes of 14×14, 7×7, 5×5, 3×3, and 1×1, for a total of 7 feature layers for detection.

[0046] Compared with the original SSD algorithm, the algorithm after improving the basic network uses one more underlying feature layer for detection.

[0047] Step 2), deconvolution operation, can solve the problem of reduced resolution of feature layer after a series of convolution and pooling operations, and expand the receptive field. Through deconvolution, the high-level feature layer used for target detection in step 1) is enlarged and fused with the bottom-level feature layer. The fused feature layer includes the advantages of both high-level feature layer and bottom-level feature layer, including both rich semantic information and rich detail information, which enhances the recognition ability of the network.

[0048] In step 3), a denser detection box is designed for target detection. At the same time, batch normalization is added after each convolutional layer in the algorithm to increase the model training speed and prevent overfitting, thereby improving the detection capability of the algorithm.

[0049] In step 1), the deep residual network Resnet-50 is used to replace the basic network VGG-16 in the original algorithm, and a total of 7 feature layers for detection are extracted. The input image size is 224×224, and the extracted feature layer sizes are 56×56, 28×28 (bottom-level feature layer), 14×14, 7×7, 5×5, 3×3, and 1×1 (high-level feature layer).

[0050] In step 2), a deconvolution operation is performed on the smaller high-level feature layers (14×14, 7×7, 5×5, 3×3, 1×1) to enlarge the size of the feature layer, and the high-level feature layers are fused with the larger low-level feature layers (56×56, 28×28) through feature fusion.

[0051] In step 3), a total of 17,340 frames are extracted from the 7 feature layers extracted in steps 1) and 2) for detection, and a batch normalization operation is added after each convolution layer in the algorithm to improve the model training speed; through the convolution operation, a 3×3 convolution kernel is used to detect the category and confidence respectively and train the detection model.

[0052] The detection process of the improved target detection algorithm is as follows:

[0053] a) Object detection based on a single-stage network model uses the idea of ​​regression to directly regress the category and bounding box of the object on the input image through a convolutional neural network. First, the deep residual network Resnet-50 is used to replace the basic network VGG-16 in the original algorithm, and the original fully connected layer of Resnet-50 is removed. After removing the fully connected layer, 3 convolutional layers are added to the Resnet-50 network; the algorithm extracts 4 feature layers conv1_x, conv2_x, conv3_x, and conv4_x from the deep residual network Resnet-50 for detection, and the newly added 3 convolutional layers conv5_x, conv6_x, and conv7_x are also used as feature layers for detection. A total of 7 feature layers are extracted for detection.

[0054] If the target stored in the bottom feature layer for detection is small, then after a series of complex convolution operations on the bottom feature layer, the target information retained on the high-level feature layer will become less, and the detection will be less sensitive. Therefore, in the SSD algorithm, the bottom feature layer contains rich detail information for detecting small targets, and the high-level feature layer contains more semantic information for detecting medium and large targets. Therefore, the improved algorithm uses 2 layers of bottom feature layers with sizes of 56×56 and 28×28, and 5 layers of high-level feature layers with sizes of 14×14, 7×7, 5×5, 3×3, and 1×1.

[0055] b) Combining the idea of ​​deconvolution and feature fusion, the feature layers at different levels are fused. The deconvolution formula is:

[0056] d=s(i-1)+k-2p (1)

[0057] In the formula, d is the deconvolution output feature layer, s is the step size, i is the input feature layer, k is the convolution kernel size, and p is the padding. Through the deconvolution operation, the rich semantic information in the high-level feature layer can be mapped to the underlying feature layer, so that the underlying feature layer has a large amount of detailed information while having sufficient semantic information, which enhances the recognition of the network.

[0058] The extraction network (hereinafter referred to as the feature map extraction network) after replacing the base network VGG-16 in the original SSD algorithm with the deep residual network Resnet-50, the extracted 7 feature layers have sizes of 56×56, 28×28, 14×14, 7×7, 5×5, 3×3, and 1×1, respectively. As described in a), the bottom feature layer contains rich detail information, and the high-level feature layer contains rich semantic information, resulting in different detection effects for large and small targets in different feature layers. Therefore, combined with feature fusion and deconvolution, the extracted conv4_x feature layer of size 7×7 is deconvolved, and the two feature layers are directly added through the element sum fusion method without changing the number of channels of the feature layer. The amount of information in the same dimension of the fused new feature layer is increased; since the replacement of the base network VGG-16 and the feature fusion operation itself introduce a large number of additional parameters and calculations, in order to minimize the amount of calculation, this fusion method is selected for fusion.

[0059] c) The 7 feature layers extracted by the feature map extraction network for detection set detection boxes of different sizes and aspect ratios in each unit. The scale of the candidate box is calculated according to the following formula:

[0060]

[0061] Among them, m represents the number of feature layers; s k Indicates the ratio of the candidate box to the image; S max and S min Represent the maximum and minimum values ​​of the ratio respectively; S max and S min The values ​​are 0.9 and 0.2 respectively. The scale of each candidate box can be obtained using the above formula (2).

[0062] For the aspect ratio, the general value is The width and height of the candidate box can be calculated according to the following formula (3):

[0063]

[0064] For a candidate box with an aspect ratio of 1, a scale of The candidate box, the center coordinates of the candidate box are i,j∈[0,|f k |),|f k | represents the size of the feature layer.

[0065] Combining formula (2) with formula (3), the scale and aspect ratio of the detection frame extracted by the improved algorithm are calculated; 4, 4, 6, 6, 6, 4, and 4 different detection frames are set for each unit in the extracted 7 feature layers, and a total of 17340 detection frames are used for detection.

[0066] d) Use a 3×3 convolution kernel to detect the category and confidence of the feature layer through convolution operation, and train the detection model. The loss function during model training is defined as the weighted sum of the position loss (localization loss, loc) and the confidence loss (conf), and the calculation formula is as follows:

[0067]

[0068] Where N is the number of matched candidate boxes; x∈{1,0} indicates whether the candidate box matches the real box. If it matches, x=1, otherwise x=0; c is the category confidence prediction value; g is the position parameter of the real box; l is the position prediction value of the predicted box; α is the weight coefficient, which is set to 1.

[0069] For the position loss function in SSD, Smooth L1 loss is used to regress the center (cx, cy) and width (w) and height (h) offsets of the candidate box. The formula is as follows:

[0070]

[0071]

[0072]

[0073] For the confidence loss function in the SSD algorithm, the typical softmax loss is used, and its formula is:

[0074]

[0075] Then the improved target detection algorithm model is trained: In this embodiment, the PASCAL VOC2007 dataset and the PASCAL VOC2012 dataset are used as the datasets for model training, and data augmentation technology is used to expand the training set images by performing operations such as horizontal flipping, random cropping, and color distortion on the dataset.

[0076] Data used in the experiment: PASCAL VOC dataset, a standardized dataset for image recognition and classification, contains 20 categories, namely people, birds, cats, cows, dogs, horses, sheep, airplanes, bicycles, ships, buses, cars, motorcycles, trains, bottles, chairs, dining tables, potted plants, sofas, and TVs. The VOC2007 dataset contains 9,963 annotated images with a total of 24,640 annotated objects; the VOC2012 dataset contains 11,540 annotated images with a total of 27,450 annotated objects.

[0077] This embodiment uses the above-mentioned VOC2007 dataset and VOC2012 dataset for training, and uses the VOC2007 dataset for testing. Stochastic gradient descent (SGD) is used for training, batch size is set to 32, initial learning rate is set to 0.001, momentum parameter momentum is set to 0.9, learning rate is reduced by 90% when the number of iterations is 180000 and 230000, and a total of 280000 training times are performed.

[0078] In order to verify the detection effect of the improved target detection algorithm based on the single-stage network model of this embodiment, the applicant selected the test set in the PASCAL VOC2007 dataset for detection, and compared the detection effect with other target detection models from both subjective and objective aspects (see Table 1).

[0079] Table 1

[0080] Tab.1Comparison of VOC2007

[0081]

[0082] Objective evaluation uses mAP (mean Average Precision) as the evaluation indicator. mAP is composed of precision and recall. The formula is as follows:

[0083] P precisi o n =TP / TP+FP (9)

[0084] P recall =TP / TP+FN (10)

[0085] In the formula, TP represents the samples where the positive samples are correctly identified; FP represents the samples where the negative samples are incorrectly identified as positive samples; and FN represents the samples where the positive samples are incorrectly identified as negative samples.

[0086] In the improved object detection algorithm, each detected category will get a curve consisting of precision and recall (PR curve). The area under the curve is the average precision (AP value), which is an evaluation index for measuring the model in a single category. The AP values ​​of all detected categories are averaged to get the mAP value, which is used to measure the quality of the model in all categories. The mAP value calculation formula is as follows:

[0087]

[0088] In the formula, Q represents the total number of categories in the detection, and AP(q) represents the AP value of the qth category in the detection.

[0089] In the objective evaluation of detection effect, the mAP value of the improved target detection algorithm has been significantly improved compared with other algorithms. However, due to the complexity of the network, the large number of detection boxes, and the introduction of other additional calculations, the detection speed fps has decreased.

[0090] In the subjective evaluation of the detection effect, the original SSD algorithm and the improved detection algorithm are compared (such as Figure 2 As shown in the figure, (a), (c), (e), (g), (i), and (k) are the original SSD algorithm detection pictures; (b), (d), (f), (h), (j), and (l) are the target detection algorithm detection pictures). It can be seen from the figure that the improved target detection algorithm has a more accurate detection box position and a higher confidence level in detecting the same target when compared with the original SSD algorithm; at the same time, it significantly improves the problems of false detection and missed detection in the original algorithm, and also has a certain detection effect under complex background conditions, and the detection effect is significantly improved compared with the original SSD algorithm.

Claims

1. An improved target detection algorithm based on a single-stage network model, characterized in that: The following steps are involved: Step 1) Use the deep residual network Resnet-50 to replace the base network VGG-16 in the original SSD algorithm. After the replacement, remove the fully connected layer of Resnet-50, and add three convolutional layers to the Resnet-50 network after removing the fully connected layer; process the input image through convolution and pooling operations to obtain multi-scale feature layers with successively reduced sizes, where: The large-size feature layer is the bottom feature layer, which is used to detect small-scale objects in the input image; The feature layer with small size is a high-level feature layer, which is used to detect medium and large-scale objects in the input image; The basic network of the original SSD algorithm is modified to a deep residual network Resnet-50, which uses 2 low-level feature layers and 5 high-level feature layers for detection; Step 2), through the deconvolution operation, the high-level feature layer used for target detection in step 1) is enlarged and fused with the bottom feature layer. The fused feature layer contains both semantic information and detail information; Step 3), design a denser detection box for target detection, and add batch normalization operations after each convolutional layer in the algorithm to increase the model training speed and prevent overfitting; The specific detection algorithm process is as follows: Combining deconvolution with feature fusion, feature layers at different levels are fused. The deconvolution formula is: d=s(i-1)+k-2p(1) Where d is the deconvolution output feature layer, s is the step size, i is the input feature layer, k is the convolution kernel size, and p is padding; Through deconvolution operation, the rich semantic information in the high-level feature layer is mapped to the underlying feature layer, so that the underlying feature layer has sufficient semantic information while having a large amount of detailed information, thus enhancing the recognition of the network; The 7 feature layers extracted for target detection set detection boxes of different sizes and aspect ratios in each unit. The scale of the candidate box is calculated according to the following formula (2): In the formula, m represents the number of feature layers; s k Indicates the ratio of the candidate box to the image; s max and min Represents the maximum and minimum values ​​of the ratio, s max The value is 0.9, s min The value is 0.2; the scale of each candidate box is obtained using the above formula (2); For the aspect ratio, the value is And the width of the candidate box is calculated according to the following formula (3): and height Perform the calculation: For the candidate box with an aspect ratio of 1, a scale is added. The candidate box has the center coordinates of: where |f k | represents the size of the feature layer; The category and confidence of the feature layer are detected by convolution operation using a 3×3 convolution kernel, and the detection model is trained. When the detection model is trained, the loss function is defined as the weighted sum of the position loss and the confidence loss. The calculation formula is shown in the following formula (4): Where N is the number of matched candidate boxes; x∈{1,0} indicates whether the candidate box matches the real box. If it matches, x=1, otherwise x=0; c is the category confidence prediction value; g is the position parameter of the real box; l is the position prediction value of the predicted box; α is the weight coefficient, which is set to 1; L conf is the confidence loss; L loc is the position loss; For the position loss function in SSD, Smooth L1 loss is used to regress the center (cx, cy) and width and height offsets of the candidate box; the formula is as follows: For the confidence loss function in SSD, the typical softmax loss is used, and its formula is:

2. The algorithm according to claim 1, characterized in that In step 1), the input image size is 224×224; the sizes of the two bottom feature layers are 56×56 and 28×28 respectively; the sizes of the five high-level feature layers are 14×14, 7×7, 5×5, 3×3, and 1×1 respectively; a deconvolution operation is performed on the smaller high-level feature layer to enlarge the size of the high-level feature layer, and the high-level feature layer is fused with the larger bottom-level feature layer through feature fusion.

3. The algorithm according to claim 1, characterized in that In step 3), a total of 17,340 frames are extracted from the bottom feature layer and the high-level feature layer extracted in step 1) and step 2) for detection.

Images