A target detection method based on improved YOLOv3

Abstract

The application discloses a target detection method based on improved YOLOv3, comprising the following steps: S1, selecting an image data set; S2, using greyscale, binarization, perspective transformation and edge segmentation to pre-process the image, and performing scaling processing on the image; S3, putting the pre-processed image into a MobileNetV2Plus backbone network to extract information; S4, selecting three feature maps of different sizes as prediction inputs, entering a feature fusion layer, and performing upsampling and splicing on the three feature maps of different sizes, and then performing convolution, and inputting the three feature maps into output layers of three different prior boxes respectively; and S5, performing convolution again on the three feature maps of different channel numbers input into the feature fusion layer, obtaining classified feature maps, and generating three prior boxes of different dimension sizes. The application only adopts a 1*1 convolution kernel, reduces the calculation dimension, and accelerates the recognition speed.

Description

TECHNICAL FIELD

[0001] The application relates to a target detection method, in particular to a target detection method based on an improved YOLOv3. BACKGROUND

[0002] In recent years, artificial intelligence technology has been widely applied, and in particular, a deep learning model based on a neural network has played a significant role in target detection and image segmentation.

[0003] Since the deep convolutional neural network model has stood out in the ImageNet large-scale image recognition competition (ILSVRC2012), the research of deep learning has entered a rapid development stage. At present, the deep convolutional neural network target detection algorithms with outstanding effects mainly include two categories: the first category is a target detection algorithm based on a candidate region, such as R-CNN (Region-CNN), FastR-CNN, Faster R-CNN and Mask R-CNN, etc., but the detection speed of this kind of target detection algorithm is slow and cannot realize real-time detection; the second category is a target detection algorithm based on regression, such as SSD, YOLO, YOLOv2 and YOLOv3, etc., and the detection speed of this kind of target detection algorithm is fast and can realize real-time detection.

[0004] However, the above network has the problems of low recall rate, low precision rate, low mAP value, large memory occupation and slow recognition rate. Although most network model calculation methods have relatively large universality, they cannot implement rapid deployment for specific tasks and cannot balance the detection accuracy and the detection speed. SUMMARY

[0005] The purpose of the application is to provide a target detection method based on an improved YOLOv3, which can realize balanced detection accuracy and detection speed.

[0006] Technical scheme: The target detection method of the application comprises the following steps:

[0007] S1, selecting image data sets;

[0008] S2, pre-processing the image by using gray-scale processing, binarization, perspective transformation and edge segmentation, and performing scaling processing on the image;

[0009] S3, putting the pre-processed image into a MobileNetV2 Plus backbone network to extract information;

[0010] S4, select three different size feature maps as prediction input into the feature fusion layer; and up-sample and splice the three different size feature maps, and then perform convolution to obtain three feature maps with output channels of 1024, 512 and 256, which are respectively input into the output layers of the large, medium and small three different prior boxes;

[0011] S5, the three different channel feature maps input into the feature fusion layer are convolved again to obtain classified feature maps, and three prior boxes with different dimensions are generated;

[0012] S6, the model is calculated by using the loss function, and the parameters are updated.

[0013] Further, in step S2, the Resize bilinear interpolation method is used to scale and transform the image.

[0014] Further, in step S3, the DW module used in the MobileNetV2 Plus network is composed of 1*1 size convolution kernels.

[0015] Further, in step S4, a 7*7*320 feature map is used as the large prior box prediction input, a 14*14*192 feature map is used as the medium prior box prediction input, and a 28*28*64 feature map is used as the small prior box prediction input;

[0016] The large prior box prediction input passes through the DW module and outputs to the Head large prior box layer, and submits the 7*7*320 feature map to the next level of sampling;

[0017] The medium prior box prediction input splices the 7*7*320 feature map input from the previous layer, passes through the DW module and splices the 14*14*320 feature map after down-sampling; then passes through the DW module to output the 14*14*512 feature map to the head medium prior box layer, and submits the 14*14*512 feature map to the next level of sampling;

[0018] The small prior box prediction input splices the 14*14*512 feature map input from the previous layer, passes through the DW and down-samples to obtain the 28*28*512 feature map; and passes through the DW module to output the 28*28*576 feature map to the head small prior box layer.

[0019] Further, in step S5, the Head large prior box layer enters the DW module and the conv layer, performs 1 / 32 sampling, and outputs a 13*13*255 feature map; the Head large prior box layer enters the DW module and the conv layer again, performs 1 / 16 sampling, and outputs a 26*26*255 feature map; and the Head large prior box layer enters the DW module and the conv layer again, performs 1 / 8 sampling, and outputs a 52*52*255 feature map.

[0020] Further, in step S6, a weight factor is added before the cross entropy to form a new loss function FL(P t ):

[0021] FL(P t )=-(1-P t ) γ log(P t )

[0022] Wherein, gamma is a weight parameter, log() is a cross entropy loss function, P t The value is the confidence of the target object;

[0023] And add weight alpha1 to the easy-to-distinguish sample and add weight alpha2 to the difficult-to-distinguish sample, so:

[0024] Loss=alpha1*FL 易区分 +alpha2*FL 难区分

[0025] Wherein, FL 易区分 Is an easy-to-distinguish sample, and the confidence is close to 1 or close to 0; FL 难区分 Is a difficult-to-distinguish sample, and the confidence is near 0.5.

[0026] Compared with the prior art, the present application has the following remarkable effects:

[0027] 1、The present application adopts a more lightweight backbone network MobileNetV2, and modifies it to propose a lightweight backbone network MobileNetV2 Plus, reduces the weight model size; and adjusts the loss function, uses the Focalloss loss function instead of the loss function in YOLOv3, improves the problem that the background sample is wrongly checked as an article in YOLOv3, so that the improved algorithm has better robustness;

[0028] 2、The present application reduces the feature scale, only uses 1*1 convolution kernel, reduces the calculation dimension, and speeds up the recognition speed. DETAILED DESCRIPTION

[0029] Figure 1 is a flowchart of the present application;

[0030] Figure 2 is a MobileNetV2 Plus model diagram;

[0031] Figure 3 is a MobileNetV2 Plus backbone network schematic diagram;

[0032] Figure 4A schematic diagram of a DW convolution module;

[0033] Figure 5 A schematic diagram of an output structure of an output layer of the network of the present application;

[0034] Figure 6(a) is a detection result diagram before improvement,

[0035] Figure 6(b) is a detection result diagram after improvement;

[0036] Figure 7(a) is a Loss curve diagram before improvement,

[0037] Figure 7(b) is a Loss curve diagram after improvement;

[0038] Figure 8 A mAP_0.5 curve diagram of the present application;

[0039] Figure 9 A recall rate curve diagram of the present application. DETAILED DESCRIPTION

[0040] The present application will be further described in detail below in combination with the accompanying drawings and specific embodiments.

[0041] As shown in the drawings, Figure 1 the main steps of the present application are as follows:

[0042] Step 1, set the image data set to be identified.

[0043] Step 2, pre-process the image using grayscale, binarization, perspective transformation and edge segmentation, and convert the image size to 224*224*3.

[0044] Step 3, put the converted (224*224*3) image into the backbone network for information extraction. The MobileNet V2 backbone network has high classification efficiency and is relatively light in computing device, because when only low-dimensional features are used, the extracted target features are limited. In order to increase the amount of information of the extracted target, a 1*1 network structure is used to map high-dimensional feature values in a low-dimensional space before the depth separable convolution, and after the depth separable convolution, an expansion link layer is used for dimension reduction, and finally the backbone network (Backbone) outputs the processed feature map.

[0045] Step 4, the feature map output by the backbone network enters the feature fusion layer (Neck). For the output of different convolution layers of the backbone network, the application selects a feature map of 28*28*576, a feature map of 14*14*512 and a feature map of 7*7*320 as inputs, enters the feature fusion layer, performs upsampling and splicing on the three feature maps of different sizes, and then performs convolution to obtain three feature maps with output channels of 1024, 512 and 256, which are respectively input to the output layers of the large, medium and small three different prior boxes. The core idea of YOLOv3 is to use three different size grids to divide the original image, and the Neck layer of the application retains the core idea of YOLOv3, improves the generalization of the model, is conducive to the training and collection of the model, and maintains the detection speed of the YOLOv3 model.

[0046] Step 5, the three corresponding input feature maps of large, medium and small enter the output layer (Head), receive the three feature maps with different channel numbers input by the feature fusion layer, and then perform convolution again to obtain the classified feature map, and generate three prior boxes with different dimensions: the large size prior box is 13*13*(4+1+80), the medium size prior box is 26*26*(4+1+80), and the small size prior box is 52*52*(4+1+80), so as to express the output information.

[0047] Step 6, the loss function is used to calculate the performance of the model, and the parameters are updated by back propagation.

[0048] Figure 2 is the network model diagram of the application. First, input the picture to be detected, and perform Resize bilinear interpolation scaling transformation on the picture to change the image size to 224*224*3.

[0049] Next, as shown in Figure 3 , the image with a size of 224*224*3 is put into the MobileNetV2 Plus network for information extraction, and the specific network architecture is as follows:

[0050] The image of 224*224*3 is put into the convolution layer, wherein the padding (feature map edge padding) of the convolution layer is 1, the stride (convolution kernel sliding step) is 2, and the size of the convolution kernel is 3*3*3*32. Then enter the BatchNormal data standardization layer. After the BatchNormal layer, the activation function GELU is used.

[0051] The formula under the normal distribution standard is as follows:

[0052]

[0053] The final output is a feature map of 112*112*32.

[0054] Thereafter, the feature map of 112*112*32 is put into the BottleNeck_1 module, and the BottleNeck_1 module is as shown in Figure 3 The feature map of 112*112*32 first enters a convolutional layer with padding of 0, stride of 1, and convolution kernel size of 1*1*32*32, then passes through a BatchNormal data standard layer and an activation function GELU, and outputs a feature map of 112*112*32. Thereafter, the feature map of 112*112*32 enters a convolutional layer with padding of 1, stride of 1, and convolution kernel size of 1*1*32*32, then passes through a BatchNormal data standard layer and an activation function GELU, and outputs a feature map of 112*112*32. Finally, the feature map of 112*112*32 first enters a convolutional layer with padding of 0, stride of 1, and convolution kernel size of 1*1*32*16, then passes through a BatchNormal data standard layer, and outputs a feature map of 112*112*16.

[0055] The feature map of 112*112*16 is then put into the BottleNeck_1 module, and the BottleNeck_2 module is as shown in Figure 3 The feature map of 112*112*16 first enters a convolutional layer with padding of 0, stride of 1, and convolution kernel size of 1*1*16*96, then passes through a BatchNormal data standard layer and an activation function GELU, and outputs a feature map of 112*112*32, which plays a role of dimensionality increasing. The feature map of 112*112*96 first enters a convolutional layer with padding of 1, stride of 2, and convolution kernel size of 3*3*1*96, then passes through a BatchNormal data standard layer and an activation function GELU, and outputs a feature map of 56*56*96, which plays a role of information extraction. The feature map of 56*56*96 first enters a convolutional layer with padding of 0, stride of 1, and convolution kernel size of 1*1*96*24, then passes through a BatchNormal data standard layer, and outputs a feature map of 56*56*24, which plays a role of dimensionality decreasing.

[0056] The feature map of 56*56*24 is then put into the BottleNeck_3 module, and the BottleNeck_3 module is as shown in Figure 4The 56*56*24 feature map first enters a convolutional layer with padding of 0, stride of 1, and convolution kernel size of 1*1*24*144, and then passes through a BatchNormal data standard layer and an activation function GELU, and the output is a 56*56*144 feature map. The 56*56*24 feature map first enters a convolutional layer with padding of 1, stride of 2, and convolution kernel size of 3*3*24*144, and then passes through a BatchNormal data standard layer and an activation function GELU, and the output is a 56*56*144 feature map. The 56*56*24 feature map first enters a convolutional layer with padding of 0, stride of 1, and convolution kernel size of 1*1*14*24, and then passes through a BatchNormal data standard layer, and the output is a 56*56*24 feature map. Next, the SUM layer adds the two feature maps output by BottleNeck_2 and BottleNeck_3.

[0057] Then, according to the network architecture as shown in Figure 3 , operations are performed until the final output 1*1*1280*c, where c is the number of classifications.

[0058] The 28*28*64 feature map (SUM4 layer output) is used as the small prior box prediction input. The 14*14*192 feature map (SUM7 layer output) is used as the middle prior box prediction input. The 7*7*320 feature map (SUM10 layer output) is used as the large prior box prediction input.

[0059] The DW (Depthwise) convolution module in the application is composed of 1*1 size convolution kernel, which has the advantages of small calculation amount and fast speed compared with the unimproved network model.

[0060] The 7*7*320 feature map output from the 5 times improved DW module (as shown in Figure 4 ) is input into the Head large prior box layer, and the 7*7*320 feature map is submitted to the next level.

[0061] The 7*7*320 feature map output from the 5 times improved DW module (as shown in Figure 4), that is, 5 convolution layers with padding 0, stride 1, and convolution kernel 1*1*512*512, and output 14*14*512 feature maps to the head prior box layer, and 14*14*512 feature maps are submitted to the next level.

[0062] The 14*14*512 feature maps of the previous layer input are spliced to the small prior box prediction input, and after DW and downsampling, the 28*28*512 feature maps are obtained; after 5 improved DW modules (such as Figure 4 ), that is, 5 convolution layers with padding 0, stride 1, and convolution kernel 1*1*576*576; and output 28*28*576 feature maps to the head small prior box layer. The head large prior box layer enters the DW module and the conv layer, and the output is 13*13*255 feature maps, such as Figure 5 1 / 32 sampling in the formula (1). The head large prior box layer enters the DW module and the conv layer again, and the output is 26*26*255 feature maps, such as Figure 5 1 / 16 sampling in the formula (1). The head large prior box layer enters the DW module and the conv layer again, and the output is 52*52*255 feature maps, such as Figure 5 1 / 8 sampling in the formula (1).

[0063] Finally, the classification according to the feature map is obtained, as shown in the formula (1). Figure 5

[0064] In actual application, the negative samples with high confidence occupy a large part of the total samples, and using the weight can remove or weaken the loss of this part, which can obviously improve the efficiency of the loss function and better calculate the size of the loss value. Therefore, a weight factor is added before the cross entropy to form a new loss function, as shown in the formula (2), wherein log() is the cross entropy loss function.

[0065] FL(P t )=-(1-P t ) γ log(P t ) (2)

[0066] Wherein, γ is the parameter of the weight, when the value is greater than 1, better effect can be obtained (in the present application, the value is 2), P t is the confidence of the target object.

[0067] The present application adds a small weight α1 to the easily distinguishable samples (confidence close to 1 or close to 0), and adds a large weight α2 to the difficult-to-distinguish samples (confidence near 0.5), as shown in the formula (3).

[0068] Loss=α1*FL 易区分 +α2*FL​难区分 (3)

[0069] The loss function calculation is dominated by the difficult-to-distinguish samples, focusing the loss function on these samples. Focal Loss is used to handle class imbalance problems. The recognition result of the original YOLO v3 algorithm is shown in Figure 6(a), which contains only one prior box and has a recognition accuracy of 0.52. The recognition result of the improved YOLO v3 algorithm used in this invention is shown in Figure 6(b). The improved algorithm not only recognizes each individual with three prior boxes, but also achieves recognition accuracies of 0.82, 0.84, and 0.85, respectively, all of which are superior to the accuracy of the original algorithm. When the final loss value reaches a fixed threshold or a set number of iterations, the parameters are updated by backpropagation based on the chain rule derivative of the loss function.

[0070] The algorithm of this invention is faster and more accurate than the original algorithm. Regarding the loss curve, this invention visualizes the loss values ​​before and after the improvement, resulting in loss curves as shown in Figures 7(a) and 7(b). When the sample size is 400, the initial loss value before the improvement (see Figure 7(a)) is higher than the initial loss value after the improvement (see Figure 7(b)), and after 200 iterations, it approaches stability and remains below 0.25. Therefore, it can be concluded that the improved algorithm can improve the speed and accuracy of target detection.

[0071] mAP_0.5 is the average precision of object detection when the Intersection over Union (IOU) is greater than 0.5. A higher mAP value indicates higher overall network accuracy in detecting bounding boxes and objects, effectively representing the network's performance. It is a commonly used evaluation metric for object detection models. The average precision of the algorithm in this invention is as follows: Figure 8 As shown.

[0072] Recall, or recall rate, is used to represent a network's ability to identify all correct instances. Figure 9 As shown, the algorithm of this invention can find 75%-80% of the correct instances in the first n instances during the final detection, while the original algorithm can only maintain about 60%-70% in the final detection, which is a great improvement in correcting the detection.

Claims

1. A target detection method based on an improved YOLOv3, characterized in that, The steps include the following: S1, Select the image dataset; S2 preprocesses the image using grayscale conversion, binarization, perspective transformation, and edge segmentation, and then scales the image. S3, the preprocessed image is put into the MobileNetV2 Plus backbone network for information extraction; S4. Select three feature maps of different sizes as prediction inputs and enter the feature fusion layer; then upsample and concatenate the three feature maps of different sizes, and then perform convolution to take three feature maps with output channels of 1024, 512 and 256, and input them into the output layers of three different prior boxes of large, medium and small respectively. S5, the feature maps with three different numbers of channels input to the feature fusion layer are convolved again to obtain the classified feature maps and generate three prior boxes of different dimensions. S6, use the loss function to calculate the model and update the parameters; In step S3, the backbone network is the MobileNetV2 Plus network, where all the DW modules used are composed of 1*1 convolutional kernels. In step S4, a 7*7*320 feature map is used as the large prior box prediction input, a 14*14*192 feature map is used as the medium prior box prediction input, and a 28*28*64 feature map is used as the small prior box prediction input. The large prior box prediction input passes through the DW module and is output to the Head large prior box layer, and is then sampled and submitted to the next level as a 7*7*320 feature map; The prior bounding box prediction input is concatenated with the 7*7*320 feature map of the previous layer input, and then passed through the DW module and concatenated with a downsampled 14*14*320 feature map; then passed through the DW module again, and output a 14*14*512 feature map to the prior bounding box layer in the head, and submitted to the next level for sampling as a 14*14*512 feature map. The small prior bounding box prediction input is concatenated with the 14*14*512 feature map of the previous layer input, and after passing through DW and downsampling, it becomes a 28*28*512 feature map; after passing through the DW module, a 28*28*576 feature map is output to the head small prior bounding box layer; In step S5, the Head large prior box layer enters the DW module and the conv layer, and is sampled at 1 / 32, outputting a 13*13*255 feature map; the Head large prior box layer then enters the DW module and the conv layer, and is sampled at 1 / 16, outputting a 26*26*255 feature map; the Head large prior box layer then enters the DW module and the conv layer, and is sampled at 1 / 8, outputting a 52*52*255 feature map.

2. The target detection method based on improved YOLOv3 according to claim 1, characterized in that, In step S2, the image is scaled using the Resize bilinear interpolation method.

3. The target detection method based on improved YOLOv3 according to claim 1, characterized in that, In step S6, a weighting factor is added before the cross-entropy to form a new loss function. : in, Here are the weight parameters, and log() is the cross-entropy loss function. The value represents the confidence level of the target object; And add weights to easily distinguishable samples. Add weights to samples that are difficult to distinguish. Then we have: in, For easily distinguishable samples, the confidence level is close to 1 or close to 0; For samples that are difficult to distinguish, the confidence level is around 0.5.

Images