A lightweight target detection method to improve small target detection accuracy

Abstract

The present invention discloses a lightweight target detection method for improving the accuracy of small target detection. The BiFPN structure in the EfficientDet model is improved to BmFPN, which replaces the original PANet of YOLOv5 as the feature network to realize feature fusion, strengthens the extraction of underlying feature information, and adds a weight mechanism to the feature map of each layer. CIoU is introduced in the loss function to replace the original GIoU. The aspect ratio influencing factor makes the target frame regression more stable. Especially when IoU is zero, the loss function tends to optimize in the direction of increasing the overlapping area. The network structure of the present invention is small in size, fast in detection speed, and high in accuracy. It fully meets the requirements of real-time scenarios and has very high practical value.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a lightweight target detection method for improving small target detection accuracy. Background Art

[0002] With the rapid development of target detection technology, its applications are spread across all walks of life. Remote sensing images are mostly generated by aircraft and satellite photography. The identifiable targets in them include lakes, forests, aircraft, vehicles, bridges, buildings and other objects. Its target recognition is widely used in agricultural output value measurement, disaster monitoring, urban construction planning and other fields. In terms of target detection algorithms, YOLO, as a representative of one-stage algorithms, has been widely used in various fields because of its real-time nature. The first generation of YOLO algorithm was proposed by Joseph Redmon et al. in 2016. This detection method discards the stage of generating candidate regions, puts feature extraction, classification and regression in a convolutional network, and directly uses the feature map at the top layer of the network to evaluate the category probability. The YOLOv5 algorithm, which can achieve a high balance between speed and accuracy, was proposed by Glenn et al. in June 2020. The algorithm obtains models of different complexities by setting flexible configuration parameters. Its characteristics such as fast detection speed and small model size enable it to be quickly embedded and deployed on mobile terminals under high real-time scenario requirements. Its input uses data augmentation algorithms such as MixUP, CutMix, etc. for multi-image fusion, adaptive image scaling, DropBlock, etc. for simulating image occlusion. Its backbone network performs downsampling 5 times in total, and its main modules are Focus and CSP structures. Among them, Focus is used to implement slicing, which is a downsampling operation that minimizes information loss while reducing the amount of calculation; CSP structures are divided into two types: one is used in the Backbone backbone network to improve the learning ability of the convolutional neural network, and the other is used in Neck to strengthen the fusion of network features. Its Neck structure references the SPP module that expands the receptive field and the PANet that enhances the communication between multi-channel features to achieve cross-layer fusion of features. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a lightweight target detection method for improving the detection accuracy of small targets in view of the defects involved in the background technology.

[0004] The present invention adopts the following technical solutions to solve the above technical problems:

[0005] A lightweight target detection method for improving the accuracy of small target detection, using the feature network BmFPN instead of PANet in the YOLOv5 structure as the feature network to achieve feature fusion, form an improved YOLOv5 structure, and perform target detection based on the improved YOLOv5 structure;

[0006] The input layer of the feature network BmFPN includes three layers: C3_in, C4_in, and C5_in, the middle layer is C4-mid, and the output layer includes three layers: C3_out, C4_out, and C5_out, where:

[0007] The inputs of C3_in, C4_in, and C5_in are the last three downsampled feature map results of the YOLOv5 model backbone network;

[0008] C4-mid is obtained by weighted fusion of C5_in and C4_in after two upsampling;

[0009] C3_out is obtained by weighted fusion of C4-mid after two upsampling and C3_in;

[0010] C4_out is obtained by weighted fusion of C4-mid, C4_in, and C3_out after 3*3 convolution;

[0011] C5_out is obtained by weighted fusion of C5_in and C4_out after 3*3 convolution.

[0012] As a further optimization scheme of the lightweight target detection method for improving the small target detection accuracy of the present invention, in the improved YOLOv5 structure, the CIoU function is used instead of the original GIoU function as the loss function.

[0013] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0014] The present invention discloses a lightweight target detection method for improving the detection accuracy of small targets under complex backgrounds, wherein the ablation experiment method is used to analyze the effectiveness of the improved module in improving the algorithm performance. mAP@0.5 refers to the average of the AP values ​​of each category when the IoU threshold is 0.5. mAP@0.5:0.95 refers to taking a step size of 0.05 and calculating the average accuracy of each category when the IoU threshold increases from 0.5 to 0.95. The BmFPN module can improve mAP@0.5 and mAP@0.5:0.95 by 0.6-0.7, indicating that the addition of bidirectional fusion and weights enriches the feature information of small targets in the feature map, so that the feature map calculated by the network can more effectively characterize the object. The detection effect of the combination of CIoU and BmFPN modules is the best, which increases by 0.9 and 0.7 at mAP@0.5 and mAP@0.5:0.95 respectively.

[0015] In the comparative experiment, the detection speed and accuracy of YOLO-BM, YOLOv5 and YOLOv3 models on the test set were compared. The smallest scale s model in the YOLOv5 model has the fastest detection speed, reaching 128.8fps; the m model sacrifices 30% of the detection speed and improves the detection accuracy by 0.5 points; although the accuracy of the YOLOv3 model is higher than that of the s model, the detection speed is very slow, only 6.3fps. The YOLO-BM model of the present invention is only 0.27ms slower than the fastest s model, while improving the detection accuracy by 1 point, and the detection speed is much higher than other algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the BmFPN feature network in the present invention. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings:

[0018] The present invention can be implemented in many different forms and should not be considered to be limited to the embodiments described herein. On the contrary, these embodiments are provided to make this disclosure thorough and complete, and will fully express the scope of the present invention to those skilled in the art. In the accompanying drawings, components are enlarged for clarity.

[0019] In order to meet the real-time requirements, the present invention is improved based on the YOLOv5s structure with fast speed and small model. In view of the characteristics of optical remote sensing images such as large picture, diverse scale changes, huge target scale differences, and close arrangement, the BmFPN structure is introduced to strengthen the network feature fusion, reduce the network parameters and calculation amount, and introduce the CIoU loss function in the final border regression.

[0020] For small target detection in large-resolution images, as the network deepens, semantic information becomes clearer, but location information is lost. If the original image (such as 16000*16000) is directly input into the detection network, the receptive field of each feature map is small after multiple downsampling (for example, after the YOLOv5 backbone network is downsampled 5 times, the 32*32 pixel target in the input image is only 1*1 in size). When the width or height of the target in the original image is smaller than the threshold, it is difficult for the network to learn the feature information of the target, and the detection effect is not good.

[0021] YOLOv5 uses the bidirectional FPN structure PANet, which has performed well in the field of image segmentation. The FPN structure conveys strong semantic information from top to bottom, and PANet conveys strong positioning features from bottom to top, so that high-order feature outputs also have low-order details. They complement each other and aggregate the features from different backbone layers to different detection layers, further improving the network's feature extraction capabilities.

[0022] The present invention discloses a lightweight target detection method for improving the detection accuracy of small targets, using a feature network BmFPN to replace PANet in the YOLOv5 structure as a feature network to achieve feature fusion, form an improved YOLOv5 structure, and perform target detection based on the improved YOLOv5 structure;

[0023] The input layer of the feature network BmFPN includes three layers: C3_in, C4_in, and C5_in, the middle layer is C4-mid, and the output layer includes three layers: C3_out, C4_out, and C5_out, where:

[0024] The inputs of C3_in, C4_in, and C5_in are the last three down-sampled feature map results of the YOLOv5 model backbone network;

[0025] C4-mid is obtained by weighted fusion of C5_in and C4_in after two upsampling;

[0026] C3_out is obtained by weighted fusion of C4-mid after two upsampling and C3_in;

[0027] C4_out is obtained by weighted fusion of C4-mid, C4_in, and C3_out after 3*3 convolution;

[0028] C5_out is obtained by weighted fusion of C5_in and C4_out after 3*3 convolution.

[0029] The present invention improves the BiFPN structure in the EfficientDet model to form a feature network BmFPN, and uses the feature network BmFPN to replace the original PANet as the feature network to achieve feature fusion. Compared with PANet, it removes nodes with only one input edge, adds cross-layer links, and achieves more advanced feature fusion through bidirectional propagation. The structure of the feature network BmFPN is as follows Figure 1 As shown in the figure, it can be understood that the attention mechanism is added to the feature fusion route, and appropriate weights are assigned to each feature map in the process. As shown in the dotted line area, The weights {ω'1, ω'2, ω'3} learned by the network during training are multiplied by the three parts and then added together. The calculation formula is:

[0030]

[0031] in The 80*80 feature map output by the backbone network at the fourth layer, is the third layer output generated after bidirectional fusion. After upsampling, The fusion result is convolved and input into the residual structure, nonlinear activation and normalization structure to obtain Middle layer:

[0032]

[0033] In traditional methods, feature maps of different resolutions are adjusted to the same resolution and then directly added together during fusion. However, in fact, their contributions to the final output features are different, so a more reasonable approach is the weighted average fusion used in the present invention.

[0034] YOLOv5 uses BCEcls binary cross entropy loss to calculate class probability, GIoU Loss and BEC Logits loss to calculate target score L obj , GIoU Loss calculates the bounding box regression L box The total network loss is L = L obj +L cls +L box .

[0035]

[0036] Where P and P gt Divided into predicted boxes and true boxes, Z is its minimum bounding rectangle. It can be seen that the optimization object of GIoU is the area between the two boxes, which can preliminarily solve the situation where IoU is zero. Although it can reflect the directional difference of the anchor box to a certain extent, it cannot reflect the specific intersection situation when the intersection values ​​of the two boxes are the same.

[0037] In order to achieve fast and stable bounding box regression when the target box wraps the prediction box, the present invention introduces CIoU into the loss instead of the original loss function GIoU.

[0038]

[0039] In the formula is the parameter of the balance ratio; It is used to measure the degree of fit between the aspect ratio of the prediction box and the target box. The aspect ratio influencing factor αv makes the target box regression more stable, especially when IoU is zero, the loss function tends to optimize in the direction of increasing the overlapping area.

[0040] Then, the algorithm was evaluated using a large-scale dataset DOTA, which contains 2860 remote sensing images collected from different sensors and platforms. In this dataset, 15 common targets are annotated by rotating boxes, which are characterized by diverse scale changes and high detection difficulty. Since the resolution of remote sensing images spans a large range, and some are as high as tens of thousands of pixels, the images should not be scaled directly, and the dataset needs to be cut. The images containing two types of targets, small vehicles (SV) and large vehicles (LV), are taken as the original images, and they are cut into 640*640 sub-images, and the overlap interval between the two sub-images is guaranteed to be 200 pixels. During the cropping process, only the annotations with the center point inside the sub-image are retained, and the annotation values ​​are adjusted. After processing, a total of 10621 images are obtained, of which 58% (6152 images) are in the training set, 14% (1513 images) are in the validation set, and 28% (2956 images) are in the test set, and the two types of targets each account for about 1 / 2. The small target remote sensing dataset M-DOTA used in this paper has a higher proportion of small and medium-sized targets than COCO, with more targets in a single image (the average number of targets per image is 9.97), and most objects are distributed in a non-central area, making detection difficult.

[0041] The designed YOLO-BM network structure was verified using Ubuntu 18.04 system as the experimental platform. Two GeForce RTX 2080SUPER were used as graphics processing units (GPUs) in both training and testing. The deep learning framework used for verification was Pytorch. The SGD optimizer was used in network training, with the initial learning rate set to 0.01 and the weight decay coefficient set to 0.0005. The same set of anchor boxes was used in the network, which was obtained by the K-Means clustering algorithm with a BPR of 0.9981.

[0042] The main evaluation indicators are recall and precision, which are defined as follows:

[0043]

[0044] TP stands for True Positive, FP stands for False Positive, and the judgment condition is whether the IoU between the object identified as a positive example and the Ground truth area is greater than the specified threshold. FN stands for False Negative, which means that the Ground truth area is omitted and not detected.

[0045] To a certain extent, precision and recall restrict each other, so a balance between the two is usually adopted. As a standard for evaluating the detection capability of the network. Geometrically speaking, the Recall value is used as the horizontal axis and the Precision value is used as the vertical axis to draw the PRC curve, and AP is the integral area enclosed by it and the coordinate axis.

[0046] In order to verify the effectiveness of the algorithm improvement proposed in this invention, the ablation experiment method is used to analyze the effectiveness of the improved module in improving the algorithm performance. Table 1 shows the comparison of the detection effect of using CIoU and GIoU for loss regression, and using FPN, PANet and the BmFPN feature fusion network designed in this paper in Neck. mAP@0.5 in the table refers to the average AP value of each category when the IoU threshold is 0.5. mAP@0.5:0.95 means taking a step size of 0.05 and calculating the average accuracy of each category when the IoU threshold increases from 0.5 to 0.95.

[0047] Table 1 Comparison of ablation experiment results

[0048]

[0049] Table 1 shows that the BmFPN module proposed in this paper can improve mAP@0.5 and mAP@0.5:0.95 by 0.6-0.7, indicating that the addition of bidirectional fusion and weights enriches the feature information of small targets in the feature map, making the feature map calculated by the network more effective in representing objects. The combination of CIoU and BmFPN modules has the best detection effect, with an increase of 0.9 and 0.7 in mAP@0.5 and mAP@0.5:0.95, respectively. This result proves the rationality and superiority of the model designed in this paper.

[0050] Table 2 Comparison of parameter quantity

[0051]

[0052] As shown in Table 2, BmFPN improves performance without significantly increasing the model size and floating-point calculation amount, and still maintains a lightweight model size, which is convenient for deployment in scenarios with limited computing power.

[0053] Table 3 Comparison of detection accuracy and speed of each algorithm

[0054]

[0055] Table 3 compares the detection speed and accuracy of the YOLO-BM, YOLOv5, and YOLOv3 models on the test set. It can be seen that the smallest scale s model in the YOLOv5 model has the fastest detection speed, reaching 128.8fps; the m model sacrifices 30% of the detection speed and improves the detection accuracy by 0.5 points; although the accuracy of the YOLOv3 model is higher than the s model, the detection speed is very slow, only 6.3fps. The YOLO-BM model designed in this paper is only 0.27ms slower than the fastest s model while improving the detection accuracy by 1 point, and the detection speed is much higher than other algorithms.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0057] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A lightweight target detection method for improving small target detection accuracy, characterized in that: The feature network BmFPN is used to replace PANet in the YOLOv5 structure as the feature network to achieve feature fusion, form an improved YOLOv5 structure, and perform target detection based on the improved YOLOv5 structure; The input layer of the feature network BmFPN includes three layers: C3_in, C4_in, and C5_in, the middle layer is C4-mid, and the output layer includes three layers: C3_out, C4_out, and C5_out, where: The inputs of C3_in, C4_in, and C5_in are the last three down-sampled feature map results of the YOLOv5 model backbone network; C4-mid is obtained by weighted fusion of C5_in and C4_in after two upsampling; C3_out is obtained by weighted fusion of C4-mid after two upsampling and C3_in; C4_out is obtained by weighted fusion of C4-mid, C4_in, and C3_out after 3*3 convolution; C5_out is obtained by weighted fusion of C5_in and C4_out after 3*3 convolution.

2. The lightweight target detection method for improving small target detection accuracy according to claim 1, characterized in that: In the improved YOLOv5 structure, the CIoU function is used instead of the original GIoU function as the loss function.

Images