Lightweight small target detection method based on detail feature enhancement

Abstract

The invention aims to provide a light-weight small target detection method based on detail feature enhancement, and the method comprises the following steps: A, constructing a neural network which comprises a Backbone network module, a Neck network module and a Head network module; b, inputting the original image into a Backbone network module to obtain a CDFFM module processing result, a first feature fusion module processing result, a second feature fusion module processing result and a fast spatial pyramid module processing result, and respectively inputting the results into a Neck network module; and C, performing multi-scale feature fusion on the input features by the Neck network module to obtain two feature layers with different resolutions, and respectively inputting the two feature layers into a Head module for detection to obtain a series of detection frames and category data, namely a final result. According to the method, detail feature information can be effectively enhanced, so that a more accurate detection effect is obtained.

Description

Technical Field

[0001] The present invention relates to the field of computer image processing, and in particular to a lightweight small target detection method based on detail feature enhancement. Background Art

[0002] The purpose of object detection is to classify instances in an image and draw bounding boxes around them for positioning. It is a very important core direction of computer vision and is one of the three main tasks of CV along with classification and segmentation. UAV aerial images are of great significance in real-world scenarios. They can provide a broad perspective and application analysis for many fields such as urban planning and management, agricultural and forestry monitoring, disaster emergency response, security law enforcement, and environmental monitoring. However, due to the presence of small target objects in aerial images, existing methods based on convolutional neural networks have poor feature extraction capabilities for small target objects, resulting in low detection performance. Summary of the invention

[0003] The present invention aims to provide a lightweight small target detection method based on detail feature enhancement, which can effectively enhance detail feature information to obtain a more accurate detection effect.

[0004] The technical solution of the present invention is as follows:

[0005] The lightweight small target detection method based on detail feature enhancement comprises the following steps:

[0006] A. Constructing a neural network, wherein the neural network includes a Backbone network module, a Neck network module, and a Head network module;

[0007] B. The original image is input into the Backbone network module and processed in sequence by the first convolution downsampling, DFCM module, first feature fusion module, first depthwise separable convolution, second feature fusion module, second depthwise separable convolution, third feature fusion module, and fast spatial pyramid module;

[0008] The first convolution downsampling processing result, the DFCM module processing result, and the first depth-separable convolution processing result are respectively input into the CDFFM module for processing to obtain the CDFFM module processing result;

[0009] After upsampling, the first depth-separable convolution processing result is input into the first DFEM module together with the first convolution downsampling processing result for processing to obtain the first DFEM module processing result;

[0010] After upsampling, the second depth-separable convolution processing result is input into the second DFEM module together with the DFCM module processing result to obtain the second DFEM module processing result;

[0011] The processing results of the CDFFM module, the first feature fusion module, the second feature fusion module, and the fast spatial pyramid module are respectively input into the Neck network module;

[0012] C. The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions, which are input into the Head module for detection respectively to obtain a series of detection boxes and category data, which is the final result.

[0013] The processing process in the DFCM module is as follows:

[0014] The input result is divided into two paths. The first path passes through two convolutional layers and upsampling to obtain the first path result. The second path passes through two convolutional layers and upsampling, and then performs a subtraction operation of adding the absolute value to the input result to obtain the second path result.

[0015] After the absolute value of the first result is subtracted from the second result, it is added and fused with the second result and processed by the convolution layer to obtain the output result.

[0016] The processing process in the first DFEM module is as follows:

[0017] The result of the first depth-wise separable convolution processing is processed by 1*1 convolution and upsampling in sequence, and then the absolute value is subtracted from the result of the first convolution downsampling processing. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the result of the first depth-wise separable convolution processing, and then passes through the convolution attention module to obtain the final output result.

[0018] The processing process in the second DFEM module is as follows:

[0019] The second depth-separable convolution processing result is processed by 1*1 convolution and upsampling in sequence, and then the absolute value subtraction operation is added to the DFCM module processing result. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the second depth-separable convolution processing result, and then comes out of the convolution attention module to obtain the final output result.

[0020] The processing process in the CDFFM module is as follows:

[0021] The first convolution downsampling processing result is subjected to downsampling processing to obtain a downsampling result; the first depth-separable convolution processing result is subjected to upsampling processing to obtain an upsampling result; the downsampling result, the upsampling result, and the DFCM module processing result are concatenated by the Concat function to obtain a first concatenation result;

[0022] The first splicing result is divided into two parts by the split function to obtain two split results; the first split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the first result; the second split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the second result;

[0023] The first result and the second result are concatenated by the Concat function to obtain a second concatenated result. The second concatenated result is processed by the Sigmoid function and then multiplied and fused with the first concatenated result to obtain an output result.

[0024] The Neck network module is a PAN network architecture.

[0025] In the neural network constructed by the method of the present invention, a detail feature enhancement module (DFEM) and a cross-scale detail feature fusion module (CDFFM) are designed between the backbone and the neck to enhance detail features and fuse cross-scale information respectively. The final feature information will be passed to the head for target classification and positioning.

[0026] In the method of the present invention, DFCM is used to replace the ordinary convolution downsampling and applied to the second layer of the network. The image is divided into a coarse-grained path and a fine-grained path. The application of DFCM effectively compensates for the small target information and reduces the key details lost due to the downsampling operation. At the same time, DFCM successfully filters out part of the background information, laying the foundation for the subsequent detail extraction module to extract small target information.

[0027] The present invention adopts two DFEM modules to enhance the feature extraction capability of deep features for small target objects.

[0028] In the method of the present invention, in order to alleviate the problem of large scale changes of target objects, a CDFFM module is designed. By multiplying the weight map with the feature map, the contribution of different feature maps can be adjusted to highlight the features with small target weights. In this way, the model can better focus on the feature information of small targets. This operation enables our CDFFM to effectively handle small targets and improve the accuracy and robustness of detection. By fusion and weighting of multi-scale features, the problem of ignoring details in previous models when dealing with scale changes is effectively solved, and the detection accuracy of small target objects is improved.

[0029] The method of the present invention can effectively enhance the detail feature information, thereby obtaining a more accurate detection effect. In addition, the present invention adopts a lightweight network structure design, which can meet the model parameter quantity required by embedded devices such as drones. At the same time, the experiment of Example 2 shows that the DFCM module, DFEM module and CDFFM module proposed by us can all bring about an improvement in detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a schematic diagram of the structure of a neural network according to Embodiment 1 of the present invention;

[0031] Figure 2 It is a structural schematic diagram of the DFCM module of Example 1;

[0032] Figure 3 It is a structural schematic diagram of the DFEM module of Example 1;

[0033] Figure 4 It is a structural schematic diagram of the CDFFM module of Example 1;

[0034] Figure 5 The figure is a comparison chart of the effects of the prior art (YOLOv9c) and the embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0036] Example 1

[0037] The lightweight small target detection method based on detail feature enhancement comprises the following steps:

[0038] A. Construct a neural network, such as Figure 1 As shown, the neural network includes a Backbone network module, a Neck network module, and a Head network module;

[0039] B. The original image is input into the Backbone network module and processed in sequence by the first convolution downsampling, DFCM module, first feature fusion module, first depthwise separable convolution, second feature fusion module, second depthwise separable convolution, third feature fusion module, and fast spatial pyramid module;

[0040] The first convolution downsampling processing result, the DFCM module processing result, and the first depth-separable convolution processing result are respectively input into the CDFFM module for processing to obtain the CDFFM module processing result;

[0041] After upsampling, the first depth-separable convolution processing result is input into the first DFEM module together with the first convolution downsampling processing result for processing to obtain the first DFEM module processing result;

[0042] After upsampling, the second depth-separable convolution processing result is input into the second DFEM module together with the DFCM module processing result to obtain the second DFEM module processing result;

[0043] The processing results of the CDFFM module, the first feature fusion module, the second feature fusion module, and the fast spatial pyramid module are respectively input into the Neck network module;

[0044] like Figure 2 As shown, the processing process in the DFCM module is as follows:

[0045] The input result is divided into two paths. The first path passes through two convolutional layers and upsampling to obtain the first path result. The second path passes through two convolutional layers and upsampling, and then performs a subtraction operation of adding the absolute value to the input result to obtain the second path result.

[0046] After the absolute value of the first result is subtracted from the second result, it is added and fused with the second result and processed by the convolution layer to obtain the output result.

[0047] like Figure 3 As shown, the processing process in the first DFEM module is as follows:

[0048] The result of the first depth-wise separable convolution processing is processed by 1*1 convolution and upsampling in sequence, and then the absolute value is subtracted from the result of the first convolution downsampling processing. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the result of the first depth-wise separable convolution processing, and then passes through the convolution attention module to obtain the final output result.

[0049] The processing process in the second DFEM module is as follows:

[0050] The second depth-separable convolution processing result is processed by 1*1 convolution and upsampling in sequence, and then the absolute value subtraction operation is added to the DFCM module processing result. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the second depth-separable convolution processing result, and then comes out of the convolution attention module to obtain the final output result.

[0051] like Figure 4 As shown, the processing process in the CDFFM module is as follows:

[0052] The first convolution downsampling processing result is subjected to downsampling processing to obtain a downsampling result; the first depth-separable convolution processing result is subjected to upsampling processing to obtain an upsampling result; the downsampling result, the upsampling result, and the DFCM module processing result are concatenated by the Concat function to obtain a first concatenation result;

[0053] The first splicing result is divided into two parts by the split function to obtain two split results; the first split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the first result; the second split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the second result;

[0054] The first result and the second result are concatenated by the Concat function to obtain a second concatenated result. The second concatenated result is processed by the Sigmoid function and then multiplied and fused with the first concatenated result to obtain an output result.

[0055] C. The Neck network module is a PAN network architecture. The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions, which are respectively input into the Head module for detection to obtain a series of detection frames and category data, which is the final result.

[0056] Example 2

[0057] For the quantitative performance evaluation of the final aerial images, we adopt the performance measurement standards widely used in the field of target detection. The specific evaluation is shown in formulas (9) and (10).

[0058]

[0059] Where AP represents the integral of R (Recall) on P (Precision), the confidence threshold ranges from 0 to 1, mAP represents the average AP value of all categories in the dataset, and N is the number of categories. The higher the mAP value, the stronger the detection performance of the model.

[0060] Table 1 summarizes the ablation experiment data of Example 1 on the UAV small target detection dataset (VisDrone). From the experimental results, when compared with the baseline, the model performance is significantly higher than the baseline performance when the DFCM, DFEM or CDFFM designed by us is added; when DFCM, DFEM and CDFFM are added at the same time, the model performance is better than the performance when DFCM, DFEM or CDFFM is used alone.

[0061] Table 1 Ablation experiment

[0062]

[0063] Example 3

[0064] The latest existing technology YOLOv9c is compared with the method of Example 1 in the UAV small target detection dataset (VisDrone). The effect comparison chart is shown in Figure 5 ,Depend on Figure 5It can be seen that the algorithm model of Example 1 of the present invention is more accurate in detection performance than the existing technology, and the existing general technology may have missed detections and false detections, while the algorithm model of Example 1 of the present invention can perform object detection more comprehensively and accurately.

Claims

1. A lightweight small target detection method based on detail feature enhancement, characterized in that: The following steps are involved: A. Constructing a neural network, wherein the neural network includes a Backbone network module, a Neck network module, and a Head network module; B. The original image is input into the Backbone network module and processed in sequence by the first convolution downsampling, DFCM module, first feature fusion module, first depthwise separable convolution, second feature fusion module, second depthwise separable convolution, third feature fusion module, and fast spatial pyramid module; The first convolution downsampling processing result, the DFCM module processing result, and the first depth-separable convolution processing result are respectively input into the CDFFM module for processing to obtain the CDFFM module processing result; After upsampling, the first depth-separable convolution processing result is input into the first DFEM module together with the first convolution downsampling processing result for processing to obtain the first DFEM module processing result; After upsampling, the second depth-separable convolution processing result is input into the second DFEM module together with the DFCM module processing result to obtain the second DFEM module processing result; The processing results of the CDFFM module, the first feature fusion module, the second feature fusion module, and the fast spatial pyramid module are respectively input into the Neck network module; C. The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions, which are input into the Head module for detection respectively to obtain a series of detection boxes and category data, which is the final result.

2. The lightweight small target detection method based on detail feature enhancement according to claim 1, characterized in that: The processing process in the DFCM module is as follows: The input result is divided into two paths. The first path passes through two convolutional layers and upsampling to obtain the first path result. The second path passes through two convolutional layers and upsampling, and then performs a subtraction operation of adding the absolute value to the input result to obtain the second path result. After the absolute value of the first result is subtracted from the second result, it is added and fused with the second result and processed by the convolution layer to obtain the output result.

3. The lightweight small target detection method based on detail feature enhancement according to claim 1, characterized in that: The processing process in the first DFEM module is as follows: The result of the first depth-wise separable convolution processing is processed by 1*1 convolution and upsampling in sequence, and then the absolute value is subtracted from the result of the first convolution downsampling processing. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the result of the first depth-wise separable convolution processing, and then passes through the convolution attention module to obtain the final output result.

4. The lightweight small target detection method based on detail feature enhancement according to claim 1, characterized in that: The processing process in the second DFEM module is as follows: The second depth-separable convolution processing result is processed by 1*1 convolution and upsampling in sequence, and then the absolute value subtraction operation is added to the DFCM module processing result. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the second depth-separable convolution processing result, and then comes out of the convolution attention module to obtain the final output result.

5. The lightweight small target detection method based on detail feature enhancement according to claim 1, characterized in that: The processing process in the CDFFM module is as follows: The first convolution downsampling processing result is subjected to downsampling processing to obtain a downsampling result; the first depth-separable convolution processing result is subjected to upsampling processing to obtain an upsampling result; the downsampling result, the upsampling result, and the DFCM module processing result are concatenated by the Concat function to obtain a first concatenation result; The first splicing result is divided into two parts by the split function to obtain two split results; the first split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the first result; the second split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the second result; The first result and the second result are concatenated by the Concat function to obtain a second concatenated result. The second concatenated result is processed by the Sigmoid function and then multiplied and fused with the first concatenated result to obtain an output result.

6. The lightweight small target detection method based on detail feature enhancement according to claim 1, characterized in that: The Neck network module is a PAN network architecture.

Images