A lightweight small target detection method based on detail feature enhancement

Abstract

The present application aims to provide a kind of light weight small target detection method based on detail feature enhancement, comprising the following steps: A, construct neural network, the neural network includes Backbone network module, Neck network module, Head network module;B, original image is input in Backbone network module, obtains CDFFM module processing result, first feature fusion module processing result, second feature fusion module, fast spatial pyramid module processing result, respectively input in Neck network module;C, Neck network module carries out multi-scale feature fusion to input feature, obtains two different resolution feature layers, respectively input in Head module and carries out detection, obtains a series of detection frame and class data, i.e. final result.The method of the present application can effectively enhance detail feature information, so as to obtain more accurate detection effect.

Description

Technical Field

[0001] The present invention relates to the field of computer image processing, and in particular to a lightweight small target detection method based on detail feature enhancement. Background Art

[0002] The goal of object detection is to classify instances in an image and locate them by drawing bounding boxes around them. It is a crucial core area of ​​computer vision, one of the three main tasks of computer vision, along with classification and segmentation. Aerial imagery from drones is crucial in real-world scenarios, providing a broad perspective and application analysis for a variety of fields, including urban planning and management, agricultural and forestry monitoring, disaster response, security enforcement, and environmental monitoring. However, existing convolutional neural network-based methods have limited ability to extract features for small objects, resulting in low detection performance. Summary of the Invention

[0003] The present invention aims to provide a lightweight small target detection method based on detail feature enhancement, which can effectively enhance detail feature information to obtain more accurate detection results.

[0004] The technical solutions of the present invention are as follows:

[0005] The lightweight small target detection method based on detail feature enhancement includes the following steps:

[0006] A. Constructing a neural network, wherein the neural network includes a Backbone network module, a Neck network module, and a Head network module;

[0007] B. The original image is input into the Backbone network module and processed in sequence by the first convolution downsampling module, the DFCM module, the first feature fusion module, the first depthwise separable convolution module, the second depthwise separable convolution module, the third feature fusion module, and the fast spatial pyramid module;

[0008] The first convolution downsampling processing result, the DFCM module processing result, and the first depth-separable convolution processing result are respectively input into the CDFFM module for processing to obtain the CDFFM module processing result;

[0009] After upsampling, the first depthwise separable convolution processing result and the first convolution downsampling processing result are input into the first DFEM module for processing to obtain the first DFEM module processing result;

[0010] The second depth-wise separable convolution processing result is upsampled and input into the second DFEM module together with the DFCM module processing result to obtain the second DFEM module processing result;

[0011] The processing results of the CDFFM module, the first feature fusion module, the second feature fusion module, and the fast spatial pyramid module are respectively input into the Neck network module;

[0012] C. The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions, which are input into the Head module for detection respectively. A series of detection boxes and category data are obtained, which is the final result.

[0013] The processing process in the DFCM module is as follows:

[0014] The input result is divided into two paths. The first path passes through two convolutional layers and upsampling to obtain the first path result. The second path passes through two convolutional layers and upsampling, and then performs a subtraction operation on the absolute value of the input result to obtain the second path result.

[0015] After the absolute value of the first result is subtracted from the second result, it is added and fused with the second result, and then processed by the convolution layer to obtain the output result.

[0016] The processing process in the first DFEM module is as follows:

[0017] The result of the first depth-wise separable convolution processing is processed by 1*1 convolution and upsampling in sequence, and then the absolute value subtraction operation is performed on the result of the first convolution downsampling processing. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the result of the first depth-wise separable convolution processing, and then comes out of the convolution attention module to obtain the final output result.

[0018] The processing process in the second DFEM module is as follows:

[0019] The result of the second depth-wise separable convolution processing is processed by 1*1 convolution and upsampling in sequence, and then the absolute value subtraction operation is added to the result of the DFCM module processing. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the result of the second depth-wise separable convolution processing, and then comes out of the convolution attention module to obtain the final output result.

[0020] The processing process in the CDFFM module is as follows:

[0021] The first convolution downsampling processing result is subjected to downsampling processing to obtain a downsampling result; the first depth-separable convolution processing result is subjected to upsampling processing to obtain an upsampling result; the downsampling result, the upsampling result, and the DFCM module processing result are concatenated by the Concat function to obtain a first concatenation result;

[0022] The first splicing result is divided into two parts by the split function to obtain two split results; the first split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the first result; the second split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the second result;

[0023] The first result and the second result are concatenated by the Concat function to obtain a second concatenated result. The second concatenated result is processed by the Sigmoid function and multiplied and fused with the first concatenated result to obtain an output result.

[0024] The Neck network module is a PAN network architecture.

[0025] In the neural network constructed using this method, a Detail Feature Enhancement Module (DFEM) and a Cross-Scale Detail Feature Fusion Module (CDFFM) are designed between the Backbone and the Neck to enhance detail features and fuse cross-scale information, respectively. This feature information is then passed to the Head for target classification and localization.

[0026] In this method, DFCM replaces conventional convolutional downsampling and is applied to the second layer of the network. By dividing the image into coarse-grained and fine-grained pathways, DFCM effectively compensates for small object information and reduces key details lost due to downsampling. Furthermore, DFCM successfully filters out some background information, laying the foundation for the subsequent detail extraction module to extract small object information.

[0027] The present invention adopts two DFEM modules to enhance the feature extraction capability of deep features for small target objects.

[0028] To alleviate issues such as large scale variations in target objects, the CDFFM module is designed in this method. By multiplying the weight map with the feature map, the contributions of different feature maps can be adjusted, highlighting features with small weights. This allows the model to better focus on the characteristic information of small objects. This operation enables our CDFFM to effectively handle small objects and improve detection accuracy and robustness. By fusing and weighting multi-scale features, it effectively solves the problem of previous models neglecting details when handling scale variations, thereby improving the detection accuracy of small objects.

[0029] The method of the present invention can effectively enhance detailed feature information, thereby achieving more accurate detection results. In addition, the present invention adopts a lightweight network structure design, which can meet the model parameter requirements of embedded devices such as drones. At the same time, the experiments in Example 2 show that the proposed DFCM module, DFEM module, and CDFFM module can all improve detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the structure of the neural network of Example 1 of the present invention;

[0031] Figure 2 This is a schematic structural diagram of the DFCM module of Example 1;

[0032] Figure 3 Schematic diagram of the structure of the DFEM module of Example 1;

[0033] Figure 4 Schematic diagram of the structure of the CDFFM module of Example 1;

[0034] Figure 5 The figure compares the effects of the prior art (YOLOv9c) and the embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0036] Example 1

[0037] The lightweight small target detection method based on detail feature enhancement includes the following steps:

[0038] A. Construct a neural network, such as Figure 1 As shown, the neural network includes a Backbone network module, a Neck network module, and a Head network module;

[0039] B. The original image is input into the Backbone network module and processed in sequence by the first convolution downsampling module, the DFCM module, the first feature fusion module, the first depthwise separable convolution module, the second depthwise separable convolution module, the third feature fusion module, and the fast spatial pyramid module;

[0040] The first convolution downsampling processing result, the DFCM module processing result, and the first depth-separable convolution processing result are respectively input into the CDFFM module for processing to obtain the CDFFM module processing result;

[0041] After upsampling, the first depthwise separable convolution processing result and the first convolution downsampling processing result are input into the first DFEM module for processing to obtain the first DFEM module processing result;

[0042] The second depth-wise separable convolution processing result is upsampled and input into the second DFEM module together with the DFCM module processing result to obtain the second DFEM module processing result;

[0043] The processing results of the CDFFM module, the first feature fusion module, the second feature fusion module, and the fast spatial pyramid module are respectively input into the Neck network module;

[0044] like Figure 2 As shown, the processing process in the DFCM module is as follows:

[0045] The input result is divided into two paths. The first path passes through two convolutional layers and upsampling to obtain the first path result. The second path passes through two convolutional layers and upsampling, and then performs a subtraction operation on the absolute value of the input result to obtain the second path result.

[0046] After the absolute value of the first result is subtracted from the second result, it is added and fused with the second result, and then processed by the convolution layer to obtain the output result.

[0047] like Figure 3 As shown, the processing process in the first DFEM module is as follows:

[0048] The result of the first depth-wise separable convolution processing is processed by 1*1 convolution and upsampling in sequence, and then the absolute value subtraction operation is performed on the result of the first convolution downsampling processing. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the result of the first depth-wise separable convolution processing, and then comes out of the convolution attention module to obtain the final output result.

[0049] The processing process in the second DFEM module is as follows:

[0050] The result of the second depth-wise separable convolution processing is processed by 1*1 convolution and upsampling in sequence, and then the absolute value subtraction operation is added to the result of the DFCM module processing. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the result of the second depth-wise separable convolution processing, and then comes out of the convolution attention module to obtain the final output result.

[0051] like Figure 4 As shown, the processing process in the CDFFM module is as follows:

[0052] The first convolution downsampling processing result is subjected to downsampling processing to obtain a downsampling result; the first depth-separable convolution processing result is subjected to upsampling processing to obtain an upsampling result; the downsampling result, the upsampling result, and the DFCM module processing result are concatenated by the Concat function to obtain a first concatenation result;

[0053] The first splicing result is divided into two parts by the split function to obtain two split results; the first split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the first result; the second split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the second result;

[0054] The first result and the second result are concatenated by the Concat function to obtain a second concatenated result. The second concatenated result is processed by the Sigmoid function and multiplied and fused with the first concatenated result to obtain an output result.

[0055] C. The Neck network module is a PAN network architecture. The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions. These are input into the Head module for detection, and a series of detection boxes and category data are obtained, which is the final result.

[0056] Example 2

[0057] For the quantitative performance evaluation of the final aerial images, we adopt the performance measurement standards widely used in the field of target detection. The specific evaluation is shown in formulas (9) and (10).

[0058]

[0059] Where AP represents the integral of R (Recall) over P (Precision), the confidence threshold ranges from 0 to 1, mAP represents the average AP value of all categories in the dataset, and N is the number of categories. The higher the mAP value, the stronger the detection performance of the model.

[0060] Table 1 summarizes the ablation experiment data from Example 1 on the VisDrone dataset for small drone target detection. The experimental results show that, compared to the baseline, the model performance significantly outperforms the baseline when our designed DFCM, DFEM, or CDFFM is added. Furthermore, the model performance is superior to that of using DFCM, DFEM, or CDFFM alone when all three are added.

[0061] Table 1 Ablation experiment

[0062]

[0063] Example 3

[0064] The latest existing technology YOLOv9c is compared with the method of Example 1 in the UAV small target detection dataset (VisDrone). The comparison of the results is shown in the figure. Figure 5 ,Depend on Figure 5It can be seen that the algorithm model of Example 1 of the present invention is more accurate in detection performance than the existing technology, and the existing general technology may have missed detections and false detections, while the algorithm model of Example 1 of the present invention can perform object detection more comprehensively and accurately.

Claims

1. A lightweight small target detection method based on detail feature enhancement, characterized in that: The following steps are involved: A. Constructing a neural network, wherein the neural network includes a Backbone network module, a Neck network module, and a Head network module; B. The original image is input into the Backbone network module and processed in sequence by the first convolution downsampling, DFCM detail feature compensation module, first feature fusion module, first depthwise separable convolution, second feature fusion module, second depthwise separable convolution, third feature fusion module, and fast spatial pyramid module; The first convolution downsampling processing result, the DFCM detail feature compensation module processing result, and the first depth-separable convolution processing result are respectively input into the CDFFM cross-scale detail feature fusion module for processing to obtain the CDFFM cross-scale detail feature fusion module processing result; After upsampling, the first depthwise separable convolution processing result and the first convolution downsampling processing result are input into the first DFEM detail feature enhancement module for processing to obtain the first DFEM detail feature enhancement module processing result; After upsampling, the second depthwise separable convolution processing result is input into the second DFEM detail feature enhancement module together with the processing result of the DFCM detail feature compensation module to obtain the processing result of the second DFEM detail feature enhancement module; The processing results of the CDFFM cross-scale detail feature fusion module, the first feature fusion module, the second feature fusion module, and the fast spatial pyramid module are input into the Neck network module respectively; C. The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions, which are input into the Head module for detection respectively. A series of detection boxes and category data are obtained, which is the final result; The processing process in the CDFFM cross-scale detail feature fusion module is as follows: The first convolution downsampling processing result is subjected to downsampling processing to obtain a downsampling result; the first depth-separable convolution processing result is subjected to upsampling processing to obtain an upsampling result; the downsampling result, the upsampling result, and the DFCM detail feature compensation module processing result are concatenated by the Concat function to obtain a first splicing result; The first splicing result is divided into two parts by the split function to obtain two split results; the first split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the first result; the second split result is sequentially processed by 3*3 convolution, 1*1 convolution, and 5*5 convolution to obtain the second result; The first result and the second result are concatenated by the Concat function to obtain a second concatenated result. The second concatenated result is processed by the Sigmoid function and multiplied and fused with the first concatenated result to obtain an output result.

2. The lightweight small target detection method based on detail feature enhancement according to claim 1, characterized in that: The processing process in the DFCM detail feature compensation module is as follows: The input result is divided into two paths. The first path passes through two convolutional layers and upsampling to obtain the first path result. The second path passes through two convolutional layers and upsampling, and then performs a subtraction operation on the absolute value of the input result to obtain the second path result. After the absolute value of the first result is subtracted from the second result, it is added and fused with the second result, and then processed by the convolution layer to obtain the output result.

3. The lightweight small target detection method based on detail feature enhancement according to claim 1, characterized in that: The processing process in the first DFEM detail feature enhancement module is as follows: The result of the first depth-wise separable convolution processing is processed by 1*1 convolution and upsampling in sequence, and then the absolute value subtraction operation is performed on the result of the first convolution downsampling processing. The result is processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the result of the first depth-wise separable convolution processing, and then comes out of the convolution attention module to obtain the final output result.

4. The lightweight small target detection method based on detail feature enhancement according to claim 1, characterized in that: The processing process in the second DFEM detail feature enhancement module is as follows: The result of the second depth-wise separable convolution processing is successively subjected to 1*1 convolution and upsampling, and then the absolute value subtraction operation is added to the result of the DFCM detail feature compensation module processing. The result is then processed by 3*3 convolution downsampling, and then processed by 1*1 convolution. It is then added and fused with the result of the second depth-wise separable convolution processing, and after passing through the convolution attention module, the final output result is obtained.

5. The lightweight small target detection method based on detail feature enhancement according to claim 1, characterized in that: The Neck network module is a PAN network architecture.

Images