Small target detection method inspired by biological vision
By introducing the XYW convolution module and attention mechanism inspired by biovision in the object detection method, the problem of insufficient detection performance of convolutional neural networks in extremely small targets is solved, and more accurate detection effects and performance improvements are achieved.
Patent Information
- Application Number
- CN202411077713.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-12-31
AI Technical Summary
The object detection method based on convolutional neural networks still needs to be improved in optical remote sensing images, especially the detection performance of extremely small targets, and the lack of guidance from biological vision mechanisms increases the inexplicability.
A small object detection method inspired by biological vision was designed. By constructing an XYW convolution module with antagonistic receptive field and a bottom-up and top-down attention mechanism, the contrast between the small object and the surrounding information was enhanced, and the contrast between the small object and the surrounding information was retained. Small target information in the shallow feature layer suppresses large target information and background information, and improves remote sensing small target detection performance.
Effectively extract detailed feature information, significantly improving the detection performance of extremely small targets, and improving the detection performance through the XYWA module and TDSA module. The network design with physiological visual basis is different from the general one-stage object detection model.
Smart Images
Figure CN119229167A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of computer image processing, and in particular to a small target detection method inspired by biological vision. Background Art
[0002] The purpose of target detection is to classify instances in an image and draw bounding boxes around them for positioning. It is a very important core direction of computer vision and is known as one of the three major tasks of CV along with classification and segmentation. Remote sensing target detection is an important application of target detection in the field of earth sciences. There are challenges such as small and densely distributed targets. Due to the rapid development of deep learning, target detection methods based on convolutional neural networks (CNNs) have developed rapidly. Most of these methods use the one-stage target detection network structure of "Backbone-Neck-Head". By improving the feature extraction method and designing feature fusion methods, good performance has been achieved. Experiments have shown that these models have achieved significant results on the Artificial Intelligence Small Target Detection Dataset (AI-TOD).
[0003] Although the one-stage target detection method based on CNN has achieved remarkable results, its detection performance for optical remote sensing image targets, especially very tiny objects, still needs to be improved. At the same time, although the method of constructing target detection methods simply through convolutional neural networks can improve performance, it increases the inexplicability due to the lack of guidance from biological visual mechanisms. Summary of the invention
[0004] The present invention aims to provide a small target detection method inspired by biological vision, which can effectively extract detail feature information to obtain a more accurate detection effect.
[0005] The technical solution of the present invention is as follows:
[0006] The small target detection method inspired by biological vision comprises the following steps:
[0007] A. Constructing a neural network, wherein the neural network includes a Backbone network, a Neck network, and a backbone neckhead module;
[0008] The Backbone network includes a first convolutional layer, a first XYW A module, a first XYWConv module, a second convolutional layer, a second XYW A module, a second XYWConv module, a third convolutional layer, a third XYWConv module, and an SPPF module, which are connected in sequence;
[0009] The SPPF module is the spatial pyramid pooling fusion module;
[0010] B. The original image is input into the Backbone network and processed by the first convolutional layer, the first XYW A module, the first XYWConv module, the second convolutional layer, the second XYW A module, the second XYWConv module, the third convolutional layer, the third XYWConv module, and the SPPF module in sequence; the first convolutional layer, the second convolutional layer, and the third convolutional layer are all downsampling operations;
[0011] The processing results of the second XYWConv module, the third XYWConv module, and the SPPF module are respectively input into the Neck network for processing, and the obtained results are input into the backbone neck head module for processing to obtain the final detection results.
[0012] The first XYWConv module, the second XYWConv module, and the third XYWConv module have the same structure, and respectively include an X cell pathway, a Y cell pathway, and a W cell pathway;
[0013] The processing process in the X cell pathway is as follows: the input result is divided into two paths, one path is processed by 1×1 convolution and ReLU function in sequence to obtain the first path processing result; the other path is processed by 3×3 convolution and ReLU function in sequence to obtain the second path processing result; the first path processing result and the second path processing result are added and fused to obtain the output result;
[0014] The processing process in the Y cell pathway is as follows: the input result is divided into three paths, the first path is processed by 1×1 convolution and ReLU function in sequence to obtain the first path processing result; the second path is processed by 5×5 dilated convolution and ReLU function in sequence to obtain the second path processing result; the third path is processed by 9×9 convolution and ReLU function in sequence to obtain the second path processing result; the first path processing result, the second path processing result, and the third path processing result are added and fused to obtain the output result;
[0015] The processing process in the W cell pathway is as follows: the input result is processed by 1×3 convolution, ReLU function, 3×1 convolution, and ReLU function in sequence to obtain the output result.
[0016] The first XYW A module and the second XYW A module have the same structure, and respectively include a channel attention module and an XYW space attention module;
[0017] The input results are divided into two paths. One path is processed by the channel attention module to obtain the channel attention processing result; the other path is processed by the XYW spatial attention module to obtain the XYW spatial attention processing result;
[0018] After the channel attention processing result and the XYW spatial attention processing result are multiplied and fused, the obtained result is multiplied and fused with the input result, and then the output result is added and fused with the input result.
[0019] The XYW spatial attention module is also provided with an X cell pathway, a Y cell pathway and a W cell pathway. The processing process in the XYW spatial attention module is as follows:
[0020] The input results are divided into three paths. The first path passes through the X cell path and 1×1 convolution in sequence to obtain the first processing result; the second path passes through the Y cell path and 1×1 convolution in sequence to obtain the second processing result; the second path passes through the W cell path and 1×1 convolution in sequence to obtain the third processing result; the first processing result, the second processing result, and the third processing result are added and fused to obtain the output result.
[0021] The processing process in the Neck network is as follows:
[0022] The processing result of the SPPF module is only used for upsampling, and then concatenated with the processing result of the second XYWConv module through the Concat function to obtain processing result N2. The processing result N2 is concatenated with the processing result of the third XYWConv module through the Concat function to obtain processing result N1. The processing result N1 is downsampled to obtain processing result N0. The processing result N2 and the processing result N1 are input into the backbone neck head module; the processing result N2 and the processing result N0 are input into the TDSA attention module for processing to obtain the TDSA processing result, which is input into the backbone neck head module.
[0023] The backbone neck head module processes the processing result N2, the processing result N0, and the TDSA processing result to obtain a final output result.
[0024] The processing process in the TDSA attention module is as follows:
[0025] The processing result N0 is upsampled to the same resolution as the processing result N2, and then processed by the Concat function and the Sigmoid function to obtain the processing result Gs. The processing result Gs is processed by the broadcast mechanism of tensor operation to obtain the suppressed attention weight RAW, which is then multiplied and fused with the processing result N2 to obtain the output result.
[0026] The calculation formula of the broadcast mechanism of tensor operation is:
[0027] RAW=1-G s (1).
[0028] The method of the present invention designs an XYW convolution module with antagonistic receptive field, which can more effectively extract the small target information in the image by enhancing the contrast between the small target and the surrounding information. At the same time, inspired by the bottom-up and top-down attention in biological vision, two different attention mechanisms are designed to better retain the small target information in the shallow feature layer, suppress the large target information and background information, and achieve the purpose of improving the remote sensing small target detection performance.
[0029] The network design of the present invention is supported by physiological vision, which is different from the general one-stage target detection model. At the same time, experiments show that the XYWA module and TDSA module we proposed can improve the detection performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 This is a schematic diagram of the structure of a neural network according to Embodiment 1 of the present invention;
[0031] Figure 2 Schematic diagram of the structure of the XYWConv module of Example 1;
[0032] Figure 3 Schematic diagram of the X cell pathway in Example 1;
[0033] Figure 4 Schematic diagram of the Y cell pathway in Example 1;
[0034] Figure 5 This is a schematic diagram of the W cell pathway of Example 1;
[0035] Figure 6 It is a schematic diagram of the structure of the XYWA module of Example 1;
[0036] Figure 7 Schematic diagram of the structure of the TDSA attention module of Example 1; DETAILED DESCRIPTION
[0037] The present invention is described in detail below with reference to the accompanying drawings and embodiments.
[0038] Example 1
[0039] The small target detection method inspired by biological vision includes the following steps:
[0040] A. Construct a neural network, such as Figure 1 As shown, the neural network includes a Backbone network, a Neck network, and a backbone neck head module;
[0041] The Backbone network includes a first convolutional layer, a first XYW A module, a first XYWConv module, a second convolutional layer, a second XYW A module, a second XYWConv module, a third convolutional layer, a third XYWConv module, and an SPPF module, which are connected in sequence;
[0042] B. The original image is input into the Backbone network and processed by the first convolutional layer, the first XYW A module, the first XYWConv module, the second convolutional layer, the second XYW A module, the second XYWConv module, the third convolutional layer, the third XYWConv module, and the SPPF module in sequence; the first convolutional layer, the second convolutional layer, and the third convolutional layer are all downsampling operations;
[0043] The processing results of the second XYWConv module, the third XYWConv module, and the SPPF module are respectively input into the Neck network for processing, and the obtained results are input into the backbone neck head module for processing to obtain the final detection results.
[0044] like Figure 2 As shown, the first XYWConv module, the second XYWConv module, and the third XYWConv module have the same structure, and include an X cell pathway, a Y cell pathway, and a W cell pathway, respectively;
[0045] The processing process in the X cell pathway is as follows: the input result is divided into two paths, one path is processed by 1×1 convolution and ReLU function in sequence to obtain the first path processing result; the other path is processed by 3×3 convolution and ReLU function in sequence to obtain the second path processing result; the first path processing result and the second path processing result are added and fused to obtain the output result;
[0046] The processing process in the Y cell pathway is as follows: the input result is divided into three paths, the first path is processed by 1×1 convolution and ReLU function in sequence to obtain the first path processing result; the second path is processed by 5×5 dilated convolution and ReLU function in sequence to obtain the second path processing result; the third path is processed by 9×9 convolution and ReLU function in sequence to obtain the second path processing result; the first path processing result, the second path processing result, and the third path processing result are added and fused to obtain the output result;
[0047] The processing process in the W cell pathway is as follows: the input result is processed by 1×3 convolution, ReLU function, 3×1 convolution, and ReLU function in sequence to obtain the output result.
[0048] The first XYW A module and the second XYW A module have the same structure, and respectively include a channel attention module and an XYW space attention module;
[0049] The input results are divided into two paths. One path is processed by the channel attention module to obtain the channel attention processing result; the other path is processed by the XYW spatial attention module to obtain the XYW spatial attention processing result;
[0050] After the channel attention processing result and the XYW spatial attention processing result are multiplied and fused, the obtained result is multiplied and fused with the input result, and then the output result is added and fused with the input result.
[0051] The XYW spatial attention module is also provided with an X cell pathway, a Y cell pathway and a W cell pathway. The processing process in the XYW spatial attention module is as follows:
[0052] The input results are divided into three paths. The first path passes through the X cell path and 1×1 convolution in sequence to obtain the first processing result; the second path passes through the Y cell path and 1×1 convolution in sequence to obtain the second processing result; the second path passes through the W cell path and 1×1 convolution in sequence to obtain the third processing result; the first processing result, the second processing result, and the third processing result are added and fused to obtain the output result.
[0053] The processing process in the Neck network is as follows:
[0054] The processing result of the SPPF module is only used for upsampling, and then concatenated with the processing result of the second XYWConv module through the Concat function to obtain processing result N2. The processing result N2 is concatenated with the processing result of the third XYWConv module through the Concat function to obtain processing result N1. The processing result N1 is downsampled to obtain processing result N0. The processing result N2 and the processing result N1 are input into the backbone neck head module; the processing result N2 and the processing result N0 are input into the TDSA attention module for processing to obtain the TDSA processing result, which is input into the backbone neck head module.
[0055] The backbone neck head module processes the processing result N2, the processing result N0, and the TDSA processing result to obtain a final output result.
[0056] The processing process in the TDSA attention module is as follows:
[0057] The processing result N0 is upsampled to the same resolution as the processing result N2, and then processed by the Concat function and the Sigmoid function to obtain the processing result Gs. The processing result Gs is processed by the broadcast mechanism of tensor operation to obtain the suppressed attention weight RAW, which is then multiplied and fused with the processing result N2 to obtain the output result.
[0058] The calculation formula of the broadcast mechanism of tensor operation is:
[0059] RAW=1-G s (1).
[0060] Example 2
[0061] To conduct a quantitative performance evaluation of the final optical remote sensing image, we adopt the performance measurement standard widely used in the field of target detection. The specific evaluation is shown in formulas (2) and (3).
[0062]
[0063] Where AP represents the integral of R (Recall) on P (precision), the confidence threshold ranges from 0 to 1, mAP represents the average AP value of all categories in the dataset, and N is the number of categories. The higher the mAP value, the stronger the detection performance of the model.
[0064] Table 1 summarizes the ablation experiment data of Example 1 on the AI-TOD dataset. From the experimental results, when compared with the baseline, the model performance is significantly higher when our designed XYWA or TDSA is added; when XYWA and TDSA are added at the same time, the model performance is better than the performance when XYWA or TDSA is used alone.
[0065] Table 1 Ablation experiment
[0066]
[0067]
Claims
1. A small target detection method inspired by biological vision, characterized in that: The following steps are involved: A. Constructing a neural network, wherein the neural network includes a Backbone network, a Neck network, and a backbone neck head module; The Backbone network includes a first convolutional layer, a first XYW A module, a first XYWConv module, a second convolutional layer, a second XYW A module, a second XYWConv module, a third convolutional layer, a third XYWConv module, and an SPPF module, which are connected in sequence; B. The original image is input into the Backbone network and processed by the first convolutional layer, the first XYW A module, the first XYWConv module, the second convolutional layer, the second XYW A module, the second XYWConv module, the third convolutional layer, the third XYWConv module, and the SPPF module in sequence; the first convolutional layer, the second convolutional layer, and the third convolutional layer are all downsampling operations; The processing results of the second XYWConv module, the third XYWConv module, and the SPPF module are respectively input into the Neck network for processing, and the obtained results are input into the backbone neck head module for processing to obtain the final detection results.
2. The small target detection method inspired by biological vision as claimed in claim 1, characterized in that: The first XYWConv module, the second XYWConv module, and the third XYWConv module have the same structure, and respectively include an X cell pathway, a Y cell pathway, and a W cell pathway; The processing process in the X cell pathway is as follows: the input result is divided into two paths, one path is processed by 1×1 convolution and ReLU function in sequence to obtain the first path processing result; the other path is processed by 3×3 convolution and ReLU function in sequence to obtain the second path processing result; the first path processing result and the second path processing result are added and fused to obtain the output result; The processing process in the Y cell pathway is as follows: the input result is divided into three paths, the first path is processed by 1×1 convolution and ReLU function in sequence to obtain the first path processing result; the second path is processed by 5×5 dilated convolution and ReLU function in sequence to obtain the second path processing result; The third path is processed by 9×9 convolution and ReLU function in sequence to obtain the processing result of the second path; the processing results of the first path, the second path, and the third path are added and fused to obtain the output result; The processing process in the W cell pathway is as follows: the input result is processed by 1×3 convolution, ReLU function, 3×1 convolution, and ReLU function in sequence to obtain the output result.
3. The small target detection method inspired by biological vision as claimed in claim 2, characterized in that: The first XYW A module and the second XYW A module have the same structure, and respectively include a channel attention module and an XYW space attention module; The input results are divided into two paths, one of which is processed by the channel attention module to obtain the channel attention processing result; The other path is processed by the XYW spatial attention module to obtain the XYW spatial attention processing result; After the channel attention processing result and the XYW spatial attention processing result are multiplied and fused, the obtained result is multiplied and fused with the input result, and then the output result is added and fused with the input result.
4. The small target detection method inspired by biological vision as claimed in claim 3, characterized in that: The XYW spatial attention module is also provided with an X cell pathway, a Y cell pathway and a W cell pathway. The processing process in the XYW spatial attention module is as follows: The input results are divided into three paths. The first path passes through the X cell path and 1×1 convolution in sequence to obtain the first path processing result; the second path passes through the Y cell path and 1×1 convolution in sequence to obtain the second path processing result; the second path passes through the W cell path and 1×1 convolution in sequence to obtain the third path processing result; The output result is obtained by adding and fusing the first processing result, the second processing result, and the third processing result.
5. The small target detection method inspired by biological vision as claimed in claim 1, characterized in that: The processing process in the Neck network is as follows: The processing result of the SPPF module is only used for upsampling, and then concatenated with the processing result of the second XYWConv module through the Concat function to obtain processing result N2. The processing result N2 is concatenated with the processing result of the third XYWConv module through the Concat function to obtain processing result N1. The processing result N1 is downsampled to obtain processing result N0. The processing result N2 and the processing result N1 are input into the backbone neck head module; the processing result N2 and the processing result N0 are input into the TDSA attention module for processing to obtain the TDSA processing result, which is input into the backbone neck head module.
6. The small target detection method inspired by biological vision as claimed in claim 5, characterized in that: The backbone neck head module processes the processing result N2, the processing result N0, and the TDSA processing result to obtain a final output result.
7. The small target detection method inspired by biological vision as claimed in claim 5, characterized in that: The processing process in the TDSA attention module is as follows: The processing result N0 is upsampled to the same resolution as the processing result N2, and then processed by the Concat function and the Sigmoid function to obtain the processing result Gs. The processing result Gs is processed by the broadcast mechanism of tensor operation to obtain the suppressed attention weight RAW, which is then multiplied and fused with the processing result N2 to obtain the output result.
8. The small target detection method inspired by biological vision as claimed in claim 7, characterized in that: The calculation formula of the broadcast mechanism of tensor operation is: RAW=1-G s (1)。