Small target detection method inspired by visual v2 area information processing mechanism

Abstract

The invention aims to provide a small target detection method inspired by a visual v2 area information processing mechanism, comprising the following steps: A, constructing a neural network which comprises a Backbone network, a Neck network module and a Head network module; a CCM module, an HCM module and a C2fDCN-D module are applied to the Backbone network module. B, inputting an original image into a Backbone network module, and processing the original image through a CCM module, an HCM module, a first C2fDCN-D module, a CBR module, a second C2fDCN-D module, an SPPF module and a third C2fDCN-D module in sequence; a processing result of the first C2fDCN-D module, a result of the second C2fDCN-D module and a processing result of the third C2fDCN-D module are input into the Neck network; and C, performing multi-scale feature fusion on the input features by the Neck network module to obtain two feature layers with different resolutions, and respectively inputting the two feature layers into the Head network module for detection to obtain a series of detection frames and category data, namely a final result. According to the method, feature extraction and fusion can be more effectively carried out, and the small target detection performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer image processing, and in particular to a small target detection method inspired by the information processing mechanism of visual area V2. Background Art

[0002] Object detection is a core problem in computer vision research. It aims to accurately locate and classify objects in images or videos, laying the foundation for higher-level vision tasks such as segmentation, scene understanding, and object tracking. In the era of deep learning, object detection algorithms have made significant progress. To improve small object detection, researchers have employed various methods based on single-stage detection models to extract features in complex backgrounds and with sparse features. These methods enhance small object detection capabilities by adding a small object detection layer and refining feature extraction fusion methods. Experimental results demonstrate that the improved model achieves promising results in small object datasets, such as drone aerial imagery.

[0003] Although the one-stage target detection method has achieved good results, in high-resolution drone aerial images, due to the low feature density and small pixels of the target to be detected, it is difficult to capture subtle details during the feature extraction process, which can easily lead to missed detections and false detections. Summary of the Invention

[0004] The present invention aims to provide a small target detection method inspired by the information processing mechanism of visual area V2. The neural network model constructed by this method can more effectively extract and fuse features to more comprehensively capture the features of the target, thereby improving the accuracy and enhancing the detection performance of small targets.

[0005] The technical solutions of the present invention are as follows: The small target detection method inspired by the information processing mechanism of visual area V2 includes the following steps: A. Constructing a neural network, wherein the neural network includes a Backbone network, a Neck network module, and a Head network module; The Backbone network module uses the CCM module, HCM module, and C2f_DCN-D module; B. The original image is input into the Backbone network module and processed in sequence by the CCM module, HCM module, first C2f_DCN-D module, CBR module, second C2f_DCN-D module, SPPF module, and third C2f_DCN-D module; The processing result of the first C2f_DCN-D module, the processing result of the second C2f_DCN-D module, and the processing result of the third C2f_DCN-D module are input into the Neck network; C. The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions, which are input into the Head network module for detection respectively. A series of detection boxes and category data are obtained, which is the final result.

[0006] The processing process in the HCM module is as follows: The input result is divided into four branches. The first branch is processed by the CCM module to obtain the first branch result; the second branch is processed by four directional convolutions of 0°, 90°, 225°, and 315° respectively. The four directional convolution results are multiplied by the weight ω respectively, and then added and fused, and then processed by 1*1 convolution to obtain the second branch result; the third branch is processed by 3*3 convolution to obtain the third branch result; the fourth branch is processed by 3*3 convolution, batch normalization, and ReLU6 function in sequence to obtain the fourth branch result; After the second branch result is added and fused with the third branch result, it is processed by batch normalization and ReLU6 function in sequence, and then spliced ​​with the first branch result and the fourth branch result through the Concat function to obtain the output result.

[0007] The processing process in the CCM module is as follows: the input result is divided into three paths. The first path is processed by 3*3 convolution to obtain the first path result; the second path is processed by 45° Gabor convolution to obtain the second path result; the third path is processed by 135° Gabor convolution to obtain the third path result; After the second and third channel results are added and fused, a single-channel addition fusion result is obtained. The single-channel addition fusion result is concatenated with the Concat function three times to obtain a three-channel result, which is then processed by 1*1 convolution. The result is added and fused with the first channel result, and then processed by batch normalization and ReLU6 function in sequence to obtain the output result.

[0008] The processing process in the Neck network module is as follows: The result of the third C2f_DCN-D module is upsampled and concatenated with the result of the first C2f_DCN-D module through the Concat function. The result is then processed by the fourth C2f_DCN-D module and the Repconv module in sequence. The result is then concatenated with the result of the second C2f_DCN-D module through the Concat function and processed by the fifth C2f_DCN-D module. The processing result of the fourth C2f_DCN-D module and the processing result of the fifth C2f_DCN-D module are input into the SEAM module for processing to obtain the SEAM module processing result; The processing results of the SEAM module and the fourth C2f_DCN-D module are respectively input into the Head network.

[0009] The first to fifth C2f_DCN-D modules have the same structure, wherein the processing process is as follows: After the input structure is convolved, it is split into the first and second parts by the Split function. The first part is processed by the Bottleneck_DCN-D module, and the result is concatenated with the first and second parts by the Concat function, and then convolved to obtain the output result.

[0010] The processing in the Bottleneck_DCN-D module is as follows: The input result is divided into two branches. The first branch undergoes two depth-wise separable convolution processes in sequence to obtain the first branch result; the second branch undergoes two deformable convolution processes in sequence to obtain the second branch result; the first branch result, the second branch result, and the input result are added and fused to obtain the output result.

[0011] The processing process in the SEAM module is as follows: The processing result of the fifth C2f_DCN-D module is processed by the GroupNorm function to obtain the group normalization result. The group normalization result is multiplied by the result of the channel normalization of the Gamma weight, and then processed by the Sigmoid function. The result is grouped based on the set value: the value greater than or equal to the set value is the key group A, and the value less than the set value is the non-key group A. After multiplying the key group A by the result of the fifth C2f_DCN-D module processing, the result is grouped according to the weight based on the set value. The value greater than or equal to the set value is grouped as key group B, and the value less than the set value is grouped as non-key group B. After multiplying the non-key group A by the result of the fifth C2f_DCN-D module processing, the result is grouped according to the weight based on the set value. The value greater than or equal to the set value is grouped as key group C, and the value less than the set value is grouped as non-key group C. The key group B is added and fused with the non-key group C, and the key group C is added and fused with the non-key group B. The two added fusion results are then concatenated using the Concat function and subjected to a 1*1 convolution. The resulting result is then added and fused with the result processed by the fourth C2f_DCN-D module to obtain the intermediate result. The intermediate results are divided into two paths. After average pooling processing, the two paths are spliced ​​by the Concat function, and then processed by convolution, BN function, and ReLU function in sequence. The results are split into the first segmentation and the second segmentation by the Split function; the first segmentation is multiplied and fused with the intermediate result to obtain the first multiplication result; the second segmentation is multiplied and fused with the intermediate result to obtain the second multiplication result; the first multiplication result is multiplied and fused with the second multiplication result to obtain the output result.

[0012] The Head network module is provided with two Head detection heads. The processing result of the SEAM module and the processing result of the fourth C2f_DCN-D module are respectively input into one Head detection head for processing.

[0013] Both Head detection heads are built-in detection heads of the YOLO network.

[0014] The present invention designs a complex cell module that mimics the visual V2 area, namely the CCM module. Other studies have shown that neurons in the visual V1 area are similar to a series of two-dimensional Gabor filters. Simple cells can be modeled using filters with different directional bandwidths, spatial frequency bandwidths, and spatial dimensions. Different neurons can correspond to edges of different frequencies and directions. The present invention uses Gabor convolution kernels in 45° and 135° directions as the optimal directional receptive fields to simulate simple cells in downstream neurons and build a complex cell module; the results of these Gabor convolutions in different directions are integrated and further processed through 1*1 convolution to simulate the sensitive characteristics of complex cells to edges, so as to extract more edge and texture information. At the same time, the original input is processed in parallel with the Gabor convolution module through 3*3 convolution to facilitate better extraction of global information.

[0015] The present invention designs a hypercomplex cell module, namely the HCM module. The receptive field of hypercomplex cells is formed by the convergence of complex cells, which is very sensitive to the direction of stimulation, and hypercomplex cells have the best stimulus response to complex graphics and special positions such as interrupted or edges, corners and corners. Inspired by the above biological mechanism, the present invention designs a hypercomplex cell module that simulates the visual v2 area. The present invention selects four directional convolution kernels of 0°, 90°, 225°, and 315° to form the Dconv convolution module, and processes it through 1*1 convolution, which can effectively extract complex edge shape features in all directions. The present invention also directly integrates the features processed by the convolution template with the features processed by the complex cell module and the features directly processed by 3*3 convolution to simulate the sensitive characteristics of hypercomplex cells to edges, complex graphics and special positions.

[0016] Inspired by the bottom-up visual attention mechanism, this paper designs a spatial attention enhancement module, or SEAM. This module simulates the bottom-up process of visual attention by modulating deep features with shallow features. The SRU (Spatial Reconstruction Unit) module from SCConv is introduced to enhance representative information through weight separation. This module identifies and suppresses redundant features in the spatial dimension, enhances feature expressiveness, and reduces the number of parameters. The position attention module from CPAM (Channel and Position Attention Mechanism) is also introduced to extract representative feature information from different channels. Shallow features first enter the spatial reconstruction module for representative information screening. After interpolation and 1x1 convolution, they are input together with deep features into the position attention module. Shallow information modulates deep information, allowing the network to focus on small targets while retaining shallow feature information.

[0017] The proposed small target detection method, inspired by the information processing mechanism of the visual field V2, and the neural network constructed by it can effectively extract detailed feature information and effectively reduce missed detections and false detections. Furthermore, the proposed method utilizes a lightweight network structure design, achieving good detection results despite a small number of parameters. Experiments have also demonstrated that the proposed CCM, HCM, and SEAM modules can all improve detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the structure of the neural network of Example 1 of the present invention; Figure 2 Schematic diagram of the structure of the HCM module of Example 1; Figure 3 Schematic diagram of the structure of the CCM module of Example 1; Figure 4 This is a schematic structural diagram of the C2f_DCN-D module of Example 1; Figure 5 Schematic diagram of the structure of the SEAM module of Example 1; Figure 6 The following figure shows the comparison results of the detection method in Example 1, Ground Truth, and Baseline on the VisDrone2019 dataset. Figure 7 The following figure shows the detection comparison results of the method in Example 1, Ground Truth, and Baseline on the AI-TODv2 dataset. DETAILED DESCRIPTION

[0019] The present invention is described in detail below with reference to the accompanying drawings and embodiments. Example 1

[0020] The small target detection method inspired by the information processing mechanism of visual area V2 includes the following steps: A. Construct Figure 1 The neural network shown includes a Backbone network, a Neck network module, and a Head network module; The Backbone network module uses the CCM module, HCM module, and C2f_DCN-D module; B. The original image is input into the Backbone network module and processed in sequence by the CCM module, HCM module, first C2f_DCN-D module, CBR module, second C2f_DCN-D module, SPPF module, and third C2f_DCN-D module; The processing result of the first C2f_DCN-D module, the processing result of the second C2f_DCN-D module, and the processing result of the third C2f_DCN-D module are input into the Neck network; like Figure 2 As shown, the processing process in the HCM module is as follows: The input result is divided into four branches. The first branch is processed by the CCM module to obtain the first branch result; the second branch is processed by four directional convolutions of 0°, 90°, 225°, and 315° respectively. The four directional convolution results are multiplied by the weight ω respectively, and then added and fused, and then processed by 1*1 convolution to obtain the second branch result; the third branch is processed by 3*3 convolution to obtain the third branch result; the fourth branch is processed by 3*3 convolution, batch normalization, and ReLU6 function in sequence to obtain the fourth branch result; in this embodiment, ω is set to 0.25; After the second branch result is added and fused with the third branch result, it is processed by batch normalization and ReLU6 function in sequence, and then spliced ​​with the first branch result and the fourth branch result through the Concat function to obtain the output result.

[0021] like Figure 3 As shown, the processing process in the CCM module is as follows: the input result is divided into three paths, the first path is processed by 3*3 convolution to obtain the first path result; the second path is processed by 45° direction Gabor convolution to obtain the second path result; the third path is processed by 135° direction Gabor convolution to obtain the third path result; After the second and third channel results are added and fused, a single-channel addition fusion result is obtained. The single-channel addition fusion result is concatenated with the Concat function three times to obtain a three-channel result, which is then processed by 1*1 convolution. The result is added and fused with the first channel result, and then processed by batch normalization and ReLU6 function in sequence to obtain the output result.

[0022] C. The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions, which are input into the Head network module for detection respectively. A series of detection boxes and category data are obtained, which is the final result.

[0023] The processing process in the Neck network module is as follows: The result of the third C2f_DCN-D module is upsampled and concatenated with the result of the first C2f_DCN-D module through the Concat function. The result is then processed by the fourth C2f_DCN-D module and the Repconv module in sequence. The result is then concatenated with the result of the second C2f_DCN-D module through the Concat function and processed by the fifth C2f_DCN-D module. The processing result of the fourth C2f_DCN-D module and the processing result of the fifth C2f_DCN-D module are input into the SEAM module for processing to obtain the SEAM module processing result; The processing results of the SEAM module and the fourth C2f_DCN-D module are respectively input into the Head network.

[0024] like Figure 4 As shown, the first to fifth C2f_DCN-D modules have the same structure, wherein the processing process is as follows: After the input structure is convolved, it is split into the first and second parts by the Split function. The first part is processed by the Bottleneck_DCN-D module, and the result is concatenated with the first and second parts by the Concat function, and then convolved to obtain the output result.

[0025] The processing in the Bottleneck_DCN-D module is as follows: The input result is divided into two branches. The first branch undergoes two depth-wise separable convolution processes in sequence to obtain the first branch result; the second branch undergoes two deformable convolution processes in sequence to obtain the second branch result; the first branch result, the second branch result, and the input result are added and fused to obtain the output result.

[0026] like Figure 5 As shown, the processing process in the SEAM module is as follows: The processing result of the fifth C2f_DCN-D module is processed by the GroupNorm function to obtain a group normalization result. The group normalization result is multiplied by the result of the channel-by-channel normalization of the Gamma weight, and then processed by the Sigmoid function. The result is then grouped based on the set value: the value greater than or equal to the set value is the key group A, and the value less than the set value is the non-key group A. In this embodiment, the set value is 0.5. After multiplying the key group A by the result of the fifth C2f_DCN-D module processing, the result is grouped according to the weight based on the set value. The value greater than or equal to the set value is grouped as key group B, and the value less than the set value is grouped as non-key group B. After multiplying the non-key group A by the result of the fifth C2f_DCN-D module processing, the result is grouped according to the weight based on the set value. The value greater than or equal to the set value is grouped as key group C, and the value less than the set value is grouped as non-key group C. The key group B is added and fused with the non-key group C, and the key group C is added and fused with the non-key group B. The two added fusion results are then concatenated using the Concat function and subjected to a 1*1 convolution. The resulting result is then added and fused with the result processed by the fourth C2f_DCN-D module to obtain the intermediate result. The intermediate results are divided into two paths. After average pooling processing, the two paths are spliced ​​by the Concat function, and then processed by convolution, BN function, and ReLU function in sequence. The results are split into the first segmentation and the second segmentation by the Split function; the first segmentation is multiplied and fused with the intermediate result to obtain the first multiplication result; the second segmentation is multiplied and fused with the intermediate result to obtain the second multiplication result; the first multiplication result is multiplied and fused with the second multiplication result to obtain the output result.

[0027] The Head network module is provided with two Head detection heads, both of which are built-in detection heads of the YOLO network. The processing results of the SEAM module and the processing results of the fourth C2f_DCN-D module are respectively input into a Head detection head for processing. Example 2

[0028] In the experiments of this example, we used evaluation metrics commonly used in the field of object detection, including Average Precision (AP) and Mean Average Precision (mAP). For the VisDrone2019 dataset, the evaluation metrics for small object detection algorithms mainly include precision, recall, mAP50, and mAP50:95. The specific evaluation methods are shown in formulas (1) and (2).

[0029] (1) (2) Here, R represents recall, P represents precision, AP represents the integral of R over P, the confidence threshold ranges from 0 to 1, mAP represents the average AP value of all categories in the dataset, and N is the number of categories. The higher the mAP value, the stronger the detection performance of the model.

[0030] In addition to using AP and mAP to evaluate model performance, this example also introduces parameter counts (Params) to analyze model efficiency. Params is a direct indicator of model size, and its calculation method is shown in Equation (3).

[0031] (3) in, * Represents the size of the convolution kernel, Indicates the number of input channels in the convolution operation, Indicates the number of output channels.

[0032] Table 1 summarizes the ablation experiment data from Example 1 on the VisDrone dataset for small drone object detection. The experimental results show that, compared to the baseline network, the model significantly outperforms the baseline when our CCM, HCM, and SEAM are added one by one. Furthermore, the proposed module achieves a good balance between detection accuracy and model lightweightness.

[0033] Table 1 Ablation experiment (VisDrone) Example 3

[0034] The method in Example 1 is compared with the Ground Truth and Baseline methods on the VisDrone2019 dataset and the AI-TODv2 dataset, respectively. The results are as follows: Figure 6 and Figure 7 As shown, Figure 6 This is the effect diagram on the VisDrone2019 dataset. Figure 7 This is the effect diagram on the AI-TODv2 dataset. In the figure, ours represents the data of Example 1. Figure 6 and Figure 7 The Chinese characters indicate the test results for each image, which are the number of correct detections (right), the number of false detections (missing), and the number of missed detections (error). Experiments show that our proposed model can effectively reduce the number of false detections and missed detections.

Claims

1. A small target detection method inspired by the information processing mechanism of visual area V2, characterized by: The following steps are involved: A. Constructing a neural network, wherein the neural network includes a Backbone network, a Neck network module, and a Head network module; The Backbone network module uses the CCM module, HCM module, and C2f_DCN-D module; B. The original image is input into the Backbone network module and processed in sequence by the CCM module, HCM module, first C2f_DCN-D module, CBR module, second C2f_DCN-D module, SPPF module, and third C2f_DCN-D module; The processing result of the first C2f_DCN-D module, the processing result of the second C2f_DCN-D module, and the processing result of the third C2f_DCN-D module are input into the Neck network; C. The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions, which are input into the Head network module for detection respectively. A series of detection boxes and category data are obtained, which is the final result.

2. The small target detection method inspired by the information processing mechanism of visual area V2 according to claim 1, characterized in that: The processing process in the HCM module is as follows: The input result is divided into four branches. The first branch is processed by the CCM module to obtain the first branch result; the second branch is processed by four directional convolutions of 0°, 90°, 225°, and 315° respectively. The four directional convolution results are multiplied by the weight ω respectively, and then added and fused, and then processed by 1*1 convolution to obtain the second branch result; the third branch is processed by 3*3 convolution to obtain the third branch result; the fourth branch is processed by 3*3 convolution, batch normalization, and ReLU6 function in sequence to obtain the fourth branch result; After the second branch result is added and fused with the third branch result, it is processed by batch normalization and ReLU6 function in sequence, and then spliced ​​with the first branch result and the fourth branch result through the Concat function to obtain the output result.

3. The small target detection method inspired by the information processing mechanism of visual area V2 according to claim 2, characterized in that: The processing process in the CCM module is as follows: the input result is divided into three paths. The first path is processed by 3*3 convolution to obtain the first path result; the second path is processed by 45° Gabor convolution to obtain the second path result; the third path is processed by 135° Gabor convolution to obtain the third path result; After the second and third channel results are added and fused, a single-channel addition fusion result is obtained. The single-channel addition fusion result is concatenated with the Concat function three times to obtain a three-channel result, which is then processed by 1*1 convolution. The result is added and fused with the first channel result, and then processed by batch normalization and ReLU6 function in sequence to obtain the output result.

4. The small target detection method inspired by the information processing mechanism of visual area V2 as claimed in claim 1, characterized in that: The processing process in the Neck network module is as follows: The result of the third C2f_DCN-D module is upsampled and concatenated with the result of the first C2f_DCN-D module through the Concat function. The result is then processed by the fourth C2f_DCN-D module and the Repconv module in sequence. The result is then concatenated with the result of the second C2f_DCN-D module through the Concat function and processed by the fifth C2f_DCN-D module. The processing result of the fourth C2f_DCN-D module and the processing result of the fifth C2f_DCN-D module are input into the SEAM module for processing to obtain the SEAM module processing result; The processing results of the SEAM module and the fourth C2f_DCN-D module are respectively input into the Head network.

5. The small target detection method inspired by the information processing mechanism of visual area V2 according to claim 4, characterized in that: The first to fifth C2f_DCN-D modules have the same structure, wherein the processing process is as follows: After the input structure is convolved, it is split into the first and second parts by the Split function. The first part is processed by the Bottleneck_DCN-D module, and the result is concatenated with the first and second parts by the Concat function, and then convolved to obtain the output result.

6. The small target detection method inspired by the information processing mechanism of visual area V2 according to claim 5, characterized in that: The processing in the Bottleneck_DCN-D module is as follows: The input result is divided into two branches. The first branch undergoes two depth-wise separable convolution processes in sequence to obtain the first branch result; the second branch undergoes two deformable convolution processes in sequence to obtain the second branch result; the first branch result, the second branch result, and the input result are added and fused to obtain the output result.

7. The small target detection method inspired by the information processing mechanism of visual area V2 according to claim 4, characterized in that: The processing process in the SEAM module is as follows: The processing result of the fifth C2f_DCN-D module is processed by the GroupNorm function to obtain the group normalization result. The group normalization result is multiplied by the result of the channel normalization of the Gamma weight, and then processed by the Sigmoid function. The result is grouped based on the set value: the value greater than or equal to the set value is the key group A, and the value less than the set value is the non-key group A. After multiplying the key group A by the result of the fifth C2f_DCN-D module processing, the result is grouped according to the weight based on the set value. The value greater than or equal to the set value is grouped as key group B, and the value less than the set value is grouped as non-key group B. After multiplying the non-key group A by the result of the fifth C2f_DCN-D module processing, the result is grouped according to the weight based on the set value. The value greater than or equal to the set value is grouped as key group C, and the value less than the set value is grouped as non-key group C. The key group B is added and fused with the non-key group C, and the key group C is added and fused with the non-key group B. The two added fusion results are then concatenated using the Concat function and subjected to a 1*1 convolution. The resulting result is then added and fused with the result processed by the fourth C2f_DCN-D module to obtain the intermediate result. The intermediate results are divided into two paths. After average pooling processing, the two paths are spliced ​​by the Concat function, and then processed by convolution, BN function, and ReLU function in sequence. The results are split into the first segmentation and the second segmentation by the Split function; the first segmentation is multiplied and fused with the intermediate result to obtain the first multiplication result; the second segmentation is multiplied and fused with the intermediate result to obtain the second multiplication result; the first multiplication result is multiplied and fused with the second multiplication result to obtain the output result.

8. The small target detection method inspired by the information processing mechanism of visual area V2 according to claim 4, characterized in that: The Head network module is provided with two Head detection heads. The processing result of the SEAM module and the processing result of the fourth C2f_DCN-D module are respectively input into one Head detection head for processing.

9. The small target detection method inspired by the information processing mechanism of visual area V2 according to claim 8, characterized in that: Both Head detection heads are built-in detection heads of the YOLO network.

Images