A small target detection method for unmanned aerial vehicle inspired by hawk eye vision mechanism

Abstract

The present application aims to provide a kind of UAV small target detection method inspired by hawk eye vision mechanism, comprising the following steps: A, construct neural network, the neural network includes Backbone network, Neck network module, Head network module;The DFB module is applied in the Backbone network module to simulate the deep foveal structure of hawk eye, and the SFB module is applied to simulate the shallow foveal structure of hawk eye;B, original image is input into Backbone network module, and the resulting processing result is input into Neck network module respectively;C, multi-scale feature fusion is carried out on input feature in Neck network module, two different resolution feature layers are obtained, and are input into Head network module for detection respectively, a series of detection frame and category data are obtained, i.e. final result.The present application method can efficiently extract the feature information of micro target, and accurately locate target in complex scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and image processing, in particular to a UAV small target detection method inspired by the hawk eye vision mechanism. BACKGROUND

[0002] Target detection technology plays an important role in the field of computer vision and is widely used in image classification, instance segmentation, target tracking and other tasks. However, the task of small target detection under the perspective of UAV faces many challenges, including small target size, complex background and large model parameter quantity. Traditional target detection methods mostly use the "Backbone-Neck-Head" structure constructed by convolutional neural network (CNN) to improve detection performance by improving feature extraction and multi-scale feature fusion. In recent years, YOLO series methods and various improved models have shown good results in UAV image detection, but there is still a lack of balance between detection accuracy and computational resource demand.

[0003] Inspired by bionic vision systems, researchers try to simulate the characteristics of biological vision systems to improve detection performance. For example, the vision system of eagles has a double fovea structure, which enables it to have excellent small target capture ability, providing design inspiration for bionic vision networks. However, existing bionic designs mostly only consider the basic structure of the vision system and do not fully utilize the characteristics of the eagle's perioptome pathway, double fovea structure and large receptive field of thalamic round nucleus, which limits the optimization between model performance and lightweight. SUMMARY

[0004] The present application aims to provide a UAV small target detection method inspired by the hawk eye vision mechanism, which can efficiently extract the feature information of small targets and accurately locate the targets in complex scenes.

[0005] The technical solution of the present application is as follows:

[0006] The UAV small target detection method inspired by the hawk eye vision mechanism comprises the following steps:

[0007] A. Construct a neural network, wherein the neural network comprises a Backbone network, a Neck network module and a Head network module;

[0008] The DFB module and the SFB module are applied in the Backbone network module;

[0009] B. The original image is input into the Backbone network module, and after being processed by the Silence module, the processing result of the Silence module is divided into two paths:

[0010] The first path is sequentially processed by the first CBS module, the second CBS module, the first DFB module, the second DFB module, the first Adown module, the third DFB module, the second Adown module and the fourth DFB module; and the second path is sequentially processed by the third CBS module and the fourth CBS module and the first SFB module.

[0011] The second DFB module processing result is processed by the first CBlinear module to obtain a first CBlinear module processing result; and the third DFB module processing result is processed by the second CBlinear module to obtain a second CBlinear module processing result.

[0012] The first SFB module processing result, the first CBlinear module processing result, the second CBlinear module processing result, the second DFB module processing result, the third DFB module processing result and the fourth DFB module processing result are input into the Neck network module.

[0013] C. The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions, which are input into the Head network module for detection to obtain a series of detection frames and category data, which are the final results.

[0014] The first DFB module, the second DFB module, the third DFB module and the fourth DFB module have the same structure, and the processing process is as follows:

[0015] The input result is processed by the GELAN module and divided into four paths; the first path is sequentially processed by a 1*1 convolution and a 3*3 convolution to obtain a first path result; the second path is sequentially processed by a 1*1 convolution, a 1*3 convolution, a 3*1 convolution and a 3*3 Atrous module to obtain a second path result; the third path is sequentially processed by a 1*1 convolution, a 3*1 convolution, a 1*3 convolution and a 3*3 Atrous module to obtain a third path result; and the fourth path is processed by a 1*1 convolution to obtain a fourth path result.

[0016] The first path result, the second path result, the third path result and the fourth path result are spliced by a Concat function to obtain an output result.

[0017] The processing process in the Neck network module is as follows:

[0018] The fifth DFB module processing result and the first CBlinear module processing result are input into the first CBFuse module for processing, and the processing result of the first CBFuse module is sequentially processed by the second SFB module and the first NRT module to obtain a first NRT module processing result.

[0019] The second SFB module processing result is processed by the third Adown module to obtain a third Adown module processing result, and the first CBlinear module processing result, the second CBlinear module processing result and the third Adown module processing result are input into the second module for processing to obtain a second CBFuse module processing result, which is processed by the third SFB module and the second NRT module to obtain a second NRT module processing result;

[0020] The fourth DFB module processing result is processed by the SPPELAN module, and then up-sampling is performed to obtain a first up-sampling result, the first up-sampling result and the third DFB module processing result are spliced by the Concat function, and the spliced result is processed by the first GELAN module to obtain a first GELAN module processing result;

[0021] The first GELAN module processing result is up-sampled, and then spliced with the second DFB module processing result by the Concat function, and the spliced result is processed by the second GELAN module and the third NRT module in sequence to obtain a third NRT module processing result;

[0022] The second GELAN module processing result is processed by the third Adown module to obtain a third Adown module processing result, and the first GELAN module processing result and the third Adown module processing result are spliced by the Concat function, and the spliced result is processed by the third GELAN module and the fourth NRT module in sequence to obtain a fourth NRT module processing result;

[0023] The first NRT module processing result, the second NRT module processing result, the third NRT module processing result and the fourth NRT module processing result are input into the Head network module.

[0024] The first SFB module, the second SFB module and the third SFB module have the same structure, and the processing process is as follows:

[0025] After the input result is processed by the GELAN module, it is divided into four branches; the first branch is processed by 1*1 convolution and 7*7 convolution in sequence to obtain a first branch result; the second branch is processed by 1*1 convolution, 1*7 convolution, 7*1 convolution and 7*7 Atrous module in sequence to obtain a second branch result; the third branch is processed by 1*1 convolution, 7*1 convolution, 1*7 convolution and 7*7 Atrous module in sequence to obtain a third branch result; and the fourth branch is processed by 1*1 convolution to obtain a fourth branch result;

[0026] The first branch result, the second branch result, the third branch result and the fourth branch result are spliced by the Concat function to obtain an output result.

[0027] The first NRT module, the second NRT module, the third NRT module and the fourth NRT module have the same structure, and the processing process is as follows:

[0028] The input result is divided into three paths, the first path and the second path are subjected to 1*1 convolution processing respectively, the obtained two convolution processing results are multiplied and fused, and then subjected to Softmax module processing to obtain a Softmax module processing result;

[0029] The third path is subjected to Rotundal cell module processing, and then the obtained result is multiplied and fused with the Softmax module processing result, and then subjected to 1*1 convolution processing, and then the obtained convolution result is added and fused with the input result to obtain an output result.

[0030] The processing process in the Rotundal cell module is as follows:

[0031] The input result is subjected to 1*3 convolution, 3*1 convolution, 1*5 deep separable convolution, 5*1 deep separable convolution and 1*1 convolution processing in sequence, and then the obtained processing result is multiplied and fused with the input result to obtain an output result.

[0032] The hole coefficients of the 1*5 deep separable convolution and the 5*1 deep separable convolution are both 3.

[0033] In the Head network module, four Head detection heads are provided, and the processing result of each NRT module is input into a Head detection head for processing.

[0034] The four Head detection heads are detection heads provided by the YOLO network.

[0035] The DFB module is used to simulate the deep fovea structure of the hawk eye, and the SFB module is used to simulate the shallow fovea structure of the hawk eye.

[0036] The DFB module simulates the introduction of multi-branch Atlas convolution, so that different scale local context information can be captured, especially the details of small objects are strengthened, so that higher visual acuity can be achieved, and small objects can be accurately perceived.

[0037] In addition to introducing multi-path Atlas convolution, the SFB module uses a larger size convolution kernel (such as 1x7, 7x1 and 7x7), so as to enhance the feature extraction capability of the model in a larger range. This design helps the model to better match and fuse global semantic information when processing large-scale background information, thereby improving the recognition effect of small objects in a complex background.

[0038] The combination of the two structures makes the hawk eye have an unparalleled visual advantage in a dynamic environment. Through the synergistic effect of the two, the EVMNet network of the application can better extract rich global-local semantic information in multi-scale feature fusion, effectively improving the detection capability of small objects, especially in complex background and dense object scenes.

[0039] The application also designs a bionic round nucleus cell module, namely the NRt module, in the Neck network part. The NRt module simulates the large receptive field and global search characteristics of the thalamic round nucleus by using a depth separable convolution and a global attention architecture. First, an equivalent large kernel convolution is used to simulate the round nucleus cell. The equivalent large kernel convolution is composed of two different depth separable convolutions, which not only maintains the lightweight, but also realizes the effect of the large kernel convolution with a receptive field equivalent to a 11 convolution kernel. Then, the round nucleus cell module is combined into the global attention architecture to further simulate the visual processing function of the thalamic round nucleus. The global attention mechanism aggregates global context information by calculating the correlation between each pair of pixels, so that the model can effectively capture the mutual relationship between different positions and improve the ability to capture long-distance dependencies, similar to the role of the thalamic round nucleus in visual processing. In this way, the NRt module can not only focus on local features, but also capture long-distance dependency information in a global range, thereby significantly improving the performance of small target detection.

[0040] The application has the following beneficial effects:

[0041] In the method of the application, the hawk eye visual neural structure is taken as a prototype, the structural characteristics of the double fovea and the peripeduncular pathway are combined, and the feature extraction module and the information fusion module are innovatively designed, thereby improving the small target detection precision under the image of an unmanned aerial vehicle. Meanwhile, the model has the lightweight characteristic and is suitable for the deployment requirements of edge devices.

[0042] The lightweight bionic unmanned aerial vehicle small target detection network EVMNet proposed in the application effectively improves the accuracy of small target detection and exhibits superior performance on multiple test data sets. On the VisDrone2019 data set, compared with YOLOv9t, the detection precision of EVMNet is improved by 12.5%, reaching 49.2% of mAP0.5, and the parameter amount is only 1.46M, which is suitable for resource-limited unmanned aerial vehicle platform applications. In addition, the bionic design of the double fovea module and the NRt module verifies the advantages of multi-scale feature extraction and background interference suppression in the experiment, so that EVMNet realizes a good balance between performance and lightweight in various indicators. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 Fig. 1 is a structural schematic diagram of the neural network of Embodiment 1 of the application;

[0044] Figure 2 Structure diagram of the DFB module of Example 1;

[0045] Figure 3 Structure diagram of the SFB module of Example 1;

[0046] Figure 4 Structure diagram of the NRt module of Example 1;

[0047] Figure 5 Structure diagram of the Rotundal cell module of Example 1;

[0048] Figure 6 Detection comparison result diagram of the method of Example 1 and MHA-YOLO. DETAILED DESCRIPTION

[0049] The application will be described in detail below with reference to the drawings and examples.

[0050] Example 1

[0051] The UAV small target detection method inspired by the hawk eye visual mechanism comprises the following steps:

[0052] A, construct a neural network, as shown in Figure 1 The neural network comprises a Backbone network, a Neck network module and a Head network module.

[0053] The DFB module and the SFB module are applied in the Backbone network module, and four Head detection heads are provided in the Head network module, wherein the four Head detection heads are detection heads provided by the YOLO network.

[0054] B, input the original image into the Backbone network module, and then perform processing on the original image through a Silence module, wherein the processing result of the Silence module is divided into two paths:

[0055] The first path sequentially passes through a first CBS module, a second CBS module, a first DFB module, a second DFB module, a first Adown module, a third DFB module, a second Adown module and a fourth DFB module; and the second path sequentially passes through a third CBS module, a fourth CBS module and a first SFB module.

[0056] The processing result of the second DFB module is processed through a first CBlinear module to obtain a first CBlinear module processing result; and the processing result of the third DFB module is processed through a second CBlinear module to obtain a second CBlinear module processing result.

[0057] The first SFB module processing result, the first CBlinear module processing result, the second CBlinear module processing result, the second DFB module processing result, the third DFB module processing result and the fourth DFB module processing result are input into the Neck network module;

[0058] The first DFB module, the second DFB module, the third DFB module and the fourth DFB module have the same structure, as shown in the figure, and the processing process is as follows: Figure 2

[0059] After the input result is processed by the GELAN module, it is divided into four paths; the first path is sequentially processed by 1*1 convolution and 3*3 convolution to obtain the first path result; the second path is sequentially processed by 1*1 convolution, 1*3 convolution, 3*1 convolution and 3*3 Atrous module to obtain the second path result; the third path is sequentially processed by 1*1 convolution, 3*1 convolution, 1*3 convolution and 3*3 Atrous module to obtain the third path result; and the fourth path is processed by 1*1 convolution to obtain the fourth path result.

[0060] The first path result, the second path result, the third path result and the fourth path result are spliced by a Concat function to obtain an output result.

[0061] The Neck network module performs multi-scale feature fusion on the input features to obtain two feature layers with different resolutions, which are input into the Head network module for detection to obtain a series of detection frames and category data, i.e., the final result.

[0062] The processing process in the Neck network module is as follows:

[0063] The fifth DFB module processing result and the first CBlinear module processing result are input into the first CBFuse module for processing, and the processing result of the first CBFuse module is sequentially processed by the second SFB module and the first NRT module to obtain the first NRT module processing result.

[0064] The second SFB module processing result is processed by the third Adown module to obtain a third Adown module processing result, and the third Adown module processing result, the first CBlinear module processing result and the second CBlinear module processing result are input into the second module for processing, and the processing result of the second CBFuse module is processed by the third SFB module and the second NRT module to obtain the second NRT module processing result.

[0065] ​The fourth DFB module processing result is processed by the SPPELAN module, and a first up-sampling result is obtained through up-sampling. The first up-sampling result is spliced with the third DFB module processing result through a Concat function, and the spliced result is processed by a first GELAN module to obtain a first GELAN module processing result.

[0066] The first GELAN module processing result is up-sampled, and is spliced with the second DFB module processing result through a Concat function. The spliced result is sequentially processed by a second GELAN module and a third NRT module to obtain a third NRT module processing result.

[0067] The second GELAN module processing result is processed by a third Adown module to obtain a third Adown module processing result. The first GELAN module processing result is spliced with the third Adown module processing result through a Concat function. The spliced result is sequentially processed by a third GELAN module and a fourth NRT module to obtain a fourth NRT module processing result.

[0068] The first NRT module processing result, the second NRT module processing result, the third NRT module processing result and the fourth NRT module processing result are input into a Head network module. The processing result of each NRT module is input into a Head detection head for processing.

[0069] The first SFB module, the second SFB module and the third SFB module have the same structure, as shown in Figure 3 The processing process is as follows:

[0070] After the input result is processed by a GELAN module, it is divided into four branches. The first branch is sequentially processed by a 1*1 convolution and a 7*7 convolution to obtain a first branch result. The second branch is sequentially processed by a 1*1 convolution, a 1*7 convolution, a 7*1 convolution and a 7*7 Atrous module to obtain a second branch result. The third branch is sequentially processed by a 1*1 convolution, a 7*1 convolution, a 1*7 convolution and a 7*7 Atrous module to obtain a third branch result. The fourth branch is processed by a 1*1 convolution to obtain a fourth branch result.

[0071] The first branch result, the second branch result, the third branch result and the fourth branch result are spliced through a Concat function to obtain an output result.

[0072] The first NRT module, the second NRT module, the third NRT module and the fourth NRT module have the same structure, as shown in Figure 4 The processing process is as follows:

[0073] The input result is divided into three paths, the first path and the second path are respectively subjected to 1*1 convolution processing, the two convolution processing results obtained are multiplied and fused, and then subjected to Softmax module processing to obtain a Softmax module processing result;

[0074] The third path is subjected to Rotundal cell module processing, the obtained result is multiplied and fused with the Softmax module processing result, then subjected to 1*1 convolution processing, and the obtained convolution result is added and fused with the input result to obtain an output result.

[0075] As shown in Figure 5 , the processing process in the Rotundal cell module is as follows:

[0076] The input result is sequentially subjected to 1*3 convolution, 3*1 convolution, 1*5 deep separable convolution, 5*1 deep separable convolution and 1*1 convolution processing, and then the obtained processing result is multiplied and fused with the input result to obtain an output result.

[0077] The hole coefficients of the 1*5 deep separable convolution and the 5*1 deep separable convolution are both 3.

[0078] Embodiment 2

[0079] For quantitative performance evaluation of the final UAV perspective small target image, the performance measurement standard widely used in the target detection field is adopted, and the specific evaluation is shown in formulas (1) and (2).

[0080]

[0081] Wherein, AP represents the integral of R (Recall) on P (precision), the confidence threshold is from 0 to 1, mAP represents the average AP value of all classes in the data set, and N is the number of classes. The higher the value of mAP, the stronger the detection performance of the model.

[0082] Table 1 summarizes the detection comparison experiment of embodiment 1 and existing YOLOv9 on the artificial intelligence UAV perspective micro target detection data set (VisDrone2019). The higher the points of the four indicators of precision, recall, mAP0.5 and mAP0.5:0.95, the stronger the detection performance of the model. The lower the parameter quantity (Param), the more lightweight the model. From the experimental effect, in comparison with the YOLOv9 framework, embodiment 1 of the application has made obvious progress in detection accuracy and parameter quantity.

[0083] Table 1 and YOLOv9 detection comparison

[0084]

[0085] Table 2 summarizes the comparison data between the model and existing small target detection models on the AI ​​drone perspective small target detection dataset (VisDrone2019). It can be seen that the neural network model of Embodiment 1 of the present invention has achieved significant results in detection performance, number of parameters, and computational cost (GFLOPS).

[0086] Table 2: Comparison of results with existing networks on the VisDrone dataset

[0087]

[0088] Example 3

[0089] To more intuitively understand the model's performance on the Visdrone dataset, this example demonstrates the visualization of detection results in various scenarios. The first column shows the original images with ground truth labels, the second column shows the detection images compared to the MHA-YOLO network, and the third column shows the detection images from Example 1. Figure 6 As shown, EVMNet demonstrates superior detection capabilities, particularly in recognizing small objects against complex backgrounds. The first row of images shows a parking lot, containing targets of different scales. It can be seen that the model from Example 1 within the yellow box accurately identified targets of different scales, almost identical to the Ground Truth annotations, while MHA-YOLO did not effectively identify objects of different scales. The second row shows a scene containing densely packed small targets. MHA-YOLO failed to detect a sufficient number of small targets within the yellow box, while EVMNet successfully identified different categories of small targets. The third row also demonstrates the accuracy of the model from Example 1.

Claims

1. A UAV small target detection method inspired by the hawk eye vision mechanism, characterized in that, The method comprises the following steps: A. constructing a neural network, wherein the neural network comprises a Backbone network module, a Neck network module and a Head network module; The DFB module and the SFB module are applied in the Backbone network module; The DFB module is applied to simulate the deep fovea structure of the hawk eye, and the processing process is as follows: After the input result is processed by the GELAN module, it is divided into four paths; the first path is sequentially processed by 1*1 convolution and 3*3 convolution to obtain a first path result; the second path is sequentially processed by 1*1 convolution, 1*3 convolution, 3*1 convolution and 3*3 Atrous module to obtain a second path result; the third path is sequentially processed by 1*1 convolution, 3*1 convolution, 1*3 convolution and 3*3 Atrous module to obtain a third path result; the fourth path is processed by 1*1 convolution to obtain a fourth path result; the first path result, the second path result, the third path result and the fourth path result are spliced by a Concat function to obtain an output result; The SFB module is applied to simulate the shallow fovea structure of the hawk eye, and the processing process is as follows: After the input result is processed by the GELAN module, it is divided into four paths; the first path is sequentially processed by 1*1 convolution and 3*3 convolution to obtain a first path result; the second path is sequentially processed by 1*1 convolution, 1*3 convolution, 3*1 convolution and 3*3 Atrous module to obtain a second path result; the third path is sequentially processed by 1*1 convolution, 3*1 convolution, 1*3 convolution and 3*3 Atrous module to obtain a third path result; the fourth path is processed by 1*1 convolution to obtain a fourth path result; the first path result, the second path result, the third path result and the fourth path result are spliced by a Concat function to obtain an output result; B. the original image is input into the Backbone network module, and the processing result of the Silence module is divided into two paths after being processed by the Silence module: The first path is sequentially processed by the first CBS module, the second CBS module, the first DFB module, the second DFB module, the first Adown module, the third DFB module, the second Adown module and the fourth DFB module; The second path is sequentially processed by the third CBS module, the fourth CBS module and the first SFB module; The processing result of the second DFB module is processed by the first CBlinear module to obtain a first CBlinear module processing result; The processing result of the third DFB module is processed by the second CBlinear module to obtain a second CBlinear module processing result; The first SFB module processing result, the first CBlinear module processing result, the second CBlinear module processing result, the second DFB module processing result, the third DFB module processing result and the fourth DFB module processing result are input into the Neck network module. ​ C. The Neck network module fuses the input features in multiple scales to obtain two feature layers with different resolutions, which are input into the Head network module to obtain a series of detection boxes and category data, i.e., the final result.

2. The UAV small target detection method inspired by the hawk eye vision mechanism according to claim 1, wherein: The first DFB module, the second DFB module, the third DFB module and the fourth DFB module have the same structure.

3. The UAV small target detection method inspired by the hawk eye vision mechanism according to claim 1, wherein: The processing procedure of the Neck network module is as follows: The first SFB module and the first CBlinear module are connected to the first CBFuse module, and the processing result of the first CBFuse module is sequentially processed by the second SFB module and the first NRT module to obtain the first NRT module processing result; The third Adown module processing result, the first CBlinear module processing result and the second CBlinear module processing result are input into the second CBFuse module, and the processing result of the second CBFuse module is processed by the third SFB module and the second NRT module to obtain the second NRT module processing result; The fourth DFB module processing result is processed by the SPPELAN module, and the first up-sampling result is obtained by up-sampling, and the first up-sampling result and the third DFB module processing result are spliced by the Concat function, and the spliced result is processed by the first GELAN module to obtain the first GELAN module processing result; The first GELAN module processing result is up-sampled and spliced with the second DFB module processing result by the Concat function, and the spliced result is sequentially processed by the second GELAN module and the third NRT module to obtain the third NRT module processing result; The second GELAN module processing result is processed by the fourth Adown module to obtain the fourth Adown module processing result; The first GELAN module processing result and the fourth Adown module processing result are spliced by the Concat function, and the spliced result is sequentially processed by the third GELAN module and the fourth NRT module to obtain the fourth NRT module processing result; The first NRT module processing result, the second NRT module processing result, the third NRT module processing result and the fourth NRT module processing result are input into the Head network module. Each NRT module is a bionic round nucleus cell module.

4. The UAV small target detection method inspired by the hawk eye vision mechanism according to claim 3, wherein: The first SFB module, the second SFB module and the third SFB module have the same structure.

5. The UAV small target detection method inspired by the hawk eye vision mechanism according to claim 3, wherein: The first NRT module, the second NRT module, the third NRT module and the fourth NRT module have the same structure, and the processing process is as follows: The input result is divided into three paths, the first path and the second path are respectively subjected to 1*1 convolution processing, the obtained two convolution processing results are multiplied and fused, then subjected to Softmax module processing to obtain a Softmax module processing result; The third path is subjected to Rotundal cell module processing, the obtained result is multiplied and fused with the Softmax module processing result, then subjected to 1*1 convolution processing, the obtained convolution result is added and fused with the input result to obtain an output result.

6. The UAV small target detection method inspired by the hawk eye vision mechanism according to claim 5, wherein: The processing process in the Rotundal cell module is as follows: The input result is sequentially subjected to 1*3 convolution, 3*1 convolution, 1*5 deep separable convolution, 5*1 deep separable convolution and 1*1 convolution processing, then the obtained processing result is multiplied and fused with the input result to obtain an output result.

7. The UAV small target detection method inspired by the hawk eye vision mechanism according to claim 6, wherein: The hole coefficients of the 1*5 deep separable convolution and the 5*1 deep separable convolution are both 3.

8. The UAV small target detection method inspired by the hawk eye vision mechanism according to claim 1, wherein: In the Head network module, four Head detection heads are provided, and the processing result of each NRT module is input into a Head detection head for processing.

9. The UAV small target detection method inspired by the hawk eye vision mechanism according to claim 8, wherein: The four Head detection heads are all detection heads provided by the YOLO network.

Images