Attention mechanism-based unmanned aerial vehicle aerial photography small target detection method

Abstract

The invention provides an unmanned aerial vehicle aerial photography small target detection method based on an attention mechanism. Comprising the following steps: collecting an unmanned aerial vehicle small target detection data set on the internet, and preprocessing the data set; according to the method, YOLO11 is used as a basic network, a novel backbone network is designed for the problems that small target feature extraction is weak and semantic and spatial feature fusion is insufficient, on the basis of a C3K2 module, a Bi-Dimensional ScConv attention mechanism module (Bi-Dimensional ScConv attention mechanism module (Bi-Dimensional ScConv attention mechanism module) is embedded behind a C4 layer and a C5 layer of Backbone, and the module constructs horizontal + vertical spatial attention through grouping normalization and a gating threshold mechanism, and accurately focuses on a small target area; and through combination of channel splitting, differential convolution and dynamic weight distribution, information interaction between channels is optimized, and the small target feature discrimination capability is enhanced. In cooperation with multi-scale feature fusion and a detection head, the whole network is trained and optimized on a preprocessed unmanned aerial vehicle small target data set, and the small target detection precision is significantly improved.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and computer vision, specifically to a method for detecting small targets in drone aerial photography based on an attention mechanism. Background Technology

[0002] In images captured by drones, target objects (such as vehicles and pedestrians) are typically very small, especially during high-altitude flights, making feature extraction difficult. Because the targets are so small, their texture, shape, and other details may be lost or become indistinct during downsampling, placing higher demands on visual feature-based target recognition. Furthermore, the small number of pixels occupied by small targets reduces the number of local feature points available to describe their characteristics, thus affecting the model's ability to learn sufficient discriminative features. This necessitates that detection models possess high perceptual capabilities, enabling them to effectively locate and identify small targets. Cameras capture images with a wide field of view and a variety of objects, resulting in complex backgrounds. In some cases, background objects may be highly similar to the actual target being detected in terms of color, texture, or shape. For example, in aerial photography of road scenes, dark manhole covers on the road surface may be similar in color and blocky in shape to small vehicles (such as motorcycles); in aerial photography of outdoor crowds, low shrubs on grass may have similar outlines to pedestrians from a high-altitude perspective (both are small, upright targets); in aerial photography of construction sites, piles of gravel on the ground and small construction equipment (such as handheld drills) may have rough textures and irregular shapes, easily causing visual confusion. This makes it difficult for convolutional neural networks to distinguish which are the actual targets to be identified and which are merely part of the background, easily leading the convolutional neural network to make incorrect judgments and attribute background information to target information.

[0003] Small targets in drone aerial images tend to cluster and occlude, posing additional challenges to object detection using convolutional neural networks (CNNs). CNNs typically extract features through multiple convolutional operations, accompanied by downsampling. When faced with densely packed small targets, whose pixel areas in the original image are already small, downsampling can cause them to become very close or even overlapping on the feature map, making it difficult for the model to distinguish them. When occlusion occurs between small targets, only a small portion of some targets may be visible, while the rest is obscured by other targets or background elements. This reduces the number of key feature points available to describe the target, further weakening the model's ability to identify it. Occlusion makes effective features for each target scarcer, forcing the model to rely on limited information to make classification and localization decisions, increasing the probability of errors. Especially for targets where only a small portion is visible, the model may misclassify or miss them entirely due to a lack of sufficient information.

[0004] To address the challenge of effectively extracting small target features in drone aerial photography, this patent proposes a BiDimSpatialAttention_ScConv attention mechanism module. Utilizing grouping normalization and gating threshold mechanisms, it constructs a bidirectional spatial attention mechanism (horizontal + vertical) to accurately distinguish between important regions (relevant to small targets) and secondary regions (background redundancy) in the feature map, enhancing the representation of subtle spatial details of small targets. Furthermore, through channel splitting and differentiated convolutional transformation, coupled with dynamic weight allocation, it achieves channel information fusion, optimizing effective interactions between channels and enhancing the channels' ability to discriminate small target features. Ultimately, this achieves accurate extraction of small target features, significantly improving the detection performance of small targets in drone aerial photography scenarios. Summary of the Invention

[0005] The purpose of this invention is to design a network for detecting small targets from the perspective of drone aerial photography, which can detect very small target areas from the perspective of drone aerial photography and has high detection accuracy.

[0006] This invention provides a small target detection network for UAV aerial photography based on an attention mechanism. It mainly includes a backbone network with a BiDimSpatialAttention_ScConv attention mechanism module, which is used to extract features from the input image and enhance the ability to express spatial details and optimize channel interaction through the BiDimSpatialAttention_ScConv module to better train subsequent networks; a multi-scale feature fusion structure to enhance feature expression capabilities; and a target detection module to output the final detection results.

[0007] This invention mainly consists of the following steps:

[0008] 1. Collect drone aerial images and videos online, obtain a large number of images using drone aerial photography, and annotate them to construct a drone aerial small target detection dataset;

[0009] 2. Construct an attention-based network for detecting small targets in drone aerial photography;

[0010] 3. Train the completed attention-based UAV aerial small target detection network on the corresponding training set for several rounds, adjust the parameters to obtain the optimal model, and then deploy the model.

[0011] In the steps described above, the completed attention-based UAV aerial small target detection network mainly includes a backbone network, a BiDimSpatialAttention_ScConv module, a multi-scale feature fusion structure, and a target detection module.

[0012] The BiDimSpatialAttention_ScConv module consists of two sub-modules. The first sub-module is the BiDimSpatialAttentionModulator (BDSAM), which enhances the learning of small target feature information in UAV aerial photography from a spatial dimension. The second sub-module is the ChannelReconstruction Unit (CRU), which enhances the learning of texture and detail information of small target features in UAV aerial photography from a channel dimension.

[0013] The construction of the BiDimSpatialAttention_ScConv module includes the following steps:

[0014] Step 1: Input features Max pooling is performed, and the maximum value is taken along the width (W) to model the vertical attention X_h, and the maximum value is taken along the height (H) to model the horizontal attention X_w.

[0015] Step 2: Perform group normalization on the vertical attention X_h obtained in Step 1. Standardize the input feature X_h by subtracting the mean μ and dividing by the standard deviation σ. The calculation formula is as follows:

[0016]

[0017] Where μ and σ are the mean and standard deviation of X_h, ε is a small positive constant added to avoid numerical instability caused by excessively small standard deviation σ, thus ensuring the stability of division operations, γ and β are trainable affine transformation parameters, corresponding to the scaling factor and offset factor in group normalization, respectively.

[0018] Step 3: Normalize the trainable scaling factor γ from the normalization in Step 2 again to obtain W. γ_h After being reweighted with the normalized features, the weights are mapped to the (0,1) range using the Sigmoid function and then gating with a threshold to obtain the information-rich weights W. h_1 And less information weight W h_2 For the information-rich branch, the weight W h_1 This is achieved by setting weights above a threshold to 1, while retaining the original values ​​for the rest. The weights W for branches with less information are then adjusted accordingly. h_2 This is achieved by setting the weights above a threshold to 0, while retaining the original values ​​for the rest. The calculation formula is as follows:

[0019]

[0020] W h_1 W h_2 =Gate(Sigmoid(Wγ_h X out_h ))

[0021] Step 4: Then, combine the vertical attention X_h obtained in Step 1 with the information enrichment weight W obtained in Step 3. h_1 And less information weight W h_2 Weighted summaries yield attention values ​​rich in information in the vertical direction. and less information attention The calculation formula is:

[0022]

[0023] in It is element-wise multiplication;

[0024] Step 5: Apply the attention obtained in Step 4, which contains rich information in the vertical direction. Divide the channel into two groups, and then apply the attention obtained in step 4, which contains less information in the vertical direction. The attention is divided into two groups along the channel. The attention values ​​of the two groups containing rich information and the two groups containing less information are cross-added and then concatenated along the channel dimension. The final vertical attention X is obtained by passing the sigmoid function. Finall_h The calculation formula is:

[0025]

[0026] Split represents grouping along the channel. It is a summation of elements;

[0027] Step 6: Perform group normalization on the horizontal attention X_w obtained in Step 1. Standardize the input feature X_w by subtracting the mean μ and dividing by the standard deviation σ. The calculation formula is:

[0028]

[0029] Step 7: Normalize the trainable scaling factor γ from the normalization in Step 6 again to obtain W. γ_w After being reweighted with the normalized features, the weights are mapped to the (0,1) range using the Sigmoid function and then gating with a threshold to obtain the information-rich weights W. w_1 And less information weight W w_2 For the information-rich branch, the weight W w_1 This is achieved by setting weights above a threshold to 1, while retaining the original values ​​for the rest. The weights W for branches with less information are then adjusted accordingly. w_2 This is achieved by setting the weights above a threshold to 0, while retaining the original values ​​for the rest. The calculation formula is as follows:

[0030]

[0031] W w_1 W w_2 =Gate(Sigmoid(W γ_w X out_w ))

[0032] Step 8: Then, combine the horizontal attention X_w obtained in Step 1 with the information enrichment weight W obtained in Step 7. h_1 And less information weight W h_2 Weighted summaries yield attention values ​​rich in information in the vertical direction. and less information attention The calculation formula is:

[0033]

[0034] Step 9: Apply the attention obtained in Step 8, which contains rich information in the vertical direction. Divide the channel into two groups, and then apply the attention obtained in step 8, which contains less information in the vertical direction. The attention is divided into two groups along the channel. The attention values ​​of the two groups containing rich information and the two groups containing less information are cross-added and then concatenated along the channel dimension. The final vertical attention X is obtained by passing the sigmoid function. Finall_w The calculation formula is:

[0035]

[0036] Step 10: Interact the input features from Step 1 sequentially with the final vertical attention X from Step 5. Finall_h And the final horizontal attention X in step 9 Finall_w After weighting, the bidirectional spatial attention weighted modulation features are obtained, and the calculation formula is as follows:

[0037]

[0038] Step 11: Divide the bidirectional spatial attention-weighted modulated features obtained in Step 10 into two groups along the channels, and then compress the number of channels by 1×1 convolution to obtain two groups of features, namely X. up and X low ;

[0039] Step 12: The first set of features X obtained in Step 11... up The output feature Y1 is obtained by combining grouped convolution and point convolution respectively.

[0040] Step 13: Use the second set of features X obtained in Step 11 lowAfter point convolution, then with X low Feature Y2 is obtained by concatting along the channel;

[0041] Step 14: Concat the output feature Y1 obtained in Step 12 and the feature Y2 obtained in Step 13 again. Then perform adaptive average pooling on the concatenated feature (compress the spatial dimension to 1×1 and focus on global channel correlation). Then normalize the pooling result through the Softmax function to generate dynamic weights for each channel (the weight size represents the importance of the corresponding channel). Finally, weight the dynamic weights with the concatenated complete feature to obtain the channel refined feature.

[0042] Step 15: Divide the refined channel features obtained in Step 14 into two groups, and then add them together to get the final output. Compress the number of channels back to the original dimension to reduce the computational cost without losing key information. At the same time, add the channels of the first group of "core information of small targets" and the channels of the second group of "detail information of small targets" to further integrate the core semantics and detailed information, making the features more "condensed".

[0043] By adopting the above technical solution, the present invention has the following advantages:

[0044] 1. This invention introduces the BDSAM module into the BiDimSpatialAttention_ScConv module. By calculating attention for the height and width dimensions respectively, it can accurately capture the positional features of small targets in two-dimensional space, avoid their spatial information being covered by the large background, be more sensitive to the "slender" and "point-like" features of small targets, retain the subtle spatial cues of small targets, suppress invalid features in the background area, allow the model to focus on the local area where the small target is located, reduce the interference of complex backgrounds in aerial images on small targets, and enable the network to better learn the feature information of small targets captured by drones.

[0045] 2. This invention introduces a CRU module into the BiDimSpatialAttentionScConv module, which solves the problem of channel representation of small targets in UAV aerial photography. By channel splitting and recombination and attention weighting, the semantic features of small targets are efficiently aggregated, thereby improving detection robustness. Attached Figure Description

[0046] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration:

[0047] Figure 1 This is a schematic diagram of the process of the UAV aerial photography small target detection method based on the attention mechanism of the present invention.

[0048] Figure 2This is a schematic diagram of the bidirectional spatial attention modulator (BDSAM) of the present invention.

[0049] Figure 3 This is a schematic diagram of the BiDimSpatialAttention_ScConv attention mechanism module of the present invention.

[0050] Figure 4 This is a schematic diagram of the small target detection network for UAV aerial photography based on the attention mechanism of the present invention. Detailed Implementation Plan

[0051] The present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the invention. The present invention proposes a method for detecting small targets in drone aerial photography based on an attention mechanism, mainly targeting the detection of small targets such as pedestrians and tricycles at a distance in drone aerial photography scenarios. The specific implementation includes the following:

[0052] Step 1: Collect drone aerial images and videos online, obtain a large number of images using drone aerial photography, and annotate them to build a drone aerial small target detection dataset;

[0053] Step 2: YOLO11n is selected as the baseline network. Its core structure includes: a backbone network (stacked C3K2 modules, responsible for feature extraction), a multi-scale feature fusion structure (fusion of deep and shallow features), and an object detection module (outputting detection boxes and class confidence). After the C4 and C5 layers of the backbone network, which pass through a downsampling convolutional layer and a C3K2 feature extraction module, BiDimSpatialAttention_ScConv modules are added respectively (to specifically enhance small object features). The module parameters are strictly adapted to the C3K2 output channels, batch_size = 32 (single GPU: NVIDIA RTX 3090), and a total of 300 rounds.

[0054] Step 3: In the appendix Figure 3 In this invention, the BiDimSpatialAttention_ScConv module is constructed. The BiDimSpatialAttention_ScConv module is the core innovative module of this invention, consisting of two sub-modules, the BDSAM module and the CRU module, connected in series. Combined with the appendix... Figure 2 The specific steps for building the BDSAM module are as follows:

[0055] Step 3-1: First, perform max pooling on the input features, taking the maximum value along the width (W) to model vertical attention, and taking the maximum value along the height (H) to model horizontal attention;

[0056] Step 3-2: Perform group normalization on the vertical attention from Step 3-1, then normalize the learnable scaling parameter γ of the group normalization layer by summation, and multiply it with the group normalized features. Map the result to the (0,1) range using the Sigmoid function, and perform gating by thresholding. Set the weights above the threshold (set to 0.5) to 1, and retain the original values ​​to obtain the first set of feature weights with rich information. Set the weights above the threshold (set to 0.5) to 0, and retain the original values ​​to obtain the second set of feature weights with less information. Then, weight the vertical attention obtained in Step 3-1 with the first set of feature weights and the second set of feature weights respectively to obtain vertical attention with rich information and attention with less information.

[0057] Step 3-3: Divide the attention with rich vertical information obtained in Step 3-2 into two groups along the channel, and then divide the attention with less vertical information obtained in Step 3-2 into two groups along the channel. Cross-add the two groups of attention with rich information and the two groups of attention with less information, and then perform Concat along the channel dimension. The final vertical attention is obtained through the Sigmoid function.

[0058] Step 3-4: Perform group normalization on the horizontal attention obtained in Step 3-1, then normalize the learnable scaling parameter γ in the group normalization by summation, and multiply it with the group normalized features. Map the result to the (0,1) range using the Sigmoid function, and perform gating by thresholding. Set the weights above the threshold (set to 0.5) to 1, and retain the original values ​​to obtain the first set of feature weights with rich information. Set the weights above the threshold (set to 0.5) to 0, and retain the original values ​​to obtain the second set of feature weights with less information. Then, weight the horizontal attention obtained in Step 3-1 with the first set of feature weights and the second set of feature weights respectively to obtain the horizontal attention with rich information and the attention with less information respectively.

[0059] Steps 3-5: Divide the attention with rich information in the horizontal direction obtained in Step 3-4 into two groups along the channel, and then divide the attention with less information in the horizontal direction obtained in Step 3-4 into two groups along the channel. Cross-add the two groups of attention with rich information and the two groups of attention with less information, and then perform Concat along the channel dimension. The final horizontal attention is obtained by using the Sigmoid function.

[0060] Step 3-6: The input features from Step 3-1 are sequentially weighted with the final vertical attention from Step 3-3 and the final horizontal attention from Step 3-5 to obtain bidirectional spatial attention weighted modulation features;

[0061] Step 4: Combine with the appendix Figure 3 To optimize channel interaction, a CRU module is added after the BDSAM module. The BiDimSpatialAttention_ScConv module is constructed using the following steps:

[0062] Step 4-1: Divide the bidirectional spatial attention-weighted modulated features obtained in Step 3-6 into two groups along the channels, and then compress the number of channels by 1×1 convolution to obtain two groups of features, namely X. up and X low ;

[0063] Step 4-2: The first set of features X obtained in Step 4-1... up The output feature Y1 is obtained by combining grouped convolution and point convolution respectively.

[0064] Step 4-3: The second set of features X obtained in Step 4-1... low After point convolution, then with X low Along the channel Concat to the supplementary feature Y2;

[0065] Step 4-4: Concat the output feature Y1 obtained in Step 4-2 and the supplementary feature Y2 obtained in Step 4-3 again, and then use adaptive average pooling and Softmax to obtain the corresponding weights. Then, weight the concatenated features to obtain the channel-refined features.

[0066] Steps 4-5: Divide the refined channel features obtained in Step 4-4 into two groups, and add them together to obtain the final output to improve expressive power;

[0067] Step 5: Train the completed attention-based UAV aerial small target detection network on the corresponding training set for several rounds, adjust the parameters to obtain the optimal model, and then deploy the model.

Claims

1.A method for detecting small targets based on aerial photography of a UAV, characterized in that, YOLO11 network model is used as the basic network, and the network is optimized according to the characteristics of the unmanned aerial vehicle aerial image, including the following steps: Step 1: Collecting unmanned aerial vehicle aerial image and video on the network, using unmanned aerial vehicle aerial image to obtain a large number of images, and labeling, constructing unmanned aerial vehicle aerial small target detection data set; Step 2: Taking YOLO11 as the basic network, the unmanned aerial vehicle aerial small target detection network is mainly embedded in the BiDimSpatialAttention_ScConv module behind the two C3K2 modules in the deepest layer of YOLO11 Backbone to enhance the attention of the target; Step 3: In the PyTorch framework, the designed network is trained on the constructed unmanned aerial vehicle aerial small target detection data set to obtain the optimal network model; Step 4: Deploy the trained model for unmanned aerial vehicle aerial target detection. 2.The method of claim 1, wherein, The BiDimSpatialAttention_ScConv module is constructed, including the following steps: Step 1: The input feature is firstly subjected to maximum pooling, respectively taking the maximum value along the width (W) to model the vertical direction attention, and taking the maximum value along the height (H) to model the horizontal direction attention; Step 2: The vertical direction attention obtained in step 1 is subjected to group normalization, and then the learnable scaling parameter γ of the group normalization layer is normalized according to the sum, and multiplied by the group normalized feature, the result is mapped to the range (0, 1) through the Sigmoid function, and the threshold is gated, the weight higher than the threshold is set to 1, and the rest is reserved to obtain the first group of weights, the weight higher than the threshold is set to 0, and the rest is reserved to obtain the second group of weights, and then the vertical direction attention obtained in step 1 is weighted with the first group of feature weights and the second group of feature weights respectively to obtain the attention containing rich information and less information in the vertical direction respectively; Step 3: The attention containing rich information in the vertical direction obtained in step 2 is evenly divided into two groups along the channel, and the attention containing less information in the vertical direction obtained in step 2 is also evenly divided into two groups along the channel, the two groups of attention containing rich information are cross added with the two groups of attention containing less information, and then concatenated along the channel dimension, and the final vertical direction attention is obtained through the Sigmoid function; Step 4: The horizontal direction attention obtained in step 1 is subjected to group normalization, and then the learnable scaling parameter γ in the group normalization is normalized according to the sum, and multiplied by the group normalized feature, the result is mapped to the range (0, 1) through the Sigmoid function, and the threshold is gated, the weight higher than the threshold is set to 1, and the rest is reserved to obtain the first group of weights, the weight higher than the threshold is set to 0, and the rest is reserved to obtain the second group of weights, and then the horizontal direction attention obtained in step 1 is weighted with the first group of feature weights and the second group of feature weights respectively to obtain the attention containing rich information and less information in the horizontal direction respectively; Step 5: The horizontal direction attention containing rich information obtained in step 4 is divided into two groups along the channel, and the horizontal direction attention containing less information obtained in step 4 is also divided into two groups along the channel. The two groups of attention containing rich information are cross-added with the two groups of attention containing less information, and then concatenated along the channel dimension. The final horizontal direction attention is obtained by passing through a Sigmoid function. Step 6: The input features in step 1 are sequentially weighted with the final vertical direction attention in step 3 and the final horizontal direction attention in step 5 to obtain the features weighted and modulated by bidirectional spatial attention. Step 7: The features weighted and modulated by bidirectional spatial attention in step 6 are divided into two groups along the channel, and then compressed by 1x1 convolution respectively to obtain two groups of features, i.e., the upper part and the lower part. Step 8: The first group of upper part features obtained in step 7 are respectively captured by group convolution to capture local spatial features, and by point convolution to capture the correlation between global channels, and then added to obtain output features. Step 9: The second group of lower part features obtained in step 7 are captured by point convolution, and then concatenated with the second group of lower part features obtained in step 7 along the channel to obtain output features. Step 10: The features obtained in step 8 and the features obtained in step 9 are spliced along the channel dimension. Then, adaptive average pooling is performed on the spliced features. The pooled results are normalized by a Softmax function to generate dynamic weights for each channel. Finally, the dynamic weights are weighted with the complete features after splicing to obtain channel refined features. Step 11: The refined features in step 10 are divided into two groups along the channel, and then added correspondingly to obtain the final output.

Images