An unmanned aerial vehicle aerial image small target detection method based on improved YOLOv11

Abstract

The application discloses a kind of based on improved YOLOv11 unmanned aerial vehicle aerial image small target detection method, to solve the problem of insufficient detection accuracy caused by target scale small, background interference strong, serious occlusion in unmanned aerial vehicle aerial image.The present application takes YOLOv11n as the basic framework, carries out three core improvements: in the first, second, fourth, sixth layer of main network embedding multi-scale attention module MSAM replaces original C3k2 module, with 1×k and k×1 strip convolution instead of matrix multiplication, combined with grouping double normalization module GDN enhances small target high-frequency detail feature capture, the calculation complexity is reduced to O (n) ;With four-stage hybrid feature pyramid network VMFPN replaces original neck FPN, the first two stages MSGDC locally mixed small target details are retained, the last two stages MSMHA globally fused integration semantic information, channel number doubles in turn (64→128→256→512) ;Convolution attention fusion module CAFM is embedded in the third fusion node of neck VMFPN, CNN local branch and Transformer global branch are fused in parallel, and finally output Fout=Fatt+Fconv.

Description

Technical Field

[0001] This invention relates to a method for small target detection in UAV aerial images based on an improved YOLOv11, belonging to the fields of computer vision and deep learning, and particularly to the improvement of lightweight target detection network structure and multi-module collaborative optimization technology. Background Technology

[0002] With the rapid development of drone technology, its applications have expanded from military reconnaissance to multiple key areas such as agricultural monitoring, disaster response, traffic control, and facility inspection. However, the unique high-altitude viewpoint, variable flight altitude and shooting angle, and complex lighting conditions of drones mean that targets in the images they capture are often small in scale (usually less than 0.12% of the image area), have blurred features, are severely occluded, and have cluttered backgrounds. This results in existing detection algorithms generally suffering from insufficient detection accuracy, high false negative and false positive rates.

[0003] Currently, deep learning-based object detection algorithms are mainly divided into two schools of thought: two-stage detection algorithms, represented by Faster R-CNN, offer high accuracy but have high computational complexity; while single-stage detection algorithms, represented by the YOLO series, offer good real-time performance but have relatively low accuracy when dealing with small targets. In recent years, algorithms such as RT-DETR and Gold-YOLO have achieved breakthroughs in speed and accuracy, but their high computational complexity limits their application on UAV edge devices. Methods specifically designed for UAV scenarios, such as ClusDet and UFPMP-Det, while optimized for dense small targets, still have shortcomings in balancing parameter count and accuracy.

[0004] The distinguishing features of this invention from the closest prior art are as follows: (1) Differences from MobileViT (Mehta et al., CVPR 2022): MobileViT discloses a technical solution that embeds Transformer blocks in a lightweight backbone. Its structure is: input feature map → flattened into sequence → multi-head self-attention (MHSA) → feedforward network (FFN) → residual connection → output. In this solution, attention calculation is implemented through matrix multiplication of query, key, and value, with a complexity of O(n²). Furthermore, the Flatten operation destroys the spatial locality of small targets, and only a single LayerNorm normalization strategy is used. The distinguishing features of the MSAM module in this invention are: ① It replaces the matrix multiplication of MHSA with 1×k and k×1 strip convolutions, where Query is the input feature map x, and Key and Value are learnable convolution kernels, completely avoiding flattening operations and preserving spatial locality; ② It introduces GDN double normalization, applying Softmax normalization in the spatial dimension and grouped L2 norm normalization in the channel dimension (number of groups G=C / 4), while MobileViT only uses LayerNorm single normalization; ③ The computational complexity is reduced to O(n), and the memory usage is significantly reduced.

[0005] (2) Differences from RepViT (Li et al., ICCV 2023): RepViT discloses a lightweight solution to reduce inference latency through structural reparameterization, but its attention mechanism is still based on global matrix operations. It does not design a special strip convolutional attention structure for high-frequency detail features of small targets in UAV aerial photography scenarios, nor does it introduce a grouped double normalization mechanism to preserve small target features at multiple granularities.

[0006] (3) Differences from Gold-YOLO MAFPN (Wang et al., NeurIPS 2023): Gold-YOLO discloses a Gather-and-Distribute neck fusion structure, in which the Gather operation guides the selection of low-level features through high-level semantic features. All stages use a uniform convolution operation and rely on pooling downsampling. The distinguishing features of the VMFPN in this invention are: ① In the four-stage structure, the first two stages (P2 / P3) use MSGDC local mixing to preserve small target details, and the last two stages (P4 / P5) use MSMHA global fusion to integrate semantic information, achieving stage-differentiated design, while the operations of each stage of MAFPN are homogeneous; ② Downsampling uses 2×2 stride 2 convolution to replace pooling operation, avoiding dilution of small target features during downsampling; ③ The number of channels doubles in each stage from 64 to 128 to 256 to 512, which is different from the non-fixed channel strategy of Gold-YOLO.

[0007] (4) Differences from BiFPN (Tan et al., CVPR 2020): BiFPN enhances multi-scale representation through weighted feature fusion, but lacks explicit token-level global modeling capabilities and cannot specifically enhance the weak responses of small targets in features at various levels. The CAFM module of this invention adds the outputs of parallel local branches (CNN) and global branches (Transformer attention with learnable scaling parameter α) to specifically enhance the weak features of small targets at the third fusion node in the neck region, which is fundamentally different from the weighted fusion mechanism of BiFPN.

[0008] Existing UAV small target detection improvement schemes mostly employ single-dimensional optimization strategies, which struggle to simultaneously overcome the core contradictions of small target features being easily lost in deep networks and insufficient perception of weak neck features. Simply enhancing the backbone feature extraction capability cannot prevent feature dilution in the neck FPN; simply improving multi-scale fusion in the neck cannot effectively enhance the feature representation of small targets. Therefore, a detection method oriented towards full-link collaborative optimization of "feature extraction—cross-layer fusion—weak feature enhancement" is urgently needed. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing a small target detection method for UAV aerial images based on an improved YOLOv11. By systematically and collaboratively improving the backbone network and the neck network, the detection accuracy and model deployment efficiency are comprehensively enhanced.

[0010] The technical solution provided by this invention is based on the YOLOv11n framework, retaining its core architecture of backbone network, neck feature fusion network and detection head, and implementing three core improvements: multi-scale attention modules (MSAM) are embedded in layers 1, 2, 4 and 6 of the backbone network to replace the original C3k2 module; the original neck FPN is replaced with a four-stage hybrid feature pyramid network (Valid Multi-Scale FPN, VMFPN); and a convolution and attention fusion module (CAFM) is embedded at the 3rd fusion node of the neck VMFPN, achieving high-precision detection of small targets in UAV aerial photography through end-to-end joint training.

[0011] Furthermore, the Multi-Scale Attention Module (MSAM) addresses the shortcomings of existing technologies, such as O(n²) complexity and the destruction of spatial locality by Flatten operations, by replacing matrix multiplication with efficient 1×k and k×1 strip convolutions (where k is an integer greater than 1). MSAM is internally composed of two residual sub-modules connected in series: a ConvAttention sub-layer and an FFN sub-layer. The ConvAttention sub-layer uses the input feature map x as the Query and performs 1×k and k×1 strip convolutions with the learnable convolutional kernel Key. The result is then subjected to spatial Softmax normalization and channel grouping L2 norm normalization by the GDN module (number of groups G=C / 4, each group is independently normalized and then concatenated). After convolution with the Value convolutional kernel, the intermediate feature f=Norm(x+ConvAttention(x)) is obtained. The FFN sub-layer consists of two cascaded 3×3 convolutional layers with BN and ReLU activation, producing the final output y=Norm(f+FFN(f)). The computational complexity is reduced from O(n²) to O(n), significantly reducing computational overhead while preserving high-frequency details of small targets with high fidelity.

[0012] Furthermore, the VMFPN structure addresses the drawback of existing neck network homogenization operations leading to the dilution of small target features during deep propagation by designing a differentiated four-level feature pyramid: Using the P2 / P3 / P4 / P5 feature maps output from the backbone network as input, after initial 4×4 convolution (stride 4) projection, the first two stages introduce the MSGDC module (three-way 3×3 grouped dilated convolutions with dilation rates of 1, 2, and 3, with 8 groups) to process local features, capturing multi-scale receptive fields and preserving small target details; the latter two stages introduce the MSMHA module (8 attention heads) to complete token-by-token feature fusion, integrating global semantic information; the downsampling layers in each stage use convolutions with a stride of 2 and a kernel size of 2×2 (instead of pooling) to double the number of channels (64→128→256→512), preventing the dilution of small target features during downsampling.

[0013] Furthermore, the Convolutional Attention Fusion Module (CAFM) addresses the lack of specific enhancement mechanisms for weak features of small targets in existing fusion methods by embedding it into the third fusion node of VMFPN (between Stage 3 and Stage 4). The local and global branches of CAFM share the same input feature map but undergo independent convolution processing: the local branch reduces the dimensionality to half the number of channels through a 1×1 convolution, then performs channel shuffling (grouping into 4) and outputs Fconv through a 3×3 depthwise separable convolution; the global branch generates Q, K, and V tensors through a 1×1 convolution and a 3×3 depthwise separable convolution, and calculates the output Fatt using attention with a learnable scaling parameter α (initialized to √C, α>0 and trainable); the final output Fout=Fatt+Fconv balances the local inductive bias of CNNs with the long-range modeling capabilities of Transformers.

[0014] Furthermore, there is a structural coupling relationship between the three improvements: the MSAM module outputs a feature map with the same number of channels as the input (residual conservation), providing VMFPN with multi-scale feature inputs with stable channel dimensions; the input of the third fusion node of VMFPN is the P3 stage feature map and the P4 stage feature map processed by MSMHA, and the local and global branches of CAFM share the same input feature map of this node; the three modules form a structural coupling through the conservation of the number of feature map channels and the hierarchical progression relationship, realizing a complete technical link of "extraction-transfer-enhancement".

[0015] Compared with the prior art, the present invention has the following beneficial technical effects: 1. The MSAM module designed in this invention replaces matrix multiplication with strip convolution, reducing the computational complexity from O(n²) to O(n). While effectively reducing computational load, it retains high-frequency and detailed information of small targets with high fidelity. When MSAM is introduced alone, mAP50 is improved by 3.2 percentage points (35.2% vs 32.0%), and the efficiency improvement per unit of parameters (ΔmAP50 / ΔParams) is 1.07, which is significantly better than mainstream attention mechanisms (MobileViT: O(n²) complexity, RepViT: no specific small target feature design).

[0016] 2. The VMFPN structure designed in this invention achieves superior cross-layer feature preservation compared to MAFPN with the same number of parameters through stage differentiation (MSGDC for local detail preservation in the first two stages + MSMHA for global semantic fusion in the last two stages) and 2×2 convolutional downsampling instead of pooling. When VMFPN is introduced alone, mAP50 is improved by 2.8 percentage points (34.8% vs 32.0%).

[0017] 3. The CAFM module designed in this invention, through a parallel structure of local branches (CNN) and global branches (Transformer attention with learnable scaling parameter α), specifically enhances the weak feature representation of small targets. When CAFM is introduced alone, mAP50 is improved by 1.5 percentage points (33.5% vs 32.0%).

[0018] 4. When the three improvements work together, the number of parameters increases by only 34.6% (from 2.6M to 3.5M), while mAP50 improves by 8.1 percentage points (40.1% vs 32.0%). The efficiency improvement per unit of parameter (ΔmAP50 / ΔParams) reaches 2.34, far exceeding the efficiency of adding any one module alone (MSAM: 1.07, VMFPN: 0.90, CAFM: 0.58) and the conventional range of existing technologies (0.4~1.1). With approximately 37% of the number of parameters in YOLOv11s (3.5M vs 9.4M), mAP50 reaches 99.3% of YOLOv11s (40.1% vs 40.4%), achieving a better balance between accuracy and efficiency. On the HIT-UAV infrared thermal imaging dataset, mAP50 is improved by 6.5 percentage points compared to the baseline, validating cross-modal generalization ability. Attached Figure Description

[0019] Figure 1 The overall structure diagram of the YOLOv11n network before improvement. Figure 2 This is a diagram of the overall structure of the improved YOLOv11n network proposed in this invention.

[0020] Figure 3 This is a detailed structural diagram of the MSAM multi-scale attention module.

[0021] Figure 4 This is a diagram of the VMFPN neck network structure.

[0022] Figure 5 This is a structural diagram of the CAFM convolutional attention fusion module. Detailed Implementation

[0023] The technical solution will now be clearly and completely described in conjunction with the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. The training hyperparameters, data augmentation strategies, and experimental environment of the baseline model YOLOv11n and the improved model of the present invention are completely identical, differing only in network structure, ensuring that the accuracy improvement comes entirely from the structural improvements of the present invention.

[0024] 1. Dataset and Experimental Environment Dataset: The experiments used the VisDrone2019 dataset, constructed by the Tianjin University team. Images were collected from diverse scenes in 14 cities across China under varying environmental and meteorological conditions, comprising 8599 images (6471 for training, 548 for validation, and 1610 for testing). Labeled categories include ten types such as pedestrians, bicycles, cars, vans, and trucks, with small targets predominating (over 85%). Generalization performance was also validated on the HIT-UAV infrared thermal imaging dataset.

[0025] Data preprocessing and enhancement: Images are denoised and contrast enhanced. The diversity of training samples is increased through random cropping (within ±10° rotation), scaling (0.5×~1.5×), color dithering (HSV perturbation), Mosaic data augmentation, and random horizontal flipping. All sample images are normalized to 640×640 resolution and divided into training, validation, and test sets in a 7:2:1 ratio.

[0026] Experimental Environment: YOLOv11n was chosen as the base model. The experimental environment consisted of an NVIDIA GeForce RTX 4090 (24GB VRAM), PyTorch 2.0.0 + cu118, Python 3.10.16, and CUDA 12.9. Training hyperparameters: Epochs=200, Batch-size=8, SGD optimizer, initial learning rate 0.01, Weight-Decay=0.0005, and cosine annealing as the learning rate scheduling strategy. All training configurations of the baseline model and the improved model were identical to those described above to ensure experimental fairness.

[0027] 2. Specific implementation of the MSAM module The MSAM module replaces the C3k2 module in layers 1, 2, 4, and 6 of the backbone network. In the ConvAttention sub-layer, the input feature map x is used as the Query and is convolved with the external learnable convolutional kernel Key using 1×k and k×1 strip convolutions (k is an integer greater than 1, k=3 in layers 1 and 2, and k=5 in layers 4 and 6). The convolution result is fed into the GDN module: Let the input feature map X∈R^(C×H×W), the number of groups G=C / 4, and the number of channels in each group Cg=C / G. X is grouped by channel to obtain {X1,...,XG}. Spatial Softmax normalization: Ag(h,w)=exp(Xg(h,w)) / Σh′,w′ exp(Xg(h′,w′)); Channel grouping L2 normalization: X̂g=Xg / (||Xg||2+ε); The final GDN output is the concatenation of the normalized results of each group; The GDN output is convolved with the learnable Value convolution kernel to obtain ConvAttention(x), and then added element-wise with the original input x and normalized to generate the intermediate feature f=Norm(x+ConvAttention(x)). In the FFN sub-layer, f is fed into a feedforward network consisting of two cascaded 3×3 convolutional layers (each followed by BN and ReLU activation), producing the final MSAM output y=Norm(f+FFN(f)). The number of channels in the MSAM output feature map is consistent with the input, and the feature map size remains unchanged through residual connections, providing a channel dimension-stable feature input for the subsequent VMFPN.

[0028] 3. Specific implementation of VMFPN neck network VMFPN takes the P3 / P4 / P5 feature maps output from the backbone network as input, projects them onto X0∈R^(64×H / 4×W / 4) after passing through an initial patch embedding layer (4×4 convolution, stride 4), and constructs a four-stage feature pyramid. In Stage 1 and Stage 2, the MSGDC module is introduced to process local features: MSGDC consists of three 3×3 grouped dilated convolutional layers (each with 8 groups), with dilation rates of 1, 2, and 3 respectively, to capture multi-scale receptive fields. In Stage 3 and Stage 4, the MSMHA module is introduced to complete token-by-token feature fusion (8 attention heads, embedding dimension consistent with the number of channels in the current stage). Each downsampling layer uses a convolutional layer with a stride of 2 and a kernel size of 2×2 (replacing pooling operations), doubling the number of activated channels (Stage 1: 64, Stage 2: 128, Stage 3: 256, Stage 4: 512). Feature embedding calculation at each stage: Initial embedding H0 = GELU(BN(Conv) 4x4(I))), X0 = H0 + Pe (Pe is the learnable position embedding); for the first two stages, H^(l-1) = MSGDC(X^(l-1)) + X^(l-1); for the last two stages, H^(l-1) = MSMHA(X^(l-1)) + X^(l-1); for each stage, X^(l) = B_MLP(H^(l-1)), where B_MLP consists of two fully connected layers (the inner layer dimension is 4 times the current number of channels, activated by GELU). MSGDC and MSMHA are both implemented using full-precision FP32 floating-point operations.

[0029] 4. Specific implementation of the CAFM module The CAFM module is embedded in the third fusion node of VMFPN (between Stage 3 and Stage 4). Its local and global branches share the same input feature map but are processed by independent convolutions. Local branch: First, a 1×1 convolution is used to reduce the number of channels to half of the input. Then, the compressed features are divided into 4 groups in the channel dimension. Cross-group information exchange is achieved by rearranging them through channel shuffle (4 groups). Then, a 3×3 depthwise separable convolution is performed to extract local spatial features, and the output is Fconv=W(3×3)(CS(W(1×1)(Y))). Global branch: Generate three tensors of shape (Ĥ×Ŵ×Ĉ) – query matrix Q, key matrix K, and value matrix V – using 1×1 convolution and 3×3 depthwise separable convolution respectively; Calculate Softmax attention with a learnable scaling parameter α (initialized to √C, where C is the number of input channels, α>0 and trainable): Attention(Q̂,K̂,V̂)=V̂·Softmax(Q̂·K̂ / α); Global branch output Fatt=W(1×1)·Attention(Q̂,K̂,V̂)+Y. CAFM module final output: Fout=Fatt+Fconv.

[0030] 7. Reasoning Implementation After training, the input image undergoes 640×640 normalization preprocessing, and then sequentially passes through the backbone network (MSAM feature extraction) → neck network (VMFPN multi-scale fusion + CAFM weak feature enhancement) → detection head (P3 / P4 / P5 three-scale prediction) → NMS post-processing, outputting the target's class label, bounding box coordinates, and confidence score. NMS thresholds: confidence 0.25, IoU threshold 0.45.

[0031] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for small target detection in UAV aerial images based on an improved YOLOv11, characterized in that, Includes the following steps: S1: Obtain the drone aerial image dataset, perform noise reduction and contrast enhancement on the images, and normalize all sample images to a preset resolution; Random cropping, rotation, scaling, Mosaic data augmentation, and random horizontal flipping are used to increase the diversity of training samples; S2: Randomly divide the dataset into training set, validation set and test set according to a preset ratio; S3: Embed the multi-scale attention module MSAM in layers 1, 2, 4 and 6 of the YOLOv11n backbone network to replace the original C3k2 module; S4: Replace the original neck FPN with a four-stage hybrid feature pyramid network VMFPN; the VMFPN takes the P3 / P4 / P5 feature maps output by the backbone network as input, and projects them to X0∈R^(64×H / 4×W / 4) after initial preset convolution to construct a four-stage feature pyramid; S5: Embed the Convolutional Attention Fusion Module (CAFM) at the 3rd fusion node of the VMFPN neck network in YOLOv11n; S6: Iteratively train the improved network model using the training set to obtain the target weights, and perform performance evaluation on the test set.

2. The method according to claim 1, characterized in that, The feature embedding calculation for each stage of VMFPN is as follows: Initial embedding H0 = GELU(BN(Conv) 4x4 (I))), X0=H0+Pe, Pe is the learnable position embedding; H^(l-1)=MSGDC(X^(l-1))+X^(l-1) in the first two stages; H^(l-1)=MSMHA(X^(l-1))+X^(l-1) in the last two stages; X^(l)=B_MLP(H^(l-1)) in each stage, where B_MLP is composed of two fully connected layers; MSGDC and MSMHA are both implemented using full-precision FP32 floating-point operations.

3. The method according to claim 1, characterized in that, In the CAFM module, the dimensionality reduction ratio of the local branch 1×1 convolution is 1 / 2, and the number of channel shuffling groups is 4; the global branch can learn the scaling parameter α, which is initialized to the square root of the number of input channels (α=√C) and α>0, and can be trained; the CAFM is embedded in the third fusion node between Stage 3 and Stage 4 of VMFPN, and the local branch and the global branch share the same input feature map but are processed by independent convolution.

4. The method according to claim 1, characterized in that, The strip convolution kernel size k is one of 3, 5 or 7; k=3 in the first and second layers of the backbone network MSAM, and k=5 in the fourth and sixth layers of MSAM; the number of channels in the output feature map of the MSAM module is consistent with the input, and the feature map size is kept unchanged through residual connections.

5. The method according to claim 1, characterized in that, The network training hyperparameters are set as follows: SGD optimizer is used, initial learning rate is 0.01, weight-decay is 0.0005, learning rate scheduling strategy is cosine annealing, total training epochs are 200, batch size is 8; Mosaic data augmentation and random horizontal flipping are enabled.

6. The method according to claim 1, characterized in that, The MSAM module outputs a feature map with the same number of channels as its input; the VMFPN third fusion node inputs a P3 stage feature map and a P4 stage feature map processed by MSMHA; the local and global branches of CAFM share the same input feature map but are processed by independent convolutions; the three modules are structurally coupled through the conservation of feature map channel count and hierarchical progression.

7. A small target detection system for drone aerial images, characterized in that, include: The feature extraction unit is the YOLOv11n backbone network, in which the C3k2 modules in layers 1, 2, 4, and 6 are replaced with MSAM modules; The MSAM module consists of a ConvAttention sub-layer and an FFN sub-layer connected in series with residuals. The ConvAttention sub-layer contains 1×k and k×1 strip convolutions, GDN grouping double normalization, and Value convolution. The FFN sub-layer contains two cascaded 3×3 convolutional layers. The feature fusion unit, the neck network, comprises a VMFPN structure and a CAFM module. The VMFPN is a four-stage feature pyramid structure. The first two stages (Stage 1 / Stage 2) use the MSGDC module (three-way dilation rate 1 / 2 / 3 3×3 grouped dilated convolutions, 8 groups), and the last two stages (Stage 3 / Stage 4) use the MSMHA module (8 heads). The number of channels doubles sequentially between stages (64→128→256→512). The CAFM module is embedded in the third fusion node of the VMFPN and includes parallel CNN local branches (1×1 convolution dimensionality reduction + channel shuffling (4 groups) + 3×3 depthwise separable convolutions) and Transformer global branches (Q / K / V projection + attention calculation with learnable parameter α). The two outputs are added together to obtain Fout=Fatt+Fconv. The detection inference unit, using the YOLOv11n detection head, is deployed at three detection scales: P3 / 8, P4 / 16, and P5 / 32, and performs classification and localization prediction on the fused features.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for detecting small targets in UAV aerial images according to any one of claims 1 to 6.

Images