A UAV aerial small target detection method based on CVG-DETR

Abstract

This invention provides a method for small target detection in UAV aerial photography based on CVG-DETR, relating to the fields of computer vision and UAV intelligent perception technology. The method includes: acquiring and preprocessing a publicly available UAV aerial small target detection dataset; constructing a CVG-DETR model based on the RT-DETR-R18 framework, wherein CMSD is used for global dependency modeling, the P2 feature layer is used to preserve shallow high-resolution details, CGSP is used for cross-scale feature adaptive fusion, and InnerMPDIoU is used to enhance the localization constraints of small target bounding boxes; training the model using the preprocessed dataset; inputting the aerial image to be detected into the trained model, and outputting the target category, bounding box position, and confidence score. This invention can enhance the feature representation capability in complex backgrounds, densely distributed, occluded, and small-scale target scenes, reducing the problems of missed detections and inaccurate localization.

Description

Technical Field

[0001] This invention relates to the field of computer vision target detection technology, and more specifically, to a method for detecting small targets in drone aerial photography based on deep learning, which is particularly suitable for target detection in drone aerial images in complex backgrounds, densely distributed targets, occlusion, lighting changes, and small-scale target scenes. Background Technology

[0002] With the rapid development of drone technology, drones, due to their high maneuverability, wide field of view, and high data acquisition efficiency, have been widely used in traffic monitoring, urban management, emergency rescue, security patrol, and low-altitude intelligent sensing. Drone aerial images can acquire large-scale scene information from an aerial perspective, providing an important data foundation for target detection in complex environments.

[0003] However, in drone aerial images, targets are typically small in scale, have a low pixel ratio, are densely distributed, heavily occluded, and have complex backgrounds. Especially from a high-altitude, top-down perspective, targets such as pedestrians and vehicles are easily confused with background areas such as roads, buildings, trees, and shadows, leading to weakened target features and increasing the risk of missed detections, false detections, and inaccurate localization.

[0004] While deep learning methods can automatically extract features, CNN-type models lack global information modeling capabilities and are prone to missing detections when dealing with dense small targets. Transformer-type models, on the other hand, have global modeling capabilities, but suffer from high computational complexity and information dilution during feature fusion, making it difficult to balance accuracy, speed, and resource consumption.

[0005] RT-DETR-R18, as an end-to-end real-time target detection network, has certain advantages in detection speed and global modeling. However, in UAV aerial photography small target detection tasks, it still suffers from problems such as easy loss of shallow detail information, insufficient multi-scale feature interaction, and limited ability to represent small target features in complex backgrounds. Therefore, there is an urgent need for a UAV aerial photography small target detection method that can balance detection accuracy, model complexity, and small target feature representation ability. Summary of the Invention

[0006] The purpose of this invention is to provide a small target detection method for UAV aerial photography based on CVG-DETR, to address the problems of weak small target features, strong interference from complex backgrounds, dense target distribution, and easy loss of feature information in existing target detection models for UAV aerial images. This method enhances the model's ability to express detailed information about small targets, global contextual information, and multi-scale semantic information by introducing a state-space feature interaction module, a high-resolution P2 feature layer, and a cross-scale feature fusion module.

[0007] The method includes the following steps: S1. Obtain a publicly available drone aerial photography small target detection dataset, and preprocess the images and annotation information in the dataset to form sample data for model training and validation; S2. Construct a CVG-DETR UAV aerial photography small target detection model based on the RT-DETR-R18 framework. The model includes a CMSD state space feature interaction module, a CGSP cross-scale feature fusion module, a high-resolution P2 feature layer, and an InnerMPDIoU loss function. S3. Input the preprocessed public drone aerial photography dataset into the CVG-DETR drone aerial photography small target detection model for training; S4. Input the drone aerial image to be detected into the trained CVG-DETR drone aerial small target detection model, and output the target category, bounding box position and corresponding confidence score.

[0008] Preferably, in step S1, the publicly disclosed drone aerial photography small target detection dataset is the VisDrone2019 dataset. This dataset contains typical drone aerial photography scenarios such as complex backgrounds, scale changes, occlusion, lighting changes, and dense target distributions, and is suitable for verifying the performance of the model in drone small target detection tasks.

[0009] Preferably, in step S2, CVG-DETR is based on the end-to-end detection paradigm of RT-DETR-R18, and the feature extraction and multi-scale feature fusion structure is redesigned for the UAV aerial small target detection task. The feature extraction network uses a C2f module to extract texture, edge, and spatial location information in the shallow stage, and introduces a CMSD state-space feature interaction module in the deep stage to enhance the modeling ability of global context information and long-range dependencies.

[0010] Furthermore, the CMSD state-space feature interaction module embeds the state-space modeling mechanism into the C2f structure and is composed of multiple MSCB structural units. Each MSCB structural unit includes a convolutional position encoding branch, a state-space modeling branch, and a feedforward network branch. The convolutional position encoding branch is used to supplement two-dimensional positional information, the state-space modeling branch is used to establish global positional dependencies, and the feedforward network branch is used to enhance channel feature representation.

[0011] Furthermore, the CVG-DETR model introduces a high-resolution P2 feature layer to preserve the edge, texture, and spatial location information of small targets in shallow features, alleviating the problem of easy loss of small target detail information during deep network feature extraction, thereby improving the model's ability to perceive weak targets.

[0012] Furthermore, the CGSP cross-scale feature fusion module replaces the Concat and RepC3 fusion structure in the original network and fuses multi-scale pyramid features from different levels. The CGSP cross-scale feature fusion module includes a CGFM context-guided feature modulation unit and a VGSP visually guided lightweight perception unit.

[0013] Furthermore, the CGFM context-guided feature modulation unit concatenates the two aligned features and generates attention weights for different branches through a channel attention mechanism. Then, it uses the attention weights to adaptively recalibrate the two features and achieves complementary enhancement between shallow spatial detail information and deep semantic information through cross-residual fusion.

[0014] Furthermore, the VGSP visually guided lightweight perception unit adopts a dual-branch structure. One branch extracts local spatial features through depthwise separable convolution and enhances the features through the GSBC module. The other branch extracts lightweight local responses through depthwise convolution. The outputs of the two branches are concatenated in the channel dimension to obtain the enhanced fused features.

[0015] Furthermore, in step S2, the InnerMPDIoU loss function is used instead of the original GIoU loss function. By constructing auxiliary boxes inside the predicted box and the ground truth box, the overlap relationship between the internal boxes is calculated to enhance the network's localization constraint on the effective area of ​​the target. At the same time, based on the minimum point distance between the corresponding corner points of the predicted box and the ground truth box, a boundary box position deviation penalty term is introduced, so that the model pays attention to the overlapping area, scale difference and position offset of the target box during the regression process, thereby improving the boundary box localization accuracy in small target detection.

[0016] Furthermore, in step S4, the image to be detected is input into the trained CVG-DETR model, which ultimately outputs the category information, bounding box coordinate information, and confidence information of each target in the image.

[0017] The beneficial effects of this invention are as follows:

[0018] This invention proposes a small target detection method for UAV aerial photography based on CVG-DETR. It enhances the model's ability to model global contextual information and long-range dependencies through the CMSD module, introduces a high-resolution P2 feature layer to improve the model's ability to preserve edge, texture, and positional information of small targets, and designs a CGSP cross-scale feature fusion module to achieve effective interaction between shallow spatial detail information and deep semantic information. Experimental results show that on the VisDrone2019 dataset, compared with the baseline RT-DETR-R18, CVG-DETR's mAP50 increases from 45.2% to 50.6%, and mAP50:95 increases from 27.7% to 31.3%, while reducing the number of parameters from 20.0M to 14.8M. This demonstrates that the invention maintains low model complexity while improving detection accuracy. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process of a UAV aerial photography small target detection method based on CVG-DETR, as described in this invention.

[0020] Figure 2 This is a schematic diagram of the overall structure of the CVG-DETR model of the UAV aerial photography small target detection method based on CVG-DETR described in this invention.

[0021] Figure 3 This is a schematic diagram of the CMSD state space feature interaction module in the UAV aerial photography small target detection method based on CVG-DETR described in this invention.

[0022] Figure 4 This is a schematic diagram of the CGSP cross-scale feature fusion module in the UAV aerial photography small target detection method based on CVG-DETR described in this invention.

[0023] Figure 5 This is a schematic diagram of the CGFM context-guided feature modulation unit in the UAV aerial photography small target detection method based on CVG-DETR described in this invention.

[0024] Figure 6 This is a schematic diagram of the structure of the VGSP visually guided lightweight sensing unit in the UAV aerial photography small target detection method based on CVG-DETR described in this invention.

[0025] Figure 7 This is a visual comparison diagram of the detection results of the UAV aerial photography small target detection method based on CVG-DETR and the comparison method in different complex scenarios, as described in this invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0027] This invention provides a method for small target detection in UAV aerial photography based on CVG-DETR, such as... Figure 1 The steps shown are as follows: S1. Obtain a publicly available drone aerial photography small target detection dataset, and preprocess the images and annotation information in the dataset to form sample data for model training and validation; S2. Construct a CVG-DETR UAV aerial photography small target detection model. The model is based on the RT-DETR-R18 framework, integrates the CMSD state space feature interaction module and the CGSP cross-scale feature fusion module, introduces a high-resolution P2 feature layer and the InnerMPDIoU loss function. S3. Input the preprocessed public drone aerial photography dataset into the CVG-DETR drone aerial photography small target detection model for training, and optimize the model parameters through the loss function; S4. Input the drone aerial image to be detected into the trained CVG-DETR drone aerial small target detection model, and output the target category, bounding box position and corresponding confidence score.

[0028] In this embodiment, the static image detection subset of the VisDrone2019 dataset contains 6471 images in the training set and 548 images in the validation set. Before model training, the image sizes are uniformly adjusted to a predetermined input size, and the image annotation information is parsed to obtain the category label and bounding box coordinates corresponding to each target.

[0029] In one specific embodiment, in step S2, the CVG-DETR model diagram is as follows: Figure 2 As shown, the input image first passes through a feature extraction network to obtain pyramid features at different scales, then passes through AIFI and CGSP for semantic enhancement and cross-scale fusion, and finally the query selection module generates the target query, and the decoder and detection head output the detection results.

[0030] In this embodiment, the feature extraction network uses a C2f module for shallow feature extraction in the early stage. The shallow C2f module can better preserve the texture, edge, and position details in the UAV aerial images, providing a high-resolution spatial feature foundation for subsequent small target detection.

[0031] In this embodiment, the CMSD module in CVG-DETR is as follows: Figure 3As shown, this module embeds the state-space modeling mechanism into the C2f framework, enhancing the model's ability to model long-range dependencies and global context information while maintaining a lightweight structure, thereby improving the model's ability to identify small targets in complex backgrounds.

[0032] Specifically, the CMSD module consists of multiple MSCB structural units. Each MSCB structural unit includes a Convolutional Position Encoder (CPE), a State Space Modeling Branch (MSSM), and a Feedforward Network Branch (MLP). The Convolutional Position Encoder supplements the two-dimensional spatial positional information, the State Space Modeling Branch aggregates spatial dependencies globally, and the Feedforward Network Branch enhances the channel representation capability. These three components work synergistically, enabling the CMSD module to simultaneously focus on local detail information and global semantic information.

[0033] In the state-space modeling branch, for 2D visual features, NSSD models on the flattened spatial tokens and combines explicit positional encoding and a bidirectional / global aggregation mechanism to mitigate scan order bias. Instead of generating independent token-level hidden states for each spatial location, NSSD aggregates features from all locations into a shared global hidden state H. The global hidden state of NSSD can be represented as:

[0034] in, Input features representing the j-th spatial location This indicates the input-related state mapping parameters. The parameter represents the state parameter used to adjust the contribution of features at this location, and L represents the sequence length. NSSD aggregates global information by sharing a global hidden state and feeds back global dependencies to each spatial location in the output stage, thereby mitigating the influence of a fixed scan order.

[0035] In this embodiment, the CVG-DETR model further introduces a high-resolution P2 feature layer. By adding the P2 feature layer, more shallow high-resolution information can be retained, improving the model's ability to perceive small and distant targets.

[0036] In this embodiment, the high-level semantic features enhanced by AIFI are gradually fused with multi-scale features such as P4, P3 and P2 to form enhanced features that combine global semantics and local details.

[0037] In this embodiment, the CGSP cross-scale feature fusion module is as follows: Figure 4As shown, CGSP is used to replace the Concat and RepC3 fusion method in the original network. Compared with simple concatenation fusion, CGSP can enhance the information interaction between features at different scales through contextual guidance and visual guidance mechanisms, enabling adaptive fusion of deep semantic information and shallow spatial detail information.

[0038] Specifically, CGSP includes a CGFM (Context-Guided Feature Modulation Unit) and a VGSP (Visually Guided Lightweight Perception Unit). CGFM is used to perform cross-scale semantic interaction and complementary feature fusion, while VGSP is used to enhance local details and perform lightweight spatial perception on the fused features.

[0039] In this embodiment, the CGFM unit is as follows: Figure 5 As shown, it first concatenates the channels of the aligned features from both paths, and then generates channel-level attention weights through the SE channel attention mechanism. The process can be represented as follows:

[0040] Where F represents the concatenated input features. Indicates global average pooling. and This represents a two-level linear mapping. Represents the ReLU activation function. This represents the Sigmoid function, where S is the generated channel attention weight. The generated weights are then applied to the corresponding branches to achieve adaptive channel recalibration.

[0041] Furthermore, CGFM uses a cross-residual fusion mechanism to supplement the recalibrated high-level semantic features into the low-level spatial features, and feeds back the recalibrated low-level detail features into the high-level semantic features, thereby achieving bidirectional enhancement of shallow detail information and deep semantic information; then the two enhanced features are concatenated in the channel dimension to obtain the output features of CGFM.

[0042] In this embodiment, the VGSP adopts a dual-branch structure, such as... Figure 6 As shown, one branch extracts local spatial features through depthwise separable convolution and further enhances these features using the GSBC module; the other branch directly extracts lightweight local responses using depthwise convolution. Finally, the outputs of the two branches are concatenated along the channel dimension to obtain the output features of VGSP.

[0043]

[0044] in, This indicates a depthwise separable convolution operation. This indicates the GSBC feature enhancement module. This indicates a channel splicing operation.

[0045] Furthermore, the GSBC unit includes a GSConv branch and a depthwise convolutional bypass branch. The GSConv branch first generates channel features through standard convolution, then extracts local spatial information through depthwise convolution, and subsequently achieves feature fusion and channel interaction through concatenation and channel rearrangement operations. This process can be represented as:

[0046] in, Represents standard convolution. This indicates a channel rearrangement operation. Standard convolution has strong channel modeling capabilities, while DWConv has low computational overhead and good local awareness. Combining the two can alleviate the channel information isolation problem caused by using depthwise separable convolution alone.

[0047] The decoder layer uses the existing decoder layer in the RT-DETR model, which will not be elaborated here.

[0048] In this embodiment, the CVG-DETR model uses the InnerMPDIoU loss function instead of the original GIoU loss function in RT-DETR-R18 to optimize the localization accuracy of the target bounding box. Let the predicted bounding box be... The real frame is ). Through scale factor By scaling the predicted bounding box and the ground truth bounding box, we can obtain the corresponding auxiliary bounding boxes:

[0049]

[0050] in, This is the auxiliary bounding box scaling factor, used to control the size of the auxiliary bounding box relative to the original bounding box. It is calculated based on the auxiliary bounding box.

[0051] Based on this, corner distance constraints from MPDIoU are introduced. Let... This represents the squared Euclidean distance between the top-left corner of the predicted bounding box and the ground truth bounding box. This represents the squared Euclidean distance between the two lower right corners. If InnerMPDIoU is the normalized scaling term, then it can be expressed as:

[0052] The final loss function is:

[0053] In one specific embodiment, in step S3, the CVG-DETR model is trained using the preprocessed VisDrone2019 public dataset. The model training platform uses an NVIDIA RTX 4090 GPU, and the software environment includes Python, PyTorch, and CUDA. The model weights are iteratively optimized using the training set, and the model's detection performance is evaluated using the validation set.

[0054] In one specific embodiment, in step S4, the drone aerial image to be detected is input into the trained CVG-DETR model, and the decoder outputs the bounding box coordinates of each detected target and its corresponding confidence score.

[0055] The drone aerial image to be detected is a single aerial image. The detection results output by the model can be used to locate and classify small-scale targets such as pedestrians and vehicles in the image, and the detection results can be overlaid on the original image in the form of a visual bounding box, or converted into structured data for subsequent scene analysis.

[0056] To further demonstrate the effectiveness of this method, related ablation experiments were conducted. Since CGSP is a complete cross-scale fusion module formed by combining CGFM and VGSP, CGFM and VGSP were added to the ablation experiments to verify the contributions of their sub-modules. Table 1 Ablation Experiment:

[0057] Ultimately, CVG-DETR achieved high detection accuracy while maintaining a low number of parameters and computational cost, achieving a better balance between accuracy and complexity.

[0058] Table 2 also compares the proposed CVG-DETR with current mainstream object detection networks.

[0059] Table 2 Comparative Experiments:

[0060] CVG-DETR achieves superior performance on mAP50 and mAP50:95 while maintaining low parameter count and computational complexity, verifying the effectiveness and lightweight advantages of the proposed model in UAV small target detection tasks.

[0061] like Figure 7As shown in the overall results, CVG-DETR exhibits more stable small target detection capabilities across various complex scenes. In dense scenes, where vehicle targets are numerous, small in scale, and closely distributed, both YOLO11m and RT-DETR-R18 show some missed detections or localization errors. However, this method can detect more distant small targets while maintaining good bounding box localization. In occluded scenes, some targets are obscured by trees, buildings, and shadows, limiting the visible area. CVG-DETR can still accurately detect occluded targets, demonstrating its strong contextual modeling capabilities. In scenes with varying lighting, strong light and shadows weaken target edges and texture information, increasing detection difficulty. Compared to the comparison methods, CVG-DETR maintains relatively stable detection results in overexposed and shadowed areas, demonstrating strong adaptability to complex lighting conditions and feature discrimination capabilities. In low-light nighttime scenes, with low image brightness and reduced target-background contrast, CVG-DETR can still effectively detect distant and weak targets, exhibiting better low-light robustness.

[0062] In summary, this invention can effectively improve the problems of missed detection and inaccurate positioning in the detection of small targets in UAV aerial photography, and provides an effective solution for target detection in complex low-altitude scenarios.

[0063] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A method for detecting small targets in UAV aerial photography based on CVG-DETR, characterized in that, The images transmitted by the drone are processed using deep learning, and an improved deep learning model is used for object detection. The detection steps are as follows: S1. Obtain a publicly available drone aerial photography small target detection dataset, and preprocess the images and annotation information in the dataset to form sample data for model training and validation; S2. Construct a CVG-DETR UAV aerial photography small target detection model. The model is based on the RT-DETR-R18 framework, integrates the CMSD state space feature interaction module and the CGSP cross-scale feature fusion module, introduces a high-resolution P2 feature layer and the InnerMPDIoU loss function. S3. Input the preprocessed public drone aerial photography dataset into the CVG-DETR drone aerial photography small target detection model for training, and optimize the model parameters through the loss function; S4. Input the drone aerial image to be detected into the trained CVG-DETR drone aerial small target detection model, and output the target category, bounding box position and corresponding confidence score.

2. The method according to claim 1, characterized in that, The preferred dataset is the VisDrone2019 dataset, which contains drone aerial images of complex backgrounds, dense target distributions, scale variations, occlusions, lighting variations, and small target scenes.

3. The method according to claim 1, characterized in that, In step S1, the preprocessing includes unifying the size of the images in the public dataset, converting the format, reading the category information, and parsing the bounding box annotation information, and dividing the dataset into a training set and a validation set.

4. The method according to claim 1, characterized in that, In step S2, the feature extraction network uses the C2f module for feature extraction in the shallow stage to preserve the texture information, edge information and spatial location information of small targets in the UAV aerial image; in the deep stage, the CMSD state space feature interaction module is introduced to enhance the network's ability to model global context information and long-range dependencies.

5. The method according to claim 1, characterized in that, In step S2, the CMSD state space feature interaction module includes multiple MSCB structural units, each of which includes a convolutional position encoding branch, a state space modeling branch, and a feedforward network branch. The convolutional position encoding branch is used to supplement two-dimensional position information, the state space modeling branch is used to establish positional dependencies in the global scope, and the feedforward network branch is used to enhance channel feature expression.

6. The method according to claim 1, characterized in that, In step S2, the CGSP cross-scale feature fusion module, composed of the CGFM context-guided feature modulation unit and the VGSP visual-guided lightweight perception unit, is used to replace the Concat and RepC3 fusion structure in the original network to fuse multi-scale pyramid features from different levels, thereby enhancing the interaction between shallow spatial detail information and deep semantic information.

7. The method according to claim 1, characterized in that, In step S2, a high-resolution P2 feature layer is introduced into the CVG-DETR UAV aerial small target detection model. The P2 feature layer is used to retain the edge, texture and position information of small targets in the shallow high-resolution features, so as to reduce the loss of small target detail information in the deep feature extraction process.

8. The method according to claim 1, characterized in that, In step S2, the InnerMPDIoU loss function constructs internal auxiliary boxes between the predicted box and the ground truth box, calculates the overlap relationship between the internal boxes, and enhances the network's localization constraint on the effective area of ​​the target. At the same time, based on the minimum point distance between the corresponding corner points of the predicted box and the ground truth box, a boundary box position deviation penalty term is introduced, so that the model pays attention to the overlapping area, scale difference and position offset of the target box during the regression process, thereby improving the boundary box localization accuracy in small target detection.

9. The method according to claim 1, characterized in that, In step S3, when the preprocessed public drone aerial photography dataset is input into the CVG-DETR drone aerial photography small target detection model for training, multi-scale features are extracted through the CMSD state space feature interaction module, different scale features are fused through the CGSP cross-scale feature fusion module, and a loss function is constructed based on the difference between the model prediction results and the real annotations. The model parameters are updated through backpropagation.

10. The method according to claim 1, characterized in that, In step S4, after inputting the drone aerial image to be detected into the trained CVG-DETR drone aerial small target detection model, the model outputs the category information, bounding box coordinate information and confidence information of each target in the image.

Images