A method, system, medium and equipment for detecting small targets in UAV aerial photography based on RT-DETR

Abstract

The present invention discloses a method, system, medium and equipment for detecting small targets in drone aerial photography based on RT-DETR, which belongs to the technical field of drone visual detection. The method collects aerial images based on drones, and inputs the images into a trained target detection model to obtain target position and category information. First, the low-level features of the image are extracted, and the information of the channel dimension and the spatial dimension are captured simultaneously based on an efficient multi-scale attention mechanism; the features are output through a convolution residual block. The key features are processed based on a single-scale feature interaction module; for the encoded feature map, the shallow and deep information are gradually fused through upsampling and lateral connection; in the downsampling stage, the contextual semantic information of the target is retained, and the information interaction between the global and local is strengthened. The present invention significantly improves the accuracy and robustness of small target detection, and expands the practicality and deployment value in actual application scenarios such as drone aerial photography and remote sensing monitoring.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle (UAV) visual detection, and specifically relates to a method, system, medium and equipment for detecting small targets in UAV aerial photography based on RT-DETR. Background Art

[0002] With the deep integration of unmanned aerial vehicle (UAV) platforms and computer vision technology, drone aerial object detection is playing a key role in agricultural pest monitoring, road network inspections, disaster search and rescue, and other fields. However, due to payload limitations during low-altitude flight, drones typically carry low- to medium-resolution cameras. This results in targets in aerial images (such as crop disease spots, traffic signs, and trapped people) being small (typically less than 32×32 pixels), with blurred texture features and dense spatial distribution. Effective detection of small targets, especially in real-world scenarios with complex background interference, variable lighting conditions, and target occlusion, has become a key bottleneck restricting the intelligent application of drones.

[0003] Currently, existing technical solutions for small target detection in drones can be divided into two categories: detection algorithms based on convolutional neural networks (CNNs) and detection algorithms based on Transformer structures.

[0004] Typical examples of the first category of detection algorithms include the YOLO series and SSD. These methods employ strategies such as feature pyramids or multi-scale fusion to enhance the model's detection capabilities for objects of varying sizes. For example, YOLOv5 improves detection accuracy while ensuring real-time performance by introducing the PANet path augmentation network and the CSP module. However, these methods, which mostly rely on a local receptive field, struggle to capture long-range dependencies in images. They are still prone to missed detections and false detections when faced with occlusion, complex backgrounds, and densely populated scenes with small objects.

[0005] The second category of detection algorithms is represented by DETR and its improved versions, such as Deformable DETR, DINO, and RT-DETR. DETR first introduced the Transformer for object detection, implementing an end-to-end prediction mechanism and avoiding redundant candidate box generation. As a highly efficient variant, RT-DETR utilizes a lightweight backbone and a dynamic query update mechanism, achieving good detection performance while maintaining low inference latency. However, RT-DETR's design focuses on balancing speed and overall detection capability, and its detailed modeling of small objects is still insufficient, especially in drone aerial imagery, where small object features are easily diluted. Furthermore, RT-DETR currently lacks a feature extraction module specifically designed for small object enhancement. In densely populated target scenes, its global matching mechanism can cause small objects to be obscured by larger objects or background information, thus affecting detection effectiveness.

[0006] Although existing object detection algorithms have made significant progress in many common scenarios, they still face many challenges and technical bottlenecks in the task of detecting small targets on UAV platforms. These challenges are mainly reflected in the following aspects:

[0007] (1) Scale-sensitivity flaws of traditional convolutional neural network models: Mainstream single-stage detectors (such as the YOLO series) rely on downsampling to extract high-level semantic features, resulting in severe attenuation of small object features in deep networks. Although structures such as the Feature Pyramid Network (FPN) attempt to alleviate this problem through feature pyramid fusion, cross-scale feature alignment bias still causes a decrease in small object localization accuracy.

[0008] (2) Computational efficiency bottleneck of the Transformer architecture: Although DETR-based detection models can utilize global attention mechanisms to enhance context modeling of small objects, their inherent high computational complexity (such as the iterative optimization mechanism of Deformable DETR) makes it difficult to meet the rigid requirements of real-time inference (>30FPS) for drone edge devices. Although existing RT-DETR variants accelerate convergence through dynamic label matching, they still do not have a dedicated attention mechanism designed to address the feature sparsity problem unique to small objects.

[0009] (3) Insufficient data-driven generalization capability: Existing methods usually rely on general data augmentation strategies such as random cropping and scaling, and lack targeted enhancement methods such as multi-scale synthesis of small aerial targets and background noise simulation, resulting in limited generalization performance of the model in real complex scenarios. Summary of the Invention

[0010] The purpose of the present invention is to provide a method, system, medium and equipment for detecting small targets in drone aerial photography based on RT-DETR, aiming to solve the problem of insufficient accuracy of the existing RT-DETR model in small target detection tasks.

[0011] The present invention is mainly achieved through the following technical solutions:

[0012] A small target detection method for UAV aerial photography based on RT-DETR, comprising the following steps:

[0013] Step S1: collecting aerial images based on drones and performing preprocessing;

[0014] Step S2: Build a target detection model based on RT-DETR, input the image into the trained target detection model, and obtain the target location and category information;

[0015] Step S21: In the backbone network, first, the convolution block is used to extract low-level features of the image, and the efficient attention convolution block is used to capture information in both the channel dimension and the spatial dimension based on the efficient multi-scale attention mechanism. Then, feature maps P2 to P4 are output through several sequentially arranged double convolution residual blocks, and feature map P5 is output through the tail deformable convolution residual block.

[0016] Step S22: In the hybrid encoder, the feature map P5 is processed based on the single-scale feature interaction module to perform context modeling and dependency capture; convolutional feature extraction is performed on the feature maps P2 to P4; then, for the encoded feature maps, the shallow high-resolution detail information and the deep high-semantic abstract information are gradually fused through upsampling and lateral connection; in the downsampling stage, the context guidance module is used to retain the contextual semantic information of the target and strengthen the information interaction between the global and local.

[0017] Step S23: IoU-aware query selection selects a fixed number of features from the feature sequence output by the encoder as the initial target query for the decoder; the Transformer decoder iteratively optimizes the target query through the auxiliary prediction head;

[0018] Step S24: The head module maps the target query vector output by the Transformer decoder to the final detection result, and the detection result includes two parts: target classification prediction and bounding box regression.

[0019] In order to better implement the present invention, further, in the step S21, a convolution block is used to extract low-level features of the image, and the feature map is spatially downsampled through the maximum pooling layer to achieve spatial compression and semantic abstraction of the feature map; then, in the efficient attention convolution block, the input channel is divided into two parts through partial convolution operations, one part of which performs convolution and efficient multi-scale attention module processing in sequence, and the other part directly skips the convolution operation, and finally the two parts of features are spliced.

[0020] In order to better implement the present invention, further, in step S21, the dual convolution residual block uses a dual convolution filter combined with convolutions of different scales or receptive fields to focus on local details and contextual information when extracting features; the deformable convolution residual block uses deformable convolution to adaptively learn sampling positions to capture the geometric deformation and non-aligned features of the target.

[0021] In order to better implement the present invention, further, in step S24, in the downsampling stage, a parallel classification branch and regression branch are used to process the output of the Transformer decoder; the classification branch uses a multi-layer perceptron to output the category probability distribution of the target corresponding to each query, and the regression branch predicts the bounding box coordinate information of the corresponding target.

[0022] A small target detection system for drone aerial photography based on RT-DETR is based on the above-mentioned small target detection method for drone aerial photography based on RT-DETR, and includes a data acquisition and processing module, a model training module, and a detection module; the data acquisition and processing module is used to acquire aerial images and perform image preprocessing; the model training module is used to build a target detection model based on RT-DETR and train the target detection model; the detection module is used to input the image into the trained target detection model to obtain target position and category information;

[0023] The target detection model includes a backbone network, a Transformer-based hybrid encoder, an IoU-aware query selection layer, a Transformer decoder, and a head module, which are arranged in sequence; the head module is used to map the target query vector output by the Transformer decoder to obtain target classification prediction and bounding box regression.

[0024] In order to better realize the present invention, further, the backbone network includes a convolution block, an efficient attention convolution block, a maximum pooling layer, three dual convolution residual blocks and a deformable convolution residual block arranged in sequence from front to back; the three dual convolution residual blocks arranged in sequence from front to back are used to output feature maps P2 to P4 respectively, and the deformable convolution residual block is used to output feature map P5.

[0025] In order to better implement the present invention, further, the dual convolution residual block includes a convolution layer, a dual convolution filter and an activation layer arranged in sequence from front to back; the deformable convolution residual block includes a convolution layer, a deformable convolution and an activation layer arranged in sequence from front to back.

[0026] In order to better implement the present invention, further, the Transformer-based hybrid encoder includes the first to fourth convolutional layers arranged in parallel and the first to sixth feature splicing layers arranged sequentially from front to back; the feature maps P2 to P4 are respectively input into the first to third convolutional layers, and the feature map P5 is input into the fourth convolutional layer;

[0027] A reparameterized convolution module, a convolution layer, and a second upsampling layer are sequentially arranged between the first feature splicing layer and the second feature splicing layer from front to back, and the convolution layer is connected to the fifth feature splicing layer; a reparameterized convolution module and a third upsampling layer are sequentially arranged between the second feature splicing layer and the third feature splicing layer from front to back, and the reparameterized convolution module is connected to the fourth feature splicing layer; a reparameterized convolution module and a context guidance module are sequentially arranged between the third feature splicing layer and the fourth feature splicing layer, and between the fourth feature splicing layer and the fifth feature splicing layer, respectively, from front to back; a reparameterized convolution module is arranged between the fifth feature splicing layer and the sixth feature splicing layer; the reparameterized convolution module between the fourth feature splicing layer and the fifth feature splicing layer is connected to the sixth feature splicing layer;

[0028] The single-scale feature interaction module, convolution layer and first upsampling layer are arranged in sequence from front to back between the fourth convolution layer and the first feature splicing layer; the first to third convolution layers are connected to the third feature splicing layer, the second feature splicing layer and the first feature splicing layer respectively.

[0029] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned RT-DETR-based small target detection method for drone aerial photography.

[0030] An electronic device comprises a memory and a processor; the memory stores a computer program; the processor is used to execute the computer program in the memory to implement the above-mentioned RT-DETR-based UAV aerial photography small target detection method.

[0031] The beneficial effects of the present invention are as follows:

[0032] (1) This invention effectively breaks through the bottleneck of traditional technologies in feature expression and spatial modeling, significantly improves the accuracy and robustness of small target detection, and expands its practicality and deployment value in practical application scenarios such as drone aerial photography and remote sensing monitoring. On the one hand, the invention introduces an efficient attention convolution block in the feature extraction stage. Based on the efficient multi-scale attention mechanism, it pays attention to the channel and spatial dimension information at the same time during the feature extraction process, and integrates the information expression capabilities of the channel dimension and the spatial dimension. Under the premise of introducing a small amount of parameters and computational overhead, it significantly improves the feature discrimination ability and representation efficiency, and effectively improves the feature expression ability. On the other hand, the invention introduces a dual convolution residual block and a deformable convolution residual block, and adopts a dual convolution structure and deformable convolution to replace some convolution operations, so that the network has stronger geometric modeling capabilities and deformation adaptability, especially enhancing the network's modeling ability for small target deformation, and showing higher robustness and accuracy when dealing with small targets with blurred boundaries, variable shapes or complex postures.

[0033] (2) In the feature fusion stage, the present invention adopts a combination of upsampling and lateral connection to integrate shallow high-resolution detail information and deep high-semantic abstract information layer by layer, thus achieving multi-layer feature fusion and effectively integrating shallow high-resolution information with deep high-semantic information. In addition, the context guidance module is introduced in the downsampling process to fully retain the key contextual semantic features and achieve the coordinated perception of local details and global semantics. While maintaining the RT-DETR decoding structure, the present invention adds a hybrid feature interaction module to the neck. The present invention is particularly suitable for detection tasks with significant target scale differences, dense targets in the scene, or complex background interference, and effectively enhances the generalization ability and scene adaptability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of the structure of the convolution block;

[0035] Figure 2 Schematic diagram of the structure of the efficient attention convolution block;

[0036] Figure 3 Schematic diagram of the structure of the efficient multi-scale attention module;

[0037] Figure 4 Schematic diagram of the structure of the residual block;

[0038] Figure 5 Schematic diagram of the structure of the double convolution filter;

[0039] Figure 6 Schematic diagram of the structure of deformable convolution;

[0040] Figure 7 This is a schematic diagram of the structure of the context guidance module;

[0041] Figure 8 Schematic diagram of the overall structure of the network model of the present invention;

[0042] Figure 9 This is a flow chart of the RT-DETR-based small target detection method for UAV aerial photography of the present invention;

[0043] Figure 10 This is the processing flow chart of the target detection model;

[0044] Figure 11 These are drone aerial images and target detection renderings in different scenarios. DETAILED DESCRIPTION

[0045] Example 1:

[0046] A small target detection method for UAV aerial photography based on RT-DETR, such as Figure 9 As shown, the following steps are included:

[0047] 1. Data collection and processing;

[0048] Using a lightweight UAV platform equipped with a visible light camera, aerial photography missions are performed at various altitudes, pitch angles, and in complex lighting environments to acquire images or video data of small targets at multiple scales, in various poses, and under various occlusion conditions. The collected data should include realistic application scenarios such as inconsistent target-background ratios, complex background interference, and a variety of small target types to enhance the diversity and robustness of the training set.

[0049] Use data annotation tools to manually and accurately annotate each small object in the captured image. This annotation includes object category and bounding box location information. The annotation results are saved in a standard format (such as YOLO or COCOJSON). The dataset is then proportionally divided into a training set, a validation set, and a test set, ensuring a uniform distribution of scenes, categories, and object sizes across the different subsets to ensure the generalization of the model evaluation results.

[0050] After reading the images and their corresponding annotation files, all images are uniformly scaled to a fixed size of 640×640 pixels to ensure network input consistency. Various data augmentation operations are also performed on the images to improve the model's generalization and small object detection performance, including but not limited to: random horizontal flipping; image normalization; image brightness, contrast, and saturation adjustments; and random cropping or object affine transformation to enhance the model's robustness to object deformation and occlusion.

[0051] 2. Build a target detection model based on RT-DETR and train the target detection model; Figure 8 and Figure 10 The specific contents are as follows:

[0052] 2.1、The input image is first processed by the backbone feature extraction network (Backbone) in the initial stage. Figure 1 and Figure 2 As shown in Figure 1, this stage mainly consists of a convolution block (ConvBlock) and two efficient attention convolution blocks PEMABlock.

[0053] 2.1.1、 Figure 1 As shown, the convolutional block consists of the following structure: 1) a 3×3 two-dimensional convolutional layer (Conv2d) for extracting local features; 2) a batch normalization layer (BatchNorm2d) to stabilize training and accelerate convergence; and 3) a nonlinear activation function (ReLU) to enhance the model's nonlinear expressiveness. After stacking the three convolutional blocks, a max pooling layer (MaxPooling) spatially downsamples the feature map. This initial stage aims to gradually extract low-level, local, detailed features of the image. The stacked structure of multiple layers of 3×3 convolutions, combined with BatchNorm and ReLU, not only improves feature stability and expressiveness, but also enhances the perception of fine-grained information such as edges and textures. Finally, max pooling achieves spatial compression and semantic abstraction of the feature map, providing more compact and representative input features for the feature extraction and object detection modules in the subsequent deep network architecture.

[0054] 2.1.2. In order to enhance the perception of small objects by shallow features, an efficient multi-scale attention mechanism is introduced in the initial stage of the backbone network, and a feature extraction residual module - efficient attention convolution block PEMABlock is designed. Figure 2 As shown in the figure, the module consists of a partial convolution layer (PConv), a 1×1 two-dimensional convolution layer, an efficient multi-scale attention module (EMA) and a ReLU activation function. A residual connection is introduced between the output of the efficient multi-scale attention module (EMA) and its input to improve information flow and feature expression capabilities.

[0055] To balance computational efficiency and feature extraction, PEMABlock employs a partial convolution (PConv) operation. This method divides the input channel into two parts, performing a standard convolution operation on one part and skipping the convolution operation on the other. The two parts are then concatenated, effectively reducing computational effort while preserving key information.

[0056] like Figure 3As shown, the efficient multi-scale attention module (EMA) can simultaneously capture information in both the channel and spatial dimensions, effectively improving feature representation capabilities while incurring minimal parameter and computational overhead. This mechanism enhances semantic modeling by reshaping some channels to mimic batch dimensions and partitioning the channels into multiple sub-feature groups, allowing spatial semantic features to be more fully represented within each sub-group, thereby enhancing fine-grained feature modeling capabilities. The module utilizes a parallel sub-network structure, which helps to exploit cross-dimensional interactions and build stronger channel-spatial dependencies, significantly improving the model's ability to model long-range dependencies. EMA demonstrates excellent performance while maintaining lightweight and computationally efficient models, validating its effectiveness and practicality in improving model representation capabilities. Furthermore, the efficient multi-scale attention module (EMA), with its compact structure and moderate computational cost, is suitable for deployment on mobile devices. It has demonstrated good adaptability across a variety of computer vision tasks, possessing broad application value and practical significance.

[0057] Specifically, if Figure 3 As shown in the figure, in the efficient multi-scale attention module (EMA), the input feature map has a size of C × H × W. It is first divided into G channel groups, each of which has a size of (C / G) × H × W. On each sub-feature map, average pooling operations are performed in the horizontal and vertical directions, respectively, to obtain feature descriptions of size (C / G) × 1 × W and (C / G) × H × 1. These two sub-feature maps are then concatenated along the spatial dimension and input into a 1×1 two-dimensional convolutional layer for feature fusion, generating an attention vector of size (C / G) × 1 (H + W). This vector is activated with a sigmoid to generate two spatial attention maps, which weight the original sub-feature maps horizontally and vertically, respectively, to enhance spatial saliency.

[0058] The weighted feature map is then normalized using GroupNorm and average pooled to generate a (C / G)×1×1 channel description vector. This vector, after softmax activation, undergoes a matrix multiplication (MatMul) with the features output by the other channel's attention path to integrate spatial and inter-channel dependencies. The features output by these two attention paths are then element-wise fused and applied to the final global attention map using a sigmoid function.

[0059] Finally, the original input feature map and the attention map are element-wise re-weighted, and the output dimension is consistent with the input, that is, C×H×W, to achieve saliency enhancement and noise suppression of multi-scale target areas.

[0060] 2.2. In the second stage of feature extraction of the backbone network, residual blocks (DualConvolutional Kernels, DualConv) and Deformable Convolution (DeConv) are used (DualConvBasicBlock and DeConv BasicBlock), such as Figure 4 Figure 2 shows a schematic diagram of the dual convolution residual block (a) and a deformable convolution residual block (b). The introduction of multiple residual blocks increases network depth while effectively alleviating the vanishing gradient problem and ensuring stable learning of deep features. Through residual connections, the model can directly transfer low-level feature information, thereby improving the completeness and multi-layered nature of feature representation. Furthermore, residual blocks help the network better learn complex feature transformations, enhancing object discrimination, especially small objects and background noise. This provides richer and more robust mid- and high-level semantic features for subsequent detection branches. DualConv combines convolutions of different scales or receptive fields to simultaneously focus on local details and broader context when extracting features, thereby improving the ability to discern the edges and shapes of small objects. DeConv adaptively learns sampling locations to flexibly capture geometric deformation and misaligned features of objects. This makes it particularly suitable for detecting small objects with irregular shapes or significant positional variations, further improving detection robustness and accuracy.

[0061] Preferably, if Figure 5 As shown in Figure 2, the dual convolution filter DualConv integrates 3×3 and 1×1 convolution kernels, allowing parallel processing of the same input feature map channels and utilizing grouped convolution techniques to efficiently arrange convolution filters. In this setting, M Indicates the number of input channels, N represents the number of output channels and convolution filters, G represents the number of groups in the double convolution. Next, we consider dividing the N filters into G Groups, where each group processes this complete feature map. 3×3 and 1×1 convolution kernels are processed simultaneously input channels, and the remaining Each channel is processed by a 1×1 convolution kernel. The summation result of the symbolic representation integrates the outputs of these processes. The grouping architecture can enhance the sparsity of the block diagonal and promote the structured learning of highly correlated filters without the need for shift arrangement. The DualConv method reduces the parameters in the backbone network by using grouped convolution and promotes information exchange between different layers. It retains the input data and uses MThe 1×1 convolution achieves optimal cross-channel transmission. Therefore, no channel reordering operation is required to construct the DualConv filter.

[0062] Specifically, assuming the size of the output feature map is D o ×D o ×N, where D o are the width and height dimensions of the output feature map. In standard convolution, the input feature map is N The size is K × K × M The convolution filter is used to filter K × K is the convolution kernel size. Therefore, the standard convolution layer FL SC The total number of FLOPs executed in is:

[0063] (1)

[0064] In DualConv, the convolution filter bank G The number of is used to control the convolution filter K × K The ratio of the convolution kernel. For a given G , the size is ( K × K +1×1) combination and the ratio of the convolution kernel is 1 / G , and the ratio of the remaining 1×1 convolution kernel is (1-1 / G ). Therefore, in G In the double convolution layer composed of convolution filter groups, the number of FLOPs of the combined convolution kernel is:

[0065] (2)

[0066] The remaining 1 point-by-point convolution kernel FLOPs number:

[0067] (3)

[0068] The total number of FLOPs is:

[0069] (4)

[0070] Compare the computational cost (FLOPs) of the double convolutional layer with the standard convolutional layer and calculate the reduction rate R DC / SC for:

[0071] (5)

[0072] From formula (5), we can see that in DualConv design, ifK =3, when G When it is larger, the speedup ratio can reach 8 to 9 times.

[0073] Preferably, if Figure 6 As shown, DeConv can better adapt to geometric changes or model geometric transformations in object scale, pose, viewpoint, and partial deformation. Traditional convolution-based networks have difficulties with geometric transformations because their convolution and pooling layers are too rigid. Constrained by a fixed geometric structure, the convolution units sample feature maps at fixed positions, which hinders their good performance under various view angles and object sizes in drone images. To enhance the adaptability of extracted features, a deformable convolution layer is added to the backbone network, which adds a two-dimensional offset to the regular grid sampling positions in the standard convolution, so that the receptive field can be dynamically adjusted. The original convolution kernel has a regular rectangular shape, while the deformable convolution kernel adds an offset to each sample point, forming an irregular arrangement. These offsets are generated by applying another convolution layer to a consistent input map.

[0074] Deformable convolution consists of two steps: 1) Using a regular grid Sample the input feature map x; 2) Perform weighted summation of the sampled values ​​with weight w. The size and expansion of the receptive field are defined:

[0075] (6)

[0076] Define a 3×3 convolution kernel with a dilation rate of 1. For each position p0 on the output feature map y, there exists:

[0077] (7)

[0078] in: y ( p 0) is the output feature map in p The value at position 0;

[0079] p 0 is the center point of the convolution kernel;

[0080] p n are n points in the grid;

[0081] w ( p n ) indicates that the convolution kernel is at the offset position p n The weight on

[0082] x (p0+ p n ) is the input feature mapx The coordinates of the sampling point are p0+ p n The value of

[0083] p0+ p n is the coordinate of the sampling point, that is, the current center point plus the offset of each convolution kernel position;

[0084] in, p n Listed In deformable convolution, the regular grid By offset To enhance, Equation (7) becomes:

[0085] (8)

[0086] Where: Δ p n is the offset in the calculation;

[0087] x (p0+ p n +Δ p n ) is the input feature map x The coordinates of the sampling point are p0+ p n +Δ p n The value of

[0088] Sampling at irregular and offset locations Considering that the offset is usually a decimal, Equation (8) is implemented using bilinear interpolation:

[0089] (9)

[0090] in: x ( p ) is the offset position calculated by bilinear interpolation p The value of the point;

[0091] x ( q ) is the adjacent integer grid q The value of the point;

[0092] in, p Indicates any (fractional) position , q lists all integer quantized spatial locations in the feature map x, and G(q, p) is the bilinear interpolation kernel.

[0093] In summary, the backbone network design combines convolutional structure with multiple enhancement modules. In the initial stage, convolutional blocks are used to extract low-level image features, supplemented by efficient multi-scale attention modules to enhance the perception of small targets. In the middle and late stages, residual block structures such as DualConv and DeConv are introduced to balance computational efficiency and feature expression, significantly improving the network's robustness and detection accuracy for small targets, deformed targets, and complex backgrounds.

[0094] 2.2. The neck part of the network adopts an efficient hybrid encoder structure that combines the single-scale feature interaction mechanism with the multi-scale feature fusion strategy. Figure 8 As shown in the figure, a Transformer-based encoder module, the Attention-based Intra-scale Feature Interaction (AIFI) module, is introduced. This module operates only on feature map P5 output by the backbone network. Specifically, the two-dimensional feature map is first flattened into a one-dimensional vector sequence and input into the AIFI module for context modeling and dependency capture. Subsequently, its output is reconstructed into a two-dimensional feature map format for use in the subsequent cross-scale fusion module. In contrast, the remaining feature maps P2, P3, and P4 use conventional convolutional feature extraction to ensure overall computational efficiency.

[0095] like Figure 8 As shown in the figure, in order to achieve accurate perception of objects of different scales, the encoded feature map is upsampled and connected horizontally to gradually fuse shallow high-resolution detail information with deep high-semantic abstract information. Figure 7 As shown in the figure, during the downsampling phase, the Context Guide Block (CGB) is innovatively used to replace traditional pooling or strided convolution. This module fully preserves the contextual semantic information of the target, optimizes the interaction between global and local information, and effectively improves detection robustness and accuracy in scenarios with large object scale variations, dense objects, or complex backgrounds.

[0096] 2.3. The network's head module primarily maps the target query vector output by the Transformer decoder into the final detection result. It consists of two parts: target classification prediction and bounding box regression. This module typically consists of two parallel sub-branches: a classification branch, employing a multi-layer perceptron (MLP) structure, outputs a probability distribution for the target category corresponding to each query; and a regression branch, responsible for predicting the bounding box coordinates of the corresponding target. This branch can employ an MLP or lightweight convolutional network to reduce computational overhead. To ensure matching accuracy and learning stability, the Hungarian matching algorithm is introduced to optimize the one-to-one correspondence between target queries and ground truth annotations, thereby improving detection accuracy and inference efficiency.

[0097] As shown in Table 1, using the same dataset and evaluation metric (mAP@0.5), a comparison was conducted between the typical two-stage detector FasterR-CNN, the single-stage detectors YOLOv5s, YOLOv7, and YOLOv8s, and the Transformer-based detector RT-DETR. The results show that Example 1 achieved a mean average precision (mAP) of 52.0% on seven representative object categories, a 3.4 percentage point improvement (approximately 7% relative increase) over the original RT-DETR, an 11.2 percentage point improvement over the current highest-accuracy single-stage model YOLOv8s, and more than double the two-stage baseline FasterR-CNN. Breaking down the categories, Example 1 achieved detection accuracies of 29.4%, 38.6%, and 51.2% for small, morphologically diverse objects such as bicycles, tricycles, and motorcycles, respectively, representing improvements of 56%, 16%, and 11% over RT-DETR, respectively. Except for a slight decrease in the "bus" category (-1.8 percentage points), all other types of targets achieved varying degrees of improvement, verifying the significant gain effect of structural improvements such as double convolution, deformable convolution, and efficient multi-scale attention on small target detection, and further confirming the practical value of the context-guided feature fusion strategy in complex backgrounds and target-dense scenes.

[0098] like Figure 11As shown, (a1)-(a3) are the original aerial images in three scenarios respectively; (b1)-(b3) are the target detection results output by the existing RT-DETR model in three scenarios respectively; (c1)-(c3) are the target detection results output by the present invention in three scenarios respectively. By comparison, it is found that the detection results output by the present invention are more accurate, which significantly improves the robustness and practicality in aerial photography applications. Specifically, the present invention enhances the feature extraction capability by fusing the attention mechanism with deformable convolution; adopts an upsampling-lateral connection structure and combines it with a context-guided module to achieve context-aware feature fusion; the present invention performs a lightweight hybrid feature encoding strategy of single-scale interaction on the key semantic feature P5. The present invention takes into account both detection accuracy and deployment efficiency, and improves the robustness and practicality of the model in applications such as aerial photography and remote sensing.

[0099] Table 1

[0100]

[0101] Example 2:

[0102] A small target detection system for aerial photography by unmanned aerial vehicles (UAVs) based on RT-DETR includes a data acquisition and processing module, a model training module, and a detection module; the data acquisition and processing module is used to acquire aerial images and perform image preprocessing; the model training module is used to build a target detection model based on RT-DETR and train the target detection model; the detection module is used to input images into the trained target detection model to obtain target position and category information.

[0103] like Figure 8 As shown in the figure, the target detection model includes a backbone network, a Transformer-based hybrid encoder, an IoU-aware query selection layer, a Transformer decoder, and a head module arranged in sequence; the head module is used to map the target query vector output by the Transformer decoder to obtain target classification prediction and bounding box regression.

[0104] The backbone network includes a convolution block, an efficient attention convolution block, a maximum pooling layer, three dual convolution residual blocks and a deformable convolution residual block arranged in sequence from front to back; the three dual convolution residual blocks arranged in sequence from front to back are used to output feature maps P2 to P4 respectively, and the deformable convolution residual block is used to output feature map P5. Figure 1 As shown in , the convolution block includes a two-dimensional convolution layer, a batch normalization layer, and an activation layer arranged in sequence from front to back. Figure 2 As shown, the efficient attention convolution block includes a partial convolution layer, a two-dimensional convolution layer, an efficient multi-scale attention module EMA and an activation layer arranged in sequence from front to back. The structure of the efficient multi-scale attention module EMA is as follows Figure 3 shown.

[0105] like Figure 4 As shown in (a) in FIG, the double convolution residual block includes a convolution layer, a double convolution filter and an activation layer arranged in sequence from front to back; Figure 4 As shown in (b), the deformable convolution residual block includes a convolution layer, a deformable convolution layer and an activation layer arranged in sequence from front to back.

[0106] like Figure 8 As shown, the Transformer-based hybrid encoder includes the first to fourth convolutional layers arranged in parallel and the first to sixth feature splicing layers arranged in sequence from front to back; the feature maps P2 to P4 are input into the first to third convolutional layers, respectively, and the feature map P5 is input into the fourth convolutional layer. A reparameterized convolution module, a convolution layer, and a second upsampling layer are arranged in sequence from front to back between the first feature splicing layer and the second feature splicing layer, and the convolution layer is connected to the fifth feature splicing layer; a reparameterized convolution module and a third upsampling layer are arranged in sequence from front to back between the second feature splicing layer and the third feature splicing layer, and the reparameterized convolution module is connected to the fourth feature splicing layer; a reparameterized convolution module and a third upsampling layer are arranged in sequence from front to back between the third feature splicing layer and the fourth feature splicing layer, and a reparameterized convolution module is connected to the fourth feature splicing layer; a reparameterized convolution module and a third upsampling layer are arranged in sequence from front to back between the fourth feature splicing layer and the fifth feature splicing layer. There is a reparameterized convolution module and a context guidance module; a reparameterized convolution module is arranged between the fifth feature splicing layer and the sixth feature splicing layer; the reparameterized convolution module between the fourth feature splicing layer and the fifth feature splicing layer is connected to the sixth feature splicing layer; the single-scale feature interaction module, convolution layer and first upsampling layer are arranged in sequence from front to back between the fourth convolution layer and the first feature splicing layer; the first convolution layer to the third convolution layer are respectively connected to the third feature splicing layer, the second feature splicing layer and the first feature splicing layer.

[0107] like Figure 7 As shown, the context-guided module includes a two-dimensional convolutional layer arranged in sequence from front to back, an expanded convolutional layer and a two-dimensional convolutional layer arranged in parallel, as well as a feature splicing layer, a batch normalization layer, a parameterized ReLU operator, a global average pooling layer and a fully connected layer.

[0108] This invention effectively breaks through the bottleneck of traditional technologies in feature expression and spatial modeling, significantly improves the accuracy and robustness of small target detection, and expands its practicality and deployment value in actual application scenarios such as drone aerial photography and remote sensing monitoring.

[0109] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A small target detection method for UAV aerial photography based on RT-DETR, characterized in that: The following steps are involved: Step S1: collecting aerial images based on drones and performing preprocessing; Step S2: Build a target detection model based on RT-DETR, input the image into the trained target detection model, and obtain the target location and category information; Step S21: In the backbone network, first, the convolution block is used to extract low-level features of the image, and the efficient attention convolution block is used to capture information in both the channel dimension and the spatial dimension based on the efficient multi-scale attention mechanism. Then, feature maps P2 to P4 are output through several sequentially arranged double convolution residual blocks, and feature map P5 is output through the tail deformable convolution residual block. Step S22: In the hybrid encoder, the feature map P5 is processed based on the single-scale feature interaction module to perform context modeling and dependency capture; convolutional feature extraction is performed on the feature maps P2 to P4; then, for the encoded feature maps, the shallow high-resolution detail information and the deep high-semantic abstract information are gradually fused through upsampling and lateral connection; in the downsampling stage, the context guidance module is used to retain the contextual semantic information of the target and strengthen the information interaction between the global and local. Step S23: IoU-aware query selection selects a fixed number of features from the feature sequence output by the encoder as the initial target query for the decoder; The Transformer decoder iteratively optimizes the target query through the auxiliary prediction head; Step S24: The head module maps the target query vector output by the Transformer decoder to the final detection result, and the detection result includes two parts: target classification prediction and bounding box regression.

2. A method for detecting small targets in drone aerial photography based on RT-DETR according to claim 1, characterized in that: In step S21, a convolution block is used to extract low-level features of the image, and the feature map is spatially downsampled through the maximum pooling layer to achieve spatial compression and semantic abstraction of the feature map; then, in the efficient attention convolution block, the input channel is divided into two parts through partial convolution operations, one part of which performs convolution and efficient multi-scale attention module processing in sequence, and the other part directly skips the convolution operation, and finally the two parts of features are spliced.

3. A method for detecting small targets in drone aerial photography based on RT-DETR according to claim 2, characterized in that: In step S21, the dual convolution residual block uses a dual convolution filter combined with convolutions of different scales or receptive fields to focus on both local details and contextual information when extracting features; the deformable convolution residual block uses deformable convolution to adaptively learn sampling positions to capture geometric deformation and non-aligned features of the target.

4. The method for detecting small targets in drone aerial photography based on RT-DETR according to claim 1, wherein: In step S24, during the downsampling phase, a parallel classification branch and regression branch are used to process the output of the Transformer decoder; the classification branch uses a multi-layer perceptron to output the category probability distribution of the target corresponding to each query, and the regression branch predicts the bounding box coordinate information of the corresponding target.

5. A UAV aerial photography small target detection system based on RT-DETR, based on the UAV aerial photography small target detection method based on RT-DETR according to any one of claims 1 to 4, characterized in that: It includes a data acquisition and processing module, a model training module and a detection module; the data acquisition and processing module is used to acquire aerial images and perform image preprocessing; the model training module is used to build a target detection model based on RT-DETR and train the target detection model; the detection module is used to input the image into the trained target detection model to obtain target location and category information; The target detection model includes a backbone network, a Transformer-based hybrid encoder, an IoU-aware query selection layer, a Transformer decoder, and a head module, which are arranged in sequence; the head module is used to map the target query vector output by the Transformer decoder to obtain target classification prediction and bounding box regression.

6. The RT-DETR-based UAV aerial photography small target detection system according to claim 5, characterized in that: The backbone network includes a convolution block, an efficient attention convolution block, a maximum pooling layer, three dual convolution residual blocks and a deformable convolution residual block arranged in sequence from front to back; the three dual convolution residual blocks arranged in sequence from front to back are used to output feature maps P2 to P4 respectively, and the deformable convolution residual block is used to output feature map P5.

7. The RT-DETR-based UAV aerial photography small target detection system according to claim 6, characterized in that: The dual convolution residual block includes a convolution layer, a dual convolution filter and an activation layer arranged in sequence from front to back; the deformable convolution residual block includes a convolution layer, a deformable convolution and an activation layer arranged in sequence from front to back.

8. The RT-DETR-based UAV aerial photography small target detection system according to claim 5, characterized in that: The Transformer-based hybrid encoder includes the first to fourth convolutional layers arranged in parallel and the first to sixth feature splicing layers arranged sequentially from front to back; the feature maps P2 to P4 are input into the first to third convolutional layers respectively, and the feature map P5 is input into the fourth convolutional layer; A reparameterized convolution module, a convolution layer, and a second upsampling layer are sequentially arranged between the first feature splicing layer and the second feature splicing layer from front to back, and the convolution layer is connected to the fifth feature splicing layer; a reparameterized convolution module and a third upsampling layer are sequentially arranged between the second feature splicing layer and the third feature splicing layer from front to back, and the reparameterized convolution module is connected to the fourth feature splicing layer; a reparameterized convolution module and a context guidance module are sequentially arranged between the third feature splicing layer and the fourth feature splicing layer, and between the fourth feature splicing layer and the fifth feature splicing layer, respectively, from front to back; a reparameterized convolution module is arranged between the fifth feature splicing layer and the sixth feature splicing layer; The reparameterized convolution module between the fourth feature splicing layer and the fifth feature splicing layer is connected to the sixth feature splicing layer; The single-scale feature interaction module, convolution layer and first upsampling layer are arranged in sequence from front to back between the fourth convolution layer and the first feature splicing layer; the first to third convolution layers are connected to the third feature splicing layer, the second feature splicing layer and the first feature splicing layer respectively.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for detecting small targets in drone aerial photography based on RT-DETR according to any one of claims 1 to 4 is implemented.

10. An electronic device, characterized in that: It includes a memory and a processor; the memory stores a computer program; the processor is used to execute the computer program in the memory to implement the RT-DETR-based UAV aerial photography small target detection method described in any one of claims 1 to 4.

Images