A method for detecting low-altitude small targets of anti-UAV based on RTDETR

Abstract

The application belongs to the technical field of computer vision and target detection, and discloses an anti-UAV low-altitude small target detection method based on RTDETR. A main network based on PVM is used to extract the features of an input image to obtain multi-scale features. A spatial gating unit based on self-attention in a hybrid encoder is used to process the highest level features in the multi-scale features to obtain gated features. The remaining features in the multi-scale features except the highest level features are taken, and the gated features are combined to input a content-guided attention fusion module to output fusion features. A candidate box with the minimum uncertainty of the fusion features is extracted as an initial reference box, and a corresponding feature vector is taken as an initial object query. The initial object query and the initial reference box are taken as a decoder based on a transformer, and the output of the decoder based on the transformer is taken through a prediction head to obtain a final target detection result. The application can significantly improve the small target detection performance while maintaining real-time performance.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and target detection technology, specifically involving a method for detecting low-altitude small targets against drones based on RTDETR (Real-Time Detection Transformer), which is applicable to drone monitoring and identification in fields such as military defense, public safety, and airport security. Background Technology

[0002] With the rapid development and widespread application of UAV technology, low-altitude small UAVs have brought new security challenges to military, public safety and critical infrastructure protection. These targets are characterized by small size, low flight altitude and fast movement speed, making it difficult for traditional detection methods to deal with them effectively. Although the RTDETR-based detection model performs well in general target detection, it still has obvious defects when dealing with small targets such as UAVs: (1) Insufficient feature extraction of small targets, the existing ResNet backbone network has limited ability to capture detailed features of small UAVs; (2) Insufficient spatial relationship modeling, the traditional feedforward network (FFN) lacks an explicit spatial modeling mechanism, making it difficult to establish long-range dependencies; (3) Low efficiency of multi-scale feature fusion, the existing fusion methods do not distinguish the importance of features at different scales, resulting in the small target features being submerged in the fusion process. Summary of the Invention

[0003] The purpose of this invention is to provide a method for detecting small low-altitude targets against drones based on RTDETR, which significantly improves the detection performance of small targets while maintaining real-time performance.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] A method for detecting low-altitude small targets against unmanned aerial vehicles based on RTDETR includes:

[0006] A PVM-based backbone network is used to extract features from the input image, resulting in multi-scale features.

[0007] The highest-level features in the multi-scale features are processed using the self-attention-based spatial gating unit in the hybrid encoder to obtain gated features;

[0008] Take the features other than the highest level features from the multi-scale features, combine them with the gated features input to the content-guided attention fusion module, calculate the attention weights from the three dimensions of channel, space and pixel, and output the fused features with the same number of input features;

[0009] A lightweight prediction head is used to generate candidate boxes for each fused feature, and the uncertainty of each candidate box is calculated. The first few candidate boxes with the smallest uncertainty are used as initial reference boxes, and the feature vectors corresponding to the first few candidate boxes with the smallest uncertainty are used as initial object queries.

[0010] The initial object query and initial reference box are used as input to the Transformer-based decoder, and the output of the Transformer-based decoder is passed through a prediction head to obtain the final object detection result.

[0011] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0012] Preferably, the step of extracting features from the input image using a PVM-based backbone network to obtain multi-scale features includes:

[0013] The input image is downsampled, and after downsampling, it is passed sequentially through two local spatial attention Mamba modules, two spatial reduction attention Mamba modules, an average pooling layer, and a linear layer to obtain multi-scale features.

[0014] The local spatial attention Mamba module includes downsampling processing, a visual Mamba state space model, and a local spatial attention module. The spatial reduction attention Mamba module includes downsampling processing, a visual Mamba state space model, and a spatial reduction self-attention module.

[0015] Preferably, the processing procedure for the visual Mamba state-space model is as follows:

[0016] The input features are doubled in dimension by linear projection and immediately split into two parallel branches.

[0017] The input features of the branch are passed through a linear layer, a convolutional layer, and an activation function layer to obtain the output of the first branch;

[0018] The input features of the branch are passed through a linear layer, a convolutional layer, an activation function layer, and a state space model to obtain the output of the second branch;

[0019] The outputs of the first and second branches are residually concatenated and then linearly projected back to the original input dimension to obtain the output features.

[0020] Preferably, the processing procedure of the local spatial attention module is as follows:

[0021] The input features are flattened and then subjected to depthwise convolution to extract spatial features;

[0022] Attention weights and value projections are generated based on the extracted spatial features, and spatial attention is calculated to obtain attention features.

[0023] The attention features and the flattened input features are joined by residual connections to obtain residual features;

[0024] The residual features are processed using a linear layer and then concatenated with the residual features to obtain the output features.

[0025] Preferably, the processing procedure of the space reduction self-attention module is as follows:

[0026] The input features are passed through a normalization layer to obtain normalized features, and query vectors, key vectors, and value vectors are generated based on the normalized features.

[0027] Input the key vector and value vector into the SR space reduction module, and perform SR space reduction operation respectively to obtain the reduced key vector and the reduced value vector.

[0028] Multi-head self-attention is calculated based on the query vector, the reduced key vector, and the reduced value vector, and attention features are output.

[0029] The attention features and normalized features are joined by residuals to obtain residual features;

[0030] The residual features are processed using a linear layer and then concatenated with the residual features to obtain the output features.

[0031] Preferably, the process of using a self-attention-based spatial gating unit in a hybrid encoder to process the highest-level features among multi-scale features to obtain gated features includes:

[0032] The highest-level feature in the multi-scale feature set is then connected to the residual through self-attention to obtain self-attention features;

[0033] The self-attention features are processed by layer normalization and the dimensions are expanded by the input projection matrix to obtain the expanded features;

[0034] The extended feature is divided into two parts. The first part of the feature is processed by layer normalization and then spatially gated modulation to obtain a gated signal. This signal is then multiplied element-wise with the second part of the feature to obtain the multiplied feature.

[0035] The multiplied features are processed through the output projection matrix to restore their dimensions, and then residually connected with the input features to obtain gated features.

[0036] Preferably, the remaining features from the multi-scale features, excluding the highest-level features, are combined with gated features input to a content-guided attention fusion module. Attention weights are calculated from three dimensions: channel, space, and pixel, and the fused features with the same number of input features are output, including:

[0037] In the multi-scale features, there are two features other than the highest level feature. The lower-level feature is set as the first feature, and the other is set as the second feature. The gated feature is set as the third feature.

[0038] The third feature is upsampled, then fused with the second feature using content-guided 3D attention, then fused with the first feature using content-guided 3D attention, and then fused with the second feature using the CSPRepLayer operation to obtain the first fused feature.

[0039] The first fused feature is downsampled, then fused with the second feature using content-guided 3D attention, and then processed by CSPRepLayer to obtain the second fused feature.

[0040] The second fused feature is downsampled, then fused with the third feature using content-guided 3D attention, and finally processed by CSPRepLayer to obtain the third fused feature.

[0041] Preferably, the content-guided 3D attention fusion is executed as follows:

[0042] Channel attention: Two input features are concatenated and then global average pooling is performed in the spatial dimension to generate channel statistical vectors. The channel statistical vectors are then passed through a two-layer bottleneck MLP to generate channel attention weights.

[0043] Spatial attention: Concatenate two input features, then apply max pooling and average pooling to the channel dimension respectively to generate two spatial feature maps. Concatenate the two spatial feature maps along the channel dimension and then pass them through a... Convolutional layers generate spatial attention weights;

[0044] Pixel attention: The channel attention weights and spatial attention weights are added to obtain a coarse attention map. The coarse attention map is then concatenated with the features obtained from the two input features and rearranged by channels. Finally, it is processed by depthwise separable convolution and activation function to generate pixel-level attention weights.

[0045] Two input features are dynamically fused using pixel-level attention weights to obtain preliminary fused features;

[0046] The initial fused features are passed through a convolutional layer and three RepVggBlock modules to obtain the first branch features. At the same time, the initial fused features are passed through a convolutional layer to obtain the second branch features. The first branch features and the second branch features are concatenated to obtain the output features.

[0047] Preferably, the calculation of the uncertainty of each candidate box includes:

[0048] Classification uncertainty: The disorder of the predicted distribution of categories is calculated using the entropy function, and is used as classification uncertainty;

[0049] Location uncertainty: The aspect ratio or coordinate variance of the predicted bounding box is taken as the location uncertainty;

[0050] Uncertainty of candidate boxes: The weighted fusion result of their classification uncertainty and localization uncertainty is used as the uncertainty of candidate boxes.

[0051] This invention provides a low-altitude small target detection method for anti-drone applications based on RTDETR. Addressing the challenges of small target drone detection, particularly the difficulty in feature extraction, this method improves detection accuracy through a triple-core architecture. First, a novel Pyramid Vision Mamba (PVM) architecture replaces the original ResNet backbone, enhancing multi-scale feature extraction capabilities and reducing computational complexity. Second, a spatial gating unit based on gMLP Blocks is introduced into the Efficient Hybrid Encoder to enhance spatial modeling capabilities. Finally, a content-guided attention mechanism is designed in the Cross-Scale Feature Fusion (CCFF) module to optimize the feature fusion process from three dimensions: channel, spatial, and pixel. This method significantly improves the detection accuracy of small target drones, maintaining real-time performance while reducing computational resource consumption, making it suitable for anti-drone applications such as border patrol and protection of key locations. Attached Figure Description

[0052] Figure 1 This is a flowchart of the anti-drone low-altitude small target detection method based on RTDETR of the present invention;

[0053] Figure 2 This is a schematic diagram of the PVM backbone network of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure of the visual Mamba state space model of the present invention;

[0055] Figure 4 This is a schematic diagram of the local spatial attention module of the present invention;

[0056] Figure 5This is a schematic diagram of the spatial reduction self-attention module of the present invention;

[0057] Figure 6 This is a schematic diagram of the structure of the gMLP Block of the present invention;

[0058] Figure 7 This is a flowchart of the multi-scale feature fusion method of the present invention;

[0059] Figure 8 A flowchart of three-dimensional attention fusion guided by the content of this invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0062] like Figure 1 As shown, this embodiment provides a method for detecting low-altitude small targets against drones based on RTDETR, specifically including the following steps:

[0063] Step 1: Use a PVM-based backbone network to extract features from the input image to obtain multi-scale features.

[0064] This embodiment modifies the backbone network of the original RTDETR model, replacing the original convolutional structure with a new Pyramid Vision Mamba (PVM) architecture. The improved backbone employs a hybrid PVM architecture, fully leveraging its multi-layered, pyramidal feature representation capabilities: preliminary feature extraction and resolution adjustment are achieved through progressive convolutional downsampling; subsequently, hybrid modules of local spatial attention and spatial reduction self-attention are introduced at different stages of MambaVision for efficient feature modeling and information aggregation. The multi-scale feature output of PVM consists of four-stage feature maps {P2 / 4, P3 / 8, P4 / 16, P5 / 32}, using increasing channel dimensions {128, 256, 384, 384} to extract semantic information from images at different scales. This architecture cleverly integrates the advantages of local feature extraction from convolutional operations, the linear complexity sequence modeling capability of the Mamba state-space model, and the global context capture capability of spatial reduction attention, thereby providing richer, higher-quality, and computationally efficient feature inputs for the subsequent detection head of the RTDETR model.

[0065] like Figure 2 As shown, this step involves the input image. , The number of channels in the input image. For the height of the input image, The input image width is progressively downsampled through a 3×3 convolutional layer with a stride of 2, reducing the image resolution to half its original size while expanding the number of channels to 64 dimensions, outputting a feature image. Then, for the feature image... Multi-scale features are obtained by sequentially passing through two local spatial attention Mamba modules, two spatial reduction attention Mamba modules, an average pooling layer, and a linear layer.

[0066] (1) The local spatial attention Mamba module includes downsampling processing, the visual Mamba state space model, and the local spatial attention module. Its processing flow is as follows: when processing the feature maps in stages P2 / 4 and P3 / 8, the feature images are... The input local spatial attention Mamba module performs deep feature enhancement. First, a 3×3 convolutional layer performs progressive downsampling with a stride of 2, reducing the image resolution to one-quarter of its original size while expanding the channel count to 128 dimensions. This feature map is then flattened and its dimensions are replaced, converting it into a sequence form that is input into the visual Mamba state-space model. Global sequence modeling is performed using discretized state-space equations and a selective scanning mechanism to capture long-range dependencies. Subsequently, the local spatial attention module modulates local features on this feature map using a large-kernel deep convolution, generating an attention weight matrix and recalibrating the features through a Hadamard product. This cascaded design allows local feature modulation to be performed based on global context information, forming a deeply enhanced feature representation. This module is repeated twice, outputting the results separately. and Two feature maps.

[0067] (1-1) As Figure 3 As shown, the detailed steps for processing the visual Mamba state-space model are as follows:

[0068] (1-1-1) Flatten the input feature map and permutate its dimensions to obtain ,in It is the sequence length. It is the embedding dimension. The input dimension is doubled through linear projection and immediately split into two parallel branches of input features.

[0069] (1-1-2) Pass the branch input features through a linear layer Convolutional layer and activation function layer This yields the output of the first branch. The convolutional branch is used to compensate for information that may be lost in the State-Space Model (SSM). .

[0070] (1-1-3) The input features of the second branch are passed through a linear layer, a convolutional layer, an activation function layer, and a state-space model to obtain the output of the second branch. The state-space model (SSM) branch performs sequence modeling. It includes discretization and selective scanning processes, with continuous parameters. By time step Discretize into , For the identity matrix, use the zero-order preservation rule:

[0071]

[0072]

[0073]

[0074]

[0075]

[0076] With the discretized system, the model performs actual sequence computations through a selective scan process:

[0077]

[0078]

[0079] The sequences are then stacked to form a feature map.

[0080]

[0081] in, for The potential state at any given moment for The potential state at any given moment for Input at any time for Output at any moment , This is the final output of the state-space model.

[0082] (1-1-4) Output of the two branches and pass The components are stitched together and then linearly projected back to the original dimension to form the final output. The formula is expressed as follows:

[0083]

[0084]

[0085] in, Features after splicing for operate, It is a linear projection.

[0086] (1-2) such as Figure 4 As shown, the detailed processing steps of the local spatial attention module are as follows:

[0087] (1-2-1) Input features flattened Spatial features are then extracted through depthwise convolution. The formula is expressed as follows: For layer normalization, For depthwise convolution:

[0088]

[0089] (1-2-2) Using spatial features Generate attention weights Sum projection The formula is expressed as follows: and For the weights and biases used to generate attention weights, and For the weights and biases used to generate the value projection:

[0090]

[0091]

[0092] (1-2-3) Through attention weights Sum projection Calculate spatial attention to obtain attention features. , For element-wise multiplication, and with Residual features are obtained by performing residual connection. residual characteristics Then processed by linear layers Output features are obtained by performing residual connections. The formula is expressed as follows:

[0093]

[0094]

[0095] (2) The spatially reduced attention Mamba module includes downsampling, a visual Mamba state space model, and a spatially reduced self-attention module. Its processing flow is as follows: When processing feature maps at stages P4 / 16 and P5 / 32, the spatially reduced self-attention Mamba module is used for efficient global modeling. First, a 3×3 convolutional layer is used for progressive downsampling. This convolutional layer operates with a stride of 2, reducing the image resolution to 1 / 8 of the original size while expanding the number of channels to 384 dimensions. Then, the visual Mamba state space model is used for sequence modeling, providing basic global context capture and state space transformation. Subsequently, the output of the visual Mamba state space model serves as the input to the spatially reduced self-attention mechanism. The Key and Value are spatially downsampled using a scaling factor, and attention calculation is performed on the reduced sequence, significantly reducing computational complexity while maintaining the global receptive field. This sequential processing ensures that the attention mechanism performs refined feature selection based on a rich global context. This module is repeated twice, outputting... and Two feature maps.

[0096] (2-1) As Figure 5 As shown, the detailed steps of the space reduction self-attention module are as follows:

[0097] (2-1-1) The input features are passed through a normalization layer to obtain normalized features, and a query vector, a key vector, and a value vector are generated based on the normalized features.

[0098] (2-1-2) Input the key vector and value vector into the SR space reduction module and perform SR space reduction operation respectively to obtain the reduced key vector and the reduced value vector.

[0099] This embodiment uses a scaling factor. Reduce the length of the key and value sequence to the original value. This reduces the computational complexity of self-attention from Reduce to The key and value matrices are reduced using the SR space reduction operation to obtain the reduced key vectors. and the reduced value vector Then, multi-head self-attention is calculated based on the query vector, the reduced key vector, and the reduced value vector to output attention features. The formula is as follows:

[0100]

[0101] The SR space reduction operation process is as follows:

[0102] Step 1: Reconstruct the spatial structure by restoring the one-dimensional sequence tensor Key (or Value) to its original two-dimensional spatial grid structure, using the output of the visual Mamba state-space model as input. to convert it into .

[0103] Step 2: Spatial dimension downsampling. A pooling operation is applied to the reshaped spatial feature map to reduce its resolution. The scaling ratio of the pooling is determined by a scaling factor. Control, among which The sequence length starts from Reduce to The formula represents:

[0104]

[0105] Step 3: Reconstruct the sequence by flattening the downsampled two-dimensional spatial feature map back into a one-dimensional sequence. Transform into .

[0106] Combining the above three steps, the entire SR space reduction operation can be concisely represented as: .

[0107] (2-1-3) Perform residual connection between attention features and normalized features to obtain residual features.

[0108] (2-1-4) The residual features are processed by a linear layer and then connected with the residual features to obtain the output features.

[0109] Multi-scale feature map output: After the complete processing flow, the network outputs four feature maps at different scales {P2 / 4, P3 / 8, P4 / 16, P5 / 32}, which correspond to the features respectively. , , and The feature maps form a multi-level feature pyramid, as shown in Table 1. All feature maps include global context information enhanced by Mamba: the P2 / 4 feature map has a resolution of 1 / 4 of the input image and 128 channels, incorporating early global context while maintaining rich spatial details; the P3 / 8 feature map has a resolution of 1 / 8 and 256 channels, balancing details and semantic information while including mid-level global context; the P4 / 16 feature map has a resolution of 1 / 16 and 384 channels, enhancing deep global context while maintaining rich semantic information; and the P5 / 32 feature map has a resolution of 1 / 32 and 384 channels, incorporating high-level global context within the strongest semantic representation. This multi-scale feature output, led by Mamba, provides comprehensive and high-quality feature input for subsequent detection heads.

[0110] Table 1. Four-stage feature pyramid structure

[0111]

[0112] Step 2: The highest-level features in the multi-scale features are processed using the self-attention-based spatial gating unit in the hybrid encoder to obtain gated features. In the efficient hybrid encoder part of the RTDETR model, this invention optimizes the feedforward network (FFN) in its core AIFI (Attention-based Intrascale Feature Interaction) module, replacing the traditional FFN structure with a gMLP Block, and explicitly enhancing spatial modeling capabilities through the spatial gating unit (SGU). For example... Figure 6 As shown, the specific steps are as follows:

[0113] Step 2.1: Expand the dimensionality of the features. Expand the feature map generated in Step 1. (That is, the highest-level feature in the multi-scale feature set) After passing through self-attention and residual connections, self-attention features are obtained. ( For batch size, For spatial location number, (For feature dimensions), perform layer normalization and expand the dimension formula as follows, where the weights are... bias ,parameter :

[0114]

[0115] Step 2.2: Perform layer normalization on the self-attention features and expand the dimensions by using the input projection matrix to obtain the expanded features. .

[0116] Step 2.3: Perform spatial gating modulation. Divide the features into two parts; the second part of the features... The first feature set carries the main content and contextual information from the input features. It is specifically responsible for learning the relationships between different locations in the spatial dimension and generating a control signal, where the feature segmentation is represented as follows:

[0117]

[0118] Spatial gated modulation is then performed: through the gate weight matrix. and bias matrix Calculate the gating signal The formula is expressed as follows:

[0119]

[0120] and the second feature Perform element-wise multiplication: , For multiplication features, the output projection matrix is ​​used. and bias matrix Restore dimensions and add residual connections. The operation ratio is 0.1. Gating features:

[0121]

[0122] The spatial gating mechanism and parameter configuration are implemented as follows:

[0123] In step 2.1 use Initialization, in step 2.3 Orthogonal initialization is used. use Uniform initialization ensures training stability.

[0124] Step 3: Extract all features from the multi-scale features except for the highest-level features. Combine this with the gated feature input to the content-guided attention fusion module. Calculate attention weights from three dimensions: channel, space, and pixel, and output the fused features with the same number of input features. In the feature pyramid fusion stage, this invention designs a content-guided attention fusion (CGAF) module to calculate attention weights from three dimensions: channel, space, and pixel, optimizing the multi-scale feature fusion process and enabling the model to pay more attention to small target regions during fusion. Through content-guided attention-based feature fusion optimization, the efficiency of small target feature fusion is improved.

[0125] In a multi-scale feature set, besides the highest-level feature, there are two other features. The feature with the lower level is designated as the first feature (i.e., the feature map). The other is set as the second feature (i.e., the feature map). And set the gating feature as the third feature (i.e., the feature map). ).

[0126] like Figure 7 As shown, the detailed steps of the multi-scale feature fusion process are as follows:

[0127] In the FPN phase: for the third feature The dimensions [B,256,H / 32,W / 32] are upsampled by 2x to [B,256,H / 16,W / 16], with the second feature The dimensions [B, 256, H / 16, W / 16] are fused together using content-guided 3D attention to obtain [B, 512, H / 16, W / 16]. Then, after a CSPREpLayer operation, the dimensions are fused back to [B, 256, H / 16, W / 16] to obtain the final fusion result. ; Fusion results Upsample to size [B, 256, H / 8, W / 8] and combine with the first feature The size [B, 256, H / 8, W / 8] is used to perform a content-guided 3D attention fusion once, resulting in [B, 512, H / 8, W / 8]. This is then fused back to the size [B, 256, H / 8, W / 8] using the CSPREpLayer operation to obtain the first fused feature. This stage transmits high-level semantic information to low-level information, generating semantically rich multi-scale features.

[0128] In the PAN stage: the first fusion feature Downsampling to [B, 256, H / 16, W / 16] using 3×3 with stride=2, and comparing with the second feature The [B,256,H / 16,W / 16] is fused into [B,512,H / 16,W / 16] through content-guided 3D attention, and then fused back into [B,256,H / 16,W / 16] through the CSPRepLayer operation to obtain the second fused feature. Then the second fusion feature Similarly, downsampled to [B,256,H / 32,W / 32], and... The dimensions [B,256,H / 32,W / 32] are fused to [B,512,H / 32,W / 32] through content-guided 3D attention, and then fused back to [B,256,H / 32,W / 32] through CSPREpLayer operation to obtain the third feature. The final output is a three-layer multi-scale feature set. Low-level detailed information is passed to higher levels, enhancing localization capabilities while preserving semantics.

[0129] like Figure 8 As shown, the detailed process of content-guided 3D attention fusion is as follows:

[0130] Channel attention: concatenating two input features Then in the spatial dimension Global average pooling is performed on the top layer to generate channel statistics vectors. Channel statistical vector Through a two-layer bottleneck structure MLP (the middle layer has a dimension of...) ,use (Activation) Performs a nonlinear transformation to generate channel attention weights. The calculation process is described as follows:

[0131]

[0132] in, For global average pooling, , For the weights of a two-layer bottleneck MLP, , The bias is for a two-layer bottleneck structure MLP.

[0133] Spatial attention: concatenate two input features, and then apply max pooling to each channel dimension. ) and average pooling ( ), generate two The spatial feature description map is obtained by concatenating two spatial feature description maps along the channel dimension, and then... Convolutional layers generate spatial attention weights. The calculation process is described as follows:

[0134]

[0135] Pixel Attention: Combining the original features and the preceding attention information, grouped convolution (groups=dim) is used for efficient computation to generate pixel-level fine-grained attention weights. Specifically, the channel attention weights and spatial attention weights are added together to obtain a coarse attention map. The coarse attention map is concatenated with the features obtained by concatenating the two input features, and then the channels are rearranged. ), and then through depthwise separable convolution ( )and Activation functions generate pixel-level attention weights The formula is expressed as follows:

[0136]

[0137]

[0138]

[0139] in, This is an element-wise addition operation.

[0140] Preliminary fusion: Two input features are dynamically fused using pixel-level attention weights to obtain preliminary fused features. ,in This represents the dot product.

[0141] Final fusion: Features are split into two paths of different complexities using CSPReplayer before fusion. The initially fused features are passed through one convolutional layer and three RepVggBlock modules to obtain the first branch features. Simultaneously, the initially fused features are passed through another convolutional layer to obtain the second branch features. The first and second branch features are then concatenated to obtain the output features. The formula is as follows:

[0142]

[0143] in, For the RepVggBlock module, where and Split into two independent convolution.

[0144] Step 4: Generate candidate boxes for each fused feature using a lightweight prediction head, and calculate the uncertainty of each candidate box. Use the first few candidate boxes with the lowest uncertainty as initial reference boxes, and use the feature vectors corresponding to these candidate boxes as initial object queries. After feature fusion, use a query selection mechanism based on the lowest uncertainty to filter out the regions most likely to contain the target from multi-scale features, generating high-quality initial object queries.

[0145] Preliminary prediction of candidate regions: Input from the multi-scale enhanced feature map in step three: , , A lightweight prediction head (containing 1×1 convolutions) is applied to each feature map to initially generate candidate boxes and their class confidence scores. Let the prediction at each location be... If there are 10 candidates, then a total of 100 candidates will be generated. One candidate, .

[0146] Uncertainty assessment: For each candidate box, calculate its uncertainty score. Uncertainty consists of two parts:

[0147] Classification uncertainty: The disorder of the class prediction distribution is calculated using an entropy function, where This indicates that the model predicts this target as a category. The confidence level is set, and the predicted category is defined as... kind:

[0148]

[0149] Location uncertainty: The aspect ratio or coordinate variance of the predicted bounding box is used as the location uncertainty to evaluate the prediction stability of the bounding box. The total uncertainty is a weighted sum, where... and For hyperparameters: .

[0150] Query selection: By Sort all candidates in ascending order, select the Top-M candidates with the least uncertainty (M is the preset number of queries, such as 300), and use their corresponding feature vectors (extracted through RoIAlign or linear projection) as the initial object queries. Meanwhile, these candidate preliminary predicted boxes will also serve as the initial reference boxes for the decoder.

[0151] Step 5: The initial object query and initial reference boxes are used as input to the Transformer-based decoder, and the output of the Transformer-based decoder is passed through a prediction head to obtain the final object detection result. A Transformer-based decoder structure is adopted, using multi-layer self-attention and cross-attention mechanisms to allow the object query to interact with multi-scale features, progressively optimizing the prediction result. The detection head then performs the final classification and bounding box regression based on the decoder output, outputting the detection result.

[0152] Decoder: The decoder consists of six identical decoding layers stacked together. Each layer contains three modules: a self-attention module, which facilitates information exchange between object queries and models the relationships between queries; a cross-attention module, which allows queries to interact with multi-scale feature maps, where the key and value come from the multi-scale features; and a cross-attention module, where queries interact with multi-scale feature maps. Query is the query vector output by the previous layer. Feedforward Network (FFN): performs further feature transformation. In this embodiment, the gMLP Block from step 2 above is used to enhance spatial modeling. Each decoder layer outputs an updated query vector. Based on the preliminary prediction results, the deep decoder iteratively optimizes the shallow output, gradually refining the prediction boxes and categories.

[0153] Detection head: Receives the output query of the last layer decoder. Each query independently uses two branches:

[0154] Classification branches: linear layers and activation functions Output category probability distribution .

[0155] Regression Branches: Linear Layers and Activation Functions Output bounding box offset This refers to the bounding box coordinates. Non-maximum suppression is used to remove redundant detection boxes, and the final detection result is output.

[0156] The Transformer-based decoder directly outputs a fixed number of non-redundant detection results, each containing a class label and bounding box coordinates. The final detection result, Final Output, is obtained. This invention constructs an efficient and accurate method for detecting small-target UAVs, significantly improving small-target detection performance while maintaining real-time performance.

[0157] DUT Anti-UAV is a visible light anti-UAV detection dataset from IEEE TITS 2022. To evaluate the performance of the method presented in this invention, comparative experiments were conducted on this dataset. The training settings included an imgsz resolution of 640x640, and the scaling factor for the method was set accordingly. Candidate formats for prediction at each location Hyperparameters Hyperparameters and the number of candidates with the least uncertainty The comparison models used were YoloV5m, YoloV8m, YoloV9m, YoloV10m, YoloV11m, YoloV12m, and Rtdetr-r18. The experimental setup consisted of an NVIDIA GeForce RTX 4090, Python 3.9, PyTorch 2.8.0, and CUDA 12.8. The training run was 300 epochs, with a batch size of 4. The performance metrics used for comparison were mAP (multi-class average precision), APs (small target precision), APm (medium target precision), Params (number of parameters), and GFLOPs (billion floating-point operations). The final experimental results are shown in Table 2.

[0158] Table 2 Experimental Results

[0159]

[0160] Experimental results show that the method of this invention achieves excellent performance in all core evaluation metrics of target detection tasks, demonstrating outstanding overall performance. In terms of detection accuracy, the AP value of this method for individual categories is significantly higher than that of the comparative models, indicating high detection accuracy for all target categories. The overall mAP index ranks first among all comparative models, fully demonstrating the adaptability and robustness of this method under different IoU threshold requirements. Regarding model efficiency, the parameter count of this method is relatively low compared to other models of the same level, significantly reducing storage overhead and memory consumption. Simultaneously, the GFLOPs per inference are low, significantly reducing computational complexity and providing a solid foundation for edge deployment. In summary, this method achieves an effective balance between detection accuracy and operational efficiency, significantly improving the detection accuracy of small target UAVs while reducing computational resource consumption, making it suitable for anti-UAV applications such as border patrol and protection of key areas.

[0161] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0162] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for detecting low-altitude small targets against unmanned aerial vehicles based on RTDETR, characterized in that, The RTDETR-based method for detecting low-altitude small targets against drones includes: A Pyramid Vision Mamba-based backbone network is used to extract features from the input image, resulting in multi-scale features. This process includes downsampling the input image, followed by sequential processing through two local spatial attention Mamba modules, two spatial reduction attention Mamba modules, an average pooling layer, and a linear layer to obtain multi-scale features. The local spatial attention Mamba modules include downsampling, a visual Mamba state space model, and a local spatial attention module. The spatial reduction attention Mamba modules include downsampling, a visual Mamba state space model, and a spatial reduction self-attention module. The highest-level features in the multi-scale features are processed by a self-attention-based spatial gating unit in a hybrid encoder to obtain gated features. This process includes: the highest-level features in the multi-scale features are processed through self-attention and residual concatenation to obtain self-attention features; the self-attention features are subjected to layer normalization and their dimensions are expanded using an input projection matrix to obtain expanded features; the expanded features are divided into two parts, the first part is processed through layer normalization and then spatially gated to obtain a gated signal, which is then multiplied element-wise with the second part to obtain multiplied features; the multiplied features are used to restore their dimensions using an output projection matrix and then residual concatenated with the input features to obtain gated features. The remaining features from the multi-scale features, excluding the highest-level features, are combined with gated features input to a content-guided attention fusion module. Attention weights are calculated from three dimensions: channel, spatial, and pixel. The output is a fused feature with the same number of input features, including: In the multi-scale features, there are two features other than the highest level feature. The lower-level feature is set as the first feature, and the other is set as the second feature. The gated feature is set as the third feature. The third feature is upsampled, then fused with the second feature using content-guided 3D attention, then fused with the first feature using content-guided 3D attention, and then fused with the second feature using the CSPRepLayer operation to obtain the first fused feature. The first fused feature is downsampled, then fused with the second feature using content-guided 3D attention, and then processed by CSPRepLayer to obtain the second fused feature. The second fusion feature is downsampled, then fused with the third feature using content-guided 3D attention, and then processed by CSPRepLayer to obtain the third fusion feature. The content-guided 3D attention fusion process is as follows: Channel attention: Two input features are concatenated and then global average pooling is performed in the spatial dimension to generate channel statistical vectors. The channel statistical vectors are then passed through a two-layer bottleneck MLP to generate channel attention weights. Spatial attention: Concatenate two input features, then apply max pooling and average pooling to the channel dimension respectively to generate two spatial feature maps. Concatenate the two spatial feature maps along the channel dimension and then pass them through a... Convolutional layers generate spatial attention weights; Pixel attention: The channel attention weights and spatial attention weights are added to obtain a coarse attention map. The coarse attention map is then concatenated with the features obtained from the two input features and rearranged by channels. Finally, it is processed by depthwise separable convolution and activation function to generate pixel-level attention weights. Two input features are dynamically fused using pixel-level attention weights to obtain preliminary fused features; The initial fused features are passed through a convolutional layer and three RepVggBlock modules to obtain the first branch features. At the same time, the initial fused features are passed through a convolutional layer to obtain the second branch features. The first branch features and the second branch features are concatenated to obtain the output features. A lightweight prediction head is used to generate candidate boxes for each fused feature, and the uncertainty of each candidate box is calculated. The first few candidate boxes with the smallest uncertainty are used as initial reference boxes, and the feature vectors corresponding to the first few candidate boxes with the smallest uncertainty are used as initial object queries. The initial object query and initial reference box are used as input to the Transformer-based decoder, and the output of the Transformer-based decoder is passed through a prediction head to obtain the final object detection result.

2. The method for detecting low-altitude small targets against unmanned aerial vehicles based on RTDETR according to claim 1, characterized in that, The processing procedure for the visual Mamba state-space model is as follows: The input features are doubled in dimension by linear projection and immediately split into two parallel branches. The input features of the branch are passed through a linear layer, a convolutional layer, and an activation function layer to obtain the output of the first branch; The input features of the branch are passed through a linear layer, a convolutional layer, an activation function layer, and a state space model to obtain the output of the second branch; The outputs of the first and second branches are residually concatenated and then linearly projected back to the original input dimension to obtain the output features.

3. The method for detecting low-altitude small targets against unmanned aerial vehicles based on RTDETR according to claim 1, characterized in that, The processing procedure of the local spatial attention module is as follows: The input features are flattened and then subjected to depthwise convolution to extract spatial features; Attention weights and value projections are generated based on the extracted spatial features, and spatial attention is calculated to obtain attention features. The attention features and the flattened input features are joined by residual connections to obtain residual features; The residual features are processed using a linear layer and then concatenated with the residual features to obtain the output features.

4. The method for detecting low-altitude small targets against unmanned aerial vehicles based on RTDETR according to claim 1, characterized in that, The processing procedure of the space reduction self-attention module is as follows: The input features are passed through a normalization layer to obtain normalized features, and query vectors, key vectors, and value vectors are generated based on the normalized features. Input the key vector and value vector into the space reduction module, perform space reduction operations respectively, and obtain the reduced key vector and the reduced value vector; Multi-head self-attention is calculated based on the query vector, the reduced key vector, and the reduced value vector, and attention features are output. The attention features and normalized features are joined by residuals to obtain residual features; The residual features are processed using a linear layer and then concatenated with the residual features to obtain the output features.

5. The method for detecting low-altitude small targets against unmanned aerial vehicles based on RTDETR according to claim 1, characterized in that, The calculation of the uncertainty for each candidate box includes: Classification uncertainty: The disorder of the predicted distribution of categories is calculated using the entropy function, and is used as classification uncertainty; Location uncertainty: The aspect ratio or coordinate variance of the predicted bounding box is taken as the location uncertainty; Uncertainty of candidate boxes: The weighted fusion result of their classification uncertainty and localization uncertainty is used as the uncertainty of candidate boxes.

Images