Unmanned aerial vehicle aerial small target detection method based on joint recalibration and query allocation

Abstract

The application discloses a UAV aerial small target detection method based on joint re-calibration and query distribution, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring a preprocessed UAV aerial dataset; building a UAV aerial target detection model; the UAV aerial target detection model comprises a backbone network composed of multi-stage space-frequency dual-domain modules, a neck network composed of a joint space-channel re-calibration module, and a head network receiving the query number output by a density perception query distribution module; the preprocessed UAV aerial dataset is used to train the UAV aerial target detection model, so that a trained detection model is obtained; the trained detection model is used to perform target detection on a to-be-processed UAV aerial image, so that a detection result is obtained and the performance of the model is evaluated. The application further discloses an electronic device and a storage medium for implementing the above method.

Description

Technical Field

[0001] This invention relates to a method for detecting small targets in UAV aerial photography, specifically a method for detecting small targets in UAV aerial photography based on joint recalibration and query allocation, belonging to the field of computer vision technology. Background Technology

[0002] With the rapid development of technologies such as drones, intelligent monitoring, remote sensing imaging, and automated inspection, small target detection has been widely applied in scenarios such as traffic monitoring, disaster relief, agricultural inspection, security monitoring, urban management, and environmental perception. Compared with conventional target detection tasks, small target detection in complex environments is more challenging. Targets in such scenarios typically occupy a small proportion of pixels in the image, with limited discernible information such as edges, textures, and shapes. They are also easily affected by factors such as complex backgrounds, lighting variations, occlusion, dense target distribution, scale differences, and viewpoint changes, resulting in weak target features, difficulty in localization, and a high rate of false positives and false negatives.

[0003] While existing UAV target detection technologies have made some progress in scenarios involving multi-scale targets, complex environments, and low light conditions, they still have significant shortcomings in practical applications, as detailed below:

[0004] First, most existing multi-scale feature fusion methods employ feature concatenation, weighted fusion, or convolutional fusion to integrate feature maps of different resolutions. While these methods can utilize the spatial details of shallow features and the semantic information of deep features, they often lack effective handling of the differences between features at different levels. Simple fusion can easily lead to problems such as insufficient feature alignment, transmission of background interference, and dilution of information about small targets.

[0005] Furthermore, in scenarios with complex backgrounds, low signal-to-noise ratios, or degraded image quality, UAV images are prone to issues such as decreased contrast, blurred edges, and degraded texture details. Existing detection methods mostly rely on spatial domain convolution or spatial attention mechanisms for feature enhancement, which underutilizes edge, texture, and structural information in the frequency domain, making it difficult to effectively compensate for the feature degradation of small targets caused by low signal-to-noise ratios. Therefore, under complex backgrounds and low signal-to-noise ratio conditions, target features are easily overwhelmed by background information, leading to missed detections and false detections.

[0006] Finally, existing DETR-type object detection methods typically use a fixed number of object queries for prediction, which is difficult to adapt to changes in the number and distribution density of targets in UAV images. In dense target scenes, insufficient query counts can lead to inadequate candidate target coverage, resulting in missed detections; in sparse target scenes, excessive queries can lead to redundant computation and invalid predictions, affecting detection efficiency and model stability. Summary of the Invention

[0007] To address the technical challenge of simultaneously achieving high-quality multi-scale feature fusion, robustness in complex environments, and real-time detection efficiency in existing technologies, this invention provides a UAV aerial image small target detection method based on joint recalibration and query allocation. This method introduces a spatial-frequency dual-domain enhancement module, a joint spatial-channel recalibration module, and a density-aware query allocation module in the backbone network, multi-scale feature fusion stage, and decoder query stage, respectively. The aim is to improve the detection accuracy and robustness of UAV aerial images in complex scenarios such as cluttered backgrounds, drastic target scale changes, dense target distribution, and indistinct small target features, while simultaneously increasing detection efficiency to meet the application requirements of real-time target detection on UAV platforms.

[0008] To achieve the above objectives, this invention provides a method for small target detection in UAV aerial photography based on joint recalibration and query allocation, comprising the following steps:

[0009] Step 1: Obtain the preprocessed drone aerial photography dataset;

[0010] Step 2: Build a drone aerial target detection model;

[0011] Step 3: Train the drone aerial target detection model using the preprocessed drone aerial photography dataset to obtain the trained detection model;

[0012] Step 4: Use the trained detection model to perform target detection on the drone aerial images to be processed, obtain the detection results, and evaluate the performance of the trained detection model.

[0013] Further, step 2 includes:

[0014] The input data is fed into the backbone network and processed by introducing a multi-stage space-frequency dual-domain module to obtain multi-scale features;

[0015] Multi-scale features are input into the neck network, and multi-scale fused features are obtained by introducing a joint spatial-channel recalibration module in the multi-scale feature fusion stage.

[0016] The high-resolution features from the multi-scale fusion features are input into the density-aware query allocation module, and the query allocation strategy is dynamically adjusted according to the target distribution density to obtain the final number of allocated queries.

[0017] The number of queries and the enhanced features obtained by feature concatenation of multi-scale fusion features are input into the head network. The decoder and the detector head predict the target category and target bounding box based on the query vector.

[0018] Furthermore, the process of the space-frequency dual-domain module processing features includes:

[0019] First, the input features are convolved to obtain intermediate features;

[0020] Subsequently, the intermediate features are divided into channels to obtain retained branch features and enhanced branch features;

[0021] Next, the enhanced branch features are input into multiple cascaded cross-domain dual attention blocks for processing to obtain the processed enhanced branch features;

[0022] Finally, the preserved branch features are concatenated with the processed enhanced branch features, and the output features are obtained by convolutional fusion.

[0023] Furthermore, the process of processing cross-domain dual attention block features includes:

[0024] First, after batch normalization and depthwise convolution processing of the input features, the residuals are added to the input features to obtain the local enhancement features;

[0025] Subsequently, the local enhancement features are divided into channels according to the channel division ratio to obtain enhanced branch features and preserved branch features;

[0026] Next, the enhanced branch features after layer normalization are input into the space frequency channel attention module for processing to obtain the processed enhanced branch features;

[0027] Finally, the enhanced branch features and the preserved branch features are concatenated and processed through a feedforward network to obtain the output features.

[0028] Furthermore, the process of the spatial frequency channel attention module processing features includes:

[0029] First, the input features after pointwise convolution are divided into spatial branch features and frequency branch features along the channel dimension;

[0030] Then, directional spatial enhancement is performed on the spatial branch features by using 3×3, 1×5 and 5×1 depth convolutions respectively, and the features obtained after feature concatenation are fused by pointwise convolution to obtain spatially enhanced features;

[0031] Next, the frequency branch features are subjected to a two-dimensional real-valued fast Fourier transform to obtain the complex frequency response; the real and imaginary parts are concatenated along the channel dimension, and the frequency response is locally jointly modulated by a 3×3 depth convolution to obtain the modulated frequency features; the modulated frequency features are then processed using a two-dimensional real-valued inverse fast Fourier transform to obtain the frequency reconstruction features.

[0032] Subsequently, the frequency reconstruction features are input into the frequency reliability gating unit, and frequency reliability weights are generated by global average pooling, 1×1 convolution and sigmoid activation function; the frequency reconstruction features are then multiplied element-wise using the frequency reliability weights to obtain the frequency enhancement features.

[0033] Finally, the spatial enhancement features and frequency enhancement features are concatenated to obtain the spatial-frequency fusion features. The spatial-frequency fusion features are then input into an efficient channel attention unit for lightweight channel recalibration, and the recalibrated spatial-frequency fusion features are output-mapped using pointwise convolution to obtain the output features.

[0034] Furthermore, the process of the joint space-channel recalibration module processing features includes:

[0035] First, the first input feature and the second input feature are respectively input into the bidirectional cross-channel relationship unit for processing to obtain the first channel relationship description vector and the second channel relationship description vector;

[0036] Then, the cross-channel relationship description vectors after vector transformation are subjected to outer product and normalization to obtain the cross-channel relationship matrix;

[0037] Next, the cross-channel relationship matrix and its transpose are used to perform bidirectional channel relationship calibration on the first and second input features, respectively, to obtain the first channel relationship calibration features and the second channel relationship calibration features; this process is expressed as:

[0038] ,

[0039] ,

[0040] in, and These represent the calibration features of the first channel relationship and the calibration features of the second channel relationship, respectively. and These respectively represent the features derived from the first input. Second input features The feature matrix obtained by flattening along the spatial dimension; This represents the cross-channel relationship matrix constructed jointly by the two channel relationship description vectors. The transpose of the matrix is ​​used for the reverse channel relationship calibration of the second feature; This indicates a size recovery operation, used to restore a C×HW matrix to a spatial feature map of H×W×C, so that it can be used for element-wise operations with subsequent spatially significant calibration weights.

[0041] Subsequently, the first channel relationship calibration feature and the second channel relationship calibration feature are respectively input into the spatial saliency calibration unit for processing to obtain the first spatial saliency calibration map and the second spatial saliency calibration map; the first spatial saliency calibration map and the second spatial saliency calibration map are respectively subjected to Softmax normalization processing to obtain the first spatial saliency calibration weight and the second spatial saliency calibration weight; the first spatial saliency calibration weight and the second spatial saliency calibration weight are respectively multiplied element-wise with the first channel relationship calibration feature and the second channel relationship calibration feature, and then added element-wise with the first input feature and the second input feature through residual connection to obtain the first spatial-channel joint recalibration feature and the second spatial-channel joint recalibration feature;

[0042] Finally, the first spatial-channel joint recalibration features and the second spatial-channel joint recalibration features are input into the weighted fusion unit for weighted fusion to obtain the output features.

[0043] Furthermore, the process of processing features by the bidirectional cross-channel relationship unit includes:

[0044] The input features are processed sequentially using average pooling, 1×1 convolution, and ReLU activation function to obtain the average pooling branch result;

[0045] The input features are processed sequentially using max pooling, 1×1 convolution, and ReLU activation function to obtain the max pooling branch result;

[0046] The average pooling branch result and the max pooling branch result are added together to obtain the channel relationship description vector;

[0047] The process of the spatially significant calibration unit processing features includes:

[0048] The channel relationship calibration features are processed by average pooling and max pooling respectively to obtain the channel average spatial description map and the channel maximum spatial description map.

[0049] The channel average spatial description map and the channel maximum spatial description map are concatenated along the channel dimension, and a spatially significant calibration map is generated through multi-layer 3×3 convolution.

[0050] Furthermore, the process of the density-aware query allocation module processing features includes:

[0051] First, the input features are rearranged spatially using pixel rearrangement to obtain spatially rearranged features.

[0052] Secondly, global average pooling and global max pooling are performed on the spatial rearrangement features respectively to obtain global average pooling features and global max pooling features;

[0053] Then, the global average pooling feature and the global max pooling feature are input into the lightweight activation unit for processing, and are added to the global average pooling feature and the global max pooling feature respectively through residual connection to obtain the residual enhancement features of the average pooling branch and the max pooling branch.

[0054] Next, the residual enhancement features of the average pooling branch and the max pooling branch are concatenated along the channel dimension, and cross-branch fusion is performed using 1×1 convolution to obtain the aggregated density features.

[0055] Finally, the aggregated density features are input into the feedforward network for processing, and the probability distribution of different target density categories is generated through Softmax normalization. The corresponding query quantity is then selected from the preset query quantity set based on the density category with the highest probability.

[0056] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0057] The present invention also provides a storage medium storing a computer program or instructions that, when the computer program or instructions are run on a computer, execute the steps of the method described.

[0058] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0059] 1. To address the issues of insufficient feature representation and low detection accuracy of small targets in UAV aerial photography against complex backgrounds, this invention designs a space-frequency dual-domain module. This module enhances the features of small targets in UAV aerial images through collaborative modeling in the spatial and frequency domains. The outer layer employs a splitting and aggregation structure, sending only a portion of the channels to a continuous cross-domain dual attention block for enhancement, while retaining some original features to maintain spatial positioning information. Inside the cross-domain dual attention block, the space-frequency channel attention module further employs parallel processing of a directional spatial enhancement branch and a frequency reconstruction branch. The directional spatial enhancement branch extracts local texture, horizontal contour, and vertical contour information, while the frequency reconstruction branch recovers the target-related frequency domain detail response through 2D real-valued fast Fourier transform, joint real-imaginary part convolution modulation, and 2D real-valued inverse fast Fourier transform. To avoid false enhancement of high-frequency background features such as road textures, shadow boundaries, and building edges, the space-frequency channel attention module introduces a frequency reliability gating unit after frequency reconstruction to screen frequency features with confidence levels before fusing them with spatial branch features. Adaptive modulation of the space-frequency features is then achieved through lightweight channel recalibration. Therefore, this invention can enhance the edge, orientation structure and frequency details of small targets while reducing computational redundancy, suppress unreliable frequency responses in complex backgrounds, and improve the detection robustness of the model in complex UAV scenarios.

[0060] 2. To address the issues of insufficient multi-scale feature fusion and the easy interference of shallow background noise with small target localization, this invention designs a joint spatial-channel recalibration module for the collaborative calibration and fusion of shallow detail features and deep semantic features. This module does not simply recalibrate using channel weights, but instead constructs a cross-feature channel relationship matrix through the channel description vectors of the two scale-aligned features, creating an explicit mutual guidance relationship between the shallow detail channels and the deep semantic channels; furthermore, it combines spatial attention to perform residual enhancement on salient regions. Existing multi-scale feature fusion methods often employ simple concatenation, addition, or convolutional fusion, which easily leads to background noise propagation, weakening of small target location information, and dilution of semantic information. This module, through joint recalibration of the spatial and channel dimensions, adaptively enhances target-related channels and key spatial regions in the fused features, while suppressing complex background textures and redundant noise interference. This allows shallow high-resolution features to provide more refined edge and location information, while deep semantic features provide more stable category semantic constraints. Therefore, the model can alleviate the problems of feature misalignment, background noise accumulation, and weakened small target response during multi-scale fusion, improving the localization accuracy of small targets and the stability of dense target detection.

[0061] 3. Addressing the issue that DETR-type models with a fixed number of queries struggle to adapt to changes in target density, this invention introduces a density-aware query allocation module. This module adaptively determines the required number of detection queries for the current image based on the target distribution density, feature response intensity, and candidate target response in the image. Traditional RT-DETR uses a fixed number of object queries, which can easily lead to missed detections in dense target scenes due to insufficient candidate coverage, and can easily result in redundant computation and invalid predictions in sparse target scenes. The density-aware query allocation module estimates the target density of the entire image through a density-aware mechanism and selects the corresponding number of queries based on the density category. This increases the candidate query capacity in dense scenes and reduces invalid queries in sparse scenes, thereby improving the recall capability for small targets, occluded targets, and dense targets. Simultaneously, it reduces redundant computation and false detection risks, improving the model's detection efficiency, prediction stability, and adaptability to complex scenes. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0063] Figure 2 This is a schematic diagram of the structure of the UAV aerial target detection model of the present invention;

[0064] Figure 3 The following are schematic diagrams of the spatial-frequency dual-domain module of the present invention: (a) is a schematic diagram of the framework of the spatial-frequency dual-domain module; (b) is a schematic diagram of the cross-domain dual attention block; and (c) is a schematic diagram of the spatial-frequency channel attention module.

[0065] Figure 4 This is a schematic diagram of the structure of the joint space-channel recalibration module of the present invention;

[0066] Figure 5 This is a schematic diagram of the density-aware query allocation module of the present invention;

[0067] Figure 6 This is a diagram illustrating the target detection effect of an unmanned aerial vehicle (UAV) in accordance with the present invention.

[0068] Figure 7 The image shows a comparison of the target detection results between the UAV aerial target detection model of this invention and the RT-DETR baseline model. (a) shows the detection results of the RT-DETR baseline model, and (b) shows the detection results of the model of this invention. Detailed Implementation

[0069] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0070] Example 1:

[0071] like Figure 1 As shown, this embodiment of the invention provides a method for detecting small targets in UAV aerial photography based on joint recalibration and query allocation, specifically including the following steps:

[0072] Step 1: Obtain drone aerial image data from the publicly available VisDrone2019-DET dataset, which includes drone aerial images with different flight altitudes, shooting angles, target scales, and complex background scenes.

[0073] The VisDrone2019 dataset was constructed by the AISKYEYE team at the Machine Learning and Data Mining Laboratory of Tianjin University. For related information, please refer to the publication "VisDrone-DET2019: The Vision Meets Drone ObjectDetection in Image Challenge Results" (ICCV Workshops 2019) by Du et al. Each image corresponds to object category annotation information and object location annotation information, with the object location annotation information including the object bounding box coordinates.

[0074] The acquired public datasets are organized and preprocessed, including image filtering, annotation file parsing, annotation format conversion, outlier removal, and image size normalization.

[0075] Specifically, the image size was uniformly adjusted to 640×640 pixels, and the target category and bounding box information were converted into a unified annotation format suitable for training object detection networks. The processed dataset was stored in a folder named Datasets, which contained two subfolders: Images and Labels. The Images folder was used to store the drone aerial image data, and the Labels folder was used to store the target category and bounding box annotation files corresponding to each image.

[0076] Furthermore, the processed dataset is divided into a training set, a test set, and a validation set according to a preset ratio of 7:2:1. The training set is used to train the improved UAV aerial target detection network model; the test set is used to test the model's target detection accuracy, robustness, and generalization ability after training; and the validation set is used to verify the model's performance, adjust hyperparameters, and save the optimal weight file during training.

[0077] Step 2, build a drone aerial target detection model, such as Figure 2 As shown.

[0078] This invention, based on the RT-DETR target detection framework, addresses the challenges of small target scale, strong background interference, dense target distribution, and detail attenuation caused by deep downsampling in UAV aerial images. It improves the feature extraction backbone, feature fusion neck, and query allocation stages. The overall network consists of three parts: a backbone network, a neck network, and a head network. The backbone network extracts features from aerial images at different scales, the neck network performs hierarchical bidirectional cross-scale feature fusion, and the head network predicts the target category and target bounding box based on the query vector.

[0079] Specifically, the input drone aerial images are first processed through multiple 3×3 convolutions for preliminary feature extraction and spatial downsampling. Spatial-Frequency Dual-domain Modules (SFDMs) are introduced at multiple stages of the backbone network. By embedding SFDMs, the network can simultaneously enhance directional spatial texture details and frequency domain structural responses during the backbone feature extraction stage. Specifically, the directional spatial branch in SFCA preserves the edges and directional contours of small targets in local regions, while the frequency branch supplements the detailed responses of small targets in the spectrum. A frequency reliability gating unit further suppresses unreliable frequency responses caused by road textures, shadows, and building edges. The backbone network outputs multi-scale features B2, B3, B4, and B5. B2, B3, and B4 are channel-aligned via 1×1 convolutions before being input into the neck network. B5 undergoes high-level semantic interaction via 1×1 convolutions and an attention-based intra-scale feature interaction module (AIFI) before participating in subsequent fusion.

[0080] In the neck network portion, this invention employs a hierarchical bidirectional cross fusion structure to bidirectionally transfer and fuse features at different scales. This structure introduces a joint spatial-channel recalibration (JSCR) module, which adaptively adjusts the importance of features at different scales in the spatial and channel dimensions, enabling full interaction between shallow detail information and deep semantic information. JSCR is deployed in both the top-down semantic guidance path and the bottom-up detail compensation path: in the upsampling path, high-level semantic features are used to calibrate low-level spatial features, resulting in S2, S3, and S4; in the downsampling path, low-level detail information continues to be fed back to high-level features, resulting in P3, P4, and P5.

[0081] Subsequently, the P3-scale fusion features enhanced by JSCR are input into the Density-Aware Query Allocation (DAQA) module to adjust the query allocation strategy based on the target density in the UAV aerial photography scene. Finally, the Cross-Union Comparison (CICC) aware query selection filter selects candidate queries with high location quality from the encoded features; Nq represents the final number of queries determined by the density-aware query allocation module; the enhanced features are fed into the decoder and detector head, outputting target category, target bounding box, and confidence information.

[0082] Specifically, the final query count Nq output by DAQA is directly used as the Top-K selection number in the Intersection over Union (IoU) perceptual query selection stage: first, candidate positions are generated from the encoded multi-scale fusion features, and the classification and localization quality scores of the candidates are calculated. Then, the top Nq high-quality candidates are selected from high to low quality scores, and the corresponding candidate features are used as the initial content queries for the decoder, and the corresponding candidate bounding boxes are used as reference boxes. The decoder follows the Transformer decoding structure of RT-DETR. Each decoding layer updates the query representation through inter-query self-attention, multi-scale feature cross-attention, and feedforward network, and refines the reference boxes layer by layer. The detection head consists of a classification branch and a bounding box regression branch, which output the target category confidence and normalized bounding box coordinates, respectively. The decoding result of the last layer is used as the detection output of the inference stage. The above query selection, decoder, and detection head belong to the existing structure of RT-DETR. The improvement of this invention lies in using DAQA to dynamically determine the number of queries Nq participating in the decoding of each image based on the target density.

[0083] The specific process of constructing each module is as follows:

[0084] 2.1 The process of constructing the space-frequency dual-domain module is as follows:

[0085] In UAV aerial target detection tasks, small targets such as vehicles and pedestrians typically occupy only a small number of pixels in the image and are easily affected by changes in flight altitude, shooting angle, shadow occlusion, road texture, and complex backgrounds. The original RT-DETR backbone network tends to weaken the edge, contour, and texture information of small targets during layer-by-layer downsampling, reducing the separability between target and background regions in deep features. To address these issues, this invention uses the Space-Frequency Dual-Domain (SFDM) module as the feature enhancement unit of the backbone network to replace the conventional feature extraction module in the original RT-DETR backbone network. The module structure is as follows: Figure 3 As shown.

[0086] SFDM consists of three parts: (1) an outer-layer splitting and aggregation structure, used to maintain the stable transmission of backbone features and reduce computational redundancy; (2) a cross-domain attention block (CDAB), used to perform spatial and frequency domain collaborative enhancement on some channels; and (3) a spatial-frequency channel attention module (SFCA), used to complete spatial-frequency fusion through directional spatial enhancement branches, frequency reconstruction branches, frequency reliability gating units, and lightweight channel recalibration units. This design can enhance the local edges, horizontal and vertical contours, and frequency domain detail response of small targets in UAV aerial photography with a relatively small computational cost, and suppress unreliable frequency components in complex backgrounds.

[0087] Figure 3 The abbreviations in the text are as follows: BN stands for Batch Normalization, DWConv stands for Depth-Wise Convolution, LayerNorm stands for Layer Normalization, PWConv stands for Point-Wise Convolution, FFN stands for Feed-Forward Network, rFFT2 and irFFT2 stand for 2D Real-Valued Fast Fourier Transform and 2D Real-Valued Inverse Fast Fourier Transform, respectively, FR Gate stands for Frequency Reliability Gate, and ECA stands for Efficient Channel Attention.

[0088] In the overall architecture, SFDM is embedded in multiple stages of the Backbone, and together with downsampling convolutions at each level, it generates multi-scale features of B2, B3, B4 and B5, which are then input into the hierarchical bidirectional cross-fusion structure of the subsequent Neck.

[0089] In detail, given the input feature map: ,

[0090] in, This represents the feature map of the input SFDM; Indicates the number of channels; and These represent the height and width of the feature map, respectively.

[0091] First, the input features are processed through convolution to obtain intermediate features: ,

[0092] in, This represents the intermediate features after convolution mapping; This indicates a convolution operation used for local feature extraction and channel mapping.

[0093] Subsequently, Dividing along the channel dimension yields preserved branch features and enhanced branch features:

[0094] ,

[0095] in, This indicates that branch features are preserved to maintain the original spatial details and gradient propagation; This indicates the enhanced branch feature, which is used as input for subsequent CDAB to perform spatial-frequency dual-domain feature enhancement; This indicates a channel partitioning operation.

[0096] Enhanced branches are A CDAB cascade is formed, making , No. The output of each CDAB is represented as follows:

[0097] ,

[0098] in, Indicates after the first Enhanced branch features after CDAB processing; Indicates the first A cross-domain dual attention block; This indicates the number of cascaded CDABs. In this embodiment, N is 2, meaning the enhancement branch passes through two CDABs sequentially.

[0099] Finally, the branches and processes will be preserved. The CDAB-enhanced features are concatenated and fused using convolution to obtain the SFDM output features: ,

[0100] in, Indicates the output characteristics of SFDM; This indicates splicing along the channel dimension; This indicates a convolutional fusion operation, used to integrate information from the preserved branch and the enhanced branch.

[0101] The goal of the outer-layer splitting and aggregation stage is to focus on modeling the enhancement branches while maintaining stable transmission of the backbone features. This structure only sends a portion of the channels into continuous CDAB calculations, avoiding the redundant overhead caused by all channels participating in complex space-frequency operations. At the same time, the retained branches can directly transmit shallow edge and position information, while the enhancement branches are responsible for supplementing directional spatial cues and frequency domain structural cues, thereby improving the stability of small target feature representation.

[0102] Furthermore, such as Figure 3 As shown in (b), CDAB first normalizes the input features and performs depthwise convolution to enhance the local spatial context and reduce the computational cost of ordinary convolution. Let the features input to CDAB be... The local enhancement results are as follows: ,

[0103] in, This represents the local enhancement features after batch normalization, depthwise convolution, and residual connections; This indicates a batch normalization operation; This represents a depthwise convolution operation, used to extract local spatial information with a low number of parameters.

[0104] Subsequently, CDAB followed the proportions. right Channel partitioning is performed to obtain the features that require space-frequency enhancement and the features that are directly retained: ,

[0105] in, Indicates the percentage Enhanced branching features; Indicates the percentage Preservation of branch features; This indicates the channel allocation ratio. In this embodiment, the channel allocation ratio is 0.5, meaning that the enhanced branch and the reserved branch each occupy 50% of the channels. It is used to control the number of channels entering SFCA.

[0106] After that, Layer normalization is performed, and SFCA is input for spatial frequency channel attention enhancement:

[0107] ,

[0108] in, This represents the branching characteristics after SFCA enhancement; Presentation layer normalization operation; This indicates the attention module for the space frequency channel.

[0109] Finally, the enhanced branch and the retained branch are concatenated, and the CDAB output features are obtained through a feedforward network:

[0110] ,

[0111] in, Indicates the output characteristics of CDAB; This represents a feedforward network used to further perform nonlinear feature transformation and channel fusion.

[0112] CDAB employs a design that combines partial channel enhancement with partial channel retention, enabling the model to capture information in both the spatial and frequency domains while controlling computational complexity. For small targets captured by UAVs, the enhancement branch strengthens directional edges, local textures, and frequency domain structural cues through SFCA, while the retention branch maintains the original spatial positioning information. After the two are re-fused in the feedforward network, the backbone network's ability to distinguish between weak targets, dense targets, and similar texture interference in complex backgrounds can be improved.

[0113] Furthermore, such as Figure 3 As shown in (c), SFCA is used to perform directional spatial enhancement and frequency reliability modulation on a portion of the channel entering the enhancement branch. Let the characteristics of the input SFCA be... First, channel mapping is completed through pointwise convolution, and then the input is divided into spatial branch input and frequency branch input along the channel dimension:

[0114] ,

[0115] ,

[0116] in, express Pointwise convolution is used to complete channel mapping and cross-channel information fusion; Represents spatial branch input features; This represents the input characteristics of the frequency branch.

[0117] In spatial branching, to simultaneously extract local edges and horizontal and vertical contours, the following methods are employed: , and Three depthwise convolution pairs Perform directional space augmentation:

[0118] ,

[0119] ,

[0120] ,

[0121] in, These represent depthwise convolutions using 3×3, 1×5, and 5×1 kernels, respectively. This represents the spatial branch input features; Represents local spatial texture features; Indicates horizontal enhancement features; This indicates a vertically enhanced feature.

[0122] Subsequently, the three spatial features are concatenated and fused using pointwise convolution to obtain spatially enhanced features:

[0123] ,

[0124] in, Represents spatially enhanced features; This represents the pointwise convolutional fusion operation in the spatial branch. Through the aforementioned spatial branching, SFCA can supplement the directional contours and local boundary information of small targets without significantly increasing computational cost, thus reducing the complexity of single-target computations. Convolution is insufficient for representing the edges of long, narrow vehicles, road orientation structures, and fine textures of small targets.

[0125] In the frequency branch, The complex frequency response is obtained by mapping it to the frequency domain using a two-dimensional real-valued fast Fourier transform:

[0126] ,

[0127] in, This represents a two-dimensional real-valued fast Fourier transform performed along the two spatial dimensions of the feature map: height and width. and These represent the real and imaginary parts of the frequency response, respectively. The imaginary unit is represented. Both the real and imaginary parts characterize frequency amplitude and phase-related information; therefore, this invention retains both for subsequent modulation.

[0128] Then, the real and imaginary parts are spliced ​​along the channel dimension, and then... Deep convolution performs local joint modulation of the frequency response: ,

[0129] in, This represents the frequency characteristics after joint convolution modulation of the real and imaginary parts. To input a two-dimensional real-valued inverse fast Fourier transform, [the following is used:] The modulated real and imaginary parts are redefined and combined to form a new complex frequency response:

[0130] ,

[0131] ,

[0132] in, and These represent the real and imaginary parts after modulation, respectively; This represents the reconstructed complex frequency response. Then, it is restored to the spatial domain using a two-dimensional real-valued inverse fast Fourier transform, yielding the reconstructed frequency characteristics:

[0133] ,

[0134] in, This represents the two-dimensional real-valued inverse fast Fourier transform; This represents the spatial features reconstructed from the frequency domain. This frequency branch can supplement texture abrupt changes, edge variations, and structural details that are difficult to model explicitly using ordinary spatial convolution.

[0135] Considering that road markings, building edges, tree textures, and shadow boundaries in UAV aerial images may also generate strong frequency responses, directly incorporating frequency reconstruction features into the fusion process could easily lead to false enhancement of high-frequency background elements. Therefore, this invention incorporates a frequency reliability gating unit (FR Gate) in the frequency branch to control frequency response. Perform reliability screening:

[0136] ,

[0137] ,

[0138] in, Indicates global average pooling; express Convolution mapping; This represents the Sigmoid activation function; Indicates frequency reliability weight; This represents element-wise multiplication; This indicates the frequency enhancement characteristic after reliability gating.

[0139] The frequency reliability gating unit can adaptively control the contribution of the frequency branch based on the global statistical response of the frequency reconstruction characteristics, retain frequency domain information when the target-related frequency response is obvious, and suppress unreliable frequency response when the background texture or noise frequency is strong.

[0140] After completing spatial branch enhancement and frequency branch gating, the spatial enhancement features are... Frequency enhancement characteristics after gating By splicing the data, the spatial-frequency fusion features are obtained: ,

[0141] To further reduce channel redundancy and enhance effective space-frequency channels, this invention employs an efficient channel attention (ECA) lightweight channel recalibration unit to perform channel attention modulation on Yc:

[0142] ,

[0143] ,

[0144] in, This represents a one-dimensional convolution performed along the channel dimension, used to model local channel dependencies without introducing a dimension-reducing fully connected layer; Indicates the channel weights generated by ECA; This represents the space-frequency fusion feature after lightweight channel recalibration. Finally, the output mapping is completed through pointwise convolution to obtain the SFCA output feature:

[0145] ,

[0146] in, Indicates the output terminal Pointwise convolution; This represents the SFCA output feature. This output maintains the same spatial dimensions as the input features and can be further fed into the feature concatenation structure and feed-forward network (FFN) in CDAB, thus maintaining consistency with the outer split-aggregation structure of SFDM and subsequent backbone multi-scale output processes.

[0147] In summary, SFDM preserves the original feature transfer through the outer-layer split-aggregation structure, achieves low-overhead spatial-frequency enhancement of some channels through CDAB, and realizes directional spatial enhancement, frequency reconstruction, frequency reliability gating, and lightweight channel recalibration through SFCA. After being embedded in multiple stages of the backbone network, this module can continuously enhance the edge, directional contour, and frequency domain texture response of small targets before and after continuous downsampling, while suppressing unreliable high-frequency information in complex backgrounds. This makes the output B2, B3, B4, and B5 features have stronger target separability and spatial localization stability when entering subsequent JSCR cross-scale fusion, thereby improving the robustness of small target detection in UAV aerial images.

[0148] 2.2 The process of constructing the joint space-channel recalibration module is as follows:

[0149] In UAV aerial target detection tasks, shallow features typically contain rich edge, texture, and positional information, which is beneficial for locating small targets; deep features have strong semantic expressive power, which is helpful for distinguishing targets from complex backgrounds. However, traditional feature pyramids, when performing cross-scale fusion, often directly use upsampling, downsampling, concatenation, or element-wise addition to complete information transfer, which can easily cause background noise to propagate between different scales and may also dilute the weak texture and boundary information of small targets by deep semantic features. This invention introduces a Joint Spatial-Channel Recalibration (JSCR) module into the hierarchical bidirectional cross-fusion structure of Neck to perform collaborative calibration on two feature paths that have already been scale-aligned at different scales. The module structure is as follows: Figure 4 As shown.

[0150] The JSCR module is used for bidirectional recalibration of the two input features. This module includes a Bidirectional Cross-Channel Relation (BCCR) unit, a Spatial Saliency Calibration (SSCM) unit, and a Weighted Fusion unit. The BCCR extracts the channel description vectors of the two features and constructs a cross-feature channel relationship matrix, enabling shallow detail features and deep semantic features to establish a mutually guiding relationship along the channel dimension. Subsequently, the SSCM generates a spatial location weight map to enhance salient regions of the target and suppress complex background interference regions. The Weighted Fusion unit adaptively outputs a single H×W×C fused feature based on the contributions of the two recalibrated features.

[0151] Therefore, JSCR can avoid indiscriminate fusion caused by simple splicing or addition, and complete feature recalibration in the order of BCCR bidirectional cross-channel relationship modeling, SSCM spatial significant calibration, and weighted fusion unit reliable fusion output, so that the subsequent DAQA module can obtain more discriminative P3 scale fusion features.

[0152] In detail, given two input feature maps: ,

[0153] in, and This represents the two feature maps input to the JSCR, both of which have the same spatial size and number of channels; , These represent the height and width of the feature map, respectively. This indicates the number of channels. In a real network, and It can represent the scale-aligned features obtained after upsampling or downsampling adjacent scale features, or it can represent the input after channel alignment of shallow detail features and deep semantic features.

[0154] The first step is the generation stage of the BCCR channel relationship description vector.

[0155] To preserve both global statistical responses and locally salient responses, JSCR optimizes input features. Average pooling and max pooling are performed respectively, and the output size of both channels is 1×1×C.

[0156] For the i-th input feature, where i=1,2, the channel relationship description vector obtained by average pooling is:

[0157] ,

[0158] in, Indicates the first Road features in the first Average pooling results across all channels; Representation of feature map In spatial location and the Feature values ​​at each channel; and These represent the height and width of the feature map, respectively. Indicates the channel index.

[0159] The channel relationship description vector obtained by max pooling is: ,

[0160] in, Indicates the first Road features in the first The max pooling results on each channel are used to preserve local salient information in small target regions or strong response regions.

[0161] Then, and Enter them separately Convolution and ReLU activation functions are applied, and the results from both paths are summed to obtain a channel relationship description vector: ,

[0162] in, Indicates the first The channel relationship description vector obtained after the road features pass through BCCR; express Convolution operations are used for channel relationship mapping; This represents a non-linear activation function.

[0163] The second step is the BCCR bidirectional cross-channel relationship modeling stage.

[0164] After obtaining the relationship description vectors between the two channels, BCCR will and Transform it into column vector form, and construct a bidirectional cross-channel relation matrix through outer product: ,

[0165] Where A represents the cross-channel relationship matrix generated by the two channel description vectors, and its dimension is C×C; and These represent the vectorized representations of the relationship description vectors between the two channels. express Transpose of; This represents the normalization function along the column direction, used to convert the channel correlation between two features into relational weights.

[0166] To facilitate the application of the cross-channel relation matrix to the feature map, f1 and f2 are expanded along the spatial dimension as F1, F2 ∈ Subsequently, the bidirectional channel relationship is calibrated for F1 and F2 using the relation matrix A constructed from the description vectors of the two channels and its transpose matrix AT, respectively, as follows:

[0167] ,

[0168] ,

[0169] in, and These represent the channel relationship calibration characteristics of the first and second paths, respectively; and They represent respectively by and The feature matrix obtained by flattening along the spatial dimension has a dimension of C×HW; Indicates by and Constructed cross-channel relationship matrix; express The transpose of the matrix is ​​used to implement reverse channel guidance; This indicates that the flattened features will be restored to their original state. The spatial feature map is used to perform element-wise operations with spatially significant calibration weights.

[0170] Through the above BCCR bidirectional cross-channel relationship modeling, shallow features can be guided by the response of deep semantic channels, and deep features can also be supplemented by edge, texture and position channel information in shallow features, thereby alleviating the problem of difficulty in coordinating semantic information and spatial detail information in conventional cross-scale fusion.

[0171] The third step is the generation of the SSCM spatial saliency calibration map.

[0172] After completing the BCCR channel relationship calibration, SSCM further... Perform spatially significant calibration modeling to enhance the response of the spatial region where small targets are located.

[0173] For input features First, average pooling and max pooling are performed along the channel dimension to obtain a spatially salient description map:

[0174] ,

[0175] ,

[0176] in, Indicates the first Average spatial description of road features; Indicates the first Maximum spatial description of the channel features of the road; Indicates the input feature branch index, which can be 1 or 2; This represents the spatial location index along the height direction of the feature map. Indicates the spatial position index along the width direction of the feature map; Indicates the number of channels.

[0177] BCCR is pooled along the H×W spatial dimension to form a 1×1×C channel description vector, while SSCM is pooled along the C channel dimension to form an H×W×1 spatial description graph. The two have different pooling dimensions, and the summation of their corresponding formulas is also different.

[0178] Then, and The splicing is performed along the channel dimension, and through two layers. Convolution generates spatially saliency calibration maps:

[0179] ,

[0180] in, Indicates the first Spatial saliency calibration map of road features; Indicates the channel dimension and To splice; express Convolution operations are used to extract local spatial context information and output spatial weights of dimension H×W×1.

[0181] The fourth step is the joint recalibration of the space-channel system and the output stage of the weighted fusion unit.

[0182] Spatial saliency calibration map Softmax normalization is performed directly on the H×W spatial location to obtain spatially significant calibration weights. Then Channel relationship calibration characteristics with BCCR output Element-wise multiplication and then combined with the original input features. By summing the residuals, we obtain the joint space-channel recalibration characteristics of the two paths:

[0183] ,

[0184] ,

[0185] in, Represents the spatially significant calibration weight map of the i-th feature; This indicates normalization along the H×W spatial dimension; This represents the intermediate feature obtained after the i-th feature undergoes spatial-channel joint recalibration using BCCR and SSCM; ⊙ represents the original input feature of the i-th path; ⊙ represents element-wise multiplication.

[0186] In this overall network, the JSCR is used as a single fusion node, outputting one fused feature. Specifically, the two output features, after recalibration by BCCR and SSCM, are input into the weighted fusion unit. The contributions of the two features are allocated through learnable weights, resulting in one fused output feature from the JSCR.

[0187] ,

[0188] ,

[0189] in, and This represents the fusion weight of the two recalibration features. and This represents the learnable weight parameters in the weighted fusion unit. Softmax is used to normalize the fusion weights. This represents one fused output feature of the JSCR module in the overall network. The weighted fusion unit does not change the spatial size and number of channels of the feature map; the output dimension remains H×W×C.

[0190] In summary, the JSCR module establishes a bidirectional cross-channel relationship between two scale-aligned features through BCCR, enhancing the mutual guidance between the shallow detail channel and the deep semantic channel in the channel dimension. It generates spatially significant calibration weights through SSCM and utilizes residual structures to preserve original feature information, highlighting the region containing small targets and suppressing interference from complex backgrounds in the spatial dimension. In the fusion output stage, learnable weights balance the contributions of the two recalibrated features, avoiding channel redundancy caused by simple stitching. After embedding a hierarchical bidirectional cross-fusion structure, this module outputs one H×W×C fusion feature as a single fusion node, effectively promoting complementary fusion between shallow detail features and deep semantic features, and improving the detection accuracy and robustness of small-scale, densely distributed, and complex background targets in UAV aerial images.

[0191] 2.3 The process of constructing the density-aware query allocation module is as follows:

[0192] In UAV aerial target detection tasks, the number and distribution of targets vary significantly across different images. For example, scenes such as open roads and parking lot edges may have fewer targets, while traffic intersections, densely parked areas, and urban roads may simultaneously contain a large number of small-scale targets. RT-DETR-type end-to-end detection models typically use a fixed number of query vectors for target prediction. When the number of queries is too small, insufficient candidate queries can easily lead to missed detections in dense, small-target scenes; conversely, when the number of queries is too large, redundant queries are introduced in sparse scenes, increasing decoding computation and interfering with target matching. Therefore, fixed query mechanisms are ill-suited to the dynamically changing target density characteristics of UAV aerial images.

[0193] To address the aforementioned issues, this invention introduces a Density-Aware Query Allocation (DAQA) module during the network's query allocation phase. The module structure is as follows: Figure 5 As shown, DAQA establishes an adaptive correspondence between image target density and query quantity by spatial rearrangement, dual-path global statistical compression, residual excitation, feature concatenation and fusion, and feedforward classification and query selection of the P3-scale enhanced fusion features output by JSCR. This module can suppress redundant queries in sparse target scenes and increase the number of candidate queries in dense target scenes, thereby improving the recall rate of small targets and reducing unnecessary computational overhead.

[0194] In detail, the working process of the DAQA module can be divided into the following four key steps:

[0195] The first step is to rearrange the pixels.

[0196] Let the input feature map of DAQA be: ,

[0197] Where F represents the feature map of the input DAQA; in this embodiment, F is taken from the P3 scale fusion feature enhanced by the JSCR module; and These represent the height and width of the P3 feature map, respectively. This represents the number of channels in the P3 feature map.

[0198] First, the input feature map F is rearranged spatially using pixel rearrangement, transferring some channel information into the spatial dimension, thereby improving the spatial resolution of the feature map without introducing additional convolution parameters. The process is represented as follows:

[0199] ,

[0200] If the upsampling factor for pixel rearrangement is Then we have: ,

[0201] in, This represents the spatial rearranged feature map obtained after pixel rearrangement; r represents the spatial rearrangement ratio. Pixel rearrangement can expand the spatial representation scale, allowing the responses of small targets that were originally distributed in the channel dimension to be rearranged into finer-grained spatial locations, which is beneficial for subsequent modules to estimate target density more accurately.

[0202] The second step is to perform dual-path global pooling compression.

[0203] In obtaining spatial rearrangement features Then, DAQA uses global average pooling and global max pooling to compress the vectors, resulting in two types of global description vectors:

[0204] ,

[0205] ,

[0206] in, This represents the statistical characteristics obtained by global average pooling. This represents the significant response features obtained by global max pooling; This indicates the Global Average Pooling operation; This indicates the Global Max Pooling operation.

[0207] Specifically, and The calculation can be expressed as:

[0208] ,

[0209] ,

[0210] in, and They represent Height and width; This represents the channel index. Global average pooling is used to describe the overall statistical characteristics of the target distribution in an image, while global max pooling is used to preserve strong response information of the target region. The two are used complementaryly to simultaneously capture both the global trend of target density and local salient target responses.

[0211] The third step is to perform dual-path residual excitation and splicing fusion.

[0212] To further enhance density-related features, suppress redundant background responses, and preserve the original statistical information in both pooling descriptions, DAQA will... and Enter by Convolution, SiLU activation function and Lightweight activation branches are formed by convolutions, and residual connections are introduced at the end of each branch. The computation process is expressed as follows:

[0213] ,

[0214] ,

[0215] ,

[0216] ,

[0217] in, and These represent the average pooling branch and the max pooling branch passing through two layers, respectively. Output characteristics after convolution and SiLU nonlinear excitation; and These represent the residual enhancement features obtained by adding the two branches element-wise to the original pooling description; and These represent the two branches in the average pooling branch. convolution; and These represent the two branches in the max pooling process. Convolution. Through residual connections, the module highlights density-related responses while preserving fundamental information about global statistical features and significant response features, thereby improving the stability of density representation.

[0218] Subsequently, the two residual enhancement features are concatenated along the channel dimension, and cross-branch information fusion is achieved using 1×1 convolution to obtain the aggregated density feature. The calculation process is as follows:

[0219] ,

[0220] ,

[0221] in, This indicates a feature concatenation operation performed along the channel dimension; This represents the joint density description formed by splicing two residual enhancement features; This indicates the use of compressed channels and fusion of dual-path information. convolution; This represents the aggregated density features fed into the subsequent feedforward network. Compared to directly adding the two branches element-wise, channel concatenation can fully preserve the complementary information of the average pooling and max pooling branches, and the subsequent fusion convolution can adaptively learn the combined weights between different statistical cues.

[0222] In this embodiment, DAQA performs the density estimation process described above only on the P3-scale fused features. P3 is located at the middle scale of the feature pyramid, possessing both high spatial resolution and semantic response after JSCR recalibration, and can better reflect the density of small targets. At the same time, it avoids repeatedly performing pixel rearrangement, pooling, residual excitation, and splicing fusion on features at scales such as P4 and P5, thereby controlling additional computational load and maintaining the lightweight nature of the query allocation module.

[0223] The fourth step is to establish a mapping between density and the number of queries.

[0224] Obtain the aggregation density characteristics after fusion Then, DAQA inputs it into a feed-forward network (FFN) and generates probability distributions for three target density categories using Softmax: ,

[0225] in, This indicates that the output of FFN is processed through... Normalized probability distribution of the three density categories; , and These represent the probabilities corresponding to low density, medium density, and high density levels, respectively. This represents a feedforward network.

[0226] Considering the characteristics of the VisDrone2019 dataset, which features a high density of small targets and a wide variation in the number of targets, three sets of query counts were pre-defined: ,

[0227] And select the corresponding number of queries based on the density category with the highest probability:

[0228] ,

[0229] ,

[0230] in, This represents the set of preset query quantities; , and These represent the query counts for low-density, medium-density, and high-density scenarios, respectively. On the VisDrone2019 dataset, the following values ​​were taken: =100、 =300、 =500; This represents the predicted density category index; This indicates the final number of queries assigned to the current image.

[0231] During the training phase, the number of real targets in the image can be used as a reference. Construct three density category labels. Considering VisDrone 2019 scenarios with dense small targets such as vehicles and pedestrians, the density category mapping relationship can be represented as follows:

[0232] ,

[0233] in, Indicates the number of real targets in the image; This represents the density category label determined by the target quantity. For VisDrone 2019, This can be considered a low-density scene. This can be considered a medium-density scene. This can be considered a high-density scenario. The above thresholds and query counts can be further fine-tuned based on the target density distribution of the specific training set; this invention is not limited to a single value.

[0234] To enable DAQA to learn an accurate density-to-query mapping, cross-entropy loss can be used to constrain the density classification branch: ,

[0235] in, This represents the density-aware query allocation loss; One-hot encoding representing the density category label; Indicates the first The predicted probability of each density category.

[0236] When training the network in batches, to ensure consistency in parallel computation of the decoder within the same batch, the maximum number of queries within the batch can be used as the number of queries for the current batch: ,

[0237] in, Indicates the number of queries used in the current batch; Indicates batch size; Indicates the first The number of queries per image is predicted by DAQA. During the inference phase, the number is determined directly based on the density prediction results of a single image. This enables adaptive selection of the number of queries per image in low, medium, and high-level scenarios.

[0238] In summary, DAQA enhances high-resolution spatial representation through pixel rearrangement, extracts global statistical information and salient response information through average pooling and max pooling respectively, balances feature enhancement and preservation of original statistical information through dual-path convolution-SiLU activation and branch residual connections, completes cross-path fusion using channel concatenation and 1×1 convolution, and establishes a mapping relationship between target density and query quantity through feedforward networks, Softmax normalization, and query selection mechanisms. Compared with strategies that use a fixed number of queries, DAQA can dynamically adjust the number of queries based on the density of target distribution in UAV aerial images, reducing redundant computation in sparse scenes and providing more candidate queries in dense, small-target scenes, thereby reducing the false negative rate.

[0239] Step 3: Train the drone aerial target detection model using the dataset;

[0240] The experimental environment was configured with a Windows operating system, an NVIDIA GeForce RTX 4070 graphics card, and the PyTorch deep learning framework. The specific configuration is shown in Table 1 below.

[0241] Table 1 Detailed Configuration

[0242]

[0243] The training set images from the dataset were input into the improved network model. Training was performed using a GPU. The improved network model parameters were set to 300 training epochs, with an initial learning rate of 0.01, a minimum learning rate of 0.0001, a weight decay coefficient of 0.00005, and a batch size of 16. During model training, the model parameters with the highest mean average precision (mAP) on the validation set were saved and named best.pt.

[0244] Step 4: Use the trained detection model to detect targets captured by the UAV and evaluate the performance of the detection model.

[0245] To objectively evaluate the detection performance and computational cost of the proposed method, this step uses precision (P), recall (R), mean precision (mAP@0.5) at an intersection-union (IU) threshold of 0.5, and parameters (Params) as evaluation metrics, and conducts a comparative evaluation with the RT-DETR baseline model. RT-DETR was proposed by Zhao et al., and the corresponding published paper is "DETRs Beat YOLOs on Real-time Object Detection," published at CVPR 2024; this comparative evaluation uses their published RT-DETR framework as the baseline model.

[0246] The formulas for calculating precision, recall, average precision (AP), and mean average precision (mAP) are as follows:

[0247] ,

[0248] ,

[0249] ,

[0250] m ,

[0251] Where R represents recall, ranging from 0 to 1; P(R) represents the precision function under different recall rates R; dR represents the integral differential of recall R, used to calculate the area under the precision-recall curve; AP(n) represents the average precision of the nth class, N represents the total number of classes, and mAP represents the average AP of all classes; TP represents the number of correct detections, FP represents the number of false positives, and FN represents the number of false negatives; Params is used to measure the model size. Subsequent experiments will systematically evaluate and analyze the model of this invention and the comparative model based on the above indicators.

[0252] Table 2 Comparison of Experimental Results for Training Models

[0253]

[0254] This invention first uses the RT-DETR model to complete the target detection task in UAV aerial images as a baseline model, with precision (P), recall (R), mAP@0.5, and number of parameters of 62.1%, 45.6%, 47.5%, and 19.88M, respectively. Then, an improved UAV aerial image target detection model is used for detection, with corresponding metrics reaching 63.2%, 47.6%, 49.8%, and 18.91M, respectively. Compared with the RT-DETR baseline model, the proposed model improves P by 1.1 percentage points, R by 2.0 percentage points, and mAP@0.5 by 2.3 percentage points, while reducing the number of parameters by 0.97M. Combining the working mechanisms of each innovative module, the spatial-frequency dual-domain module enhances the edge and texture response of weak targets through directional spatial enhancement and frequency reliability modulation; the joint spatial-channel recalibration module improves multi-scale feature localization through cross-channel relationships and spatially significant region collaborative calibration; and the density-aware query allocation module dynamically adjusts the query capacity according to the target density to improve dense target coverage and reduce redundant queries in sparse scenes. The overall comparison results shown in Table 2 constitute the quantitative basis for the synergistic effect of the above three improvements: the simultaneous improvement of P, R and mAP@0.5 is consistent with the reduction of false detections, the enhancement of detection capability and the improvement of overall detection performance, respectively. The decrease in the number of parameters indicates that the partial channel enhancement and lightweight query allocation adopted have controlled the model complexity while improving performance.

[0255] Meanwhile, to verify the effectiveness of the improved model in target detection tasks using UAV aerial images, this study conducted inference verification on test images containing complex road backgrounds, small-scale targets, dense target distributions, and different shooting angles. The results are as follows: Figure 6 As shown. Figure 6 In the diagram, the boundary of each rectangle represents the target location and scale predicted by the model; the English category labels above the boxes indicate the predicted category, where car represents a car, van represents a van, truck represents a truck, bus represents a bus, pedestrian represents a pedestrian, people represents a crowd, bicycle represents a bicycle, motor represents a motorcycle, and tricycle represents a tricycle; the 0 to 1 value after the category label represents the detection confidence, the larger the value, the higher the confidence of the prediction, and the different box colors are only used to distinguish the categories. Figure 6 The visualization demonstrates that the model of this invention can still provide relatively complete category and location predictions under conditions of small targets, dense distribution, partial occlusion, and complex backgrounds: the spatial-frequency dual-domain module helps preserve the edges and textures of weak targets; the joint spatial-channel recalibration module helps improve the localization consistency of multi-scale targets; and the density-aware query allocation module helps maintain sufficient candidate queries in dense scenes. These visualization results, together with the quantitative improvements in P, R, and mAP@0.5 shown in Table 2, illustrate the effectiveness of the improved scheme of this invention.

[0256] To further facilitate observation and comparison of the detection performance of the model of this invention with the RT-DETR baseline model, images of the same dense targets at road intersections, squares, and sports fields were selected for visualization and comparison, with confidence scores and target labels disabled in the test images. The results are as follows. Figure 7 As shown in the figure, (a) is the detection result of the RT-DETR baseline model, and (b) is the detection result of the model of the present invention. Figure 7 In the middle, from top to bottom, are scenes of road intersections, pedestrians in squares, and dense targets in sports fields.

[0257] Depend on Figure 7 As can be seen, in the road intersection scenario, the model of this invention can retain the detection of distant small-scale vehicles and pedestrians, while reducing redundant detection boxes in some non-target areas; in the square pedestrian scenario, the RT-DETR baseline model produces abnormal detection boxes covering multiple pedestrians and a large background area, while the model of this invention can form detection boxes with reasonable boundary ranges for adjacent pedestrians; in the sports field dense target scenario, the model of this invention maintains the ability to detect dense small-scale pedestrians while reducing the large-scale abnormal detection boxes generated by the baseline model in sparse target areas.

[0258] therefore, Figure 7 Qualitatively, the model of this invention has good localization stability in complex backgrounds, adjacent targets, and dense small target scenes, and can reduce some abnormally large detection boxes and redundant predictions.

[0259] In summary, compared with the RT-DETR baseline model, the model of this invention can better adapt to the detection requirements of small-scale, densely distributed and complex background targets in UAV aerial images, and improve detection accuracy, recall and adaptability to complex scenes while reducing the number of parameters.

[0260] Example 2:

[0261] This invention provides an electronic device, including a processor and a memory. The memory stores program instructions for implementing the UAV aerial small target detection method based on joint recalibration and query allocation as described in Embodiment 1. The processor executes the program instructions stored in the memory to perform UAV aerial small target detection.

[0262] The processor can also be called a Central Processing Unit (CPU). A processor may be an integrated circuit chip with signal processing capabilities. A processor can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0263] The methods described in Embodiment 1 of this invention are applicable to this electronic device embodiment. The specific functions implemented by this electronic device embodiment are the same as those in the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0264] Example 3:

[0265] This embodiment provides a storage medium storing a computer program or instruction capable of implementing the above-described UAV aerial small target detection method based on joint recalibration and query allocation. When the computer program or instruction is run on a computer, it executes the steps of the above-described UAV aerial small target detection method based on joint recalibration and query allocation.

[0266] The computer program or instructions can be stored in the aforementioned storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0267] The methods described in Embodiment 1 of this invention are all applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0268] This invention provides a method for small target detection in UAV aerial photography based on joint recalibration and query allocation. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for small target detection in UAV aerial photography based on joint recalibration and query allocation, characterized in that, Includes the following steps: Step 1: Obtain the preprocessed drone aerial photography dataset; Step 2: Build a drone aerial target detection model; Step 3: Train the drone aerial target detection model using the preprocessed drone aerial photography dataset to obtain the trained detection model; Step 4: Use the trained detection model to perform target detection on the drone aerial images to be processed, obtain the detection results, and evaluate the performance of the trained detection model.

2. The method for detecting small targets in UAV aerial photography based on joint recalibration and query allocation according to claim 1, characterized in that, Step 2 includes: The input data is fed into the backbone network and processed by introducing a multi-stage space-frequency dual-domain module to obtain multi-scale features; Multi-scale features are input into the neck network, and multi-scale fused features are obtained by introducing a joint spatial-channel recalibration module in the multi-scale feature fusion stage. The high-resolution features from the multi-scale fusion features are input into the density-aware query allocation module, and the query allocation strategy is dynamically adjusted according to the target distribution density to obtain the final number of allocated queries. The number of queries and the enhanced features obtained by feature concatenation of multi-scale fusion features are input into the head network. The decoder and the detector head predict the target category and target bounding box based on the query vector.

3. The method for detecting small targets in UAV aerial photography based on joint recalibration and query allocation according to claim 2, characterized in that, The process of the spatial-frequency dual-domain module processing features includes: First, the input features are convolved to obtain intermediate features; Subsequently, the intermediate features are divided into channels to obtain retained branch features and enhanced branch features; Next, the enhanced branch features are input into multiple cascaded cross-domain dual attention blocks for processing to obtain the processed enhanced branch features; Finally, the preserved branch features are concatenated with the processed enhanced branch features, and the output features are obtained by convolutional fusion.

4. The UAV aerial photography small target detection method based on joint recalibration and query allocation according to claim 3, characterized in that, The process of processing cross-domain dual attention block features includes: First, after batch normalization and depthwise convolution processing of the input features, the residuals are added to the input features to obtain the local enhancement features; Subsequently, the local enhancement features are divided into channels according to the channel division ratio to obtain enhanced branch features and preserved branch features; Next, the enhanced branch features after layer normalization are input into the space frequency channel attention module for processing to obtain the processed enhanced branch features; Finally, the enhanced branch features and the preserved branch features are concatenated and processed through a feedforward network to obtain the output features.

5. The UAV aerial photography small target detection method based on joint recalibration and query allocation according to claim 4, characterized in that, The process of feature processing by the spatial frequency channel attention module includes: First, the input features after pointwise convolution are divided into spatial branch features and frequency branch features along the channel dimension; Then, directional spatial enhancement is performed on the spatial branch features by using 3×3, 1×5 and 5×1 depth convolutions respectively, and the features obtained after feature concatenation are fused by pointwise convolution to obtain spatially enhanced features; Next, the frequency branch features are subjected to a two-dimensional real-valued fast Fourier transform to obtain the complex frequency response; the real and imaginary parts are concatenated along the channel dimension, and the frequency response is locally jointly modulated by a 3×3 depth convolution to obtain the modulated frequency features; the modulated frequency features are then processed using a two-dimensional real-valued inverse fast Fourier transform to obtain the frequency reconstruction features. Subsequently, the frequency reconstruction features are input into the frequency reliability gating unit, and frequency reliability weights are generated by global average pooling, 1×1 convolution and sigmoid activation function; the frequency reconstruction features are then multiplied element-wise using the frequency reliability weights to obtain the frequency enhancement features. Finally, the spatial enhancement features and frequency enhancement features are concatenated to obtain the spatial-frequency fusion features. The spatial-frequency fusion features are then input into an efficient channel attention unit for lightweight channel recalibration, and the recalibrated spatial-frequency fusion features are output-mapped using pointwise convolution to obtain the output features.

6. The UAV aerial photography small target detection method based on joint recalibration and query allocation according to claim 2, characterized in that, The process of feature processing by the joint space-channel recalibration module includes: First, the first input feature and the second input feature are respectively input into the bidirectional cross-channel relationship unit for processing to obtain the first channel relationship description vector and the second channel relationship description vector; Then, the cross-channel relationship description vectors after vector transformation are subjected to outer product and normalization to obtain the cross-channel relationship matrix; Next, the cross-channel relationship matrix and its transpose are used to perform bidirectional channel relationship calibration on the first and second input features, respectively, to obtain the first channel relationship calibration features and the second channel relationship calibration features; this process is expressed as: ， ， in, and These represent the calibration features of the first channel relationship and the calibration features of the second channel relationship, respectively. and These respectively represent the features derived from the first input. Second input features The feature matrix obtained by flattening along the spatial dimension; Representing the cross-channel relationship matrix The transpose of the matrix; This indicates a size restoration operation; Subsequently, the first channel relationship calibration feature and the second channel relationship calibration feature are respectively input into the spatial saliency calibration unit for processing to obtain the first spatial saliency calibration map and the second spatial saliency calibration map; the first spatial saliency calibration map and the second spatial saliency calibration map are respectively normalized to obtain the first spatial saliency calibration weight and the second spatial saliency calibration weight; the first spatial saliency calibration weight and the second spatial saliency calibration weight are respectively multiplied element-wise with the first channel relationship calibration feature and the second channel relationship calibration feature, and then added element-wise with the first input feature and the second input feature through residual connection to obtain the first spatial-channel joint recalibration feature and the second spatial-channel joint recalibration feature; Finally, the first spatial-channel joint recalibration features and the second spatial-channel joint recalibration features are input into the weighted fusion unit for weighted fusion to obtain the output features.

7. The UAV aerial photography small target detection method based on joint recalibration and query allocation according to claim 6, characterized in that, The process of processing features in the bidirectional cross-channel relationship unit includes: The input features are processed sequentially using average pooling, 1×1 convolution, and ReLU activation function to obtain the average pooling branch result; The input features are processed sequentially using max pooling, 1×1 convolution, and ReLU activation function to obtain the max pooling branch result; The average pooling branch result and the max pooling branch result are added together to obtain the channel relationship description vector; The process of the spatially significant calibration unit processing features includes: The channel relationship calibration features are processed by average pooling and max pooling respectively to obtain the channel average spatial description map and the channel maximum spatial description map. The channel average spatial description map and the channel maximum spatial description map are concatenated along the channel dimension, and a spatially significant calibration map is generated through multi-layer 3×3 convolution.

8. The method for small target detection in UAV aerial photography based on joint recalibration and query allocation according to claim 2, characterized in that, The process of feature processing by the density-aware query allocation module includes: First, the input features are rearranged spatially using pixel rearrangement to obtain spatially rearranged features. Secondly, global average pooling and global max pooling are performed on the spatial rearrangement features respectively to obtain global average pooling features and global max pooling features; Then, the global average pooling feature and the global max pooling feature are input into the lightweight activation unit for processing, and are added to the global average pooling feature and the global max pooling feature respectively through residual connection to obtain the residual enhancement features of the average pooling branch and the max pooling branch. Next, the residual enhancement features of the average pooling branch and the max pooling branch are concatenated along the channel dimension, and cross-branch fusion is performed using 1×1 convolution to obtain the aggregated density features. Finally, the aggregated density features are input into the feedforward network for processing, and the probability distribution of different target density categories is generated through Softmax normalization. The corresponding query quantity is then selected from the preset query quantity set based on the density category with the highest probability.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the UAV aerial small target detection method based on joint recalibration and query allocation as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The device stores a computer program or instructions that, when executed on a computer, perform the steps of the UAV aerial small target detection method based on joint recalibration and query allocation as described in any one of claims 1 to 8.

Images