An unmanned aerial vehicle small target detection method and system based on dual-domain feature extraction and dynamic scale adaptive query selection

Abstract

The application discloses a kind of based on dual-domain feature extraction and dynamic scale self-adapting query selection UAV small target detection method and system.Method includes: obtaining UAV aerial image data and pre-processing, and the image after pre-processing is input into target detection model and extracts multi-scale feature, carries out frequency detail compensation and space structure reservation to multi-scale feature, obtains dual-domain enhanced feature;Dual-domain enhanced feature and other scale features are input into encoder, and the local feature calibration of context information aggregation and semantic guidance is executed, and encoder candidate feature is obtained;According to the category response and scale attribute of encoder candidate feature, dynamically determine the number of query candidates of small target under the preset query condition, and select query candidate from small target candidate set and remaining candidate set respectively, to obtain final query candidate set;According to final query candidate set, decoder query is constructed, and target class and target bounding box are output through decoder.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, UAV remote sensing image processing, and artificial intelligence, and more specifically to a method and system for detecting small targets on UAVs based on dual-domain feature extraction and dynamic scale adaptive query selection. Background Technology

[0002] Unmanned aerial vehicle (UAV) platforms, with their advantages of high mobility, wide coverage, and flexible deployment, have been widely used in tasks such as traffic monitoring, urban management, disaster assessment, agricultural inspection, and emergency search and rescue. UAV target detection identifies and locates vehicles, pedestrians, bicycles, and other targets in aerial images, providing foundational information for subsequent target tracking, behavior analysis, and decision control.

[0003] Compared to ordinary natural images, UAV images typically have a wide field of view, long shooting distance, small target scale, dense instance distribution, complex background texture, and significant scale variations. Especially under conditions of high-altitude overhead view, low illumination, motion blur, and occlusion, small targets occupy only a few pixels, and discernible information such as edges, textures, and contours is weak, making them prone to degradation during continuous downsampling, deep feature transformation, and multi-scale propagation.

[0004] Existing convolutional neural network object detection methods typically enhance small object features through feature pyramids, multi-scale fusion, attention mechanisms, or by adding high-resolution detection layers. However, simple feature concatenation or frequency enhancement can easily amplify high-frequency background textures, increasing feature redundancy and false detections. Some methods improve local object visibility through cropping, magnification, or density region localization, but these often require additional region processing steps and struggle to maintain an end-to-end detection structure.

[0005] Transformer-based object detectors utilize the interaction between object queries and image features to complete ensemble predictions, reducing manual steps such as anchor box design and non-maximum suppression. However, they still have the following shortcomings in small target scenarios involving UAVs: First, fine-grained edges, textures, contours, and high-frequency information are easily attenuated during backbone network and multi-scale propagation; second, large background areas in UAV images dominate context modeling, and the responses of weak targets are easily diluted by the background context; third, query initialization usually relies on classification confidence or foreground responses for global ranking, and small targets have weak responses before decoding, making them easily excluded by large targets or high-response background candidates under a fixed query budget.

[0006] Therefore, existing technologies have not yet formed a continuous small-target perception and retention mechanism that runs through feature extraction, context aggregation, and query initialization, making it difficult to simultaneously solve the problems of fine-grained information degradation, background interference, and insufficient small-scale candidate representation. How to enhance weak target features, suppress background-dominated context interference, and increase the chance of small-scale candidates entering the decoder without increasing the total number of queries is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a method and system for detecting small targets in unmanned aerial vehicles based on dual-domain feature extraction and dynamic scale adaptive query selection, which solves the problems of easy degradation of fine-grained features of small targets, weak target response suppressed by complex background context, and insufficient small-scale candidate representation under fixed query budget in the prior art.

[0008] This invention does not enhance small targets in a single stage of the detection process, but establishes a continuous collaborative processing flow consisting of dual-domain feature extraction, semantically guided context aggregation, and dynamic scale-adaptive query selection, so that the perception information of small targets is gradually preserved in the feature extraction, encoder interaction, and query initialization stages.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A method for detecting small targets on unmanned aerial vehicles (UAVs) based on dual-domain feature extraction and dynamic scale adaptive query selection includes: S1. Acquire drone aerial image data and preprocess it, then input the preprocessed image into the target detection model; S2. Extract multi-scale features from the input image using the target detection model, and perform frequency detail compensation and spatial structure preservation on the multi-scale features to obtain dual-domain enhanced features; S3. Input the dual-domain enhanced features and other scale features into the encoder, perform context information aggregation and semantically guided local feature calibration to obtain encoder candidate features; S4. Based on the category response and scale attribute of the encoder candidate features, under the preset query conditions, dynamically determine the number of small target candidates to be queried, and select query candidates from the small target candidate set and the remaining candidate set respectively to obtain the final query candidate set. S5. Construct a decoder query based on the final query candidate set, and output the target category and target bounding box through the decoder.

[0010] Preferably, the target detection model in S1 is based on an end-to-end Transformer target detector, including a backbone network, an encoder, an adaptive query selection module, and a decoder; The backbone network is used to extract feature maps at multiple scales from the input image; The encoder is used to perform same-scale interaction and cross-scale fusion on feature maps of multiple scales; The adaptive query selection module selects the decoder query based on the category response and scale attribute of the encoder candidate features; The decoder predicts the target category and target bounding box based on the selected query, and outputs the target category and target bounding box.

[0011] Preferably, in S2, the spatial-frequency sensing feature extraction module performs frequency detail compensation and spatial structure preservation on multi-scale features. The spatial-frequency sensing feature extraction module is set in multiple feature extraction stages of the backbone network to enhance the fine-grained information of weak targets. The space-frequency sensing feature extraction module includes a frequency detail compensation path and a spatial structure preservation path; The frequency detail compensation path is used to perform frequency domain transformation on the multi-scale features, extract and adaptively modulate the spectral features related to small target edges, textures and high-frequency details, and obtain frequency domain reconstructed features through inverse frequency domain transformation; The spatial structure preservation path is used to extract local edge, texture, contour and neighborhood structure information from the multi-scale features to obtain spatial structure features; The frequency domain reconstruction features and spatial structure features are fused together, and residual connections are established with the multi-scale features to obtain dual-domain enhanced features.

[0012] Preferably, in S3, context information aggregation and semantically guided local feature calibration are performed through a semantically guided context aggregation module. The semantically guided context aggregation module is set in the encoder and includes a linear context aggregation unit and a semantically gated local calibration unit. The linear context aggregation unit is used to construct channel context relationships based on the multi-scale features and redistribute channel context information to each spatial location; The semantically gated local calibration unit is used to generate channel semantic weights based on global semantic information, and to modulate local structural features using channel semantic weights to suppress background response and enhance small target-related features.

[0013] Preferably, the number of queries for dynamically determining small target candidates in S4 includes: Obtain the scale attribute of each encoder candidate feature, and divide the encoder candidate features into a small target candidate set and a remaining candidate set according to the scale attribute; Based on the proportion of the small target candidate set in all candidates, the number of small target candidate queries and the number of remaining candidate queries are dynamically determined while the preset total number of queries remains unchanged. The candidate features of the encoder are sorted according to their category response and scale attributes, and a corresponding number of query candidates are selected from the small target candidate set and the remaining candidate set respectively to obtain the final query candidate set.

[0014] Preferably, step S5 determines the corresponding candidate features and candidate reference anchor boxes based on the final query candidate set; Spatial location codes are generated based on the positions of candidate reference anchor boxes, and then fused with candidate features to form decoder content queries. The candidate reference anchor boxes are positioned and corrected based on the candidate features to form a geometric reference box; Input the decoder content query and geometric reference box into the decoder, and output the target category and target bounding box.

[0015] Preferably, a UAV small target detection system based on dual-domain feature extraction and dynamic scale adaptive query selection includes: The image acquisition and preprocessing module is used to acquire and preprocess drone aerial image data. The dual-domain feature extraction module is used to extract multi-scale features and obtain dual-domain enhanced features through frequency detail compensation and spatial structure preservation. The semantic context aggregation module is used to perform context aggregation and semantically guided local feature calibration on dual-domain enhanced features; The dynamic query selection module is used to dynamically select query candidates based on the category response and scale attributes of the encoder candidate features, while keeping the total number of queries constant. The detection output module is used to construct a decoder query based on the selected query candidates and output the target category and target bounding box.

[0016] As can be seen from the above technical solution, compared with the prior art, the present invention provides a method and system for detecting small targets of unmanned aerial vehicles based on dual-domain feature extraction and dynamic scale adaptive query selection, which has the following beneficial effects: 1) By jointly modeling the frequency detail compensation path and the spatial structure preservation path, it is possible to recover the edges, textures and high-frequency weak responses of small targets, and to use local structural constraints to suppress the indiscriminate amplification of background textures. 2) By using linear channel context aggregation and semantic gating local calibration, the complexity of dense spatial interactions is reduced, while the interference of large background areas on the semantic response of weak targets is reduced, thus improving the discriminability of encoder features. 3) Without increasing the total number of queries, dynamically adjust the query budget of small-scale candidates according to the scenario scale composition, and increase the probability of small targets entering the decoder by prioritizing small-scale candidates and supplementing the remaining candidates. 4) Content query, spatial location encoding, and geometric reference boxes are used together for decoder initialization to provide clear semantic and location priors for subsequent cross-scale interaction and bounding box iterative optimization; 5) Form a progressive small target perception and preservation process that continuously coordinates features, context, and queries, which is suitable for UAV detection scenarios such as dense targets, large scale variations, cluttered backgrounds, low contrast, and weak textures. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the overall network structure of the UAV small target detection model provided by the present invention; Figure 2 A schematic diagram of the spatial-frequency sensing feature extraction module provided by the present invention; Figure 3 This is a schematic diagram of the semantic guidance context aggregation module structure provided by the present invention; Figure 4 The flowchart of the adaptive query selection module provided by this invention; Figure 5 A comparison chart of the detection results of the model provided in this invention and the comparison model on the VisDrone2019 dataset; Figure 6 A comparison chart showing the performance and model complexity of the model provided in this invention and the comparison model on the VisDrone2019 dataset for small object detection; Figure 7 A heatmap comparison of the feature response of the model provided in this invention and the basic detection model on the CODrone dataset; Figure 8 A comparison diagram of the small target detection results provided by this invention and the comparison model on the TinyPerson dataset; Figure 9 A comparison diagram of the P2 layer feature response before and after the introduction of the spatial-frequency sensing feature extraction module provided by the present invention; Figure 10 The method flowchart provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figure 10 As shown, this invention discloses a method for detecting small targets on unmanned aerial vehicles (UAVs) based on dual-domain feature extraction and dynamic scale adaptive query selection, including: S1. Acquire drone aerial image data and preprocess it, then input the preprocessed image into the target detection model; S2. Extract multi-scale features from the input image using the target detection model, and perform frequency detail compensation and spatial structure preservation on the multi-scale features to obtain dual-domain enhanced features; S3. Input the dual-domain enhanced features and other scale features into the encoder, perform context information aggregation and semantically guided local feature calibration to obtain encoder candidate features; S4. Based on the category response and scale attributes of the encoder candidate features, and with the total number of queries for the preset decoder remaining constant, dynamically determine the number of queries for small target candidates, and select the corresponding number of query candidates from the small target candidate set and the remaining candidate set respectively to obtain the final query candidate set. That is, under the condition of a fixed total number of queries, dynamically determine the number of queries allocated to small target candidates and the remaining candidates (medium and large targets) based on the proportion of small target candidates in the current encoder candidate features, and then perform candidate selection separately.

[0021] S5. Construct a decoder query based on the final query candidate set, and output the target category and target bounding box through the decoder.

[0022] In a specific embodiment of the present invention, referring to... Figure 1 As shown, in S1, the acquired UAV aerial images are subjected to size adjustment, normalization, and data format conversion processing; in an optional embodiment, the adjusted image size is 640×640.

[0023] Furthermore, in S2, the space-frequency sensing feature extraction module includes a frequency detail compensation path and a spatial structure preservation path, which are used to extract frequency domain detail information and local spatial structure information, respectively.

[0024] Furthermore, in S3, the semantic-guided context aggregation module includes a linear context aggregation unit and a semantic-gated local calibration unit, which are used to implement global context modeling and local semantic calibration, respectively.

[0025] Furthermore, in S4, the candidates are grouped according to the scale attribute of the encoder candidate features, and the number of small target candidate queries is dynamically adjusted within a fixed total query volume based on the proportion of small target candidates in the current image.

[0026] Furthermore, in S5, the decoder input is constructed based on the content features, location encoding, and reference boxes corresponding to the query candidates, and the decoder is used to predict the target category and iteratively optimize the bounding box.

[0027] In one specific embodiment, the following is included: This embodiment uses drone aerial images as model input. The drone aerial images can be acquired by visible light imaging devices, infrared imaging devices, or multimodal imaging devices mounted on the drone platform. The images include vehicles, pedestrians, bicycles, motorcycles, and other targets to be detected. Drone aerial images typically have characteristics such as a large field of view, long shooting distance, small target pixel ratio, dense target distribution, and complex background texture. To ensure the image data meets the input requirements of the target detection model, the original drone aerial images are preprocessed.

[0028] The preprocessing includes the following: The input images are resized to convert drone aerial images of different sizes into a uniform size; the image pixel values ​​are normalized; and the image data is converted into a tensor format that the object detection model can process. In this embodiment, the input image size is set to 640×640. It should be noted that 640×640 is only one optional implementation; in practical applications, other sizes can be set according to the drone's imaging resolution, target size, and computing device performance.

[0029] Specifically, the target detection model in S1 is based on an end-to-end Transformer target detector, including a backbone network, an encoder, an adaptive query selection module, and a decoder; The backbone network is used to extract feature maps at multiple scales from the input image; The encoder is used to perform same-scale interaction and cross-scale fusion on feature maps of multiple scales; The adaptive query selection module selects the decoder query based on the category response and scale attribute of the encoder candidate features; The decoder predicts the target category and target bounding box based on the selected query, and outputs the target category and target bounding box. In other words, the decoder performs decoding processing based on the selected decoder query and outputs the target category and target bounding box.

[0030] In a specific embodiment of the present invention, the target detection model is based on an end-to-end Transformer target detector, including a backbone network, a feature encoder, a query selection module, and a target decoder, preferably using RT-DETR-R18 as the base model. Without changing the main end-to-end ensemble prediction process, improvements are made to the backbone feature extraction, encoder context aggregation, and decoder query initialization to form the D2SQ-DETR model.

[0031] Specifically, in S2, the spatial-frequency sensing feature extraction module performs frequency detail compensation and spatial structure preservation on multi-scale features. The spatial-frequency sensing feature extraction module is set in multiple feature extraction stages of the backbone network to enhance the fine-grained information of weak targets. The space-frequency sensing feature extraction module includes a frequency detail compensation path and a spatial structure preservation path; The frequency detail compensation path is used to perform frequency domain transformation on the multi-scale features, extract and adaptively modulate the spectral features related to small target edges, textures and high-frequency details, and obtain frequency domain reconstructed features through inverse frequency domain transformation; The spatial structure preservation path is used to extract local edge, texture, contour and neighborhood structure information from the multi-scale features to obtain spatial structure features; The frequency domain reconstruction features and spatial structure features are fused together, and residual connections are established with the multi-scale features to obtain dual-domain enhanced features.

[0032] In one specific embodiment of the present invention, small targets in UAV images typically occupy only a few pixels, and their discrimination mainly relies on edges, textures, contours, and local high-frequency information. During continuous downsampling and deep feature transformation in the network, this fine-grained information tends to gradually decay. To address this issue, this embodiment incorporates a spatial-frequency-aware feature extraction module in the backbone network. For example... Figure 2 As shown, the space-frequency sensing feature extraction module includes a frequency detail compensation path and a spatial structure preservation path set in parallel.

[0033] Let the input features of the space-frequency sensing feature extraction module be: ; Where C represents the number of channels of the input feature, and H and W represent the height and width of the input feature, respectively.

[0034] First, perform a two-dimensional discrete Fourier transform on the input feature X to map the spatial domain feature to the frequency domain, obtaining the complex spectrum: ; Where T(·) represents the two-dimensional discrete Fourier transform, and F represents the complex spectrum after the transform.

[0035] Complex spectra include both real and imaginary parts. Compared to processing methods that only use spectral amplitude, preserving both real and imaginary parts avoids premature loss of phase information, allowing the model to utilize spectral features related to the geometric position and contour structure of small targets.

[0036] The real and imaginary parts of the complex spectrum are concatenated along the channel dimension: ; Where Re(F) represents the real part of the complex spectrum, Im(F) represents the imaginary part of the complex spectrum, and Concat(·) represents the channel splicing operation.

[0037] For the spliced ​​feature F c Perform lightweight convolution processing to obtain spectral features F s This convolution operation is used to learn the real part, the imaginary part, and the joint frequency response between different channels.

[0038] Directly enhancing spectral features may simultaneously amplify high-frequency background information such as road edges, building textures, and tree textures. Therefore, this embodiment does not use a fixed frequency band or a fixed enhancement factor, but instead generates adaptive modulation weights based on the relative spectral response within the current sample.

[0039] Let the spectral energy response obtained from the spectral features be E, then perform intra-sample normalization on it: ; Where μ(E) represents the mean of the spectral energy response in the spatial dimension, σ(E) represents the standard deviation of the spectral energy response in the spatial dimension, and ε represents the numerical stability constant used to avoid the denominator being zero.

[0040] The normalized spectral energy response is mapped using a nonlinear function to obtain the spectral modulation diagram. : ; This formula represents the conversion of the normalized spectral energy response into a spectral modulation map through a nonlinear mapping function. Here, A() represents the nonlinear mapping operation, and E represents the spectral energy response. This represents the generated spectrum modulation diagram.

[0041] And mean correction is performed on the aforementioned spectrum modulation plot: ; This formula describes the application of mean-centering / zero-centering to the spectral modulation map, which involves subtracting the average response along its spatial dimensions. This ensures that subsequent spectral enhancement is based on the relative responses between different locations, rather than increasing the overall response across all locations. Mean correction makes spectral enhancement focus on relative differences, avoiding an overall boost to all spatial locations.

[0042] Meanwhile, the content calibration branch applies spectral features F s Global average pooling and pointwise convolution are performed to generate channel-level content calibration weights C. s Content calibration weights are used to limit frequency enhancement based on the semantic content of the input image, preventing high-frequency background responses from being over-amplified.

[0043] Based on the zero-center spectral modulation diagram Ãe and the content calibration weight C s Constructing spectrum-gated weights: ; Where ⊙ denotes element-wise multiplication. The constant 1 in the formula constitutes the residual reference, used to maintain the stability of the original spectral characteristics.

[0044] The spectral features are modulated using the aforementioned spectral gating weights: ; The modulated spectral features are re-splitled into real and imaginary parts and then reassembled into a complex spectrum: ; This formula represents the recombination of the real and imaginary parts after spectral gating modulation into a complex spectrum, where... and These represent the real and imaginary parts of the modulated spectral characteristics obtained by channel splitting, respectively, with i representing the imaginary unit.

[0045] Perform a two-dimensional inverse Fourier transform on the reconstructed complex spectrum, and obtain the frequency domain reconstructed features through the reconstruction mapping: ; Among them, T -1 (·) denotes the two-dimensional inverse Fourier transform, ψ f (·) indicates a mapping operation used for channel recovery and feature reconstruction.

[0046] Furthermore, while frequency domain information can compensate for weakened high-frequency details, some background textures also exhibit strong high-frequency responses. To avoid background noise amplification due to relying solely on frequency enhancement, this embodiment sets up a spatial structure preservation path parallel to the frequency detail compensation path.

[0047] Spatial structure preservation path extracts local spatial structure information through depthwise convolution and pointwise convolution to obtain spatial structure features X. s The spatial structural features mainly include small target edges, textures, contours, local geometric continuity, and neighborhood topological information.

[0048] Depthwise convolution is used to extract local spatial relationships within each channel, while pointwise convolution is used to enable information exchange between channels. This structure can effectively preserve spatial details in shallow features while controlling computational cost.

[0049] Reconstructing feature X in the frequency domain f With spatial structural features X s The components are stitched together along the channel dimension and then merged using channel mapping operations:

[0050] Where Φ(·) represents the channel fusion mapping, X out This represents the output of the spatial-frequency sensing feature extraction module.

[0051] By preserving the original input features through residual connections, frequency domain reconstructed features and spatial structure features are injected into the original representation in a complementary form. Thus, the frequency detail compensation path is used to recover the weak high-frequency response that is easily attenuated during deep propagation, while the spatial structure preservation path is used to maintain the local geometric relationship of the target and limit high-frequency background noise. Together, they form a dual-domain enhanced feature.

[0052] Specifically, in S3, context information aggregation and semantically guided local feature calibration are performed through the semantically guided context aggregation module. The semantically guided context aggregation module is set in the encoder and includes a linear context aggregation unit and a semantically gated local calibration unit. The linear context aggregation unit is used to construct channel context relationships based on the multi-scale features and redistribute channel context information to each spatial location; The semantically gated local calibration unit is used to generate channel semantic weights based on global semantic information, and to modulate local structural features using channel semantic weights to suppress background response and enhance small target-related features.

[0053] In one specific embodiment of the present invention, drone aerial images typically contain large areas of roads, buildings, vegetation, and shadows, with small targets occupying a relatively low proportion of the overall image. When performing direct global spatial interaction, the large background area tends to dominate the context aggregation process, diluting the response of small targets.

[0054] To address this issue, this embodiment incorporates a semantic guidance context aggregation module within the encoder. For example... Figure 3 As shown, this module includes a linear context aggregation unit and a semantically gated local calibration unit.

[0055] Let X1 be the input feature of the semantically guided context aggregation module. Using a pre-normalized residual structure, the processing procedure is as follows: ; ; Where N1(·) and N2(·) represent normalization operations, A(·) represents linear context aggregation operation, F(·) represents semantic gated local calibration operation, and P(·) represents identity mapping or pointwise convolution for channel alignment.

[0056] Perform pointwise convolution mapping on the normalized input features to generate query features, key features, and value features: ; After multi-head reshaping, query features are normalized along the spatial dimension, and key features are normalized along the channel dimension: ; ; Where N = H × W represents the total number of spatial locations, and d represents the channel dimension corresponding to each attention head.

[0057] First, a compact channel context relationship is constructed based on value features and key features. Then, the channel context relationship is redistributed to various spatial locations based on query features. ; The processing does not directly construct an N×N dense relation matrix between spatial locations, thereby reducing the computational complexity of high-resolution features. The linear context aggregation unit achieves global information interaction through the channel context matrix, enabling different spatial locations to share target-related semantics, while reducing the repeated aggregation of large background areas in the spatial relation matrix.

[0058] Even after linear context aggregation, the features may still contain both target and background responses. Therefore, this embodiment further uses a semantically gated local calibration unit to selectively process the aggregated features.

[0059] First, channel expansion and nonlinear activation are performed on the normalized input: ; Where W1 represents pointwise convolution and δ(·) represents a nonlinear activation function.

[0060] Global average pooling is performed on the intermediate feature U, and channel semantic gates are generated through two pointwise convolutions: ; Where GAP(·) represents global average pooling, σ(·) represents the Sigmoid activation function, and g represents the channel semantic weight.

[0061] Simultaneously, depthwise convolution is performed on the intermediate feature U to obtain the local structural response: ; Modulation of local structural responses using channel semantic gates: ; Finally, local calibration features are obtained through pointwise convolution: ; In this design, channel semantic gates are used to select target-relevant channels, while depthwise convolution preserves local spatial structure. The combination of these two techniques suppresses background responses generated by road, building textures, and vegetation areas, while enhancing weak target-relevant features.

[0062] Specifically, the dynamic determination of the number of small target candidates in S4 includes: Obtain the scale attribute of each encoder candidate feature, and divide the encoder candidate features into a small target candidate set and a remaining candidate set according to the scale attribute; Based on the proportion of the small target candidate set in all candidates, the number of small target candidate queries and the number of remaining candidate queries are dynamically determined while the preset total number of queries remains unchanged. The candidate features of the encoder are sorted according to their category response and scale attributes, and a corresponding number of query candidates are selected from the small target candidate set and the remaining candidate set respectively to obtain the final query candidate set.

[0063] In one specific embodiment of the present invention, a traditional end-to-end Transformer detector typically sets a fixed number of decoder queries and performs global sorting based on the classification confidence or foreground response of encoder candidates. Since small targets on UAVs usually have weak classification responses before entering the decoder, if the query selection is based solely on classification confidence, small target candidates are easily excluded by large targets or high-response background candidates.

[0064] Therefore, this embodiment includes an adaptive query selection module. For example... Figure 4 As shown, this module includes query quantity allocation based on scale distribution, joint sorting of category and scale, and hierarchical candidate selection.

[0065] (1) Acquisition of candidate scale attributes

[0066] The encoder's multi-scale features are flattened into candidate features, and the scale response of each candidate feature is obtained through a scale prediction branch. Let the scale response of the i-th candidate feature be s. i The corresponding scale probability is: ; Where σ(·) represents the Sigmoid activation function.

[0067] The encoder candidates are divided into a small target candidate set and a remaining candidate set based on the scale probability. When a candidate satisfies pi <τ small When τ is reached, it is assigned to the small target candidate set. small This represents the threshold for judging small target scale. In one optional implementation, τ small Set to 0.1. This value is for illustrative purposes only and does not constitute a limitation of the present invention.

[0068] (2) Dynamic allocation of query quantity

[0069] If the total number of candidate features of the encoder is N, then the proportion of small target candidates in the current image is: ; Where I(·) represents the indicator function.

[0070] Since the proportions of small targets can vary significantly between different images, directly allocating the number of queries based on the proportion of a single image may cause sudden changes in the number of queries. Therefore, this embodiment uses a smooth update method to obtain the running proportion: ; The formula uses exponential smoothing to smooth the proportion of small target candidates at different update times, so as to avoid drastic fluctuations in the number of queries caused by changes in the proportion of small targets in a single image or a single batch.

[0071] in, This represents the proportion of small target candidates obtained based on the current input at the t-th update; This represents the proportion of smoothed small target candidates obtained after the t-th update; represents the proportion of smoothed small target candidates obtained from the previous update; μ represents the smoothing coefficient, used to control the contribution of historical candidate proportions to the current candidate proportion; t represents the number of updates.

[0072] If the decoder's preset total query count is K, then the number of candidate queries for the small target is: ; The number of remaining candidate queries is: ; The above query allocation process only adjusts the ratio between small target candidate queries and remaining candidate queries, without increasing the total number of decoder queries.

[0073] (3) Joint ranking of categories and scales

[0074] To simultaneously consider the class discriminative power and scale attribute of candidates during the query selection process, this embodiment constructs a comprehensive ranking score. Let S be the classification confidence of the i-th candidate feature. cls,i The scale modulation factor is β iIf the weight of the continuous scale bias is λ, then the overall ranking score is: ; Wherein, when the i-th candidate belongs to the small target candidate, the scale modulation factor β i Set to a value greater than 1; when the i-th candidate does not belong to the small target candidate, the scale modulation factor β i Set to 1. By using a scale modulation factor and a continuous scale bias, the competitiveness of weak target candidates in the ranking process can be improved.

[0075] (4) Hierarchical query selection

[0076] First, from the small goal candidate set T small In the process, the top N are selected based on their comprehensive ranking scores. small One candidate: ; Subsequently, from the remaining candidate set T remain Select the top N remain One candidate: ; The two candidate sets are merged to obtain the final query candidate set: ; Through the above-mentioned hierarchical selection method, small target candidates first compete within an independent candidate set, reducing the chance of them being suppressed by large targets and high-response background candidates; the remaining number of queries is still used to select targets of other scales, thereby maintaining multi-scale detection capability.

[0077] Specifically, step S5 determines the corresponding candidate features and candidate reference anchor boxes based on the final query candidate set; Spatial location codes are generated based on the positions of candidate reference anchor boxes, and then fused with candidate features to form decoder content queries. The candidate reference anchor boxes are positioned and corrected based on the candidate features to form a geometric reference box; Input the decoder content query and geometric reference box into the decoder, and output the target category and target bounding box.

[0078] Each candidate in the final query candidate set corresponds to a candidate feature and a candidate reference anchor box. The candidate features form the content query, and the candidate reference anchor box provides the spatial / geometric location prior. Then, the content query and the geometric reference box are used together as the decoder initialization information, and the category prediction and bounding box prediction are obtained through multi-layer decoding.

[0079] In a specific embodiment of the present invention, after obtaining the final query candidate set, a decoder query is constructed based on the candidate features and candidate reference anchor boxes.

[0080] Let F be the candidate feature corresponding to the final query candidate set. topK The corresponding candidate reference anchor boxes are: ; Where, x i and y i w represents the center coordinates of the candidate reference anchor box. i and h i These represent the width and height of the candidate reference anchor frame, respectively.

[0081] A positional code is generated based on the center coordinates of the candidate reference anchor boxes, and then fused with the candidate content features to obtain the content query for the i-th decoder: ; in, Φ represents the candidate content feature, Φ(·) represents the position encoding function, and δ represents the position prior scaling factor.

[0082] Simultaneously, the bounding box offset ΔB is predicted based on candidate features. i And correct the candidate reference anchor boxes: ; in, This represents the initial geometric reference box corresponding to the i-th query.

[0083] Query the decoder content Q i and initial geometric reference box The input is a pairwise decoder. The decoder performs multi-layer iterative processing using query self-attention, feature cross-attention, and a feedforward network, updating the target category and bounding box layer by layer. The final output includes the target category, target classification confidence, target bounding box center position, and target bounding box width and height.

[0084] Specifically, a UAV small target detection system based on dual-domain feature extraction and dynamic scale adaptive query selection includes: The image acquisition and preprocessing module is used to acquire and preprocess drone aerial image data. The dual-domain feature extraction module is used to extract multi-scale features and obtain dual-domain enhanced features through frequency detail compensation and spatial structure preservation. The semantic context aggregation module is used to perform context aggregation and semantically guided local feature calibration on dual-domain enhanced features; The dynamic query selection module is used to dynamically select query candidates based on the category response and scale attributes of the encoder candidate features, while keeping the total number of queries constant. The detection output module is used to construct a decoder query based on the selected query candidates and output the target category and target bounding box.

[0085] In one specific embodiment of this invention, all experiments were implemented in PyTorch and performed on four NVIDIA RTX3080 GPUs. The development language was Python 3.9, and the CUDA version was 11.8. This embodiment used RT-DETR-R18 as the baseline and employed uniform settings for fair comparison. The input resolution was 640×640, all models were trained for 300 epochs, and the global batch size was 8. AdamW was used with an initial learning rate of 1×10⁻⁶. -4 The weight decay was 0.05. Unless otherwise stated, data augmentation, label assignment, training strategy, and inference settings remained consistent with the baseline, as shown in Table 1.

[0086] Table 1 Experimental Parameter Configuration Table

[0087] The COCO average accuracy is used as the evaluation metric, including AP and AP. 50 AP 75 AP S AP M and AP L AP S Used to evaluate the performance of small target detection; by comparing the average accuracy AP and AP 50 AP 75 Small target average accuracy (AP) S Medium target average accuracy AP M Large target average accuracy AP L The number of parameters and the number of floating-point operations are used to comprehensively evaluate the detection accuracy, positioning ability, small target recognition ability and model complexity of the model of this invention.

[0088] To verify the effectiveness of the proposed UAV small target detection method based on dual-domain feature extraction and dynamic scale adaptive query selection, comparative and ablation experiments were conducted on the VisDrone2019, CODrone, TinyPerson, and BDD100K datasets. The experiments used RT-DETR-R18 as the base model. The standard model constructed in this invention is denoted as D2SQ-DETR, and the lightweight model built for low-computational-consumption applications is denoted as D2SQ-DETR-S. See Table 2 for details.

[0089] Table 2. Performance comparison with representative detectors on the VisDrone 2019 validation set.

[0090] As shown in Table 2, the D2SQ-DETR achieved 33.0 AP and 53.1 AP.50 34.4 AP 75 and 25.5 AP S Compared to the base model RT-DETR-R18, AP, AP 50 AP 75 and AP S They increased by 6.8, 9.0, 8.2, and 7.3 percentage points respectively. AP S The significant improvement demonstrates that the embodiments of the present invention can enhance the effective characterization of small-scale targets in UAV aerial images; AP 75 The improvement indicates that the embodiments of the present invention can not only detect more small targets, but also improve the degree of overlap between the predicted bounding box and the real target.

[0091] The D2SQ-DETR-S achieved 30.4 AP and 22.9 AP. S Meanwhile, the number of parameters was reduced from 20.0M to 10.2M, and the number of floating-point operations was reduced from 60.0 GFLOPs to 54.5 GFLOPs, indicating that the lightweight implementation can still maintain good small target detection performance while reducing model size and computational overhead.

[0092] To further verify the actual detection effect of the UAV small target detection method described in this embodiment of the invention in complex aerial photography scenarios, the detection results of typical scenes in the VisDrone2019 dataset are visualized, as shown below. Figure 5 As shown. By Figure 5 It is known that in scenes with dense pedestrians, distant traffic targets, and low-contrast targets against cluttered backgrounds, RT-DETR-R18, UAV-DETR-R50, and FBRT-YOLO-L still exhibit varying degrees of missed detection of small targets and false detections of the background. The D2SQ-DETR of this invention can detect more small targets with weak appearance information and can generate more complete target bounding boxes for closely spaced target instances.

[0093] For distant vehicle and pedestrian targets, the prediction results of the model in this embodiment are more stable; for densely arranged targets, the model can reduce confusion between adjacent targets; for complex background areas, the model can reduce the misclassification of road textures, building structures or other background responses as targets.

[0094] The above detection results show that the embodiments of the present invention can improve the effective detection capability of weak targets in complex UAV aerial photography scenarios and reduce missed detections and false detections through the synergistic effect of spatial-frequency sensing feature extraction, semantic-guided context aggregation and dynamic scale adaptive query selection.

[0095] To comprehensively evaluate the small target detection accuracy and model complexity of the models, the AP of different detection models was measured on the VisDrone2019 dataset. S The number of floating-point operations and the number of parameters were compared, and the results are as follows: Figure 6 As shown. By Figure 6 It can be seen that the lightweight model D2SQ-DETR-S achieves higher average accuracy for small targets while having fewer parameters and fewer floating-point operations than the basic model RT-DETR-R18. D2SQ-DETR-S reduces the number of parameters from 20.0M to 10.2M and the number of floating-point operations from 60.0 GFLOPs to 54.5 GFLOPs, while simultaneously improving AP and AP. S These represent increases of 4.2 and 4.7 percentage points, respectively. The standard model D2SQ-DETR has 17.9M parameters, lower than RT-DETR-R18 and RT-DETR-R50, while achieving an AP of 25.5. S Although its high-resolution feature interaction increases computational cost, it achieves further improvements in small target detection accuracy and strict bounding box localization metrics, as shown in Tables 3 and 4.

[0096] Table 3 Performance of D2SQ-DETR on the CODrone validation set compared to representative detectors

[0097] Table 4 Performance of D2SQ-DETR-S on the CODrone validation set compared to representative detectors

[0098] As shown in Tables 3 and 4, D2SQ-DETR achieved 21.0 AP and 39.3 AP on the CODrone validation set. 50 19.8 AP 75 4.9 AP S and 17.4 AP M Compared to RT-DETR-R50, AP, AP 75 AP S and AP M The performance improved by 2.5, 3.0, 1.6, and 3.1 percentage points respectively, while the number of parameters decreased from 42.0M to 17.9M. The D2SQ-DETR-S achieved 19.0 AP and 36.7 AP. 50 17.5 AP 75 3.8AP S and 16.0 AP M This indicates that the lightweight implementation still maintains a good balance between accuracy and efficiency in mixed-scale target detection scenarios. For example... Figure 7As shown, under low light, long distance, and cluttered background conditions in the CODrone dataset, compared with the basic detection model RT-DETR-R18, D2SQ-DETR forms a more concentrated and complete feature response in the real small target region and reduces the activation of irrelevant background regions, indicating that the embodiments of the present invention can enhance the features of weak small targets and suppress complex background interference.

[0099] As shown in Table 5, the D2SQ-DETR achieved 10.7 AP and 30.8 AP. 50 4.8 AP 75 and 9.4 AP S Compared to RT-DETR-R18, the above indicators improved by 1.6, 3.6, 1.1, and 1.5 percentage points, respectively. AP 50 and AP S The improvement indicates that the present invention can improve the effective detection capability of small pedestrians with weak textures; AP 75 The improvement indicates that the increased target recall did not come at the cost of reduced positioning quality.

[0100] Table 5 Performance comparison with representative detectors on the TinyPerson validation set.

[0101] like Figure 8 As shown, in complex backgrounds, dense pedestrians, and distant scenes in the TinyPerson dataset, D2SQ-DETR can detect more tiny pedestrians that are difficult to distinguish from the surrounding environment, and generate more complete predicted bounding boxes for targets that are close to each other, reducing the missed detection of low-contrast and weak-response targets.

[0102] Table 6. Performance comparison with representative detectors on the BDD100K validation set.

[0103] As shown in Table 6, the D2SQ-DETR achieved 33.5 AP and 59.6 AP. 50 32.1AP 75 15.9AP S 38.6 AP M and 57.0 AP L Compared to RT-DETR-R18, AP, AP 50 AP 75 AP S and AP MThe improvements were 2.8, 4.3, 3.5, 1.9, and 3.0 percentage points, respectively. Distant vehicles and pedestrians in BDD100K images and small targets in UAV images share similar characteristics of weak texture, low pixel ratio, and complex backgrounds. These results demonstrate that the present invention also has certain applicability in non-aerial photography scenarios such as road traffic.

[0104] As shown in Table 7, after introducing SAFE alone, AP increased from 16.3 to 18.3. S The AP improved from 2.8 to 3.8, while both the number of parameters and computational cost decreased, indicating that dual-domain feature extraction can enhance the details of small targets while controlling complexity. Introducing SGCA improved the AP to 17.6, demonstrating that channel context aggregation and semantically guided local calibration can reduce the interference of complex backgrounds on weak target responses. Adding the P2 feature layer improved the AP to 18.0, but significantly increased the computational cost, indicating that high-resolution features are helpful for small target detection, but incur additional computational overhead.

[0105] When SAFE, SGCA, P2, and DSQS are used simultaneously, the model achieves 21.0 AP and 39.3 AP. 50 19.8 AP 75 and 4.8AP S The results are all higher than those obtained by using the modules individually or in combination. This shows that each module plays a role in the feature extraction, context interaction, and query initialization stages, and together they form a small target perception and preservation process that runs through the entire detection process.

[0106] like Figure 9 As shown, compared with the model without the spatial-frequency-aware feature extraction module, the P2 layer generates more concentrated and clear feature responses around the small target area after the module is introduced. At the same time, the activation of irrelevant background areas is significantly reduced. This indicates that the module can effectively preserve the edge, texture and local contour information of small targets and slow down the degradation of fine-grained features during network propagation.

[0107] Table 7 Ablation experiments of each module of the present invention on the CODrone validation set.

[0108] In summary, through the synergistic effect of the spatial-frequency-aware feature extraction module, the semantic-guided context aggregation module, and the adaptive query selection module, stable detection performance improvements were achieved in different datasets and scenarios, demonstrating that the present invention has good small target detection accuracy, localization capability, cross-scenario adaptability, and deployment flexibility.

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0110] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting small targets in unmanned aerial vehicles (UAVs) based on dual-domain feature extraction and dynamic scale adaptive query selection, characterized in that, include: S1. Acquire drone aerial image data and preprocess it, then input the preprocessed image into the target detection model; S2. Extract multi-scale features from the input image using the target detection model, and perform frequency detail compensation and spatial structure preservation on the multi-scale features to obtain dual-domain enhanced features; S3. Input the dual-domain enhanced features and other scale features into the encoder, perform context information aggregation and semantically guided local feature calibration to obtain encoder candidate features; S4. Based on the category response and scale attribute of the encoder candidate features, under the preset query conditions, dynamically determine the number of small target candidates to be queried, and select query candidates from the small target candidate set and the remaining candidate set respectively to obtain the final query candidate set. S5. Construct a decoder query based on the final query candidate set, and output the target category and target bounding box through the decoder.

2. The method for detecting small UAV targets based on dual-domain feature extraction and dynamic scale adaptive query selection according to claim 1, characterized in that, The target detection model in S1 is based on an end-to-end Transformer target detector, including a backbone network, encoder, adaptive query selection module and decoder; The backbone network is used to extract feature maps at multiple scales from the input image; The encoder is used to perform same-scale interaction and cross-scale fusion on feature maps of multiple scales; The adaptive query selection module selects the decoder query based on the category response and scale attribute of the encoder candidate features; The decoder predicts the target category and target bounding box based on the selected query and outputs the target category and target bounding box.

3. The method for detecting small UAV targets based on dual-domain feature extraction and dynamic scale adaptive query selection according to claim 1, characterized in that, In S2, the spatial-frequency sensing feature extraction module performs frequency detail compensation and spatial structure preservation on multi-scale features. The spatial-frequency sensing feature extraction module is set in multiple feature extraction stages of the backbone network to enhance the fine-grained information of weak targets. The space-frequency sensing feature extraction module includes a frequency detail compensation path and a spatial structure preservation path; The frequency detail compensation path is used to perform frequency domain transformation on the multi-scale features, extract and adaptively modulate the spectral features related to small target edges, textures and high-frequency details, and obtain frequency domain reconstructed features through inverse frequency domain transformation; The spatial structure preservation path is used to extract local edge, texture, contour and neighborhood structure information from the multi-scale features to obtain spatial structure features; The frequency domain reconstruction features and spatial structure features are fused together, and residual connections are established with the multi-scale features to obtain dual-domain enhanced features.

4. The method for detecting small UAV targets based on dual-domain feature extraction and dynamic scale adaptive query selection according to claim 1, characterized in that, In S3, context information aggregation and semantically guided local feature calibration are performed through the semantically guided context aggregation module. The semantically guided context aggregation module is set in the encoder and includes a linear context aggregation unit and a semantically gated local calibration unit. The linear context aggregation unit is used to construct channel context relationships based on the multi-scale features and redistribute channel context information to each spatial location; The semantically gated local calibration unit is used to generate channel semantic weights based on global semantic information, and to modulate local structural features using channel semantic weights to suppress background response and enhance small target-related features.

5. The method for detecting small UAV targets based on dual-domain feature extraction and dynamic scale adaptive query selection according to claim 1, characterized in that, The number of queries for dynamically determining small target candidates in S4 includes: Obtain the scale attribute of each encoder candidate feature, and divide the encoder candidate features into a small target candidate set and a remaining candidate set according to the scale attribute; Based on the proportion of the small target candidate set in all candidates, the number of small target candidate queries and the number of remaining candidate queries are dynamically determined while the preset total number of queries remains unchanged. The candidate features of the encoder are sorted according to their category response and scale attributes, and a corresponding number of query candidates are selected from the small target candidate set and the remaining candidate set respectively to obtain the final query candidate set.

6. The method for detecting small UAV targets based on dual-domain feature extraction and dynamic scale adaptive query selection according to claim 1, characterized in that, S5 determines the corresponding candidate features and candidate reference anchor boxes based on the final query candidate set; Spatial location codes are generated based on the positions of candidate reference anchor boxes, and then fused with candidate features to form decoder content queries. The candidate reference anchor boxes are positioned and corrected based on the candidate features to form a geometric reference box; Input the decoder content query and geometric reference box into the decoder, and output the target category and target bounding box.

7. A UAV small target detection system based on dual-domain feature extraction and dynamic scale adaptive query selection, characterized in that, include: The image acquisition and preprocessing module is used to acquire and preprocess drone aerial image data. The dual-domain feature extraction module is used to extract multi-scale features and obtain dual-domain enhanced features through frequency detail compensation and spatial structure preservation. The semantic context aggregation module is used to perform context aggregation and semantically guided local feature calibration on dual-domain enhanced features; The dynamic query selection module is used to dynamically select query candidates based on the category response and scale attributes of the encoder candidate features, while keeping the total number of queries constant. The detection output module is used to construct a decoder query based on the selected query candidates and output the target category and target bounding box.

Images