An unmanned aerial vehicle image small target detection method based on frequency domain enhancement

Abstract

This invention proposes a method for small target detection in UAV images based on frequency domain enhancement, comprising: extracting multi-scale features from the UAV image to obtain shallow, mid-level, and deep feature maps; performing frequency domain decomposition on the deep feature map to extract high-frequency and low-frequency feature components, and applying differential attention enhancement to the high-frequency structural information and low-frequency semantic information respectively to obtain a frequency-domain enhanced deep feature representation; performing cross-scale feature fusion based on the frequency-domain enhanced deep features and the shallow and mid-level features through a frequency adaptive approach, and adaptively compensating for high-frequency detail information during the fusion process; progressively aggregating the fused multi-scale features and inputting them into a detection head to output the small target detection result in the UAV image. This method can effectively recover the suppressed high-frequency structural information in the deep features, improving the detection accuracy of small targets in complex backgrounds and under multi-scale conditions.

Description

Technical Field

[0001] This invention relates to the technical field of target detection, and more particularly to a method for detecting small targets in UAV images. Background Technology

[0002] With the rapid development of UAV remote sensing and aerial monitoring technologies, UAV imagery is widely used in urban surveillance, traffic management, emergency rescue, and other scenarios. However, due to the high imaging altitude and wide field of view of UAVs, targets in the images typically exhibit characteristics such as small scale, low pixel ratio, and blurred edges, making the detection of small targets in UAV images a significant challenge.

[0003] To address the aforementioned issues, existing technologies have proposed various small target detection methods based on deep features. Some of these methods attempt to incorporate frequency domain analysis, wavelet transform, or attention mechanisms to enhance the representation of detailed features of small targets. For example, frequency decomposition of feature maps is used to model high-frequency and low-frequency components separately, aiming to recover suppressed edge and structural information in deep features. However, this process is usually limited to applying a uniform form of weighting or attention operation to the separated high-frequency and low-frequency features. The frequency domain enhancement lacks differentiated design, which can easily lead to excessive smoothing of high-frequency details or disruption of the consistency of low-frequency semantics, ultimately resulting in a double decrease in the discrimination ability and localization stability of small targets. Furthermore, existing methods still generally rely on fixed interpolation operators or simple feature concatenation to complete cross-scale alignment during the multi-scale feature fusion stage. Frequency domain information is not involved in the construction process of the feature fusion operator, causing the high-frequency structural information on which small targets depend to be further smoothed and weakened during feature upsampling and scale alignment. This results in the model being unable to accurately capture the boundaries of small targets, leading to unstable boundary localization. Meanwhile, in scenarios with complex backgrounds or significant multi-scale variations, the aforementioned spatial-frequency fusion or frequency-domain enhancement methods struggle to adaptively adjust to the varying needs for detailed information at different spatial locations. This makes them unsuitable for complex backgrounds and multi-scale variations, easily leading to problems such as unstable localization of small target boundaries, false detections, or missed detections. In summary, existing technologies suffer from issues such as the weakening of high-frequency structural information of small targets and poor spatial adaptability during the deep feature extraction and multi-scale fusion stages. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a small target detection method for UAV images based on frequency domain enhancement. Building upon wavelet decomposition, this invention introduces a differentiated dual-frequency attention modeling approach to address the differences in spatial structure and semantic representation among different frequency components, achieving differentiated frequency domain enhancement for high and low frequency components. Furthermore, unlike existing methods that only use frequency domain information as a feature enhancement result in subsequent detection, this invention explicitly introduces a frequency adaptive mechanism in the cross-scale feature fusion stage, allowing frequency domain information to directly participate in the construction of the feature fusion operator, solving the problem of smoothing high-frequency details in cross-scale fusion. By combining spatially location-related adaptive filtering strategies, targeted compensation for high-frequency structural information is performed during feature upsampling and scale alignment, thereby maintaining the consistency and integrity of small target detail information across multi-scale features, improving the stability and detection accuracy of small target localization in complex scenes.

[0005] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0006] A method for small target detection in UAV images based on frequency domain enhancement is as follows:

[0007] Acquire drone images, input the drone images into a trained small target detection network, and the processing steps in the small target detection network are as follows:

[0008] S1: Use a backbone network to extract features from the UAV image to obtain shallow features, mid-level features, and deep features;

[0009] S2: The deep features are enhanced by frequency domain differential attention using a wavelet dual-frequency attention module to obtain enhanced deep features;

[0010] S3: Using the enhanced deep features and the mid-level features as input, cross-scale fusion based on adaptive high-pass dual filtering is performed using the first frequency adaptive feature fusion module to obtain the mid-level fused features;

[0011] S4: Using the shallow features of the middle-layer fusion features as input, cross-scale fusion based on adaptive high-pass dual filtering is performed using the second frequency adaptive feature fusion module to obtain shallow fusion features;

[0012] S5: Using shallow fusion features, mid-level fusion features, and enhanced deep features as inputs, cross-scale fusion from shallow to deep is performed based on the feature fusion module to obtain the final fused features;

[0013] S6: After decoding the final fused features through the decoder, input them into the detection head, output the target category and bounding box information, and obtain the detection result.

[0014] Furthermore, the backbone network is selected as either a high-precision backbone network or a lightweight backbone network.

[0015] The precision-priority backbone network is used to adapt to precision-priority scenarios. The precision-priority backbone network includes one of HGNet, ResNet-50, ResNet-101, HRNet, ConvNeXt, and Swing Transformer.

[0016] The lightweight backbone network is used to adapt to real-time priority scenarios. The lightweight backbone network includes one of ResNet-18, MobileNetV2, MobileNetV3, ShuffleNetV2, and GhostNet.

[0017] Furthermore, the deep features are enhanced with frequency-domain differential attention using a wavelet dual-frequency attention module, including:

[0018] S21: The deep features are decomposed in the frequency domain using translation-invariant wavelet transform to generate low-frequency component LL and high-frequency component, including vertical high frequency HL, horizontal high frequency LH and diagonal high frequency HH.

[0019] S22: Combining low-frequency and high-frequency components into a low-frequency branch characteristic based on frequency band characteristics. High-frequency branching characteristics ;

[0020] S23: Set the adjustable parameter α to adjust the attention head in the multi-head self-attention mechanism. According to (1−α):α, it is divided into high-frequency attention heads and low-frequency attention heads;

[0021] S24: Perform window self-attention enhancement on the high-frequency branch features based on the high-frequency attention head to obtain the enhanced high-frequency branch features. ;

[0022] S25: Perform asymmetric self-attention enhancement on the low-frequency branch features based on the low-frequency attention head to obtain the enhanced low-frequency branch features. ;

[0023] S26: The enhanced high-frequency branch features and the enhanced low-frequency branch features are spliced ​​together by channel, and residual connections are made with the deep features to obtain the enhanced deep features.

[0024] Window self-attention enhancement is performed on the high-frequency branch features based on the high-frequency attention head, including:

[0025] S241. High-frequency branch features are displayed according to a preset fixed window size. The space is divided into n non-overlapping local windows to obtain the high-frequency branch features after the division. , where i is the window number;

[0026] S242. High-frequency branch features are obtained through the projection matrix of the high-frequency attention head. Perform linear transformations to generate query vectors respectively. Key vector Sum value vector m is the attention head number;

[0027] S243. Perform multi-head self-attention calculation within the window and obtain the multi-head attention enhancement features within the window. :

[0028] ;

[0029] ;

[0030] in, For the window attention enhancement feature of the m-th high-frequency attention head in the i-th window, For the attention dimension, express Normalization function, For transpose, This is a channel-based splicing operation;

[0031] S244. Rearrange the attention-enhanced features within all windows according to their original spatial positions to obtain the enhanced high-frequency branch features. .

[0032] Based on the low-frequency attention head, asymmetric self-attention enhancement is performed on the low-frequency branch features, including...

[0033] S251. Perform average pooling downsampling on the low-frequency branch features to reduce spatial resolution and obtain the feature map. ;

[0034] S252. Utilize the query matrix of low-frequency attention heads to analyze low-frequency branch features. Generate query vectors by performing linear mapping The key matrix and value matrix of the low-frequency attention head are used to analyze the feature map. Generate key vectors by performing linear mapping Sum value vector ;

[0035] S253. Perform asymmetric self-attention calculation to obtain the enhanced low-frequency branch features. :

[0036] ;

[0037] ;

[0038] in, This represents the asymmetric self-attention enhancement feature of the b-th low-frequency attention head.

[0039] Furthermore, the first frequency-adaptive feature fusion module and the second frequency-adaptive feature fusion module are multiplexed modules; cross-scale fusion based on adaptive high-pass dual filtering is performed using the frequency-adaptive feature fusion module, including:

[0040] ST1. Obtain initial high-resolution features and initial low-resolution features for fusion, and perform channel alignment on the high-resolution features and low-resolution features to obtain channel-consistent first high-resolution features and first low-resolution features.

[0041] ST2. Generate a first adaptive high-pass filter kernel related to spatial location based on the first high-resolution feature, and use the first high-resolution feature and the first low-resolution feature as input to perform cross-scale fusion based on high-frequency enhancement and low-frequency removal using the first adaptive filter kernel to obtain intermediate fused features;

[0042] ST3. Generate a second adaptive high-pass filter kernel related to spatial location based on the intermediate fusion features. Using the initial high-resolution features and initial low-resolution features as input, perform cross-scale fusion based on high-frequency enhancement and low-frequency removal using the second adaptive filter kernel to obtain the final fused features output by the adaptive feature fusion module.

[0043] Furthermore, the first adaptive filter core and the second adaptive filter core are high-pass filter cores or high-pass filter cores constructed by differentially constructing a low-pass filter core and a unit pulse core.

[0044] Furthermore, the method for constructing a high-pass filter kernel by differentially combining a low-pass filter kernel and a unit pulse kernel is as follows:

[0045] A. Input Features First, local features are extracted using a 3×3 convolution to obtain the convolutional feature map. :

[0046] ;

[0047] in, A 3×3 convolution operation is used to extract local features. The feature map described In position The value at that location, For feature dimensions;

[0048] B. Feature maps after convolution processing Perform point-by-point normalization to generate the corresponding low-pass filter weight distribution:

[0049] ;

[0050] in, For Haiming Window, This represents element-wise multiplication. That is, location The low-pass filter weight distribution (low-pass kernel) is obtained by point-by-point normalization. For the first Haiming Window One element;

[0051] C. By performing a differential operation between the low-pass filter weights and the unit pulse kernel, a spatially related adaptive high-pass filter kernel is obtained.

[0052] Furthermore, cross-scale fusion based on high-frequency enhancement and low-frequency removal is performed, including:

[0053] An adaptive filtering kernel is used to perform convolution operation on high-resolution features to achieve spatial location-related high-frequency enhancement filtering, resulting in high-frequency enhanced features. These features are then added element-wise to the high-resolution features to obtain high-frequency enhanced fusion features.

[0054] The low-resolution features are convolved using an adaptive filtering kernel to achieve spatially related low-frequency removal filtering, resulting in low-frequency removed features. The low-frequency removed features are then upsampled through pixel rearrangement to obtain low-frequency removed upsampled features.

[0055] The high-frequency enhancement fusion feature and the low-frequency removal upsampling feature are fused to obtain the cross-scale fusion output of high-frequency enhancement and low-frequency removal.

[0056] Furthermore, in step S5, cross-scale fusion from shallow to deep is performed based on the feature fusion module, using a path aggregation network; the decoder and detector head are based on the RT-DETR model.

[0057] The beneficial effects of this invention are as follows:

[0058] This application achieves accurate frequency domain decomposition of deep features through translation-invariant wavelet transform, avoiding detail loss caused by feature displacement. For high-frequency components, a window-based self-attention mechanism is designed for local modeling, focusing on fine-grained structural information such as edges and textures of small targets while avoiding global background interference. For low-frequency components, an asymmetric self-attention mechanism is designed to achieve lightweight global modeling, reducing computational complexity while preserving the overall target outline and global semantic consistency. Combined with adjustable attention head allocation parameters, the ability to flexibly balance high-frequency detail enhancement and low-frequency semantic preservation is achieved, resulting in a dual improvement in the detail separability and semantic discriminability of small targets, while also enhancing the model's adaptability. This comprehensive approach achieves accurate and differentiated frequency domain enhancement of high and low frequency components, effectively strengthening the feature discrimination capability of small targets.

[0059] This application integrates a spatially location-dependent adaptive high-pass filter kernel generation mechanism into cross-scale fusion, enabling the filter enhancement intensity to dynamically adjust with the feature spatial location. This achieves targeted high-frequency enhancement of small target regions while simultaneously enhancing the spatial adaptability of the model. Pixel rearrangement replaces traditional fixed interpolation for upsampling, and the dual-filter fusion design strengthens high-frequency details, effectively avoiding the loss of high-frequency details caused by interpolation smoothing. This achieves accurate compensation and enhancement of high-frequency structural information during cross-scale fusion, ensuring the consistency and integrity of small target feature information in multi-scale fusion. By deeply involving frequency domain information in cross-scale fusion construction, the accuracy and stability of small target boundary localization are significantly improved, effectively solving problems such as missed detections, false detections, and localization offsets of small targets in complex scenarios. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram of the overall network structure of the UAV small target detection method based on frequency domain enhanced feature fusion according to the present invention;

[0062] Figure 2 This is a schematic diagram of the wavelet dual-frequency attention module of the present invention;

[0063] Figure 3 This is a schematic diagram of the frequency adaptive feature fusion module of the present invention;

[0064] Figure 4 This is a schematic diagram of the high-pass filter generator of the present invention;

[0065] Figure 5This is a visualization of the small target detection results in UAV images and remote sensing images presented in this invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] A method for small target detection in UAV images based on frequency domain enhancement is as follows:

[0068] Acquire drone images, input the drone images into a trained small target detection network, and in the small target detection network, such as... Figure 1 As shown, the processing steps are as follows:

[0069] S1: Feature extraction is performed on the UAV image using a backbone network to obtain shallow, mid-level, and deep features. Among them, the shallow feature map has high spatial resolution and contains rich edge and texture information, while the deep feature map has strong semantic expressive power.

[0070] The backbone network can be selected based on application requirements, prioritizing accuracy or using a lightweight backbone network.

[0071] In a preferred embodiment, the precision-priority backbone network is HGNet, which, through a hierarchical feature extraction structure and an efficient feature reuse mechanism, possesses strong feature representation capabilities while ensuring computational efficiency. This enables it to provide richer and more stable deep semantic representations for subsequent frequency domain modeling and fine-grained feature enhancement for small targets. Furthermore, the precision-priority backbone network can also employ network structures such as ResNet-50, ResNet-101, HRNet, ConvNeXt, and Swin Transformer.

[0072] In another preferred embodiment, the lightweight backbone network is ResNet-18. Through a relatively simple network structure and residual connection mechanism, it maintains good feature extraction capabilities while reducing the number of parameters and computational complexity, making it suitable for small target detection scenarios involving UAVs with limited real-time requirements or computational resources. Furthermore, lightweight backbone networks can also employ lightweight network structures such as MobileNetV2, MobileNetV3, ShuffleNetV2, and GhostNet. The accuracy-priority backbone network can also employ network structures such as ResNet-50, ResNet-101, HRNet, ConvNeXt, and Swing Transformer.

[0073] S2: The deep features are enhanced by frequency-domain differential attention using a wavelet dual-frequency attention module, resulting in enhanced deep features, such as... Figure 2 As shown.

[0074] High-frequency feature components mainly contain fine-grained information such as the edges, textures, and local structures of small targets, and their spatial correlation is usually concentrated in the local neighborhood. Low-frequency feature components, on the other hand, mainly represent the overall outline, spatial layout, and global semantic relationships of the target, exhibiting stronger cross-regional consistency. Blindly applying the same attention modeling approach to these different frequency components can easily lead to over-smoothing of high-frequency details during global modeling or destruction of low-frequency semantics during local modeling, thereby weakening the discrimination ability and localization stability of small targets. To avoid these problems, this application employs differentiated self-attention modeling methods to enhance high-frequency and low-frequency feature components based on their characteristic differences, thereby preserving high-frequency structural details while maintaining low-frequency semantic consistency, ultimately obtaining a frequency-domain enhanced deep feature map.

[0075] In this embodiment, a wavelet dual-frequency attention module is used to perform frequency-domain differential attention enhancement on the deep features, such as... Figure 2 As shown, it includes:

[0076] S21: Using translation-invariant wavelet transform (UWT) to analyze deep features Frequency domain decomposition is performed to generate low-frequency components (LL) and high-frequency components, including vertical high-frequency (HL), horizontal high-frequency (LH), and diagonal high-frequency (HH). The low-frequency components are used to characterize the overall structure and semantic information of the target, while the high-frequency components are used to characterize the target's edges, texture, and detailed structure. The formula is as follows:

[0077] ;

[0078] in, Represents the deep feature map The wavelet transform operator is executed.

[0079] Unlike existing methods that use Discrete Wavelet Transform (DWT), this method employs translation-invariant wavelet transform to avoid the translation sensitivity problem introduced by downsampling during frequency domain decomposition. This ensures that features at different spatial locations maintain consistency in frequency domain representation, which helps preserve the edge and structural information of small targets, reduces response instability caused by feature displacement, and improves the robustness and positioning accuracy of small target detection.

[0080] S22: Based on frequency band characteristics, the low-frequency component and the high-frequency component are combined into low-frequency branch features and high-frequency branch features. Specifically, the low-frequency component LL is used as the low-frequency branch feature. The horizontal, vertical, and diagonal high-frequency components are spliced ​​together along the channel dimension and then processed... Convolutional fusion, as a high-frequency branch feature, is expressed as:

[0081] ;

[0082] in, Indicates low-frequency branching characteristics. Indicates high-frequency branching characteristics, This indicates a channel dimension splicing operation. express Convolution operation.

[0083] S23: Set the adjustable parameter α to adjust the attention head in the multi-head self-attention mechanism. The attention heads are divided into high-frequency attention heads and low-frequency attention heads according to (1−α):α. The high-frequency attention head is used for attention calculation of the high-frequency branch features, and the low-frequency attention head is used for attention calculation of the low-frequency branch features.

[0084] By adjusting the value ratio of the attention head allocation parameter α, a balance can be struck between enhancing the high-frequency response capability of small target edges and local structural information and maintaining the low-frequency modeling capability of the overall semantic consistency of the target. This allows for adaptive adjustment of the influence of high-frequency and low-frequency information on the detection results according to different application scenarios and data characteristics, thereby improving the stability and robustness of small target detection.

[0085] S24: Perform window self-attention enhancement on the high-frequency branch features based on the high-frequency attention head to obtain the enhanced high-frequency branch features;

[0086] Specifically, window self-attention enhancement is performed on the high-frequency branch features based on the high-frequency attention head, including:

[0087] First, high-frequency branch features are divided according to a preset fixed window size (2×2). The space is divided into n non-overlapping local windows to obtain the high-frequency branch features after the division. , where i is the window number;

[0088] Furthermore, the high-frequency branch features are analyzed through the projection matrix of the high-frequency attention head. Perform linear transformations to generate query vectors respectively. Key vector Sum value vector , m is the attention head number:

[0089] ;

[0090] in, , , These are the query matrix, key matrix, and value matrix of the high-frequency attention head, respectively;

[0091] Furthermore, self-attention calculation within the window: Calculate the attention enhancement features of the m-th high-frequency attention head in the i-th window. The attention enhancement features within the i-th window are obtained by concatenating them along the channel dimension. :

[0092] ;

[0093] ;

[0094] in, For the attention dimension, For the window attention enhancement feature of the m-th high-frequency attention head in the i-th window, express Normalization function, For transpose, This is a channel-based splicing operation;

[0095] Furthermore, attention enhancement features are applied to all windows. Rearrange the elements according to their original spatial positions to obtain the enhanced high-frequency branch features. .

[0096] Considering that high-frequency components mainly characterize the edges, textures, and local structural information of small targets, and their spatial correlation is usually limited to relatively nearby pixel regions, this application employs a window self-attention mechanism to enhance high-frequency branch features. This approach allows attention computation to focus more on local edges and texture regions, which is beneficial for enhancing the fine-grained structural response of small targets, while avoiding the problems of unnecessary long-range interference and significantly increased computational complexity introduced by the global self-attention mechanism when processing high-frequency details.

[0097] S25: Based on the low-frequency attention head, perform asymmetric self-attention enhancement on the low-frequency branch features to obtain the enhanced low-frequency branch features; for the low-frequency branch features, first perform average pooling downsampling on the input features to reduce spatial resolution; during the self-attention calculation process, the query vector Q is generated from the unsampled low-frequency branch features, and the key vector K and the numerical vector V are generated from the downsampled low-frequency feature map, thus forming an asymmetric self-attention calculation method to reduce the attention calculation complexity while maintaining the global semantic modeling capability.

[0098] Specifically, the asymmetric self-attention enhancement of the low-frequency branch features based on the low-frequency attention head includes:

[0099] First, the low-frequency branch features are downsampled using average pooling to reduce the spatial resolution, resulting in a feature map. , Indicates average pooling;

[0100] Furthermore, the query matrix of the low-frequency attention head is used to analyze low-frequency branch features. Generate query vectors by performing linear mapping The key matrix and value matrix of the low-frequency attention head are used to analyze the feature map. Generate key vectors by performing linear mapping Sum value vector ;

[0101] ;

[0102] in, , , These are the query matrix, key matrix, and value matrix for low-frequency attention heads, respectively.

[0103] Furthermore, asymmetric self-attention computation is performed to obtain the enhanced low-frequency branch features. :

[0104] ;

[0105] ;

[0106] in, This represents the asymmetric self-attention enhancement feature of the b-th low-frequency attention head.

[0107] Considering that low-frequency components mainly carry the overall outline, structural relationships, and global semantic information of the target, their effective modeling relies on cross-regional contextual association rather than local detail aggregation. Furthermore, low-frequency features are prone to introducing redundant computation and noise interference at high spatial resolutions. This application first performs average pooling downsampling on the low-frequency branch features to compress spatial dimensions and highlight structural information, and then performs asymmetric self-attention computation. This approach effectively reduces the complexity of attention computation without compromising the semantic consistency of low frequencies, and avoids the problems of semantic oversmoothing or computational redundancy that easily arise when existing symmetric global self-attention or local attention mechanisms are directly applied to low-frequency components.

[0108] S26: Concatenate the enhanced high-frequency branch features and enhanced low-frequency branch features according to the channel, and combine them with the deep features. Perform residual connections to obtain enhanced deep features. It compensates for the high-frequency structural information lost during deep feature downsampling and improves the ability to distinguish small targets.

[0109] S3: Using the enhanced deep features and the mid-level features as input, cross-scale fusion based on adaptive high-pass dual filtering is performed using the first frequency adaptive feature fusion module to obtain the mid-level fused features.

[0110] S4: Using the shallow features of the middle-layer fusion features as input, cross-scale fusion based on adaptive high-pass dual filtering is performed using the second frequency adaptive feature fusion module to obtain shallow fusion features.

[0111] The calculation methods of the first frequency adaptive feature fusion module in step S3 and the second frequency adaptive feature fusion module in step S4 are the same, only the inputs are different. The frequency adaptive feature fusion module generates a corresponding high-pass filter kernel at each spatial location of the input features, and dynamically adjusts the high-pass filter kernel according to the spatial location, thereby adaptively adjusting the high-frequency enhancement intensity for the feature distribution of different regions. By introducing a position-dependent high-pass filtering mechanism in the spatial dimension, regions containing small targets can obtain stronger high-frequency enhancement, while the low-frequency response of the background region is effectively suppressed. This is beneficial for highlighting the edge and structural information of small targets, improving the accuracy and stability of small target localization during cross-scale feature fusion.

[0112] In this embodiment, a frequency-adaptive feature fusion module is used to perform cross-scale fusion based on adaptive high-pass dual filtering, such as... Figure 3 As shown, it includes:

[0113] ST1. Obtain initial high-resolution features and initial low-resolution features for fusion, and perform channel alignment on the high-resolution features and low-resolution features to obtain channel-consistent first high-resolution features and first low-resolution features.

[0114] Specifically, let the initial high-resolution features be... The initial low-resolution features are Where s represents the downsampling ratio, , These represent the original number of channels. The feature map height and width are respectively used as inputs. A 1×1 convolution is applied to the initial high-resolution feature and the initial low-resolution feature respectively to perform channel alignment.

[0115] ;

[0116] in, This is a 1×1 convolution operation. , , representing the feature map after channel alignment.

[0117] ST2. Generate a first adaptive high-frequency filter kernel related to spatial location based on the first high-resolution feature. Using the first high-resolution feature and the first low-resolution feature as input, perform cross-scale fusion based on high-frequency enhancement and low-frequency removal using the first adaptive filter kernel to obtain intermediate fused features. In the cross-scale feature fusion process, by adaptively generating a high-pass filter kernel, perform high-frequency enhancement and low-frequency removal on the high-resolution feature and the upsampled low-resolution feature to obtain fused features containing rich high-frequency detail information.

[0118] ST3. Generate a second adaptive high-frequency filter kernel related to spatial location based on the intermediate fusion features. Using the initial high-resolution features and initial low-resolution features as input, perform cross-scale fusion based on high-frequency enhancement and low-frequency removal using the second adaptive filter kernel to obtain the final fused features output by the adaptive feature fusion module.

[0119] Specifically, in the above process, the first and second adaptive high-frequency filter kernels are generated using the same method, employing either a high-pass filter kernel or a high-pass filter kernel constructed by differentially combining a low-pass filter kernel and a unit pulse kernel. The high-pass filter kernel is used to enhance high-resolution features at high frequencies while removing low-resolution features at low frequencies, thereby mitigating the smoothing effect introduced by interpolation upsampling.

[0120] Specifically, when using a high-pass filter kernel, the formula is:

[0121] ;

[0122] in, This represents the first high-resolution feature or intermediate fused feature. This represents the mapping function used to generate the high-pass filter kernel, such as using a high-pass filter generator. , This represents the filter kernel size.

[0123] In the embodiments of this application, the high-pass filter kernel is constructed by differentially constructing a low-pass filter kernel and a unit pulse kernel, such as... Figure 4 As shown, it includes:

[0124] First, input features First, local features are extracted using a 3×3 convolution to obtain the convolutional feature map. :

[0125] ;

[0126] Among them, input features For the first high-resolution feature or intermediate fused feature, A 3×3 convolution operation is used to extract local features. The feature map described In position The value at that location, For feature dimensions;

[0127] Furthermore, a point-by-point normalization operation is performed on the convolutional feature map M to generate the corresponding low-pass filter weight distribution:

[0128] ;

[0129] in, It is a Hamming window. This represents element-wise multiplication. That is, location The low-pass filter weight distribution (low-pass kernel) is obtained by point-by-point normalization. For the first Haiming Window Each element.

[0130] Furthermore, by performing a differential operation between the low-pass filter weights and the unit pulse kernel, a spatially related adaptive high-pass filter kernel is obtained, thereby highlighting high-frequency information in the features and suppressing low-frequency background interference.

[0131] In this embodiment, a first adaptive filtering kernel is used to perform cross-scale fusion based on high-frequency enhancement and low-frequency removal, such as... Figure 3 As shown, it includes:

[0132] First, the first high-resolution feature is convolved using the first adaptive filtering kernel to achieve spatial location-related high-frequency enhancement filtering, resulting in the first high-frequency enhancement feature. Then, the first high-resolution feature is added element-wise to the first high-resolution feature to obtain the first high-frequency enhancement fusion feature.

[0133] Furthermore, the first low-resolution feature is convolved using the first adaptive filtering kernel to achieve spatially related low-frequency removal filtering, resulting in the first low-frequency removed feature. The first low-frequency removed feature is then upsampled through pixel rearrangement to obtain the first low-frequency removed upsampled feature.

[0134] Furthermore, the first high-frequency enhanced fusion feature and the first low-frequency removed upsampling feature are fused (element-by-element addition) to obtain the intermediate fusion feature.

[0135] In this embodiment, a second adaptive filtering kernel is used to perform cross-scale fusion based on high-frequency enhancement and low-frequency removal, such as... Figure 3 As shown, it includes:

[0136] First, the initial high-resolution features are convolved using the second adaptive filter kernel to achieve spatial location-related high-frequency enhancement filtering, resulting in the second high-frequency enhanced features. These features are then added element-wise to the initial high-resolution features to obtain the second high-frequency enhanced fusion features.

[0137] Furthermore, the initial low-resolution features are convolved using the second adaptive filter kernel to achieve spatially related low-frequency removal filtering, resulting in the second low-frequency removal features. The second low-frequency removal features are then upsampled through pixel rearrangement to obtain the second low-frequency removal upsampled features.

[0138] Furthermore, the second high-frequency enhanced fusion feature and the second low-frequency removed upsampling feature are fused (element-by-element addition) to obtain the final fused feature output by the adaptive feature fusion module.

[0139] S5: Using shallow fusion features, mid-level fusion features, and enhanced deep features as inputs, cross-scale fusion from shallow to deep is performed based on the feature fusion module to obtain the final fused features.

[0140] In this embodiment of the application, cross-scale fusion from shallow to deep is performed based on the feature fusion module to obtain the final fused features, including:

[0141] The enhanced deep features obtained from the frequency domain enhancement in step S2 are input to the path aggregation network (PAN) along with the outputs of the two frequency adaptive feature fusion modules. The feature fusion module transmits detailed information from shallow to deep, thereby enhancing the localization capability at multiple scales.

[0142] S6: After decoding the final fused features through the decoder, input them into the detection head, output the target category and bounding box information, and obtain the detection result.

[0143] This embodiment further illustrates the effectiveness and superiority of the UAV image small target detection method based on frequency domain enhanced feature fusion proposed in this application through experiments:

[0144] The experiment in this embodiment was run on the Linux operating system, the experimental code was written in Python, and the detection method was implemented based on the deep learning framework PyTorch (2.1.0+cu118) and ran on an NVIDIA 4090 (24G) workstation.

[0145] In this embodiment, the method of this application is trained and tested using publicly available small object detection datasets VisDron2019, AI-TOD, and SEU-PML. During training, the input images are uniformly adjusted to a preset size to ensure consistent experimental conditions between different methods. The baseline model of this invention is RT-DETR.

[0146] Since the publicly available small object detection dataset has already been pre-divided into training, validation, and test sets, the corresponding data partitioning results can be directly read for training and evaluation. First, the input images are preprocessed, including uniformly adjusting them to 640×640 pixels, and employing data preprocessing and data augmentation strategies consistent with RT-DETR to ensure consistency in the training process.

[0147] Subsequently, the preprocessed training images are input into the detection model to be trained for forward propagation.

[0148] The Inner-SIoU loss function is adopted. Based on the loss value, the gradient is calculated through backpropagation, and the AdamW optimizer is used to iteratively update the model parameters, where the initial learning rate is... The momentum is 0.9, and the learning rate is dynamically adjusted during training using a learning rate scheduling strategy.

[0149] The training rounds consisted of 300 epochs with a batch size of 4, and an early stopping strategy was employed with a patience period of 30 epochs.

[0150] To objectively evaluate the detection performance, this embodiment uses average accuracy (mAP50) and small target evaluation accuracy (APs) as the main evaluation indicators. The experimental results are shown in Tables 1, 2, and 3.

[0151] Table 1 compares the results of this application and existing object detection methods on the VisDron2019 dataset:

[0152]

[0153] Table 2 compares the results of this application and existing object detection methods on the AI-TOD dataset:

[0154]

[0155] Table 3 Comparison of the results of this application and existing object detection methods on the SEU-PML dataset:

[0156]

[0157] As shown in Tables 1, 2, and 3, the proposed method achieves superior detection performance compared to existing technologies on three publicly available UAV target detection datasets: VisDrone2019, AI-TOD, and SEU-PML. On the VisDrone2019 dataset, the proposed method achieves an mAP_50 of 52.8% and a small target detection metric AP_s of 21.6%, both significantly improved compared to other methods. On the AI-TOD dataset, which primarily features very small targets, the proposed method achieves mAP_50 of 48.4% and AP_s of 18.4%, significantly outperforming comparative methods while maintaining relatively constant parameter size and computational complexity. On the SEU-PML dataset, characterized by high-density and heavily occluded scenes, the proposed method achieves an mAP_50 of 75.2%, while also maintaining a high level in the small target metric AP_s.

[0158] The present invention visualizes the small target detection results in UAV images and remote sensing images as follows: Figure 5 As shown, the results indicate that the present invention significantly reduces the false negative rate of small targets and effectively solves the problem of false negative detection in small target detection.

[0159] The experimental results above show that the frequency enhancement-based UAV small target detection method proposed in this application can stably improve detection accuracy under different scenarios and target scales without increasing additional computational overhead. It has good practical application value and is especially suitable for embedded or edge computing devices with limited resources.

[0160] In summary, this invention introduces a wavelet dual-frequency attention module, which enhances the high-frequency components to significantly strengthen the edges and texture details of small targets; and enhances the low-frequency components to maintain the stability of the target semantics and overall structural representation, thereby achieving a balance between detail separability and semantic consistency.

[0161] The frequency adaptive feature fusion module dynamically compensates for high-frequency details during cross-scale fusion, effectively avoiding the impact of interpolation smoothing on the positioning accuracy of small targets.

[0162] The method proposed in this application combines global semantic modeling with local detail enhancement, maintaining high computational efficiency while ensuring detection accuracy, and is suitable for small target detection scenarios of UAVs.

[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for small target detection in UAV images based on frequency domain enhancement, characterized in that, The method is as follows: Acquire drone images, input the drone images into a trained small target detection network, and the processing steps in the small target detection network are as follows: S1: Use a backbone network to extract features from the UAV image to obtain shallow features, mid-level features, and deep features; S2: The deep features are enhanced by frequency domain differential attention using a wavelet dual-frequency attention module to obtain enhanced deep features; S3: Using the enhanced deep features and the mid-level features as input, cross-scale fusion based on adaptive high-pass dual filtering is performed using the first frequency adaptive feature fusion module to obtain the mid-level fused features; S4: Using the shallow features of the middle-layer fusion features as input, cross-scale fusion based on adaptive high-pass dual filtering is performed using the second frequency adaptive feature fusion module to obtain shallow fusion features; S5: Using shallow fusion features, mid-level fusion features, and enhanced deep features as inputs, cross-scale fusion from shallow to deep is performed based on the feature fusion module to obtain the final fused features; S6: After decoding the final fused features through the decoder, input them into the detection head, output the target category and bounding box information, and obtain the detection result.

2. The method for small target detection in UAV images based on frequency domain enhancement according to claim 1, characterized in that, The backbone network is selected from either a high-precision backbone network or a lightweight backbone network. The precision-priority backbone network is used to adapt to precision-priority scenarios. The precision-priority backbone network includes one of HGNet, ResNet-50, ResNet-101, HRNet, ConvNeXt, and Swing Transformer. The lightweight backbone network is used to adapt to real-time priority scenarios. The lightweight backbone network includes one of ResNet-18, MobileNetV2, MobileNetV3, ShuffleNetV2, and GhostNet.

3. The method for small target detection in UAV images based on frequency domain enhancement according to claim 1 or 2, characterized in that, The deep features are enhanced with frequency-domain differential attention using a wavelet dual-frequency attention module, including: S21: The deep features are decomposed in the frequency domain using translation-invariant wavelet transform to generate low-frequency component LL and high-frequency component, including vertical high frequency HL, horizontal high frequency LH and diagonal high frequency HH. S22: Combining low-frequency and high-frequency components into a low-frequency branch characteristic based on frequency band characteristics. High-frequency branching characteristics ; S23: Set the adjustable parameter α to adjust the attention head in the multi-head self-attention mechanism. According to (1−α):α, it is divided into high-frequency attention heads and low-frequency attention heads; S24: Perform window self-attention enhancement on the high-frequency branch features based on the high-frequency attention head to obtain the enhanced high-frequency branch features. ; S25: Perform asymmetric self-attention enhancement on the low-frequency branch features based on the low-frequency attention head to obtain the enhanced low-frequency branch features. ; S26: The enhanced high-frequency branch features and the enhanced low-frequency branch features are spliced ​​together by channel, and residual connections are made with the deep features to obtain the enhanced deep features.

4. The method for small target detection in UAV images based on frequency domain enhancement according to claim 3, characterized in that, Window self-attention enhancement is performed on the high-frequency branch features based on the high-frequency attention head, including: S241. High-frequency branch features are displayed according to a preset fixed window size. The space is divided into n non-overlapping local windows to obtain the high-frequency branch features after the division. , where i is the window number; S242. High-frequency branch features are obtained through the projection matrix of the high-frequency attention head. Perform linear transformations to generate query vectors respectively. Key vector Sum value vector m is the attention head number; S243. Perform multi-head self-attention calculation within the window and obtain the multi-head attention enhancement features within the window. : ； ； in, For the window attention enhancement feature of the m-th high-frequency attention head in the i-th window, For the attention dimension, express Normalization function, For transpose, This is a channel-based splicing operation; S244. Rearrange the attention-enhanced features within all windows according to their original spatial positions to obtain the enhanced high-frequency branch features. .

5. The method for small target detection in UAV images based on frequency domain enhancement according to claim 4, characterized in that, Based on the low-frequency attention head, asymmetric self-attention enhancement is performed on the low-frequency branch features, including... S251. Perform average pooling downsampling on the low-frequency branch features to reduce spatial resolution and obtain the feature map. ; S252. Utilize the query matrix of low-frequency attention heads to analyze low-frequency branch features. Generate query vectors by performing linear mapping The key matrix and value matrix of the low-frequency attention head are used to analyze the feature map. Generate key vectors by performing linear mapping Sum value vector ; S253. Perform asymmetric self-attention calculation to obtain the enhanced low-frequency branch features. : ； ； in, This represents the asymmetric self-attention enhancement feature of the b-th low-frequency attention head.

6. The method for small target detection in UAV images based on frequency domain enhancement according to any one of claims 1, 2, 4 or 5, characterized in that, The first frequency adaptive feature fusion module and the second frequency adaptive feature fusion module are multiplexing modules; Cross-scale fusion based on adaptive high-pass dual filtering is performed using a frequency-adaptive feature fusion module, including: ST1. Obtain initial high-resolution features and initial low-resolution features for fusion, and perform channel alignment on the high-resolution features and low-resolution features to obtain channel-consistent first high-resolution features and first low-resolution features. ST2. Generate a first adaptive high-pass filter kernel related to spatial location based on the first high-resolution feature, and use the first high-resolution feature and the first low-resolution feature as input to perform cross-scale fusion based on high-frequency enhancement and low-frequency removal using the first adaptive filter kernel to obtain intermediate fused features; ST3. Generate a second adaptive high-pass filter kernel related to spatial location based on the intermediate fusion features. Using the initial high-resolution features and initial low-resolution features as input, perform cross-scale fusion based on high-frequency enhancement and low-frequency removal using the second adaptive filter kernel to obtain the final fused features output by the adaptive feature fusion module.

7. The method for small target detection in UAV images based on frequency domain enhancement according to claim 6, characterized in that, The first adaptive filter core and the second adaptive filter core are either high-pass filter cores or high-pass filter cores constructed by differentially constructing a low-pass filter core and a unit pulse core.

8. The method for small target detection in UAV images based on frequency domain enhancement according to claim 7, characterized in that, The method for constructing a high-pass filter kernel by differentially combining a low-pass filter kernel and a unit pulse kernel is as follows: A. Input Features First, local features are extracted using a 3×3 convolution to obtain the convolutional feature map. : ； in, A 3×3 convolution operation is used to extract local features. The feature map described In position The value at that location, For feature dimensions; B. Feature maps after convolution processing Perform point-by-point normalization to generate the corresponding low-pass filter weight distribution: ； in, For Haiming Window, This represents element-wise multiplication. That is, location The low-pass filter weight distribution (low-pass kernel) is obtained by point-by-point normalization. For the first Haiming Window One element; C. By performing a differential operation between the low-pass filter weights and the unit pulse kernel, a spatially related adaptive high-pass filter kernel is obtained.

9. The method for small target detection in UAV images based on frequency domain enhancement according to claim 7 or 8, characterized in that, Perform cross-scale fusion based on high-frequency enhancement and low-frequency removal, including: An adaptive filtering kernel is used to perform convolution operation on high-resolution features to achieve spatial location-related high-frequency enhancement filtering, resulting in high-frequency enhanced features. These features are then added element-wise to the high-resolution features to obtain high-frequency enhanced fusion features. The low-resolution features are convolved using an adaptive filtering kernel to achieve spatially related low-frequency removal filtering, resulting in low-frequency removed features. The low-frequency removed features are then upsampled through pixel rearrangement to obtain low-frequency removed upsampled features. The high-frequency enhancement fusion feature and the low-frequency removal upsampling feature are fused to obtain the cross-scale fusion output of high-frequency enhancement and low-frequency removal.

10. The method for small target detection in UAV images based on frequency domain enhancement according to claim 1, characterized in that, In step S5, cross-scale fusion from shallow to deep is performed based on the feature fusion module, using a path aggregation network; The decoder and detector head are based on the RT-DETR model.

Images