Unmanned aerial vehicle target detection method based on frequency-space joint attention and dynamic fusion

Abstract

The invention relates to the technical field of computer vision detection, in particular to an unmanned aerial vehicle target detection method based on frequency-space joint attention and dynamic fusion, and the method comprises the steps: obtaining an unmanned aerial vehicle image data set, carrying out the preprocessing, and dividing a training set and a test set; constructing a target detection model, inputting the training set into the target detection model to extract image features, sequentially performing frequency domain detail enhancement, spatial domain salient region extraction and multi-scale feature adaptive fusion based on the image features, and establishing a feature sequence; screening the feature sequence to obtain an initial target query, and finishing target classification and positioning on the initial target query through a decoder; training a target detection model by using the training set, and inputting the test set into the trained target detection model to generate a detection result; on the premise that the real-time reasoning advantage of RT-DETR is kept as much as possible, the problems that in an unmanned aerial vehicle scene, a target is prone to missing detection, the scale change is large, the background is complex, and the target is fuzzy are effectively solved, and the detection precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision detection, and particularly relates to a UAV target detection method based on frequency-space joint attention and dynamic fusion. BACKGROUND

[0002] Target detection is a core task of computer vision, which studies the identification and positioning of targets in images. Small object detection (SOD), as an important subfield, focuses on small target detection and has irreplaceable theoretical and application value in practical scenarios such as security monitoring, unmanned airport scene analysis, pedestrian detection, and automatic driving traffic sign recognition. Among them, unmanned aerial vehicles (UAVs) have been widely used in intelligent transportation, security monitoring, and agriculture due to their flexibility and economy. However, due to their special imaging perspective, UAV images have wide field of view, complex background, different target orientations, and dense small target blur.

[0003] In recent years, breakthroughs have been made in the field of target detection, and a series of deep neural network-based methods have been widely applied to downstream tasks. Early methods improved detection performance but often relied on manually designed components such as anchor boxes (anchors) and non-maximum suppression (NMS), which had limited adaptability in complex scenarios. In 2020, the Facebook AI team proposed the end-to-end target detection model DETR (Detection Transformer), which first eliminated the dependence on anchor boxes and NMS, and achieved integrated prediction modeling for target detection through the Transformer architecture, laying an important foundation for subsequent DETR-like model improvements. The Baidu team proposed the RT-DETR (Real Time-Detection Transformer) series, focusing on lightweight feature encoding, decoder design, and efficient training strategies, while maintaining the end-to-end paradigm and achieving real-time performance close to or exceeding the same generation of YOLO (You Only Look Once).

[0004] However, the current mainstream detector is mainly designed for natural images, and there is a problem of difficult to balance global structure information and detail texture expression in small target detection in unmanned aerial vehicle aerial image, resulting in easy loss of small target features; RT-DETR still has two outstanding problems when facing unmanned aerial vehicle aerial scene: first, the feature fusion method depends on simple splicing, lacks adaptive modeling between different levels of features, and it is difficult to fully play the complementarity of high-level and shallow semantic information; second, the interaction process of cross-scale features is static and fixed, lacking modeling of dynamic dependency between scales, and it is difficult to effectively meet the feature alignment requirements of small size target dense distribution and complex background. SUMMARY

[0005] In order to solve the technical problems that the existing RT-DETR model cannot adaptively model according to feature changes, lacks modeling of dynamic dependency between scales, and cannot meet the requirements of small size target dense distribution and complex background, the purpose of the present application is to provide an unmanned aerial vehicle target detection method based on frequency-space joint attention and dynamic fusion, and the technical scheme adopted is as follows:

[0006] An unmanned aerial vehicle image dataset is obtained and preprocessed to divide a training set and a test set;

[0007] A target detection model is constructed, the training set is input into the target detection model to extract image features, frequency domain detail enhancement, space domain salient region extraction and multi-scale feature adaptive fusion are sequentially performed based on the image features, and a feature sequence is established;

[0008] The feature sequence is screened to obtain an initial target query, and the target classification and positioning are completed by the decoder on the initial target query;

[0009] The training set is used to train the target detection model, and the test set is input into the trained target detection model to generate a detection result.

[0010] Preferably, the unmanned aerial vehicle image dataset is obtained and preprocessed to divide a training set and a test set, specifically:

[0011] The unmanned aerial vehicle image dataset is converted into YOLO format, cropped to a uniform size, and divided into a training set and a test set.

[0012] Preferably, the target detection model comprises a backbone network and an efficient hybrid encoder, the efficient hybrid encoder comprises a frequency-space dynamic fusion unit, and the frequency-space dynamic fusion unit comprises a frequency-space joint attention enhancement module, a dual-domain adaptive splicing module and a dynamic multi-scale feature fusion module.

[0013] Preferably, the training set is input into the target detection model to extract image features, and based on the image features, frequency domain detail enhancement, spatial domain salient region extraction and multi-scale feature adaptive fusion are sequentially performed to establish a feature sequence, including:

[0014] The training set is input into the backbone network to extract image features, and the image features include a plurality of feature layer output features, denoted as ;

[0015] The image features are input into the efficient hybrid encoder, and the output features are interacted to construct image interaction features as , and the image interaction features and the remaining feature layer output features are integrated to obtain a feature map;

[0016] The feature map is input into the frequency-space joint attention enhancement module for feature enhancement to obtain an enhanced feature map, denoted as ;

[0017] The enhanced feature map is up-sampled, and a fusion feature is output by combining the dual-domain adaptive splicing module;

[0018] The image features are processed by the dynamic multi-scale feature fusion module to obtain aggregated features, the aggregated features, the enhanced feature map and the fusion feature are residual connected to determine an output result, and the feature sequence is established by integrating the output result.

[0019] Preferably, the feature map is input into the frequency-space joint attention enhancement module for feature enhancement to obtain an enhanced feature map, including:

[0020] Based on the feature map, a frequency domain feature is obtained, the frequency domain feature is separated by using a high-frequency mask and a low-frequency mask, a low-frequency feature and a high-frequency feature are obtained based on the frequency domain feature, the low-frequency feature is processed by using a learnable filter weight, and the processed frequency domain feature is obtained by combining the high-frequency feature, and the frequency domain enhanced feature is obtained by inverse transforming the processed frequency domain feature;

[0021] Based on each pixel position in the feature map, channel average pooling and maximum pooling are performed and spliced, a spatial attention weight is determined by combining a convolution layer and an activation function, and the spatial domain enhanced feature is obtained by combining the spatial attention weight and the feature map;

[0022] The frequency domain enhanced feature and the spatial domain enhanced feature are integrated to obtain the enhanced feature map.

[0023] Preferably, the enhanced feature map is up-sampled, and a fusion feature is output by combining the dual-domain adaptive splicing module, specifically:

[0024] The enhanced feature map is up-sampled to the enhanced feature map , and the output feature The same spatial resolution, the enhanced feature map after upsampling is completed , the enhanced feature map and the output feature The first fusion feature is obtained by inputting the dual-domain adaptive splicing module, and the enhanced feature map and the output feature The second fusion feature is obtained by inputting the dual-domain adaptive splicing module.

[0025] Preferably, the image features are processed by the dynamic multi-scale feature fusion module to obtain the aggregated features, including:

[0026] The image features are spatially scaled and stacked in the channel dimension to form a multi-scale tensor;

[0027] The multi-scale tensor is sequentially subjected to three-dimensional convolution, normalization and activation function to highlight the significant feature response, and the redundant information is compressed through pooling to obtain the aggregated features.

[0028] Preferably, the initial target query is obtained by screening the feature sequence, specifically:

[0029] The features with high intersection-over-union and high classification confidence are selected from the feature sequence using the IoU-aware query selection mechanism, and are defined as the initial target query.

[0030] Preferably, the detection result includes the target category, the positioning frame and the confidence score.

[0031] The present application has the following advantages:

[0032] Based on the RT-DETR-R18 network structure in RT-DETR, the frequency domain detail enhancement, spatial domain significant region extraction and multi-scale feature adaptive fusion operation are sequentially performed, that is, through the frequency-space joint attention enhancement module, the significant region in the spatial domain is highlighted, at the same time, the feature map is subjected to frequency domain transformation and high-low frequency component decomposition, and the low frequency component is adaptively weighted by using a learnable filter, the global structure and detail texture are considered, and the feature expression of small targets is effectively enhanced; the dual-domain adaptive splicing module is used to replace the traditional simple splicing mode, different features are integrated, and the complementarity and robustness between different hierarchical features are enhanced; combined with the dynamic multi-scale feature fusion module, the cross-scale sequence dependency relationship is constructed, the dynamic alignment and information interaction between multi-scale features are realized, and the detection capability of small size targets in unmanned aerial vehicle aerial images is particularly improved.

[0033] The present application effectively solves the problems of easy missed detection, large scale change, complex background and target blur in unmanned aerial vehicle scenes while maintaining the real-time inference advantage of RT-DETR, realizes the improvement of detection accuracy, and has high engineering application value and popularization prospect. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings required to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0035] Figure 1 An architecture diagram of a target detection model of a UAV target detection method based on frequency-space joint attention and dynamic fusion provided by an embodiment of the present application;

[0036] Figure 2 A step flowchart of a UAV target detection method based on frequency-space joint attention and dynamic fusion provided by an embodiment of the present application;

[0037] Figure 3 An architecture diagram of a frequency-space joint attention enhancement module of a UAV target detection method based on frequency-space joint attention and dynamic fusion provided by an embodiment of the present application;

[0038] Figure 4 An architecture diagram of a dual-domain adaptive splicing module of a UAV target detection method based on frequency-space joint attention and dynamic fusion provided by an embodiment of the present application;

[0039] Figure 5 An architecture diagram of a dynamic multi-scale feature fusion module of a UAV target detection method based on frequency-space joint attention and dynamic fusion provided by an embodiment of the present application;

[0040] Figure 6 A detection effect diagram of a UAV target detection method based on frequency-space joint attention and dynamic fusion provided by an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined purposes, the specific embodiments, structures, features and effects of a UAV target detection method based on frequency-space joint attention and dynamic fusion according to the present application are described in detail as follows in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0043] The application provides a UAV target detection method based on frequency-space joint attention and dynamic fusion.

[0044] Please refer to Figure 1 and Figure 2 , which show the architecture diagram and step flow chart of a target detection model of a UAV target detection method based on frequency-space joint attention and dynamic fusion provided by the first embodiment of the application, the method comprising:

[0045] Step S1: acquiring a UAV image dataset and performing preprocessing to divide a training set and a test set;

[0046] Step S2: constructing a target detection model, inputting the training set into the target detection model to extract image features, sequentially performing frequency domain detail enhancement, space domain significant region extraction and multi-scale feature adaptive fusion based on the image features, and establishing a feature sequence;

[0047] Step S3: screening the feature sequence to obtain an initial target query, and completing target classification and positioning through a decoder on the initial target query;

[0048] Step S4: training the target detection model using the training set, and inputting the test set into the trained target detection model to generate a detection result.

[0049] For better illustration, the RT-DETR series in the prior art has application limitations, and the sensitivity to small targets is insufficient. In a complex background such as a UAV aerial scene, the RT-DETR series is easily disturbed by factors such as occlusion, light changes and target size differences, leading to target positioning deviation and recognition accuracy decline, and it is difficult to meet the high requirements of the monitoring speed of the real-time flight task of the UAV. Then, the RT-DETR-R18 network structure in the RT-DETR series is improved, which is a lightweight and widely used convolutional neural network basic model, and can realize high detection speed and accuracy under limited computing resources, so as to improve the detection accuracy of the UAV aerial image target and enhance the reliability of the efficient deployment of the UAV in the fields of security inspection, agricultural monitoring and the like.

[0050] Further, in step S1, specifically:

[0051] The UAV image dataset is converted into a YOLO format, cropped into a uniform size, and divided into a training set and a test set.

[0052] As an optional implementation, in the embodiment, the unmanned aerial vehicle image dataset is a VisDrone2019 dataset, containing 288 video clips and 10209 static images shot in 14 cities in China, covering different scenes such as urban streets and rural areas, the dataset is collected by using unmanned aerial vehicles of different models under various weather and lighting conditions, more than 2.6 million target bounding boxes are manually labeled, and important attributes such as scene visibility, object category and occlusion are provided, which contains 10 categories of targets such as people, pedestrians, cars, trucks, buses, trucks, motorcycles, bicycles, sunshade tricycles and tricycles, and the image types are relatively rich and diversified.

[0053] Specifically, the unmanned aerial vehicle image dataset is preprocessed to remove noise, adjust contrast and brightness, etc., to ensure the quality of the prior data is reliable, the unmanned aerial vehicle image dataset is converted into a YOLO format that meets the input of the RT-DETR model, the annotation file of the YOLO format is usually a text file, each line corresponds to a target, and contains a category ID, a target center point coordinate, a target positioning box width and a target positioning box height; and the image size is adjusted to 640x640, and then the YOLO format is normalized based on the adjusted size of the image to unify the data dimension; the unmanned aerial vehicle image dataset with unified size is divided into a training set and a test set according to a certain proportion, and a 7:3 or 8:2 proportion is usually used for division.

[0054] Further, the target detection model includes a backbone network and an efficient hybrid encoder, the efficient hybrid encoder includes a frequency-space dynamic fusion unit, the frequency-space dynamic fusion unit includes a frequency-space joint attention enhancement module, a dual-domain adaptive splicing module and a dynamic multi-scale feature fusion module.

[0055] It can be explained that the target detection model is based on the RT-DETR model as the basic architecture, in the embodiment, it includes a backbone network, which is used to extract basic features from the training set images, preferably, the backbone network is a ResNet18 network.

[0056] The CCFM (Cross-Scale Feature Fusion Module) module of the efficient hybrid encoder is improved to improve the adaptability of the model when processing scale changes, and then the FSDM (Frequency-Spatial Dynamic Fusion Module) is proposed; it includes the FSE (Frequency-Spatial Enhanced) module to pay attention to the frequency domain and the spatial domain of the image at the same time; the FSAC (Frequency-Spatial Adaptive Concatenation) module is used for feature splicing to better fuse information in different domains and improve the understanding ability of the model; the DySF (Dynamic Multi-Scale Feature Fusion) module is used to adjust the features to adapt to different scenes and needs.

[0057] Further, in step S2, comprising:

[0058] Step S21: input the training set into the backbone network to extract image features, the image features including multiple feature layer output features, denoted as ;

[0059] It is explained that the ResNet18 network is used to extract multi-scale image features of the unmanned aerial vehicle image in the training set, i.e. three layers of multi-scale image features, wherein the ResNet18 network contains multiple convolution blocks and residual connections, in the network structure, the shallower layer can capture low-level detail features such as edges and textures of the image; the middle layer extracts more abstract local semantic features; the deep layer is used to obtain global high-level semantic features, and then multi-layer multi-scale image features are obtained.

[0060] Step S22: input the image features into the efficient hybrid encoder to perform feature interaction on the output features to obtain image interaction features , and integrate the image interaction features and the remaining feature layer output features to obtain a feature map.

[0061] It is explained that the feature layer output features perform adjacent feature interaction within feature layers (AIFI) to obtain image interaction features, that is, while maintaining the original scale of the output features, the information interaction within the output features is enhanced; wherein the output features As the deepest feature, the contained semantic information is more advanced, so only the output feature Intra-scale interaction can improve accuracy while reducing computational complexity; then integrate the remaining feature layer output features to construct a feature map, denoted as .

[0062] Step S23: input the feature map into the space-frequency joint attention enhancement module for feature enhancement, and obtain the enhanced feature map, denoted as ;

[0063] It can be understood that the high layer to the shallow layer, that is, the feature layer output feature towards the feature layer output feature , the semantic information is transmitted, which makes the target image feature more clear, and can enhance the semantic expression of multiple scales; then the low dimension transmits the semantic information to the high dimension again, which enhances the positioning ability of multiple scales from bottom to top; at the same time, in the process of top-down feature transmission, through the space-frequency joint attention enhancement module, the edge information and detailed texture of the target are captured, and more discriminative enhanced feature map is obtained.

[0064] Please refer to Figure 3 , wherein the space-frequency joint attention enhancement module includes a spatial domain attention branch and a frequency domain filter enhancement branch, which realizes global feature learning from a whole perspective.

[0065] Further, in step S23, it includes:

[0066] Step S231: based on the feature map, the frequency domain feature is obtained, the frequency domain feature is separated by using the high-frequency mask and the low-frequency mask, the low-frequency feature and the high-frequency feature are obtained by combining the frequency domain feature, the low-frequency feature is processed by using the learnable filter weight, and the processed frequency domain feature is obtained by combining the high-frequency feature, and the processed frequency domain feature is inverse transformed to obtain the frequency domain enhanced feature.

[0067] Specifically, first, the feature map is subjected to two-dimensional Fourier transform to obtain the frequency domain feature, and the pixel value in the spatial domain is mapped to the frequency domain; then the frequency domain feature is separated by using the high-frequency mask and the low-frequency mask, wherein the low-frequency mask is a square region with a side length of assigned as 1, and the remaining positions are 0; on the contrary, the high-frequency mask is assigned as 0 in the square region of the spectrum center, and the remaining positions are 1.

[0068] It is explained that the spectrum is the frequency distribution of the image corresponding to the frequency domain feature, each point represents the amplitude and phase of the spatial frequency, the low-frequency semantic information changes slowly including large area, contour and background, and the high-frequency semantic information changes rapidly including edge, texture and noise; further, the mask refers to a window taken in the center region of the spectrum, that is, a square region with a side length of assigned as 1, and the remaining positions are 0; on the contrary, the high-frequency mask is assigned as 0 in the square region of the spectrum center, and the remaining positions are 1. a square region of size 2N+1; the corresponding calculation formula is:

[0069]

[0070] wherein, denotes a low-frequency mask, , denotes the height of the feature map, denotes the width of the feature map; denotes a high-frequency mask, ; denotes the position of the pixel in the frequency domain.

[0071] Next, the high-frequency mask and the low-frequency mask are multiplied with the elements of the frequency domain feature to obtain high-frequency features and low-frequency features, the low-frequency features are multiplied with the learnable filter weights and added to the high-frequency features to obtain processed frequency domain features, and finally the processed frequency domain features are inverse transformed to restore the spatial domain features, denoted as frequency domain enhanced features, and the corresponding calculation formula is:

[0072]

[0073] wherein, denotes the frequency domain enhanced features; denotes the feature map, i.e. any feature in , denotes the batch size of the feature map, denotes the channel number of the feature map, denotes the height of the feature map, denotes the width of the feature map; denotes a two-dimensional Fourier transform; denotes centering the output of ; denotes the frequency domain feature; denotes the learnable filter weight, ; denotes an element-wise product; denotes the processed frequency domain feature; denotes an inverse transform operation; denotes a real part taking operation.

[0074] Step S232: performing channel average pooling and maximum pooling based on each pixel position in the feature map, splicing, determining spatial attention weights in combination with a convolution layer and an activation function, and obtaining spatial domain enhanced features in combination with the spatial attention weights and the feature map.

[0075] ​​Specifically, the channel average pooling and the maximum pooling of each pixel position in the feature map, i.e., the channel average pooling is to average all channel values of the pixel position to obtain a representative average value; the channel maximum pooling is to select the maximum value from all channel values of the pixel position; the two pooling results are spliced; then the The spatial attention weight is obtained by a convolution layer and a sigmoid activation function, wherein the convolution layer is to perform convolution operation on the input to extract key spatial information; the activation function is used to map the output of the convolution layer to a value range of 0-1 to generate the spatial attention weight, which reflects the importance degree of each region in the feature map; the spatial attention weight and the feature map are multiplied to obtain the spatial domain enhanced feature, and the corresponding calculation formula is:

[0076]

[0077] wherein, represents the spatial domain enhanced feature; represents a convolution layer operation; represents a splicing operation; represents the channel average pooling; represents the channel maximum pooling; represents the feature map; represents an element-wise product; represents a sigmoid activation function.

[0078] Step S233: integrating the frequency domain enhanced feature and the spatial domain enhanced feature to obtain an enhanced feature map.

[0079] Specifically, the frequency domain enhanced feature obtained in step S231 is added to the spatial domain enhanced feature obtained in step S232 to obtain the output of the FSE module, i.e., the enhanced feature map, and the corresponding calculation formula is:

[0080]

[0081] wherein, represents the enhanced feature map.

[0082] Similarly, the enhanced feature map of each feature in the feature map is obtained, i.e., .

[0083] Please refer to Figure 4 Step S24: up-sampling the enhanced feature map to combine the output fusion feature of the dual-domain adaptive splicing module.

[0084] ​​​To make an explanation, the top-down dual-domain feature transmission process of the FSDM is realized by the frequency-space joint attention enhancement module and the dual-domain adaptive splicing module to enhance the frequency domain information and fuse it with the time domain features, and the obtained feature map contains more rich semantic information.

[0085] Further, in step S24, specifically:

[0086] to the enhanced feature map up-sampling to the enhanced feature map , the output feature with the same spatial resolution, the enhanced feature map after up-sampling , the enhanced feature map and the output feature are input into the dual-domain adaptive splicing module to obtain the first fused feature, and the enhanced feature map after up-sampling and the output feature are input into the dual-domain adaptive splicing module to obtain the second fused feature.

[0087] Specifically, in the present embodiment, the original simple two-input splicing is improved to be capable of receiving three feature tensors of different sources, including the up-sampled higher layer frequency domain enhanced feature , i.e., the enhanced feature map after up-sampling or the first fused feature after up-sampling; the relatively shallow layer feature extracted by the backbone network , i.e., any one of the output features ; the relatively shallow layer feature after frequency domain feature enhancement, i.e., the enhanced feature map or the enhanced feature map

[0088] ; scale alignment is realized by adaptive pooling and interpolation up-sampling, and multi-layer feature fusion is realized by splicing in the channel dimension, i.e., the size of the input feature is dynamically adjusted according to the target size by adaptive pooling to ensure that the features of different levels have consistent resolution in the spatial dimension; interpolation up-sampling can magnify the low-resolution feature to completely match the high-resolution feature in the spatial scale, effectively avoiding information loss or redundancy caused by feature size difference; the corresponding calculation formula is:

[0089] wherein, represents the fused feature, i.e., the output of the FSAC module; represents the intermediate representation of the up-sampled higher layer frequency domain enhanced feature ; represents adaptive maximum pooling; represents adaptive average pooling; represents the relatively shallow layer feature extracted by the backbone network a spatial dimension, represents high, represents wide; represents a shallow layer feature extracted by the backbone network enhanced feature map processed by the FSE module; represents FSE module processing; represents a splicing operation.

[0090] Please refer to Figure 5 , step S25: processing the image features through the dynamic multi-scale feature fusion module to obtain aggregated features, performing residual connection on the aggregated features, the enhanced feature map and the fused features, determining an output result, integrating the output result to establish a feature sequence.

[0091] It can be understood that the core role of the dynamic multi-scale feature fusion module DySF is to perform dynamic scale alignment and cross-scale fusion on the multi-scale features , use three-dimensional convolution and pooling to establish sequence dependence in the scale dimension, and enhance the information interaction between different levels of features, so as to meet the subsequent details of different feature layers, improve the target, and especially the detection effect of small targets.

[0092] Further, in step S25, the aggregated features are obtained by processing the image features through the dynamic multi-scale feature fusion module, including:

[0093] Step S251: performing spatial scale alignment on the image features, and stacking in the channel dimension to form a multi-scale tensor.

[0094] Specifically, the DySF module first receives multi-scale feature maps from the backbone network, i.e., image features , unifies the number of channels thereof through convolution; then, according to the target scale output by the frequency-space dynamic fusion unit FSDM, the image features are respectively subjected to spatial scale alignment using adjacent interpolation up-sampling or adaptive maximum pooling; the aligned features are stacked in the channel dimension to form a multi-scale feature tensor.

[0095] Step S252: sequentially performing three-dimensional convolution, normalization and activation function on the multi-scale tensor to highlight significant feature responses, and compressing redundant information through pooling to obtain aggregated features.

[0096] Specifically, the multi-scale tensor is subjected to three-dimensional convolution, batch normalization and sigmoid activation function to highlight significant feature responses; then, three-dimensional maximum pooling is used to compress the redundant information of the multi-scale tensor, and the most discriminative scale and spatial features are retained, providing a more discriminative fusion representation for the subsequent detection head. The corresponding calculation formula is:

[0097]

[0098] in, This represents the aggregated features, which are the final output features of the DySF module; Represents a multiscale tensor; This represents the sigmoid activation function; This indicates a batch normalization operation; Represents 3D convolution; Represents stacked tensors; This represents the features after scale alignment.

[0099] It can be explained that the calculation formula for scale alignment is:

[0100]

[0101]

[0102] in, Image features representing a uniform number of channels; express Convolution operation; Represents any feature in the image features; Indicates an upsampling operation; Indicates the high value corresponding to the image feature; Indicates the FSDM output terminal and The corresponding layer has high features; This indicates adaptive max pooling; Indicates the FSDM output terminal and The feature width of the corresponding layer; that is Indicates the FSDM output terminal and The scale of the corresponding layer.

[0103] It can be explained that in step S25, the aggregated features, enhanced feature maps, and fused features are residually connected to determine the output result. This preserves the original semantics while injecting cross-scale compensation information, significantly improving the robustness of detecting small targets and targets with uneven scales. The corresponding calculation formula is as follows:

[0104]

[0105] in, Indicates the first The output of the layer; Indicates the first in FSDM The output features are the enhanced feature map and the fused feature; This represents the aggregated features output by the DySF module.

[0106] Next, the output results of the corresponding layers are integrated to establish a feature sequence.

[0107] Further, in step S3, the initial target query is obtained by filtering the feature sequence, specifically as follows:

[0108] The IoU-aware query selection mechanism is used to filter features with high intersection-union ratio and high classification confidence from the feature sequence and define them as the initial target query.

[0109] The explanation is as follows: an IoU-aware query selection mechanism is used based on feature sequences. This mechanism not only focuses on the classification score of each feature but also on the intersection-union ratio (IU) of the features. This addresses the problem that traditional query methods, which are mainly based on classification scores, do not have a high degree of overlap between the selected query and the real target. By considering the IU score, a fixed number of image features that are more likely to describe the real target are selected as the initial target query for the decoder. Due to the special nature of the selection mechanism, it has higher positional accuracy and can provide the decoder with an initial state that is closer to the real target. This significantly improves the matching degree between the query and the real target and reduces the problems of missed detections and false detections caused by the bias of the initial query.

[0110] It can be explained that in step S4, the object detection model is trained using the training set. That is, the object detection model is trained for 400 rounds on the NVIDIA GeForce RTX3090 with an early stopping mechanism 40. Each training input consists of 4 images. After training, the trained weights are saved, and the test set is used as the detection result image of the comparison model.

[0111] Furthermore, the detection results include target category, bounding box, and confidence score; where target category refers to the category to which the object belongs as identified by the target detection model; bounding box is a rectangular region used to represent the position of the target in an image or video frame; and confidence score is used to quantify the detection results.

[0112] Please see Figure 6 To better illustrate this, experiments were conducted based on the VisDrone2019 dataset and compared with existing RT-DETR-R18 networks and other UAV-related detection networks. All comparison methods were tested using default parameters, resulting in Table 1, which shows the experimental comparison between this invention and existing models.

[0113] Table 1. Experimental Comparison of the Invention and Existing Models

[0114]

[0115] To make an illustration, the method of the present application achieves a gain of +0.9% on the mAP50-95 indicator and an increase of +1.4% on the mAP50 indicator. At the same time, under the premise of maintaining relatively optimal accuracy, the parameter amount (23.4M) and the calculation amount (63.0GFLOPs) of the method are still within a reasonable range, indicating that the method has achieved a good balance between detection performance and model complexity. Compared with the latest YOLO series models such as YOLOv10, YOLOv11, YOLOv12 and HIC-YOLOv5, the method of the present application shows strong competitiveness in overall detection accuracy, further verifying its effectiveness in small target detection tasks.

[0116] Next, the detection map of the improved model and the original model are compared, the left is the original model detection map, and the right is the improved model detection map, and the original image is located above the two images.

[0117] It can be explained that, Figure 6 (a) shown in, in the unmanned aerial vehicle aerial image target detection, the originally unrecognized dense vehicles can be successfully recognized by the improved model; Figure 6 (b) shown in, when the image is blurred, the original model will appear overlapping boxes and false detection, while the improved model is more accurate; and according to Figure 6 the display can be known that the target detection model proposed in the present application has better detection performance than the source model under multiple angles.

[0118] Understandably, based on the RT-DETR-R18 network structure in RT-DETR, frequency domain detail enhancement, spatial domain salient region extraction and multi-scale feature adaptive fusion operation are sequentially performed, that is, through the frequency-space joint attention enhancement module, the salient region in the spatial domain is highlighted, at the same time, the feature map is subjected to frequency domain transformation and high-low frequency component decomposition, and the low frequency component is adaptively weighted by using a learnable filter, the global structure and detail texture are considered, and the feature expression of small targets is effectively enhanced; the dual-domain adaptive splicing module is used to replace the traditional simple splicing mode, different features are integrated, and the complementarity and robustness between different level features are enhanced; combined with the dynamic multi-scale feature fusion module, the cross-scale sequence dependence relationship is constructed, the dynamic alignment and information interaction between multi-scale features are realized, and the detection capability for small size targets in the unmanned aerial vehicle aerial image is particularly improved.

[0119] The present application effectively solves the problems of easy missed detection, large scale change, complex background, target blur and the like in the unmanned aerial vehicle scene under the premise of maintaining the real-time inference advantage of RT-DETR, realizes the improvement of detection accuracy, and has high engineering application value and popularization prospect.

[0120] It is to be noted that the sequential order of the above-described embodiments of the present application only for the purpose of description, but not the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0121] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments.

Claims

1. A UAV target detection method based on frequency-space joint attention and dynamic fusion, characterized in that, The method includes: Acquire the drone image dataset and preprocess it to divide it into training and test sets; A target detection model is constructed. The training set is input into the target detection model to extract image features. Based on the image features, frequency domain detail enhancement, spatial domain salient region extraction and multi-scale feature adaptive fusion are performed sequentially to establish a feature sequence. The initial target query is obtained by filtering the feature sequence, and the target classification and localization are completed by the decoder. The target detection model is trained using the training set, and the test set is input into the trained target detection model to generate detection results.

2. The UAV target detection method based on frequency-space joint attention and dynamic fusion according to claim 1, characterized in that, Obtain the drone image dataset and preprocess it to divide it into training and test sets, specifically as follows: The drone image dataset was converted to YOLO format, cropped to a uniform size, and divided into training and testing sets.

3. The UAV target detection method based on frequency-space joint attention and dynamic fusion according to claim 1, characterized in that, The target detection model includes a backbone network and an efficient hybrid encoder. The efficient hybrid encoder includes a frequency-space dynamic fusion unit, which includes a frequency-space joint attention enhancement module, a dual-domain adaptive stitching module, and a dynamic multi-scale feature fusion module.

4. The UAV target detection method based on frequency-space joint attention and dynamic fusion according to claim 3, characterized in that, The training set is input into the object detection model to extract image features. Based on the image features, frequency domain detail enhancement, spatial domain salient region extraction, and multi-scale feature adaptive fusion are performed sequentially to establish a feature sequence, including: The training set is input into the backbone network to extract image features. These image features include output features from multiple feature layers, denoted as... ; Image features are input into a high-efficiency hybrid encoder, and the output features are processed. Construct image interaction features by performing feature interactions. The feature map is obtained by integrating the image interaction features with the output features of the remaining feature layer; The feature map is input into the frequency-space joint attention enhancement module for feature enhancement, resulting in an enhanced feature map, denoted as . ; Upsample the enhanced feature map and output the fused feature by combining it with the dual-domain adaptive stitching module; Image features are processed by a dynamic multi-scale feature fusion module to obtain aggregated features. The aggregated features, enhanced feature maps and fused features are then residually connected to determine the output results. The output results are then integrated to establish a feature sequence.

5. The UAV target detection method based on frequency-space joint attention and dynamic fusion according to claim 4, characterized in that, The feature map is input into the frequency-space joint attention enhancement module for feature enhancement, resulting in an enhanced feature map, including: Frequency domain features are obtained based on feature maps. High-frequency and low-frequency masks are used to separate the frequency domain features. Low-frequency and high-frequency features are obtained by combining the frequency domain features. Low-frequency features are processed by learnable filter weights, and high-frequency features are obtained by combining them with high-frequency features. The processed frequency domain features are then subjected to inverse transformation to obtain frequency domain enhanced features. Channel average pooling and max pooling are performed on each pixel position in the feature map, and the data is concatenated. Spatial attention weights are determined by combining convolutional layers and activation functions. Spatial attention weights are combined with the feature map to obtain spatial enhancement features. The enhanced feature map is obtained by integrating the frequency domain enhancement features and the spatial domain enhancement features.

6. The UAV target detection method based on frequency-space joint attention and dynamic fusion according to claim 5, characterized in that, The enhanced feature map is upsampled, and the fused features are output by combining the dual-domain adaptive stitching module, specifically as follows: Enhanced feature maps Upsampling to enhanced feature map Output features With the same spatial resolution, the upsampled enhanced feature map will be completed. Enhanced feature maps and output features The first fused feature is obtained by inputting the dual-domain adaptive concatenation module. The first fused feature is upsampled and then compared with the enhanced feature map. and output features The second fused feature is obtained by inputting the dual-domain adaptive stitching module.

7. The UAV target detection method based on frequency-space joint attention and dynamic fusion according to claim 6, characterized in that, Image features are processed by a dynamic multi-scale feature fusion module to obtain aggregated features, including: Image features are spatially scale aligned and stacked along the channel dimension to form a multi-scale tensor; The multi-scale tensor is sequentially subjected to 3D convolution, normalization, and activation functions to highlight salient feature responses, and redundant information is compressed by pooling to obtain aggregated features.

8. The UAV target detection method based on frequency-space joint attention and dynamic fusion according to claim 1, characterized in that, The initial target query is obtained by filtering the feature sequence, specifically as follows: The IoU-aware query selection mechanism is used to filter features with high intersection-union ratio and high classification confidence from the feature sequence and define them as the initial target query.

9. The UAV target detection method based on frequency-space joint attention and dynamic fusion according to claim 8, characterized in that, The detection results include target category, bounding box, and confidence score.

Images