Unmanned aerial vehicle image target detection method based on collaborative feature fusion

Abstract

The invention discloses an unmanned aerial vehicle image target detection method based on collaborative feature fusion, and the method comprises the steps: carrying out the detection of a to-be-detected unmanned aerial vehicle image through employing a trained unmanned aerial vehicle image target detection model, and obtaining a detection result; the model comprises a backbone network, a neck network and a detection head network; the backbone network is integrated with a multi-scale dynamic double-domain coupling module, a multi-scale feature map is extracted through frequency domain-space domain combined processing, and the multi-scale feature map serves as input of a neck network after target edge features are enhanced; the neck network adopts a collaborative feature pyramid network to fuse features layer by layer, linear deformable convolution is used in a P2 layer to enhance details, a wide-area sensing module is combined with large kernel convolution to capture a long-range context in a P3 layer, and finally, the detection head network outputs a target detection result. The method solves the technical problems that the extraction precision of small target features in the aerial image of the unmanned aerial vehicle is not high, background noise interference is serious, and multi-scale target fusion is insufficient.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image processing, and in particular to a method for detecting targets in unmanned aerial vehicle (UAV) images based on collaborative feature fusion. Background Art

[0002] In the task of object detection in aerial imagery, drones, with their high flexibility and wide field of view, have become a crucial platform for efficient monitoring. Their applications are increasingly widespread in fields such as military reconnaissance, urban surveillance, environmental protection, and agricultural management. With the rapid development of deep neural networks, general object detection technology has also achieved significant progress. FasterRCNN, DETR, and the YOLO series have become the dominant paradigm in general object detection and have shone brightly in natural image detection. Drone aerial imagery, typically captured at high altitudes and from diverse angles, can be subject to dynamic backgrounds and environmental noise. The wide field of view of aerial imagery increases the proportion of background areas and background noise. Objects exhibit a wide range of scale distributions, with a high proportion of small objects. This scale imbalance not only places higher demands on the generalization capabilities of object detection models, but also significantly increases the difficulty of feature learning due to the inherently scarce information content of small objects and the dominance of background noise. Furthermore, compared to natural images, aerial imagery has a higher resolution, requiring more complex data processing and computational workloads.

[0003] To address the challenge of object recognition from drones, existing researchers have primarily sought to improve detection accuracy through approaches such as contextual information fusion, frequency domain enhancement, and multi-scale feature fusion. Contextual information fusion methods employ a small region proposal generator to aggregate semantic associations between the target and its surroundings to enhance small target detection. However, these computationally intensive operations can lead to significant memory overhead when processing high-resolution images and may introduce irrelevant background noise. Frequency domain enhancement methods utilize Fourier transforms or learnable filters to enhance high-frequency features (such as edges and textures), effectively improving the representation of motion-blurred targets. However, frequency domain transforms can destroy discriminative features in the spatial domain, and some methods are computationally complex, limiting their practical applications. Multi-scale feature fusion methods alleviate the scale imbalance problem by combining feature information from different scales. Early image pyramid networks (FPNs) employed end-to-end multi-scale feature construction but failed to fully consider the synergistic relationships between layers. Bidirectional Feature Pyramid Networks (BiFPNs) employ a bidirectional weighted fusion mechanism to enhance feature correlation and contextual information, but this mechanism can also lead to potential information loss. Furthermore, compared with the feature information of medium and large targets, the feature information of small targets is relatively scarce, and it is difficult to obtain effective positive feedback in the weighted fusion mechanism, which leads to excessive suppression of low-level features in the fusion process, causing the model to tend to learn large targets, thereby weakening the expression of subtle features of small targets.

[0004] Therefore, how to improve the feature extraction accuracy of small targets in drone aerial images and achieve multi-scale target fusion has become a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0005] In response to the above-mentioned shortcomings of the existing technology, the present invention provides a UAV image target detection method based on collaborative feature fusion. Through a multi-scale dynamic dual-domain coupling module and a collaborative feature pyramid network, it solves the technical problems of low accuracy in feature extraction of small targets in UAV aerial images, severe background noise interference, and insufficient multi-scale target fusion, thereby significantly improving detection accuracy and robustness.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A method for target detection in drone images based on collaborative feature fusion is proposed. The method obtains a drone image to be detected and inputs it into a pre-trained drone image target detection model to obtain a target detection result for the drone image to be detected.

[0008] The drone image target detection model includes a backbone network integrated with a multi-scale dynamic dual-domain coupling module, a neck network based on a collaborative feature pyramid network, and a detection head network for performing target detection; the drone image target detection model uses the input drone image as the input of the backbone network, uses the multi-scale dynamic dual-domain coupling module to perform adaptive average pooling to generate a multi-scale feature map, and extracts the edge information of the multi-scale feature map, thereby obtaining a dual-domain enhanced multi-scale feature map; the dual-domain enhanced multi-scale feature map is used as the input of the neck network, the dual-domain enhanced multi-scale features are fused layer by layer through the collaborative feature pyramid network, and the extraction of detail information is enhanced by linear deformable convolution, and the target detail information and context information are captured through the wide-area perception module to output the further enhanced multi-scale fusion features and input them into the detection head network to obtain the target detection result of the drone image as the output of the drone image target detection model.

[0009] As a preferred solution, the backbone network includes a Stem layer and four CPS modules connected in sequence; the Stem layer includes two cascaded CBS units, and the CPS module includes a cascaded CBS unit, multiple serial multi-scale dynamic dual-domain coupling modules and a splicing layer, wherein the CBS unit includes a cascaded convolutional layer, a BN layer and a SiLU activation function; in the CPS module, after the input feature map is feature extracted by the CBS unit, it is split into two branches according to the channel dimension by Split, and one of the branches is subjected to dual-domain edge enhancement by multiple serial multi-scale dynamic dual-domain coupling modules. The enhanced feature map and the output of the first multi-scale dynamic dual-domain coupling module are spliced ​​with the features of the other branch in the splicing layer, and the output is obtained as the output of the CSP module;

[0010] In the backbone network, the drone image, which serves as the input to the drone image object detection model, is fed into the backbone network's Stem layer. The Stem layer's output is downsampled and fed into the first CPS module. The output of each CPS module serves as the input to the next layer in the backbone network, and the feature map output by the last CPS module serves as the backbone network's output. The backbone network's final output feature map has a size of H / 32 × W / 32 × C5, where H and W are the height and width of the backbone network's input drone image, and C5 is the number of channels in the feature map output by the last CPS module in the backbone network.

[0011] As a preferred solution, the multi-scale dynamic dual-domain coupling module includes an adaptive average pooling layer, a 1×1 convolution unit, a 3×3 grouped convolution unit, an upsampling layer, a dual-domain edge information extraction module, and a splicing layer connected in sequence; wherein the 1×1 convolution unit includes a cascaded 1×1 convolution layer, a BN layer, and a SiLU activation function, and the 3×3 grouped convolution unit includes a cascaded 3×3 convolution layer, a BN layer, and a SiLU activation function;

[0012] In the multi-scale dynamic dual-domain coupling module, feature extraction of two branches is performed on the input feature map respectively, one of which is used as the input of the adaptive averaging layer, the output of the adaptive averaging layer is used as the input of the 1×1 convolution unit, the output of the 1×1 convolution unit is used as the input of the 3×3 grouped convolution unit, the output of the 3×3 grouped convolution unit is used as the input of the upsampling layer, and the output of the upsampling layer is used as the input of the dual-domain edge information extraction module. The output of the dual-domain edge information extraction module is spliced ​​with the output of the other branch after passing through the 3×3 grouped convolution unit and the upsampling layer in the splicing layer, and the output is used as the overall output of the multi-scale dynamic dual-domain coupling module.

[0013] As a preferred solution, the processing process of the dual-domain edge information extraction module includes the following steps:

[0014] The output feature map of the upsampling layer in the multi-scale dynamic dual-domain coupling module is used as the input of the dual-domain edge information extraction. The input feature map is subjected to feature extraction of two branches. One branch is smoothed by a 3×3 average pooling layer to calculate the low-frequency feature component, and the low-frequency feature component is subtracted from the input feature map to extract the high-frequency feature component; the other branch converts the spatial domain representation into the frequency domain representation by a two-dimensional discrete Fourier transform, and a high-pass filter is applied to the frequency domain feature map to retain the high-frequency information and suppress the low-frequency components to obtain the frequency domain feature map after high-pass filtering; the other branch is smoothed by a 3×3 average pooling layer to calculate the low-frequency feature component and subtract the low-frequency feature component from the input feature map to extract the high-frequency feature component. The high-frequency feature components are localized for contrast calculation to obtain an edge intensity map of the high-frequency feature components, and the edge intensity map is used as a guide to enhance the amplitude of the frequency domain feature map after high-pass filtering to obtain an amplitude-enhanced frequency domain feature map; the amplitude-enhanced frequency domain feature map is frequency-sharpened to obtain a sharpened frequency domain feature map; the sharpened frequency domain feature map is converted back to the spatial domain representation through a two-dimensional discrete inverse Fourier transform to obtain a sharpened feature map; the original input feature map, the high-frequency feature components and the sharpened feature map are spliced ​​and output as the overall output of the dual-domain edge information extraction module.

[0015] As a preferred solution, the processing process of the dual-domain edge information extraction module is shown by the following formula:

[0016]

[0017] Where, Xhigh (i, j) represents the high-frequency feature component, X up (i, j) represents the input two-dimensional spatial domain feature map, i and j represent the row and column coordinates of the feature map in the spatial domain, m and n represent the offset within the window, and k represents the pooling kernel size;

[0018]

[0019] Where, X up (u,v) represents the frequency domain feature map obtained by two-dimensional discrete Fourier transform of the original input feature map, F(·) represents the two-dimensional discrete Fourier transform, represents the kernel function of Fourier transform, represents phase rotation, M and N represent width and height in the spatial domain respectively, i represents the imaginary unit, and denotes the spatial variation of u and v frequencies of the feature map in the horizontal and vertical directions, respectively;

[0020]

[0021]

[0022] M(X up (u,v)) and P(X up (u,v)) represent the frequency domain feature map X up The amplitude and phase of (u, v) at position (u, v), Re(·) and Im(·) represent the real and imaginary parts of the frequency domain feature map respectively, M hpf (u, v) represents the amplitude spectrum after thresholding, α represents the amplitude threshold, X hpf (u,v) represents the frequency domain feature map after high-pass filtering, e i·P(·) represents Euler's formula;

[0023]

[0024] ME(u,v)=|X hpf (u,v)|·(1+β·S(i,j));

[0025]

[0026] Where S(i,j) represents the pixel value at position (i,j) in the edge intensity image, R represents the domain size factor, p and q represent the offset relative to the current position (i,j) in the domain, ME(u,v) represents the amplitude enhanced by the edge intensity map, β represents the intensity factor, and X e (u,v) represents the frequency domain feature map after amplitude enhancement;

[0027]

[0028] Where, X fs (u,v) represents the frequency domain feature map after sharpening, and γ represents the sharpening factor;

[0029]

[0030] Where, X fs (i, j) represents the sharpened feature map obtained by two-dimensional discrete inverse Fourier transform, F -1 (·) denotes two-dimensional discrete inverse Fourier transform;

[0031] X DEIE (i,j)=[X up (i,j);X high (i,j);X fs (i,j)];

[0032] Where, X DEIE (i, j) represents the output of the dual-domain edge information extraction module.

[0033] As a preferred solution, the neck network based on the collaborative feature pyramid network includes four network layers, marked as P2, P3, P4, and P5 layers, respectively corresponding to the outputs of the first CPS module, the second CPS module, the third CPS module, and the fourth CPS module in the backbone network; the neck network processing process based on the collaborative feature pyramid network includes a front-end part and a back-end part;

[0034] In the front end of the collaborative feature pyramid network, the feature map output by the first CPS module in the backbone network (corresponding to Figure 1 The feature map corresponding to the C2 layer in the collaborative feature pyramid network is sent to the P3 layer after the linear deformable convolution of the P2 layer; the feature maps output by the second, third, and fourth CPS modules in the backbone network (corresponding to Figure 1 The feature maps corresponding to the C3, C4, and C5 layers in the neck network are first processed by the front-end CBS modules of the P3, P4, and P5 layers in the neck network for channel adjustment. The output of the front-end CBS module of the P5 layer is spliced ​​with the output of the front-end CBS module of the P4 layer through the upsampling module, and then used as the input of the CSP module of the P4 layer; the output of the CSP module of the P4 layer is spliced ​​with the output of the front-end CBS module of the P3 layer through the upsampling module, and then used as the input of the CSP module of the P3 layer; the output of the CSP module of the P3 layer is then spliced ​​with the output features of the linear deformable convolution of the P2 layer, and used as the input of the wide-area perception module, which captures target detail information and context information;

[0035] In the back-end part of the collaborative feature pyramid network, the output of the wide-area perception module is used as the input of the cross-stage convolution block of the P3 layer. The output of the cross-stage convolution block of the P3 layer is spliced ​​with the output of the front-end CBS module of the P4 layer and the output of the CSP module, and then used as the input of the cross-stage convolution block of the P4 layer; the output of the cross-stage convolution block of the P4 layer is spliced ​​with the output of the front-end CBS module of the P5 layer, and then used as the input of the CSP module of the P5 layer. The output of the CSP module of the P5 layer is used as the overall output of the collaborative pyramid network; wherein, the cross-stage convolution block includes a cascaded back-end CSP module and a back-end CBS module, and the back-end CSP modules of the cross-stage convolution blocks in the P3 and P4 layers also output the output feature maps of the corresponding network layers, which are combined with the output feature maps of the CSP module of the P5 layer. Figure 1 It serves as the overall output of the collaborative pyramid network and is passed to the probe network for UAV target image detection.

[0036] As a preferred solution, the processing process of the linear deformable convolution is shown by the following formula:

[0037]

[0038] P' n =P0+ΔP n ;

[0039] Where LDConv·) represents linear deformable convolution, P0 represents the spatial coordinate position of the feature map output by the first CPS module in the backbone network, N represents the number of parameters of the convolution kernel, and ω n represents the convolution kernel parameters, P' n represents the sampling coordinate after offset, x(P' n ) represents the characteristic value of the sampling coordinate after migration, ΔP n Indicates the offset.

[0040] As a preferred solution, the wide-area perception module includes a convolution splitting unit, a first 1×1 convolution layer, a parallel depth convolution unit, a second 1×1 convolution layer and a splicing layer connected in sequence; the parallel depth convolution unit includes a parallel 1×1 convolution layer, a 1×31 convolution layer, a 31×1 convolution layer and a 31×31 convolution layer;

[0041] In the wide-area perception module, the input features are processed through the convolution splitting unit, and the input feature channels are proportionally split into two branches using Split. One of the branches with a small number of channels is first processed by the first 1×1 convolution layer. Then, the feature map after the convolution operation is input to the parallel depth convolution unit, and the local features are captured using 1×1, 1×31 and 31×1 convolution kernels. The direction-sensitive features are captured through the asymmetric structure using the 31×31 large kernel strip convolution. The output of the parallel depth convolution unit is merged with the output of the first 1×1 convolution layer in the channel dimension, and then fused through the second 1×1 convolution layer. Finally, the fused features are spliced ​​with the other branch with a large number of channels through the splicing layer, and the output is used as the overall output of the wide-area perception module.

[0042] As a preferred solution, the detection head network includes two parallel one-to-many detection heads and one-to-one detection heads; in the training stage of the drone image target detection model, the model parameters are optimized and updated by the one-to-many detection heads and the one-to-one detection heads at the same time to obtain the trained drone image target detection model; in the inference stage of the drone image target detection model, the final target detection result is obtained through the output of the one-to-one detection head.

[0043] As a preferred solution, the trained drone image target detection model is specifically trained through the following steps:

[0044] UAV images with pre-annotated UAV target area positions are used as training samples to form a training sample set, which is input into the UAV image target detection model. Through two parallel one-to-many detection heads and one-to-one detection heads in the detection head network, the model parameters of the UAV image target detection model are optimized and updated with the goal of minimizing the total loss function composed of the regression loss function and the classification loss function, thereby training the UAV image target detection model.

[0045] Compared with the prior art, the present invention has the following technical effects:

[0046] (1) Compared with the traditional spatial convolution method, the frequency domain technology can more accurately distinguish the high-frequency features of the target edge and the low-frequency components of the background noise through the spectrum decomposition mechanism. The present invention utilizes the advantages of frequency domain technology in feature decoupling and combines it with the spatial domain to design a multi-scale dynamic dual-domain coupling module (MDDC). In this module, the feature map is converted into the frequency domain by Fourier transform through the dual-domain edge information extraction module. The spectrum separation characteristics of high-frequency components and low-frequency components are utilized, and the frequency domain analysis is used to realize the spectral space decoupling of multi-scale target edge features and background noise. The interference components are eliminated by selective frequency band suppression, thereby achieving accurate edge information extraction and noise robustness.

[0047] (2) In order to solve the problem that the feature pyramid in the existing neck network cannot represent the features of small targets and relies too much on redundant detection heads to cause computational overhead, the present invention proposes a collaborative feature pyramid network (SFPN). The network transmits high-level semantic information to the lower layer through a top-down path through a bidirectional path, thereby enhancing the context perception of small targets, and transmits low-level details to the upper layer through a bottom-up path, thereby improving the positioning accuracy of large targets. The neck network of the present invention does not perform feature fusion on the P2 layer, but only uses linear deformable convolution to enhance the extraction of detail information, which can make the small target information in the low-level features more prominent. In the P3 layer, the designed wide-area perception module is used to further capture the target's detail information and context information. In this way, SFPN can effectively fuse multi-scale context information, thereby enhancing the model's detection ability on targets of different sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to make the purpose, technical solutions and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0049] Figure 1 Schematic diagram of the architecture of the drone image target detection model used in the present invention;

[0050] Figure 2 Schematic diagram of the architecture of the dual-domain edge information extraction module used in the present invention;

[0051] Figure 3 Schematic diagram of the architecture of the collaborative feature pyramid network used in the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0053] The present invention will be described in further detail below with reference to the accompanying drawings.

[0054] Existing object detection in aerial imagery faces three core challenges: First, the target scale distribution is extremely uneven, with a high proportion of small targets and scarce feature information; second, complex background interference is severe, and dynamic environments and noise lead to high false detection rates in traditional methods; third, target geometric deformations are diverse, and multi-angle shooting by drones leads to prominent problems such as object rotation and occlusion. Existing technologies such as Faster R-CNN, YOLO and other general detection frameworks perform poorly when directly applied to aerial scenes, while improved methods based on context fusion, frequency domain enhancement or multi-scale feature pyramids (such as BiFPN) still have defects such as high computational overhead and easy loss of small target features. Especially in high-resolution aerial imagery, background noise and target edge spectrum overlap, making it difficult for traditional single-domain convolution to achieve accurate feature decoupling. Therefore, in response to the above-mentioned problems and shortcomings, the present invention proposes a UAV image target detection method based on collaborative feature fusion. This method addresses the dual challenges of the existing backbone network's insufficient multi-scale edge feature representation capability and limited background noise suppression efficiency, and proposes a Multi-Scale Dynamic Dual-Domain Coupling (MDDC) module. This module is integrated into the feature extraction stage of the backbone network, and breaks through the limitations of traditional single-domain convolution by constructing a frequency domain-spatial domain dual-drive architecture. In response to the problems of insufficient small target feature representation in the existing feature pyramid in the neck and the computational overhead introduced by over-reliance on redundant detection heads, a Synergistic Feature Pyramid Network (SFPN) is proposed. This method fuses multi-scale contextual information through linear deformable convolution and wide-area perception modules to enhance the model's detection capability on targets of different sizes.

[0055] Specifically, the method for drone image target detection based on collaborative feature fusion proposed in the present invention is to input the acquired drone image to be detected into a pre-trained drone image target detection model to obtain target detection results for the drone image to be detected. The drone image target detection model includes a backbone network integrated with a multi-scale dynamic dual-domain coupling module, a neck network based on a collaborative feature pyramid network, and a detection head network for performing target detection. The drone image target detection model uses the input drone image as the input of the backbone network, uses the multi-scale dynamic dual-domain coupling module to perform adaptive average pooling to generate a multi-scale feature map, and extracts edge information from the multi-scale feature map to obtain a dual-domain enhanced multi-scale feature map. The dual-domain enhanced multi-scale feature map is used as the input of the neck network, and the dual-domain enhanced multi-scale features are layer-by-layer fused through the collaborative feature pyramid network. The extraction of detail information is enhanced using linear deformable convolution. The target detail information and context information are then captured through a wide-area perception module to output further enhanced multi-scale fused features, which are input into the detection head network to obtain the drone image target detection results as the output of the drone image target detection model.

[0056] The following is a more detailed description of the UAV image target detection method based on collaborative feature fusion of the present invention.

[0057] 1. UAV Image Target Detection Model

[0058] This paper takes the convolutional neural network (CNN) as the basic structure, introduces a multi-scale processing mechanism and frequency domain edge feature extraction technology, and proposes a synergistic feature fusion network with dual-domain edge enhancement for UAV image object detection (SFFNet).

[0059] like Figure 1As shown, the drone image target detection model of the present invention includes a backbone network integrated with a multi-scale dynamic dual-domain coupling module, a neck network based on a collaborative feature pyramid network, and a detection head network for performing target detection; the drone image target detection model takes the input drone image as the input of the backbone network, uses the multi-scale dynamic dual-domain coupling module to perform adaptive average pooling to generate a multi-scale feature map, and extracts the edge information of the multi-scale feature map, thereby obtaining a dual-domain enhanced multi-scale feature map; the dual-domain enhanced multi-scale feature map is used as the input of the neck network, and the dual-domain enhanced multi-scale features are fused layer by layer through the collaborative feature pyramid network, and the extraction of detail information is enhanced by linear deformable convolution, and then the target detail information and context information are captured through the wide-area perception module to output the further enhanced multi-scale fusion features and input them into the detection head network to obtain the target detection result of the drone image as the output of the drone image target detection model.

[0060] The following is a detailed description of the component modules of the drone target detection model.

[0061] 1.1. Backbone network integrating multi-scale dynamic dual-domain coupling modules

[0062] The backbone network includes a Stem layer and four CPS modules connected in sequence; the Stem layer includes two cascaded CBS units, and the CPS module includes a cascaded CBS unit, multiple serial multi-scale dynamic dual-domain coupling modules and a splicing layer, wherein the CBS unit includes a cascaded convolutional layer, a BN layer and a SiLU activation function; in the CPS module, after the input feature map is extracted by the CBS unit, it is split into two branches according to the channel dimension through Split, and one of the branches is subjected to dual-domain edge enhancement through multiple serial multi-scale dynamic dual-domain coupling modules. The enhanced feature map is spliced ​​with the output of the first multi-scale dynamic dual-domain coupling module and the features of the other branch in the splicing layer, and the output is obtained as the output of the CSP module;

[0063] In the backbone network, the drone image as the input of the drone image target detection model is used as the input of the backbone network Stem layer. The output of the Stem layer is downsampled as the input of the first CPS module. The output of each CPS module is used as the input of the next layer in the backbone network. The feature map output by the last CPS module is used as the output of the backbone network. The size of the feature map output by the backbone network is H / 32×W / 32×C C5 , H, W are the height and width of the input drone image of the backbone network, C C5 The last CPS module in the backbone network ( Figure 1The number of channels of the feature map output by the C5 layer in the network is determined by the number of convolution kernels in the backbone network.

[0064] Compared with the traditional spatial convolution method, the frequency domain technology can more accurately distinguish the high-frequency features of the target edge from the low-frequency components of the background noise through the spectrum decomposition mechanism. The present invention takes advantage of the frequency domain technology in feature decoupling and combines it with the spatial domain to design a multi-scale dynamic dual-domain coupling module (MDDC). MDDC improves the model's ability to characterize the edge details of small targets through operations such as adaptive average pooling, dual-domain edge information extraction and convolution. Its structure is as follows: Figure 2 shown.

[0065] The multi-scale dynamic dual-domain coupling module includes an adaptive average pooling layer, a 1×1 convolution unit, a 3×3 grouped convolution unit, an upsampling layer, a dual-domain edge information extraction module and a splicing layer connected in sequence; wherein the 1×1 convolution unit includes a cascaded 1×1 convolution layer, a BN layer and a SiLU activation function, and the 3×3 grouped convolution unit includes a cascaded 3×3 convolution layer, a BN layer and a SiLU activation function; in the multi-scale dynamic dual-domain coupling module, two branches of feature extraction are performed on the input feature map, one of which is used as the input of the adaptive averaging layer, the output of the adaptive averaging layer is used as the input of the 1×1 convolution unit, the output of the 1×1 convolution unit is used as the input of the 3×3 grouped convolution unit, the output of the 3×3 grouped convolution unit is used as the input of the upsampling layer, the output of the upsampling layer is used as the input of the dual-domain edge information extraction module, the output of the dual-domain edge information extraction module and the output of the other branch after passing through the 3×3 grouped convolution unit and the upsampling layer are spliced ​​in the splicing layer, and the output is used as the overall output of the multi-scale dynamic dual-domain coupling module.

[0066] The processing of the dual-domain edge information extraction module (DEIE) includes the following steps:

[0067] The output feature map of the upsampling layer in the multi-scale dynamic dual-domain coupling module is used as the input of the dual-domain edge information extraction. The input feature map is subjected to feature extraction of two branches. One branch is smoothed by a 3×3 average pooling layer to calculate the low-frequency feature component, and the low-frequency feature component is subtracted from the input feature map to extract the high-frequency feature component; the other branch converts the spatial domain representation into the frequency domain representation by a two-dimensional discrete Fourier transform, and a high-pass filter is applied to the frequency domain feature map to retain the high-frequency information and suppress the low-frequency components to obtain the frequency domain feature map after high-pass filtering; the other branch is smoothed by a 3×3 average pooling layer to calculate the low-frequency feature component and subtract the low-frequency feature component from the input feature map to extract the high-frequency feature component. The high-frequency feature components are localized for contrast calculation to obtain an edge intensity map of the high-frequency feature components, and the edge intensity map is used as a guide to enhance the amplitude of the frequency domain feature map after high-pass filtering to obtain an amplitude-enhanced frequency domain feature map; the amplitude-enhanced frequency domain feature map is frequency-sharpened to obtain a sharpened frequency domain feature map; the sharpened frequency domain feature map is converted back to the spatial domain representation through a two-dimensional discrete inverse Fourier transform to obtain a sharpened feature map; the original input feature map, the high-frequency feature components and the sharpened feature map are spliced ​​and output as the overall output of the dual-domain edge information extraction module.

[0068] The multi-scale dynamic dual-domain coupling module processing process includes the following steps:

[0069] First, MDDC first receives the input feature map After adaptive average pooling operation, feature maps X of different scales are generated s , where s∈{3,6,9,12} represents the size of the feature map after downsampling, and its expression is:

[0070] X s =AAP s (X);

[0071] Where AAP s (·) represents the adaptive average pooling operation;

[0072] Next, the feature map of each scale passes through 1×1 convolution and 3×3 grouped convolution in turn; among them, 1×1 convolution compresses the number of channels of the feature map to one-fourth of the original to extract more concise features, and 3×3 grouped convolution adopts a channel decoupling strategy, which effectively maintains the channel specificity of the feature representation by performing independent spatial convolution operations on each channel. The representations are:

[0073]

[0074] Where, represents the output of 1×1 convolution, Represents the output of 3×3 grouped convolution;

[0075] Then, the feature map extracted by convolution After upsampling, the resolution of the original input feature map X is restored and the channels are aligned to obtain the multi-scale feature map X. up The edge information of the features is extracted by the dual-domain edge information extraction module;

[0076] In the dual-domain edge information extraction module, the forward propagation of the DEIE module first transforms the input feature map Then, the low-frequency components are calculated by smoothing through a 3×3 average pooling layer. up Subtract the low-frequency characteristic components from the high , and then further processed by a set of 1×1 convolutional layers, which is expressed as:

[0077]

[0078] Where, X high (i, j) represents the high-frequency feature component, X up (i, j) represents the input two-dimensional spatial domain feature map, i and j represent the row and column coordinates of the feature map in the spatial domain, m and n represent the offset within the window, and k represents the pooling kernel size;

[0079] On the other hand, in order to extract specific high-frequency components, the input feature map X up The two-dimensional discrete Fourier transform is used to convert the spatial domain into the frequency domain, and its expression is:

[0080]

[0081] Where, X up (u, v) represents the frequency domain feature map obtained by two-dimensional discrete Fourier transform of the original input feature map; F(·) represents the two-dimensional discrete Fourier transform; represents the kernel function of Fourier transform; represents phase rotation; M and N represent width and height in the spatial domain respectively; i represents the imaginary unit; and Represents the spatial variation of u and v frequencies of the feature map in the horizontal and vertical directions, respectively, and is used to modulate the pixel values ​​in the i and j directions;

[0082] In the frequency domain, a high-pass filter (HPF) is applied to retain high-frequency information and suppress low-frequency components. The specific operation is to reset the frequencies with amplitudes below the threshold to zero while retaining the original phase information to avoid distorting the overall structure of the image. Its expression is:

[0083]

[0084] M(X up (u,v)) and P(X up (u,v)) represent the frequency domain feature map X up The amplitude and phase of (u, v) at position (u, v), Re(·) and Im(·) represent the real and imaginary parts of the frequency domain feature map respectively, M hpf (u, v) represents the amplitude spectrum after thresholding, α represents the amplitude threshold, X hpf (u,v) represents the frequency domain feature map after high-pass filtering, e i·P(·) represents Euler's formula;

[0085] After processing the high-pass filtered frequency domain image X hpf (u,v), it is necessary to further adaptively enhance the amplitude of specific frequency components according to the edge strength of the image content. We take the high frequency component X high The edge strength map (StrengthMap) is used as a guide to enhance the amplitude while ensuring that the structural features of the reconstructed image are not destroyed. The edge strength map is obtained by high The local contrast is calculated to depict the intensity change of the local area of ​​the image; its expression is:

[0086]

[0087] ME(u,v)=|X hpf (u,v)|·(1+β·S(i,j));

[0088]

[0089] Where S(i, j) represents the pixel value at position (i, j) in the edge intensity image; R represents the domain size factor; p and q represent the offset relative to the current position (i, j) in the domain; ME(u, v) represents the amplitude enhanced by the edge intensity map; β represents the intensity factor, which is set to 1.5 in this embodiment; X e (u,v) represents the frequency domain feature map after amplitude enhancement;

[0090] For the adaptively enhanced frequency domain image X e Frequency sharpening (FS) is performed on (u, v) to reduce artifacts while increasing clarity. Its expression is:

[0091]

[0092] Where, X fs (u, v) represents the frequency domain feature map after sharpening, γ represents the sharpening factor, and in this embodiment, γ is set to 1.2;

[0093] The part after frequency domain sharpening X fs (u,v) is converted back to the spatial domain by two-dimensional discrete inverse Fourier transform to obtain the enhanced component X fs (i,j):

[0094]

[0095] Where, Represents the sharpened feature map obtained by two-dimensional discrete inverse Fourier transform, F -1 (·) denotes two-dimensional discrete inverse Fourier transform;

[0096] Finally, the original input image X is transformed into up , high frequency component X high and sharpening components Splicing output

[0097]

[0098] Where, X DEIE (i, j) represents the output of the dual-domain edge information extraction module.

[0099] 1.2 Neck Network Based on Collaborative Feature Pyramid Network

[0100] The neck network is constructed based on a collaborative feature pyramid network. The low-level features in the pyramid can provide more details and spatial information, which helps to improve the accuracy of small target detection. High-level features contain rich semantic information, which can help the model identify object categories and contextual environments. Therefore, it is crucial to make full use of features at multiple scales so that the model can extract information from different levels and thus reduce the cross-level gap. In order to retain more low-level feature information in the fusion stage and effectively weigh the contributions of features at different levels, the present invention proposes an efficient collaborative feature pyramid network (SFPN) as the neck network of the drone image target detection model.

[0101] The collaborative feature pyramid network (SFPN) includes four network layers, labeled P2, P3, P4, and P5, corresponding to the outputs of the first, second, third, and fourth CPS modules in the backbone network (i.e., the outputs of the C2, C3, C4, and C5 layers in the backbone network). SFPN achieves efficient feature expression and enhancement by fusing multi-scale features layer by layer. In the neck network based on the collaborative feature pyramid network, the dual-domain enhanced multi-scale feature map output by the backbone network is input into the collaborative feature pyramid network for layer-by-layer fusion. Specifically, the neck network processing process based on the collaborative feature pyramid network includes a front-end part and a back-end part.

[0102] In the front end of the collaborative feature pyramid network, the feature map output by the first CPS module in the backbone network (corresponding to Figure 1 The feature map corresponding to the C2 layer in the collaborative feature pyramid network is sent to the P3 layer after the linear deformable convolution of the P2 layer; the feature maps output by the second, third, and fourth CPS modules in the backbone network (corresponding to Figure 1 The feature maps corresponding to the C3, C4, and C5 layers in the neck network are first processed by the front-end CBS modules of the P3, P4, and P5 layers in the neck network for channel adjustment. The output of the front-end CBS module of the P5 layer is spliced ​​with the output of the front-end CBS module of the P4 layer through the upsampling module, and then used as the input of the CSP module of the P4 layer; the output of the CSP module of the P4 layer is spliced ​​with the output of the front-end CBS module of the P3 layer through the upsampling module, and then used as the input of the CSP module of the P3 layer; the output of the CSP module of the P3 layer is then spliced ​​with the output features of the linear deformable convolution of the P2 layer, and used as the input of the wide-area perception module, which captures target detail information and context information;

[0103] In the back-end part of the collaborative feature pyramid network, the output of the wide-area perception module is used as the input of the cross-stage convolution block of the P3 layer. The output of the cross-stage convolution block of the P3 layer is spliced ​​with the output of the front-end CBS module of the P4 layer and the output of the CSP module, and then used as the input of the cross-stage convolution block of the P4 layer; the output of the cross-stage convolution block of the P4 layer is spliced ​​with the output of the front-end CBS module of the P5 layer, and then used as the input of the CSP module of the P5 layer. The output of the CSP module of the P5 layer is used as the overall output of the collaborative pyramid network; wherein, the cross-stage convolution block includes a cascaded back-end CSP module and a back-end CBS module, and the back-end CSP modules of the cross-stage convolution blocks in the P3 and P4 layers also output the output feature maps of the corresponding network layers, which are combined with the output feature maps of the CSP module of the P5 layer. Figure 1 It serves as the overall output of the collaborative pyramid network and is passed to the probe network for UAV target image detection.

[0104] In the neck network layer, no feature fusion is performed in the P2 layer, and only linear deformable convolution is used to enhance the extraction of detail information, thereby making the small target information in the low-level features more prominent; in the P3 layer, the designed wide-area perception module is used to further capture the target's detailed information and contextual information.

[0105] The linear deformable convolution and wide-area perception modules in the collaborative feature pyramid network are described in detail below.

[0106] 1.2.1 Linear Deformable Convolution

[0107] From a drone's perspective, the shapes of objects are often diverse. This is particularly true during object detection tasks, where the shapes can change dramatically due to factors such as perspective, distance, rotation, and scale. Traditional convolution operations are typically based on fixed convolution kernels, whose size and shape are shared across the entire image or feature map. This fixed nature limits convolution in capturing shape variations. When the shape of an object changes, traditional convolution often struggles to effectively adapt to these changes, resulting in reduced detection accuracy for complex objects.

[0108] Linear deformable convolution dynamically adjusts the sampling position of the convolution kernel, making the convolution operation more flexible and adaptively capturing changes in target shape, effectively improving target detection accuracy. Furthermore, this convolution excels at reducing computational complexity by correcting the growth trend of the number of parameters to a linear growth, making the convolution operation more efficient. Therefore, with the help of this convolution, drone vision systems can more accurately identify variable target shapes and are computationally more efficient, adapting to the needs of real-world drone scenarios.

[0109] Based on the above advantages and combined with the distribution trend of feature details in the feature pyramid, this paper applies Linear Deformable Convolution (LDConv) at the bottom of the top-down path aggregation of SFPN, that is, the P2 feature layer. This design can extract more information about small objects from low-level features, thereby enhancing the spatial expressiveness and detail capture capabilities of features.

[0110] Specifically, the convolution first generates the initial sampling coordinates P based on the number of convolution kernel parameters N n , and its sampling grid set is R={(x1,y1),(x2,y2),…,(x N ,y N )}; Then, the offset ΔP is calculated by convolution operation n and the initial sampling coordinates P n Add together to generate the offset sampling coordinates P' n , to achieve dynamic adjustment of the sampling position; finally, the features are interpolated and resampled at these new sampling positions, and the convolution operation is used to extract the features. The processing process is shown in the following formula:

[0111]

[0112] P′ n =P0+ΔP n ;

[0113] Where LDCOnv·) represents linear deformable convolution, P0 represents the spatial coordinate position of the feature map output by the first CPS module in the backbone network, N represents the number of parameters of the convolution kernel, ω n represents the convolution kernel parameters, P' n represents the sampling coordinate after offset, x(P' n ) represents the characteristic value of the sampling coordinate after migration, ΔP n Indicates the offset.

[0114] In this way, the model can extract more detailed information from low-level features and pass this information to the P3 feature layer for fusion, while achieving a linear growth in the number of parameters and supporting irregular sampling shapes, providing a better balance between flexibility and performance.

[0115] 1.2.2 Wide-Area Perception Module

[0116] Accurate object detection requires extensive contextual information, and different objects have varying contextual requirements. Traditional small-scale convolutional kernels are limited by their local receptive field, making it difficult to model long-range spatial dependencies. This is especially true when dealing with objects with large aspect ratios or that are sensitive to orientation. Large strip convolutions are good feature representation learners for remote object detection, and using a few large convolutional kernels rather than a stack of small kernels may be a more powerful paradigm.

[0117] Based on this, the present invention designs a wide-area perception module (WPM) at the feature pyramid P3 layer that combines the complementary advantages of large kernel convolution and strip convolution, so as to fully utilize the unique prior knowledge presented in the aerial photography scene. The design details of WPM are as follows: Figure 3 As shown in Figure 2. Based on this practical consideration, feature maps near the lower layers of the feature pyramid are larger in size, which can easily lead to an increase in computational complexity. In a specific application embodiment, only one-quarter of the feature channels are fed into a set of parallel depthwise convolutions to reduce computational complexity.

[0118] The wide-area perception module includes a sequentially connected convolution splitting unit, a first 1×1 convolution layer, a parallel depth convolution unit, a second 1×1 convolution layer, and a splicing layer; the parallel depth convolution unit includes a parallel 1×1 convolution layer, a 1×31 convolution layer, a 31×1 convolution layer, and a 31×31 convolution layer;

[0119] In the wide-area perception module, the input features are split into two parts in proportion using the convolution splitting unit. and Two branches, The branch is first processed by the first 1×1 convolution layer. Then, the feature map after the convolution operation is input to the parallel depth convolution unit, and the 1×1, 1×31 and 31×1 convolution kernels are used to capture local features. The 31×31 large kernel strip convolution is used to capture the direction sensitivity feature through the asymmetric structure. The output of the parallel depth convolution unit is merged with the output of the first 1×1 convolution layer in the channel dimension, and then fused through the second 1×1 convolution layer. Finally, the fused features are combined with The branches are then spliced ​​together in the splicing layer, and the output is used as the overall output of the wide-area perception module.

[0120] 1.3 Detection Head Network

[0121] In the present invention, the detection head network includes two parallel one-to-many detection heads and one-to-one detection heads, which are used to decode the semantic category and spatial coordinates of the target; in the training stage of the drone image target detection model, the model parameters are optimized and updated by the one-to-many detection heads and the one-to-one detection heads at the same time to obtain the trained drone image target detection model; in the inference stage of the drone image target detection model, the final target detection result is obtained through the output of the one-to-one detection head.

[0122] 2. Training of drone image target detection model

[0123] The drone image target detection model in this invention is trained through the following steps:

[0124] UAV images with pre-annotated target area locations are used as training samples to form a training sample set, which is input into the UAV image target detection model. The model parameters of the UAV image target detection model are optimized and updated by two parallel one-to-many detection heads and one-to-one detection heads in the detection head network with the goal of minimizing the total loss function composed of the regression loss function and the classification loss function, thereby training the UAV image target detection model; wherein the classification loss function adopts a binary cross entropy loss function, and the regression loss function adopts an overall loss function composed of a CIoU loss function and a DFL loss function;

[0125] The total loss function is expressed as:

[0126] Loss=λ1·L cls +λ2·L CIoU +λ3·L DFL ,

[0127] Among them, Loss represents the total loss; L cls represents the binary cross entropy loss, L CIoU represents the CIoU loss, L DFLrepresents DFL loss; λ1, λ2, and λ3 represent the corresponding binary cross entropy loss L cls , CIoU loss L CIoU 、DFL loss L DFL The preset weight coefficient; in this embodiment, λ1, λ2, and λ3 can be set to 0.5, 7.5, and 1.5 respectively;

[0128] Among them, the binary cross entropy loss function is expressed as:

[0129]

[0130] Where, L DFL represents the binary cross entropy loss function, t i represents the true category label of sample i, f i Represents the probability of the i-th sample predicted by the model.

[0131] The CIoU loss function is expressed as:

[0132]

[0133] Among them, L CIoU represents CIoU loss; Represents the center point b of the target area positioning frame predicted by the model and the center point b of the real target area positioning frame gt c represents the diagonal length of the minimum bounding rectangle covering the predicted target region positioning frame and the true target region positioning frame; α is the preset weight coefficient; v is the consistency parameter used to measure the difference in aspect ratio between the predicted target region positioning frame and the true target region positioning frame, and w and h are the width and height of the predicted target area positioning box respectively, w gt 、h gt are the width and height of the predicted target area positioning box respectively; IoU represents the ratio of the intersection area to the union area of ​​the predicted target area positioning box and the true target area positioning box, that is:

[0134]

[0135] Among them, B, B gt They represent the predicted target area positioning box and the real target area positioning box respectively, and Area(·) represents area operation.

[0136] The DFL loss function is expressed as:

[0137]

[0138] Where, L DFL represents the DFL loss function; y represents the true continuous target value; Indicates that the model needs to predict the probability distribution of all discrete positions; y i and y i+1 Respectively represent the two integer boundaries adjacent to the true value y after discretization; and Respectively represent the discretized probability distribution of the model prediction, corresponding to y i and y i+1 The probability value of .

[0139] 3. Examples

[0140] In order to verify the effectiveness of the drone image target detection method based on collaborative feature fusion of the present invention, this example shows the quantitative results of the detection performance of some of the most advanced methods and the drone image target detection model proposed in this example on the VisDrone validation set.

[0141] As shown in Table 1, the red part represents the best result achieved under this indicator, the green part represents the second best result, and the gray part represents the third place result. “-” indicates that the result is not reported or not publicly available. It can be seen that the model of this embodiment surpasses the existing methods in most evaluation indicators. It is particularly important to point out that traditional general target detection frameworks, such as Faster-RCNN and Cascade-RCNN, are limited by the dense distribution of small targets and scale imbalance characteristics unique to aerial images, and their AP are only 24.8% and 22.6% respectively. In contrast, methods based on density map cropping (such as NE-CDMNet) have significantly improved detection performance, with an AP of 35.9%. In addition, the results also show that the SFFNet model of this embodiment has made significant progress in the detection of small and medium objects, while the detection performance of large objects has been improved less. Among the top three detection results, the series of models of this embodiment almost occupy most positions.

[0142] Table 1 Comparison of the model in this embodiment with the most advanced technical methods on the VisDrone validation set

[0143]

[0144] Table 2 Comparison of the model of this embodiment with the most advanced technical methods on the UAVDT dataset

[0145]

[0146] At the same time, the UAVDT dataset contains a large number of small targets and covers many images with low light and complex backgrounds. These features can more realistically reflect the performance of the network in small object detection tasks. As shown in Table 2, the bold part represents the best results achieved under this indicator. "-" indicates that the results are not reported or not publicly available. It can be seen that compared with the baseline YOLOv10-X, the method of this embodiment improves on AP50 by 2.1%, surpassing many existing state-of-the-art methods. Compared with the latest method SCLNet, the detection performance of this embodiment is improved by 0.6% and 1.3% on AP and AP50 indicators respectively. These results show that the method of this embodiment performs well in the task of target detection in drone images.

[0147] Experimental results on two challenging aerial datasets (VisDrone and UAVDT) show that SFFNet-X achieved excellent scores of 36.8AP and 20.6AP on these datasets, respectively. At the same time, the lightweight version (N / S) maintains the dual advantages of accuracy and parameter count among similar detectors, demonstrating its excellent performance. SFFNet can stably maintain high accuracy in complex environments and shows strong adaptability under challenging conditions such as low light and dynamic backgrounds. Qualitative analysis also verifies the superiority of SFFNet. Compared with the baseline method, it can effectively suppress low-frequency noise interference and show stronger robustness when dealing with multi-scale targets.

[0148] 4. Overview

[0149] In summary, this embodiment discloses a method for drone image target detection based on collaborative feature fusion, proposing an innovative end-to-end model, SFFNet, specifically designed for drone aerial photography target detection tasks. First, this embodiment addresses the issues of limited target feature expression and background noise interference by proposing an MDDC module. This module effectively addresses the challenges of inaccurate target feature extraction and the impact of noise interference on detection results by adaptively modeling dual-domain edge features in information streams at different scales. The introduction of the MDDC module greatly improves feature extraction accuracy, enabling the model to better capture target details and structural information, thereby enhancing target detection accuracy. Second, this embodiment also explores the potential of feature pyramids in processing target geometry and long-range contextual information. By creating a SFPN, collaborative modeling of multi-scale targets is achieved. The SFPN can effectively integrate multi-scale contextual information, thereby enhancing the model's detection capabilities for targets of different sizes. Through the MDDC module and SFPN, this embodiment addresses the technical issues of low feature extraction accuracy for small targets in drone aerial images, severe background noise interference, and insufficient multi-scale target fusion, significantly improving detection accuracy and robustness.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described with reference to the preferred embodiments of the present invention, it should be understood by those skilled in the art that various changes can be made in form and details without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for target detection in UAV images based on collaborative feature fusion, characterized in that: Obtain the drone image to be detected, input it into a pre-trained drone image target detection model, and obtain the target detection result of the drone image to be detected; The drone image target detection model includes a backbone network integrated with a multi-scale dynamic dual-domain coupling module, a neck network based on a collaborative feature pyramid network, and a detection head network for performing target detection; the drone image target detection model uses the input drone image as the input of the backbone network, uses the multi-scale dynamic dual-domain coupling module to perform adaptive average pooling to generate a multi-scale feature map, and extracts the edge information of the multi-scale feature map, thereby obtaining a dual-domain enhanced multi-scale feature map; the dual-domain enhanced multi-scale feature map is used as the input of the neck network, the dual-domain enhanced multi-scale features are fused layer by layer through the collaborative feature pyramid network, and the extraction of detail information is enhanced by linear deformable convolution, and the target detail information and context information are captured through the wide-area perception module to output the further enhanced multi-scale fusion features and input them into the detection head network to obtain the target detection result of the drone image as the output of the drone image target detection model.

2. The method for target detection in UAV images based on collaborative feature fusion according to claim 1, characterized in that: The backbone network includes a Stem layer and four CPS modules connected in sequence; the Stem layer includes two cascaded CBS units, and the CPS module includes a cascaded CBS unit, multiple serial multi-scale dynamic dual-domain coupling modules and a splicing layer, wherein the CBS unit includes a cascaded convolutional layer, a BN layer and a SiLU activation function; in the CPS module, after the input feature map is extracted by the CBS unit, it is split into two branches according to the channel dimension by Split, and one of the branches is subjected to dual-domain edge enhancement by multiple serial multi-scale dynamic dual-domain coupling modules. The enhanced feature map is spliced ​​with the output of the first multi-scale dynamic dual-domain coupling module and the features of the other branch in the splicing layer, and the output is obtained as the output of the CSP module; In the backbone network, the drone image, which is the input of the drone image target detection model, is used as the input of the backbone network Stem layer. The output of the Stem layer is downsampled as the input of the first CPS module. The output of each CPS module is used as the input of the next layer in the backbone network. The feature map output by the last CPS module is used as the output of the backbone network.

3. The method for target detection in UAV images based on collaborative feature fusion according to claim 2, characterized in that: The multi-scale dynamic dual-domain coupling module includes an adaptive average pooling layer, a 1×1 convolution unit, a 3×3 grouped convolution unit, an upsampling layer, a dual-domain edge information extraction module, and a splicing layer connected in sequence; wherein the 1×1 convolution unit includes a cascaded 1×1 convolution layer, a BN layer, and a SiLU activation function, and the 3×3 grouped convolution unit includes a cascaded 3×3 convolution layer, a BN layer, and a SiLU activation function; In the multi-scale dynamic dual-domain coupling module, feature extraction of two branches is performed on the input feature map respectively, one of which is used as the input of the adaptive averaging layer, the output of the adaptive averaging layer is used as the input of the 1×1 convolution unit, the output of the 1×1 convolution unit is used as the input of the 3×3 grouped convolution unit, the output of the 3×3 grouped convolution unit is used as the input of the upsampling layer, and the output of the upsampling layer is used as the input of the dual-domain edge information extraction module. The output of the dual-domain edge information extraction module is spliced ​​with the output of the other branch after passing through the 3×3 grouped convolution unit and the upsampling layer in the splicing layer, and the output is used as the overall output of the multi-scale dynamic dual-domain coupling module.

4. The method for target detection in UAV images based on collaborative feature fusion according to claim 3, characterized in that: The processing process of the dual-domain edge information extraction module includes the following steps: The output feature map of the upsampling layer in the multi-scale dynamic dual-domain coupling module is used as the input of the dual-domain edge information extraction, and the input feature map is subjected to feature extraction of two branches, one of which is smoothed by a 3×3 average pooling layer to calculate the low-frequency feature component, and the low-frequency feature component is subtracted from the input feature map to extract the high-frequency feature component; the other branch converts the spatial domain representation into the frequency domain representation by a two-dimensional discrete Fourier transform, and applies a high-pass filter to the frequency domain feature map to retain the high-frequency information and suppress the low-frequency component to obtain the frequency domain feature map after high-pass filtering; the high-frequency feature component obtained by the other branch is subjected to local contrast calculation to obtain the edge intensity map of the high-frequency feature component, and the edge intensity map is used as a guide to enhance the amplitude of the frequency domain feature map after high-pass filtering to obtain the frequency domain feature map after amplitude enhancement; the frequency domain feature map after amplitude enhancement is subjected to frequency domain sharpening to obtain the sharpened frequency domain feature map; The sharpened frequency domain feature map is converted back to the spatial domain representation through a two-dimensional discrete inverse Fourier transform to obtain the sharpened feature map; the original input feature map, the high-frequency feature component and the sharpened feature map are spliced ​​together and output as the overall output of the dual-domain edge information extraction module.

5. The method for target detection in UAV images based on collaborative feature fusion according to claim 4, characterized in that: The processing process of the dual-domain edge information extraction module is shown in the following formula: Where, X high (i, j) represents the high-frequency feature component, X up (i, j) represents the input two-dimensional spatial domain feature map, i and j represent the row and column coordinates of the feature map in the spatial domain, m and n represent the offset within the window, and k represents the pooling kernel size; Where, X up (u,v) represents the frequency domain feature map obtained by two-dimensional discrete Fourier transform of the original input feature map, F(·) represents the two-dimensional discrete Fourier transform, represents the kernel function of Fourier transform, represents phase rotation, M and N represent width and height in the spatial domain respectively, i represents the imaginary unit, and denotes the spatial variation of u and v frequencies of the feature map in the horizontal and vertical directions, respectively; M(X up (u,v)) and P(X up (u,v)) represent the amplitude and phase of the frequency domain feature map Xup(u,v) at position (u,v), Re(·) and Im(·) represent the real and imaginary parts of the frequency domain feature map, respectively. hpf (u, v) represents the amplitude spectrum after thresholding, α represents the amplitude threshold, X hpf (u,v) represents the frequency domain feature map after high-pass filtering, e i·P(·) represents Euler's formula; ME(u,v)=|X hpf (u,v)|·(1+β·S(i,j)); Where S(i,j) represents the pixel value at position (i,j) in the edge intensity image, R represents the domain size factor, p and q represent the offset relative to the current position (i,j) in the domain, ME(u,v) represents the amplitude enhanced by the edge intensity map, β represents the intensity factor, and X e (u,v) represents the frequency domain feature map after amplitude enhancement; Where, X fs (u,v) represents the frequency domain feature map after sharpening, and γ represents the sharpening factor; Where, X fs (i, j) represents the sharpened feature map obtained by two-dimensional discrete inverse Fourier transform, F -1 (·) denotes two-dimensional discrete inverse Fourier transform; X DEIE (i,j)=[X up (i,j);X high (i,j);X fs (i,j)]; Where, X DEIE (i, j) represents the output of the dual-domain edge information extraction module.

6. The method for target detection in UAV images based on collaborative feature fusion according to claim 2, characterized in that: The neck network based on the collaborative feature pyramid network includes four network layers, marked as P2, P3, P4, and P5 layers, respectively corresponding to the outputs of the first CPS module, the second CPS module, the third CPS module, and the fourth CPS module in the backbone network; the neck network processing process based on the collaborative feature pyramid network includes a front-end part and a back-end part; In the front-end part of the collaborative feature pyramid network, the feature map output by the first CPS module in the backbone network (corresponding to the feature map corresponding to the C2 layer in Figure 1) is sent to the P3 layer after passing through the linear deformable convolution in the P2 layer of the collaborative feature pyramid network; the feature maps output by the second, third, and fourth CPS modules in the backbone network (corresponding to the feature maps corresponding to the C3, C4, and C5 layers in Figure 1) are first respectively processed by the front-end CBS modules of the P3, P4, and P5 layers in the neck network for channel adjustment. The output of the front-end CBS module of the P5 layer is spliced ​​with the output of the front-end CBS module of the P4 layer through the upsampling module and then used as the input of the CSP module of the P4 layer; The output of the P4 layer CSP module is spliced ​​with the output of the P3 layer front-end CBS module through the upsampling module and then used as the input of the P3 layer CSP module; The output of the CSP module in the P3 layer is then concatenated with the output features of the linear deformable convolution in the P2 layer as the input of the wide-area perception module, which captures target details and context information. In the back-end part of the collaborative feature pyramid network, the output of the wide-area perception module is used as the input of the cross-stage convolution block of the P3 layer. The output of the cross-stage convolution block of the P3 layer is spliced ​​with the output of the front-end CBS module of the P4 layer and the output of the CSP module, and then used as the input of the cross-stage convolution block of the P4 layer; the output of the cross-stage convolution block of the P4 layer is spliced ​​with the output of the front-end CBS module of the P5 layer, and then used as the input of the CSP module of the P5 layer. The output of the CSP module of the P5 layer is used as the overall output of the collaborative pyramid network; wherein, the cross-stage convolution block includes a cascaded back-end CSP module and a back-end CBS module, and the back-end CSP module of the cross-stage convolution block in the P3 and P4 layers also outputs the output feature map of the corresponding network layer respectively, which together with the output feature map of the CSP module of the P5 layer are used as the overall output of the collaborative pyramid network and passed to the probe network for UAV target image detection.

7. The method for target detection in UAV images based on collaborative feature fusion according to claim 6, characterized in that: The processing process of the linear deformable convolution is shown by the following formula: P’ n =P0+ΔP n ; Where LDConv(·) represents linear deformable convolution, P0 represents the spatial coordinate position of the feature map output by the first CPS module in the backbone network, N represents the number of parameters of the convolution kernel, and ω n represents the convolution kernel parameters, P ' n represents the sampling coordinate after offset, x(P' n ) represents the characteristic value of the sampling coordinate after migration, ΔP n Indicates the offset.

8. The method for target detection in UAV images based on collaborative feature fusion according to claim 6, characterized in that: The wide-area perception module includes a convolution splitting unit, a first 1×1 convolution layer, a parallel depth convolution unit, a second 1×1 convolution layer, and a splicing layer connected in sequence; the parallel depth convolution unit includes a parallel 1×1 convolution layer, a 1×31 convolution layer, a 31×1 convolution layer, and a 31×31 convolution layer; In the wide-area perception module, the input features are processed through the convolution splitting unit, and the input feature channels are proportionally split into two branches using Split. One of the branches with a small number of channels is first processed by the first 1×1 convolution layer. Then, the feature map after the convolution operation is input to the parallel depth convolution unit, and the local features are captured using 1×1, 1×31 and 31×1 convolution kernels. The direction-sensitive features are captured through the asymmetric structure using the 31×31 large kernel strip convolution. The output of the parallel depth convolution unit is merged with the output of the first 1×1 convolution layer in the channel dimension, and then fused through the second 1×1 convolution layer. Finally, the fused features are spliced ​​with the other branch with a large number of channels through the splicing layer, and the output is used as the overall output of the wide-area perception module.

9. The method for target detection in UAV images based on collaborative feature fusion according to claim 1, characterized in that: The detection head network includes two parallel one-to-many detection heads and one-to-one detection heads. During the training phase of the drone image target detection model, the model parameters are optimized and updated simultaneously through the one-to-many detection heads and the one-to-one detection heads to obtain the trained drone image target detection model. During the inference phase of the drone image target detection model, the final target detection result is obtained through the output of the one-to-one detection head.

10. The method for target detection in UAV images based on collaborative feature fusion according to claim 9, characterized in that: The trained drone image target detection model is specifically trained through the following steps: UAV images with pre-annotated target area locations are used as training samples to form a training sample set, which is input into the UAV image target detection model. The model parameters of the UAV image target detection model are optimized and updated by two parallel one-to-many detection heads and one-to-one detection heads in the detection head network with the goal of minimizing the total loss function composed of the regression loss function and the classification loss function, thereby training the UAV image target detection model; wherein the classification loss function adopts a binary cross entropy loss function, and the regression loss function adopts an overall loss function composed of a CIoU loss function and a DFL loss function; The total loss function is expressed as: Loss=λ1·L cls +λ2·L CIoU +λ3·L DFL , Among them, Loss represents the total loss; L cls represents the binary cross entropy loss, L CIoU represents the CIoU loss, L DFL represents DFL loss; λ1, λ2, and λ3 represent the corresponding binary cross entropy loss L cls , CIoU loss L CIoU 、DFL loss L DFL The preset weight coefficient of .

Images