Unmanned aerial vehicle target detection method based on multi-modal feature fusion and wavelet adaptive weighting

Abstract

The application relates to the field of target intelligent detection, and discloses an unmanned aerial vehicle target detection method based on multi-modal feature fusion and wavelet adaptive weighting, which comprises the following steps: acquiring multi-modal image data of an unmanned aerial vehicle and carrying out pretreatment; inputting the pretreated multi-modal image data of the unmanned aerial vehicle into a double-main-stem YOLOv11 target detection network; carrying out feature down-sampling and purification treatment according to a wavelet adaptive weighting down-sampling module of an infrared main-stem network branch; acquiring multi-modal fusion features according to a deformable fusion module; optimizing parameters of a detection head according to an NWD loss function; and acquiring target detection results according to the multi-modal fusion features and carrying out detection evaluation. The application improves the small target detection precision and robustness of the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of intelligent target detection, and in particular to a method for UAV target detection using multimodal feature fusion and wavelet adaptive weighting. Background Technology

[0002] With the booming low-altitude economy and the increasing maturity of UAV technology, UAV vision systems are playing an irreplaceable role in traffic patrol, agricultural monitoring, disaster relief, and security reconnaissance. Target detection, as the core fundamental task of UAV vision systems, aims to accurately locate and identify targets of interest (such as pedestrians and vehicles) from aerial images. However, compared to conventional ground views, targets viewed from a wide angle by a UAV typically occupy only a few pixels, making them typical small targets.

[0003] Traditional machine learning algorithms are no longer sufficient to meet the current needs of drones for detecting small targets. In recent years, single-stage convolutional neural networks, represented by the YOLO series of algorithms, have made significant progress in the field of general target detection. However, conventional convolutional neural networks have architectural bottlenecks when processing small targets on drones: as the number of network layers increases and continuous downsampling operations are performed, the limited shallow spatial details and high-frequency texture information of small targets are severely lost, resulting in a lack of high-resolution representations required for locating small targets in deep feature maps, leading to serious missed detections and false detections. In addition, the perspective tilt and motion blur generated by drones during dynamic flight further exacerbate the difficulty of extracting features from small targets. Traditional vision algorithms and conventional deep learning models are prone to serious missed detections and false detections when dealing with such scenarios. Furthermore, in actual operation, drones often face low light at night, strong backlight, rain and fog, and complex background interference. The image data acquired by traditional single-modal visible light cameras in these environments contains a lot of noise, causing vision algorithms to completely fail. In order to overcome the perception barrier of single-modality, the academic and industrial communities have widely adopted a scheme of joint detection using visible light and infrared multimodal sensors.

[0004] Therefore, designing a UAV target detection method to avoid the influence of complex environments and improve the detection accuracy and stability of small targets has become an urgent problem to be solved. Summary of the Invention

[0005] Based on this, the present invention proposes a UAV target detection method based on multimodal feature fusion and wavelet adaptive weighting, which solves the technical problems of poor environmental adaptability of single-modal models, loss of high-frequency features of small targets due to conventional convolutional downsampling, and easy feature aliasing in existing pre / mid-stage fusion strategies. By using multimodal feature fusion and wavelet adaptive weighting, the method reduces information loss in feature propagation, effectively preserves key features of small targets, and improves the accuracy and robustness of UAV small target detection.

[0006] This invention proposes a multimodal feature fusion and wavelet adaptive weighting method for UAV target detection, comprising: Acquire and preprocess multimodal image data of UAVs, including visible light image data and infrared image data, and the preprocessing includes cropping and enhancement processing; The preprocessed UAV multimodal image data is input into a dual-backbone YOLOv11 target detection network. The dual-backbone YOLOv11 target detection network is based on a dual-backbone network structure, which includes two independent backbone network branches. The backbone network branches include a visible light backbone network branch and an infrared backbone network branch. The visible light backbone network branch and the infrared backbone network branch extract visible light image features and infrared image features, respectively. Feature downsampling and purification are performed using the wavelet adaptive weighted downsampling module of the infrared backbone network branch. The wavelet adaptive weighted downsampling module is set in the fourth, sixth and eighth layers of the infrared backbone network branch. Multimodal fusion features are obtained according to the deformable fusion module, which is set between the dual backbone network and the neck network. The deformable fusion module guides the deformable convolution operation of visible light image features based on infrared image features. The detection head parameters are optimized based on the NWD loss function, and the target detection results are obtained and evaluated based on the multimodal fusion features.

[0007] Furthermore, the step of performing feature downsampling and purification processing based on the wavelet adaptive weighted downsampling module of the infrared backbone network branch specifically includes: The wavelet adaptive weighted downsampling module includes a frequency domain decoupling unit, a basic weight initialization unit, an adaptive weight generation unit, and a splicing and fusion unit; The frequency domain decoupling unit employs a Haar wavelet decomposition structure to perform two-dimensional discrete wavelet decomposition on the input image, outputting one low-frequency component and three high-frequency components. The specific operation is as follows: By using two-dimensional Haar wavelet transform, the local neighborhood of the input image is sequentially processed by difference and smoothing operations. Spatial downsampling is completed without feature energy loss, and the input image is decoupled into low-frequency components and high-frequency components. The low-frequency components represent global structural information, and the high-frequency components include horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components. The horizontal, vertical, and diagonal high-frequency components respectively include edge texture details in the horizontal, vertical, and diagonal directions. The basic weight initialization unit consists of an initial weight assignment module and a low-frequency feature re-determination module, which is used to construct initial weights for the decoupled low-frequency and high-frequency components, and to perform background noise filtering and feature purification on the low-frequency components. The adaptive weight generation unit consists of a dual-path pooling module, a weight generation module, and a feature weighting layer. It is used to adaptively generate feature weights for low-frequency and high-frequency components. The specific operation is as follows: The dual-path pooling module inputs three high-frequency components in parallel into the dual-path pooling layer for spatial dimension feature compression and statistical processing to obtain a fused high-frequency feature vector. The specific algorithm of the dual-path pooling module is as follows: , , , in, This represents the result of average pooling, where H and W represent the height and width of the feature map, respectively, and i and j represent the row and column numbers of the feature map, respectively. This represents the input characteristics of the dual-path pooling module. This represents the result of max pooling, where max represents the max pooling operation. This represents the fused high-frequency feature vector output by the dual-path pooling layer. This indicates a channel splicing operation; The weight generation module sequentially inputs the fused high-frequency feature vector into a dimensionality-reducing convolutional layer, a group normalization layer, and a nonlinear activation function to compress the channel dimension and extract the nonlinear dependencies between high-frequency channels. Then, it performs a linear mapping on the nonlinearly activated feature vector to project it to three output branches, generating unnormalized response values ​​for the corresponding low-frequency and high-frequency components. The activation function normalizes the unnormalized response value to generate adaptive normalized weights, which range from 0 to 1. The specific algorithm for the weight generation module is as follows: , , , in, ReLU represents the feature vector of the intermediate hidden layer after compression and linear transformation, and ReLU represents the nonlinear activation function. This indicates a group normalization operation. This represents a dimension reduction convolution operation. This represents the unnormalized response value of the k-th frequency branch. This represents the length of the feature vector of the intermediate hidden layer, and c represents the element ordinal number of the feature vector of the intermediate hidden layer. Indicates the mapping weights. This represents the c-th element of the feature vector of the intermediate hidden layer. This represents the bias function. represents adaptive normalized weights, and sigmoid represents the sigmoid activation function; The feature weighting layer performs feature weighting on high-frequency components according to adaptive normalization weights to obtain purified and enhanced feature components. The specific algorithm for enhancing feature components is as follows: , in, Indicates enhanced feature components, This represents the frequency characteristic map after wavelet decomposition of high-frequency components. The splicing and fusion unit consists of a channel splicing layer and a feature enhancement convolutional layer. It is used to re-splice the weighted low-frequency features and high-frequency features, fuse information of different frequencies through convolution operations and restore the original number of channels to output the enhanced target features. The channel stitching layer spatially aligns and stitches the low-frequency components, processed by the low-frequency feature re-determination module, with the adaptively weighted high-frequency components along the channel dimension to obtain high-low frequency mixed features. These high-low frequency mixed features are spatial-frequency multi-scale feature representations. The specific algorithm of the channel stitching layer is as follows: , in, This indicates the high- and low-frequency mixed characteristics obtained after splicing. This indicates the low-frequency characteristics after purification. , , These represent the high-frequency features in the horizontal direction, the high-frequency features in the vertical direction, and the high-frequency features in the diagonal direction, respectively. This indicates a channel splicing operation; The feature enhancement convolutional layer inputs high- and low-frequency mixed features. The convolutional layer performs a cross-channel linear combination of high-frequency textures and low-frequency main body contours, adjusts the output dimension, and then accelerates network convergence through a batch normalization layer and performs non-linear activation. The specific algorithm of the feature enhancement convolutional layer is as follows: , in, This represents the feature map after wavelet frequency domain decoupling and adaptive enhancement, where ReLU represents the nonlinear activation function. This indicates a batch normalization operation. This represents feature-interactive convolution.

[0008] Furthermore, the basic weight initialization unit is specifically as follows: The basic weight initialization unit includes an initial weight assignment module and a low-frequency feature re-determination module; The initial weighting module assigns initial weights to low-frequency components and performs preliminary suppression of background noise mixed in high-frequency components. The specific algorithm of the initial weighting module is as follows: , in, , , , Let represent the initial weights of the low-frequency components, the initial weights of the horizontal high-frequency components, the initial weights of the vertical high-frequency components, and the initial weights of the diagonal high-frequency components, respectively. Let LL, LH, HL, and HH represent the low-frequency components, the horizontal high-frequency components, the vertical high-frequency components, and the diagonal high-frequency components, respectively. , , , Indicates the initial weight value; The low-frequency feature re-determination module compresses the low-frequency components, after initial weight allocation, into a one-dimensional channel descriptor in spatial dimension through global average pooling. This one-dimensional channel descriptor is then processed by a one-dimensional convolution with an adaptive kernel size to capture local cross-channel interactions. A sigmoid activation function generates low-frequency channel-specific weights, which are then multiplied element-wise with the low-frequency components to output the refined low-frequency features. The specific algorithm of the low-frequency feature re-determination module is as follows: , , in, This represents the weights specific to the low-frequency channel, and Sigmoid represents the Sigmoid activation function. This represents a one-dimensional convolution operation with an adaptive kernel size, and GAP represents a global average pooling operation. This represents the low-frequency components after initial weighting. This indicates the low-frequency characteristics after purification. This indicates element-wise multiplication.

[0009] Furthermore, the step of obtaining multimodal fusion features based on the deformable fusion module specifically includes: The deformable fusion module includes an offset and mask generation module, a deformation space sampling and alignment module, and a feature stitching module; The offset and mask generation module consists of a depthwise separable convolutional network and channel-segmented layers. It uses infrared image features from the infrared backbone network branch as prior guidance to calculate the offset and modulation mask. These offset and modulation mask are used for spatial sampling of visible light features. The specific operation of the offset and mask generation module is as follows: Depthwise separable convolutional networks utilize kernel sizes of The depthwise separable convolutional network is used to calculate infrared image features. The specific algorithm of the depthwise separable convolutional network is as follows: , in, Indicates infrared image features, Indicates the kernel size as Depth-separable convolution, This represents the joint offset and mask tensor. H and W represent the height and width of the feature map, respectively, and K represents the total number of sampling points of the deformable convolution kernel. The channel segmentation divides the offset and mask joint tensor along the channel dimension. The first 2K channels constitute the coordinate offset, and the last K channels are processed by the Sigmoid activation function to generate modulation mask weights. The specific algorithm for the channel segmentation is as follows: , , in, Indicates the coordinate offset. This indicates the operations of the first 2K channels. This indicates the operations on the last K channels, and Sigmoid represents the Sigmoid activation function. Indicates the modulation mask weights; The deformation space sampling alignment module consists of a deformable convolution module, a bilinear interpolation calculation module, and a visible light feature weighting layer. It uses the coordinate offset and modulation mask weights as prior guidance to perform deformation sampling alignment of visible light image features using an irregular network. The specific operation is as follows: The deformable convolution module adds coordinate offsets to preset regular sampling point positions to obtain dynamic irregular sampling point coordinates. The specific algorithm for obtaining the irregular sampling point coordinates is as follows: , in, Indicates the coordinates of irregular sampling points. Represents any center position on the feature map. Indicates the location of the rule sampling point; The bilinear interpolation calculation module calculates the feature response of irregular sampling points using a two-dimensional bilinear interpolation algorithm. The specific algorithm of the bilinear interpolation calculation module is as follows: , , in, This represents the characteristic response of irregular sampling points. This represents the four integer coordinates of the four points in two-dimensional space that are closest to the irregular sampling point. This represents the feature response of the four integer coordinates closest to the irregular sampling point in two-dimensional space. ReLU represents the kernel function for two-dimensional bilinear interpolation, and ReLU represents the nonlinear activation function. , and , These represent the horizontal and vertical coordinates of the corresponding point, respectively. The visible light feature weighting layer weights the interpolated sample point feature responses with the modulation mask weights to generate aligned visible light features. The specific algorithm for the visible light feature weighting layer is as follows: , in, This represents the aligned visible light characteristics. Representing the The convolutional kernel weights corresponding to each sampling location; The feature stitching module consists of a channel stitching layer, a dimensionality reduction convolutional layer, a normalization module, and a nonlinear activation function. It stitches the aligned visible light features and infrared image features along the channel dimension, and then inputs them... The convolutional layer performs dimensionality reduction, and then passes through a batch normalization module and a non-linear activation function to output multimodal fusion features. The specific algorithm of the feature concatenation module is as follows: , , in, This represents the splicing feature between visible light characteristics and infrared image characteristics. This indicates a channel splicing operation. Indicates multimodal fusion features, This represents the SiLU nonlinear activation function. This indicates a batch normalization operation. express Convolutional layer.

[0010] Furthermore, the step of optimizing the detection head parameters based on the NWD loss function specifically includes: The bounding boxes output by the dual-backbone YOLOv11 object detection network and the labeled ground truth bounding boxes are modeled as independent two-dimensional Gaussian distribution models. The second-order Wasserstein distance between the output bounding boxes and the labeled ground truth bounding boxes is calculated to measure the spatial distribution difference between the two. Then, the second-order Wasserstein distance is nonlinearly normalized according to the exponential function to map it to the (0,1] interval to obtain the normalized distance metric. The bounding box regression loss function is constructed based on the normalized distance metric. The dual-backbone YOLOv11 object detection network is then jointly backpropagated based on the bounding box regression loss function and the object classification loss function.

[0011] Furthermore, the step of obtaining the target detection result based on the multimodal fusion features and performing detection evaluation specifically includes: The detection and evaluation are based on evaluation metrics, which include mean precision, precision, recall, and frame rate. The specific algorithm for the detection and evaluation is as follows: , , , , in, Indicates the mean of the precision; Indicates accuracy. Indicates recall rate, Indicates frame rate, Indicates the total number of categories. This indicates the number of samples that are actually positive and are also predicted to be positive. This indicates the number of samples that were actually negative but were predicted to be positive. This represents the number of samples that were actually positive but were predicted as negative. Indicates the number of images detected. This indicates the total time taken to detect the image.

[0012] In summary, based on the aforementioned UAV target detection method using multimodal feature fusion and wavelet adaptive weighting, infrared prior guidance is used to extract high-frequency information of small targets and capture feature centers in multimodal images, thereby providing accurate spatial anchor points for targets from a cross-modal perspective. Cross-modal offset and mask generation operations are used to dynamically adjust the spatial sampling trajectory of visible light features, avoiding the extraction of complex background clutter and providing alignment criteria for features from a low-level geometric perspective. More importantly, due to the extremely complex background in visible light images from a UAV's top-down view, traditional single-modal deformation sampling easily leads to sampling points diverging and obscuring the real target. Cross-modal prior guidance can force the sampling points of visible light features to converge within the physical boundaries of the real target, improving the model's ability to process the pure texture of small, weak targets against complex backgrounds. Simultaneously, around the extracted infrared feature centers, the coordinate offset and importance modulation mask required for visible light spatial sampling are directly predicted through a feature mapping convolutional layer. The cross-modal generated offset effectively solves the target geometric distortion problem caused by changes in UAV perspective and perspective distortion. Actual irregular sampling points are dynamically generated by combining the visible light base spatial grid with infrared predicted offsets. After acquiring continuous features on the visible light feature map using two-dimensional bilinear interpolation, pixel-level weighting is applied using an infrared predicted mask to obtain precisely aligned deformable geometric features. This compensates for the spatial inaccuracies and feature aliasing problems caused by simple cross-modal channel stitching. Simultaneously, this cross-modal adaptive geometric alignment mechanism allows the model to dynamically fuse infrared edge details with the fine contours of visible light, thereby improving the effectiveness and specificity of the model's collaborative representation of multimodal heterogeneous features. By constructing a dual-backbone network to independently extract heterogeneous modal features, and innovatively replacing the traditional downsampling layer with the wavelet adaptive weighted downsampling module DWTAC for the infrared feature extraction branch, the input feature map is decoupled in the frequency domain without loss using two-dimensional Haar wavelet transform, fundamentally avoiding the irreversible annihilation of high-frequency information caused by traditional pooling or stride convolution. In addition, feature weights of each frequency component are dynamically generated through mechanisms such as dual-path pooling and dimensionality reduction convolution, realizing deep collaborative interaction between spatial and frequency domain features. This effectively suppresses redundant background noise while significantly enhancing the model's ability to extract target high-frequency features. At the same time, an infrared feature-guided deformable fusion module IGDAM is introduced in front of the neck network. It uses infrared thermal prior to dynamically guide visible light features for accurate spatial deformation sampling and geometric alignment, which not only effectively overcomes the perceptual limitations of single-modal vision in harsh environments, but also avoids the spatial inaccuracy and feature aliasing problems caused by traditional simple stitching. Finally, by combining the introduction of the NWD loss function and the transfer learning strategy, the method provided in this application effectively overcomes the defect of target bounding box regression metric sensitivity, achieves an excellent balance between accuracy and inference performance on UAV platforms with limited computing resources, and significantly improves the generalization ability and system robustness of small target detection in complex low-altitude environments.

[0013] This invention proposes a multimodal feature fusion and wavelet adaptive weighting UAV target detection system, comprising: The preprocessing module is used to acquire and preprocess UAV multimodal image data, which includes visible light image data and infrared image data. The preprocessing includes cropping and enhancement processing. The basic feature extraction module is used to input the preprocessed UAV multimodal image data into the dual-backbone YOLOv11 target detection network. The dual-backbone YOLOv11 target detection network is based on a dual-backbone network structure, which includes two independent backbone network branches. The backbone network branches include a visible light backbone network branch and an infrared backbone network branch. The visible light backbone network branch and the infrared backbone network branch extract visible light image features and infrared image features, respectively. The downsampling module is used to perform feature downsampling and purification processing based on the wavelet adaptive weighted downsampling module of the infrared backbone network branch. The wavelet adaptive weighted downsampling module is set in the fourth, sixth and eighth layers of the infrared backbone network branch. A multimodal fusion module is used to obtain multimodal fusion features based on a deformable fusion module. The deformable fusion module is set between a dual backbone network and a neck network. The deformable fusion module guides deformable convolution operations of visible light image features based on infrared image features. The optimization and evaluation module is used to optimize the parameters of the detection head based on the NWD loss function, and then obtain the target detection results and perform detection evaluation based on the multimodal fusion features.

[0014] The present invention also provides a storage medium that stores one or more programs, which, when executed by a processor, implement the UAV target detection method with multimodal feature fusion and wavelet adaptive weighting as described above.

[0015] The present invention also provides a computer device, the computer device including a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the UAV target detection method with multimodal feature fusion and wavelet adaptive weighting as described above. Attached Figure Description

[0016] Figure 1 This is a flowchart of the UAV target detection method based on multimodal feature fusion and wavelet adaptive weighting proposed in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the UAV target detection system with multimodal feature fusion and wavelet adaptive weighting proposed in the second embodiment of the present invention; Figure 3 This is a schematic diagram of the deformable fusion module structure proposed in the first embodiment of the present invention. Figure 4 This is a schematic diagram of the wavelet adaptive weighted downsampling module structure proposed in the first embodiment of the present invention. Figure 5 This is a schematic diagram of the dual-backbone YOLOv11 target detection network proposed in the first embodiment of the present invention.

[0017] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0018] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0019] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0021] Please see Figure 1 The diagram shows a flowchart of the UAV target detection method based on multimodal feature fusion and wavelet adaptive weighting proposed in the first embodiment of the present invention. This UAV target detection method based on multimodal feature fusion and wavelet adaptive weighting includes steps S01 to S05, wherein: Step S01: Acquire multimodal image data from the UAV and perform preprocessing; It should be noted that in this embodiment, the UAV multimodal image data includes visible light image data and infrared image data. The preprocessing includes cropping and enhancement processing. The UAV is used to photograph the target to obtain a UAV target detection dataset containing the target. The target in the UAV target detection dataset is labeled. The target includes pedestrians, crowds, bicycles, vehicles, trucks, tricycles, tricycles with sunshades, buses, and motorcycles, and they are divided into training set, validation set, and test set.

[0022] Based on feature requirements, the first step is to obtain high-quality images of the target to be detected, typically using high-resolution drones to acquire images. Then, the acquired images are classified into positive and negative samples (i.e., containing and not containing the target), constructing a preliminary drone target detection dataset. Each image in the dataset needs to be finely labeled; this process can be completed by professionals or with the help of semi-automatic labeling tools. The labeled categories cover common drone detection targets, such as pedestrians, crowds, bicycles, vehicles, trucks, tricycles, tricycles with canopies, buses, and motorcycles. The labeling should ensure accurate classification and completely record the target's location, size, and category information. To improve the model's generalization ability and robustness, the dataset needs to be divided into training, validation, and test sets after labeling. The training set is used for model training, the validation set is used for hyperparameter tuning and training monitoring, and the test set is used for final performance evaluation. Finally, the preprocessed dataset should be quality-assessed to ensure accurate labeling, effective augmentation, and reasonable partitioning.

[0023] Through detailed category labeling, the algorithm can learn more accurate and specific target features, thereby improving detection accuracy; imaging techniques in the preprocessing process can optimize the performance of the algorithm, making it more efficient and stable when processing large-scale datasets; accurate category labeling and rich datasets can accelerate the convergence speed of the algorithm and improve training efficiency.

[0024] Step S02: Input the preprocessed UAV multimodal image data into the dual-backbone YOLOv11 target detection network; It should be noted that in this embodiment, the dual-backbone YOLOv11 target detection network is based on a dual-backbone network structure, which includes two independent backbone network branches: a visible light backbone network branch and an infrared backbone network branch. These branches extract visible light image features and infrared image features, respectively. Addressing the heterogeneous physical imaging mechanisms of infrared and visible light images, this embodiment abandons the traditional early data-level channel stitching single-stream network architecture and innovatively constructs a dual-backbone network. This dual-backbone network includes a visible light feature extraction branch and an infrared feature extraction branch. While the two branches maintain parallel correspondence in the macroscopic network topology, their network weights are independent and not shared, ensuring no interference during the initial feature extraction phase and thus preserving the independent physical properties of each modality to the greatest extent possible. For the specific structure of the dual-backbone YOLOv11 target detection network, please refer to [link to specific details]. Figure 5 ; Step S03: Perform feature downsampling and purification processing based on the wavelet adaptive weighted downsampling module of the infrared backbone network branch; It should be noted that in this embodiment, the wavelet adaptive weighted downsampling module is set in the fourth, sixth, and eighth layers of the infrared backbone network branch. The wavelet adaptive weighted downsampling module includes a frequency domain decoupling unit, a basic weight initialization unit, an adaptive weight generation unit, and a splicing and fusion unit; for the specific structure of the wavelet adaptive weighted downsampling module, please refer to [link to relevant documentation]. Figure 4 ; The frequency domain decoupling unit employs a Haar wavelet decomposition structure to perform two-dimensional discrete wavelet decomposition on the input image, outputting one low-frequency component and three high-frequency components. The specific operation is as follows: By using two-dimensional Haar wavelet transform, the local neighborhood of the input image is sequentially processed by difference and smoothing operations. Spatial downsampling is completed without feature energy loss, and the input image is decoupled into low-frequency components and high-frequency components. The low-frequency components represent global structural information, and the high-frequency components include horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components. The horizontal, vertical, and diagonal high-frequency components respectively include edge texture details in the horizontal, vertical, and diagonal directions. The basic weight initialization unit consists of an initial weight assignment module and a low-frequency feature re-determination module, which is used to construct initial weights for the decoupled low-frequency and high-frequency components, and to perform background noise filtering and feature purification on the low-frequency components. The adaptive weight generation unit consists of a dual-path pooling module, a weight generation module, and a feature weighting layer. It is used to adaptively generate feature weights for low-frequency and high-frequency components. The specific operation is as follows: The dual-path pooling module inputs three high-frequency components in parallel into the dual-path pooling layer for spatial dimension feature compression and statistical processing to obtain a fused high-frequency feature vector. The specific algorithm of the dual-path pooling module is as follows: , , , in, This represents the result of average pooling, where H and W represent the height and width of the feature map, respectively, and i and j represent the row and column numbers of the feature map, respectively. This represents the input characteristics of the dual-path pooling module. This represents the result of max pooling, where max represents the max pooling operation. This represents the fused high-frequency feature vector output by the dual-path pooling layer. This indicates a channel splicing operation; The weight generation module sequentially inputs the fused high-frequency feature vector into a dimensionality-reducing convolutional layer, a group normalization layer, and a nonlinear activation function to compress the channel dimension and extract the nonlinear dependencies between high-frequency channels. Then, it performs a linear mapping on the nonlinearly activated feature vector to project it to three output branches, generating unnormalized response values ​​for the corresponding low-frequency and high-frequency components. The activation function normalizes the unnormalized response value to generate adaptive normalized weights, which range from 0 to 1. The specific algorithm for the weight generation module is as follows: , , , in, ReLU represents the feature vector of the intermediate hidden layer after compression and linear transformation, and ReLU represents the nonlinear activation function. This indicates a group normalization operation. This represents a dimension reduction convolution operation. This represents the unnormalized response value of the k-th frequency branch. This represents the length of the feature vector of the intermediate hidden layer, and c represents the element ordinal number of the feature vector of the intermediate hidden layer. Indicates the mapping weights. This represents the c-th element of the feature vector of the intermediate hidden layer. This represents the bias function. represents adaptive normalized weights, and sigmoid represents the sigmoid activation function; The feature weighting layer performs feature weighting on high-frequency components according to adaptive normalization weights to obtain purified and enhanced feature components. The specific algorithm for enhancing feature components is as follows: , in, Indicates enhanced feature components, This represents the frequency characteristic map after wavelet decomposition of high-frequency components. The splicing and fusion unit consists of a channel splicing layer and a feature enhancement convolutional layer. It is used to re-splice the weighted low-frequency features and high-frequency features, fuse information of different frequencies through convolution operations and restore the original number of channels to output the enhanced target features. The channel stitching layer spatially aligns and stitches the low-frequency components, processed by the low-frequency feature re-determination module, with the adaptively weighted high-frequency components along the channel dimension to obtain high-low frequency mixed features. These high-low frequency mixed features are spatial-frequency multi-scale feature representations. The specific algorithm of the channel stitching layer is as follows: , in, This indicates the high- and low-frequency mixed characteristics obtained after splicing. This indicates the low-frequency characteristics after purification. , , These represent the high-frequency features in the horizontal direction, the high-frequency features in the vertical direction, and the high-frequency features in the diagonal direction, respectively. This indicates a channel splicing operation; The feature enhancement convolutional layer inputs high- and low-frequency mixed features. The convolutional layer performs a cross-channel linear combination of high-frequency textures and low-frequency main body contours, adjusts the output dimension, and then accelerates network convergence through a batch normalization layer and performs non-linear activation. The specific algorithm of the feature enhancement convolutional layer is as follows: , in, This represents the feature map after wavelet frequency domain decoupling and adaptive enhancement, where ReLU represents the nonlinear activation function. This indicates a batch normalization operation. This represents feature-interactive convolution.

[0025] The basic weight initialization unit includes an initial weight assignment module and a low-frequency feature re-determination module; The initial weighting module assigns initial weights to low-frequency components and performs preliminary suppression of background noise mixed in high-frequency components. The specific algorithm of the initial weighting module is as follows: , in, , , , Let represent the initial weights of the low-frequency components, the initial weights of the horizontal high-frequency components, the initial weights of the vertical high-frequency components, and the initial weights of the diagonal high-frequency components, respectively. Let LL, LH, HL, and HH represent the low-frequency components, the horizontal high-frequency components, the vertical high-frequency components, and the diagonal high-frequency components, respectively. , , , Indicates the initial weight value; The low-frequency feature re-determination module compresses the low-frequency components, after initial weight allocation, into a one-dimensional channel descriptor in spatial dimension through global average pooling. This one-dimensional channel descriptor is then processed by a one-dimensional convolution with an adaptive kernel size to capture local cross-channel interactions. A sigmoid activation function generates low-frequency channel-specific weights, which are then multiplied element-wise with the low-frequency components to output the refined low-frequency features. The specific algorithm of the low-frequency feature re-determination module is as follows: , , in, This represents the weights specific to the low-frequency channel, and Sigmoid represents the Sigmoid activation function. This represents a one-dimensional convolution operation with an adaptive kernel size, and GAP represents a global average pooling operation. This represents the low-frequency components after initial weighting. This indicates the low-frequency characteristics after purification. This indicates element-wise multiplication.

[0026] To address the fundamental challenges of feature extraction and cross-modal fusion for small targets from a UAV perspective, this invention constructs a dual-backbone target detection architecture and innovatively employs the wavelet adaptive weighted downsampling module (DWTAC) to replace the traditional stride convolution or pooling downsampling mechanism in the infrared feature extraction branch. The core motivation lies in the fact that conventional spatial downsampling causes irreversible resolution loss, leading to severe dispersion of weak infrared thermal features of small targets; while wavelet transform can achieve physical dimensionality reduction with zero feature energy loss, losslessly decoupling the high-frequency thermal salient features crucial for the spatial localization of small targets from the low-frequency general information characterizing the background environment.

[0027] For the heterogeneous frequency band characteristics after decoupling, the adaptive weight generation mechanism built into the DWTAC module plays a precise role in feature recalibration. Specifically, the module constructs a nonlinear soft-gated modulation mechanism, applying four dynamically generated normalized weight coefficients to the corresponding low-frequency and high-frequency components respectively, performing channel-level element-wise multiplication. This operation deeply integrates the contextual semantics of the feature map, not only adaptively attenuating redundant infrared background noise, but also greatly highlighting the infrared feature boundaries of small targets.

[0028] Subsequently, the four weighted and refined feature components are concatenated along the channel dimension and input into a standard convolutional layer with a kernel size of 1×1 to perform cross-frequency band information exchange and channel compression. At this point, the DWTAC module outputs a high-quality infrared feature map with halved spatial resolution and maximized infrared feature amplification. More importantly, this high-quality infrared feature map serves as a core spatial navigation prior, directly inputting into the subsequent infrared feature-guided deformable fusion module IGDAM. This module guides the visible light branch across modalities for precise geometric deformation sampling and feature alignment, thus completely establishing a collaborative fusion link between frequency domain decoupling and refinement and spatial domain cross-modal alignment at the network's underlying layer.

[0029] Step S04: Obtain multimodal fusion features based on the deformable fusion module; It should be noted that in this embodiment, the deformable fusion module is located between the dual backbone network and the neck network. The deformable fusion module performs deformable convolution operations on visible light image features guided by infrared image features. The deformable fusion module includes an offset and mask generation module, a deformation space sampling alignment module, and a feature stitching module. For the specific structure of the deformable fusion module, please refer to [link to relevant documentation]. Figure 3 ; The offset and mask generation module consists of a depthwise separable convolutional network and channel-segmented layers. It uses infrared image features from the infrared backbone network branch as prior guidance to calculate the offset and modulation mask. These offset and modulation mask are used for spatial sampling of visible light features. The specific operation of the offset and mask generation module is as follows: Depthwise separable convolutional networks utilize kernel sizes of The depthwise separable convolutional network is used to calculate infrared image features. The specific algorithm of the depthwise separable convolutional network is as follows: , in, Indicates infrared image features, Indicates the kernel size as Depth-separable convolution, This represents the joint offset and mask tensor. H and W represent the height and width of the feature map, respectively, and K represents the total number of sampling points of the deformable convolution kernel. The channel segmentation divides the offset and mask joint tensor along the channel dimension. The first 2K channels constitute the coordinate offset, and the last K channels are processed by the Sigmoid activation function to generate modulation mask weights. The specific algorithm for the channel segmentation is as follows: , , in, Indicates the coordinate offset. This indicates the operations of the first 2K channels. This indicates the operations on the last K channels, and Sigmoid represents the Sigmoid activation function. Indicates the modulation mask weights; The deformation space sampling alignment module consists of a deformable convolution module, a bilinear interpolation calculation module, and a visible light feature weighting layer. It uses the coordinate offset and modulation mask weights as prior guidance to perform deformation sampling alignment of visible light image features using an irregular network. The specific operation is as follows: The deformable convolution module adds coordinate offsets to preset regular sampling point positions to obtain dynamic irregular sampling point coordinates. The specific algorithm for obtaining the irregular sampling point coordinates is as follows: , in, Indicates the coordinates of irregular sampling points. Represents any center position on the feature map. Indicates the location of the rule sampling point; The bilinear interpolation calculation module calculates the feature response of irregular sampling points using a two-dimensional bilinear interpolation algorithm. The specific algorithm of the bilinear interpolation calculation module is as follows: , , in, This represents the characteristic response of irregular sampling points. This represents the four integer coordinates of the four points in two-dimensional space that are closest to the irregular sampling point. This represents the feature response of the four integer coordinates closest to the irregular sampling point in two-dimensional space. ReLU represents the kernel function for two-dimensional bilinear interpolation, and ReLU represents the nonlinear activation function. , and , These represent the horizontal and vertical coordinates of the corresponding point, respectively. The visible light feature weighting layer weights the interpolated sample point feature responses with the modulation mask weights to generate aligned visible light features. The specific algorithm for the visible light feature weighting layer is as follows: , in, This represents the aligned visible light characteristics. Representing the The convolutional kernel weights corresponding to each sampling location; The feature stitching module consists of a channel stitching layer, a dimensionality reduction convolutional layer, a normalization module, and a nonlinear activation function. It stitches the aligned visible light features and infrared image features along the channel dimension, and then inputs them... The convolutional layer performs dimensionality reduction, and then passes through a batch normalization module and a non-linear activation function to output multimodal fusion features. The specific algorithm of the feature concatenation module is as follows: , , in, This represents the splicing feature of visible light characteristics and infrared image characteristics. This indicates a channel splicing operation. Indicates multimodal fusion features, This represents the SiLU nonlinear activation function. This indicates a batch normalization operation. express Convolutional layer.

[0030] Step S05: Optimize parameters based on the NWD loss function, and then obtain target detection results and perform detection evaluation based on multimodal fusion features; It should be noted that in this embodiment, the bounding boxes output by the dual-backbone YOLOv11 object detection network and the labeled ground truth bounding boxes are modeled as independent two-dimensional Gaussian distribution models, and the second-order Wasserstein distance between the output bounding boxes and the labeled ground truth bounding boxes is calculated to measure the difference in their spatial distribution. Then, the second-order Wasserstein distance is nonlinearly normalized according to the exponential function to map it to the (0,1] interval to obtain the normalized distance metric. The bounding box regression loss function is constructed based on the normalized distance metric, and the dual-backbone YOLOv11 object detection network is jointly backpropagated based on the bounding box regression loss function and the object classification loss function.

[0031] Based on the multimodal fusion features extracted using the dual-backbone architecture, the NWD loss function models the bounding boxes as a two-dimensional Gaussian distribution while simultaneously measuring the distribution similarity between the predicted and ground truth boxes, achieving more robust regression supervision. This mechanism not only overcomes the extreme sensitivity of traditional IoU metrics to small positional deviations but also introduces a continuous distance metric at the distribution level, thereby enhancing the model's ability to localize targets at extremely small scales. By introducing NWD loss during the training phase of the dual-backbone architecture, the positive sample allocation process can be further optimized, enabling the final prediction results to possess both global convergence stability and precise localization characteristics, significantly improving the detection accuracy of small targets in UAV aerial photography. Thanks to the smoothness of Gaussian modeling, the model exhibits higher recognition sensitivity and localization accuracy for targets with extremely low pixel proportions and lack of overlapping areas. The refined distance metric also helps solve the gradient vanishing problem in small target bounding box regression, reducing false alarms and missed detections, thus comprehensively improving detection performance. Furthermore, the design framework of the NWD loss function gives it strong noise resistance, enabling it to adapt to UAV detection tasks under different lighting and background scenarios in multimodal data. During training, the model learns the Wasserstein distance between the predicted and true distributions, autonomously adjusting its focus on weak signal features to enhance its ability to capture unknown small targets in complex environments. Furthermore, the normalized exponential mapping mechanism employed by NWD allows for adaptive gradient adjustment based on target scale differences, mitigating numerical oscillations during regression and further improving the model's training convergence efficiency.

[0032] The detection and evaluation are based on evaluation metrics, which include mean precision, precision, recall, and frame rate. The specific algorithm for the detection and evaluation is as follows: , , , , in, Indicates the mean of the precision; Indicates accuracy. Indicates recall rate, Indicates frame rate, Indicates the total number of categories. This indicates the number of samples that are actually positive and are also predicted to be positive. This indicates the number of samples that were actually negative but were predicted to be positive. This represents the number of samples that were actually positive but were predicted as negative. Indicates the number of images detected. This indicates the total time taken to detect the image.

[0033] It is one of the core metrics for evaluating the performance of object detection algorithms. It comprehensively reflects the model's recognition ability across different categories by integrating the detection accuracy of all categories and calculating the average. A higher accuracy indicates a higher performance. The value represents the algorithm's higher accuracy in target recognition and its more stable performance across different categories. Precision measures the proportion of instances predicted as positive that are actually positive; higher precision indicates stronger reliability and fewer false positives. Recall represents the proportion of correctly identified positive instances out of all true positive instances; higher recall reflects that the algorithm can capture more true targets and has a lower false negative rate. (Frames per second) is a key metric for evaluating algorithm processing speed, representing the number of images that can be processed per second. A higher frame rate... Higher values ​​correspond to faster processing speeds, facilitating real-time or near-real-time image processing. In UAV target detection tasks, high... This value is of great significance for ensuring the efficient operation of the system and reducing latency and costs.

[0034] In summary, this embodiment provides a UAV target detection method based on multimodal feature fusion and wavelet adaptive weighting. The specific implementation process is as follows: First, acquire and preprocess UAV visible light and infrared image datasets from the same scene. Second, construct an improved target detection network model based on a dual-backbone architecture, using two independent branches to extract visible light spatial texture and infrared detail features respectively. For the infrared branch, introduce a wavelet adaptive weighted downsampling module (DWTAC) to achieve lossless decoupling and dynamic purification of infrared features in the frequency domain. Subsequently, in the intermediate feature fusion stage, use the infrared feature-guided cross-modal deformation alignment fusion module (TGDAM) to guide visible light features for spatial deformation sampling, completing the geometric alignment and multi-scale aggregation of heterogeneous features. Further, in the model training stage, introduce the normalized Gaussian Wasserstein distance (NWD) loss function, and iteratively optimize the network parameters until convergence through pre-training and transfer learning strategies. Finally, infer the image to be detected based on the optimal network weights to achieve accurate localization and recognition of small UAV targets.

[0035] The core technical advantages of this invention are as follows: First, it constructs an asymmetric dual-backbone feature extraction system. Unlike traditional front-end pixel-level stitching, this invention processes heterogeneous modal data through independent branches, preserving high-resolution textures of visible light while completely solving the problem of irreversible annihilation of high-frequency thermal radiation features of small targets caused by traditional pooling by utilizing the wavelet adaptive weighted downsampling module DWTAC during downsampling, thanks to the wavelet frequency domain decoupling mechanism. Second, it achieves thermal saliency-driven spatial geometric alignment. Relying on the deformable fusion module TGDAM, this invention breaks the deadlock of sampling divergence in traditional fusion mechanisms, using infrared features as spatial navigation anchors to force visible light convolution kernels to achieve accurate edge capture in complex backgrounds, overcoming the challenges of cross-modal feature space inaccuracy and feature aliasing from the ground up. Third, it reconstructs the bounding box regression mechanism for extremely small scales. By introducing the NWD loss function to model the target as a two-dimensional Gaussian distribution, it effectively alleviates the problem of severe IoU oscillations and positive sample allocation failure caused by small positional shifts, significantly enhancing the training stability and localization accuracy of the model at extreme scales.

[0036] In summary, this invention achieves automated and high-precision detection of small targets in UAV aerial photography through deep collaboration of frequency domain decoupling and purification, spatial deformation alignment, and distribution smoothing regression. It not only successfully overcomes the visual perception bottleneck in complex low-altitude environments, significantly reducing the false negative and false positive rates for extremely small targets, but also significantly improves the system's model generalization ability and all-weather robustness in actual traffic patrols, security reconnaissance, and other tasks.

[0037] Please see Figure 2 The figure shown is a schematic diagram of the structure of the UAV target detection system based on multimodal feature fusion and wavelet adaptive weighting proposed in the second embodiment of the present invention. The system includes: Preprocessing module 10 is used to acquire and preprocess UAV multimodal image data, wherein the UAV multimodal image data includes visible light image data and infrared image data, and the preprocessing includes cropping and enhancement processing. The basic feature extraction module 20 is used to input the preprocessed UAV multimodal image data into the dual-backbone YOLOv11 target detection network. The dual-backbone YOLOv11 target detection network is based on a dual-backbone network structure, which includes two independent backbone network branches. The backbone network branches include a visible light backbone network branch and an infrared backbone network branch. The visible light backbone network branch and the infrared backbone network branch extract visible light image features and infrared image features, respectively. The downsampling module 30 is used to perform feature downsampling and purification processing based on the wavelet adaptive weighted downsampling module of the infrared backbone network branch. The wavelet adaptive weighted downsampling module is set in the fourth, sixth and eighth layers of the infrared backbone network branch. The multimodal fusion module 40 is used to obtain multimodal fusion features based on the deformable fusion module. The deformable fusion module is set between the dual backbone network and the neck network. The deformable fusion module guides the deformable convolution operation of visible light image features based on infrared image features. The optimization and evaluation module 50 is used to optimize the parameters of the detection head according to the NWD loss function, and then obtain the target detection results and perform detection evaluation based on the multimodal fusion features.

[0038] The present invention also proposes a computer storage medium storing one or more programs that, when executed by a processor, implement the above-described method for UAV target detection using multimodal feature fusion and wavelet adaptive weighting.

[0039] The present invention also proposes a computer device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to realize the above-mentioned multimodal feature fusion and wavelet adaptive weighting UAV target detection method.

[0040] Those skilled in the art will understand that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain stored, communicated, propagated, or transmitted programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0041] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0042] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0043] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0044] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for UAV target detection using multimodal feature fusion and wavelet adaptive weighting, characterized in that, include: Acquire and preprocess multimodal image data of UAVs, including visible light image data and infrared image data, and the preprocessing includes cropping and enhancement processing; The preprocessed UAV multimodal image data is input into a dual-backbone YOLOv11 target detection network. The dual-backbone YOLOv11 target detection network is based on a dual-backbone network structure, which includes two independent backbone network branches. The backbone network branches include a visible light backbone network branch and an infrared backbone network branch. The visible light backbone network branch and the infrared backbone network branch extract visible light image features and infrared image features, respectively. Feature downsampling and purification are performed using the wavelet adaptive weighted downsampling module of the infrared backbone network branch. The wavelet adaptive weighted downsampling module is set in the fourth, sixth and eighth layers of the infrared backbone network branch. The steps of feature downsampling and purification based on the wavelet adaptive weighted downsampling module of the infrared backbone network branch specifically include: The wavelet adaptive weighted downsampling module includes a frequency domain decoupling unit, a basic weight initialization unit, an adaptive weight generation unit, and a splicing and fusion unit; The frequency domain decoupling unit employs a Haar wavelet decomposition structure to perform two-dimensional discrete wavelet decomposition on the input image, outputting one low-frequency component and three high-frequency components. The specific operation is as follows: By using two-dimensional Haar wavelet transform, the local neighborhood of the input image is sequentially processed by difference and smoothing operations. Spatial downsampling is completed without feature energy loss, and the input image is decoupled into low-frequency components and high-frequency components. The low-frequency components represent global structural information, and the high-frequency components include horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components. The horizontal, vertical, and diagonal high-frequency components respectively include edge texture details in the horizontal, vertical, and diagonal directions. The basic weight initialization unit consists of an initial weight assignment module and a low-frequency feature re-determination module, which is used to construct initial weights for the decoupled low-frequency and high-frequency components, and to perform background noise filtering and feature purification on the low-frequency components. The adaptive weight generation unit consists of a dual-path pooling module, a weight generation module, and a feature weighting layer. It is used to adaptively generate feature weights for low-frequency and high-frequency components. The specific operation is as follows: The dual-path pooling module inputs three high-frequency components in parallel into the dual-path pooling layer for spatial dimension feature compression and statistical processing to obtain a fused high-frequency feature vector. The specific algorithm of the dual-path pooling module is as follows: ， ， ， in, This represents the result of average pooling, where H and W represent the height and width of the feature map, respectively, and i and j represent the row and column numbers of the feature map, respectively. This represents the input characteristics of the dual-path pooling module. This represents the result of max pooling, where max represents the max pooling operation. This represents the fused high-frequency feature vector output by the dual-path pooling layer. This indicates a channel splicing operation; The weight generation module sequentially inputs the fused high-frequency feature vector into a dimensionality-reducing convolutional layer, a group normalization layer, and a nonlinear activation function to compress the channel dimension and extract the nonlinear dependencies between high-frequency channels. Then, it performs a linear mapping on the nonlinearly activated feature vector to project it to three output branches, generating unnormalized response values ​​for the corresponding low-frequency and high-frequency components. The activation function normalizes the unnormalized response value to generate adaptive normalized weights, which range from 0 to 1. The specific algorithm for the weight generation module is as follows: ， ， ， in, ReLU represents the feature vector of the intermediate hidden layer after compression and linear transformation, and ReLU represents the nonlinear activation function. This indicates a group normalization operation. This represents a dimension reduction convolution operation. This represents the unnormalized response value of the k-th frequency branch. This represents the length of the feature vector of the intermediate hidden layer, and c represents the element ordinal number of the feature vector of the intermediate hidden layer. Indicates the mapping weights. This represents the c-th element of the feature vector of the intermediate hidden layer. This represents the bias function. represents adaptive normalized weights, and sigmoid represents the sigmoid activation function; The feature weighting layer performs feature weighting on high-frequency components according to adaptive normalization weights to obtain purified and enhanced feature components. The specific algorithm for enhancing feature components is as follows: ， in, Indicates enhanced feature components, This represents the frequency characteristic map after wavelet decomposition of high-frequency components. The splicing and fusion unit consists of a channel splicing layer and a feature enhancement convolutional layer. It is used to re-splice the weighted low-frequency features and high-frequency features, fuse information of different frequencies through convolution operations and restore the original number of channels to output the enhanced target features. The channel stitching layer spatially aligns and stitches the low-frequency components, processed by the low-frequency feature re-determination module, with the adaptively weighted high-frequency components along the channel dimension to obtain high-low frequency mixed features. These high-low frequency mixed features are spatial-frequency multi-scale feature representations. The specific algorithm of the channel stitching layer is as follows: ， in, This indicates the high- and low-frequency mixed characteristics obtained after splicing. This indicates the low-frequency characteristics after purification. , , These represent the high-frequency features in the horizontal direction, the high-frequency features in the vertical direction, and the high-frequency features in the diagonal direction, respectively. This indicates a channel splicing operation; The feature enhancement convolutional layer inputs high- and low-frequency mixed features. The convolutional layer performs a cross-channel linear combination of high-frequency textures and low-frequency main body contours, adjusts the output dimension, and then accelerates network convergence through a batch normalization layer and performs non-linear activation. The specific algorithm of the feature enhancement convolutional layer is as follows: ， in, This represents the feature map after wavelet frequency domain decoupling and adaptive enhancement, where ReLU represents the nonlinear activation function. This indicates a batch normalization operation. Represents feature-interactive convolution; Multimodal fusion features are obtained according to the deformable fusion module, which is set between the dual backbone network and the neck network. The deformable fusion module guides the deformable convolution operation of visible light image features based on infrared image features. The step of obtaining multimodal fusion features based on the deformable fusion module specifically includes: The deformable fusion module includes an offset and mask generation module, a deformation space sampling and alignment module, and a feature stitching module; The offset and mask generation module consists of a depthwise separable convolutional network and channel-segmented layers. It uses infrared image features from the infrared backbone network branch as prior guidance to calculate the offset and modulation mask. These offset and modulation mask are used for spatial sampling of visible light features. The specific operation of the offset and mask generation module is as follows: Depthwise separable convolutional networks utilize kernel sizes of The depthwise separable convolutional network is used to calculate infrared image features. The specific algorithm of the depthwise separable convolutional network is as follows: ， in, Indicates infrared image features, Indicates the kernel size as Depth-separable convolution, This represents the joint offset and mask tensor. H and W represent the height and width of the feature map, respectively, and K represents the total number of sampling points of the deformable convolution kernel. The channel segmentation divides the offset and mask joint tensor along the channel dimension. The first 2K channels constitute the coordinate offset, and the last K channels are processed by the Sigmoid activation function to generate modulation mask weights. The specific algorithm for the channel segmentation is as follows: ， ， in, Indicates the coordinate offset. This indicates the operations of the first 2K channels. This indicates the operations on the last K channels, and Sigmoid represents the Sigmoid activation function. Indicates the modulation mask weights; The deformation space sampling alignment module consists of a deformable convolution module, a bilinear interpolation calculation module, and a visible light feature weighting layer. It uses the coordinate offset and modulation mask weights as prior guidance to perform deformation sampling alignment of visible light image features using an irregular network. The specific operation is as follows: The deformable convolution module adds coordinate offsets to preset regular sampling point positions to obtain dynamic irregular sampling point coordinates. The specific algorithm for obtaining the irregular sampling point coordinates is as follows: ， in, Indicates the coordinates of irregular sampling points. Represents any center position on the feature map. Indicates the location of the rule sampling point; The bilinear interpolation calculation module calculates the feature response of irregular sampling points using a two-dimensional bilinear interpolation algorithm. The specific algorithm of the bilinear interpolation calculation module is as follows: ， ， in, This represents the characteristic response of irregular sampling points. This represents the four integer coordinates of the four points in two-dimensional space that are closest to the irregular sampling point. This represents the feature response of the four integer coordinates closest to the irregular sampling point in two-dimensional space. ReLU represents the kernel function for two-dimensional bilinear interpolation, and ReLU represents the nonlinear activation function. , and , These represent the horizontal and vertical coordinates of the corresponding point, respectively. The visible light feature weighting layer weights the interpolated sample point feature responses with the modulation mask weights to generate aligned visible light features. The specific algorithm for the visible light feature weighting layer is as follows: ， in, This represents the aligned visible light characteristics. Representing the The convolutional kernel weights corresponding to each sampling location; The feature stitching module consists of a channel stitching layer, a dimensionality reduction convolutional layer, a normalization module, and a nonlinear activation function. It stitches the aligned visible light features and infrared image features along the channel dimension, and then inputs them... The convolutional layer performs dimensionality reduction, and then passes through a batch normalization module and a non-linear activation function to output multimodal fusion features. The specific algorithm of the feature concatenation module is as follows: ， ， in, This represents the splicing feature between visible light characteristics and infrared image characteristics. This indicates a channel splicing operation. Indicates multimodal fusion features, This represents the SiLU nonlinear activation function. This indicates a batch normalization operation. express Convolutional layer; The detection head parameters are optimized based on the NWD loss function, and the target detection results are obtained and evaluated based on the multimodal fusion features.

2. The UAV target detection method based on multimodal feature fusion and wavelet adaptive weighting according to claim 1, wherein the basic weight initialization unit is specifically as follows: The basic weight initialization unit includes an initial weight assignment module and a low-frequency feature re-determination module; The initial weighting module assigns initial weights to low-frequency components and performs preliminary suppression of background noise mixed in high-frequency components. The specific algorithm of the initial weighting module is as follows: ， in, , , , Let represent the initial weights of the low-frequency components, the initial weights of the horizontal high-frequency components, the initial weights of the vertical high-frequency components, and the initial weights of the diagonal high-frequency components, respectively. Let LL, LH, HL, and HH represent the low-frequency components, the horizontal high-frequency components, the vertical high-frequency components, and the diagonal high-frequency components, respectively. , , Indicates the initial weight value; The low-frequency feature re-determination module compresses the low-frequency components, after initial weight allocation, into a one-dimensional channel descriptor in spatial dimension through global average pooling. This one-dimensional channel descriptor is then processed by a one-dimensional convolution with an adaptive kernel size to capture local cross-channel interactions. A sigmoid activation function generates low-frequency channel-specific weights, which are then multiplied element-wise with the low-frequency components to output the refined low-frequency features. The specific algorithm of the low-frequency feature re-determination module is as follows: ， ， in, This represents the weights specific to the low-frequency channel, and Sigmoid represents the Sigmoid activation function. This represents a one-dimensional convolution operation with an adaptive kernel size, and GAP represents a global average pooling operation. This represents the low-frequency components after initial weighting. This indicates the low-frequency characteristics after purification. This indicates element-wise multiplication.

3. The UAV target detection method based on multimodal feature fusion and wavelet adaptive weighting according to claim 1, characterized in that, The step of optimizing the detection head parameters based on the NWD loss function specifically includes: The bounding boxes output by the dual-backbone YOLOv11 object detection network and the labeled ground truth bounding boxes are modeled as independent two-dimensional Gaussian distribution models. The second-order Wasserstein distance between the output bounding boxes and the labeled ground truth bounding boxes is calculated to measure the spatial distribution difference between the two. Then, the second-order Wasserstein distance is nonlinearly normalized according to the exponential function to map it to the (0,1] interval to obtain the normalized distance metric. The bounding box regression loss function is constructed based on the normalized distance metric. The dual-backbone YOLOv11 object detection network is then jointly backpropagated based on the bounding box regression loss function and the object classification loss function.

4. The UAV target detection method based on multimodal feature fusion and wavelet adaptive weighting according to claim 1, characterized in that, The step of obtaining target detection results based on multimodal fusion features and performing detection evaluation specifically includes: The detection and evaluation are based on evaluation metrics, which include mean precision, precision, recall, and frame rate. The specific algorithm for the detection and evaluation is as follows: ， ， ， ， in, Indicates the mean of the precision; Indicates accuracy. Indicates recall rate, Indicates frame rate, Indicates the total number of categories. This indicates the number of samples that are actually positive and are also predicted to be positive. This indicates the number of samples that were actually negative but were predicted to be positive. This represents the number of samples that were actually positive but were predicted as negative. Indicates the number of images detected. This indicates the total time taken to detect the image.

5. A UAV target detection system based on multimodal feature fusion and wavelet adaptive weighting, characterized in that, include: The preprocessing module is used to acquire and preprocess UAV multimodal image data, which includes visible light image data and infrared image data. The preprocessing includes cropping and enhancement processing. The basic feature extraction module is used to input the preprocessed UAV multimodal image data into the dual-backbone YOLOv11 target detection network. The dual-backbone YOLOv11 target detection network is based on a dual-backbone network structure, which includes two independent backbone network branches. The backbone network branches include a visible light backbone network branch and an infrared backbone network branch. The visible light backbone network branch and the infrared backbone network branch extract visible light image features and infrared image features, respectively. The downsampling module is used to perform feature downsampling and purification processing based on the wavelet adaptive weighted downsampling module of the infrared backbone network branch. The wavelet adaptive weighted downsampling module is set in the fourth, sixth and eighth layers of the infrared backbone network branch. The steps of feature downsampling and purification based on the wavelet adaptive weighted downsampling module of the infrared backbone network branch specifically include: The wavelet adaptive weighted downsampling module includes a frequency domain decoupling unit, a basic weight initialization unit, an adaptive weight generation unit, and a splicing and fusion unit; The frequency domain decoupling unit employs a Haar wavelet decomposition structure to perform two-dimensional discrete wavelet decomposition on the input image, outputting one low-frequency component and three high-frequency components. The specific operation is as follows: By using two-dimensional Haar wavelet transform, the local neighborhood of the input image is sequentially processed by difference and smoothing operations. Spatial downsampling is completed without feature energy loss, and the input image is decoupled into low-frequency components and high-frequency components. The low-frequency components represent global structural information, and the high-frequency components include horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components. The horizontal, vertical, and diagonal high-frequency components respectively include edge texture details in the horizontal, vertical, and diagonal directions. The basic weight initialization unit consists of an initial weight assignment module and a low-frequency feature re-determination module, which is used to construct initial weights for the decoupled low-frequency and high-frequency components, and to perform background noise filtering and feature purification on the low-frequency components. The adaptive weight generation unit consists of a dual-path pooling module, a weight generation module, and a feature weighting layer. It is used to adaptively generate feature weights for low-frequency and high-frequency components. The specific operation is as follows: The dual-path pooling module inputs three high-frequency components in parallel into the dual-path pooling layer for spatial dimension feature compression and statistical processing to obtain a fused high-frequency feature vector. The specific algorithm of the dual-path pooling module is as follows: ， ， ， in, This represents the result of average pooling, where H and W represent the height and width of the feature map, respectively, and i and j represent the row and column numbers of the feature map, respectively. This represents the input characteristics of the dual-path pooling module. This represents the result of max pooling, where max represents the max pooling operation. This represents the fused high-frequency feature vector output by the dual-path pooling layer. This indicates a channel splicing operation; The weight generation module sequentially inputs the fused high-frequency feature vector into a dimensionality-reducing convolutional layer, a group normalization layer, and a nonlinear activation function to compress the channel dimension and extract the nonlinear dependencies between high-frequency channels. Then, it performs a linear mapping on the nonlinearly activated feature vector to project it to three output branches, generating unnormalized response values ​​for the corresponding low-frequency and high-frequency components. The activation function normalizes the unnormalized response value to generate adaptive normalized weights, which range from 0 to 1. The specific algorithm for the weight generation module is as follows: ， ， ， in, ReLU represents the feature vector of the intermediate hidden layer after compression and linear transformation, and ReLU represents the nonlinear activation function. This indicates a group normalization operation. This represents a dimension reduction convolution operation. This represents the unnormalized response value of the k-th frequency branch. represents the length of the feature vector of the intermediate hidden layer, and c represents the element ordinal number of the feature vector of the intermediate hidden layer. Indicates the mapping weights. This represents the c-th element of the feature vector of the intermediate hidden layer. This represents the bias function. represents adaptive normalized weights, and sigmoid represents the sigmoid activation function; The feature weighting layer performs feature weighting on high-frequency components according to adaptive normalization weights to obtain purified and enhanced feature components. The specific algorithm for enhancing feature components is as follows: ， in, Indicates enhanced feature components, This represents the frequency characteristic map after wavelet decomposition of high-frequency components. The splicing and fusion unit consists of a channel splicing layer and a feature enhancement convolutional layer. It is used to re-splice the weighted low-frequency features and high-frequency features, fuse information of different frequencies through convolution operations and restore the original number of channels to output the enhanced target features. The channel stitching layer spatially aligns and stitches the low-frequency components, processed by the low-frequency feature re-determination module, with the adaptively weighted high-frequency components along the channel dimension to obtain high-low frequency mixed features. These high-low frequency mixed features are spatial-frequency multi-scale feature representations. The specific algorithm of the channel stitching layer is as follows: ， in, This indicates the high- and low-frequency mixed characteristics obtained after splicing. This indicates the low-frequency characteristics after purification. , , These represent the high-frequency features in the horizontal direction, the high-frequency features in the vertical direction, and the high-frequency features in the diagonal direction, respectively. This indicates a channel splicing operation; The feature enhancement convolutional layer inputs high- and low-frequency mixed features. The convolutional layer performs a cross-channel linear combination of high-frequency textures and low-frequency main body contours, adjusts the output dimension, and then accelerates network convergence through a batch normalization layer and performs non-linear activation. The specific algorithm of the feature enhancement convolutional layer is as follows: ， in, This represents the feature map after wavelet frequency domain decoupling and adaptive enhancement, where ReLU represents the nonlinear activation function. This indicates a batch normalization operation. Represents feature-interactive convolution; A multimodal fusion module is used to obtain multimodal fusion features based on a deformable fusion module. The deformable fusion module is set between a dual backbone network and a neck network. The deformable fusion module guides deformable convolution operations of visible light image features based on infrared image features. The step of obtaining multimodal fusion features based on the deformable fusion module specifically includes: The deformable fusion module includes an offset and mask generation module, a deformation space sampling and alignment module, and a feature stitching module; The offset and mask generation module consists of a depthwise separable convolutional network and channel-segmented layers. It uses infrared image features from the infrared backbone network branch as prior guidance to calculate the offset and modulation mask. These offset and modulation mask are used for spatial sampling of visible light features. The specific operation of the offset and mask generation module is as follows: Depthwise separable convolutional networks utilize kernel sizes of The depthwise separable convolutional network is used to calculate infrared image features. The specific algorithm of the depthwise separable convolutional network is as follows: ， in, Indicates infrared image features, Indicates the kernel size as Depth-separable convolution, This represents the joint offset and mask tensor. H and W represent the height and width of the feature map, respectively, and K represents the total number of sampling points of the deformable convolution kernel. The channel segmentation divides the offset and mask joint tensor along the channel dimension. The first 2K channels constitute the coordinate offset, and the last K channels are processed by the Sigmoid activation function to generate modulation mask weights. The specific algorithm for the channel segmentation is as follows: ， ， in, Indicates the coordinate offset. This indicates the operations of the first 2K channels. This indicates the operations on the last K channels, and Sigmoid represents the Sigmoid activation function. Indicates the modulation mask weights; The deformation space sampling alignment module consists of a deformable convolution module, a bilinear interpolation calculation module, and a visible light feature weighting layer. It uses the coordinate offset and modulation mask weights as prior guidance to perform deformation sampling alignment of visible light image features using an irregular network. The specific operation is as follows: The deformable convolution module adds coordinate offsets to preset regular sampling point positions to obtain dynamic irregular sampling point coordinates. The specific algorithm for obtaining the irregular sampling point coordinates is as follows: ， in, Indicates the coordinates of irregular sampling points. Represents any center position on the feature map. Indicates the location of the rule sampling point; The bilinear interpolation calculation module calculates the feature response of irregular sampling points using a two-dimensional bilinear interpolation algorithm. The specific algorithm of the bilinear interpolation calculation module is as follows: ， ， in, This represents the characteristic response of irregular sampling points. This represents the four integer coordinates of the four points in two-dimensional space that are closest to the irregular sampling point. This represents the feature response of the four integer coordinates closest to the irregular sampling point in two-dimensional space. ReLU represents the kernel function for two-dimensional bilinear interpolation, and ReLU represents the nonlinear activation function. , and , These represent the horizontal and vertical coordinates of the corresponding point, respectively. The visible light feature weighting layer weights the interpolated sample point feature responses with the modulation mask weights to generate aligned visible light features. The specific algorithm for the visible light feature weighting layer is as follows: ， in, This represents the aligned visible light characteristics. Representing the The convolutional kernel weights corresponding to each sampling location; The feature stitching module consists of a channel stitching layer, a dimensionality reduction convolutional layer, a normalization module, and a nonlinear activation function. It stitches the aligned visible light features and infrared image features along the channel dimension, and then inputs them... The convolutional layer performs dimensionality reduction, and then passes through a batch normalization module and a non-linear activation function to output multimodal fusion features. The specific algorithm of the feature concatenation module is as follows: ， ， in, This represents the splicing feature between visible light characteristics and infrared image characteristics. This indicates a channel splicing operation. Indicates multimodal fusion features, This represents the SiLU nonlinear activation function. This indicates a batch normalization operation. express Convolutional layer; The optimization and evaluation module is used to optimize the parameters of the detection head based on the NWD loss function, and then obtain the target detection results and perform detection evaluation based on the multimodal fusion features.

6. A storage medium, characterized in that, The storage medium stores one or more programs that, when executed by a processor, implement the UAV target detection method based on multimodal feature fusion and wavelet adaptive weighting as described in any one of claims 1-4.

7. A computer device, characterized in that, The computer device includes a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the UAV target detection method of multimodal feature fusion and wavelet adaptive weighting as described in any one of claims 1-4.

Images