A method for small target detection by drones in adverse weather conditions

Abstract

A method for detecting small targets on unmanned aerial vehicles (UAVs) under adverse weather conditions belongs to the field of computer vision and target detection technology. Its key features include the following steps: Step 1, preprocessing and initially extracting features from the source and target domain images to obtain feature maps; Step 2, filtering low-entropy redundant features from the feature maps; Step 3, decomposing the feature maps, then performing depthwise convolution on each frequency sub-band after each decomposition, and recombining them; Step 4, matching different attenuation coefficients to the feature extraction backbone and prediction head using a grouped exponential moving average strategy; Step 5, detecting small targets on UAVs by constructing a total loss function. This method effectively solves the problem of decreased detection accuracy caused by image degradation and feature blurring under adverse weather conditions, significantly improving the accuracy and robustness of small target detection on UAVs under such conditions.

Description

Technical Field

[0001] A method for detecting small targets on drones under adverse weather conditions belongs to the field of computer vision and target detection technology. Background Technology

[0002] In recent years, drones have been widely used due to their maneuverability and flexibility. Small target detection is one of the core tasks of drone low-altitude monitoring, such as identifying trapped people in rubble at long distances during disaster relief and tracking small vehicles violating traffic rules in traffic monitoring. However, these small targets usually have low pixel ratios, inconspicuous features, and are easily affected by background interference. In addition, in practical applications, the operating environment of drones is often affected by severe weather such as fog, haze, rain, and snow. Specifically, severe weather such as fog, haze, rain, and snow can cause the effective features of small targets to be obscured, and the performance of general detection models will drop sharply.

[0003] To address the challenges of small target detection by drones, academia and industry have conducted extensive research. Early methods primarily focused on enhancing the feature representation capabilities of small targets in images, such as fusing multi-scale features through feature pyramid networks, using attention mechanisms to strengthen the weights of target regions, or combining super-resolution reconstruction techniques to magnify small targets. These methods achieved some success in ideal environments, but their robustness to adverse weather conditions was insufficient.

[0004] To address the impact of severe weather on small target detection by UAVs, a traditional approach is to combine image enhancement with target detection. Traditional image enhancement methods (such as dark channel prior dehazing, wavelet transform denoising, and physics-based rain and snow removal) enhance image sharpness through preprocessing before inputting the enhanced image into the detection model. However, these methods may over-smooth the image, neglecting local details in small target regions; furthermore, the enhancement process is not correlated with the detection task, making it difficult to guarantee the preservation of target-sensitive features. More importantly, these methods are typically designed for specific weather conditions, have limited generalization ability, and struggle to handle complex scenarios involving mixed fog, haze, rain, and snow.

[0005] In recent years, domain adaptation techniques have provided new research ideas for cross-scene detection. These methods maintain model performance in unlabeled target domains by aligning the feature distributions of the source domain (e.g., clear weather) and the target domain (e.g., severe weather). For example, the domain-adaptive detection Transformer model effectively mitigates cross-domain differences through class-level prototype alignment and global statistical learning, demonstrating excellent performance in general object detection tasks. However, for small target detection by UAVs under severe weather conditions, the domain-adaptive detection Transformer method still has limitations: First, at the feature level, existing domain adaptation methods mostly employ global operation mechanisms for feature alignment, lacking the ability to distinguish between "noise-target" features, making it difficult for the model to focus on the key information of small targets. For example, in foggy images, the blurred regions of the background and the low-resolution regions of small targets may have similar distribution characteristics; simple domain alignment would confuse the features of the two.

[0006] Secondly, in terms of model structure, traditional convolutional operations have a limited receptive field, making it difficult to capture the global contextual relationships of small targets in complex backgrounds. Small targets in the view of drones are often scattered, and their surrounding environment (such as building edges and vegetation textures) may contain important auxiliary information. However, traditional convolution focuses more on local features and cannot effectively integrate these global cues, resulting in a decrease in detection accuracy.

[0007] Finally, regarding parameter optimization, most existing methods employ a uniform parameter update strategy, making it difficult to balance model stability and adaptability. The feature extraction backbone needs to maintain a stable ability to capture general features, while the prediction head needs to quickly adapt to changes in feature distribution under severe weather conditions. Therefore, a single update strategy may lead to the backbone overfitting noisy features, or the prediction head failing to adjust in a timely manner. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method for detecting small targets of UAVs in adverse weather conditions that effectively solves the problem of decreased detection accuracy caused by image degradation and feature blurring under adverse weather conditions, and significantly improves the accuracy and robustness of UAV small target detection under adverse weather conditions.

[0009] The technical solution adopted by this invention to solve its technical problem is: a method for detecting small targets by unmanned aerial vehicles under adverse weather conditions, characterized by the following steps: Step 1: Preprocess the source domain image and the target domain image and extract initial features, then feed them into the encoder-decoder structure of Transformer to obtain feature maps; Step 2: Filter low-entropy redundant features from the feature map obtained in Step 1 by calculating the Shannon entropy of the feature channels. Step 3: Decompose the feature map obtained in Step 1 into different frequency sub-bands, and then perform depth convolution on each frequency sub-band after each level of decomposition to generate a feature map after depth convolution. Finally, use inverse wavelet transform to recombine the feature maps after convolution of each frequency band. Step 4: Using a grouped exponential moving average strategy, different decay coefficients are matched to the feature extraction backbone and the prediction head; Step 5: Detect small targets on the UAV by constructing a total loss function.

[0010] Preferably, step 2 includes the following steps: Step 2-1: For the feature map X output by the decoder, calculate the Shannon entropy value of each channel along the channel dimension; Step 2-2: Convert the entropy values ​​of the feature map into attention weights, normalize the entropy vector, and generate attention weights ω. Steps 2-3 involve generating attention weights. The weighted feature map X' is obtained by multiplying it channel by channel with the original feature map output from the decoder. Steps 2-4 introduce complex entropy-aware attention filtering to filter low-entropy redundant features.

[0011] Preferably, in step 2-1, the Shannon entropy value of each channel is calculated along the channel dimension, and the calculation formula is as follows: Among them, H c Let represent the entropy of the c-th channel, N = H * W be the total number of pixels in the feature map, and p(Xi) represent the probability of pixel Xi appearing.

[0012] Preferably, in step 2-2, the formula for generating the attention weight ω is: in, For hyperparameters, ω c H represents the weight of the c-th channel of ω. c Let represent the entropy of the c-th channel.

[0013] Preferably, step 3 includes the following steps: Step 3-1: Perform J-level discrete wavelet transform on the feature map X to decompose it into sub-bands of different frequencies. Step 3-2: For each frequency sub-band after each level of decomposition, perform depthwise convolution using a 3*3 convolution kernel to generate a feature map after depthwise convolution. Step 3-3: Use inverse wavelet transform to recombine the feature maps after convolution of each frequency band, and restore the sub-band features to spatial domain feature maps.

[0014] Preferably, in step 3-1, the low-frequency components are further recursively decomposed, and the decomposition formula is as follows: Where DWT(·) represents the discrete wavelet transform, This represents the low-frequency component of the j-th level decomposition of the feature map X.

[0015] Preferably, in step 4, the parameter update calculation formula for the feature extraction backbone is as follows: in, To extract the backbone parameters of the teacher model for the t-th iteration, To extract the backbone parameters of the teacher model for the (t-1)th iteration, To extract the backbone parameters of the student model for the t-th iteration, To extract the attenuation coefficient of the main trunk; For the prediction head, the parameter update calculation formula is as follows: in, Let be the prediction head parameters of the teacher model in the t-th iteration. These are the prediction head parameters of the teacher model in the (t-1)th iteration. Let be the prediction head parameters of the student model in the t-th iteration. This is the attenuation coefficient of the prediction head.

[0016] Preferably, step 1 includes the following steps: Step 1-1: Scale the source domain image and the target domain image to the same size, and perform standard preprocessing on the scaled images; Steps 1-2 involve inputting the preprocessed source domain image and target domain image into a shared backbone network to extract initial features, and then feeding the feature-extracted image into a domain adaptive detection Transformer encoder-decoder structure to obtain feature maps.

[0017] Compared with the prior art, the beneficial effects of this invention are: In the UAV small target detection method under severe weather conditions proposed in this application, the initial features of the source and target domain images are first extracted through a backbone network. Then, a complex entropy-aware attention module filters out low-entropy redundant features, retains high-entropy effective information, reduces weather noise interference, and enhances the focusing of key features of small targets. Next, a wavelet convolution feature enhancement module is used to replace traditional convolution, strengthening the ability to capture the global context of small targets and the discriminative power of noise. Then, a grouped exponential moving average parameter update strategy is designed to balance the stability and adaptability of the model through differentiated decay coefficients. Finally, a total loss function is constructed by combining the detection loss of the domain adaptive detection model and the complex entropy-aware attention loss, and features and parameters are synergistically optimized for small target detection. This invention effectively solves the problem of decreased detection accuracy caused by image degradation and feature blurring under severe weather conditions, and significantly improves the accuracy and robustness of UAV small target detection under severe weather conditions.

[0018] In this application, based on the domain adaptive detection Transformer model, the accurate detection of small targets under severe weather conditions is achieved by integrating the collaborative optimization strategy of "entropy filtering-wavelet transform-grouped exponential moving average parameters". The complex entropy-aware attention mechanism quantifies the feature information entropy, adaptively adjusts the attention weight, strengthens the feature expression of small targets, and suppresses background noise.

[0019] By replacing traditional convolution operations with wavelet convolution and leveraging its unique time-frequency analysis characteristics, the model's ability to capture the global context of small targets is enhanced, improving the distinction between targets and noise and overcoming the limitations of traditional convolutional local feature extraction. A differentiated parameter update strategy using grouped exponential moving average parameters is employed, designing different attenuation coefficients based on the functional differences between the feature extraction backbone and the prediction head, thus balancing the stability of general features with adaptability to severe weather conditions.

[0020] Through the synergistic effect of feature selection, enhancement, and parameter optimization, the accuracy and robustness of UAV small target detection are significantly improved under adverse weather conditions. Attached Figure Description

[0021] Figure 1 This is a flowchart of a method for detecting small targets using drones in adverse weather conditions.

[0022] Figure 2 This is a schematic diagram of the network structure for a small target detection method for drones under severe weather conditions. Detailed Implementation

[0023] Figures 1-2 This is the preferred embodiment of the present invention, which is described below in conjunction with the accompanying drawings. Figures 1-2 The present invention will be further described below.

[0024] like Figure 1 As shown, a method for detecting small targets using a drone under adverse weather conditions includes the following steps: Step 1: Preprocess the source domain image and the target domain image and extract initial features, then feed them into the encoder-decoder structure of Transformer to obtain feature maps.

[0025] Combination Figure 2 The source and target domain images are fed into a shared backbone network to extract initial features, which are then fed into the encoder-decoder structure of the domain adaptive detection Transformer. After passing through class-level prototype alignment and dataset-level alignment modules, cross-domain feature class consistency optimization is achieved.

[0026] Step 1 includes the following steps: Step 1-1: Scale the source domain image and the target domain image to the same size, and perform standard preprocessing on the scaled images; Steps 1-2 involve inputting the preprocessed source domain image and target domain image into a shared backbone network to extract initial features. Then, the image with extracted features is fed into a domain adaptive detection Transformer encoder-decoder structure to obtain feature maps, thereby achieving cross-domain feature class consistency optimization.

[0027] Step 2: Filter low-entropy redundant features from the feature map obtained by the Transformer encoder-decoder by calculating the Shannon entropy of the feature channels; The feature map output by the Transformer encoder-decoder in step 1 is used to filter out low-entropy redundant features by calculating the Shannon entropy of the feature channels, while retaining high-entropy effective information to generate a spatial attention map. This reduces attention ambiguity under complex weather conditions and enhances the focus on key features of small targets. Step 2 includes the following steps: Step 2-1, process the feature map output by the decoder. Where H, W, and C are the height, width, and number of channels of the feature map, respectively. The Shannon entropy value for each channel is calculated along the channel dimension using the following formula: Among them, H c Let represent the entropy of the c-th channel, N = H * W be the total number of pixels in the feature map, and p(Xi) represent the probability of pixel Xi appearing.

[0028] Step 2-2: Convert the entropy values ​​of the feature map into attention weights. Introduce a non-linear mapping function to normalize the entropy vector and generate the attention weights ω. The weight generation formula is: in, ω is a hyperparameter that controls the sensitivity of weight generation. c This represents the weight of the c-th channel of ω. A larger weight indicates higher uncertainty in the corresponding channel's features, potentially containing more key information about small targets.

[0029] Steps 2-3 involve multiplying the generated attention weights ω channel-by-channel with the original feature map output by the decoder to enhance key features of small targets and suppress background noise, resulting in a weighted feature map X'. The weighting formula is as follows: in, Weighted feature map The One channel.

[0030] Steps 2-4 introduce complex entropy-aware attention loss. By constraining the correlation between high-entropy feature channels and small target regions, the model is guided to pay more attention to high-entropy features containing key information about small targets. The formula for calculating the loss of complex entropy-aware attention loss is as follows: Where M is the number of small targets, ω c This represents the weight of the c-th channel of ω. ω represents the complex entropy-perceptual attention loss.c This represents the weight of the c-th channel of ω.

[0031] Step 3: Decomposition and recombination of feature maps; Wavelet convolution is introduced to replace traditional convolution operations. By leveraging the multi-scale decomposition characteristics and large receptive field of wavelet transform in the frequency domain, the ability to capture global contextual information of small targets under adverse weather conditions is enhanced; at the same time, the frequency domain filtering characteristics are used to strengthen the distinction between targets and noise.

[0032] Step 3 includes the following steps: Step 3-1: Perform J-level discrete wavelet transform on the feature map X to decompose it into sub-bands of different frequencies. Where j represents the j-th level decomposition.

[0033] The low-frequency components are further recursively decomposed into To further improve the frequency resolution of low frequencies and reduce its spatial resolution, the decomposition formula is as follows: Where DWT(·) represents the discrete wavelet transform, This represents the low-frequency component of the j-th level decomposition of the feature map X.

[0034] Step 3-2: For each frequency sub-band after each level of decomposition, perform depthwise convolution using a 3*3 convolution kernel to generate a feature map after depthwise convolution. Depthwise convolution is used to separate convolutions of different frequency components, allowing smaller convolutional kernels to operate over a larger area of ​​the original input, thus expanding the overall receptive field. The convolution process formula is as follows: in, Indicates different subband frequencies. This indicates the weights of the corresponding subband convolution kernel. For the feature map of the corresponding subband, ConV(·) represents the convolution operation.

[0035] Step 3-3: Use inverse wavelet transform to recombine the feature maps after convolution of each frequency band.

[0036] Wavelet transform and its inverse transform are linear operations. By summing the outputs of convolutions at different levels, they aggregate information of different frequencies. Wavelet convolution output... The calculation is as follows: in, This represents the inverse wavelet transform, which restores the sub-band features to a spatial domain feature map.

[0037] Step 4: Using a grouped exponential moving average strategy, different decay coefficients are matched to the feature extraction backbone and the prediction head; A grouped exponential moving average strategy is proposed: a higher decay coefficient is used for the feature extraction backbone to maintain feature stability, while a lower decay coefficient is used for the prediction head to enhance its dynamic adaptability. A differentiated parameter update strategy is employed to balance the model's generalization across weather scenarios with its specificity for small target detection tasks. Step 4 includes the following steps: Step 4-1: Design differentiated attenuation coefficients to address the functional differences between the feature extraction backbone and the prediction head. A higher attenuation coefficient is applied to the feature extraction backbone to maintain feature stability. The formula for updating the feature extraction backbone parameters is as follows: in, To extract the backbone parameters of the teacher model for the t-th iteration, To extract the backbone parameters of the teacher model for the (t-1)th iteration, To extract the backbone parameters of the student model for the t-th iteration, To extract the attenuation coefficient of the main trunk.

[0038] Step 4-2: A lower attenuation coefficient is applied to the prediction head to enhance its dynamic adaptability. The formula for updating the prediction head parameters is as follows: in, Let be the prediction head parameters of the teacher model in the t-th iteration. These are the prediction head parameters of the teacher model in the (t-1)th iteration. Let be the prediction head parameters of the student model in the t-th iteration. This is the attenuation coefficient of the prediction head.

[0039] Step 5: Detect small targets using the UAV by constructing a total loss function; Construct a domain-adaptive detection Transformer detection loss and complex entropy-perceived attention loss The total loss function, consisting of two parts, is continuously iterated and optimized through feature selection, enhancement, and parameter updates to ultimately achieve the detection of small targets.

[0040] Step 5 specifically includes the following steps: The domain-adaptive detection Transformer detection loss L d and complex entropy-perceived attention loss The two parts are combined to form the total loss function for model training, and its calculation formula is as follows: in, Attention loss for complex entropy perception The weight.

[0041] The following experiment verifies the aforementioned method for detecting small targets using drones under adverse weather conditions: The datasets used in the experiment were the HazyDet dataset and the VisDrone dataset. The HazyDet dataset is designed specifically for target detection by UAVs in foggy conditions. This dataset contains 383,000 instances and the data sources include two parts: field images collected in natural foggy environments and data generated by simulating and generating data for foggy conditions based on atmospheric scattering models on normal scene images.

[0042] The VisDrone dataset is a large-scale drone vision dataset containing 288 video clips (261,908 frames) and 10,209 still images. All data was collected from cameras mounted on various drones. The dataset boasts diverse and multi-dimensional data sources, covering 14 different cities in China, including diverse scenes such as urban and rural areas, and encompassing over 10 common targets such as pedestrians and vehicles, covering varying scene densities, including sparse and dense scenes. Data was collected using multiple drone platforms under different weather and lighting conditions, including sunny, cloudy, and foggy days.

[0043] To test the capabilities of UAV small target detection methods across weather conditions, the proposed method was tested on the two datasets mentioned above. On the VisDrone dataset, the mAP50 was 78.7% and the mAP was 55.9%. The proposed method, through the collaborative design of complex entropy-aware attention and wavelet convolution, comprehensively outperforms the baseline model-domain adaptive detection Transformer method in target detection performance, demonstrating significant advantages in key aspects of small target detection and robustness across weather conditions in UAV scenarios. Specifically, compared to the baseline method, mAP50 is improved by 7.6%, mAP by 3.6%, and the three sub-indices APS, APM, and APL are improved by 4.6%, 6.7%, and 4.7%, respectively. Compared to the latest CNN-based and Transformer-based methods, the proposed method, using ResNet-50 as the backbone network, achieves the best performance across all indicators. In particular, compared to the U-ShapeNet method using the Swin-T backbone network, even without a superior backbone network performance, mAP50 is improved by 1.1% and mAP by 0.5%.

[0044] On the HazyDet dataset, the mAP50 is 56.4%, and the mAP is 33.4%, representing a 5.2% improvement in mAP50 and a 3.1% improvement in mAP compared to the baseline DATR method. APS, APM, and APL are improved by 7.9%, 4.3%, and 2.8%, respectively. Compared to state-of-the-art CNN-based and Transformer-based methods, our proposed method, using ResNet-50 as the backbone, achieves optimal detection performance. Compared to the U-ShapeNet method, it improves mAP50 by 21.9% and mAP by 13.9%.

[0045] Experimental results show that the proposed collaborative design of complex entropy-aware attention and wavelet convolution can effectively compensate for the performance shortcomings of the backbone network. By optimizing feature quality and feature allocation efficiency, the proposed method achieves better performance in the task of small target detection in severe weather.

[0046] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for detecting small targets using a drone under adverse weather conditions, characterized in that: Includes the following steps: Step 1: Preprocess the source domain image and the target domain image and extract initial features, then feed them into the encoder-decoder structure of Transformer to obtain feature maps; Step 2: Filter low-entropy redundant features from the feature map obtained in Step 1 by calculating the Shannon entropy of the feature channels. Step 3: Decompose the feature map obtained in Step 1 into different frequency sub-bands, and then perform depth convolution on each frequency sub-band after each level of decomposition to generate a feature map after depth convolution. Finally, use inverse wavelet transform to recombine the feature maps after convolution of each frequency band. Step 4: Using a grouped exponential moving average strategy, different decay coefficients are matched to the feature extraction backbone and the prediction head; Step 5: Detect small targets on the UAV by constructing a total loss function.

2. The method for detecting small targets by unmanned aerial vehicles under adverse weather conditions according to claim 1, characterized in that: Step 2 includes the following steps: Step 2-1: For the feature map X output by the decoder, calculate the Shannon entropy value of each channel along the channel dimension; Step 2-2: Convert the entropy values ​​of the feature map into attention weights, normalize the entropy vector, and generate attention weights ω. Steps 2-3: Multiply the generated attention weights ω with the original feature map output by the decoder channel by channel to obtain the weighted feature map X'. Steps 2-4 introduce complex entropy-aware attention filtering to filter low-entropy redundant features.

3. The method for detecting small targets by unmanned aerial vehicles under adverse weather conditions according to claim 2, characterized in that: In step 2-1, the Shannon entropy value of each channel is calculated along the channel dimension. The calculation formula is as follows: Among them, H c Let represent the entropy of the c-th channel, N = H * W be the total number of pixels in the feature map, and p(Xi) represent the probability of pixel Xi appearing.

4. The method for detecting small targets by unmanned aerial vehicles under adverse weather conditions according to claim 2, characterized in that: In step 2-2, the formula for generating the attention weight ω is: in, For hyperparameters, ω c H represents the weight of the c-th channel of ω. c Let represent the entropy of the c-th channel.

5. The method for detecting small targets by unmanned aerial vehicles under adverse weather conditions according to claim 1, characterized in that: Step 3 includes the following steps: Step 3-1: Perform J-level discrete wavelet transform on the feature map X to decompose the feature map X into sub-bands of different frequencies; Step 3-2: For each frequency sub-band after each level of decomposition, perform depthwise convolution using a 3*3 convolution kernel to generate a feature map after depthwise convolution. Step 3-3: Use inverse wavelet transform to recombine the feature maps after convolution of each frequency band, and restore the sub-band features to spatial domain feature maps.

6. The method for detecting small targets by unmanned aerial vehicles under adverse weather conditions according to claim 1, characterized in that: In step 3-1, the low-frequency components are further recursively decomposed, and the decomposition formula is as follows: Where DWT(·) represents the discrete wavelet transform, This represents the low-frequency component of the j-th level decomposition of the feature map X.

7. The method for detecting small targets by unmanned aerial vehicles under adverse weather conditions according to claim 1, characterized in that: In step 4, the parameter update calculation formula for the feature extraction backbone is as follows: in, To extract the backbone parameters of the teacher model for the t-th iteration, To extract the backbone parameters of the teacher model for the (t-1)th iteration, To extract the backbone parameters of the student model for the t-th iteration, To extract the attenuation coefficient of the main trunk; For the prediction head, the parameter update calculation formula is as follows: in, Let be the prediction head parameters of the teacher model in the t-th iteration. These are the prediction head parameters of the teacher model in the (t-1)th iteration. Let be the prediction head parameters of the student model in the t-th iteration. This is the attenuation coefficient of the prediction head.

8. The method for detecting small targets by unmanned aerial vehicles under adverse weather conditions according to claim 1, characterized in that: Step 1 includes the following steps: Step 1-1: Scale the source domain image and the target domain image to the same size, and perform standard preprocessing on the scaled images; Steps 1-2 involve inputting the preprocessed source domain image and target domain image into a shared backbone network to extract initial features, and then feeding the feature-extracted image into a domain adaptive detection Transformer encoder-decoder structure to obtain feature maps.

Images