An Unmanned Aerial Vehicle Remote Sensing Image Dehazing Method, System, Device and Medium Based on UAVD-Net

Abstract

A method, system, device and medium for dehazing UAV remote sensing images based on UAVD-Net, which constructs a non-uniform haze remote sensing image dataset including a training set, a validation set and a test set; constructs a UAV remote sensing image dehazing network based on UAVD-Net; uses the training set and the validation set to train the UAV remote sensing image dehazing network based on UAVD-Net to obtain an optimal training weight file Dehaze.pt; uses the test set and the optimal training weight file Dehaze.pt to perform UAV remote sensing image dehazing on the UAV remote sensing image dehazing network based on UAVD-Net to obtain a UAV remote sensing image dehazing result; the system, device and medium are used to implement this method; the present invention effectively fuses the global information and local features of the image by introducing a multi-level global information capture module, an adaptive local information enhancement module and a cross-channel feature fusion module into the network, realizes complementarity, and effectively improves the dehazing accuracy of UAV remote sensing images in the case of non-uniform haze, having the advantages of strong adaptability and high dehazing accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and further relates to deep learning and digital image processing technologies. Specifically, it is a method, system, device, and medium for dehazing UAV remote sensing images based on UAVD-Net. The present invention can be used for dehazing UAV optical remote sensing images. Background Art

[0002] Unmanned aerial vehicles (UAVs), as a flexible and efficient aerial remote sensing tool, have played an important role in fields such as environmental monitoring, disaster assessment, urban planning, and agricultural management. Especially in complex terrains or extreme climate conditions, UAVs can acquire high-resolution image data, providing crucial support for decision-making. However, due to the presence of haze, dust, and other particles in the atmosphere, remote sensing images are often interfered with during the acquisition process, resulting in a decrease in image contrast and blurred details, affecting the reliability and accuracy of the data. Image dehazing technology has thus become an important topic in UAV remote sensing applications.

[0003] Traditional image dehazing methods are mostly based on theories and technologies in fields such as image enhancement and image restoration. However, in practical applications, especially images taken by UAVs often face challenges such as complex lighting, changing scenes, and uneven haze levels, and the performance of existing methods is limited. With the rapid development of deep learning and computer vision technologies, dehazing methods based on convolutional neural networks (CNNs) and transformers have gradually become a research hotspot, showing great potential in extracting image features and improving detail restoration.

[0004] The patent application document with the publication number CN118052737A discloses a perception-guided UAV image dehazing algorithm based on superpixel scene prior. The implementation steps of this method are as follows: introducing superpixel scene prior to reduce the calculation in the UAV dehazing process, and the image dehazing process can be converted from the RGB color space to the Lab color space to avoid confusion between different chromaticities and redundant information, so as to select a reliable area for efficient dehazing through the L channel. Considering the influence of light non-uniformity during the dehazing process, the invention designs a guided filtering algorithm based on simple linear iterative clustering, which replaces a large number of guiding windows with superpixel clustering windows with similar color blocks while retaining complementary information. In order to improve the perception ability after dehazing, a quantitative analysis is carried out on the superpixel segmentation and target detection results, and an alternating direction multiplier method is used to design a collaborative optimization feedback iteration mechanism for perception and dehazing to enhance the effectiveness of UAV vision tasks in foggy environments. However, the present invention still has the following deficiencies: this method relies on the atmospheric light intensity value, and when there are large blank areas in the image, the dehazing effect of this method is poor.

[0005] The patent application document with the publication number CN118314053A discloses a method for removing haze from UAV inspection aerial images based on a neural network. The implementation steps of this method are as follows: constructing an image dataset based on UAV inspection aerial images; constructing a heuristic perception dehazing neural network, and training the heuristic perception dehazing neural network based on the image dataset to obtain a trained heuristic perception dehazing neural network; inputting the UAV inspection aerial image to be dehazed into the trained heuristic perception dehazing neural network for processing to obtain a haze-free clear aerial image. However, the still existing drawback of this invention is that this method can only process and ignore local details in the image, resulting in blurring or detail loss in local areas of the dehazed image.

[0006] The defects and deficiencies of the above prior art are summarized as follows:

[0007] 1. Many existing methods assume a uniform haze distribution and use simple models to estimate the haze effect. However, in actual scenarios, haze often exhibits non-uniformity, which can lead to a decline in the quality of the dehazing results, especially in images with high dynamic range or complex terrain.

[0008] 2. Existing dehazing algorithms tend to ignore local details in the image when dealing with global haze, resulting in blurring or detail loss in local areas of the dehazed image. Summary of the Invention

[0009] In order to overcome the above deficiencies of the prior art, the purpose of the present invention is to provide a method, system, device and medium for removing haze from UAV remote sensing images based on UAVD-Net. By designing a multi-level global information capture module (MGIC), global feature layers are extracted and fused layer by layer to enhance the model's ability to understand complex scenarios and improve the accuracy of dehazing; by designing an adaptive local information enhancement module (ALIE), texture detail information in the image can be effectively obtained and enhanced to improve the dehazing accuracy; designing a cross-channel feature fusion module (CFF) to fuse global and local information through a cross-channel mechanism to maintain the overall structure of the image and enhance the local detail clarity, thereby obtaining a natural and clear dehazed image; this method is used to solve the problems of low dehazing quality of existing UAV dehazing methods in non-uniform haze conditions and blurring or detail loss in local areas of the dehazed image.

[0010] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0011] A method for removing haze from UAV remote sensing images based on UAVD-Net, comprising the following steps:

[0012] Step 1: Construct a non-uniform haze remote sensing image dataset, including a training set, a validation set and a test set;

[0013] Step 2: Construct a UAV remote sensing image dehazing network based on UAVD-Net, including an Encoder, a Texture Detail Feature Extraction sub-network, and a Decoder;

[0014] Step 3: Use the training set and validation set in the non-uniform haze remote sensing image dataset constructed in Step 1 to train the UAV remote sensing image dehazing network based on UAVD-Net constructed in Step 2 to obtain the optimal training weight file Dehaze.pt;

[0015] Step 4: Use the test set in the non-uniform haze remote sensing image dataset constructed in Step 1 and the optimal training weight file Dehaze.pt obtained in Step 3 to perform UAV remote sensing image dehazing on the UAV remote sensing image dehazing network based on UAVD-Net constructed in Step 2 to obtain the UAV remote sensing image dehazing result.

[0016] The UAV remote sensing image dehazing network based on UAVD-Net in Step 2 includes an Encoder, a Texture Detail Feature Extraction sub-network, and a Decoder;

[0017] The Encoder includes a CNN network layer, five multi-level global information capture modules (MGIC), and a summation layer; the multi-level global information capture module (MGIC) includes a patch embedding (PEb) layer, a positional encoding layer (Positional Encoding, PE), a multi-level encoder (Multi-Level Encoder, MLE), and a dehazing enhancement layer (Dehazing Enhancement Layer, DEL), where the multi-level encoder includes three multi-head self-attention mechanism layers (Multi-head Self-Attention Mechanism, MHSA), three feed-forward network layers (Feed Forward Network, FFN), two normalization layers (Batch Normalization, BN), and a linear normalization layer (Linear Normalization, LN); the dehazing enhancement layer includes a convolutional layer (Conv), a ReLU activation function, three convolution block attention module (CBAM) layers, and three normalization (BN) layers; the multi-level global information capture module (MGIC) is expressed as the following formula:

[0018] Foutput = DEL(MLE(PE + PEb(F input )));

[0019] The texture detail feature extraction sub-network (Texture Detail Feature Extraction) includes a Conv layer and four adaptive local information enhancement modules (ALIE); the adaptive local information enhancement module (ALIE) includes two consecutive Conv layers, two multi-head self-attention mechanism layers (Multi-head Self-Attention Mechanism, MHSA), two addition and normalization layers (Add&Norm, A&N), and a positional encoding layer (Positional Encoding, PE); the adaptive local information enhancement module (ALIE) is expressed as the following formula:

[0020] F ALIE = F3 + Conv(Conv(F'))

[0021] F3 = A&N(MHSA(F2), F2)

[0022] F2 = A&N(MHSA(F1))

[0023] F1 = Conv(Conv(F')) + PE;

[0024] The decoder (Decoder) includes a cross-channel feature fusion module (CFF); the cross-channel feature fusion module (CFF) fuses global features and local features, and includes an upsampling layer (Upsampling, up), a channel attention mechanism (ChannelAttention Mechanism, CAM), two Conv layers, a ReLU activation function, two concatenation layers (Cat), a Conv layer with a convolution kernel of 3, a Conv layer with a convolution kernel of 5, and a Conv layer with a convolution kernel of 7; the cross-channel feature fusion module (CFF) is expressed as the following formula:

[0025] F * = Conv(Cat(Conv(Cat(ReLU(Conv(up(F G ), CAM(F L ))), up(F)))) i ))。

[0026] In step 3, set the number of training epochs to be greater than or equal to 150, and the batch size to be greater than or equal to 32.

[0027] In step 4, set the test batch size to be greater than or equal to 8.

[0028] The present invention also provides a UAV remote sensing image dehazing system based on UAVD-Net, including:

[0029] Data acquisition module: used to construct a non-uniform haze remote sensing image dataset, including a training set, a validation set, and a test set;

[0030] Network construction module: used to construct a UAV remote sensing image dehazing network based on UAVD-Net, including an encoder (Encoder), a texture detail feature extraction sub-network (Texture Detail Feature Extraction), and a decoder (Decoder);

[0031] Network training module: used to train the UAV remote sensing image dehazing network based on UAVD-Net using the training set and the validation set in the non-uniform haze remote sensing image dataset to obtain an optimal training weight file Dehaze.pt;

[0032] Dehazing module: used to perform UAV remote sensing image dehazing on the UAV remote sensing image dehazing network based on UAVD-Net using the test set in the non-uniform haze remote sensing image dataset and the optimal training weight file Dehaze.pt to obtain the UAV remote sensing image dehazing result.

[0033] The present invention also provides a UAV remote sensing image dehazing device based on UAVD-Net, including:

[0034] Memory: stores a computer program for the above-mentioned UAV remote sensing image dehazing method based on UAVD-Net, which is a computer-readable device;

[0035] Processor: used to implement the above-mentioned UAV remote sensing image dehazing method when executing the computer program.

[0036] The present invention also provides a computer-readable storage medium, which stores a computer program that can implement the above-mentioned UAV remote sensing image dehazing method when executed by a processor.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] 1. The present invention designs five multi-level global information capture modules (MGIC) in the encoder. This module enhances the model's ability to understand complex scenes and improves the accuracy of non-uniform haze dehazing by extracting and fusing global feature layers layer by layer.

[0039] 2. In the present invention, four Adaptive Local Information Enhancement Modules (ALIE) are designed in the network for texture detail feature extraction. This module can effectively acquire and enhance the texture detail information in the image, improving the defogging accuracy.

[0040] 3. In the decoder of the present invention, a Cross-channel Feature Fusion Module (CFF) is designed. This module fuses global and local information through a cross-channel mechanism to maintain the overall structure of the image and enhance the local detail clarity, thereby obtaining a natural and clear defogged image.

[0041] In summary, the present invention designs a brand-new network structure. By introducing the multi-level global information capture MGIC module, Adaptive Local Information Enhancement Module (ALIE), and Cross-channel Feature Fusion Module CFF in the network, the global information and local features of the image are effectively fused, achieving complementarity. It can effectively improve the defogging accuracy of UAV remote sensing images in the case of non-uniform haze, and has the advantages of strong adaptability and high defogging accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a schematic diagram of the principle process of an embodiment of the present invention.

[0043] Figure 2 is a structure diagram of the UAV remote sensing image defogging network based on UAVD-Net according to an embodiment of the present invention.

[0044] Figure 3 is a structure diagram of the MGIC module according to an embodiment of the present invention.

[0045] Figure 4 is a structure diagram of the ALIE module according to an embodiment of the present invention.

[0046] Figure 5 is a structure diagram of the CFF module according to an embodiment of the present invention.

[0047] Figure 6 is a graph of the defogging accuracy of UAV remote sensing images of the present invention and existing methods. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] The technical solution of the present invention will be further described in detail below with reference to the drawings.

[0049] As Figure 1 shown, a method for defogging UAV remote sensing images based on UAVD-Net includes the following steps:

[0050] Step 1: Construct a non-uniform haze remote sensing image dataset, including a training set, a validation set, and a test set;

[0051] Select evenly hazy images from the publicly available UAV hazy remote sensing image dataset to form a non-uniform hazy remote sensing image dataset, and divide the non-uniform hazy remote sensing image dataset into a training set, a validation set, and a test set according to a ratio.

[0052] As Figure 2 shown, Step 2: Construct a UAV remote sensing image dehazing network based on UAVD-Net, including an encoder (Encoder), a texture detail feature extraction sub-network (Texture Detail Feature Extraction), and a decoder (Decoder);

[0053] As Figure 3 shown, Step 201: The encoder includes a CNN network layer, five multi-level global information capture modules (MGIC), and a summation layer. The multi-level global information capture module (MGIC) includes a patch embedding (PEb) layer, a positional encoding layer (Positional Encoding, PE), a multi-level encoder (Multi-Level Encoder, MLE), and a dehazing enhancement layer (Dehazing Enhancement Layer, DEL), where the multi-level encoder includes three multi-head self-attention mechanism layers (Multi-head Self-Attention Mechanism, MHSA), three feed forward network layers (Feed Forward Network, FFN), two normalization layers (Batch Normalization, BN), and one linear normalization layer (Linear Normalization, LN); the dehazing enhancement layer includes a convolutional layer (Conv), a ReLU activation function, three convolution block attention module (CBAM) layers, and three normalization (BN) layers; the multi-level global information capture module (MGIC) is expressed as the following formula:

[0054] F output = DEL(MLE(PE + PEb(F input )));

[0055] As Figure 4As shown, step 202: The Texture Detail Feature Extraction sub-network includes a Conv layer and four Adaptive Local Information Enhancement (ALIE) modules; the Adaptive Local Information Enhancement (ALIE) module includes two consecutive Conv layers, two Multi-head Self-Attention Mechanism (MHSA) layers, two Add&Norm (A&N) layers, and one Positional Encoding (PE) layer; the Adaptive Local Information Enhancement (ALIE) module is expressed as the following formula:

[0056] F ALIE = F3 + Conv(Conv(F'))

[0057] F3 = A&N(MHSA(F2), F2)

[0058] F2 = A&N(MHSA(F1))

[0059] F1 = Conv(Conv(F')) + PE;

[0060] As Figure 5 shown, step 203: The Decoder includes a Cross-channel Feature Fusion (CFF) module; the Cross-channel Feature Fusion (CFF) module fuses global features and local features, and includes an Upsampling (up) layer, a Channel Attention Mechanism (CAM), two Conv layers, a ReLU activation function, two Concatenate (Cat) layers, a Conv layer with a kernel size of 3, a Conv layer with a kernel size of 5, and a Conv layer with a kernel size of 7. The Cross-channel Feature Fusion (CFF) module is expressed as the following formula:

[0061] F * = Conv(Cat(Conv(Cat(ReLU(Conv(up(F G ), CAM(F L ))), up(F)))) i ).

[0062] Step 3: Set the number of training epochs to be greater than or equal to 150, and the batch size to be greater than or equal to 32. Use the training set and validation set in the non-uniform haze remote sensing image dataset constructed in Step 1 to train the UAVD-Net-based UAV remote sensing image dehazing network constructed in Step 2 to obtain the optimal training weight file Dehaze.pt.

[0063] Step 4: Set the test batch size to be greater than or equal to 8, and use the test set in the non-uniform haze remote sensing image dataset constructed in Step 1 and the optimal training weight file Dehaze.pt obtained in Step 3 to perform haze removal on the UAV remote sensing image haze removal network constructed in Step 2 for UAV remote sensing images, and obtain the UAV remote sensing image haze removal result.

[0064] The present invention also provides a UAV remote sensing image haze removal system based on UAVD-Net, including:

[0065] Data acquisition module: used to construct a non-uniform haze remote sensing image dataset in Step 1, including a training set, a validation set, and a test set;

[0066] Network construction module: used to construct a UAV remote sensing image haze removal network based on UAVD-Net in Step 2, including an encoder (Encoder), a texture detail feature extraction sub-network (Texture Detail Feature Extraction), and a decoder (Decoder);

[0067] Network training module: used to train the UAV remote sensing image haze removal network constructed in Step 2 using the training set and the validation set in the non-uniform haze remote sensing image dataset constructed in Step 1 in Step 3, and obtain the optimal training weight file Dehaze.pt;

[0068] Haze removal module: used to perform haze removal on the UAV remote sensing image haze removal network constructed in Step 2 for UAV remote sensing images using the test set in the non-uniform haze remote sensing image dataset constructed in Step 1 and the optimal training weight file Dehaze.pt obtained in Step 3, and obtain the UAV remote sensing image haze removal result.

[0069] The present invention also provides a UAV remote sensing image haze removal device based on UAVD-Net, including:

[0070] Memory: stores the computer program of the above-mentioned UAV remote sensing image haze removal method based on UAVD-Net, and is a computer-readable device;

[0071] Processor: used to implement the above-mentioned UAV remote sensing image haze removal method when executing the computer program.

[0072] The present invention also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement the above-mentioned UAV remote sensing image haze removal method based on UAVD-Net.

[0073] The effects of the present invention will be further described below in conjunction with simulation experiments.

[0074] 1. Simulation experiment conditions

[0075] The hardware platform for the simulation experiment of the present invention is as follows: the processor is Intel i9-13900K, the main frequency is 3.0GHz, and the operating memory is 64G.

[0076] The software platform for the simulation experiment platform of the present invention is: Windows 11 operating system and PyCharm, PyTorch 2.1.0, CUDA 12.1.

[0077] 2. Simulation steps

[0078] The training set and validation set in the publicly available non-uniform haze remote sensing image dataset constructed in Step 1 are input into the UAV remote sensing image dehazing network based on UAVD-Net for optimization training. All remote sensing image dehazing adopts single-scale training, the image input size is 512×512 pixels, and the number of iterations epoch is set to 150.

[0079] 3. Simulation content and its result analysis

[0080] The simulation experiment of the present invention performs dehazing processing on 3 UAV remote sensing images containing non-uniform haze, and the results are averaged. The results are as Figure 6 shown.

[0081] The following combines Figure 6 to further describe the simulation effect of the present invention.

[0082] Figure 6 is a comparison chart of the dehazing accuracy between the present invention and existing methods. The comparison algorithm is DehazeFormer (Song, Y., He, Z., Qian, H., & Du, X. [J] (2022). Vision Transformers for Single Image Dehazing. IEEE Transactions on Image Processing, 32, 1927-1941.). The size of the test image is 512×512, and the accuracy evaluation indicators are peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). Among them, PSNR is an index used to measure the quality of an image or video. The higher the value, the better the image quality and the less noise. SSIM evaluates the similarity between two images by comparing aspects such as brightness, contrast, and structure. The SSIM value is between 0 and 1, and the closer the value is to 1, the more similar the two images are.

[0083] As Figure 6As shown, it can be seen that both the PSNR and SSIM of the present invention are higher than those of DehazeFormer. The experimental results show that the multi-level global information capture module (MGIC) designed in the present invention enhances the model's ability to understand complex scenes and improves the accuracy of non-uniform haze dehazing by extracting and fusing global feature layers layer by layer. At the same time, the adaptive local information enhancement module (ALIE) can effectively acquire and enhance the texture detail information in the image, improving the dehazing accuracy. In addition, the cross-channel feature fusion module (CFF) fuses global and local information through a cross-channel mechanism to maintain the overall structure of the image and enhance the local detail clarity, thereby obtaining a natural and clear dehazed image. Generally speaking, the UAV remote sensing image dehazing method based on UAVD-Net proposed in the present invention significantly improves the dehazing accuracy of remote sensing images and meets the requirements of high-precision dehazing.

Claims

1. A method for removing haze from UAV remote sensing images based on UAVD-Net, characterized in that, It includes the following steps: Step 1: Construct a non-uniform haze remote sensing image dataset, including a training set, a validation set, and a test set; Step 2: Construct a UAV remote sensing image dehazing network based on UAVD-Net, including an encoder, a texture detail feature extraction sub-network, and a decoder; The encoder includes a CNN network layer, five multi-level global information capture modules, and a summation layer; The multi-level global information capture module includes a patch embedding layer, a position encoding layer, a multi-level encoder, and a dehazing enhancement layer. The multi-level encoder includes three multi-head self-attention mechanism layers, three feed-forward network layers, two normalization layers, and a linear normalization layer; the dehazing enhancement layer includes a convolutional layer, a ReLU activation function, three hybrid attention mechanism layers, and three normalization layers; The multi-level global information capture module is expressed as the following formula: F output = DEL(MLE(PE + PEb(F input ))); The texture detail feature extraction sub-network includes a Conv layer and four adaptive local information enhancement modules; The adaptive local information enhancement module includes two serially connected Conv layers, two multi-head self-attention mechanism layers, two summation normalization layers, and a position encoding layer; the adaptive local information enhancement module is expressed as the following formula: F ALIE = F3 + Conv(Conv(F')) F3 = A&N(MHSA(F2), F2) F2 = A&N(MHSA(F1)) F1 = Conv(Conv(F')) + PE; The decoder includes a cross-channel feature fusion module; the cross-channel feature fusion module is used to fuse global features and local features. The cross-channel feature fusion module includes an upsampling layer, a channel attention mechanism, two Conv layers, a ReLU activation function, two summation layers, a Conv layer with a convolution kernel of 3, a Conv layer with a convolution kernel of 5, and a Conv layer with a convolution kernel of 7; the cross-channel feature fusion module is expressed as the following formula: F * = Conv(Cat(Conv(Cat(ReLU(Conv(up(F G ),CAM(F L ))),up(F G ))) i )); Step 3: Use the training set and the validation set in the non-uniform haze remote sensing image dataset constructed in Step 1 to train the UAV remote sensing image dehazing network based on UAVD-Net constructed in Step 2 to obtain the optimal training weight file Dehaze.pt; Step 4: Use the test set in the non-uniform haze remote sensing image dataset constructed in Step 1 and the optimal training weight file Dehaze.pt obtained in Step 3 to perform UAV remote sensing image dehazing on the UAV remote sensing image dehazing network based on UAVD-Net constructed in Step 2 to obtain the UAV remote sensing image dehazing result.

2. The method for removing haze from UAV remote sensing images based on UAVD-Net according to claim 1, characterized in that, In Step 3, set the number of training epochs to be greater than or equal to 150, and the batch size to be greater than or equal to 32.

3. A method for removing haze from UAV remote sensing images based on UAVD-Net according to claim 1, characterized in that, In Step 4, set the test batch size to be greater than or equal to 8.

4. A UAV remote sensing image dehazing system based on the method according to any one of claims 1 to 3, characterized in that, It includes: Data acquisition module: used to construct a non-uniform haze remote sensing image dataset, including a training set, a validation set, and a test set; Network construction module: used to construct a UAV remote sensing image dehazing network based on UAVD-Net, including an encoder, a texture detail feature extraction sub-network, and a decoder; Network training module: It is used to train the UAV remote sensing image dehazing network based on UAVD-Net using the training set and validation set in the non-uniform haze remote sensing image dataset to obtain the optimal training weight file Dehaze.pt; Dehazing module: It is used to perform UAV remote sensing image dehazing on the UAV remote sensing image dehazing network based on UAVD-Net using the test set in the non-uniform haze remote sensing image dataset and the optimal training weight file Dehaze.pt to obtain the UAV remote sensing image dehazing result.

5. An unmanned aerial vehicle (UAV) remote sensing image dehazing device based on UAVD-Net, characterized in that, Including: Memory: Store the computer program of a UAV remote sensing image dehazing method according to any one of claims 1-3, which is a computer-readable device; Processor: It is used to implement a UAV remote sensing image dehazing method according to any one of claims 1-3 when executing the computer program.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it can implement a UAV remote sensing image dehazing method according to any one of claims 1-3.

Images