A weakly supervised dehazing method based on uncertainty-driven

Abstract

The present invention discloses a weakly supervised defogging method based on uncertainty driving, comprising a weakly supervised defogging network using a U-Net architecture, wherein the weakly supervised defogging network comprises a regional dynamic attention module (RDAM), a downsampling layer, a multi-scale feature fusion module (MSFFM), and an upsampling layer; the RDAM is used to process an input image or the output of a previous RDAM layer, construct a dynamic convolution layer, and obtain a regional dynamic attention feature map through the dynamic convolution layer; the downsampling layer is used to downsample the output of the RDAM to reduce the spatial resolution; the MSFFM is used to process input features through a window multi-head self-attention mechanism to capture long-distance dependencies between pixels and perform multi-scale feature extraction and fusion; the upsampling layer is used to restore the feature map output by the MSFFM to the resolution of the feature map output by the previous MSFFM layer and perform feature splicing therewith; the upsampled feature map passes through a 3x3 convolution layer, a ReLU activation function, and a 1x1 convolution layer in sequence, and finally outputs a defogged image.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and specifically relates to an image defogging method, in particular a weakly supervised defogging network based on deep learning. Background Art

[0002] Images captured under hazy weather conditions are often affected by uneven brightness, reduced contrast, color distortion, and blurred details, which greatly hinders the analysis and understanding of these images by vision-based industrial systems. Therefore, dehazing, as a necessary step for downstream tasks such as object detection, has become an important research direction in vision-based industrial systems. [1] . Current dehazing methods are mainly divided into prior-based dehazing methods and deep learning-based dehazing methods. The former relies on the atmospheric scattering model and identifies the relationship between them by analyzing a large number of foggy and fog-free images. However, these methods often fail in scenes that do not conform to the prior information, resulting in significant color distortion and vignetting effects. The latter estimates the atmospheric scattering model parameters or directly maps foggy images to fog-free images by training neural networks using synthetic datasets. Although the effect is significant, the generalization performance in real scenes is poor.

[0003] Weakly supervised dehazing methods have gradually become one of the research directions to solve this problem. [2,3,4] The model is trained on loosely paired foggy and non-fog images, but these methods are complex to train and have poor model stability. In addition, some teacher-student networks based on convolutional neural networks (CNNs) [5,6,7] Structures are used for dehazing, but due to the limitation of their local receptive field, they have difficulty capturing long-range dependencies. Recently, weakly supervised dehazing methods based on transformers have been proposed. [8] Although they excel in capturing global dependencies, they often neglect local details. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and solve the effectiveness and uncertainty problems of existing defogging methods in the real world. It provides a weakly supervised defogging method driven by uncertainty, which aims to improve the defogging effect in non-uniform defogging tasks by integrating the regional dynamic attention module (RDAM) and the multi-scale feature fusion module (MSFFM) based on the window multi-head self-attention mechanism (W-MSA). In particular, it can better preserve the local details and global consistency of the image when processing real scenes.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A weakly supervised dehazing method based on uncertainty driving includes a weakly supervised dehazing network using a U-Net architecture. The weakly supervised dehazing network consists of a regional dynamic attention module (RDAM), a downsampling layer, a multi-scale feature fusion module (MSFFM), an upsampling layer, a 3x3 convolutional layer, a ReLU activation function, and a 1x1 convolutional layer.

[0007] The regional dynamic attention module (RDAM) processes the input image and the output of the previous RDAM layer and constructs a dynamic convolution layer using spatial attention features. The RDAM uses two parallel convolution paths to calculate channel attention and pixel attention respectively, and then uses learnable parameters to weightedly fuse these two attentions to obtain a channel-pixel collaborative attention feature map. The regional dynamic attention feature map is obtained through the dynamic convolution layer.

[0008] The downsampling layer is used to downsample the output of the regional dynamic attention module RDAM in the previous layer, reducing the spatial resolution to obtain the input of the regional dynamic attention module RDAM in the next layer;

[0009] The multi-scale feature fusion module (MSFFM) processes the features input to the MSFFM through a windowed multi-head self-attention mechanism to capture long-range dependencies between pixels and obtain windowed multi-head self-attention features. The MSFFM performs deconvolution and average pooling on the windowed multi-head self-attention feature map to extract and fuse information at several levels. The module also introduces parallel dilated convolution to enhance the fusion of local and global information.

[0010] The upsampling layer is used to restore the feature map output by the multi-scale feature fusion module MSFFM to the resolution of the feature map output by the multi-scale feature fusion module MSFFM of the previous layer, and perform feature splicing with the feature map output by the multi-scale feature fusion module MSFFM of the previous layer; finally, a 3x3 convolution layer, a ReLU activation function and a 1x1 convolution layer are used to output the dehazed image.

[0011] Furthermore, in the regional dynamic attention module RDAM, channel attention and pixel attention are calculated by the following formulas respectively:

[0012] f c =f in ·σ(Conv(δ(Conv(AvgPool(f in ))))));

[0013] f in represents the input feature map of the regional dynamic attention module, f crepresents the channel attention feature; AvgPool(·) represents the average pooling operation; Conv(·) is the convolution layer; δ(·) is the ReLU activation function; σ(·) is the Sigmoid activation function, which is used to limit the weight to the range of (0,1);

[0014] By inputting feature map f in Perform average pooling, then pass through two convolutional layers and nonlinear activation calculations;

[0015] f p =f in ·σ(Conv(δ(Conv(f in ))));

[0016] f p Represents pixel attention features, directly input feature map f in This is obtained by performing two convolutional layers and activation function calculations; then the learnable parameter σ(θ) is used to adaptively fuse the channel attention features and the pixel attention features:

[0017] f mix =σ(θ)·f c +(1-σ(θ))·f p ;

[0018] σ(θ) is a learnable parameter obtained by the Sigmoid function, with a value range of (0,1); f mix Represents the channel-pixel collaborative attention feature map, which represents the weighted sum of channel attention and pixel attention and is used for subsequent dynamic convolution;

[0019] In order to achieve regional adaptive convolution, the regional dynamic attention module RDAM is equipped with a dynamic convolution layer based on spatial attention;

[0020] First, calculate the spatial attention feature f s :

[0021] f s =f in ·σ(Conv(concat(AvgPool(f in ),MaxPool(f in ))));

[0022] Enhance the spatial feature expression of the input feature map through spatial attention and calculate the regional weight W i :

[0023] W i =Softmax(Linear(AvgPool(f s ))),i=1,2,3,...,k;

[0024] where f s Represents spatial attention features; AvgPool(·) represents the average pooling operation; concat(·) represents the feature concatenation operation; MaxPool(·) represents the maximum pooling operation, Linear(·) represents the fully connected layer; Softmax(·) is the Softmax activation function; k is the number of regions; then the weight W is applied to the k convolution kernels i Generate a set of region-adaptive dynamic convolution kernels conv i , the specific calculation process is:

[0025] conv i =W i conv0,i=1,2,3,...,k;

[0026] Where conv0 is the initial convolution kernel; the obtained dynamic convolution kernel is used to construct the dynamic convolution layer DyConv();

[0027] Finally, the channel-pixel collaborative attention feature f mix After the dynamic convolution layer DyConv(), the regional dynamic attention feature f is obtained out for:

[0028] f out =DyConv(f mix ).

[0029] Furthermore, the specific workflow in the multi-scale feature fusion module MSFFM is as follows:

[0030] The multi-scale feature fusion module MSFFM processes the input x through the window multi-head self-attention mechanism in To learn the long-distance dependencies between image pixels, we can obtain the window multi-head self-attention feature x W-MSA , then perform deconvolution and average pooling operations to obtain 2 times up-sampled features x2 and 1 / 2 times down-sampled features x of different scales 1 / 2 , 1 / 4 times downsampled feature x 1 / 4 Finally, refine and fuse to obtain the multi-scale fusion feature Y out ; The specific operations are as follows:

[0031] x W-MSA =W-MSA(x in )

[0032] x in is the feature input to the multi-scale feature fusion module MSFFM; W-MSA(·) is the window multi-head self-attention mechanism operation; then the window multi-head self-attention feature x W-MSARefine it through a separable shared convolution layer and a dilated convolution layer to obtain the original scale refined feature Y;

[0033] In parallel, multi-head self-attention features of the window x W-MSA Perform deconvolution and average pooling operations:

[0034] x2=Deconv(x W-MSA )

[0035] Deconv(·) is a deconvolution operation used to convert x W-MSA The resolution is doubled to obtain a 2x upsampled feature x2; then it is refined through a convolutional layer, a separable shared convolutional layer, and a dilated convolutional layer to obtain a 2x upsampled scale refined feature Y2;

[0036] x 1 / 2 =AvgPool2×2(x W-MSA )

[0037] AvgPool2×2(·) is a 2x2 average pooling operation used to convert x W-MSA The resolution is reduced by half to obtain 1 / 2 times downsampled features x 1 / 2 , and then refined through a transposed convolution layer, a separable shared convolution layer and an expanded convolution layer to obtain a 1 / 2 times downsampling scale refined feature Y 1 / 2 ;

[0038] x 1 / 4 =AvgPool4×4(x W-MSA )

[0039] AvgPool4×4(·) is a 4x4 average pooling operation, which obtains 1 / 4 times the downsampled feature x 1 / 4 , and then refined through two transposed convolutional layers, a separable shared convolutional layer and an expanded convolutional layer to obtain a 1 / 4 times downsampling scale refined feature Y 1 / 4 ;

[0040] Then, the above multi-scale refined features are spliced, and after a convolution layer and a ReLU activation function, the multi-scale fusion feature Y is output. out , the calculation process is:

[0041] Y out =Relu(Conv(concat(Y 1 / 4 , Y 1 / 2 , Y, Y2)))

[0042] Through these operations, the multi-scale feature fusion module MSFFM combines the global and local information of the input image and enhances the feature expression capability of the dehazing task.

[0043] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, characterized in that when the processor executes the program, the steps of the uncertainty-driven weakly supervised defogging method are implemented.

[0044] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the uncertainty-driven weakly supervised defogging method.

[0045] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0046] 1. Efficient dehazing effect: The present invention adopts a weakly supervised dehazing network with a U-Net architecture, combined with downsampling and upsampling modules, to extract and restore multi-scale features of the image, thereby providing an efficient dehazing effect in complex scenes. The network can effectively remove haze from the image while maintaining the details and clarity of the image. Through quantitative comparison, the present invention achieved the highest PSNR (30.32) and SSIM (0.972) among the weakly supervised methods on the HSTS test set, indicating that it has superior performance in the image dehazing task. In addition, the present invention exhibits stronger robustness when processing complex real scenes.

[0047] 2. Regional dynamic adaptive weight adjustment: The regional dynamic attention module (RDAM) of the present invention can dynamically adjust the weights of different channels and pixels, focusing on areas with higher haze concentrations in the image. This channel-pixel collaborative attention mechanism ensures that the network can adaptively handle the distribution of haze in complex scenes, so that key areas in the image can be better restored. By adaptively adjusting the weights of different channels and pixels, effective dehazing is performed on different areas in the image, avoiding residual haze, color shift and other phenomena in other methods, making the final dehazed image more realistic and delicate.

[0048] 3. Capturing long-range dependencies: By introducing a multi-scale feature fusion module (MSFFM) based on a windowed multi-head self-attention mechanism (W-MSA), the present invention can capture long-range dependencies between different image blocks and fully combine the local and global information of the image, thereby more effectively improving the dehazing effect, especially when processing larger scenes, and better maintaining overall consistency.

[0049] 4. Enhanced detail recovery capability: The Multi-Scale Feature Fusion Module (MSFFM) effectively fuses local and global features through parallel dilated convolutions, ensuring that the network not only retains the global structural information of the image during image recovery, but also refines local details, improving the quality of the dehazed image and avoiding detail blurring and color distortion.

[0050] 5. Good generalization performance in real-world scenarios: This paper adopts a weakly supervised learning approach combined with an uncertainty-driven strategy, enabling the network to have good generalization capabilities in real-world scenarios even without a large amount of labeled data, and to adapt to different haze environments and complex lighting conditions.

[0051] 6. Excellent Visual Results: Qualitative comparisons show that our method can produce high-quality dehazed images in the RTTS test set from real-world scenes. Compared to other weakly supervised methods, our method effectively removes haze, avoiding issues such as oversaturation and localized distortion, while preserving edge details and contour information, resulting in more natural and clear dehazed images.

[0052] 7. Improved model generalization: Compared to other methods that rely on generative adversarial networks (GANs), this method significantly improves the network's generalization through uncertainty-driven weakly supervised learning. Even without training on real haze images, it maintains good dehazing performance in real scenes, overcomes domain shift issues, and adapts to the uneven haze distribution in different scenes.

[0053] 8. Enhanced training stability: Compared with the training instability and poor convergence problems of GAN models such as ART-SS-DA, the network architecture design and uncertainty-driven mechanism adopted in this paper make the model training process more stable, with faster convergence speed, improved training efficiency, and reduced model complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Schematic diagram of the uncertainty-driven weakly supervised dehazing network structure;

[0055] Figure 2 This is a schematic diagram of the structure of the regional dynamic attention module RDAM;

[0056] Figure 3 It is a structural diagram of the multi-scale feature fusion module MSFFM;

[0057] Figure 4 A comparison of the visual effects of different dehazing methods on the RTTS test set. DETAILED DESCRIPTION

[0058] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] This example proposes an uncertainty-driven weakly supervised dehazing method based on a weakly supervised dehazing network (Uncertainty-Driven Weakly Supervised Dehazing Network). The network consists of a Regional Dynamic Attention Module (RDAM), a downsampling layer, a Multi-Scale Feature Fusion Module (MSFFM), an upsampling layer, a 3x3 convolutional layer, a Relu activation function, and a 1x1 convolutional layer.

[0060] Regional Dynamic Attention Module (RDAM): Unlike traditional methods that focus solely on channel or spatial weight distribution, RDAM dynamically adjusts the weights on different channels and pixels through the Channel-Pixel Collaborative Attention (CP-CA) mechanism. This mechanism combines channel attention and pixel attention to dynamically adjust weights, improving the network's perception of key channels and regions. Specifically, channel attention uses average pooling to calculate the importance of each channel, while pixel attention calculates the importance of each pixel through convolution operations. The fusion of the two is used to weight the input features, enabling the network to focus on areas with higher fog concentrations.

[0061] Multi-Scale Feature Fusion Module (MSFFM): MSFFM utilizes W-MSA to learn long-range dependencies between image patches and further fuses and refines them through parallel dilated convolutions, effectively complementing global semantics with local details in dehazing tasks. The W-MSA module calculates global dependencies in the input feature map through a multi-head self-attention mechanism, then fuses multi-scale features through deconvolution and average pooling. Finally, parallel dilated convolutions are used to enhance the fusion of multi-scale features.

[0062] like Figure 1 As shown in Figure 1, the dehazing network structure of this embodiment is similar to the U-Net[9] architecture, which consists of a regional dynamic attention module (RDAM), a downsampling layer, a multi-scale feature fusion module (MSFFM), an upsampling layer, a 3x3 convolution layer, a ReLU activation function, and a 1x1 convolution layer. The specific process is as follows:

[0063] Input and multi-layer processing of regional dynamic attention modules (RDAM): The input of synthetic haze images or real-world haze images is first processed by three layers of regional dynamic attention modules (RDAM). The output of the previous RDAM layer is processed by a downsampling module and serves as the input of the next RDAM layer.

[0064] Multi-layer processing in the Multi-Scale Feature Fusion Module (MSFFM): The output features of each RDAM layer enter the Multi-Scale Feature Fusion Module (MSFFM) of the corresponding layer. The MSFFM is responsible for fusing features at different scales at multiple levels. The features input to the MSFFM are processed by the W-MSA module to capture long-range dependencies between pixels. The MSFFM performs deconvolution and average pooling on the input features, integrating features at different scales to achieve multi-level information extraction and fusion. The MSFFM also introduces parallel dilated convolution to enhance the fusion of local and global information.

[0065] Upsampling and Output: The upsampling layer restores the output of the third-layer multi-scale feature fusion module (MSFFM) to the output resolution of the second-layer multi-scale feature fusion module (MSFFM). It then performs feature concatenation with the output of the second-layer multi-scale feature fusion module (MSFFM). After a 3x3 convolutional layer, a Relu activation function, and a 1x1 convolutional layer, it is upsampled again and concatenated with the output of the first-layer multi-scale feature fusion module (MSFFM). After a 3x3 convolutional layer, a Relu activation function, and a 1x1 convolutional layer, the dehazed image is output.

[0066] Specifically, regarding the Regional Dynamic Attention Module (RDAM):

[0067] like Figure 2 As shown in Figure 2, RDAM designs a channel-pixel collaborative attention mechanism (CP-CA) to dynamically assign weights to different channels and pixels based on fog distribution characteristics. Channel attention and pixel attention are calculated using the following formulas:

[0068] f c =f in ·σ(Conv(δ(Conv(AvgPool(f in ))))))

[0069] f in represents the input feature map of the regional dynamic attention module; f c represents the channel attention feature map; AvgPool(·) represents the average pooling operation; Conv(·) is the convolution layer; δ(·) is the ReLU activation function; σ(·) is the Sigmoid activation function, which is used to limit the weight to the range of (0,1);

[0070] By inputting feature map f to the regional dynamic attention module in Perform average pooling, then pass through two convolutional layers and nonlinear activation calculations;

[0071] f p =f in ·σ(Conv(δ(Conv(f in ))))

[0072] f p Represents the pixel attention feature map, directly input feature map f in It is obtained by performing two convolutional layers and activation function calculations;

[0073] Then, the channel attention feature map and the pixel attention feature map are adaptively fused using the learnable parameter σ(θ):

[0074] f mix =σ(θ)·f c +(1-σ(θ))·f p

[0075] σ(θ) is a learnable parameter obtained through the Sigmoid function, and its value range is (0,1). mix Represents the channel-pixel collaborative attention feature map, which represents the weighted sum of channel attention and pixel attention, and is used for subsequent dynamic convolution.

[0076] In order to achieve region-adaptive convolution, RDAM designs a dynamic convolution layer based on spatial attention.

[0077] First, calculate the spatial attention feature f s :

[0078] f s =f in ·σ(Conv(concat(AvgPool(f in ),MaxPool(f in ))))

[0079] Enhance the spatial feature expression of the input feature map through spatial attention and calculate the regional weight W i :

[0080] W i =Softmax(Linear(AvgPool(f s ))),i=1,2,3,...,k

[0081] where f sRepresents spatial attention features; AvgPool(·) represents the average pooling operation; concat(·) represents the feature concatenation operation; MaxPool(·) represents the maximum pooling operation, Linear(·) represents the fully connected layer; Softmax(·) is the Softmax activation function; k is the number of regions; then, weight W is applied to k convolution kernels i Generate a set of region-adaptive dynamic convolution kernels conv i , the specific calculation process is:

[0082] conv i =W i conv0,i=1,2,3,...,k

[0083] Where conv0 is the initial convolution kernel. The obtained dynamic convolution kernel is used to construct the dynamic convolution layer DyConv().

[0084] Finally, the channel-pixel collaborative attention feature f mix After the dynamic convolution layer DyConv(), the regional dynamic attention feature f is obtained out for:

[0085] f out =DyConv(f mix )

[0086] Specifically, regarding the multi-scale feature fusion module (MSFFM) based on W-MSA:

[0087] like Figure 3 As shown, MSFFM processes the input x of the multi-scale feature fusion module through W-MSA in To learn the long-distance dependencies between image pixels, we can obtain the window multi-head self-attention feature x W-MSA , then perform deconvolution and average pooling operations to obtain 2 times up-sampled features x2 and 1 / 2 times down-sampled features x of different scales 1 / 2 , 1 / 4 times downsampled feature x 1 / 4 Finally, refine and fuse to obtain the multi-scale fusion feature Y out The specific operations are as follows:

[0088] x W-MSA =W-MSA(x in )

[0089] x in is the feature input to the multi-scale feature fusion module MSFFM; W-MSA(·) is the window multi-head self-attention mechanism operation; then the window multi-head self-attention feature x W-MSA Refine it through a separable shared convolution layer and a dilated convolution layer to obtain the original scale refined feature Y;

[0090] In parallel, multi-head self-attention features of the window x W-MSA Perform deconvolution and average pooling operations:

[0091] x2=Deconv(x W-MSA )

[0092] Deconv(·) is a deconvolution operation used to convert x W-MSA The resolution is doubled to obtain a 2x upsampled feature x2; then it is refined through a convolutional layer, a separable shared convolutional layer, and a dilated convolutional layer to obtain a 2x upsampled scale refined feature Y2;

[0093] x 1 / 2 =AvgPool2×2(x W-MSA )

[0094] AvgPool2×2(·) is a 2x2 average pooling operation used to convert x W-MSA The resolution is reduced by half to obtain 1 / 2 times downsampled features x 1 / 2 , and then refined through a transposed convolution layer, a separable shared convolution layer and an expanded convolution layer to obtain a 1 / 2 times downsampling scale refined feature Y 1 / 2 ;

[0095] x 1 / 4 =AvgPool4×4(x W-MSA )

[0096] AvgPool4×4(·) is a 4x4 average pooling operation, which obtains 1 / 4 times the downsampled feature x 1 / 4 , and then refined through two transposed convolutional layers, a separable shared convolutional layer and an expanded convolutional layer to obtain a 1 / 4 times downsampling scale refinement feature Y 1 / 4 ;

[0097] Then, the multi-scale refined features are concatenated and passed through a convolution layer and a ReLU activation function to output the multi-scale fusion feature Y out The calculation process is:

[0098] Y out =Relu(Conv(concat(Y 1 / 4 , Y 1 / 2 , Y, Y2)))

[0099] Through these operations, MSFFM effectively combines the global and local information of the input image, providing stronger feature expression capabilities for the dehazing task.

[0100] Specifically, regarding the setting of the loss function:

[0101] This example designs an unsupervised loss function based on heteroscedastic uncertainty. This loss function includes contrast loss, edge-preserving loss, and saturation loss. This allows the network to adaptively correct contrast, enhance edges, and improve saturation based on regional uncertainty during the dehazing process. Furthermore, for labeled data, a mean squared error loss function is used to measure the error between the predicted result and the true value.

[0102] (1) Heteroscedastic Uncertainty-Based Loss Function

[0103] Heteroscedastic uncertainty loss function assumes that the samples {x1,x2,…,x n} are independent and follow a Gaussian distribution, and each sample has a different variance, expressed as:

[0104]

[0105] Among them, μ i and Represents the i-th sample x i The mean and variance of , i = 1, 2…n. It is generally believed that a larger variance indicates a higher uncertainty in the pixels and features of the image.

[0106] x i : represents the i-th sample of the foggy image. μ i : is the mean of the i-th sample. is the variance of the i-th sample, indicating the uncertainty of pixels and features.

[0107] In order to enhance the visual quality of the dehazed image, the contrast loss L is used. contrast , edge preservation loss L edge and saturation loss L sat Adaptively correct contrast, enhance edges and improve saturation. This embodiment calculates the heteroscedastic uncertainty loss L uncertainty The process is as follows:

[0108]

[0109] in:

[0110] L contrast =C J -C I

[0111]

[0112] L sat =SJ -S I

[0113]

[0114] Where logσ i represents the logarithm of the variance, J(i, j) and I(i, j) are the pixel values ​​of the defogged image and the foggy image at position (i, j), respectively. and are the mean of the dehazed image and the hazy image, respectively. H and W are the height and width of the image, respectively. Canny(·) represents the Canny edge detection operator. γ is a learnable parameter obtained through the Sigmoid activation function, ranging from (0, 1).

[0115] (2) Mean Squared Error Loss Function

[0116] The mean squared error loss function is used for labeled data and calculates the average of the squared differences between the dehazed image and the haze-free image:

[0117]

[0118] J(i,j) represents the pixel value of the dehazed image at position (i,j). G(i,j) represents the pixel value of the haze-free image at position (i,j). H and W are the height and width of the image, respectively.

[0119] (3) Total Loss Function

[0120] The total loss function combines the above two loss functions in a linear weighted sum:

[0121] L total =λ uncertainty L uncertainty +λ mse L mse

[0122] λ uncertainty and λ mse are the hyperparameters of their respective loss functions. According to experience, when these coefficients are set to λ uncertainty = 0.5 and λ mse When =0.72, the dehazing result is the best.

[0123] Specifically, the dataset and training settings:

[0124] In a Linux environment, a weakly supervised dehazing network was built and trained using the PyTorch framework. An NVIDIA GeForce RTX 3090 GPU was used for training. The Adam optimizer was used to adjust model parameters during training, with momentum decay exponents set to β1 = 0.9 and β2 = 0.999. The input image size was set to 128 × 128, and the batch size was 16.

[0125] To facilitate training and evaluation, this example uses a public haze dataset. Specifically, from Haze4K [5] 3000 synthetic haze images were selected from the dataset as labeled training sets, and

[10] 3000 real haze images were selected from the Real-World Task-Driven Test Set (RTTS) in the dataset as the unlabeled training set. For testing, this embodiment uses the Synthetic Objective Test Set (SOTS) and Hybrid Subjective Test Set (HSTS) in RESIDE. SOTS includes 500 outdoor and 500 indoor synthetic haze images, while HSTS includes 10 outdoor synthetic haze images. In addition, we also added HazeRD

[11] The synthetic haze dataset contains 15 clear images and 60 haze images generated under different visibility conditions (1000m, 500m, 200m, and 100m). To evaluate the performance of the algorithm in real scenes, 500 real haze images were selected from RTTS as the test set to ensure that these images were not used in the training process.

[0126] In order to verify the defogging effect of the present invention, this embodiment adopts subjective and objective evaluation methods and compares them with mainstream methods. These methods are divided into three categories: (1) Defogging methods based on priors: DCP

[12] (2010) and FVR

[13] (2015); (2) Supervised learning dehazing method: DehazeNet

[14] (2016), AOD-Net

[15] (2017), EPDN

[16] (2019), SwinIR

[17] (2021), Dehamer

[18] (2022) and Restormer

[19] (2023); (3) Weakly supervised dehazing method: RefineDNet [2] (2021), PSD

[20] (2021), DMT [5] (2021), ART-SS-SSIM

[21] (2022), ART-SS-DA

[21] (2022), Semi-UFormer

[22] (2023), SDA-GAN

[23] (2023) and Semi-SGDRL

[24] (2024).

[0127] 1. Quantitative Comparison

[0128] Table 1 shows that the proposed method achieves the highest PSNR and SSIM values ​​among weakly supervised methods on the HSTS test set, with PSNR of 30.32 and SSIM of 0.972, respectively. Although it lags slightly behind ART-SS-DA on the HazeRD test set, with ART-SS-DA achieving a PSNR of 19.50 and our method achieving 18.88; and on the SOTS test set, with ART-SS-DA achieving a PSNR of 29.15 and our method achieving 29.23, it is worth noting that ART-SS-DA only enhances the generalization ability of the model by introducing real haze images, without considering the uncertainty factors in the image degradation process, which results in poor visual quality of the dehazed images. In addition, the core network of ART-SS-DA uses a generative adversarial network, which suffers from training instability and poor convergence. On the other hand, supervised approaches Restormer and DeHamer achieved leading performance on the SOTS dataset, with Restormer achieving a PSNR of 37.81 and an SSIM of 0.990, while DeHamer achieved a PSNR of 35.18 and an SSIM of 0.986. This significant performance can be attributed to the fact that the SOTS dataset is a synthetic dataset with more uniform scenes, allowing Transformer-based models to better capture and generalize the characteristics of the data.

[0129] Table 1: Quantitative comparison results of the present invention and mainstream technologies on SOTS, HSTS and HazeRD datasets.

[0130]

[0131]

[0132] 2. Qualitative Comparison

[0133] In order to verify the defogging effect of the present invention in real scenes, this embodiment compares it with the mainstream weakly supervised defogging method on the RTTS test set of real scenes. Figure 4(a)-(g) show the dehazed images generated using various weakly supervised methods. Since PSD relies too much on prior information, the image restoration performance is poor in real scenes with insufficient prior information, such as Figure 4 (b) shows oversaturated colors. Similarly, although RefineDNet alleviates the domain shift problem to some extent through weakly supervised training, it fails to fully consider the non-uniform distribution of haze in real scenes, resulting in over-dehazing problems, such as Figure 4 The ground area in (c) is too dark. In addition, DMT has designed sub-networks for estimating atmospheric light value and transmission rate respectively, which leads to error accumulation and local distortion in the dehazed image, such as Figure 4 However, ART-SS-DA and SDA-GAN fail to achieve adaptive weight distribution between channels and pixels, resulting in different degrees of distortion in the dehazed image, including Figure 4 (e) and Figure 4 (f) shows the residual haze and color shift. Figure 4 (g) demonstrates that the method of the present invention can effectively remove haze while retaining rich edge and contour details, producing excellent visual effects.

[0134] Preferably, the embodiments of the present application further provide a specific implementation of an electronic device capable of implementing all steps in the uncertainty-driven weakly supervised defogging method in the above embodiment, and the electronic device specifically includes the following contents:

[0135] Processor, memory, communications interface, and bus;

[0136] Among them, the processor, memory, and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between related devices such as server-side devices, metering devices, and user-side devices.

[0137] The processor is used to call the computer program in the memory, and when the processor executes the computer program, all steps of the uncertainty-driven weakly supervised defogging method in the above embodiment are implemented.

[0138] An embodiment of the present application also provides a computer-readable storage medium capable of implementing all steps of the uncertainty-driven weakly supervised defogging method in the above embodiment. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements all steps of the uncertainty-driven weakly supervised defogging method in the above embodiment.

[0139] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the hardware + program embodiments are generally similar to the method embodiments, so their description is relatively simple. For relevant portions, refer to the description of the method embodiments.

[0140] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0141] Although the present application provides method operation steps such as embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative work. The order of steps listed in the embodiments is only one way of executing the steps among many steps and does not represent the only execution order. When an actual device or client product is executed, it can be executed in the order shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0142] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0145] The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solutions of the present invention. The above specific embodiments are merely illustrative and not restrictive. Without departing from the scope of the present invention and the scope of protection of the claims, those skilled in the art may make various specific modifications based on the teachings of the present invention, all of which fall within the scope of protection of the present invention.

[0146] References:

[0147] [1]Singh M,Laxmi V,Faruki P.Visibility enhancement and dehazing:Research contribution challenges and direction[J].Computer Science Review,2022,44:100473.

[0148] [2]Zhao S, Zhang L, Shen Y, et al.Refinednet: A weakly supervisedrefinement framework for single image dehazing[J]. IEEE Transactions on ImageProcessing, 2021,30:3391-3404.

[0149] [3]Dong Y, Li Y, Dong Q, et al. Semi-supervised domain alignment learning for single image dehazing[J]. IEEE Transactions on Cybernetics, 2022.

[0150] [4]Zhang S, Zhang

[0151] [5]Liu Y,Zhu L,Pei S,et al.From synthetic to real:Image dehazingcollaborating with unlabeled real data[C] / / Proceedings of the 29th ACMinternational conference on multimedia.ACM,2021:50-58.

[0152] [6]Li L,Dong Y,Ren W,et al.Semi-supervised image dehazing[J].IEEETransactions on Image Processing,2019,29:2766-2779.

[0153] [7]Yu H,Huang J,Liu Y,et al.Source-free domain adaptation for real-world image dehazing[C] / / Proceedings of the 30th ACM International Conferenceon Multimedia.ACM,2022:6645-6654.

[0154] [8]Song Y,He Z,Qian H,et al.Vision transformers for single imagedehazing[J].IEEE Transactions on Image Processing,2023,32:1927-1941.

[0155] [9]Ronneberger O,Fischer P,Brox T.U-net:Convolutional networks forbiomedical image segmentation[C] / / Medical Image Computing and Computer-Assisted Intervention–MICCAI.Springer,2015:234-241.

[0156]

[10] Li B,Ren W,Fu D,et al.Benchmarking single-image dehazing andbeyond[J].IEEE Transactions on Image Processing,2018,28:492-505.

[0157]

[11] Zhang Y,Ding L,Sharma G.Hazerd:an outdoor scene dataset andbenchmark for single image dehazing[C] / / 2017 IEEE International Conference onImage Processing(ICIP).IEEE,2017:3205-3209.

[0158]

[12] He K,Sun J,Tang X.Single image haze removal using dark channelprior[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,33(12):2341-2353.

[0159]

[13] Tarel J,Hautiere N.Fast visibility restoration from a singlecolor or gray level image[C] / / 2009IEEE 12th International Conference onComputer Vision.IEEE,2009:2201-2208.

[0160]

[14] Cai B,Xu X,Jia K,et al.Dehazenet:An end-to-end system for singleimage haze removal[J].IEEE Transactions on Image Processing,2016,25(11):5187-5198.

[0161]

[15] Li B,Peng X,Wang Z,et al.Aod-net:All-in-one dehazing network[C] / / Proceedings of the IEEE international conference on computer vision.IEEE,2017:4770-4778.

[0162]

[16] Qu Y,Chen Y,Huang J,et al.Enhanced pix2pix dehazing network[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and PatternRecognition.IEEE,2019:8160-8168.

[0163]

[17] Liang J,Cao J,Sun G,et al.Swinir:Image restoration using swintransformer[C] / / Proceedings of the IEEE / CVF International Conference onComputer Vision,2021:1833-1844.

[0164]

[18] Guo C L,Yan Q,Anwar S,et al.Image dehazing transformer withtransmission-aware 3d position embedding[C] / / Proceedings of the IEEE / CVFconference on computer vision and pattern recognition,2022:5812-5820.

[0165]

[19] Zamir S W,Arora A,Khan S,et al.Restormer:Efficient transformerfor high-resolution image restoration[C] / / Proceedings of the IEEE / CVFconference on computer vision and pattern recognition,2022:5728-5739.

[0166]

[20] Chen Z,Wang Y,Yang Y,et al.Psd:Principled synthetic-to-realdehazing guided by physical priors[C] / / Proceedings of the IEEE / CVF Conferenceon Computer Vision and Pattern Recognition.IEEE,2021:7180-7189.

[0167]

[21] Yasarla R,Priebe C,Patel V.Art-ss:An adaptive rejection techniquefor semi-supervised restoration for adverse weather-affected images[C] / / European Conference on Computer Vision.Springer,2022:699-718.

[0168]

[22] Tong M,Wang Y,Cui P,et al.Semi-uformer:Semi-superviseduncertainty-aware transformer for image dehazing[J / OL].arXiv preprint arXiv:2210.16057,2022.

[0169]

[23] Dong Y,Li Y,Dong Q,et al.Semi-supervised domain alignmentlearning for single image dehazing[J].IEEE Transactions on Cybernetics,2022.

[0170]

[24] Jia T,Li J,Zhuo L,et al.Self-guided disentangled representationlearning for single image dehazing[J].Neural Networks,2024,172:106107.

Claims

1. A weakly supervised dehazing method based on uncertainty driving, characterized in that: It includes a weakly supervised dehazing network using the U-Net architecture. The weakly supervised dehazing network consists of a regional dynamic attention module (RDAM), a downsampling layer, a multi-scale feature fusion module (MSFFM), an upsampling layer, a 3x3 convolution layer, a ReLU activation function, and a 1x1 convolution layer. The regional dynamic attention module (RDAM) processes the input image and the output of the previous RDAM layer and constructs a dynamic convolution layer using spatial attention features. The RDAM uses two parallel convolution paths to calculate channel attention and pixel attention respectively, and then uses learnable parameters to weightedly fuse these two attentions to obtain a channel-pixel collaborative attention feature map. The regional dynamic attention feature map is obtained through the dynamic convolution layer. The downsampling layer is used to downsample the output of the regional dynamic attention module RDAM in the previous layer, reducing the spatial resolution to obtain the input of the regional dynamic attention module RDAM in the next layer; The multi-scale feature fusion module (MSFFM) processes the features input to the MSFFM through a windowed multi-head self-attention mechanism to capture long-range dependencies between pixels and obtain windowed multi-head self-attention features. The MSFFM performs deconvolution and average pooling on the windowed multi-head self-attention feature map to extract and fuse information at several levels. The module also introduces parallel dilated convolution to enhance the fusion of local and global information. The upsampling layer is used to restore the feature map output by the multi-scale feature fusion module MSFFM to the resolution of the feature map output by the multi-scale feature fusion module MSFFM in the previous layer, and perform feature splicing with the feature map output by the multi-scale feature fusion module MSFFM in the previous layer; finally, a 3x3 convolution layer, a ReLU activation function, and a 1x1 convolution layer are used to output the dehazed image; An unsupervised loss function based on heteroscedastic uncertainty is set, including contrast loss, edge preservation loss and saturation loss, so that the weakly supervised dehazing network can adaptively correct contrast, enhance edges and improve saturation according to regional uncertainty during the dehazing process.

2. The uncertainty-driven weakly supervised defogging method according to claim 1, characterized in that: In the regional dynamic attention module RDAM, channel attention and pixel attention are calculated by the following formulas respectively: ； represents the input feature map of the regional dynamic attention module, Represents channel attention features; represents the average pooling operation; is the convolutional layer; is the ReLU activation function; Sigmoid activation function is used to limit the weight to the range of (0, 1); By inputting feature maps Perform average pooling, then pass through two convolutional layers and nonlinear activation calculations; ； Represents pixel attention features, directly input feature map It is obtained by performing two convolutional layers and activation function calculations; Then use the learnable parameters Adaptively fuse channel attention features and pixel attention features: ； is a learnable parameter obtained through the Sigmoid function, with a value range of (0, 1); Represents the channel-pixel collaborative attention feature map, which represents the weighted sum of channel attention and pixel attention and is used for subsequent dynamic convolution; In order to achieve regional adaptive convolution, the regional dynamic attention module RDAM is equipped with a dynamic convolution layer based on spatial attention; First, calculate the spatial attention feature f s : ； Enhance the spatial feature expression of the input feature map through spatial attention and calculate the regional weight : ； where f s Represents spatial attention characteristics; represents the average pooling operation; Represents feature splicing operation; Max represents the maximum pooling operation, Represents a fully connected layer; Softmax is the Softmax activation function is the number of regions; Convolution kernels are weighted Generate a set of region-adaptive dynamic convolution kernels , the specific calculation process is: ； in is the initial convolution kernel; the dynamic convolution layer is constructed using the obtained dynamic convolution kernel ; Finally, the channel-pixel collaborative attention feature After the dynamic convolution layer , get the regional dynamic attention feature for: 。 3. The uncertainty-driven weakly supervised defogging method according to claim 1, characterized in that: The specific workflow in the multi-scale feature fusion module MSFFM is as follows: The multi-scale feature fusion module MSFFM processes the input through the window multi-head self-attention mechanism To learn the long-distance dependencies between image pixels, we can obtain the window multi-head self-attention features. , then perform deconvolution and average pooling operations to obtain 2x upsampled features of different scales , 1 / 2 times down-sampled features , 1 / 4 times down-sampled features Finally, refine and fuse to obtain multi-scale fusion features ; The specific operations are as follows: ； It is the feature input to the multi-scale feature fusion module MSFFM; Operate the multi-head self-attention mechanism for the window; then Refine it through a separable shared convolution layer and a dilated convolution layer to obtain the original scale refined feature Y; Parallel, multi-head self-attention features for windows Perform deconvolution and average pooling operations: ； is the deconvolution operation, which is used to The resolution is doubled, and 2x upsampling features are obtained. ; Then it is refined through a convolutional layer, a separable shared convolutional layer and a dilated convolutional layer to obtain 2 times upsampling scale refinement features ; ； is a 2x2 average pooling operation, used to The resolution is reduced by half to obtain 1 / 2 times down-sampled features , and then refined through a transposed convolution layer, a separable shared convolution layer and a dilated convolution layer to obtain 1 / 2 times downsampling scale refinement features ; ； It is a 4x4 average pooling operation to obtain 1 / 4 times down-sampled features , and then refined through two transposed convolutional layers, a separable shared convolutional layer and an expanded convolutional layer to obtain 1 / 4 times downsampling scale refinement features ; Then, the multi-scale refined features are spliced, and after a convolution layer and a Relu activation function, the multi-scale fusion features are output. , the calculation process is: ； Through these operations, the multi-scale feature fusion module MSFFM combines the global and local information of the input image and enhances the feature expression capability of the dehazing task.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the uncertainty-driven weakly supervised defogging method according to any one of claims 1 to 3 are implemented.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the uncertainty-driven weakly supervised defogging method according to any one of claims 1 to 3 are implemented.

Images