Single image defogging method and system

By combining the dynamic convolution module and the Transformer module in the image dehazing network, the problem of insufficient long-range image feature modeling in the existing technology is solved, and better dehazing effect and network convergence are achieved.

CN117036182BActive Publication Date: 2025-09-19XIDIAN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310705338.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-06-14
Publication Date
2025-09-19
Estimated Expiration
2043-06-14

AI Technical Summary

Technical Problem

Existing convolutional neural networks can only effectively model short-range image features during the dehazing process and have difficulty capturing long-range image features.

Method used

Combining dynamic convolution modules and Transformer modules, adding an attention mechanism, and designing an image defogging network based on an encoder-decoder architecture, the dynamic convolution modules and Transformer modules are used to capture the relationship between foggy feature maps, and the feature maps of different stages are fused in the model. The jump structure and gating module are used to optimize the network.

Benefits of technology

It achieves effective modeling of long-range image features, improves the dehazing effect, and enables the network to update gradients faster, alleviating the problems of gradient vanishing and gradient diffusion, thereby improving dehazing performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117036182B_ABST
    Figure CN117036182B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of image processing and computer vision technology, and discloses a single image defogging method and system. The method combines a dynamic convolution module and a Transformer module. The dynamic convolution can transform the parameters of the convolution kernel according to the input, and the Transformer can capture the relationship between the foggy feature maps. The combination of the two modules is used for defogging to achieve better results. At the same time, an attention mechanism is added to the two modules to make the model pay attention to the foggy area. The model is based on an encoder-decoder architecture, and the foggy image is encoded by the encoder and then decoded by the decoder to obtain a defogged image. A gating module is added to the design of the model, which can fuse the feature maps extracted at different stages, which is beneficial to the improvement of the defogging effect. The decoder adds a jump structure, which makes the gradient of the network updated faster, alleviates the gradient vanishing and gradient diffusion caused by the network being too deep, and makes the model easier to converge.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and computer vision, and in particular relates to a single image defogging method and system. Background Art

[0002] Currently, under weather conditions such as haze, the scattering effect of atmospheric particles severely degrades the quality of images captured by sensors. Therefore, image dehazing algorithms have become a key research area in image processing and computer vision. The primary task of image dehazing technology is to remove the impact of weather factors on image quality and enhance image visibility. In 1999, Srinivasa G. Narasimhan et al. developed an atmospheric scattering model to describe the formation of haze images.

[0003] Before the advent of deep learning, image dehazing algorithms primarily relied on image enhancement and image restoration. Image enhancement-based dehazing algorithms primarily aim to improve the contrast and brightness of foggy images to achieve dehazing. Image enhancement-based dehazing algorithms include histogram equalization, the Retinex algorithm, wavelet transforms, homomorphic filtering, and image enhancement algorithms based on fractional-order partial differential equations. Image restoration-based dehazing algorithms include dark channel priors, Fattal's single image dehazing algorithm, Tan's single image dehazing algorithm, Tarel's fast image restoration algorithm, and the Bayesian dehazing algorithm. Following the widespread adoption of deep learning, numerous convolutional neural network-based dehazing methods have been proposed. However, neural network-based dehazing algorithms can be divided into two categories. One category relies on atmospheric scattering models. Leveraging the powerful learning capabilities of neural networks, they learn the transmittance t and atmospheric light A, and then use these models to remove haze. Representative methods include DehazeNet and the densely connected pyramid dehazing network. The second category is that the network directly learns the mapping relationship between foggy images and dehazed images. Representative methods include All-in-OneNet, GCANet, Cycle-Dehaze and other networks.

[0004] Through the above analysis, the problems and defects of the existing technology are: the convolutional neural network can only model short-range image features during the dehazing process, and cannot effectively model long-range image features. Summary of the Invention

[0005] To address the above issues, the present invention designs a single-image defogging method that combines a dynamic convolution module and a Transformer module. Dynamic convolution transforms the parameters of the convolution kernel according to the input, while the Transformer captures the relationship between foggy feature maps. The combination of the two achieves good defogging results. An attention mechanism is also added to enable the model to focus on foggy areas. The model is designed based on an encoder-decoder architecture, encoding the foggy image through the encoder and then decoding it through the decoder to obtain the defogged image. A gating module is added to the model to fuse the feature maps extracted at different stages. The decoder uses transposed convolution to restore the size of the feature map to its original resolution. Finally, a jump structure is added to the end to enable faster updates of the network gradient.

[0006] Furthermore, the single image defogging method comprises the following steps:

[0007] Step 1: Build a training dataset;

[0008] Step 2: Build a network model to achieve image defogging: The network model is based on the U-Net structure. The encoding area first passes through three depth-separable modules to extract shallow features of the foggy image. The back end of the encoding area makes the dynamic convolution module and the transformer module run in parallel. Their features are fused in the parallel process. Finally, the gating module is used to fuse the low, medium and high groups of features to generate the final features of the encoding module. The decoding area uses two deconvolution modules for upsampling. Before the features enter the deconvolution module, a jump structure is used to add the primary features through the channel and pixel attention modules and the features entering the deconvolution module. Finally, a normal convolution layer is used to reduce the number of channels to 3 channels. The feature map is then added to the original foggy image to obtain the final defogging image.

[0009] Step 3: Use SmoothL1 loss and contrast loss to optimize the entire network by minimizing the error, as follows:

[0010]

[0011] Where: x represents the dehazed image:

[0012]

[0013] Where: w i is the weight coefficient, I represents the foggy image;

[0014] D represents L1 distance, G i represents the hidden layer features extracted from the i-th layer of the VGG19 pre-trained network, represents the fog-free image generated by the dehazing network after the foggy image passes through, w represents the weight of the network, and n represents the n hidden layers;

[0015] Step 4: Train the above dehazing network model;

[0016] Step 5: Test: Input the fog image to be dehazed into the network model trained in step 4, and the final output image is the dehazed image.

[0017] Furthermore, in step 2, the dynamic convolution module includes a dynamic dense residual block, a dynamic convolution layer, a channel attention, a pixel attention and a skip connection structure;

[0018] The dynamic dense residual block is composed of three dynamic convolutional layers and a dynamic convolutional layer with a final convolution kernel size of 1*1 and a jump connection structure;

[0019] The dynamic convolution layer is to combine four different convolution kernels into a weighted attention layer to form the final convolution kernel. The dynamic convolution process is expressed as follows:

[0020]

[0021] Where C represents the dynamic convolution operation, W(x) represents the convolution kernel parameters generated by sample adaptive, K represents the number of generated weights, and A k (x) represents the kth attention weight, W k Represents the kth convolution kernel parameter.

[0022] Furthermore, in step 2, channel attention is to first perform global average pooling on the features, a convolution layer reduces the features to 1 / 8 of the original, and then a relu activation function is performed. A convolution layer restores the number of channels of the features to the original number of channels, and then a sigmoid activation function is used to generate channel weights. The original features and weights are multiplied element by element, which is expressed as:

[0023] F r =sigmoid(conv(relu(conv(gap(x)))))*x

[0024] Where gap represents global average pooling, conv represents convolution operation, relu represents ReLU activation function, sigmoid represents Sigmoid activation function, and F r Represents the generated features;

[0025] Pixel attention first reduces the number of channels to 1 / 8 of the original through a convolution operation, then passes through the relu activation function, reduces the number of channels to 1 through a convolution operation, and generates weights through sigmoid, which is expressed as:

[0026] F r =sigmoid(conv(relu(conv(x))))*x

[0027] The Transformer module consists of layer normalization, MDTA module and GDFN module. The MDTA module is a multi-head attention module that generates corresponding Q, K, V for the feature map, weights V according to Q and K, and adds a jump structure; GDFN is a gated deep convolutional feedforward network, which consists of a convolution kernel with a 1*1 convolution kernel, a depth convolution kernel with a 3*3 convolution kernel, a gelu activation function and a jump structure.

[0028] Furthermore, the fusion module in step 2 is a convolutional layer that fuses the features passed through the tranformer module and the features passed through the dynamic convolution module, which can be expressed as:

[0029] F = conv(cat(x1, x2))

[0030] Among them, x1 represents the feature map generated by the transformer block, x2 is the feature map generated by the convolution module, cat represents the splicing operation, and the feature maps x1 and x2 are spliced ​​according to the channel. Conv represents the convolution operation.

[0031] Furthermore, the gating module in step 2 is a convolutional layer, which takes as input the low, medium, and high feature sets and outputs 3 channels. This means that weights corresponding to the three feature sets are generated, and then the context fusion features are obtained by multiplying and summing them according to the weights, which is expressed as:

[0032]

[0033] where x l , x m , x h They represent low, medium and high features respectively. Gate is a gating module. The features are weighted accordingly, and the weights are multiplied and summed with the original features to obtain the fused features.

[0034] The object of the present invention is to provide a single image defogging system for implementing the single image defogging method, the single image defogging system comprising:

[0035] Image scaling module, used to obtain foggy images and scale them to a resolution of 256*256;

[0036] Parameter loading module, used to load trained model parameters;

[0037] The image processing module is used to input the foggy image into the network to obtain the defogged image.

[0038] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0039] First, in response to the technical problems existing in the above-mentioned prior art and the difficulty of solving them, this paper, closely combining the technical solutions claimed by the present invention with the results and data from the research and development process, presents a detailed and in-depth analysis of the technical problems solved by the technical solutions of the present invention and the innovative technical effects achieved by solving these problems. Specifically, the paper describes the following: Combining a dynamic convolution module with a Transformer module, the dynamic convolution transforms the parameters of the convolution kernel according to the input, and the Transformer captures the relationship between foggy feature maps. Combining these two modules for defogging achieves excellent results. An attention mechanism is incorporated into both modules, enabling the model to focus on foggy areas. The model design is based on an encoder-decoder architecture, encoding the foggy image through the encoder and then decoding it through the decoder to produce the defogged image. A gating module is incorporated into the model design to fuse feature maps extracted at different stages, improving defogging performance. The decoder design uses transposed convolution to restore the feature map size to its original resolution. A skip-connection structure is added at the end to enable faster updates of the network's gradients, alleviating the gradient vanishing and gradient dispersion that can occur with overly deep networks, making the model more easily converged. Thanks to the added attention mechanism, the network can assign different weights to different fog concentrations based on the image input, thereby removing haze. Usage steps: Step 1: Obtain a foggy image; Step 2: Scale the image to a resolution of 256*256; Step 3: Load the trained model parameters; Step 4: Input the foggy image into the network to obtain the dehazed image.

[0040] Second, by employing parallel dynamic convolution and transformer blocks, channel attention and pixel attention are incorporated into both modules, enabling the network to effectively detect and maintain focus on foggy areas in the image. Dynamic convolution dynamically adjusts the convolution kernel based on the input, hence its adoption in the network design process. The Transformer block, formed by collectively referencing the Reformer architecture, comprises the MDTA module and the GDFN module. This transformer module captures long-range information essential for dehazing features. Fusion of features acquired through dynamic convolution and transformer processing facilitates the formation of better dehazing features, improving dehazing performance.

[0041] Third, the present invention introduces the Transformer block into the network, realizing the application of Transformer in the field of image defogging, so that CNN and Transformer can complement each other to improve the defogging effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1is a flow chart of a single image defogging method provided by an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of a single image defogging method according to an embodiment of the present invention;

[0044] Figure 3 This is a block diagram of the Transformer module provided by an embodiment of the present invention;

[0045] Figure 4 This is a block diagram of the MDTA module provided by an embodiment of the present invention;

[0046] Figure 5 This is a block diagram of a GDFN module provided by an embodiment of the present invention;

[0047] Figure 6 This is a block diagram of a dynamic residual module provided by an embodiment of the present invention;

[0048] Figure 7 It is a schematic diagram of some results of the SOTS test set provided by an embodiment of the present invention;

[0049] Figure 8 、 Figure 9 Schematic diagram of five NH-HAZE test images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] like Figure 1 As shown, the single image defogging method provided by the embodiment of the present invention includes the following steps:

[0052] S101: Obtain a foggy image;

[0053] S102: scaling the image to a resolution of 256*256;

[0054] S103: Load the trained model parameters;

[0055] S104: Input the foggy image into the network to obtain a defogged image.

[0056] The single image defogging method based on dynamic convolution and transformer provided in an embodiment of the present invention includes the following steps:

[0057] Step 1. Construction of training dataset: The present invention selects the Reside dataset and the NH-HAZE dataset as training sets. Specifically, the ITS data subset in the Reside dataset is used as the training set. The dataset consists of 1,399 clear indoor images. Different transmittances and atmospheric lights are selected to synthesize each clear image into 10 foggy images, and these foggy and clear image pairs are used as training sets. Since the NH-HAZE dataset has a small number of images, the dataset is expanded by cropping the original images, and finally the foggy and non-fog image pairs in the dataset are expanded to 3,877 pairs. The test images of the Reside dataset use the SOTS dataset (a subset of the Reside dataset), and the test images of the NH-HAZE dataset take the last 5 images in the dataset.

[0058] Step 2: Build a network to achieve image dehazing.

[0059] The overall network structure is as follows Figure 2 As shown, the network is based on the U-Net structure.

[0060] In the coding area, the Transformer module is used. Figure 3 As shown, the MDTA module of the Transformer module is as follows Figure 4 As shown, the GDFN module of the Transformer module is as follows Figure 5 As shown, the dynamic residual module is as follows Figure 6 As shown, their features are fused. The decoding area uses transposed convolution to restore the resolution of the feature map to the original resolution. Specifically, in the encoding area, three depth-separable convolution modules are first used to extract primary fog image features. The features are then input into the parallel dynamic convolution module and transformer module. There are three groups of dynamic convolution modules and transformer modules. After the features of each group are output, such features are combined to form a combined feature and input into the next group. At the end of the encoding module, the features of the three groups of parallel modules are aggregated through the gated fusion module as the final encoding output.

[0061] The decoding area includes a deconvolution module and a normal convolution module. Before the features enter the deconvolution module, the output features of the encoder module and the primary fog image features are added together through channel attention and pixel attention and then input into the deconvolution module. After the two deconvolution modules, the resolution of the features is increased to the original resolution. Then, the feature map and the features of the first depthwise convolution module are added together through channel attention and pixel attention and input into the last normal convolution layer. The output of this convolution layer is added to the original foggy image to obtain the final dehazed image.

[0062] Step 3: Determine the loss function. The present invention sets the weight of the contrast loss to 1, that is, the overall loss function is the SmoothL1 loss plus the contrast loss. The specific contrast loss function is to input the dehazed image, the foggy image, and the fog-free image into the vgg19 network, and use the feature maps output by the 2nd, 7th, 12th, 21st, and 30th layers of the vgg19 network. The corresponding weights are 1 / 32, 1 / 16, 1 / 8, 1 / 4, and 1 respectively to form the final contrast loss function.

[0063] Step 4: Train the above dehazing model. During the training process, the network parameters are updated iteratively by calculating the loss function, and finally an optimized dehazing model is obtained to perform image dehazing.

[0064] Step 5: Test. Input the hazy and clean image pairs from the test set into the dehazing network. The resulting output image is the dehazed image. Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM) are used as evaluation metrics.

[0065] Example 1:

[0066] The present invention is a single image defogging algorithm based on a dynamic convolution module and a transformer module. The single image defogging method comprises the following steps:

[0067] Step 1: Build a training dataset;

[0068] Step 2: Build a network model for image dehazing: The network model is based on the U-Net structure. The encoding region first passes through three depth-wise separable modules to extract shallow features of the foggy image. The back-end of the encoding region parallelizes the dynamic convolution module and the transformer module, fusing their features during the parallel process. Finally, a gating module is used to fuse the low, medium, and high feature groups to generate the final features of the encoding module. The decoding region uses two deconvolution modules for upsampling. Before the features enter the deconvolution module, a jump structure is used to add the primary features through the channel and pixel attention modules and the features entering the deconvolution module. Finally, a normal convolution layer is used to reduce the number of channels to 3 channels. This feature map is then added to the original foggy image to obtain the final dehazed image.

[0069] Step 3: Optimize the entire network by minimizing the error using SmoothL1 loss and contrast loss as follows:

[0070]

[0071] Where: x represents the dehazed image:

[0072]

[0073] Where: wi is the weight coefficient, I represents the foggy image;

[0074] D represents L1 distance, G i represents the hidden layer features extracted from the i-th layer of the VGG19 pre-trained network, represents the fog-free image generated by the dehazing network after the foggy image passes through, w represents the weight of the network, and n represents the n hidden layers;

[0075] Step 4: Train the above dehazing network model;

[0076] Step 5: Test: Input the fog image to be dehazed into the network model trained in step 4, and the final output image is the dehazed image.

[0077] In step 2, the dynamic convolution module includes dynamic dense residual blocks, dynamic convolution layers, channel attention, pixel attention, and skip structures.

[0078] The dynamic dense residual block is composed of three dynamic convolutional layers and a dynamic convolutional layer with a final convolution kernel size of 1*1 and a jump connection structure;

[0079] The dynamic convolution layer is to combine four different convolution kernels into a weighted attention kernel to form the final convolution kernel. The dynamic convolution process can be expressed as:

[0080]

[0081] Where C represents the dynamic convolution operation, W(x) represents the convolution kernel parameters generated by sample adaptive, K represents the number of generated weights, and A k (x) represents the kth attention weight, W k Represents the kth convolution kernel parameter;

[0082] Channel attention first performs global average pooling on the features, then a convolutional layer reduces the features to 1 / 8 of the original size, and then a relu activation function is performed. A convolutional layer restores the number of channels of the features to the original number of channels, and then a sigmoid activation function is used to generate channel weights. The original features and weights are multiplied element by element, which can be expressed as:

[0083] F r =sigmoid(conv(relu(conv(gap(x)))))*x

[0084] Where gap represents global average pooling, conv represents convolution operation, relu represents ReLU activation function, sigmoid represents Sigmoid activation function, and F rRepresents the generated features;

[0085] Pixel attention first reduces the number of channels to 1 / 8 of the original through a convolution operation, then passes through the relu activation function, and then reduces the number of channels to 1 through a convolution operation. The weight is generated through sigmoid, which can be expressed as:

[0086] F r =sigmoid(conv(relu(conv(x))))*x

[0087] Among them, conv, relu, sigmoid, F r Same as above;

[0088] The Transformer module consists of layer normalization, the MDTA module, and the GDFN module. The MDTA module is a multi-head attention module that generates the corresponding Q, K, and V for the feature map, weights V based on Q and K, and incorporates a skip connection structure. The GDFN is a gated deep convolutional feedforward network, primarily consisting of 1x1 convolutions, 3x3 depthwise convolutions, the gelu activation function, and a skip connection structure.

[0089] The fusion module in step 2 is a convolutional layer that fuses the features passed through the tranformer module and the features passed through the dynamic convolution module, which can be expressed as:

[0090] F = conv(cat(x1, x2))

[0091] Where x1 represents the feature map generated by the transformer block, x2 is the feature map generated by the convolution module, cat represents the splicing operation, and the feature maps x1 and x2 are spliced ​​according to the channel, and conv represents the convolution operation;

[0092] The gating module in step 2 is a convolutional layer. Its input is the low, medium, and high sets of features. Its output channel number is 3, which means that the weights corresponding to the three sets of features are generated. Then, the context fusion features are obtained by multiplying them together according to the weights. It can be expressed as:

[0093]

[0094] where x l , x m , x h They represent low, medium and high features respectively. Gate is a gating module. Features can generate corresponding weights through it. The weights are multiplied and summed with the original features to obtain the fused features.

[0095] The single image defogging system provided by an embodiment of the present invention includes:

[0096] Image scaling module, used to obtain foggy images and scale them to a resolution of 256*256;

[0097] Parameter loading module, used to load trained model parameters;

[0098] The image processing module is used to input the foggy image into the network to obtain the defogged image.

[0099] Through the above model design, implementation and training, the present invention tests the defogging effect of the model in the test set as an application example.

[0100] The dehazing results of the test set are shown. The visual results of the image dehazing model on the 500 images of the SOTS test set are as follows Figure 7 As shown in the figure, only five test photos are shown. The average peak signal-to-noise ratio obtained in this dataset is 27.0256, and the average structural similarity is 0.9330. The highest peak signal-to-noise ratio is 33.472, and the highest structural similarity is 0.9831. A good dehazing result is achieved. The visual results of the five images tested in NH-HAZE are as follows Figure 8 and Figure 9 As shown, the average peak signal-to-noise ratio and structural similarity of the five images are 17.525 and 0.6119 respectively.

[0101] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A single image defogging method, characterized in that: The proposed single-image defogging method combines a dynamic convolution module and a Transformer module. The dynamic convolution transforms the parameters of the convolution kernel according to the input, while the Transformer captures the relationship between the foggy feature maps. The combination of the two achieves good defogging results. At the same time, the addition of an attention mechanism enables the model to focus on foggy areas. The model is designed based on an encoder-decoder architecture. The foggy image is encoded by the encoder and then decoded by the decoder to obtain the defogged image. A gating module is added to the model to fuse the feature maps extracted at different stages. The decoder uses transposed convolution to restore the size of the feature map to the original resolution. A skip connection structure is added at the end to achieve faster network gradient updates. The specific steps are as follows: Step 1: Build a training dataset; Step 2: Build a network model for image dehazing: The network model is based on the U-Net structure. In the encoding region, three depth-wise separable modules are used to extract shallow features of foggy images. The backend of the encoding region runs the dynamic convolution module and the transformer module in parallel, fusing their features during the parallel process. Finally, a gating module is used to fuse the low, medium, and high feature sets to generate the final features of the encoding module. The decoding area is upsampled using two deconvolution modules. Before the features enter the deconvolution module, a skip-connection structure is used to add the primary features after passing through the channel and pixel attention modules and the features entering the deconvolution module. Finally, a normal convolution layer is used to reduce the number of channels to 3. The feature map is then added to the original foggy image to obtain the final dehazed image. Step 3: Use SmoothL1 loss and contrast loss to optimize the entire network by minimizing the error, as follows: Where: x represents the dehazed image: Where: wi is the weight coefficient, I represents the foggy image; D represents L1 distance, G i represents the hidden layer features extracted from the i-th layer of the VGG19 pre-trained network, represents the fog-free image generated by the dehazing network after the foggy image passes through, w represents the weight of the network, and n represents the n hidden layers; Step 4: Train the above dehazing network model; Step 5: Test: Input the fog image to be dehazed into the network model trained in step 4, and the final output image is the dehazed image.

2. The single image defogging method according to claim 1, wherein: In step 2, the dynamic convolution module includes dynamic dense residual blocks, dynamic convolution layers, channel attention, pixel attention, and skip connection structures; The dynamic dense residual block is composed of three dynamic convolutional layers and a dynamic convolutional layer with a final convolution kernel size of 1*1 and a jump connection structure; The dynamic convolution layer is to combine four different convolution kernels into a weighted attention layer to form the final convolution kernel. The dynamic convolution process is expressed as follows: Where C represents the dynamic convolution operation, W(x) represents the convolution kernel parameters generated by sample adaptive, K represents the number of generated weights, and A k (x) represents the kth attention weight, W k Represents the kth convolution kernel parameter.

3. The single image defogging method according to claim 2, wherein: In step 2, channel attention first performs global average pooling on the features, a convolutional layer reduces the features to 1 / 8 of the original, and then performs a relu activation function. A convolutional layer restores the number of channels of the features to the original number of channels, and then generates channel weights through a sigmoid activation function. The original features and weights are multiplied element by element, which is expressed as: F r =sigmoid(conv(relu(conv(gap(x)))))*x Where gap represents global average pooling, conv represents convolution operation, relu represents ReLU activation function, sigmoid represents Sigmoid activation function, and F r Represents the generated features; Pixel attention first reduces the number of channels to 1 / 8 of the original through a convolution operation, then passes through the relu activation function, reduces the number of channels to 1 through a convolution operation, and generates weights through sigmoid, which is expressed as: F r =sigmoid(conv(relu(conv(x))))*x The Transformer module consists of layer normalization, MDTA module and GDFN module. The MDTA module is a multi-head attention module that generates corresponding Q, K, V for the feature map, weights V according to Q and K, and adds a skip structure; GDFN is a gated deep convolutional feedforward network, which consists of a convolution with a convolution kernel of 1*1, a depth convolution with a convolution kernel of 3*3, a gelu activation function and a skip structure.

4. The single image defogging method according to claim 1, wherein: The fusion module in step 2 is a convolutional layer that fuses the features passed through the tranformer module and the features passed through the dynamic convolution module, which can be expressed as: F = conv(cat(x1, x2)) Among them, x1 represents the feature map generated by the transformer block, x2 is the feature map generated by the convolution module, cat represents the splicing operation, and the feature maps x1 and x2 are spliced ​​according to the channel. Conv represents the convolution operation.

5. The single image defogging method according to claim 1, wherein: The gating module in step 2 is a convolutional layer. The input is the low, medium, and high feature sets, and the output channel number is 3. That is, the weights corresponding to the three sets of features are generated, and then they are multiplied and summed according to the weights to obtain the context fusion feature, which is expressed as: [0026] where x l , x m , x h They represent low, medium and high features respectively. Gate is the gating module. The features generate corresponding weights, and the fused features are obtained by multiplying the weights and the original features.

6. A single image defogging system for implementing the single image defogging method according to any one of claims 1 to 5, characterized in that: The single image defogging system comprises: Image scaling module, used to obtain foggy images and scale them to a resolution of 256*256; Parameter loading module, used to load trained model parameters; The image processing module is used to input the foggy image into the network to obtain the defogged image.

Citation Information

Patent Citations

  • Image blind denoising method and system based on enhanced Transform

    CN114820341A

  • Image defogging method based on codec structure

    CN114820388A