Single image defogging method and device

Abstract

The application discloses a single-image defogging method and device, and the method comprises the following steps: obtaining an image to be defogged; inputting the image to be defogged into a pre-constructed defogging network model to obtain a defogged image; wherein the construction process of the defogging network model comprises the following steps: obtaining a clear image dataset, processing clear images in the clear image dataset through an atmospheric scattering model to generate foggy images; constructing a training dataset by one-to-one correspondence between the clear images and the foggy images; constructing a defogging network model based on a U-Net network model, training the defogging network model according to the training dataset, and obtaining the constructed defogging network model after the training is completed. The application can effectively remove fog and retain more image detail information.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and apparatus for dehazing a single image. Background Technology

[0002] In contemporary visual communication, images occupy a crucial media position. However, the impact of smog pollution on image quality, especially in high-tech applications such as target recognition, satellite remote sensing analysis, and image classification, poses a significant challenge. Specifically, the light propagation process in smoggy environments is interfered with by the scattering and absorption of suspended water droplets and particles, resulting in reduced edge sharpness of the subject. Image sharpness and color saturation are significantly decreased, thus weakening the overall visual expressiveness of the image. This phenomenon not only reduces the direct resolvability of images but also potentially obscures key information within the image, and in severe cases, hinders the accuracy and reliability of subsequent scientific research, highlighting the restrictive effect of smog on the application of high-precision image analysis techniques.

[0003] From the perspective of image processing theory, dehazing algorithms can be systematically classified into three categories: (1) image dehazing algorithms based on image restoration; (2) image dehazing algorithms based on image enhancement; and (3) image dehazing algorithms based on deep learning. Image restoration algorithms fundamentally explore the image generation mechanism and construct a comprehensive atmospheric scattering model by precisely analyzing the imaging effect caused by fog. This model integrates key physical processes such as light scattering and attenuation. This type of algorithm can accurately distinguish and extract the original image components affected by fog by having a thorough understanding of the deep physical mechanism of image degradation. The effectiveness of this type of method is highly dependent on the preset a priori conditions, which limits its application scope. Moreover, the dehazing effect may not be satisfactory when encountering high-intensity interference environments, showing certain limitations. Dehazing strategies based on image enhancement technology aim to improve the visual quality affected by fog by exploring the low brightness and low contrast features contained in the image. However, while pursuing the improvement of image clarity and visibility, this strategy inevitably faces the challenge of potentially damaging fine structural details and color fidelity. This trade-off may result in a compromise between detail resolution and color accuracy in the dehazed image, with overall dehazing performance reaching only a moderate level. With the rapid development of deep learning, methods for obtaining haze-free images based on deep learning have gradually become mainstream. The core of the algorithm is to construct a convolutional neural network as a deep learning model, utilizing the learning capabilities of the neural network to predict parameters to achieve the dehazing goal. Although end-to-end deep learning algorithms can effectively mitigate the negative impact of fog on images to a large extent, their dehazing mechanism may produce side effects. While striving to eliminate fog, it reduces some of the detailed features of the original image, reflecting limitations in maintaining the fidelity of the image's inherent detail texture. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and apparatus for dehazing a single image, which can retain more image detail information while effectively dehazing.

[0005] To achieve the above objectives, the present invention is implemented using the following technical solution: In a first aspect, the present invention provides a method for dehazing a single image, comprising: Obtain the image to be dehazed; The image to be dehazed is input into a pre-constructed dehazing network model to obtain the dehazed image; The construction process of the defogging network model includes: Obtain a clear image dataset, and process the clear images in the clear image dataset using an atmospheric scattering model to generate images with fog; A training dataset is constructed by matching the clear images and the fogged images one-to-one. A dehazing network model is constructed based on the U-Net network model. The dehazing network model is trained using the training dataset. After training, the constructed dehazing network model is obtained. The U-Net network model includes a symmetric encoder and decoder. The encoder includes multiple standard convolutional layers and pooling layers. A predetermined number of standard convolutional layers at the end are replaced with depthwise separable convolutional layers, and a densely connected layer is introduced at the end of the depthwise separable convolutional layers. A multi-head attention mechanism is introduced in the pooling layers. The decoder includes multiple interleaved deconvolutional layers and feature enhancement layers.

[0006] Optionally, the depth-separable convolutional layer is used to perform depthwise convolution and pointwise convolution on the input image data; The depthwise convolution includes: For each channel of the input image, a depthwise convolution operation is performed using a depthwise convolution kernel to extract feature maps: ; In the formula, The feature map is extracted by depthwise convolution. For feature map Upper position The The values ​​corresponding to each channel For the input image Upper position The The values ​​corresponding to each channel The height and width of the depthwise convolution kernel. For depthwise convolution kernels at position The The weights corresponding to each channel; The pointwise convolution includes: For the feature map extracted by the depthwise convolution, a 1 Pointwise convolution kernels are used to perform pointwise convolution operations to extract feature maps: ; In the formula, The feature map is extracted by pointwise convolution. For feature map Upper position The The values ​​corresponding to each channel 1 Pointwise convolution kernel at the th The input channel and the first The weights of each output channel; 1 The number of input channels for the pointwise convolution kernel.

[0007] Optionally, the dense connection layer is used to connect the feature maps output by the current layer and all previous layers.

[0008] Optionally, the pooling layer, after introducing a multi-head attention mechanism, performs pooling processing on the feature map output by the depthwise separable convolutional layer, including: For the input feature map The feature map is obtained by mapping to a new feature space through convolution operations. ; It has Head attention mechanisms, calculating the weights of each head attention mechanism: ; In the formula, , It is a linear transformation matrix; According to weight Calculate attention head : ; In the formula, The dimension of the attention head; All attention heads are spliced ​​and linearly transformed: ; In the formula, It is a linear transformation matrix; The splicing and linear transformation results With feature map The pooling result is obtained by summing. : .

[0009] Optionally, the feature enhancement layer is used to perform a nonlinear transformation on the output of the deconvolution layer using the ReLU activation function.

[0010] Secondly, the present invention provides a single-image dehazing device, the device comprising: The image acquisition module is used to acquire the image to be dehazed; The image dehazing module is used to input the image to be dehazed into a pre-constructed dehazing network model to obtain the dehazed image; The construction process of the defogging network model includes: Obtain a clear image dataset, and process the clear images in the clear image dataset using an atmospheric scattering model to generate images with fog; A training dataset is constructed by matching the clear images and the fogged images one-to-one. A dehazing network model is constructed based on the U-Net network model. The dehazing network model is trained using the training dataset. After training, the constructed dehazing network model is obtained. The U-Net network model includes a symmetric encoder and decoder. The encoder includes multiple standard convolutional layers and pooling layers. A predetermined number of standard convolutional layers at the end are replaced with depthwise separable convolutional layers, and a densely connected layer is introduced at the end of the depthwise separable convolutional layers. A multi-head attention mechanism is introduced in the pooling layers. The decoder includes multiple interleaved deconvolutional layers and feature enhancement layers.

[0011] Thirdly, the present invention provides an electronic device, including a processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps according to the method described above.

[0012] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0013] Fifthly, the present invention provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0014] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention provides a single-image dehazing method and apparatus that, while dehazing the image as a whole, focuses more on restoring the fine details of the hazy image. This overcomes the problem in existing dehazing algorithms that tend to remove some details from the original image along with the image. It can retain more image details, reduce the halos at the edges of the dehazed image, generate a high-quality dehazed image, and reduce the computation time consumed by image processing. Attached Figure Description

[0015] Figure 1 This is a schematic flowchart of the single-image dehazing method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the defogging network model provided in an embodiment of the present invention. Detailed Implementation

[0016] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0017] Example 1:

[0018] like Figure 1 As shown, this embodiment of the invention provides a method for dehazing a single image, including the following steps: Step S01: Obtain the image to be dehazed.

[0019] Step S02: Input the image to be dehazed into the pre-built dehazing network model to obtain the dehazed image.

[0020] The process of constructing the defogging network model includes the following steps: Step S11: Obtain a clear image dataset, and process the clear images in the clear image dataset using an atmospheric scattering model to generate images with fog.

[0021] Step S12: Construct a training dataset by matching clear images and foggy images one-to-one.

[0022] Step S13: Construct a dehazing network model based on the U-Net network model. Train the dehazing network model using the training dataset. After training, the constructed dehazing network model is obtained.

[0023] The U-Net network model includes a symmetric encoder and decoder. The encoder includes multiple standard convolutional layers and pooling layers, with a predetermined number of standard convolutional layers at the tail replaced by depthwise separable convolutional layers, and densely connected layers introduced at the tail of the depthwise separable convolutional layers. A multi-head attention mechanism is introduced in the pooling layers. The decoder includes multiple interleaved deconvolutional layers and feature enhancement layers.

[0024] like Figure 2 As shown, the defogging network model includes one standard convolutional layer with a 7×7 kernel, two depthwise separable convolutional layers with a 3×3 kernel, two pooling layers with multi-head attention mechanisms, two densely connected layers, one fully connected layer, two deconvolutional layers with a 3×3 kernel, three enhancement modules, and one 7×7 deconvolutional layer.

[0025] Depthwise separable convolutional layers are used to perform depthwise convolution and pointwise convolution on input image data. Depthwise convolution operates in the channel dimension, while pointwise convolution operates in the spatial dimension. This significantly reduces the number of parameters in the model.

[0026] Depthwise convolution includes: For each channel of the input image, a depthwise convolution operation is performed using a depthwise convolution kernel to extract feature maps: ; In the formula, The feature map is extracted by depthwise convolution. For feature map Upper position The The values ​​corresponding to each channel For the input image Upper position The The values ​​corresponding to each channel The height and width of the depthwise convolution kernel. For depthwise convolution kernels at position The The weights corresponding to each channel; Pointwise convolution includes: For the feature map extracted by depthwise convolution, a 1 Pointwise convolution kernels are used to perform pointwise convolution operations to extract feature maps: ; In the formula, The feature map is extracted by pointwise convolution. For feature map Upper position The The values ​​corresponding to each channel 1 Pointwise convolution kernel at the th The input channel and the first The weights of each output channel; 1 The number of input channels for the pointwise convolution kernel.

[0027] Densely connected layers connect the feature maps output by the current layer to all previous layers. Densely connected layers allow each layer to directly connect to all subsequent layers. In a densely connected module, the output of each layer is directly connected to all its subsequent layers, not just the next layer. This means that the input of the nth layer is the set of outputs from all preceding layers. This allows for more efficient use of the feature information from previous layers, as each layer can access features from all previous layers, thus better preserving low-level and mid-level features. This is extremely useful for tasks requiring detail preservation, such as image dehazing, thereby improving the network's semantic understanding capabilities.

[0028] Pooling layers, after introducing a multi-head attention mechanism, perform pooling processing on the feature maps output by depthwise separable convolutional layers, including: For the input feature map The feature map is obtained by mapping to a new feature space through convolution operations. ; It has Head attention mechanisms, calculating the weights of each head attention mechanism: ; In the formula, , It is a linear transformation matrix; According to weight Calculate attention head : ; In the formula, The dimension of the attention head; All attention heads are spliced ​​and linearly transformed: ; In the formula, It is a linear transformation matrix; The splicing and linear transformation results With feature map The pooling result is obtained by summing. : .

[0029] An attention mechanism is introduced into the pooling layer, enabling the network to focus on foggy areas in the image. This helps to remove fog more effectively, improving the accuracy and stability of the dehazing effect. It also helps the network retain important details in the image, such as texture and edges, thereby improving the clarity and quality of the dehazed image. By adaptively adjusting the contrast and brightness of different regions, attention pooling makes the network more adaptable to different lighting conditions and levels of fog, improving the network's adaptability and robustness.

[0030] The feature enhancement layer is used to perform a non-linear transformation on the output of the deconvolution layer using the ReLU activation function. The feature enhancement layer is primarily used to further process and improve the quality of the dehazed image. It enhances the contrast, sharpness, and details of the dehazed image to improve the overall visual effect and clarity.

[0031] Example 2: This invention provides a single-image dehazing device, the device comprising: The image acquisition module is used to acquire the image to be dehazed; The image dehazing module is used to input the image to be dehazed into a pre-built dehazing network model to obtain the dehazed image; The construction process of the defogging network model includes: Obtain a clear image dataset, and process the clear images in the clear image dataset using an atmospheric scattering model to generate images with fog; A training dataset is constructed by matching clear images and foggy images one-to-one. A dehazing network model is constructed based on the U-Net network model. The dehazing network model is trained using the training dataset, and the constructed dehazing network model is obtained after training. The U-Net network model includes a symmetric encoder and decoder. The encoder includes multiple standard convolutional layers and pooling layers. A predetermined number of standard convolutional layers at the end are replaced with depthwise separable convolutional layers, and a densely connected layer is introduced at the end of the depthwise separable convolutional layers. A multi-head attention mechanism is introduced in the pooling layers. The decoder includes multiple interleaved deconvolutional layers and feature enhancement layers.

[0032] Example 2:

[0033] Based on Embodiment 1, this embodiment of the invention provides an electronic device, including a processor and a storage medium; Storage media are used to store instructions; The processor is used to perform operations according to instructions to execute the steps according to the method described above.

[0034] Example 3:

[0035] Based on Embodiment 1, this embodiment of the invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0036] Example 4:

[0037] Based on Embodiment 1, this embodiment of the invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-described method.

[0038] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0039] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0040] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0041] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0042] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for dehazing a single image, characterized in that, include: Obtain the image to be dehazed; The image to be dehazed is input into a pre-constructed dehazing network model to obtain the dehazed image; The construction process of the defogging network model includes: Obtain a clear image dataset, and process the clear images in the clear image dataset using an atmospheric scattering model to generate images with fog; A training dataset is constructed by matching the clear images and the fogged images one-to-one. A dehazing network model is constructed based on the U-Net network model. The dehazing network model is trained using the training dataset. After training, the constructed dehazing network model is obtained. The U-Net network model includes a symmetric encoder and decoder. The encoder includes multiple standard convolutional layers and pooling layers. A predetermined number of standard convolutional layers at the end are replaced with depthwise separable convolutional layers, and a densely connected layer is introduced at the end of the depthwise separable convolutional layers. A multi-head attention mechanism is introduced in the pooling layers. The decoder includes multiple interleaved deconvolutional layers and feature enhancement layers. The depth-separable convolutional layer is used to perform depth convolution and pointwise convolution on the input image data; The depthwise convolution includes: For each channel of the input image, a depthwise convolution operation is performed using a depthwise convolution kernel to extract feature maps: ； In the formula, The feature map is extracted by depthwise convolution. For feature map Upper position The The values ​​corresponding to each channel For the input image Upper position The The values ​​corresponding to each channel The height and width of the depthwise convolution kernel. For depthwise convolution kernels at position The The weights corresponding to each channel; The pointwise convolution includes: For the feature map extracted by the depthwise convolution, a 1 Pointwise convolution kernels are used to perform pointwise convolution operations to extract feature maps: ； In the formula, The feature map is extracted by pointwise convolution. For feature map Upper position The The values ​​corresponding to each channel 1 Pointwise convolution kernel at the th The input channel and the first The weights of each output channel; 1 The number of input channels for a pointwise convolution kernel; The pooling layer performs pooling processing on the feature map output by the depthwise separable convolutional layer after introducing a multi-head attention mechanism, including: For the input feature map The feature map is obtained by mapping to a new feature space through convolution operations. ; It has Head attention mechanisms, calculating the weights of each head attention mechanism: ； In the formula, , It is a linear transformation matrix; According to weight Calculate attention head : ； In the formula, The dimension of the attention head; All attention heads are spliced ​​and linearly transformed: ； In the formula, It is a linear transformation matrix; The splicing and linear transformation results With feature map The pooling result is obtained by summing. : 。 2. The single-image dehazing method according to claim 1, characterized in that, The dense connection layer is used to connect the feature maps output by the current layer and all previous layers.

3. The single-image dehazing method according to claim 1, characterized in that, The feature enhancement layer is used to perform a nonlinear transformation on the output of the deconvolution layer using the ReLU activation function.

4. A single-image dehazing device, characterized in that, The apparatus is configured to perform the steps of the method as described in any one of claims 1-3, the apparatus comprising: The image acquisition module is used to acquire the image to be dehazed; The image dehazing module is used to input the image to be dehazed into a pre-constructed dehazing network model to obtain the dehazed image; The construction process of the defogging network model includes: Obtain a clear image dataset, and process the clear images in the clear image dataset using an atmospheric scattering model to generate images with fog; A training dataset is constructed by matching the clear images and the fogged images one-to-one. A dehazing network model is constructed based on the U-Net network model. The dehazing network model is trained using the training dataset. After training, the constructed dehazing network model is obtained. The U-Net network model includes a symmetric encoder and decoder. The encoder includes multiple standard convolutional layers and pooling layers. A predetermined number of standard convolutional layers at the end are replaced with depthwise separable convolutional layers, and a densely connected layer is introduced at the end of the depthwise separable convolutional layers. A multi-head attention mechanism is introduced in the pooling layers. The decoder includes multiple interleaved deconvolutional layers and feature enhancement layers.

5. An electronic device, characterized in that, Including processor and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1-3.

7. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-3.

Images