Real haze image defogging method based on haze degradation model

Abstract

The invention discloses a real haze image defogging method based on a haze degradation model. The method comprises the following steps: constructing a haze degradation model fusing multiple scattering effects and multiple image degradation factors; using the haze degradation model to construct a training data set including the clear image and the corresponding pseudo haze image; constructing a defogging network for a real haze scene; training the defogging network by adopting the training data set until a preset loss function is converged; and inputting a to-be-defogged image into the trained defogging network to obtain a defogging result. According to the haze degradation model constructed by the invention, the difference between a synthetic domain and a real domain is effectively relieved; a designed space-frequency hybrid module improves the adaptability of the model to complex degradation characteristics; the prior-guided feed-forward network fully excavates and fuses dark channel prior information, and the sensing and modeling capability of the model to the haze area is effectively enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer image processing, and in particular to a real fog and haze image defogging method based on a fog and haze degradation model. BACKGROUND

[0002] Under the condition of fog and haze, the suspended particles in the atmosphere will scatter and absorb light, resulting in degradation phenomena such as contrast reduction, color distortion, and detail blurring in the captured image. This degradation not only affects the subjective perception of the human eye, but also reduces the accuracy of advanced vision tasks such as automatic driving, video monitoring, and target detection. In recent years, with the rapid development of computer vision technology, image defogging as an important research direction of bottom-level vision processing has attracted widespread attention.

[0003] Existing image defogging methods can be divided into image prior-based defogging methods and deep learning-based defogging methods. Image prior-based defogging methods usually rely on the atmospheric scattering model. These methods solve the transmission rate and global atmospheric light in the imaging process through image statistical prior, and realize image defogging. However, due to the complexity and uncertainty of fog and haze, the prior assumptions relied on by traditional methods cannot accurately capture the changes of all fog and haze scenes. In some scenes where prior knowledge does not hold, the estimated transmission rate will deviate, resulting in artifacts in the defogging results.

[0004] With the rapid development of deep learning, deep learning-based defogging methods have made significant progress. Early defogging networks mainly use large-scale synthetic datasets for training, and learn the mapping relationship from fog and haze images to clear images through end-to-end learning. These methods perform well on synthetic images, but their performance drops significantly when faced with real fog and haze images, making it difficult to adapt to complex and variable actual scenes. In recent years, some defogging methods for real fog and haze images have been proposed to address the significant domain difference between synthetic data and real data. These methods usually use semi-supervised or unsupervised strategies to improve the network's generalization ability in real environments. However, existing methods still have difficulty in fully simulating the diverse degradation characteristics of real fog and haze, resulting in deficiencies in brightness, color, and detail recovery in the defogging results. SUMMARY

[0005] To solve the above technical problems, the present application provides a real fog and haze image defogging method based on a fog and haze degradation model, which has simple algorithm and good defogging effect.

[0006] The technical solution of the present application to solve the above technical problems is: a real fog and haze image defogging method based on a fog and haze degradation model, comprising the following steps:

[0007] Step 1: Construct a fog and haze degradation model that fuses multiple scattering effects and multiple image degradation factors;

[0008] Step two: use the haze degradation model to build a training dataset containing clear images and corresponding pseudo-haze images;

[0009] Step three: build a haze removal network for real haze scenes;

[0010] Step four: train the haze removal network using the training dataset until the pre-set loss function converges;

[0011] Step five: input the image to be dehazed into the trained haze removal network to obtain the dehazing result.

[0012] The above real haze image dehazing method based on the haze degradation model, in step one, the haze degradation model is constructed as follows:

[0013]

[0014] wherein, represents a pixel; represents the haze image intensity obtained; represents the real clear image intensity; represents the transmittance, reflecting the degree of attenuation of light in the propagation process; represents the global atmospheric light; represents the multiple scattering term, wherein represents the multiple scattering domain centered on the pixel , represents the scattering contribution of the pixel , represents the intensity of the real clear image at the pixel , represents the transmittance at the pixel , and respectively represent the scattering coefficient and the scene depth; represents the multiple scattering weight coefficient, used to adjust the intensity of the multiple scattering effect; represents the Gaussian noise; represents the color deviation disturbance term; represents the compression artifact. The above real haze image dehazing method based on the haze degradation model, in step two, 500 real clear images are selected, the depth information map of the clear image is estimated using a pre-trained depth estimation network, the value range of each parameter in the haze degradation model is set, and a parameter combination is randomly selected within the value range. Each clear image is dynamically generated a corresponding pseudo-haze image through the haze degradation model.

[0015]

[0016] ​In the above-mentioned real haze image dehazing method based on the haze degradation model, in step three, the dehazing network is a three-stage symmetrical encoder-decoder structure main branch network; the input image first extracts shallow features through a 3×3 convolutional layer, and then enters a multi-scale feature extraction process; each stage is composed of stacked MFBlocks of different numbers, and the MFBlocks of different stages have different feature map resolutions and channel numbers to capture multi-level semantic information; in the encoder part, downsampling operations are performed after the first and second stage MFBlocks to gradually compress the spatial size and expand the number of channels; In the decoder section, upsampling operations are performed sequentially after the second and third stages of MFBlock to gradually restore resolution and channel count. A spatial-frequency domain hybrid module is introduced into MFBlock to collaboratively extract local spatial texture and frequency domain global structure information. At the same time, a priori-guided feedforward network is embedded in MFBlock to enhance the quality of texture edge and detail reconstruction by introducing prior feature information. Skip connections are set between each stage to achieve cross-layer feature fusion between the codec and the decoder. The network finally outputs a residual image through a 3×3 convolutional layer. The residual image is added to the input image to obtain the final dehazed image.

[0017] In the above-mentioned method for dehazing real haze images based on a haze degradation model, step three involves a spatial-frequency domain hybrid module introduced in MFBlock, which includes two parallel processing branches: a spatial branch and a frequency domain branch.

[0018] Spatial branching pairs of input features Spatial features are extracted by sequentially performing 3 × 3 convolution, 3 × 3 depthwise separable convolution, and 3 × 3 convolution operations, and the original feature information is preserved through residual connections to obtain the output features of the spatial branch. ;

[0019] Frequency domain branching will input features The frequency domain representation is obtained by converting the data to the frequency domain using a Fast Fourier Transform. The details are as follows:

[0020]

[0021] in, Represents spatial domain coordinates, , These are the horizontal and vertical coordinates of the spatial domain, respectively. Represents frequency domain coordinates, , These are the horizontal and vertical axes in the frequency domain, respectively. It is an imaginary number. and These represent the height and width of the input feature, respectively; by Obtain the real part and imaginary part and the amplitude spectrum is calculated accordingly and the phase spectrum is calculated accordingly , specifically as follows:

[0022]

[0023]

[0024] wherein, is an inverse tangent function;

[0025] Subsequently, the amplitude spectrum is enhanced by 1x1 convolution to obtain the enhanced amplitude spectrum feature ; the phase spectrum is sequentially subjected to feature extraction by 1x1 convolution, SE channel attention mechanism and 1x1 convolution, and the original feature information is retained through residual connection to obtain the enhanced phase spectrum feature ; then, the enhanced amplitude spectrum feature and the phase spectrum feature are converted back to real part and imaginary part , specifically as follows:

[0026]

[0027]

[0028] The feature map is remapped to the spatial domain by inverse Fourier transform, and the original feature information is retained through residual connection to obtain the output feature of the frequency domain branch , specifically as follows:

[0029]

[0030] wherein, represents inverse Fourier transform;

[0031] The output feature of the spatial branch is spliced with the output feature of the frequency domain branch in the channel dimension, and feature fusion is performed by 1x1 convolution to obtain the fused feature , specifically as follows:

[0032]

[0033] wherein, represents channel dimension splicing operation, represents 1x1 convolution; subsequently, the adaptive fusion weight of the spatial branch is generated in the channel dimension using function Adaptive fusion weights for frequency domain branches :

[0034]

[0035] The final output of the spatial-frequency domain hybrid module The weighted fusion result of the two features:

[0036] .

[0037] The above-mentioned dehazing method for real haze images based on a haze degradation model, in step three, the prior-guided feedforward network embedded in MFBlock contains two parallel branches: the first branch processes the input prior features... The process involves sequentially performing 3×3 depthwise convolution, GELU activation, 1×1 convolution, and sigmoid activation to generate a gated mask. This mask is then restored using a 1×1 convolution to recover the channel dimension, yielding the output features. The second branch corresponds to the final output of the space-frequency hybrid module. The enhanced features are obtained by sequentially applying 3×3 depthwise separable convolution, GELU activation, 1×1 convolution, and GELU activation. These enhanced features are then multiplied element-wise with a gating mask, and finally, a 1×1 convolution is applied to obtain the output features. .

[0038] In the above-mentioned method for dehazing real haze images based on a haze degradation model, in step three, the prior features are generated and transmitted as follows: First, a preliminary dehazed image is obtained by using a dark channel prior on the input image. Then, the spatial size and channel dimension of the preliminary dehazed image are adjusted through upsampling and downsampling operations to match the size of each stage of the encoder-decoder. The adjusted preliminary dehazed image is used as a prior feature and input to the prior-guided feedforward network in the first MFBlock of each stage. In the same stage, the prior-guided feedforward network of subsequent MFBlocks receives the output features of the corresponding network in the previous MFBlock as its input prior features. This achieves the step-by-step transmission and fusion of prior information throughout the network, guiding the feature enhancement and detail recovery process.

[0039] The beneficial effects of this invention are as follows: By constructing a haze degradation model that integrates multiple scattering effects and various image degradation factors, this invention effectively alleviates the difference between the synthetic and real domains; the designed spatial-frequency hybrid module enhances the model's adaptability to complex degradation features; and the prior-guided feedforward network fully mines and integrates prior information from the dark channel, effectively enhancing the model's perception and modeling capabilities for haze regions. This invention effectively improves the dehazing capability of real haze images, exhibiting significant advantages in brightness restoration, color fidelity, and detail enhancement. Attached Figure Description

[0040] Figure 1 This is the overall flowchart of the present invention.

[0041] Figure 2 This is a schematic diagram of the pseudo-haze image obtained by the present invention.

[0042] Figure 3 This is a schematic diagram of the defogging network of the present invention.

[0043] Figure 4 This is a schematic diagram of the MFBlock structure of the present invention.

[0044] Figure 5 This is a schematic diagram of the spatial-frequency domain hybrid module structure of the present invention.

[0045] Figure 6 This is a schematic diagram of the priori-guided feedforward network structure of the present invention.

[0046] Figure 7 This is a schematic diagram of the defogging result of the present invention, wherein (a) to (e) are the original fog images, and (f) to (j) are the result images of defogging (a) to (e) using the present invention. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] like Figure 1 As shown, a method for dehazing real haze images based on a haze degradation model includes the following steps:

[0049] Step 1: Construct a haze degradation model that integrates multiple scattering effects and various image degradation factors.

[0050] The constructed haze degradation model is as follows:

[0051]

[0052] in, Represents pixels; This indicates the intensity of the acquired haze image; Indicates the intensity of a true, clear image; Transmittance represents the degree of light attenuation during propagation; Represents global atmospheric light; Denotes the multiple scattering term, where Represented in pixels The centered multiple scattering domain Represents pixels For pixels scattering contribution, Represents a true, clear image in pixels the intensity of the light, denotes the transmittance at the pixel denotes the transmittance at the pixel and denote the scattering coefficient and the scene depth, respectively; denotes a multiple scattering weight coefficient for adjusting the intensity of the multiple scattering effect; denotes a Gaussian noise; denotes a color bias perturbation term; denotes a compression artifact. is preferably a natural number between 0 and 1, is preferably a natural number between 0.3 and 1.5, is preferably a natural number between 0.75 and 0.95, is preferably a natural number between -0.025 and 0.025, is calculated using a Gaussian weight function with a window size of 5.

[0053] Step two: Construct a training dataset containing clear images and corresponding pseudo-haze images using the haze degradation model.

[0054] 500 real clear images are selected, the depth information map of the clear image is estimated using the pre-trained depth estimation network, the value range of each parameter in the haze degradation model is set, and the parameter combination is randomly selected within the value range. Each clear image is dynamically generated a corresponding pseudo-haze image by the haze degradation model, as shown in Figure 2 .

[0055] Step three: Construct a haze removal network for real haze scenes.

[0056] As shown in Figure 3 , the haze removal network is a three-stage symmetric encoder-decoder structure main branch network; the input image is first extracted by a 3x3 convolutional layer to obtain shallow features, and then enters the multi-scale feature extraction process; each stage is composed of different numbers of MFBlock stacks, and the MFBlock in different stages has different feature map resolutions and channel numbers, which are used to capture multi-level semantic information; in the encoder part, down-sampling operations are performed after the MFBlock of the first and second stages, respectively, to gradually compress the spatial size and expand the channel number; in the decoder part, up-sampling operations are performed after the MFBlock of the second and third stages, respectively, to gradually restore the resolution and channel number; a spatial-frequency hybrid module is introduced in the MFBlock, and the structure is as shown in Figure 4As shown, the local spatial texture and the global structure information in the frequency domain are extracted cooperatively; meanwhile, a feedforward network with prior guidance is embedded in the MFBlock to enhance the texture edge and detail reconstruction quality by introducing prior feature information; skip connections are set between stages to realize cross-layer feature fusion between the encoder and the decoder; finally, the network outputs a residual image through a 3 × 3 convolution layer, and the final defogging image is obtained by adding the residual image to the input image.

[0057] As shown in FIG. 1, the MFBlock is composed of three stages: the first stage is a feature extraction stage, the second stage is a feature refinement stage, and the third stage is a feature fusion stage. Figure 5 As shown, the spatial-frequency hybrid module introduced in the MFBlock contains two parallel processing branches: a spatial branch and a frequency domain branch.

[0058] The spatial branch sequentially performs 3 × 3 convolution, 3 × 3 depth separable convolution, and 3 × 3 convolution operations on the input feature to extract spatial features, and retains the original feature information through a residual connection to obtain the output feature of the spatial branch.

[0059] The frequency domain branch converts the input feature to the frequency domain through a fast Fourier transform to obtain the corresponding frequency domain representation , which is as follows:

[0060]

[0061] wherein, denotes a spatial domain coordinate, , are the horizontal and vertical coordinates of the spatial domain, denotes a frequency domain coordinate, , are the horizontal and vertical coordinates of the frequency domain, is an imaginary number, and denote the height and width of the input feature, respectively; the real part and the imaginary part are obtained from , and the amplitude spectrum and the phase spectrum are calculated therefrom, which are as follows:

[0062]

[0063]

[0064] wherein, is an arctangent function;

[0065] Subsequently, the amplitude spectrum is enhanced through a 1 × 1 convolution to obtain the enhanced amplitude spectrum feature ; phase spectrum , sequentially pass through 1x1 convolution, SE channel attention mechanism and 1x1 convolution for feature extraction, and retain the original feature information through residual connection to obtain enhanced phase spectrum features ; then, the enhanced amplitude spectrum features and phase spectrum features are converted back to real part and imaginary part , as follows:

[0066]

[0067]

[0068] The feature map is remapped to the spatial domain by inverse Fourier transform, and the original feature information is retained through residual connection to obtain the output features of the frequency domain branch , as follows:

[0069]

[0070] wherein, represents inverse Fourier transform;

[0071] The output features of the spatial branch are spliced with the output features of the frequency domain branch in the channel dimension, and the features are fused by 1x1 convolution to obtain the fused features , as follows:

[0072]

[0073] wherein, represents channel dimension splicing operation, represents 1x1 convolution; subsequently, the function is used to generate adaptive fusion weights of the spatial branch and adaptive fusion weights of the frequency domain branch in the channel dimension:

[0074]

[0075] The final output of the spatial-frequency hybrid module is the weighted fusion result of the two kinds of features:

[0076] .

[0077] The feedforward network embedded in the MFBlock with prior guidance has the structure as shown in Figure 6 , which contains two parallel branches: the first branch processes the input prior features The 3*3 depth convolution, the GELU activation, the 1*1 convolution and the Sigmoid activation are sequentially performed to generate a gating mask, and then the channel dimension is restored through 1*1 convolution to obtain the output feature ; the second branch is used to process the final output of the spatial-frequency domain hybrid module The 3*3 depth separable convolution, the GELU activation, the 1*1 convolution and the GELU activation are sequentially performed to obtain an enhanced feature, then the enhanced feature is multiplied by the gating mask element by element, and finally the 1*1 convolution is performed to obtain the output feature .

[0078] The prior feature is generated and transmitted in the following manner: first, the dark channel prior is used to obtain a preliminary defogging image from the input image, and then the spatial size and channel dimension of the preliminary defogging image are adjusted through upsampling and downsampling operations to match the size of each stage of the encoder-decoder; the adjusted preliminary defogging image is input into the prior-guided feedforward network in the first MFBlock of each stage as a prior feature; the input prior feature of the prior-guided feedforward network in the subsequent MFBlock in the same stage receives the output feature of the corresponding network in the previous MFBlock, so as to realize the step-by-step transmission and fusion of the prior information in the entire network and guide the feature enhancement and detail recovery process.

[0079] Step four: the training data set is used to train the defogging network until the pre-set loss function converges.

[0080] Step five: the image to be defogged is input into the trained defogging network to obtain the defogging result.

[0081] The method of the present application is used to perform experiments on the RTTS and Fattal public data sets. The RTTS data set contains 4322 real fog images, and the Fattal data set contains 34 real fog images. Figure 7 The image defogging results obtained by using the present application are given, (a)-(e) are original fog images, (f)-(j) are the results of defogging (a)-(e) using the present application, respectively. It can be seen that the present application can effectively remove the fog interference in the image and obtain high-quality images with high contrast and clear details.

Claims

1. A method for dehazing realistic haze images based on a haze degradation model, characterized in that, Includes the following steps: Step 1: Construct a haze degradation model that integrates multiple scattering effects and various image degradation factors; Step 2: Using the haze degradation model, construct a training dataset containing clear images and corresponding pseudo haze images; Step 3: Construct a defogging network for real-world smog scenarios; Step 4: Train the dehazing network using the training dataset until the pre-set loss function converges; Step 5: Input the image to be dehazed into the trained dehazing network to obtain the dehazing result.

2. The method for dehazing real haze images based on a haze degradation model according to claim 1, characterized in that, In step one, the haze degradation model constructed is as follows: ； in, Represents pixels; This indicates the intensity of the acquired haze image; Indicates the intensity of a true, clear image; Transmittance represents the degree of light attenuation during propagation; Represents global atmospheric light; Denotes the multiple scattering term, where Represented in pixels The centered multiple scattering domain Represents pixels For pixels scattering contribution, Represents a true, clear image in pixels The strength, Represents pixels Transmittance at that location and These represent the scattering coefficient and scene depth, respectively. This represents the multiple scattering weighting coefficient, used to adjust the intensity of the multiple scattering effect; Indicates Gaussian noise; This indicates the color deviation disturbance term; This indicates compression artifacts.

3. The method for dehazing real haze images based on a haze degradation model according to claim 2, characterized in that, In step two, 500 real and clear images are selected, and a pre-trained depth estimation network is used to estimate the depth information map of the clear images. The value range of each parameter in the haze degradation model is set, and parameter combinations are randomly selected within the value range. The haze degradation model is used to dynamically generate a corresponding pseudo haze image for each clear image.

4. The method for dehazing real haze images based on a haze degradation model according to claim 2, characterized in that, In step three, the dehazing network is a main branch network with a three-stage symmetric encoder-decoder structure; the input image first extracts shallow features through a 3×3 convolutional layer, and then enters the multi-scale feature extraction process. Each stage consists of stacks of MFBlocks of varying numbers. The MFBlocks in different stages have different feature map resolutions and number of channels, which are used to capture multi-level semantic information. In the encoder section, downsampling operations are performed after the first and second MFBlock stages to gradually compress the spatial size and expand the number of channels. In the decoder section, upsampling operations are performed sequentially after the second and third MFBlock stages to gradually restore the resolution and number of channels. A spatial-frequency domain hybrid module is introduced into the MFBlock to collaboratively extract local spatial texture and frequency domain global structure information. At the same time, a priori-guided feedforward network is embedded in the MFBlock to enhance the quality of texture edge and detail reconstruction by introducing prior feature information. Skip connections are set between each stage to achieve cross-layer feature fusion between the encoder and decoder. The network finally outputs a residual image through a 3×3 convolutional layer. The residual image is added to the input image to obtain the final dehazed image.

5. The method for dehazing real haze images based on a haze degradation model according to claim 4, characterized in that, In step three, the spatial-frequency hybrid module introduced in MFBlock contains two parallel processing branches: a spatial branch and a frequency branch. Spatial branching pairs of input features Spatial features are extracted by sequentially performing 3 × 3 convolution, 3 × 3 depthwise separable convolution, and 3 × 3 convolution operations, and the original feature information is preserved through residual connections to obtain the output features of the spatial branch. ; Frequency domain branching will input features The frequency domain representation is obtained by converting the data to the frequency domain using a Fast Fourier Transform. The details are as follows: ； in, Represents spatial domain coordinates, , These are the horizontal and vertical coordinates of the spatial domain, respectively. Represents frequency domain coordinates, , These are the horizontal and vertical coordinates in the frequency domain, respectively. It is an imaginary number. and These represent the height and width of the input feature, respectively; by Obtain the real part and the virtual part And based on this, the amplitude spectrum is calculated. and phase spectrum The details are as follows: ； ； in, It is the arctangent function; Subsequently, the amplitude spectrum Enhancement is achieved through 1×1 convolution, resulting in the enhanced amplitude spectrum features. ; for phase spectrum Feature extraction is performed sequentially using 1×1 convolution, SE channel attention mechanism, and 1×1 convolution, and the original feature information is preserved through residual connection to obtain the enhanced phase spectrum features. Next, the enhanced amplitude spectrum features and phase spectrum characteristics Convert back to real part and the virtual part The details are as follows: ； ； The feature map is remapped to the spatial domain using inverse Fourier transform, and the original feature information is preserved through residual connection to obtain the output features of the frequency domain branch. The details are as follows: ； in, Indicates the inverse Fourier transform; Output features of spatial branching Output characteristics of frequency domain branch The features are concatenated along the channel dimension and then fused using a 1×1 convolution to obtain the fused features. The details are as follows: ； in, This indicates a channel-level concatenation operation. This represents a 1×1 convolution; subsequently, using Adaptive fusion weights that generate spatial branches in the channel dimension Adaptive fusion weights for frequency domain branch : ； The final output of the spatial-frequency domain hybrid module The weighted fusion result of the two features: 。 6. The method for dehazing real haze images based on a haze degradation model according to claim 5, characterized in that, In step three, the prior-guided feedforward network embedded in MFBlock contains two parallel branches: the first branch considers the prior features of the input. The process involves sequentially performing 3×3 depthwise convolution, GELU activation, 1×1 convolution, and sigmoid activation to generate a gated mask. This mask is then restored using a 1×1 convolution to recover the channel dimension, yielding the output features. The second branch corresponds to the final output of the space-frequency hybrid module. The enhanced features are obtained by sequentially applying 3×3 depthwise separable convolution, GELU activation, 1×1 convolution, and GELU activation. These enhanced features are then multiplied element-wise with a gating mask, and finally, a 1×1 convolution is applied to obtain the output features. .

7. The method for dehazing real haze images based on a haze degradation model according to claim 6, characterized in that, In step three, the prior features are generated and transmitted as follows: First, a preliminary dehazed image is obtained by using a dark channel prior on the input image. Then, the spatial size and channel dimension of the preliminary dehazed image are adjusted by upsampling and downsampling operations to match the size of each stage of the encoder-decoder. The adjusted preliminary dehazed image is used as a prior feature and input to the prior-guided feedforward network in the first MFBlock of each stage. In the same stage, the prior-guided feedforward network of the subsequent MFBlock receives the output features of the corresponding network in the previous MFBlock as its input prior features. This realizes the step-by-step transmission and fusion of prior information in the entire network, guiding the feature enhancement and detail recovery process.

Images