A haze image restoration method based on deep learning and gamma correction

Abstract

The present invention discloses a haze image restoration method based on deep learning and gamma correction, comprising a transformer-based image restoration architecture Uformer as a generator of a generative adversarial network; inputting a hazy image into the generator of the adversarial network to generate a dehazed image; a discriminator comparing the dehazed image generated by the generator with a real image without haze, and optimizing the parameters of the generative adversarial network model by minimizing the Charbonnier loss function; repeating the above operation until the number of iterations set by training, and adding a gamma correction function in the forward propagation inference process of the trained generator to perform a power transformation on the image data to adjust its nonlinear response so as to achieve more accurate light intensity representation; splitting the gamma-corrected image into three RGB channels, multiplying each channel by a corresponding weight, merging the channels and performing standardization processing to obtain a final restoration result, thereby making the enhanced image have a better perceptual range and application value.

Description

Technical Field

[0001] The present invention relates to the technical field of computer digital image processing, and in particular to a haze image restoration method based on deep learning and gamma correction. Background Art

[0002] With the advancement of current computer technology and the continuous improvement of hardware performance, image processing technology plays an increasingly important role in today's social development. New technologies based on computer vision are increasingly being applied in fields such as intelligent security, industrial automation, satellite imaging, and navigation and guidance. Over the past few decades, severe weather conditions such as smog and dust storms have become frequent worldwide. The atmospheric scattering caused by smog and other weather conditions significantly degrades the imaging quality of optical equipment, resulting in reduced image contrast and chromatic aberration. This significantly limits the application scenarios of many optical devices and significantly reduces the success rate of target recognition and detection by computer vision systems.

[0003] Therefore, effective image dehazing and enhancement methods are of great value in improving the imaging quality of optical systems and improving the image perception level. They are an important basis for subsequent image segmentation, image classification, target detection and other related work.

[0004] In previous studies, many methods restored degraded images based on traditional prior information or constructed atmospheric physics models, and restored images by analyzing information such as brightness and color distribution. Such methods often have poor robustness and limited dehazing effects on complex scenes such as dense fog environments. The restored images may have problems such as color aberration and noise, and cannot restore the input degraded images with high quality.

[0005] To address the above problems, the present invention proposes an image defogging method that combines a deep learning uFormer network structure with gamma correction to improve image quality. Summary of the Invention

[0006] The purpose of this invention is to provide a haze image restoration method based on deep learning and gamma correction, which can effectively solve the problem.

[0007] To solve the above technical problems, the present invention adopts the following technical solution: a haze image restoration method based on deep learning and gamma correction, characterized by comprising:

[0008] The transformer-based image restoration architecture Uformer is used as the generator of the generative adversarial network;

[0009] Input the foggy image into the generator of the adversarial network to generate the defogged image;

[0010] The discriminator compares the dehazed image generated by the generator with the real image without haze, and optimizes the parameters of the generative adversarial network model by minimizing the Charbonnier loss function;

[0011] Repeat the above steps for the number of iterations set by the training, and

[0012] During the forward propagation inference process of the trained generator, a gamma correction function is added to correct the input dehazed image and output a preliminary restored image;

[0013] The gamma-corrected image is split into three RGB channels, each channel is multiplied by the corresponding weight, and the final restoration result is obtained after merging the channels and standardizing them.

[0014] Preferably, the dark channel prior principle and atmospheric light estimation are used in combination with transmittance estimation to generate a foggy image. The atmospheric scattering model formula is:

[0015] I = J·t + A(Et);

[0016] Where I represents the synthetic haze image, J represents the clear image, t is the atmospheric transmittance, A is the atmospheric light value, and E is the unit matrix.

[0017] Preferably, the generator generates the defogging image as follows:

[0018] The input foggy image first passes through a convolution layer, extracts the underlying features through a 3×3 convolution kernel, and is activated by LeakyReLU to output a feature map.

[0019] Use the encoder output as the decoder input;

[0020] The encoder output feature map and the decoder feature map at the corresponding resolution are spliced ​​at the channel position through jump connections to achieve multi-scale feature information fusion.

[0021] Preferably, the encoder uses a multi-head self-attention module with multiple local enhancement windows, which includes a non-overlapping window multi-head attention mechanism and a local enhancement feedforward network, and is combined with a downsampling module to capture global dependencies; and

[0022] The decoder also contains multiple self-attention modules with local enhancement windows to upsample the input feature information and increase the resolution of the feature map.

[0023] Preferably, the encoder specifically operates as follows:

[0024] Step 1: Window Multi-Head Attention Mechanism (W-MSA) for a given input sequence Divide X into i windows with no overlap between them. For each window, W-MSA applies a multi-head self-attention module to calculate the attention weights between elements in the window and generate representations of the elements in the window. The self-attention of the K-th head can be expressed as:

[0025]

[0026] in, The mapping matrices representing the query, key, and value of the K-th head respectively.

[0027] Step 2: Calculate the attention weight based on the known Q, K, and V. The formula can be expressed as:

[0028]

[0029] Wherein, B is the relative position deviation, and its value is obtained through learning.

[0030] Step 3: Calculate the multi-head self-attention module of the local enhancement window. The calculation formula is expressed as:

[0031] X′ m =W-MSA(LN(X m-1 ))+X m-1 ;

[0032] X m =LeFF(LN(X m ′))+X m ';

[0033] Among them, X′ m and X m They are the outputs of W-MSA and LeFF respectively, and LN represents layer normalization.

[0034] Preferably, downsampling uses a 4×4 convolution kernel with a stride of 2 to reduce the image resolution to 1 / 4 and double the number of channels. Upsampling uses a 2×2 convolution kernel with a stride of 2 transposed convolution. After processing, the image resolution is quadrupled and the number of channels is halved.

[0035] Preferably, a skip connection is introduced, that is, in the decoder, the output of the encoder and the residual term are added as the input of the decoder, thereby combining low-level and high-level feature information, capturing features at different levels, and improving the performance and convergence speed of the model during training.

[0036] Preferably, the Charbonnier loss function formula is:

[0037]

[0038] in is a clear image, ∈ is an empirical constant.

[0039] Preferably, the correction of the input defogging image is specifically performed as follows:

[0040] Step 1: Normalize the pixel values ​​of the dehazed image and map them to between 0 and 1;

[0041] Step 2: Calculate the normalized data using 1 / gamma as the exponent.

[0042] Step 3: Inversely transform the pre-compensated real value into an integer value between 0 and 255.

[0043] Preferably, the standardization process is:

[0044]

[0045] Where X is the gamma-corrected image, μ is the mean of the image, σ is the standard deviation of the image, and Z is the normalized image.

[0046] Beneficial effects: The generator of the generative adversarial network is modified using the transformer-based image restoration architecture Uformer. The self-attention mechanism is used to effectively capture the global and local features in the image, and can effectively achieve image dehazing and produce clear restored images.

[0047] Furthermore, by adding a gamma correction function and applying a power transformation to the image data to adjust its nonlinear response, we achieve more accurate light intensity representation. Combined with RGB channel weighting and data normalization, the enhanced image has a greater perceptual range and application value. This method is crucial for maintaining the accuracy and visual comfort of the dehazed image information and is a prerequisite for subsequent visual system processing and acquisition of information. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0049] In the attached figure:

[0050] Figure 1 This is a flow chart of the haze image restoration method based on deep learning and gamma correction of the present invention;

[0051] Figure 2 is the degraded image with fog according to the present invention;

[0052] Figure 3 A clear image restored by the deep learning method of the present invention;

[0053] Figure 4 This is a clear image restored by the deep learning method of the present invention with gamma correction. DETAILED DESCRIPTION

[0054] To make the objects and advantages of the present invention more clearly understood, the present invention is described in detail below with reference to the following embodiments. It should be understood that the following text is merely used to describe a haze image restoration method based on deep learning and gamma correction or several specific implementations of the present invention, and does not strictly limit the scope of protection specifically claimed in the present invention.

[0055] Example: Figure 1 A haze image restoration method based on deep learning and gamma correction is shown, comprising the following steps:

[0056] Step 1: Create a synthetic foggy dataset based on the atmospheric scattering model;

[0057] Among them, an image defogging dataset is established, and the dark channel prior principle and atmospheric light estimation are used in combination with transmittance estimation to generate foggy images. The atmospheric scattering model formula is:

[0058] I = J·t + A(Et);

[0059] Where I represents the synthetic haze image, J represents the clear image, t is the atmospheric transmittance, A is the atmospheric light value, and E is the unit matrix.

[0060] Step 2: Initialize the model parameters, pass the foggy image through the first convolutional layer to extract the underlying features, and input it into the encoder module;

[0061] Among them, the underlying features are extracted through the 3×3 convolution kernel, and the feature map is obtained after activation by LeakyReLU;

[0062] Step 3: The encoder uses a self-attention module with multiple local enhancement windows, which includes a non-overlapping window multi-head attention module (W-MSA) and a local enhancement feedforward network (LeFF), and is combined with a downsampling module to reduce computational complexity and capture global dependencies. Downsampling reduces the image resolution to 1 / 4 and doubles the number of channels through a 4×4 convolution kernel with a stride of 2. The main operations are:

[0063] (1) W-MSA for a given input sequence Divide X into i windows with no overlap between them. For each window, W-MSA applies a multi-head self-attention mechanism to calculate the attention weights between elements in the window and generate representations of the elements in the window. The self-attention of the k-th head can be expressed as:

[0064]

[0065] in The mapping matrices representing the query, key, and value of the K-th head respectively;

[0066] (2) Calculate the attention weight based on the known Q, K, and V. The formula can be expressed as:

[0067]

[0068] Where B is the relative position deviation, and its value is obtained through learning;

[0069] (3) Calculate the multi-head self-attention module of the local enhancement window. The calculation formula is expressed as:

[0070] X′ m =W-MSA(LN(X m-1 ))+X m-1 ;

[0071] X m =LeFF(LN(X m ′))+X m ';

[0072] where X′ m and Xm are the outputs of W-MSA and LeFF respectively, and LN represents layer normalization;

[0073] Step 4: The output of the encoder is used as the input of the decoder. The decoder also contains a self-attention module with multiple local enhancement windows to upsample the input feature information and increase the resolution of the feature map. The upsampling uses a 2×2 convolution kernel with a stride of 2 and a transposed convolution. After processing, the image resolution is quadrupled and the number of channels is halved.

[0074] Step 5: The encoder output feature map and the decoder feature map at the corresponding resolution are spliced ​​at the channel position through jump connections to achieve multi-scale feature information fusion, thereby further improving the image dehazing effect; in the decoder, the encoder output and the residual term are added as the decoder input to capture feature information at different levels, thereby improving the performance and convergence speed of the model during training.

[0075] Step 6: The discriminator compares the dehazed image generated by the generator with the real image without fog. That is, the discriminator extracts features through 4×4 convolution kernel and Leaky ReLU layer downsampling, determines the difference between the generated dehazed image and the actual clear image, and optimizes the model parameters by minimizing the Charbonnier loss function. Charbonnier_loss is a loss function similar to L1 loss for image generation tasks. Its formula is:

[0076]

[0077] Among them, I′ is the model generated image, is a clear image, ∈ is an empirical constant;

[0078] Step 7: Repeat steps 2 to 6 until the number of iterations set for model training is reached;

[0079] Step 8: Add gamma correction to the forward propagation inference process of the trained dehazing model, edit the gamma curve of the image, and finally output the restored image; specifically:

[0080] (1) Normalize the pixel values ​​of the dehazed image and map them to between 0 and 1;

[0081] (2) The normalized data is calculated using 1 / gamma as the exponent;

[0082] (3) Inversely transforming the pre-compensated real value into an integer value between 0 and 255;

[0083] Step 9: Split the gamma-corrected image into three channels, RGB, and multiply each channel by the corresponding weight, that is, multiply by the corresponding weights WR, WG, and WB, to obtain a new channel value. After merging the channels and standardizing, the final restoration result is obtained; the formula for standardization is:

[0084]

[0085] Where X is the gamma-corrected image, μ is the mean of the image, σ is the standard deviation of the image, and Z is the normalized image.

[0086] Based on the above operations, the results are as follows Figure 2-Figure 4 As shown, Figure 2 and Figure 3 In comparison, the deep learning self-attention mechanism effectively captures the global and local features in the image. This method can effectively achieve image dehazing and produce clear restored images. It can be seen that the visibility of the image is significantly improved after dehazing, and the image details are effectively restored. Figure 3 and Figure 4 In comparison, a method of adjusting the nonlinear response of image data by power transformation is used to achieve more accurate light intensity representation, and RGB channel weight processing and data normalization operations are combined to make the enhanced image have a better perceptual range and application value.

[0087] The above describes the embodiments of the present invention in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. After knowing the contents described in the present invention, ordinary technicians in this technical field can make several equivalent changes and substitutions without departing from the principles of the present invention. These equivalent changes and substitutions should also be regarded as falling within the scope of protection of the present invention.

Claims

1. A haze image restoration method based on deep learning and gamma correction, characterized in that: include: The transformer-based image restoration architecture Uformer is used as the generator of the generative adversarial network; Input the foggy image into the generator of the adversarial network to generate the defogged image; The discriminator compares the dehazed image generated by the generator with the real image without haze, and optimizes the parameters of the generative adversarial network model by minimizing the Charbonnier loss function; Repeat the above steps for the number of iterations set by the training, and During the forward propagation inference process of the trained generator, a gamma correction function is added to correct the input dehazed image and output a preliminary restored image; The gamma-corrected image is split into three RGB channels, each channel is multiplied by the corresponding weight, and the final restoration result is obtained after merging the channels and standardizing them.

2. The haze image restoration method based on deep learning and gamma correction according to claim 1, characterized in that: Using the dark channel prior principle and atmospheric light estimation, combined with transmittance estimation, a foggy image is generated. The atmospheric scattering model formula is: I = J·t + A(Et); Where I represents the synthetic haze image, J represents the clear image, t is the atmospheric transmittance, A is the atmospheric light value, and E is the unit matrix.

3. The haze image restoration method based on deep learning and gamma correction according to claim 1, characterized in that: The generator generates dehazed images as follows: The input foggy image first passes through a convolution layer, extracts the underlying features through a 3×3 convolution kernel, and is activated by LeakyReLU to output a feature map. Use the encoder output as the decoder input; The encoder output feature map and the decoder feature map at the corresponding resolution are spliced ​​at the channel position through jump connections to achieve multi-scale feature information fusion.

4. The haze image restoration method based on deep learning and gamma correction according to claim 3, characterized in that: The encoder uses a multi-head self-attention module with multiple local enhancement windows. The multi-head self-attention module includes a non-overlapping window multi-head attention mechanism and a local enhancement feedforward network, and is combined with a downsampling module to capture global dependencies; and The decoder also contains multiple self-attention modules with local enhancement windows to upsample the input feature information and increase the resolution of the feature map.

5. The haze image restoration method based on deep learning and gamma correction according to claim 4, characterized in that: The specific operations of the encoder are: Step 1: W-MSA is used to calculate the input sequence X∈R with a given length H, width W and number of channels C. C×H×W , split X into F P×P windows with no overlap between windows; for each window, W-MSA applies a multi-head self-attention module to calculate the attention weights between elements in the window and generate representations of the elements in the window; the self-attention of the k-th head can be expressed as: in, The mapping matrices representing the query, key, and value of the k-th head respectively; Step 2: Calculate the attention weight based on the known Q, K, and V. The formula can be expressed as: Among them, the dimension of the attention head is d k =C / k, B is the relative position deviation, its value is obtained through learning; Step 3: Calculate the multi-head self-attention module of the local enhancement window. The calculation formula is expressed as: X′ m =W-MSA(LN(X m-1 ))+X m-1 ; X m =LeFF(LN(X m ′))+X m ′; Among them, X′ m and X m They are the outputs of W-MSA and LeFF respectively, and LN represents layer normalization.

6. The haze image restoration method based on deep learning and gamma correction according to claim 4, characterized in that: Downsampling uses a 4×4 convolution kernel with a stride of 2 to reduce the image resolution to 1 / 4 and double the number of channels. Upsampling uses a 2×2 convolution kernel with a stride of 2 transposed convolution. After processing, the image resolution is quadrupled and the number of channels is halved.

7. The haze image restoration method based on deep learning and gamma correction according to claim 3, characterized in that: The skip connection is introduced, that is, in the decoder, the output of the encoder and the residual term are added as the input of the decoder, thereby combining low-level and high-level feature information, capturing features at different levels, and improving the performance and convergence speed of the model during training.

8. The haze image restoration method based on deep learning and gamma correction according to claim 1, characterized in that: The Charbonnier loss function formula is: Where I′ is the model generated image, is a clear image, and ε is an empirical constant.

9. The haze image restoration method based on deep learning and gamma correction according to claim 1, characterized in that: Correction of the input defogging image is specifically as follows: Step 1: Normalize the pixel values ​​of the dehazed image and map them to between 0 and 1; Step 2: Calculate the normalized data using 1 / gamma as the exponent; Step 3: Inversely transform the pre-compensated real value into an integer value between 0 and 255.

10. The haze image restoration method based on deep learning and gamma correction according to claim 1, characterized in that: The normalized processing is: Where X is the gamma-corrected image, μ is the mean of the image, σ is the standard deviation of the image, and Z is the normalized image.

Images