Two-stage image defogging method combining atmospheric model and fusion strategy

By combining the two-stage image defogging method of atmospheric model and fusion strategy, the atmospheric light value and transmission map are estimated using quadtree subdivision search and linear transformation model, combined with Gamma correction and Gaussian differential pyramid fusion technology, the existing technology is solved The problem of missing detailed information of the image defog removal algorithm and halo artifacts achieves a more efficient image defog removal effect.

CN119006335APending Publication Date: 2024-11-22CHANGCHUN UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411072933.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing image defog removal algorithm can easily lead to missing details, halo artifacts and sky area distortion when processing foggy environment images. Deep learning-based methods are prone to overfitting, resulting in the defog removal effect of real natural scene images. good.

Method used

The two-stage image defogging method combined with atmospheric model and fusion strategy is adopted to estimate the atmospheric light value through the quad-tree subdivision search algorithm, and the transmission map is obtained using linear transformation model and adaptive control parameters to recover the initial foggy-free image in reverse solution. , and reconstruct the foggy-free image by Gamma-corrected and non-downsampled Gaussian differential pyramid fusion image.

Benefits of technology

It effectively solves the problems of sky area distortion and halo artifacts, retains gradient information such as edges and contours, improves the accuracy and efficiency of image fog removal, and reduces information loss.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006335A_ABST
    Figure CN119006335A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a two-stage image defogging method combining an atmospheric model and a fusion strategy, which is mainly composed of an image preliminary defogging module, an exposure module and an image fusion module. The method comprises the following steps: firstly, applying a quadtree-based subdivision search estimation algorithm and a linear model to a foggy image to obtain an atmospheric light value and a transmission image, and then, carrying out reverse solution by utilizing an atmospheric scattering model to obtain an initial fogless image; thirdly, performing different Gamma value correction on the initial fog-free image to generate a series of images with different exposure degrees; and finally, performing multi-scale fusion by adopting a non-subsampled Gaussian difference pyramid, and recovering a fog-free clear image. The method is superior to an existing typical defogging method in subjective and objective evaluation aspects, the defogging effect is remarkable, and the problems of detail information loss, halo artifacts, sky region distortion and the like in the image can be effectively solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and specifically is a two-stage image defogging method combining an atmospheric model with a fusion strategy. Background Art

[0002] Images taken in foggy environments usually suffer from quality degradation, which affects the execution of high-level tasks of computer vision systems, such as image segmentation, target detection, etc. Image defogging algorithms can eliminate or reduce the impact of fog, restore the original clear image, and improve the visibility and quality of images through methods such as model building, image analysis, and optimization solutions.

[0003] At present, image dehazing algorithms are mainly divided into three categories: based on traditional image enhancement, based on physical models, and based on deep learning. Traditional image enhancement dehazing algorithms mainly enhance images globally and locally, highlight details, and improve contrast and brightness to improve image quality and visual experience. They mainly include Retinex algorithm, histogram equalization algorithm, partial differential equation algorithm, wavelet transform algorithm, etc. However, this type of algorithm does not consider the cause of image degradation, but simply enhances contrast to improve visual effects, which is prone to over-enhancement and loss of significant information in the original image.

[0004] The defogging algorithm based on physical models mainly restores a clear image by integrating the prior relationship between the fog-free image and certain parameters in the atmospheric scattering model, and performing inversion operations according to the formation process of the foggy image. It mainly includes algorithms based on dark channel prior and color attenuation prior. This type of algorithm has low complexity and good defogging effect, but if there are scenes similar to atmospheric light such as white walls and the sky in the image, there will be problems such as residual fog, excessive contrast, and color distortion.

[0005] The deep learning-based method is to construct an end-to-end supervised learning network based on a convolutional neural network, and then obtain a fog-free and clear image based on the atmospheric scattering model or directly process the foggy image. This type of method has a good defogging effect and a strong ability to restore details. However, since the data sets of fog-free and foggy images are artificially synthesized later, overfitting is prone to occur, resulting in the defogging effect of real natural scene images being not as good as expected.

[0006] In recent years, with the development of image fusion technology, defogging algorithms based on artificial exposure image fusion have attracted widespread attention from scholars. This type of algorithm first performs Gamma correction on the foggy image to generate a series of images with different exposures, and then uses a fusion algorithm to integrate these images into one image to enhance the image information. However, the Gamma value used by this type of algorithm for foggy images of different concentrations is fixed, resulting in unstable defogging effects for different images. Therefore, this field urgently needs an image defogging method that can overcome the above defects. Summary of the invention

[0007] 1. Technical issues to be solved

[0008] The presence of fog causes the light intensity received by the imaging system to be attenuated, resulting in degradation of the acquired image quality; and the current dehazing algorithms mainly focus on improving the global contrast or optimizing and estimating through supervised learning, which leads to problems such as missing detail information, halo artifacts and distortion of the sky area in the image.

[0009] (II) Technical solution

[0010] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0011] A two-stage image defogging method combining an atmospheric model with a fusion strategy comprises: an image preliminary defogging module, an exposure module, and an image fusion module. The method comprises the following steps: step 1: estimating the atmospheric light value of a foggy image by using a quadtree subdivision search algorithm; step 2: obtaining a transmission map by using a linear transformation model and adaptive control parameters; step 3: restoring an initial fog-free image by inverse solution using an atmospheric scattering model; step 4: obtaining a multi-exposure image by using Gamma correction; step 5: fusing images by using a non-subsampled Gaussian difference pyramid, and reconstructing a fog-free image by using an inverse transformation;

[0012] The step 1 includes a search estimation method based on quadtree subdivision, and the specific process is as follows: the input image I is evenly divided into four regions of the same size, denoted as R i , i∈{1,2,3,4}; calculate R i The mean and variance of pixels in the region: M i =mean(R i ), S i =Var(R i );Use mean M i Subtract the variance S i , as the score of the area, recorded as Score i , Score i =M i -S i ; Get Score i The maximum value in , and its corresponding area is assigned to Set the threshold τ. If HScore is greater than the threshold τ, Assign to I, repeat the above steps, if it is less than the threshold τ, then is the final area; The average of the top 10% brightest pixels As the atmospheric light value A;

[0013] The step 2 includes a method for estimating the transmittance of the atmospheric medium in foggy weather. According to the linear transformation principle and the concept that the segmented area of ​​the quadratic function intersecting the oblique line is approximately represented as a straight line, the transmittance calculation formula can be obtained: Using Gaussian filter to filter t(x), we have Where G(x) is a Gaussian filter, and Where μ is the mean of x and σ is the variance of x;

[0014] In step 3, the atmospheric light value A and the transmittance t are known, and the formula I(x)=J(x)t(x)+A is used to obtain the preliminary defogging image J(x); in step 4, four different gamma values ​​are used to artificially expose the initial defogging image to generate four images with different exposures; in step 5, the exposure image is decomposed by non-sampled Gaussian pyramid and non-downsampled Gaussian difference pyramid, which is expressed as and r is the exposure image index, l is the decomposition scale index, 0≤l≤N, N is the maximum decomposition scale number, and the image decomposed by the non-subsampled Gaussian difference pyramid is weighted fused, with the weight value W i r The calculation formula is: Then the fused image S at each scale i for: Finally, the fused image F is reconstructed as:

[0015] Furthermore, the atmospheric light value obtained by the method based on quadtree subdivision search estimation selects the pixel value of the sky position.

[0016] Furthermore, the parameters in the transmittance are: A ∞ is the atmospheric light value at infinity, and the λ value is adjusted adaptively according to the brightness value, λ(x)=1-I L (x), 0≤λ≤1 The control factor λ is used to constrain the speed of the linear change of the above formula, I L (x) is the brightness component of the foggy image, I Lmin and I Lmax I L(x) The minimum and maximum values ​​of .

[0017] Furthermore, the non-sampled Gaussian decomposition formula is: G 0 (i, j) = I (i, j), where 0 ≤ l ≤ N is the scale index, N is the highest decomposition level, G l represents the NSGP image of the lth layer, G l-1 represents the l-1th layer NSGP image, g(m,n) is the separable Gaussian kernel weight function, G 0represents the 0th layer NSGP image, that is, the original input image. The decomposition process is as follows: first, the original image is used as the bottom layer image G 0 (NSGP layer 0), and convolve with the Gaussian kernel weight function ω (size 5×5) to obtain the image G 1 (NSGP layer 1), iterates N times to form an N+1 layer pyramidal image data structure, namely, the non-sampled Gaussian pyramid image, represented by G 0 ,G 1 ,...G N .

[0018] Furthermore, the non-subsampled Gaussian difference pyramid process can be expressed as: LP k (i,j)=G k+1 (i,j)-G k (i,j), LP N (i,j)=G N (i, j), where 0≤k≤N-1 is the scale index, N is the highest decomposition level, LP k represents the k-th layer NSDGP image, LP N represents the Nth layer NSDGP image; the decomposition process: using the 1st layer NSGP image and the 0th layer NSGP image to perform a differential operation to obtain the image LP 0 (NSDGP image layer 0), iterate N-1 times to generate N layers of non-subsampled Gaussian difference pyramid image, denoted as LP 0 ,LP 1 ,...,LP N-1 , L.P. N It is called the base image.

[0019] Furthermore, the gamma correction is defined as follows: I(x)→α·I γ (x), where α and γ are real numbers. When γ < 1, the visual effect of fog is enhanced; when γ > 1, part of the fog in the image can be eliminated. Therefore, images with different exposures can be generated by selecting different values ​​of γ > 1.

[0020] (III) Beneficial effects

[0021] Compared with the prior art, the present invention provides a two-stage image defogging method combining an atmospheric model with a fusion strategy, which has the following beneficial effects:

[0022] The present invention obtains a more accurate atmospheric light value A and a linear transformation model and a transmission map by adaptively controlling parameters based on a quadtree subdivision search estimation method, thereby solving the problems of sky area distortion and halo artifacts in the preliminary defogging image.

[0023] The present invention obtains multi-exposure images through Gamma exposure and decomposes the multi-exposure images using a non-subsampled Gaussian difference pyramid, thereby better retaining gradient information such as edges and contours, reducing information loss in image defogging, and facilitating improving the accuracy and efficiency of image defogging. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flow chart of a two-stage image defogging method combining an atmospheric model and a fusion strategy according to the present invention;

[0025] Figure 2 It is a non-subsampled Gaussian difference pyramid decomposition diagram of the present invention;

[0026] Figure 3 The non-subsampled Gaussian difference pyramid decomposition example diagram of the present invention, wherein Figures (a), (b), and (c) are the original image, the first layer detail image, and the second layer base image respectively;

[0027] Figure 4 The exposure images obtained by processing the same image with different exposure values ​​in the present invention, where (a) is the original image with fog; (b)-(h) are the images generated when α=1 and γ is 0.1, 0.5, 0.8, 1.1, 1.5, 1.7, and 2 respectively;

[0028] Figure 5 This is a comparison diagram of atmospheric illumination estimation of the present invention, wherein (a) is the original image, (b) and (c) are atmospheric illumination values ​​determined by the traditional method and the method of the present invention, respectively;

[0029] Figure 6 The defogging effects of different λ values ​​of the present invention are shown in Figure 1, where (a) is the original image with fog; (b) to (f) are images generated when λ is 0.1, 0.4, 0.6, 0.9 and adaptive λ respectively;

[0030] Figure 7 This is a diagram of the defogging results of the two-stage image defogging method that combines the atmospheric model with the fusion strategy of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Example

[0033] like Figure 1As shown, an embodiment of the present invention proposes a two-stage image defogging method combining an atmospheric model with a fusion strategy, including: an image preliminary defogging module, an exposure module, and an image fusion module. The method includes the following steps:

[0034] Step 1: Use a quadtree subdivision search algorithm to estimate the atmospheric light value of the fog image. Step 1 includes a quadtree subdivision search estimation method. The specific process is as follows: divide the input image I into four regions of equal size, denoted as R i , i∈{1,2,3,4}; calculate R i The mean and variance of pixels in the region: M i =mean(R i ), S i =Var(R i );Use mean M i Subtract the variance S i , as the score of the area, recorded as Score i , Score i =M i -S i ; Get Score i The maximum value in , and its corresponding area is assigned to Set the threshold τ. If HScore is greater than the threshold τ, Assign to I, repeat the above steps, if it is less than the threshold τ, then is the final area; The average of the top 10% brightest pixels As the atmospheric light value A;

[0035] Step 2: Obtain the transmission map using the linear transformation model and adaptive control parameters, including a method for estimating the transmittance of the atmospheric medium on foggy days. Based on the linear transformation principle and the concept that the segmented area of ​​the quadratic function intersecting the oblique line is approximately represented as a straight line, the transmittance calculation formula can be obtained: Using Gaussian filter to filter t(x), we have Where G(x) is a Gaussian filter, and Where μ is the mean of x, σ is the variance of x, and the parameters of the transmittance are: A ∞ is the atmospheric light value at infinity, λ(x)=1-I L (x), I L (x) is the brightness component of the foggy image, I Lmin and I Lmax I L(x) The minimum and maximum values ​​of ;

[0036] Step 3: Use the atmospheric scattering model to inversely solve and restore the initial fog-free image. Given the atmospheric light value A and transmittance t, the formula I(x) = J(x)t(x) + A can be used to obtain the initial defogging image J(x);

[0037] Step 4: Use Gamma correction to obtain multi-exposure images. Use 4 different Gamma values ​​to artificially expose the initial fog-free image to generate 4 images with different exposures.

[0038] Step 5: Use non-subsampled Gaussian difference pyramid to fuse the images and reconstruct the fog-free image using inverse transformation. The specific steps are: decompose the exposure image by non-subsampled Gaussian pyramid and non-subsampled Gaussian difference pyramid, expressed as and r is the exposure image index, l is the decomposition scale index, 0≤l≤N, N is the maximum decomposition scale number, and the image decomposed by the non-subsampled Gaussian difference pyramid is weighted fused, with the weight value W i r The calculation formula is: Then the fused image S at each scale i for: Finally, the fused image F is reconstructed as:

[0039] like Figure 2 and 3 As shown, the non-subsampled Gaussian difference pyramid decomposition image and specific example diagrams include: The non-subsampled Gaussian decomposition formula is G 0 (i, j) = I (i, j), where 0 ≤ l ≤ N is the scale index, N is the highest decomposition level, G l represents the NSGP image of the lth layer, G l-1 represents the l-1th layer NSGP image, g(m,n) is the separable Gaussian kernel weight function, G 0 represents the 0th layer NSGP image, that is, the original input image. The decomposition process is as follows: first, the original image is used as the bottom layer image G 0 (NSGP layer 0), and convolve with the Gaussian kernel weight function ω (size 5×5) to obtain the image G 1 (NSGP layer 1), iterates N times to form an N+1 layer pyramidal image data structure, namely, the non-sampled Gaussian pyramid image, represented by G 0 ,G 1 ,...G N ; The non-subsampled Gaussian difference pyramid process can be expressed as: LP k (i,j)=G k+1 (i,j)-G k (i,j), LPN (i,j)=G N (i, j), where 0≤k≤N-1 is the scale index, N is the highest decomposition level, LP k represents the k-th layer NSDGP image, LP N represents the Nth layer NSDGP image; the decomposition process: using the 1st layer NSGP image and the 0th layer NSGP image to perform a differential operation to obtain the image LP 0 (NSDGP image layer 0), iterate N-1 times to generate N layers of non-subsampled Gaussian difference pyramid image, denoted as LP 0 ,LP 1 ,...,LP N-1 , L.P. N It is called the base image.

[0040] like Figure 4 As shown in the figure, the exposure graph obtained by processing the same image with different exposure values, the gamma correction is defined as follows: I(x)→α·I γ (x), where α and γ are real numbers. When γ < 1, the visual effect of fog is enhanced; when γ > 1, part of the fog in the image can be eliminated. Therefore, images with different exposures can be generated by selecting different values ​​of γ > 1.

[0041] like Figure 5 As shown in FIG. 1 , it is a comparison diagram of atmospheric illumination estimation. Since the brightest pixel in the original image appears in the car light part in the foreground, the atmospheric illumination value selected by the traditional method is the pixel value of the car light part, while the atmospheric illumination value estimated by the quadtree subdivision method used in this method selects the sky position, which is more accurate.

[0042] like Figure 6 As shown in the figure, the defogging results with different λ values ​​are different. Different λ values ​​have obvious differences in the image restoration effects in different areas of the image. Therefore, λ takes different values ​​in different areas to more effectively restore the image, and it should not be a fixed value. At the same time, the brightness value of the fog image can well reflect the fog concentration. The fog in the distant area of ​​the image is thicker, and the fog in the background image is thinner. Therefore, the λ value can be adaptively adjusted according to the brightness value, λ(x) = 1-I L (x), 0≤λ≤1, and the control factor λ is used to constrain the speed of the linear change in the above formula.

[0043] like Figure 7 As shown, it is the defogging result diagram of the present invention, wherein Figure (a) is a real foggy image, an indoor artificially synthesized foggy image and an outdoor artificially synthesized foggy image from left to right, and Figure (b) is the corresponding fog image defogging result diagram from left to right.

[0044] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A two-stage image dehazing method combining an atmospheric model with a fusion strategy, characterized in that: The method is mainly composed of an image preliminary defogging module, an exposure module, and an image fusion module. The method includes the following steps: Step 1: Determine the atmospheric light value of the fog image using a quadtree subdivision search estimation algorithm; Step 2: Obtain a transmission map using a linear transformation model and adaptive control parameters; Step 3: Restore the initial fog-free image using an inverse solution of the atmospheric scattering model; Step 4: Obtain a multi-exposure image using Gamma correction; Step 5: Use a non-subsampled Gaussian difference pyramid to fuse the images, and reconstruct the fog-free image using an inverse transformation; The step 1 includes a search estimation method based on quadtree subdivision, and the specific process is as follows: the input image I is evenly divided into four regions of the same size, denoted as R i , i∈{1,2,3,4}; calculate R i The mean and variance of pixels in the region: M i =mean(R i ), S i =Var(R i );Use mean M i Subtract the variance S i , as the score of the area, recorded as Score i , Score i =M i -S i ; Get Score i The maximum value in , and its corresponding area is assigned to HScore=max(Score i ); Set the threshold τ, if HScore is greater than the threshold τ, Assign to I, repeat step1-step5, if it is less than the threshold τ, then is the final area; The average of the top 10% brightest pixels As the atmospheric light value A; The step 2 includes a method for estimating the transmittance of the atmospheric medium in foggy weather. According to the linear transformation principle and the concept that the segmented area of ​​the quadratic function intersecting the oblique line is approximately represented as a straight line, the transmittance calculation formula can be obtained: Using Gaussian filter to filter t(x), we have Where G(x) is a Gaussian filter, and Where μ is the mean of x and σ is the variance of x; In step 3, the atmospheric light value A and the transmittance t are known, and the formula I(x)=J(x)t(x)+A is used to obtain the preliminary defogging image J(x); in step 4, four different gamma values ​​are used to artificially expose the initial defogging image to generate four images with different exposures; in step 5, the exposure image is decomposed by non-sampled Gaussian pyramid and non-downsampled Gaussian difference pyramid, which is expressed as and r is the exposure image index, l is the decomposition scale index, 0≤l≤N, N is the maximum decomposition scale number, and the image decomposed by the non-subsampled Gaussian difference pyramid is weighted fused, with the weight value W i r The calculation formula is: Then the fused image S at each scale i for: Finally, the fused image F is reconstructed as:

2. The two-stage image defogging method combining an atmospheric model and a fusion strategy according to claim 1, characterized in that: The atmospheric light value obtained by the quadtree subdivision search estimation method is the pixel value of the sky position.

3. The two-stage image defogging method combining an atmospheric model and a fusion strategy according to claim 1, characterized in that: The parameters in the transmittance are: A ∞ is the atmospheric light value at infinity, and the λ value is adjusted adaptively according to the brightness value, λ(x)=1-I L (x), 0≤λ≤1 The control factor λ is used to constrain the speed of the linear change of the above formula, I L (x) is the brightness component of the foggy image, I Lmin and I Lmax I L(x) The minimum and maximum values ​​of .

4. The two-stage image defogging method combining an atmospheric model and a fusion strategy according to claim 1, characterized in that: The non-sampled Gaussian decomposition formula is: G0(i,j)=I(i,j), where 0≤l≤N is the scale index, N is the highest decomposition level, G l represents the NSGP image of the lth layer, G l-1 represents the l-1th layer NSGP image, g(m,n) is a separable Gaussian kernel weight function, G0 represents the 0th layer NSGP image, that is, the original input image, and the decomposition process is as follows: first, the original image is used as the bottom image G0 (the 0th layer of NSGP), and a convolution operation is performed with the Gaussian kernel weight function ω (size is 5×5) to obtain the image G1 (the 1st layer of NSGP), and it is iterated N times to form an N+1-layer pyramidal image data structure, that is, a non-sampled Gaussian pyramid image, represented by G0, G1, ... G N .

5. The two-stage image defogging method combining an atmospheric model and a fusion strategy according to claim 1, characterized in that: The non-subsampled Gaussian difference pyramid process can be expressed as: LP k (i,j)=G k+1 (i,j)-G k (i,j)LP N (i,j)=G N (i, j), where 0≤k≤N-1 is the scale index, N is the highest decomposition level, LP k represents the k-th layer NSDGP image, LP N represents the Nth layer NSDGP image; the decomposition process: using the 1st layer NSGP image and the 0th layer NSGP image to perform a differential operation to obtain image LP0 (the 0th layer of the NSDGP image), iterating N-1 times, generating N layers of non-subsampled Gaussian difference pyramid images, represented as LP0, LP1, ..., LP N-1 , L.P. N It is called the base image.

6. The two-stage image defogging method combining an atmospheric model and a fusion strategy according to claim 1, characterized in that: The definition of the gamma correction is as follows: I(x)→α·I γ (x), where α and γ are real numbers. When γ < 1, the visual effect of fog is enhanced; when γ > 1, part of the fog in the image can be eliminated. Therefore, images with different exposures can be generated by selecting different values ​​of γ > 1.