Fine-grained sky segmentation and transmittance fusion image dehazing method

Abstract

This invention relates to a refined sky segmentation and transmittance fusion method for image dehazing, belonging to the field of image processing technology. Existing techniques, when processing images with both sky and fog, often result in a white scene due to the excessive brightness of the sky, which contradicts the dark channel prior rule, leading to severe color distortion in the sky portion of the dehazed image. This invention utilizes an algorithm combining the Canny operator and Otsu's method to extract edge features from images with both sky and fog, initially dividing them into sky and non-sky regions. Side-window filtering is used to smooth image edges and clarify the boundaries between sky and non-sky regions. The transmittance of the sky and non-sky regions is estimated using dark channel prior algorithms and bright channel prior algorithms, respectively. After weighted fusion processing, the transmittance of the entire image is obtained. Atmospheric light values ​​are obtained based on the pixel mean of the sky region. Finally, an atmospheric scattering model is used to obtain the recovered dehazed image. This invention is applied to image recognition and target tracking in foggy image monitoring processes.

Description

Technical Field

[0001] This invention relates to a technical solution called a fine sky segmentation and transmittance fusion image dehazing method, which is applied to image recognition and target tracking in the process of monitoring foggy images and belongs to the field of image processing technology. Background Technology

[0002] In foggy weather, the scattering and absorption of light by water vapor and fog droplets in the air can cause image degradation such as reduced clarity and color shift, resulting in a serious decline in the quality of image recognition and target tracking in outdoor scenes.

[0003] To improve the image quality captured during image monitoring in foggy weather, existing technologies propose three types of dehazing algorithms based on image processing principles: deep learning-based dehazing algorithms, non-physical model-based dehazing algorithms, and physical model-based dehazing algorithms.

[0004] Deep learning-based dehazing algorithms use convolutional neural networks to build a deep learning model. This model leverages the learning capabilities of neural networks to perform supervised learning on large amounts of labeled data, analyzing and extracting effective features from images relevant to haze removal. For example, the DehazeNet dehazing model is trained using simulated hazy images. The hazy image is input into the dehazing model, features are extracted layer by layer, and the model estimates the image's transmittance through learning, thus outputting the dehazed image.

[0005] Dehazing algorithms based on non-physical models enhance image details and strengthen image edges to obtain more detailed information, ultimately achieving a dehazing effect. These algorithms specifically include histogram equalization, color level enhancement algorithms, Retinex algorithms, wavelet transforms, and curvelet transforms.

[0006] Physically based dehazing algorithms are the mainstream dehazing methods in current technology. They mainly consider the imaging principle of fog, light scattering and attenuation, and then establish a model. A typical physically based dehazing algorithm is the Dark Channel Prior (DCP) algorithm, which includes:

[0007] Based on the principle of atmospheric scattering model, the hazy image and the restored dehazed image are correlated using the following formula:

[0008] I(x)=J(x)t(x)+A(1-t(x))

[0009] In the formula: x represents the pixel coordinates, I(x) is the hazy image, J(x) is the restored dehazed image, A is the atmospheric light value, and t(x) is the atmospheric transmittance. After identity transformation, the formula for solving the dehazed image is obtained:

[0010]

[0011] By combining the dark channel prior with the atmospheric scattering model, it is found that in a certain local region, there are always some pixels whose grayscale value in a certain color channel is so small as to be close to 0, i.e.:

[0012]

[0013] In the formula: J dark (x) represents the dark primary color channel, J c (y) represents one of the three channels R, G, B, Ω(x) represents a region centered at x, and y represents the pixel coordinates in the region Ω(x).

[0014] Assuming the atmospheric light value A is already determined, performing an identity transformation on the formula and then filtering by minimum value yields:

[0015]

[0016] Because J dark Since the pixel grayscale value of (x) approaches 0, we can obtain:

[0017]

[0018] At the same time, to make the image appear more natural and realistic, a constant parameter ω (ω = 0.95) is introduced, thus:

[0019]

[0020] The method for selecting atmospheric light value A is as follows: First, select the pixels with the largest gray values ​​(0 to 255 gray values) in the top 0.1% of the pixels in the dark channel image. These pixels are the least transparent points of fog. These pixels correspond to the image with fog. Select the gray value of the pixel with the largest gray value as the atmospheric light value A.

[0021] Based on the transmittance t(x) and the known atmospheric light value A, a hazy image can be restored to a dehazed image:

[0022]

[0023] Although the dark channel prior dehazing algorithm has achieved good dehazing results for most foggy images without sky, it fails when used to process foggy images with sky. The sky appears as a white scene due to its excessive brightness, which does not conform to the dark channel prior rule. As a result, the original overly bright areas, such as the sky, have severe color distortion and noise in the dehazed image. Summary of the Invention

[0024] To avoid color distortion and noise in overly bright areas such as the sky in foggy images when using dark channel prior dehazing algorithms to dehaze foggy image surfaces, we propose a technical solution called Fine Sky Segmentation and Transmittance Fusion Image Dehazing Method.

[0025] The fine sky segmentation and transmittance fusion image dehazing method of the present invention is characterized by:

[0026] 1. Obtain a color image I(x) with sky and fog;

[0027] 2. Perform grayscale processing on I(x) to obtain the grayscale image I′(x) of I(x);

[0028] 3. Delineate the non-sky and sky regions in the grayscale image I′(x).

[0029] First, the maximum inter-class variance δ is obtained using Otsu's method. 2 :

[0030]

[0031] Where: k is the gray value, the average gray values ​​of the non-sky region W0 and the sky region W1 are h0(k) and h1(k) respectively, and the probability of a pixel falling into the non-sky region or the sky region of the image is: h(k) is the average gray value of the entire image; and the maximum inter-class variance δ 2 The corresponding k value is the optimal threshold, and this optimal threshold is set as the high threshold T in the Canny operator. h , lower threshold T l Set as high threshold T h of The high threshold T h Low threshold T l It becomes the edge feature between the non-sky region and the sky region;

[0032] Secondly, after applying morphological dilation and erosion operations to the grayscale image I′(x), edge features are filled to obtain the binary image I(x)′.

[0033] 4. Perform edge smoothing on the non-sky regions in the binary image I(x)′.

[0034] A weighted filtering method is used to apply side-window filtering to the binary image I(x)′. By default, the sky region appears above the non-sky region. Five types of windows are constructed with the image boundary pixels of the non-sky region as the center point, labeled as bottom (d), left (l), right (r), lower left (ld), and lower right (rd), respectively. Figure 1As shown, the five types of windows are labeled S = {d, l, r, ld, rd}, with n as the index of each type. The boundary pixel to be processed is labeled i, and the neighboring pixels of i are labeled j. The image after weighted filtering by a certain type of window is I. n :

[0035]

[0036] In the formula: For the window at the boundary of pixel i to be processed, w ij As the weight kernel, w ij Located near the boundary pixel i to be processed, q j N is the pixel value of the neighboring pixel j. n This is the sum of the weights of a certain type of window, i.e.:

[0037]

[0038] Cost function E i for:

[0039]

[0040] In the formula: q i Let i be the pixel value of the boundary pixel to be processed.

[0041] Then the best window image I m for:

[0042] I m =argmin n∈S E i ,

[0043] From the best window image I m Filtering is applied to the boundary pixel i to prevent the edge diffusion of the non-sky region in the binary image I(x)′, resulting in the final binary image I(x)″. In I(x)″, the edge information of the non-sky region is more obvious and the edges are smoother.

[0044] 5. Estimate transmittance

[0045] First, the average gray value of all pixels in the sky region is selected as the atmospheric light value A in the final binary image I(x)″.

[0046] Secondly, the dark channel prior algorithm is used to estimate the transmittance t in the non-sky region. non-sky (x):

[0047]

[0048] In the formula: ω is a constant parameter, Ω(x) represents a region centered at x, y represents the pixel coordinates within the Ω(x) region, and I...c (y) represents the image of one channel among the R, G, and B channels. A c This represents the atmospheric light value of one of the three channels: R, G, and B.

[0049] Third, the transmittance t of the sky region is estimated using a bright channel prior algorithm. sky (x):

[0050]

[0051] 6. Fusion transmittance

[0052] The transmittance t of the non-sky region is fused using a weighted fusion method. non-sky (x), Sky area transmittance t sky (x):

[0053] The probability λ of the sky region appearing in the final binary image I(x)″ is:

[0054]

[0055] In the formula: Q is the total number of pixels in the sky region, H is the height of the entire image, and W is the width of the entire image;

[0056] Transmittance t(x) of the entire image:

[0057] t(x)=λt sky (x)+(1-λ)t non-sky (x)-ζ,

[0058] In the formula: ζ is a user-defined parameter that adds constraints to adjust the transmittance t(x) of the entire image.

[0059]

[0060] 7. According to the formula Obtain the recovered dehazed image J(x).

[0061] The technical advantages of this invention are as follows:

[0062] Unlike existing technologies, this invention uses a combination of the Canny operator and the Otsu method to extract image edge features, resulting in more accurate threshold determination and more targeted extraction based on specific images. It also uses a side window filter to smooth the image edges, preserving more image information, making the boundaries between sky and non-sky areas more prominent, the edge features more obvious, and the boundary division clearer.

[0063] This invention calculates the transmittance of the sky region and the non-sky region separately, and then performs weighted fusion to obtain the transmittance of the entire image. This enables the dehazing of images with both sky and fog using a dark channel prior dehazing algorithm, overcoming the shortcomings of existing technologies. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of five types of windows for side window filtering in this invention. Detailed Implementation

[0065] The grayscale processing refers to taking a color image I(x) with sky and fog, calculating the weighted sum of the R, G, and B channel components, and then determining the corresponding coefficients based on the human eye's sensitivity to the R, G, and B colors to obtain a grayscale image I′(x). Furthermore, using a method of automatically adjusting image contrast through grayscale transformation, the grayscale transformation function is obtained by applying the probability density function of the grayscale level, thereby making the grayscale image I′(x) more contrasting.

[0066] By applying artificial coefficient constraints to the atmospheric light value A, color distortion in the sky region of the recovered dehazed image J(x) can be avoided. After trial calculation, the coefficient is set to 0.8.

[0067] Estimating the transmittance t in the non-sky region non-sky During the process of (x), the constant parameter ω = 0.95.

Claims

1. A method for image dehazing using fine sky segmentation and transmittance fusion, characterized by: (1) Obtain a color image I(x) with sky and fog; (2) Perform grayscale processing on I(x) to obtain the grayscale image I′(x) of I(x); (3) Delineate the non-sky region and sky region in the grayscale image I′(x). First, the maximum inter-class variance δ is obtained using Otsu's method. 2 : Where: k is the gray value, the average gray values ​​of the non-sky region W0 and the sky region W1 are h0(k) and h1(k) respectively, and the probability of a pixel falling into the non-sky region or the sky region of the image is: h(k) is the average gray value of the entire image; and the maximum inter-class variance δ 2 The corresponding k value is the optimal threshold, and this optimal threshold is set as the high threshold T in the Canny operator. h , lower threshold T l Set as high threshold T h of The high threshold T h Low threshold T l It becomes the edge feature between the non-sky region and the sky region; Secondly, after applying morphological dilation and erosion operations to the grayscale image I′(x), edge features are filled to obtain the binary image I(x)′. (4) Perform edge smoothing on the non-sky regions in the binary image I(x)′. A weighted filtering method is applied to the binary image I(x)′ using side window filtering. By default, the sky region appears above the non-sky region. Five types of windows are constructed, centered on the image boundary pixels of the non-sky region, labeled as bottom (d), left (l), right (r), lower left (ld), and lower right (rd). These five types of windows are denoted as S={d,l,r,ld,rd}, with n as the index. The boundary pixel to be processed is labeled i, and the neighboring pixels of i are labeled j. The image after weighted filtering using a certain type of window is I. n : In the formula: For the window at the boundary of pixel i to be processed, w ij As the weight kernel, w ij Located near the boundary pixel i to be processed, q j N is the pixel value of the neighboring pixel j. n This is the sum of the weights of a certain type of window, i.e.: Cost function E i for: In the formula: q i Let i be the pixel value of the boundary pixel to be processed. Then the best window image I m for: I m =argmin n∈S E i , From the best window image I m Filtering is applied to the boundary pixel i to prevent the edge diffusion of the non-sky region in the binary image I(x)′, resulting in the final binary image I(x)″. In I(x)″, the edge information of the non-sky region is more obvious and the edges are smoother. (5) Estimating transmittance First, the average gray value of all pixels in the sky region is selected as the atmospheric light value A in the final binary image I(x)″. Secondly, the dark channel prior algorithm is used to estimate the transmittance t in the non-sky region. non-sky (x): In the formula: ω is a constant parameter, Ω(x) represents a region centered at x, y represents the pixel coordinates within the Ω(x) region, and I... c (y) represents the image of one channel among the R, G, and B channels. A c This represents the atmospheric light value of one of the three channels: R, G, and B. Third, the transmittance t of the sky region is estimated using a bright channel prior algorithm. sky (x): (6) Fusion transmittance The transmittance t of the non-sky region is fused using a weighted fusion method. non-sky (x), Sky area transmittance t sky (x): The probability λ of the sky region appearing in the final binary image I(x)″ is: In the formula: Q is the total number of pixels in the sky region, H is the height of the entire image, and W is the width of the entire image; Transmittance t(x) of the entire image: t(x)=λt sky (x)+(1-λ)t non-sky (x)-ζ, In the formula: ζ is a user-defined parameter that adds constraints to adjust the transmittance t(x) of the entire image. (7) According to the formula Obtain the recovered dehazed image J(x).

2. The fine sky segmentation and transmittance fusion image dehazing method according to claim 1, characterized in that, The grayscale processing refers to taking a color image I(x) with sky and fog, calculating the weighted sum of the R, G, and B channel components, and then determining the corresponding coefficients based on the human eye's sensitivity to the R, G, and B colors to obtain a grayscale image I′(x). Furthermore, using a method of automatically adjusting image contrast through grayscale transformation, the grayscale transformation function is obtained by applying the probability density function of the grayscale level, thereby making the grayscale image I′(x) more contrasting.

3. The fine sky segmentation and transmittance fusion image dehazing method according to claim 1, characterized in that, By applying artificial coefficient constraints to the atmospheric light value A, color distortion in the sky region of the recovered dehazed image J(x) can be avoided. After trial calculation, the coefficient is set to 0.

8.

4. The fine sky segmentation and transmittance fusion image dehazing method according to claim 1, characterized in that, Estimating the transmittance t in the non-sky region non-sky During the process of (x), the constant parameter ω = 0.95.

Images