A fast image defogging method based on sky region segmentation

Abstract

The application discloses a kind of based on the quick image defogging method of sky area segmentation, it includes the following steps: to original fog map image enhancement is carried out and to the image after enhancement is carried out two threshold segmentation, obtain corresponding segmentation graph;Respectively to two segmentation graphs are opened and filtering are carried out, and superimposition is obtained segmentation mask graph;According to different foreground and background atmospheric light and transmissivity to original fog map is guided filtering, and by atmospheric scattering model is carried out inverse solution, obtain initial defogging image;Initial defogging image is fused with segmentation mask image, obtain fusion image;To fusion image image enhancement is carried out, obtain final defogging image.The application is carried out foreground and background segmentation processing after image enhancement, and the part of sudden change in depth of field is superimposed and segmented, to retain the details of image;Different depth factors are used to original fog map defogging, improve the flexibility and generalization of image defogging, and the image defogging effect in various scenes is ideal.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a fast image dehazing method based on segmenting sky regions. Background Technology

[0002] Images / videos captured in hazy weather are subject to the absorption, scattering, and refraction of light by atmospheric turbidity (such as hydrosols, fog, and haze), reducing the imaging accuracy of optical sensors and directly leading to weakened image visibility, such as blurred contrast, color shift, and reduced dynamic range. Since many applications in computer vision systems, including image understanding, target recognition, and target tracking, rely on input images or videos captured under ideal weather conditions, image clarity significantly impacts the accuracy of subsequent interpretation, analysis, recognition, and measurement results. Therefore, the research on fast image dehazing algorithms is increasingly attracting the attention of researchers.

[0003] Existing image dehazing methods employ image enhancement techniques to improve contrast or highlight image features, thereby clarifying foggy images. Alternatively, they rely on atmospheric scattering physics models for fog image restoration, utilizing prior knowledge of image degradation for dehazing; however, this prior theory fails when processing images containing sky. Another approach involves using deep learning to estimate model parameters or construct end-to-end dehazing networks, but these methods have proven ineffective in practical applications. Deep learning methods also suffer from high hardware computing power and data acquisition costs, poor generalization ability, and high power consumption, making them unsuitable for real-time on-site detection requirements. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention provides a fast image dehazing method based on sky region segmentation, which solves the problems of unsatisfactory dehazing effect, low generalization, sensitivity to environment, and high real-time detection requirements of existing methods.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0006] A fast image dehazing method based on sky region segmentation is provided, which includes the following steps:

[0007] S1. Enhance the original fog image to obtain the enhanced image; perform local thresholding and global optimal thresholding on the enhanced image to obtain the local segmentation map and the global segmentation map.

[0008] S2. Perform opening and filtering operations on the local segmentation image and the global segmentation image respectively to obtain the filtered local segmentation image and the global segmentation image; superimpose the filtered local segmentation image and the global segmentation image to obtain the segmentation mask image;

[0009] S3. Calculate the atmospheric light and transmittance of different foreground and background scenes in the original fog image according to different depth factors; perform guided filtering on the original fog image according to the atmospheric light and transmittance of different foreground and background scenes, and perform inverse solution through atmospheric scattering model to obtain the initial defogging image.

[0010] S4. Fuse the initial dehazed image with the segmentation mask image to obtain the fused image;

[0011] S5. Perform image enhancement on the fused image to obtain the final dehazed image.

[0012] Furthermore, the image enhancement in step S1 includes Unsharp filtering and adaptive histogram equalization.

[0013] Furthermore, the specific method for global optimal threshold segmentation in step S1 is as follows:

[0014] Initialize the segmentation threshold T, and divide the enhanced image into foreground and background according to the segmentation threshold T;

[0015] According to the formula:

[0016]

[0017]

[0018]

[0019]

[0020] b 2 =P1×(m1-mg) 2 +P2×(m2-mg) 2

[0021] The inter-class variance b of the enhanced image is obtained. 2 Where P1 and P2 represent the cumulative probabilities of the foreground and background, respectively, and p i Let represent the i-th histogram component, ∑(·) represent the summation function, m1 and m2 represent the cumulative probability mean of the foreground and background respectively, n represent the number of pixels in the foreground, L represent the total number of pixels in the enhanced image, and mg represent the global grayscale mean of the enhanced image.

[0022] By sequentially selecting grayscale values ​​from 0 to 255 as the segmentation threshold T and performing the above operation, the corresponding inter-class variance b can be obtained. 2 Select the inter-class variance b 2 The gray value corresponding to the maximum value is used as the segmentation threshold K; the enhanced image is segmented according to the segmentation threshold K to obtain the global segmentation map;

[0023] The specific method for local threshold segmentation in step S1 is as follows:

[0024] Calculate the standard deviation and mean of each pixel value and its neighborhood in the enhanced image; according to the formula:

[0025] T xy =aσ xy +bm xy

[0026] Obtain the variable local threshold T for each pixel xy Where a and b represent constants, σ xy m represents the standard deviation of the neighborhood of that pixel. xy This represents the mean value of the neighborhood of that pixel;

[0027] Based on the variable local threshold T for each pixel xy Segment each pixel; when the pixel value is greater than the corresponding variable local threshold T xy If the pixel value is less than or equal to the corresponding variable local threshold T, then the pixel value is set to 1; xy If the pixel value is set to 0, a local segmentation image is obtained.

[0028] Furthermore, the inter-class variance b 2 If there are multiple maximum values, the corresponding gray values ​​are calculated to obtain the mean value, and the mean value is used as the segmentation threshold K.

[0029] Furthermore, the filtering in step S2 employs inverse harmonic mean filtering and mask sliding filtering;

[0030] Inverse harmonic mean filtering is used to filter out salt noise in local and global segmentation maps while retaining pepper noise.

[0031] Mask sliding filter is used to filter out salt-and-pepper noise in local and global segmentation maps.

[0032] Furthermore, the specific steps of step S3 are as follows:

[0033] S3-1, According to the formula:

[0034]

[0035] Obtain layer J of the original fog map min(RGB) (x); where x represents a pixel, Ω(x) represents a window centered at pixel x, y represents the number of pixels within the window centered at pixel x, c represents the RGB channel, and J c (y) represents the value of pixel y selected in each of the RGB channels, and min(·) represents the minimum value function;

[0036] S3-2, According to the formula:

[0037] t(x)=e -βd(x)

[0038] The transmittance t(x) is obtained; where β represents the atmospheric scattering parameter and d(x) represents the depth of field.

[0039] S3-3, Take layer J min(RGB) The top 0.1% of pixels in (x) arranged by brightness are taken as atmospheric light value A;

[0040] S3-4. According to the formula:

[0041]

[0042] The original fog image, transmittance t(x), and atmospheric light value A are input into the atmospheric scattering physical model to obtain the initial defogging image J(x); where I(x) represents the original fog image.

[0043] Furthermore, the specific method for image enhancement in step S5 is as follows: based on the segmentation mask image, select pixel matrices for the sky and foreground parts in the fused image and assign weights to them to obtain updated sky and foreground parts; superimpose the updated sky and foreground parts to obtain the final dehazed image.

[0044] The beneficial effects of this invention are as follows: the fast image dehazing method performs foreground and background segmentation after image enhancement, and performs superimposed segmentation on parts with abrupt changes in depth of field to preserve image details; it uses different depth factors to dehaze the original fog image, which improves the flexibility and generalization of image dehazing, and achieves ideal image dehazing effect in various scenarios. Attached Figure Description

[0045] Figure 1 This is a detailed flowchart of the present invention;

[0046] Figure 2 This is an image segmentation diagram of the present invention;

[0047] Figure 3 This is a comparison chart showing the defogging effects of various defogging methods. Detailed Implementation

[0048] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0049] like Figure 1As shown, a fast image dehazing method based on sky region segmentation includes the following steps:

[0050] S1. Enhance the original fog image to obtain the enhanced image; perform local thresholding and global optimal thresholding on the enhanced image to obtain the local segmentation map and the global segmentation map.

[0051] S2. Perform opening and filtering operations on the local segmentation image and the global segmentation image respectively to obtain the filtered local segmentation image and the global segmentation image; superimpose the filtered local segmentation image and the global segmentation image to obtain the segmentation mask image;

[0052] S3. Calculate the atmospheric light and transmittance of different foreground and background scenes in the original fog image according to different depth factors; perform guided filtering on the original fog image according to the atmospheric light and transmittance of different foreground and background scenes, and perform inverse solution through atmospheric scattering model to obtain the initial defogging image.

[0053] S4. Fuse the initial dehazed image with the segmentation mask image to obtain the fused image;

[0054] S5. Perform image enhancement on the fused image to obtain the final dehazed image.

[0055] The image enhancement in step S1 includes Unsharp filtering and adaptive histogram equalization.

[0056] like Figure 2 As shown (a, b, c, and d represent the original fog map, global segmentation map, local segmentation map, and segmentation mask map, respectively), the specific method for global optimal threshold segmentation in step S1 is as follows:

[0057] Initialize the segmentation threshold T, and divide the enhanced image into foreground and background according to the segmentation threshold T;

[0058] According to the formula:

[0059]

[0060]

[0061]

[0062]

[0063] b 2 =P1×(m1-mg) 2 +P2×(m2-mg) 2

[0064] The inter-class variance b of the enhanced image is obtained. 2Where P1 and P2 represent the cumulative probabilities of the foreground and background, respectively, and p i Let represent the i-th histogram component, ∑(·) represent the summation function, m1 and m2 represent the cumulative probability mean of the foreground and background respectively, n represent the number of pixels in the foreground, L represent the total number of pixels in the enhanced image, and mg represent the global grayscale mean of the enhanced image.

[0065] By sequentially selecting grayscale values ​​from 0 to 255 as the segmentation threshold T and performing the above operation, the corresponding inter-class variance b can be obtained. 2 Select the inter-class variance b 2 The gray value corresponding to the maximum value is used as the segmentation threshold K; the enhanced image is segmented according to the segmentation threshold K to obtain the global segmentation map;

[0066] The specific method for local threshold segmentation in step S1 is as follows:

[0067] Calculate the standard deviation and mean of each pixel value and its neighborhood in the enhanced image; according to the formula:

[0068] T xy =aσ xy +bm xy

[0069] Obtain the variable local threshold T for each pixel xy Where a and b represent constants, σ xy m represents the standard deviation of the neighborhood of that pixel. xy This represents the mean value of the neighborhood of that pixel;

[0070] Based on the variable local threshold T for each pixel xy Segment each pixel; when the pixel value is greater than the corresponding variable local threshold T xy If the pixel value is less than or equal to the corresponding variable local threshold T, then the pixel value is set to 1; xy If the pixel value is set to 0, a local segmentation image is obtained.

[0071] Between-class variance b 2 If there are multiple maximum values, the corresponding gray values ​​are calculated to obtain the mean value, and the mean value is used as the segmentation threshold K.

[0072] The filtering in step S2 uses inverse harmonic mean filtering and mask sliding filtering;

[0073] Inverse harmonic mean filtering is used to filter out salt noise in local and global segmentation maps while retaining pepper noise.

[0074] Mask sliding filter is used to filter out salt-and-pepper noise in local and global segmentation maps.

[0075] The specific steps of step S3 are as follows:

[0076] S3-1, According to the formula:

[0077]

[0078] Obtain layer J of the original fog map min(RGB) (x); where x represents a pixel, Ω(x) represents a window centered at pixel x, y represents the number of pixels within the window centered at pixel x, c represents the RGB channel, and J c (y) represents the value of pixel y selected in each of the RGB channels, and min(·) represents the minimum value function;

[0079] S3-2, According to the formula:

[0080] t(x)=e -βd(x)

[0081] The transmittance t(x) is obtained; where β represents the atmospheric scattering parameter and d(x) represents the depth of field.

[0082] S3-3, Take layer J min(RGB) The top 0.1% of pixels in (x) arranged by brightness are taken as atmospheric light value A;

[0083] S3-4. According to the formula:

[0084]

[0085] The original fog image, transmittance t(x), and atmospheric light value A are input into the atmospheric scattering physical model to obtain the initial defogging image J(x); where I(x) represents the original fog image.

[0086] The specific method for image enhancement in step S5 is as follows: based on the segmentation mask image, select pixel matrices for the sky and foreground parts in the fused image and assign weights to them to obtain the updated sky and foreground parts; superimpose the updated sky and foreground parts to obtain the final dehazed image.

[0087] In one embodiment of the present invention, the inverse solution process of the atmospheric scattering model is as follows:

[0088] Since the minimum values ​​of each channel (RGB) in the original fog image will not approach zero, the minimum layer of the original fog image is determined. Assuming the atmospheric light value A is known, the atmospheric scattering physical model is deformed, and two minimum value calculations are performed to derive the estimated transmittance. The formula is as follows:

[0089]

[0090]

[0091] Introducing the depth-of-field preservation parameter ω, we obtain:

[0092]

[0093] By controlling and retaining a certain amount of fog, a better visual effect can be achieved.

[0094] By incorporating transmittance and atmospheric light into the transformation formula of the atmospheric scattering model, and to correct the image's overall tendency towards white field caused by excessively low transmittance t(x), a threshold T0 can be set. When the value of transmittance t(x) is less than the threshold T0, the value of transmittance t(x) is set equal to the threshold T0, resulting in:

[0095]

[0096] Where the threshold T0 is set to 0.1, and max(·) represents the maximum value function.

[0097] The inverse solution formula for the atmospheric scattering physics model is:

[0098] I(x)=J(x)t(x)+A(1-t(x))

[0099] t(x)=e -βd(x)

[0100] Where A(1-t(x)) represents the light rays that are superimposed on the original fog map after atmospheric light A degrades in the fog, and J(x)t(x) represents the direct attenuation part of the light rays after passing through the original fog map.

[0101] Combining the above formulas, we obtain the formula corresponding to the initial dehazed image J(x) in step S3-4.

[0102] This method uses information entropy, peak signal-to-noise ratio, gray-level contrast ratio, average gradient, effective detail intensity, tone reproduction, structural information, and comprehensive evaluation as assessment metrics. Information entropy is calculated based on the entropy value of an image to determine its information content. Higher image complexity results in higher information entropy, indicating greater information content and better image quality. The formula is as follows:

[0103]

[0104] Where H(A) represents the information entropy of the image, a represents the gray value, and P A (a) represents the gray-scale probability distribution, log(·) represents the logarithmic function, and ∑(·) represents the summation function.

[0105] Peak signal-to-noise ratio (PSNR) measures the ratio of effective information to noise in an image, reflecting the degree of image distortion. A higher PSNR value indicates better image quality. The formula is as follows:

[0106]

[0107] Where PSNR represents the peak signal-to-noise ratio of the image, MSE represents the mean square error, and z represents the maximum possible value of the pixel value.

[0108] Gray-level contrast refers to the magnitude of the difference in gray levels within an image, reflecting the details of the image. Its formula is as follows:

[0109]

[0110] Where C represents the gray-level contrast of the image, δ(i,j) represents the gray-level difference between adjacent pixels i and j, and P δ (i,j) represents the pixel distribution probability of the gray level difference δ(i,j) between adjacent pixels i and j.

[0111] The average gradient refers to the significant difference in gray levels near the boundaries or shadows of an image. It reflects the contrast and texture characteristics of image details. The larger the value, the higher the image sharpness. The formula is as follows:

[0112]

[0113] Where G represents the average gradient of the image, and M×N represents the size of the image. Represents the gradient in the horizontal direction. This represents the gradient in the vertical direction.

[0114] Since higher image contrast is not always better, noise and the Halo effect are particularly noticeable at object edges or abrupt changes in depth of field. The Halo effect renders the contrast evaluation component ineffective and significantly impacts the visual quality of the dehazed image. Subtracting the Halo intensity from the total detail intensity yields the effective detail intensity of the dehazed image, as shown in the following formula:

[0115]

[0116] Among them, I valid L represents the effective detail intensity of an image. s This represents the total detail intensity; n represents the number of pixels in the window Ω, using a 7×7 area.

[0117] The degree of color reproduction reflects whether an image is realistic and natural; its formula is as follows:

[0118]

[0119] Where, d correl h represents the degree of color reproduction of an image. k and h k 'Represents the original image and the final image, respectively. and represents the mean of the original image and the mean of the final image, respectively, and k represents the gray level of the histogram.

[0120] The brightness of an object's surface as perceived by the human eye is the product of illuminance and reflectance. However, the structure of objects in a scene is independent of illuminance. To obtain structural information from an image, the influence of illuminance is usually removed. Therefore, by filtering out the illuminance image from both the pre- and post-dehazing images, only the reflectance images R1 and R2 are compared for structural information. The formula for structural information is as follows:

[0121]

[0122]

[0123] Where S(R1,R2) represents the structural information of reflectance maps R1 and R2, M represents the number of pixels, u1 and u2 represent the image mean of reflectance maps R1 and R2 respectively, σ1 and σ2 represent the standard deviation of reflectance maps R1 and R2 respectively, and σ 12 R represents the covariance of reflection maps R1 and R2. 1,i and R 2,i Let R1 and R2 represent the i-th pixel in the reflection maps R1 and R2, respectively.

[0124] The formula for comprehensive evaluation is as follows:

[0125] Q = (L valid ) α ·(d correl ) β ·(S) γ

[0126] Where Q represents the comprehensive evaluation, and α, β, and γ represent the weights in the comprehensive evaluation, all of which are 1.

[0127] Comparative experiments were conducted with the proposed method using He dehazing, RetineX dehazing, efficient multi-scale correlation wavelet dehazing, and manual multi-exposure image fusion dehazing. Table 1 shows the comparison of evaluation values ​​for various methods, as shown below:

[0128] Table 1 Comparison of Evaluation Values ​​for Multiple Methods

[0129]

[0130] As shown in Table 1, the information entropy of the dehazed image obtained by this method is lower than that obtained by the He dehazing method but higher than that obtained by the other three dehazing methods, indicating that the dehazed image obtained by this method has higher information content and better quality. The peak signal-to-noise ratio (PSNR) of the dehazed image obtained by this method is higher than that obtained by the other four dehazing methods, indicating that the dehazed image obtained by this method has the lowest distortion and excellent dehazing effect. The gray-level contrast of the dehazed image obtained by this method is lower than that obtained by the efficient multi-scale correlation wavelet dehazing method but higher than that obtained by the other three dehazing methods, indicating that the dehazed image obtained by this method retains more details and has excellent dehazing effect. The average gradient of the dehazed image obtained by this method is lower than that obtained by the He dehazing method but higher than that obtained by the other three dehazing methods. The average gradient of the dehazed image obtained by this method indicates that the clarity of the dehazed image obtained by this method is excellent. The effective detail intensity of the dehazed image obtained by this method is lower than that of the dehazed images obtained by the efficient multi-scale correlation wavelet dehazing method and the artificial multi-exposure image fusion dehazing method, but higher than that of the dehazed images obtained by the other two dehazing methods, indicating that the visual effect of the dehazed image obtained by this method is better. The hue of the dehazed image obtained by this method is higher than that of the dehazed images obtained by the other four dehazing methods, indicating that the restoration degree of the dehazed image obtained by this method is the highest. The structural information of the dehazed image obtained by this method is higher than that of the dehazed images obtained by the other four dehazing methods, indicating that the dehazing effect of this method is excellent. The overall evaluation of the dehazed image obtained by this method is higher than that of the dehazed images obtained by the other four dehazing methods, indicating that the dehazing effect of this method is ideal.

[0131] Table 2 is the subjective defogging evaluation table, as shown below:

[0132] Table 2 Subjective Defogging Evaluation Table

[0133]

[0134] From Table 2 and Figure 3 As shown, the image quality, detail retention, and sharpness of the dehazed image obtained by this method are superior to those obtained by the other four methods. Figure 3 In this diagram, e represents the original fog image, f represents the He dehazing method, g represents the RetineX dehazing method, h represents the efficient multi-scale correlation wavelet dehazing method, i represents the artificial multi-exposure image fusion dehazing method, and j represents the present method.

[0135] In summary, this invention performs foreground and background segmentation after image enhancement, and performs superimposed segmentation on parts with abrupt changes in depth of field to preserve image details; it uses different depth factors to dehaze the original fog image, improving the flexibility and generalization of image dehazing, and achieving ideal image dehazing results in various scenarios.

Claims

1. A fast image dehazing method based on sky region segmentation, characterized in that: Includes the following steps: S1. Enhance the original fog image to obtain the enhanced image; perform local thresholding and global optimal thresholding on the enhanced image to obtain the local segmentation map and the global segmentation map. S2. Perform opening and filtering operations on the local segmentation map and the global segmentation map respectively to obtain the filtered local segmentation map and the global segmentation map; The filtered local segmentation image and the global segmentation image are superimposed to obtain the segmentation mask image; S3. Calculate the atmospheric light and transmittance of different foreground and background scenes in the original fog image according to different depth factors; perform guided filtering on the original fog image according to the atmospheric light and transmittance of different foreground and background scenes, and perform inverse solution through atmospheric scattering model to obtain the initial defogging image. S4. Fuse the initial dehazed image with the segmentation mask image to obtain the fused image; S5. Perform image enhancement on the fused image to obtain the final dehazed image; The specific method for global optimal threshold segmentation in step S1 is as follows: Initialize the segmentation threshold T, and divide the enhanced image into foreground and background according to the segmentation threshold T; According to the formula: The inter-class variance of the enhanced image is obtained. ;in, , Let these represent the cumulative probabilities of the foreground and background, respectively. Indicates the first Each histogram component This represents the summation function. , These represent the cumulative probability mean of the foreground and background, respectively. Indicates the number of pixels in the foreground. This represents the total number of pixels in the enhanced image. This represents the global grayscale mean of the enhanced image; By sequentially selecting grayscale values ​​from 0 to 255 as the segmentation threshold T and performing the above operation, the corresponding inter-class variance can be obtained. Selecting inter-class variance The gray value corresponding to the maximum value is used as the segmentation threshold K; the enhanced image is segmented according to the segmentation threshold K to obtain the global segmentation map; The specific method for local threshold segmentation in step S1 is as follows: Calculate the standard deviation and mean of each pixel value and its neighborhood in the enhanced image; according to the formula: Obtain the variable local threshold for each pixel ;in, and Represents a constant. This represents the standard deviation of the pixel's neighborhood. This represents the mean value of the neighborhood of that pixel; Based on the variable local threshold of each pixel Segment each pixel; when the pixel value is greater than the corresponding variable local threshold... If the pixel value is less than or equal to the corresponding variable local threshold, then set the pixel value to 1; If the pixel value is set to 0, a local segmentation image is obtained. The filtering in step S2 employs inverse harmonic mean filtering and mask sliding filtering. The inverse harmonic mean filter is used to filter out salt noise in the local segmentation map and the global segmentation map separately while retaining pepper noise; The mask sliding filter is used to filter out salt-and-pepper noise in the local segmentation map and the global segmentation map; The specific steps of step S3 are as follows: S3-1, According to the formula: Obtain the layers of the original fog map ;in, Represents pixels, Represented in pixels The central window, Represents pixels The pixels within the center window, Indicates RGB channels. This indicates selecting pixels in the RGB channels respectively. The value, Describes the minimum value function; S3-2, According to the formula: Obtain transmittance ;in, Indicates atmospheric scattering parameters, Indicates depth of field; S3-3, Apply Layer The top 0.1% of pixels, arranged by brightness, are used as atmospheric light values. ; S3-4. According to the formula: Original fog map, transmittance and atmospheric light value The image is input into the atmospheric scattering physics model to obtain the initial dehazed image. ;in, This represents the original fog map.

2. The fast image dehazing method based on sky region segmentation according to claim 1, characterized in that: The image enhancement in step S1 includes Unsharp filtering and adaptive histogram equalization.

3. The fast image dehazing method based on sky region segmentation according to claim 1, characterized in that: The inter-class variance If there are multiple maximum values, the corresponding gray values ​​are calculated to obtain the mean value, and the mean value is used as the segmentation threshold K.

4. The fast image dehazing method based on sky region segmentation according to claim 1, characterized in that: The specific method for image enhancement in step S5 is as follows: based on the segmentation mask image, select pixel matrices for the sky and foreground parts in the fused image and assign weights to them to obtain updated sky and foreground parts; superimpose the updated sky and foreground parts to obtain the final dehazed image.

Images