An improved dark channel prior dehazing algorithm

Through the improved dark channel prior defog removal algorithm, combined with Rayleigh scattering theory and active contour model, the problems of slow image defog removal speed, color distortion and incomplete defog removal are solved, and more efficient image defog removal is achieved.

CN113920136BActive Publication Date: 2024-11-01LIAONING TECHNICAL UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111191635.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-13
Publication Date
2024-11-01
Estimated Expiration
2041-10-13

AI Technical Summary

Technical Problem

The prior art has problems of slow fog removal speed, distortion of color and incomplete fog removal during image defog removal, especially when dealing with fog images in real weather.

Method used

Using the improved dark channel prior defog removal algorithm, Rayleigh scattering theory is used to improve the estimation of atmospheric light, the sky area is segmented based on the active contour model, and the dark channel value is estimated using filters of different sizes, and the transmission is simplified. The process is used to improve the processing speed of the algorithm and the removal effect of the fog.

Benefits of technology

It improves the speed and quality of image defog removal, reduces color distortion and halo effects, and enhances the real-time and defog removal effect of the algorithm.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113920136B_ABST
    Figure CN113920136B_ABST
Patent Text Reader

Abstract

The present invention discloses an improved dark channel prior defogging algorithm, the steps of which are: using Rayleigh scattering theory to improve the estimated value of atmospheric light; using a segmentation method based on an active contour model to segment the sky area to obtain a sky area and a non-sky area; using filters of different sizes to estimate the dark channel value; simplifying the determination of transmittance to improve the processing speed of the algorithm. The present invention first adds Rayleigh scattering theory to improve the estimation method of atmospheric light on the basis of the original dark channel prior defogging algorithm, then divides the fog image into sky and non-sky areas, fuses dark channel images of different filter sizes to obtain a fused dark channel, and finally simplifies the solution process of transmittance to improve the running speed of the algorithm and enhance real-time performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image defogging, and in particular relates to an improved dark channel priori defogging algorithm. Background Art

[0002] The quality of outdoor images taken in foggy and hazy environments will be affected to varying degrees in terms of contrast and color due to the presence of a large number of suspended particles in the atmosphere. However, in many image processing systems such as target detection and satellite remote sensing, people have strict requirements on the quality of input images. Therefore, it is of great practical significance in the field of computer vision to sharpen degraded foggy images.

[0003] In the early days of image defogging, methods such as improving image contrast and saturation were often used to express the main features of the image. This method can achieve the purpose of image clarity, but it has low reliability and often results in a large amount of information loss. With the continuous development of modern science and technology and the continuous improvement of image quality requirements, the atmospheric scattering model has been applied to the field of image defogging, and a large number of defogging algorithms relying on the atmospheric scattering model have emerged. These algorithms fully consider the image degradation process and physically remove the fog in the image. At present, with the rise of deep learning, the application of deep learning methods to process foggy images has gradually become a mainstream, but such methods are limited by many factors, and there are still problems such as incomplete defogging when clarifying images in real scenes.

[0004] In summary, the methods for clearing foggy images can be divided into three categories: algorithms based on image enhancement, algorithms based on deep learning, and defogging algorithms based on physical models. Defogging algorithms based on image enhancement can effectively improve the contrast and color information of images, but will sacrifice some detail information, such as defogging algorithms based on histogram equalization, curvelet transform, and Retinex. Image defogging algorithms based on deep learning have appeared in recent years. Ren et al. proposed a convolutional neural network for multi-scale feature extraction and defogging by estimating the transmittance of fog images. However, the performance is often unsatisfactory when processing fog images in real weather. The defogging algorithm based on physical models analyzes the process of image quality degradation and is more targeted. Jackson et al. proposed a defogging method that uses Rayleigh scattering, establishes a scattering coefficient model to estimate the initial transmittance map, and uses guided filtering to optimize the transmittance map, which improves the calculation speed, but the image defogging is not thorough enough. He et al. proposed a dark channel defogging algorithm, which can effectively restore most foggy images, but because the bright sky area does not meet the dark channel prior, the processed sky area will produce a serious halo effect. Later, He et al. proposed to use guided filtering instead of soft cutout method to optimize transmittance based on their dark channel prior (DCP) theory, which improved the defogging efficiency. However, after processing, there are still problems such as incomplete defogging and slow processing speed. In view of the shortcomings of He et al.'s method, Liu Guo et al. proposed to introduce adaptive threshold in the process of obtaining dark primary color, which improved the accuracy of transmittance, but the operating efficiency is still relatively low. Summary of the invention

[0005] Based on the above deficiencies in the prior art, the object of the present invention is to provide an improved dark channel prior defogging algorithm to solve the problems of slow defogging speed, color distortion and incomplete defogging in traditional defogging methods.

[0006] In order to solve the above technical problems, the present invention is implemented by the following technical solutions:

[0007] The present invention provides an improved dark channel priori defogging algorithm, comprising the following steps:

[0008] S1: Apply Rayleigh scattering theory to improve estimates of atmospheric light;

[0009] S2: Segment the sky area using a segmentation method based on an active contour model to obtain a sky area and a non-sky area;

[0010] S3: Use filters of different sizes to estimate the dark channel value;

[0011] S4: Simplify the calculation of transmittance and improve the processing speed of the algorithm.

[0012] Optionally, the step S1 includes the following steps:

[0013] S11, according to Rayleigh scattering, select the pixel sequence with the highest intensity from the blue channel of the input image by column and take the average value of the sequence data to obtain A1 to eliminate the halo effect. A1 is expressed as follows:

[0014] A1=avg(max(I B列向 ))

[0015] Where: A1 is the average maximum value of the blue channel of the fog image, I B is the blue channel value of the fog image;

[0016] S12, calculate the average value of the lowest pixel in the red and green channels A2, and further eliminate unnecessary texture information after A1:

[0017] A2=avg(min(I G )+min(I R ))

[0018] Where: I G is the green channel value of the fog image, I R is the red channel value of the fog image, A2 is the average of the darkest green channel and the darkest red channel;

[0019] S13. Calculate the sum of A1 and A2 to estimate the atmospheric light A:

[0020] A=A1+A2.

[0021] Optionally, in step S2, dark channel images of different filter sizes are fused, and the edge area focuses on using a dark channel image with a smaller filter size to better extract scene structure information in depth discontinuous areas and avoid halo artifacts in the edge area; a dark channel image with a larger filter size is used in the smooth area to ensure that the restored image does not produce color cast due to excessive enhancement.

[0022] Furthermore, the step S3 includes the following steps:

[0023] S31, convert RGB to grayscale histogram;

[0024] S32, down-sampling the original image to reduce it to 1 / 4 of the original image;

[0025] S33, using interpolation method to obtain the transmittance of the original image.

[0026] Based on the above, the present invention first adds Rayleigh scattering theory to improve the estimation method of atmospheric light on the basis of the original dark channel priori dehazing algorithm, then divides the fog image into sky and non-sky areas, fuses dark channel images of different filter sizes to obtain a fused dark channel, and finally simplifies the transmittance solution process to improve the algorithm's running speed and enhance real-time performance.

[0027] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following is a detailed description in combination with the preferred embodiments and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings of the embodiment are briefly introduced below.

[0029] Figure 1 This is a flow chart of the improved dark channel prior defogging algorithm of the present invention. DETAILED DESCRIPTION

[0030] The following is a detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings, which is a part of this specification and illustrates the principles of the present invention through examples. Other aspects, features and advantages of the present invention will become clear from the detailed description. In the referenced drawings, the same or similar components in different figures are represented by the same reference numerals.

[0031] In order to solve the problems of slow defogging speed, color distortion, incomplete defogging, etc. of traditional defogging methods, the present invention proposes an improved dark channel prior defogging algorithm. The algorithm flow is as follows: Figure 1 The improved dark channel prior defogging algorithm of the present invention comprises the following steps:

[0032] S1: Apply Rayleigh scattering theory to improve estimates of atmospheric light;

[0033] S2: Segment the sky area using a segmentation method based on an active contour model to obtain a sky area and a non-sky area;

[0034] S3: Use filters of different sizes to estimate the dark channel value;

[0035] S4: Simplify the calculation of transmittance and improve the processing speed of the algorithm.

[0036] Atmospheric light value is an important parameter for image defogging. Large calculation amount or large error will lead to a decrease in the defogging performance of the entire image. Traditional defogging algorithms estimate A by selecting the largest pixel. In order to accurately estimate the brightness of the pixel to represent the haze density, the Rayleigh scattering theory is used. Blue light is more easily scattered than other colors of light. The B (blue) channel of the RGB three-color channel of the fogged image is maximized, while the other two channels R (red) and G (green) are minimized. Therefore, an improved atmospheric light calculation method is proposed, which includes both the brightest and darkest pixels in the calculation. The main steps are as follows:

[0037] (1) According to Rayleigh scattering, blue light is scattered the most in RGB images. In order to comprehensively consider the image parts with different depths of field, the pixel sequence with the highest intensity is selected from the blue channel of the input image by column and the average value of the sequence data is taken to obtain A1 to eliminate the halo effect. A1 is expressed as follows:

[0038] A1=avg(max(I B列向 ))

[0039] Where: A1 is the average maximum value of the blue channel of the fog image, I B is the blue channel value of the fog map.

[0040] (2) Calculate the average value of the lowest pixels in the red and green channels as A2, and further eliminate unnecessary texture information after A1.

[0041] A2=avg(min(I G )+min(I R ))

[0042] Where: I G is the green channel value of the fog image, I R is the red channel value of the fog image, and A2 is the average value of the darkest green channel and the darkest red channel.

[0043] (3) Calculate the sum of A1 and A2 to estimate the atmospheric light A

[0044] A=A1+A2.

[0045] The Active Contour Model (ACM) method is a very representative method. ACM combines the constraint information on the image with prior knowledge to define a closed and continuous energy curve. This energy curve continuously approaches the target contour under the combined action of internal and external forces. When the energy of the curve reaches the minimum value, it is considered that the energy curve reaches the target contour, that is, the target edge is obtained. First, define the curve, X(s) = (x(s), y(s)), s∈[0,1]. The energy function of the curve is:

[0046]

[0047] In the formula, E int is the internal energy term of the curve, E ext is the external energy term of the curve, which is specifically expressed as:

[0048]

[0049]

[0050] α(s) and β(s) are weight coefficients; α(s)|X'(s)| 2 is the elastic energy of the curve, which controls the tension of the curve and ensures the continuity of the curve; β(s)|X″(s)| 2 is the rigid energy of the curve, which controls the curvature of the curve and ensures the smoothness of the curve. ▽ represents the gradient operator, I(x, y) represents the image, and G σ represents a Gaussian filter operator with a standard deviation of σ, * represents convolution, E ext It is the energy constraint given by the image.

[0051] The GVF-Snake model introduces the GVF field V(x, y) = [μ(x, y), ν(x, y)] based on the Snake model, and uses the GVF field as a new external force field to replace the Gaussian external force field of the Snake model. The energy functional of the GVF-Snake model is obtained as follows:

[0052]

[0053] μ is a parameter that controls the integrand and is set according to the amount of noise in the image. Generally speaking, the greater the noise, the greater its value. x 、u y 、v x 、v y is the first-order derivative of u and v in the x and y directions. The first term is the smoothing energy term, which can produce a slowly diffusing vector field; the second term is the edge energy term. At the edge of the image, V = ▽f, which makes the gradient force point to the edge, thereby pushing the curve to approach the edge, and far away from the edge, |▽f| tends to 0. The first term plays a major role, making the edge information diffuse to the smooth area, thereby increasing the capture range of the contour curve.

[0054] In the process of using the minimum filter operation to obtain the dark channel, the filter size is a key parameter that affects the final dehazing effect. A larger filter size is more consistent with the dark channel theory because it contains more dark pixels, but at the same time it will cause a halo effect due to the large-size filter; on the contrary, a smaller filter size can effectively suppress the halo effect in the edge area, but it will cause the restored image to be oversaturated.

[0055] In order to solve the above shortcomings, the present invention proposes an improved dark channel estimation method, which fuses dark channel images of different filter sizes. The edge area focuses on using dark channel images with smaller filter sizes to better extract scene structure information in depth discontinuous areas and avoid halo artifacts in edge areas; the smooth area uses dark channel images with larger filter sizes to ensure that the restored image will not produce color cast due to excessive enhancement. Therefore, the key to fusion is to construct a weight map that shows the depth discontinuous area, and the median filtering technology, as a nonlinear filtering operation, can effectively suppress the impulse noise component while retaining the edge information. The specific edge information can be expressed as:

[0056]

[0057] Where Ω1(x) is the filter area centered on x (a 15×15 square matrix), median is the median filter operation, and W is the minimum channel of the input image.

[0058] The image edge information map is used as the weight map, and the dark channel maps of different sizes are fused using pixel-level fusion to obtain the fused dark channel I fdark (x), the expression is:

[0059]

[0060] In the formula, For a dark channel image with a filter size of 3×3, The dark channel image with a filtering size of 15×15.

[0061] The algorithm of the present invention utilizes a dark channel with a smaller filter size to effectively compensate for the edge area of ​​the dark channel. The haze-free image restored by the fused dark channel avoids the oversaturation problem caused by the small-size dark channel and suppresses the halo effect caused by the larger-size dark channel to a certain extent.

[0062] Since the transmittance map of dark channel defogging is more detailed than other algorithms, and it takes up a lot of processing time in the whole algorithm, reducing the processing efficiency of the program, so if the accuracy is appropriately reduced, the defogging effect should not be much different in theory. The method adopted by the present invention does not calculate the transmittance of the original image, but uses the following method to simplify the calculation of the transmittance:

[0063] 1. Convert RGB to grayscale histogram;

[0064] 2. Downsample the original image to 1 / 4 of the original image, store it in V(x), and solve the transmittance of V(x);

[0065] 3. Use interpolation method to obtain the approximate transmittance of the original image.

[0066] Experiments have shown that this method greatly improves the execution speed, and the dehazing effect is basically the same as the original solution. However, if the reduction factor is not very large, such as reducing it to 0.5 times the original size, the time spent on the two scaling may offset the profit gained from calculating the transmittance map of the small image, so the downsampling rate must be reasonably selected.

[0067] The study found that the traditional dark channel prior defogging has a poor effect on the sky part, and the sky often has large areas of texture and block phenomena. The fundamental reason is that the sky part basically does not meet the dark channel defogging prior conditions. To this end, the present invention adopts a method to avoid excessive enhancement of the sky part in the image defogging algorithm: (1) convert the image to grayscale, and use a decolorization algorithm with contrast preservation function to retain more edge information; (2) calculate the gradient information of the grayscale image, and use an edge detection operator to achieve it; (3) denoise and filter the gradient information; (4) distinguish the gradient information according to the set gradient threshold and brightness threshold; (5) perform Gaussian feathering on the distinguished image.

[0068] After obtaining the sky region, the transmittance map of the sky region is uniformly set to a fixed value, but this is not good, and appropriate corrections should be made according to the specific value. In the above operation of He, the obtained sky region does not really belong to the sky, and a certain point does not necessarily belong to the sky completely or not at all. Therefore, the present invention proposes an optimization method as follows:

[0069] J x =(t p ×I x +J x ×(255-I x )) / 255

[0070] Among them J x refers to the point of the dark channel, t p It is similar to the fixed transmittance value mentioned by He, which is specified according to the image processing effect debugging. In the above formula, if I(x)=255, that is, it belongs completely to the sky, then the transmittance of the point is a fixed value. If I(x)=0, that is, it does not belong to the sky at all, the value of the calculation formula remains unchanged and does not affect normal defogging.

[0071] In addition, regarding the calculation of the atmospheric light value A, He proposed to use the average value of the pixels in the sky area as A, which is also very reasonable. However, in actual processing, for some images without sky at all, the detected sky area may be very small (obviously a false detection, but the program does not know), and it is unreasonable to use this as the atmospheric light value. Therefore, the processing method of the present invention is to first calculate the A value of the sky area, and then detect the proportion of sky pixels in the entire image.

[0072] The above is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and changes can be made without departing from the principle of the present invention, and these improvements and changes are also regarded as the protection scope of the present invention.

Claims

1. An improved dark channel prior dehazing algorithm, characterized in that: The following steps are involved: S1: Apply Rayleigh scattering theory to improve estimates of atmospheric light; S2: Segment the sky area using a segmentation method based on an active contour model to obtain a sky area and a non-sky area; S3: Use filters of different sizes to estimate the dark channel value; S4: Simplify the calculation of transmittance and improve the processing speed of the algorithm; In step S2, dark channel images of different filter sizes are fused, and the edge area focuses on using a dark channel image of a smaller filter size, so as to better extract the scene structure information of the depth discontinuous area and avoid halo artifacts in the edge area; a dark channel image of a larger filter size is used in the smooth area to ensure that the restored image will not produce color cast due to excessive enhancement; the specific edge information is expressed as: Where Ω1(x) is the filter area centered on x, median is the median filter operation, and W is the minimum channel of the input image; The image edge information map is used as the weight map, and the dark channel maps of different sizes are fused using pixel-level fusion to obtain the fused dark channel I fdark (x), the expression is: In the formula, For a dark channel image with a filter size of 3×3, The dark channel image with a filter size of 15×15; The step S3 comprises the following steps: S31, convert RGB to grayscale histogram; S32, down-sampling the original image to reduce it to 1 / 4 of the original image; S33, using an interpolation method to obtain the transmittance of the original image; The method used to avoid excessive enhancement of the sky part in the image dehazing algorithm is: (1) Convert the image to grayscale and use a decolorization algorithm with contrast preservation function to retain more edge information; (2) Obtain the gradient information of the grayscale image using an edge detection operator; (3) De-noising and filtering the gradient information; (4) Differentiate the gradient information according to the set gradient threshold and brightness threshold; (5) performing Gaussian feathering processing on the differentiated image; After obtaining the sky area, the transmittance map of the sky area is uniformly set to a fixed value, and an optimization method is proposed as follows: J x =(t p ×I x +J x ×(255-I x )) / 255 Among them J x refers to the point of the dark channel, t p It is a fixed transmittance value. In the above formula, if I(x) = 255, that is, it belongs entirely to the sky, then the transmittance of the point is a fixed value. If I(x) = 0, that is, it does not belong to the sky at all, the value of the calculation formula remains unchanged and does not affect normal defogging.

2. The improved dark channel prior defogging algorithm as claimed in claim 1, characterized in that: The step S1 comprises the following steps: S11, according to Rayleigh scattering, select the pixel sequence with the highest intensity from the blue channel of the input image by column and take the average value of the sequence data to obtain A1 to eliminate the halo effect. A1 is expressed as follows: A1=avg(max(I B )) Where: A1 is the average maximum value of the blue channel of the fog image, I B is the blue channel value of the fog image; S12, calculate the average value of the lowest pixel in the red and green channels A2, and further eliminate unnecessary texture information after A1: A2=avg(min(I G )+min(I R )) Where: I G is the green channel value of the fog image, I R is the red channel value of the fog image, A2 is the average of the darkest green channel and the darkest red channel; S13. Calculate the sum of A1 and A2 to estimate the atmospheric light A: A=A1+A2.