Image defogging method and device, storage medium and electronic equipment

Abstract

The invention discloses an image defogging method and device, a storage medium and electronic equipment, and relates to the technical field of image processing. The image defogging method comprises the following steps: acquiring an image to be defogged; based on a dark channel prior algorithm, respectively calculating a dark channel and a bright channel of the to-be-defogged image to obtain a dark channel graph and a bright channel graph; estimating an atmospheric light value based on the dark channel map and the bright channel map to obtain an atmospheric light value corresponding to each pixel point; obtaining a perspective rate based on an atmospheric scattering model and the atmospheric light value corresponding to each pixel point; and based on the atmospheric light value corresponding to each pixel point and the perspective rate, performing fogless image restoration on the to-be-defogged image to obtain a defogged image. By combining the dark channel and the bright channel, the atmospheric light value of each point is obtained, the situation that the perspective rate estimation of a local area is inaccurate due to the fact that a global atmospheric light is used is avoided, the perspective rate estimation accuracy is improved, and then the image defogging effect is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image defogging method, an image defogging device, a machine-readable storage medium and an electronic device. Background Art

[0002] At present, at construction sites, various operating machines will collect on-site images through cameras and then operate according to the on-site images. However, haze and dust are common phenomena at construction sites. These suspended particles will scatter and absorb light, which will not only reduce the visibility of the human eye, but also cause the loss of image details, reduced contrast, and color distortion in the images collected by the camera. For example, when using an excavator for operation, haze and dust will cause the loss of image details, reduced contrast, and color distortion in the images collected by the excavator camera. The collected images will interfere with the operator's perception of the environment around the excavator, reducing the safety and efficiency of the operation. Therefore, the application of image defogging technology is of great significance to construction machinery such as excavators.

[0003] The dark channel algorithm is an image dehazing technology widely used in the field of computer vision. However, the existing dark channel algorithm directly uses the atmospheric light value estimated by the dark channel, resulting in inaccurate perspective estimation in local areas. If the sky area in the image is too bright or there are bright objects, it will cause color spots or color aberration in the dehazed image, resulting in insufficient image dehazing effect. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image defogging method, an image defogging device, a machine-readable storage medium and an electronic device to solve the problem in the prior art of using a global atmospheric light, resulting in inaccurate estimation of the perspective rate of the local area and insufficient image defogging effect.

[0005] In order to achieve the above-mentioned object, the first aspect of the present application provides an image defogging method, comprising:

[0006] Obtain the image to be dehazed;

[0007] Based on a dark channel priori algorithm, a dark channel and a bright channel of the image to be dehazed are calculated respectively to obtain a dark channel map and a bright channel map;

[0008] Based on the dark channel image and the bright channel image, the atmospheric light value is estimated to obtain the atmospheric light value corresponding to each pixel point;

[0009] Obtaining a perspective rate based on an atmospheric scattering model and an atmospheric light value corresponding to each pixel point;

[0010] Based on the atmospheric light value corresponding to each pixel point and the perspective rate, the image to be defogged is restored to a non-fogged image to obtain a defogged image.

[0011] In the embodiment of the present application, the atmospheric light value is estimated based on the dark channel map and the bright channel map to obtain the atmospheric light value corresponding to each pixel point, including:

[0012] According to the brightness of each pixel point in the dark channel image, the dark channel atmospheric light value is calculated;

[0013] Based on the pixel value of each pixel in the bright channel image, obtain the bright channel atmospheric light value of each pixel;

[0014] Based on the dark channel atmospheric light value and the bright channel atmospheric light value of each pixel point, the atmospheric light value corresponding to each pixel point is obtained.

[0015] In the embodiment of the present application, the dark channel atmospheric light value is calculated according to the brightness of each pixel in the dark channel image, including:

[0016] Determine a plurality of pixels to be calculated according to the brightness of each pixel in the dark channel image;

[0017] The average value of the pixel values ​​of the plurality of pixels to be calculated is obtained to obtain the dark channel atmospheric light value.

[0018] In the embodiment of the present application, obtaining the atmospheric light value corresponding to each pixel point based on the atmospheric light value of the dark channel and the atmospheric light value of the bright channel of each pixel point includes:

[0019] Substitute the dark channel atmospheric light value and the bright channel atmospheric light value of each pixel into a preset atmospheric light value calculation formula to obtain the atmospheric light value corresponding to each pixel. The preset atmospheric light value calculation formula is:

[0020] A(X)=αA light (X)+βA0

[0021] Among them, α is the bright channel weight adjustment factor, β is the dark channel weight adjustment factor, α+β<1.0, A0 is the atmospheric light value of the dark channel, A light (X) is the atmospheric light value of the bright channel of each pixel, and A(X) is the atmospheric light value corresponding to each pixel.

[0022] In the embodiment of the present application, the bright channel image is represented as:

[0023]

[0024] Among them, U(X) is the local neighborhood centered on X in the image to be dehazed, c represents the three color channels of R, G, and B, and I c (Y) is the pixel value of a certain pixel in a certain color channel in the image to be dehazed, Ilight (X) is the bright channel image of the image to be dehazed, and Y is any pixel point in the local neighborhood centered on X in the image to be dehazed.

[0025] In the embodiment of the present application, obtaining the perspective based on the atmospheric scattering model and the atmospheric light value corresponding to each pixel point includes:

[0026] Calculating a perspective estimation value based on an atmospheric scattering model and the atmospheric light value corresponding to each pixel point;

[0027] The perspective estimation value is compensated to obtain the perspective.

[0028] In an embodiment of the present application, the step of obtaining the image to be defogged includes:

[0029] Get the initial image;

[0030] The Retinex algorithm is used to perform image defogging processing on the initial image, and / or image enhancement is performed on the initial image to obtain an image to be defogged.

[0031] A second aspect of the present application provides an image defogging device, comprising:

[0032] An acquisition module, used for acquiring the image to be dehazed;

[0033] A first calculation module is used to calculate the dark channel and the bright channel of the image to be defogged based on a dark channel priori algorithm, and obtain a dark channel map and a bright channel map;

[0034] A second calculation module is used to estimate the atmospheric light value based on the dark channel image and the bright channel image to obtain the atmospheric light value corresponding to each pixel point;

[0035] A third calculation module, used for obtaining a perspective rate based on an atmospheric scattering model and an atmospheric light value corresponding to each pixel point;

[0036] The restoration module is used to restore the image to be defogged based on the atmospheric light value corresponding to each pixel point and the perspective rate to obtain a defogged image.

[0037] A third aspect of the present application provides an electronic device, the electronic device comprising:

[0038] at least one processor;

[0039] a memory connected to the at least one processor;

[0040] The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the above-mentioned image defogging method by executing the instructions stored in the memory.

[0041] A fourth aspect of the present application provides a machine-readable storage medium having instructions stored thereon, which, when executed by a processor, configure the processor to execute the above-mentioned image defogging method.

[0042] Through the above technical scheme, the image to be defogged is obtained; based on the dark channel prior algorithm, the dark channel and the bright channel of the image to be defogged are calculated respectively to obtain a dark channel map and a bright channel map; based on the dark channel map and the bright channel map, the atmospheric light value is estimated to obtain the atmospheric light value corresponding to each pixel; based on the atmospheric scattering model and the atmospheric light value corresponding to each pixel, the perspective is obtained; based on the atmospheric light value corresponding to each pixel and the perspective, the image to be defogged is restored to a fog-free image to obtain a defogged image. By combining the dark channel and the bright channel, the atmospheric light value at each point is obtained, avoiding the use of a global atmospheric light, which causes inaccurate perspective estimation in the local area, improving the accuracy of perspective estimation, and thus improving the image defogging effect.

[0043] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0045] Figure 1 A schematic diagram of a process of an image defogging method according to an embodiment of the present application is schematically shown;

[0046] Figure 2 A schematic diagram of a specific image defogging step according to an embodiment of the present application is shown schematically;

[0047] Figure 3 Schematically showing a comparison diagram of an image before and after defogging according to an embodiment of the present application;

[0048] Figure 4 The structure diagram of an image defogging device according to an embodiment of the present application is schematically shown;

[0049] Figure 5 The internal structure diagram of a computer device according to an embodiment of the present application is schematically shown.

[0050] Description of Reference Numerals

[0051] 410 - acquisition module; 420 - first calculation module; 430 - second calculation module; 440 - third calculation module; 450 - recovery module; A01 - processor; A02 - network interface; A03 - internal memory; A04 - display screen; A05 - input device; A06 - non-volatile storage medium; B01 - operating system; B02 - computer program. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0053] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0054] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0055] Figure 1 The following schematically shows a flow chart of an image defogging method according to an embodiment of the present application. Figure 1 As shown, the embodiment of the present application provides an image defogging method, which may include the following steps:

[0056] Step 210: Acquire the image to be defogged;

[0057] In this embodiment, the above-mentioned image to be defogged can be an image captured by a camera at the scene, or an image captured by a camera at the scene after preprocessing. The above-mentioned preprocessing can be image enhancement, image denoising and other processing, which can be determined according to actual needs.

[0058] Step 220: Based on a dark channel priori algorithm, respectively calculate the dark channel and the bright channel of the image to be dehazed to obtain a dark channel map and a bright channel map;

[0059] In this embodiment, the dark channel prior algorithm is based on the observation and statistics of a large number of fog-free images, and the dark channel prior law is discovered, that is, in the local area of ​​most fog-free images, the intensity value of at least one color channel is very low and close to 0. When calculating the dark channel map and the bright channel map, the following atmospheric scattering model can be used to describe the image with fog and dust, that is, the image to be defogged can be expressed as:

[0060] I(X)=J(X)t(X)+A(X)(1-t(X))

[0061] Among them, I(X) represents the image to be defogged, J(X) represents the image to be restored without fog and dust, A(X) is the atmospheric light component, and t(X) is the perspective.

[0062] According to the idea of ​​the dark channel prior algorithm, in most non-sky local areas, some pixels will always have at least one color channel with a very low value. For an image J(X) without fog and dust, the mathematical definition of its dark channel can be expressed as:

[0063]

[0064] Among them, U(X) is the local neighborhood centered on X in the image to be dehazed, c represents the three color channels of R, G, and B, and J c (Y) is the pixel value of a certain pixel in a certain color channel in an image without fog and dust, J dark (X) is the dark channel image, and Y is any pixel in the local neighborhood centered on X in the image to be dehazed.

[0065] The dark channel prior algorithm points out that the dark channel of the image J(X) without fog and dust in the non-sky area tends to 0. Correspondingly, similar to the dark channel prior, the basic idea of ​​the bright channel prior is that one of the three color channels of some pixels in the defogging image will always have a larger intensity. Therefore, in some embodiments, the bright channel map can be expressed as:

[0066]

[0067] Among them, U(X) is the local neighborhood centered on X in the image to be dehazed, c represents the three color channels of R, G, and B, and I c (Y) is the pixel value of a certain pixel in a certain color channel in the image to be dehazed, I light (X) is the bright channel image of the image to be dehazed.

[0068] Step 230: estimating the atmospheric light value based on the dark channel image and the bright channel image to obtain the atmospheric light value corresponding to each pixel point;

[0069] In this embodiment, the estimation may be performed by combining the pixel value of each pixel point in the dark channel image and the bright channel image.

[0070] In some embodiments, the estimating the atmospheric light value based on the dark channel map and the bright channel map to obtain the atmospheric light value corresponding to each pixel point includes the following steps:

[0071] First, the dark channel atmospheric light value is calculated according to the brightness of each pixel in the dark channel image;

[0072] Specifically, the dark channel atmospheric light value is calculated according to the brightness of each pixel in the dark channel image, comprising the following steps:

[0073] The first step is to determine a plurality of pixels to be calculated according to the brightness of each pixel in the dark channel image;

[0074] The second step is to calculate the average value of the pixel values ​​of the plurality of pixels to be calculated to obtain the dark channel atmospheric light value.

[0075] In this embodiment, when calculating the atmospheric light value of the dark channel, the brightest top 0.1% of the pixels in the dark channel image can be taken; then the pixels at the corresponding positions are found in the image to be dehazed I(X), and the pixel values ​​of these pixels are obtained, and then the average value of the pixel values ​​of these pixels is calculated, and the average value is used as the atmospheric light value estimated for the dark channel, that is, the atmospheric light value of the dark channel, which can be denoted as A0.

[0076] By calculating the average of the pixel values ​​of the brightest 0.1% of pixels in the dark channel image, a more accurate dark channel atmospheric light value can be obtained.

[0077] Then, based on the pixel value of each pixel in the bright channel image, a bright channel atmospheric light value of each pixel is obtained;

[0078] In this embodiment, according to the bright channel prior theory, the bright channel intensity at the pixel point X approaches the atmospheric light intensity of the fog-free image, that is,

[0079] I light (X) → A light (X)

[0080] Among them, A light (X) is the atmospheric light value of the bright channel of each pixel, I light(X) is the pixel value of each pixel. Therefore, the bright channel atmospheric light value of each pixel is the pixel value of each pixel in the bright channel image.

[0081] Finally, based on the dark channel atmospheric light value and the bright channel atmospheric light value of each pixel point, the atmospheric light value corresponding to each pixel point is obtained.

[0082] In this embodiment, the atmospheric light value corresponding to each pixel point can be obtained by combining the atmospheric light value of the dark channel and the atmospheric light value of the bright channel.

[0083] Specifically, obtaining the atmospheric light value corresponding to each pixel point based on the atmospheric light value of the dark channel and the atmospheric light value of the bright channel of each pixel point includes:

[0084] Substitute the dark channel atmospheric light value and the bright channel atmospheric light value of each pixel into a preset atmospheric light value calculation formula to obtain the atmospheric light value corresponding to each pixel. The preset atmospheric light value calculation formula is:

[0085] A(X)=αA light (X)+βA0

[0086] Among them, α is the bright channel weight adjustment factor, β is the dark channel weight adjustment factor, α+β<1.0, A0 is the atmospheric light value of the dark channel, A light (X) is the atmospheric light value of the bright channel of each pixel, and A(X) is the atmospheric light value corresponding to each pixel.

[0087] In this embodiment, since the pixel value of the bright channel is close to the atmospheric light value of the fog-free image, the α weight can be set to be larger, and the atmospheric light value weight of the dark channel can be set to be smaller, and the β weight can be set to be smaller, and α+β<1.0 is satisfied. For example, the default α and β can be set to α=0.7 and β=0.2. The specific adjustment can be made according to the actual situation.

[0088] By calculating the atmospheric light value corresponding to the dark channel and the atmospheric light value corresponding to the bright channel of each pixel respectively, and then combining the atmospheric light value corresponding to the dark channel and the atmospheric light value corresponding to the bright channel to obtain the atmospheric light value, the atmospheric light value at each pixel can be obtained, avoiding the use of a global atmospheric light value. By setting different weight adjustment factors for the bright channel and the dark channel respectively, and then weighted summing the atmospheric light value of the dark channel and the atmospheric light value of the bright channel of each pixel, the atmospheric light value corresponding to each pixel obtained can be made more accurate.

[0089] Step 240: Obtaining perspective based on the atmospheric scattering model and the atmospheric light value corresponding to each pixel point;

[0090] In this embodiment, after obtaining the atmospheric light value corresponding to each pixel point, the perspective rate can be obtained by estimating the perspective rate according to the dark channel prior principle and the atmospheric light value; or the perspective rate can be estimated according to the dark channel prior principle and the atmospheric light value, and then the estimated perspective rate is further optimized to obtain the perspective rate. The perspective rate is used to maintain a certain sense of fog in the depth of field, making the image look more real and natural.

[0091] In some embodiments, since the dark channel prior algorithm may cause the image to appear too bright or produce a halo effect, it is necessary to compensate the estimated perspective. That is, the perspective is obtained based on the atmospheric scattering model and the atmospheric light value corresponding to each pixel point, including the following steps:

[0092] Firstly, based on the atmospheric scattering model and the atmospheric light value corresponding to each pixel point, a perspective estimation value is calculated;

[0093] In this embodiment, the perspective estimation value is obtained by estimating the perspective using the atmospheric scattering model and the atmospheric light value corresponding to each pixel. Assuming that the atmospheric light value corresponding to each pixel has been obtained and represented as A(X), the atmospheric scattering model can be converted into the following equation:

[0094]

[0095] Among them, c represents the three color channels of R, G, and B, and J c (X) The pixel value of a pixel in a certain color channel in an image without fog and dust, I c (X) is the pixel value of a certain pixel in a certain color channel in the image to be dehazed, and t(X) is the estimated value of the perspective.

[0096] Assuming that the transmittance in each filter window is constant, recorded as t(X), and then calculating the dark channel on both sides of the equation of the above atmospheric scattering model, we can get:

[0097]

[0098] According to the dark channel prior theory, and knowing that A(X) is a positive number, we can get:

[0099]

[0100] Therefore, the perspective can be obtained as:

[0101]

[0102] If the foggy part is completely removed, the image will be unrealistic because the aerial perspective effect is lost. Therefore, a constant ω (0<ω<1) is introduced in the perspective estimation process to retain a small amount of fog in the image. That is:

[0103]

[0104] Here, ω is usually 0.95. This is the estimated value of the perspective.

[0105] Then, the perspective estimation value is compensated to obtain the perspective.

[0106] In this embodiment, the compensation can be performed by using methods such as mean filtering and guided filtering, wherein guided filtering can reduce noise, smooth details and remove fog. The process is as follows:

[0107] First, the guide map I and the perspective rate t to be processed are input; that is, the image to be dehazed and the perspective rate estimation value are input.

[0108] Then, calculate the mean I of the guide image I within the filter window mean and variance Var(I), mean t of the perspective rate t to be processed mean , the product of the guide map I and the perspective rate t I*t, its mean can be expressed as (I*t) mean .

[0109] Then, calculate the coefficients a, b, where:

[0110]

[0111] b=t mean -aI mean

[0112] Where ε is a very small positive number.

[0113] Then, calculate the mean of coefficients a and b, expressed as a and b respectively. mean , b mean .

[0114] Finally, the filtering result is obtained according to the result, that is:

[0115] t I =a medn *I+b mean

[0116] Among them, t I To calculate the output image, that is, the optimized transmittance map, traverse all windows on the entire image and repeat the above steps to obtain the final transmittance compensation result, which is the perspective estimation value.

[0117] It should be noted that the above-mentioned methods of using mean filtering, guided filtering, etc. to compensate the perspective estimation value belong to the existing technology and will not be described in detail here.

[0118] Step 250: Based on the atmospheric light value corresponding to each pixel point and the perspective rate, perform haze-free image restoration on the image to be defogged to obtain a defogged image.

[0119] In this embodiment, since the pixel value of the restored image will be too large when the perspective rate t is too small, it is necessary to set a lower limit t0 (generally 0.1) for the perspective rate. When t<t0, t=t0. Substituting the obtained perspective rate and the atmospheric light value corresponding to each pixel into the expression of the restored image, the defogging image can be obtained. The expression of the above restored image is:

[0120]

[0121] Among them, t I (X) is the estimated value of perspective.

[0122] In the above implementation process, the image to be defogged is obtained; based on the dark channel prior algorithm, the dark channel and the bright channel of the image to be defogged are calculated respectively to obtain a dark channel map and a bright channel map; based on the dark channel map and the bright channel map, the atmospheric light value is estimated to obtain the atmospheric light value corresponding to each pixel; based on the atmospheric scattering model and the atmospheric light value corresponding to each pixel, the perspective is obtained; based on the atmospheric light value corresponding to each pixel and the perspective, the image to be defogged is restored to a fog-free image to obtain a defogged image. By combining the dark channel and the bright channel, the atmospheric light value at each point is obtained, avoiding the use of a global atmospheric light, which causes inaccurate perspective estimation in the local area, and improving the accuracy of perspective estimation, thereby improving the image defogging effect.

[0123] The above steps 210-240 implement an improved dark channel algorithm. However, considering that the dark channel algorithm will make the restored image darker overall, and overexposure is prone to occur in the sky area, the Retinex retinal algorithm has a certain improvement effect on the image being too dark and overexposed. Therefore, the improved dark channel algorithm and the Retinex retinal algorithm can be combined for defogging. Specifically, the Retinex retinal algorithm can be used to defog the original image to obtain an initial image, and then the above steps 210-240 can be used to implement defogging based on the dark channel algorithm; or the original image can be used as the image to be defogged, and the above steps 210-240 can be used to implement defogging based on the dark channel algorithm, and then the defogged image can be further defogged using the Retinex retinal algorithm to obtain the final defogged image. By combining the improved dark channel algorithm and the Retinex retinal algorithm, the image detail information can be retained, the interference of uneven external illumination can be reduced, the color distortion of the restored image can be avoided, and the image defogging effect can be further improved.

[0124] In some embodiments, obtaining the image to be defogged includes:

[0125] First, obtain the initial image;

[0126] In this embodiment, the initial image may be an image captured by a camera at the scene, or may be an image captured by a camera at the scene after being processed by denoising, image enhancement, etc.

[0127] Then, the Retinex algorithm is used to perform image defogging processing on the initial image to obtain an image to be defogged.

[0128] In this embodiment, the Retinex algorithm has a certain improvement effect on the image being too dark or overexposed. The Retinex algorithm considers that the image S(x, y) is composed of the illumination image L(x, y) and the reflection image R(x, y). The image S(x, y) here is the initial image, that is:

[0129] S(x, y) = L(x, y) * R(x, y)

[0130] Where: S(x, y) represents the image signal being observed or received by the camera, x, y represent the size of the image signal; L(x, y) represents the illumination component of the ambient light; R(x, y) represents the reflection component of the target object carrying the image detail information.

[0131] Take the logarithm of both sides of the above equation:

[0132]

[0133] Because R(x, y) represents the reflective properties of the object, that is, the intrinsic properties of the image, it should be retained to the greatest extent; L(x, y) represents the incident light image, which determines the dynamic range that the image pixels can reach and should be removed as much as possible.

[0134] Usually L(x, y) is approximately represented by the convolution of S(x, y) and a Gaussian kernel, which is equivalent to Gaussian filtering of the image, which will cause edge blurring and loss of high-frequency details. Therefore, bilateral filtering is considered. Bilateral filtering introduces two factors, spatial proximity and pixel value similarity, so that pixels near the edge will not be affected by pixels far away, thereby better preserving edge information.

[0135] Bilateral filtering consists of two Gaussian kernels, a spatial domain kernel and a range kernel. The closer the pixels are, the greater the spatial domain weight; the closer the pixel values ​​are, the greater the range weight.

[0136] Spatial domain kernel template weight:

[0137]

[0138] Among them, w d Represents the distance from a point (i, j) in the neighborhood to the center point (k, l). Represents the variance of the Gaussian function.

[0139] Range kernel template weight:

[0140]

[0141] S(i, j) represents the pixel value of image S at point (i, j), S(k, l) represents the pixel value of image S at the center point (k, l) of the template, Represents the variance of the Gaussian function.

[0142] Multiplying them together gives the template weights for bilateral filtering:

[0143]

[0144] After filtering, the value of L(x, y) at point (i, j) is L(i, j):

[0145]

[0146] It should be noted that the bilateral filtering can be performed by filtering the image with multiple bilateral filters respectively and then weighting. The above image filtering can also be performed by using a single-scale Gaussian filter or a multi-scale Gaussian filter with color restoration, which is not limited in this embodiment.

[0147] Substitute the obtained L(x, y) into the expression of r(x, y) to obtain r(x, y), and then normalize r(x, y) to a pixel value in the range of [0, 255], that is:

[0148]

[0149] Among them, I(i, j) is the value of I(x, y) at point (i, j), r(i, j) is the value of r(x, y) at point (i, j), min is the minimum value in r(x, y), and max is the maximum value in r(x, y). The obtained I(i, j) is the value of each pixel in the image after defogging, so the image to be defogged is obtained.

[0150] In the above implementation process, the image restored by the dark channel algorithm is generally dark and overexposure is prone to occur in the sky area, while the Retinex algorithm has a certain improvement effect on the image being too dark and overexposed. Therefore, by combining the Retinex algorithm and the dark channel algorithm to perform image dust removal and defogging, the image detail information can be retained, the interference of uneven external lighting can be reduced, the color distortion of the restored image can be avoided, and the image defogging effect can be further improved.

[0151] In some embodiments, the step of obtaining the image to be defogged comprises the following steps:

[0152] First, obtain the initial image;

[0153] Then, the initial image is enhanced to obtain the image to be defogged.

[0154] In this embodiment, the image enhancement may be grayscale transformation, histogram equalization, image smoothing and sharpening of the initial image. For example, the image enhancement may be performed using a contrast-limited adaptive histogram equalization algorithm (CLAHE). The steps are mainly as follows: the initial image is divided into a number of small blocks, and then each small block is subjected to histogram equalization. A threshold may be pre-set to crop the histogram to prevent contrast from being excessively enhanced and reduce noise amplification. The image enhancement may be achieved using existing technology, which will not be described in detail here.

[0155] It should be noted that the above-mentioned image enhancement process may also be placed after defogging, that is, image enhancement processing is performed on the defogging image to obtain the final defogging image.

[0156] In the above implementation process, image enhancement can enhance the contrast of the initial image, increase the brightness difference between different areas in the image, make the initial image more vivid, and also remove noise, reduce random noise in the initial image, improve the signal-to-noise ratio of the initial image, make the initial image cleaner, and further improve the image defogging effect.

[0157] It should be noted that the above-mentioned Retinex algorithm and image enhancement can exist at the same time. For example, image enhancement can be performed first, and then the Retinex algorithm is used to perform image dehazing on the initial image after image enhancement, so that the dehazed image can retain image detail information, reduce interference from uneven external lighting, avoid color distortion of the restored image, and improve image contrast.

[0158] The following is a specific example to illustrate the solution. Figure 2 , Figure 2 The schematic diagram of the specific image defogging steps according to the embodiment of the present application is schematically shown. First, after obtaining the original image, the original image is first subjected to limited contrast adaptive histogram equalization to enhance the original image, and then the enhanced original image is defogged using the Retinex retina algorithm, and then the dark channel prior algorithm implemented in the above steps 210 to 240 is used to defog, and finally the final defogged image is obtained. Please refer to Figure 3 , Figure 3 A comparison diagram of an image before and after defogging according to an embodiment of the present application is schematically shown. Figure 3 The left image is the original image, and the right image is the dehazed image. Figure 3 It can be seen that the above scheme has a better defogging effect.

[0159] Figure 1 FIG. 1 is a flow chart of the image defogging method in the embodiment. It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0160] Please see Figure 4 , Figure 4 The structure diagram of an image defogging device according to an embodiment of the present application is schematically shown. This embodiment provides an image defogging device, including an acquisition module 410, a first calculation module 420, a second calculation module 430, a third calculation module 440 and a recovery module 450, wherein:

[0161] An acquisition module 410 is used to acquire an image to be defogged;

[0162] A first calculation module 420 is used to calculate the dark channel and the bright channel of the image to be defogged based on a dark channel priori algorithm, and obtain a dark channel map and a bright channel map;

[0163] A second calculation module 430 is used to estimate the atmospheric light value based on the dark channel image and the bright channel image to obtain the atmospheric light value corresponding to each pixel point;

[0164] A third calculation module 440 is used to obtain a perspective rate based on an atmospheric scattering model and an atmospheric light value corresponding to each pixel point;

[0165] The restoration module 450 is used to restore the image to be defogged based on the atmospheric light value corresponding to each pixel point and the perspective rate to obtain a defogged image.

[0166] The image defogging method device includes a processor and a memory. The above-mentioned acquisition module 410, the first calculation module 420, the second calculation module 430, the third calculation module 440 and the recovery module 450 are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0167] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and image defogging can be achieved by adjusting kernel parameters.

[0168] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0169] An embodiment of the present invention provides a machine-readable storage medium on which a program is stored. When the program is executed by a processor, the image defogging method is implemented.

[0170] An embodiment of the present invention provides a processor, which is used to run a program, wherein the image defogging method is executed when the program is run.

[0171] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05 and a memory (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A06. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor A01, an image defogging method is implemented. The display screen A04 of the computer device can be a liquid crystal display or an electronic ink display, and the input device A05 of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0172] Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0173] In one embodiment, the image defogging method and device provided in the present application can be implemented in the form of a computer program. Figure 5 The computer device shown in the figure is run. The memory of the computer device can store various program modules constituting the image defogging method device, such as: Figure 4 The acquisition module 410, the first calculation module 420, the second calculation module 430, the third calculation module 440 and the restoration module 450 are shown. The computer program composed of various program modules enables the processor to execute the steps of the image defogging method of each embodiment of the present application described in this specification.

[0174] Figure 5 The computer device shown can be Figure 4 The acquisition module 410 in the image defogging method device shown executes step 210. The computer device may execute step 220 through the first calculation module 420. The computer device may execute step 230 through the second calculation module 430. The computer device may execute step 240 through the third calculation module 440. The computer device may execute step 250 through the recovery module 450.

[0175] An embodiment of the present application provides an electronic device, the electronic device comprising: at least one processor; a memory connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, the at least one processor implements the above-mentioned image defogging method by executing the instructions stored in the memory, and the processor implements the following steps when executing the instructions:

[0176] Obtain the image to be dehazed;

[0177] Based on a dark channel priori algorithm, a dark channel and a bright channel of the image to be dehazed are calculated respectively to obtain a dark channel map and a bright channel map;

[0178] Based on the dark channel image and the bright channel image, the atmospheric light value is estimated to obtain the atmospheric light value corresponding to each pixel point;

[0179] Obtaining a perspective rate based on an atmospheric scattering model and an atmospheric light value corresponding to each pixel point;

[0180] Based on the atmospheric light value corresponding to each pixel point and the perspective rate, the image to be defogged is restored to a non-fogged image to obtain a defogged image.

[0181] In one embodiment, estimating the atmospheric light value based on the dark channel map and the bright channel map to obtain the atmospheric light value corresponding to each pixel point includes:

[0182] According to the brightness of each pixel point in the dark channel image, the dark channel atmospheric light value is calculated;

[0183] Based on the pixel value of each pixel in the bright channel image, obtain the bright channel atmospheric light value of each pixel;

[0184] Based on the dark channel atmospheric light value and the bright channel atmospheric light value of each pixel point, the atmospheric light value corresponding to each pixel point is obtained.

[0185] In one embodiment, calculating the dark channel atmospheric light value according to the brightness of each pixel in the dark channel image includes:

[0186] Determine a plurality of pixels to be calculated according to the brightness of each pixel in the dark channel image;

[0187] The average value of the pixel values ​​of the plurality of pixels to be calculated is obtained to obtain the dark channel atmospheric light value.

[0188] In one embodiment, obtaining the atmospheric light value corresponding to each pixel point based on the atmospheric light value of the dark channel and the atmospheric light value of the bright channel of each pixel point includes:

[0189] Substitute the dark channel atmospheric light value and the bright channel atmospheric light value of each pixel into a preset atmospheric light value calculation formula to obtain the atmospheric light value corresponding to each pixel. The preset atmospheric light value calculation formula is:

[0190] A(X)=αA light (X)+βA0

[0191] Among them, α is the bright channel weight adjustment factor, β is the dark channel weight adjustment factor, α+β<1.0, A0 is the atmospheric light value of the dark channel, A light (X) is the atmospheric light value of the bright channel of each pixel, and A(X) is the atmospheric light value corresponding to each pixel.

[0192] In one embodiment, the bright channel map is represented as:

[0193]

[0194] Among them, U(X) is the local neighborhood centered on X in the image to be dehazed, c represents the three color channels of R, G, and B, and I c (Y) is the pixel value of a certain pixel in a certain color channel in the image to be dehazed, I light(X) is the bright channel image of the image to be dehazed, and Y is any pixel point in the local neighborhood centered on X in the image to be dehazed.

[0195] In one embodiment, obtaining the perspective based on the atmospheric scattering model and the atmospheric light value corresponding to each pixel point includes:

[0196] Calculating a perspective estimation value based on an atmospheric scattering model and the atmospheric light value corresponding to each pixel point;

[0197] The perspective estimation value is compensated to obtain the perspective.

[0198] In one embodiment, obtaining the image to be defogged includes:

[0199] Get the initial image;

[0200] The Retinex algorithm is used to perform image defogging processing on the initial image, and / or image enhancement is performed on the initial image to obtain an image to be defogged.

[0201] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0202] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0203] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0205] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0206] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0207] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0208] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0209] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An image defogging method, characterized in that: include: Obtain the image to be dehazed; Based on a dark channel priori algorithm, a dark channel and a bright channel of the image to be dehazed are calculated respectively to obtain a dark channel map and a bright channel map; Based on the dark channel image and the bright channel image, the atmospheric light value is estimated to obtain the atmospheric light value corresponding to each pixel point; Obtaining a perspective rate based on an atmospheric scattering model and an atmospheric light value corresponding to each pixel point; Based on the atmospheric light value corresponding to each pixel point and the perspective rate, the image to be defogged is restored to a non-fogged image to obtain a defogged image.

2. The method according to claim 1, characterized in that The estimating the atmospheric light value based on the dark channel image and the bright channel image to obtain the atmospheric light value corresponding to each pixel point includes: According to the brightness of each pixel point in the dark channel image, the dark channel atmospheric light value is calculated; Based on the pixel value of each pixel in the bright channel image, obtain the bright channel atmospheric light value of each pixel; Based on the dark channel atmospheric light value and the bright channel atmospheric light value of each pixel point, the atmospheric light value corresponding to each pixel point is obtained.

3. The method according to claim 2, characterized in that The step of calculating the dark channel atmospheric light value according to the brightness of each pixel in the dark channel image comprises: Determine a plurality of pixels to be calculated according to the brightness of each pixel in the dark channel image; The average value of the pixel values ​​of the plurality of pixels to be calculated is obtained to obtain the dark channel atmospheric light value.

4. The method according to claim 2, characterized in that: The obtaining, based on the dark channel atmospheric light value and the bright channel atmospheric light value of each pixel point, the atmospheric light value corresponding to each pixel point comprises: Substitute the dark channel atmospheric light value and the bright channel atmospheric light value of each pixel into a preset atmospheric light value calculation formula to obtain the atmospheric light value corresponding to each pixel. The preset atmospheric light value calculation formula is: A(X)=αA light (X)+βA0 Among them, α is the bright channel weight adjustment factor, β is the dark channel weight adjustment factor, α+β<1.0, A0 is the atmospheric light value of the dark channel, A light (X) is the atmospheric light value of the bright channel of each pixel, and A(X) is the atmospheric light value corresponding to each pixel.

5. The method according to claim 1, characterized in that The bright channel image is represented as: Among them, U(X) is the local neighborhood centered on X in the image to be dehazed, c represents the three color channels of R, G, and B, and I c (Y) is the pixel value of a certain pixel in a certain color channel in the image to be dehazed, I light (X) is the bright channel image of the image to be dehazed, and Y is any pixel point in the local neighborhood centered on X in the image to be dehazed.

6. The method according to claim 1, characterized in that The obtaining of the perspective rate based on the atmospheric scattering model and the atmospheric light value corresponding to each pixel point includes: Calculating a perspective estimation value based on an atmospheric scattering model and the atmospheric light value corresponding to each pixel point; The perspective estimation value is compensated to obtain the perspective.

7. The method according to claim 1, characterized in that The step of obtaining the image to be defogged comprises: Get the initial image; The Retinex algorithm is used to perform image defogging processing on the initial image, and / or image enhancement is performed on the initial image to obtain an image to be defogged.

8. An image defogging device, characterized in that: include: An acquisition module, used for acquiring the image to be dehazed; A first calculation module is used to calculate the dark channel and the bright channel of the image to be defogged based on a dark channel priori algorithm, and obtain a dark channel map and a bright channel map; A second calculation module is used to estimate the atmospheric light value based on the dark channel image and the bright channel image to obtain the atmospheric light value corresponding to each pixel point; A third calculation module, used for obtaining a perspective rate based on an atmospheric scattering model and an atmospheric light value corresponding to each pixel point; The restoration module is used to restore the image to be defogged based on the atmospheric light value corresponding to each pixel point and the perspective rate to obtain a defogged image.

9. An electronic device, characterized in that: The electronic device includes: at least one processor; a memory connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the image defogging method according to any one of claims 1 to 7 by executing the instructions stored in the memory.

10. A machine-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor is configured to perform the image defogging method according to any one of claims 1 to 7.

Images