Vehicle-mounted video image defogging method and system based on guide filtering, and storage medium

Abstract

The invention discloses a vehicle-mounted video image defogging method and system based on guided filtering, and a storage medium. The method comprises the following steps: acquiring a degraded image captured by a camera, and converting the degraded image into a grayscale image; setting a gray threshold, calculating a proportionality coefficient of pixel points lower than the gray threshold in the gray image, and determining a fog concentration coefficient of the gray image according to the proportionality coefficient; taking the grayscale image as a guide image, and sequentially filtering each pixel point of the grayscale image by adopting a guide filtering method so as to determine ambient atmospheric light; and determining a restored image based on the fog concentration coefficient and the ambient atmospheric light according to a preset haze degradation model.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle-mounted image defogging methods, and in particular to a vehicle-mounted video image defogging method, system and storage medium based on guided filtering. Background Art

[0002] Bad weather such as rain, snow and fog is often one of the important causes of serious traffic accidents. According to research, the probability of traffic accidents in rainy weather is 6.4 times that in sunny weather, and the risk of traffic accidents in foggy weather is 3.4 times higher than that in sunny weather. How to help drivers improve their ability to identify obstacles such as vehicles in front of them in rainy and foggy weather has been a research hotspot in automotive active safety technology in recent years. The most commonly used method is to collect videos, images and other information in front of the vehicle, and then process them using image processing technology and present them to the driver. The current method is insufficient in continuous image processing and defogging effects.

[0003] Existing dehazing algorithms can often only process a single image. The dehazing model has the disadvantages of large computational complexity, slow computational speed, and can only process a single image. It is difficult to meet the requirements of continuous in-vehicle video image processing. Summary of the invention

[0004] To solve at least one aspect of the above problems, the present invention provides a vehicle-mounted video image defogging method based on guided filtering, comprising: obtaining a degraded image captured by a camera, and converting the degraded image into a grayscale image; setting a grayscale threshold, calculating the proportion coefficient of pixels in the grayscale image that are lower than the grayscale threshold, and determining the fog concentration coefficient of the grayscale image according to the proportion coefficient; using the grayscale image as a guiding image, and using a guided filtering method to filter each pixel of the grayscale image in turn to determine the ambient atmospheric light; and determining a repaired image based on the fog concentration coefficient and the ambient atmospheric light according to a preset haze degradation model.

[0005] Preferably, the preset haze degradation model adopts:

[0006] E=f β (II ∞ )+I ∞ ,

[0007] Among them, E is the repaired image, f β is the fog concentration coefficient, I is the degraded image, I ∞ is the ambient atmospheric light.

[0008] Preferably, the step of converting the degraded image into the grayscale image further comprises: converting the degraded image into the grayscale image using an integer method.

[0009] Preferably, the grayscale threshold is 50.

[0010] Preferably, the step of determining the fog concentration coefficient of the grayscale image according to the proportional coefficient further comprises: determining a corresponding relationship between the proportional coefficient and the fog concentration coefficient according to a value range of the fog concentration coefficient and the proportional coefficient:

[0011] f β =8-200*p,

[0012] Wherein, p is the ratio coefficient of the pixel points in the grayscale image that are lower than the grayscale threshold.

[0013] Preferably, it also includes: calculating the image information entropy and image variance of the degraded image and the restored image respectively; when the image information entropy and image variance of the restored image are both greater than the image information entropy and image variance of the degraded image, judging that the restored image is qualified; when the image information entropy of the restored image is less than or equal to the image information entropy of the degraded image, or when the image variance of the restored image is less than or equal to the image variance of the degraded image, judging that the restored image is unqualified.

[0014] On the other hand, a vehicle-mounted video image defogging system based on guided filtering is provided, comprising: an image collector, which is used to collect degraded images in real time; a controller, which is electrically connected to the image collector to receive the degraded image, and the controller preferably outputs a repaired image based on the degraded image using any of the above methods; and a display, which is connected to the controller and is used to receive and display the repaired image.

[0015] On the other hand, a computer-readable storage medium is provided, comprising program instructions, which, when executed by a computer, enable the computer to perform any of the above methods.

[0016] The embodiments of the present invention have the following beneficial effects: firstly, the atmospheric attenuation exponential law of light energy and the physical model of haze degradation are provided, then the attenuation law and the degradation model are corrected to obtain a simplified defogging model; the two parameters of the fog concentration coefficient and the ambient atmospheric light in the model are determined by using the grayscale image method and the guided filtering method; the defogging effect of the simplified defogging model is verified by using computer simulation. The defogging effect is obvious for continuous videos, while reducing the defogging time cost and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to better understand the above and other purposes, features, advantages and functions of the present invention, reference may be made to the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings refer to the same components. It should be understood by those skilled in the art that the accompanying drawings are intended to schematically illustrate the preferred embodiments of the present invention and have no limiting effect on the scope of the present invention, and the components in the drawings are not drawn to scale.

[0018] Figure 1 It is a schematic diagram of the process of a vehicle-mounted video image defogging method based on guided filtering according to an embodiment of the present invention;

[0019] Figure 2 A block diagram of a preset haze degradation model of a vehicle-mounted video image defogging method based on guided filtering according to an embodiment of the present invention;

[0020] Figure 3 A degraded image in a vehicle-mounted video image defogging method based on guided filtering according to an embodiment of the invention;

[0021] Figure 4 This is a restored image in the vehicle video image defogging method based on guided filtering according to an embodiment of the invention. DETAILED DESCRIPTION

[0022] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0024] In order to at least partially solve one or more of the above-mentioned problems and other potential problems, an embodiment of the present disclosure proposes a vehicle-mounted video image defogging method based on guided filtering, including: obtaining a degraded image captured by a camera, and converting the degraded image into a grayscale image; setting a grayscale threshold, calculating the proportion coefficient of pixels below the grayscale threshold in the grayscale image, and determining the fog concentration coefficient of the grayscale image according to the proportion coefficient; using the grayscale image as a guiding image, and using a guided filtering method to filter each pixel of the grayscale image in turn to determine the ambient atmospheric light; determining a repaired image based on the fog concentration coefficient and the ambient atmospheric light according to a preset haze degradation model.

[0025] Step s1, obtain the degraded image captured by the camera, and convert the degraded image into a grayscale image. The details are as follows:

[0026] The degraded image captured by the vehicle-mounted camera can be a continuous frame of a video, or it can be an independent or multiple images of a specified point obtained according to a preset time interval. The vehicle-mounted camera adopts a color camera, and the degraded image is a color original image of the environment captured by the vehicle-mounted camera.

[0027] The logarithmic relationship between white and black is divided into several levels, which is called grayscale. An image represented by grayscale is called a grayscale image. Grayscale images are usually obtained by measuring the brightness of each pixel in a single electromagnetic spectrum (such as visible light). Grayscale images used for display are usually saved with a nonlinear scale of 8 bits per sampled pixel, so that there can be 256 levels of grayscale. "0" represents pure black, "255" represents pure white, and the numbers in between represent the transition color from black to white from small to large. According to the three primary color channel values ​​of each pixel in the degraded image, the degraded image is converted into a grayscale image using a common grayscale image conversion method.

[0028] Step s2, setting a grayscale threshold, calculating the proportion coefficient of pixels below the grayscale threshold in the grayscale image, and determining the fog concentration coefficient of the grayscale image according to the proportion coefficient. The details are as follows:

[0029] The description of grayscale image still reflects the distribution and characteristics of the overall and local chromaticity and brightness levels of the whole image, just like color image. The existence of fog will affect the brightness of the image, especially the grayscale image. The grayscale distribution of the image will be different at different concentrations. The higher the proportion of pixels below the grayscale threshold, the clearer the image and the lower the fog concentration coefficient; the lower the proportion of pixels below the grayscale threshold, the lower the image clarity and the higher the fog concentration coefficient.

[0030] According to the fact that the fog concentration coefficient is inversely proportional to the ratio coefficient of the pixels below the grayscale threshold in the degraded image, the corresponding relationship between the fog concentration coefficient and the ratio coefficient of the pixels below the grayscale threshold in the degraded image is defined as:

[0031] f β =k1-k2*p,

[0032] Among them, f β is the fog concentration coefficient, p is the proportion coefficient of pixels below the grayscale threshold in the grayscale image, and k1 and k2 are constant parameters.

[0033] According to the value range of the fog concentration coefficient [1, 8], the constant parameter k1 is determined to be 8. The value range of p is determined according to the gray threshold, for example, the value range of p corresponding to the gray threshold is determined according to the empirical value, and the value of the constant parameter k2 is determined according to the determined maximum value of p, so that when the equation k2*p is maximized, the fog concentration coefficient takes the minimum value of 1.

[0034] Step s3, using the grayscale image as a guide image, and using the guided filtering method to filter each pixel of the grayscale image in turn to determine the ambient atmospheric light. The details are as follows:

[0035] The grayscale image is filtered using the guided filtering method to determine the ambient atmospheric light. The grayscale image of the foggy image is filtered and used as the ambient atmospheric light. ∞ The estimated value of . In the definition of guided filtering, a local linear model is used. This model believes that a point on a function has a linear relationship with the points in its neighboring parts. A complex function can be represented by many local linear functions. When the value of a point on the function is required, it is only necessary to calculate the values ​​of all linear functions containing the point and average them. Assuming that the output and input of the function satisfy a linear relationship in a two-dimensional window, image guided filtering is a linear variable filter, and its filter kernel is as shown in formula (1.1):

[0036]

[0037] In the formula, w k is the filter window centered at pixel k, a k and b k For a given unit window w k The linear transformation coefficient a k and b k In the window, it is a fixed value. The radius of the filter window is r. For example, the filter radius is 30, and i is the filter window w. k The pixel index in ,q i is the output image, I i To guide the image.

[0038] The guided filter must ensure that the difference between the input image p and the output image q is minimal, and it is necessary to minimize equation (1.2):

[0039]

[0040] The above formula is the objective function, which can be solved by the least squares method:

[0041]

[0042] In the formula, ε is the regularization smoothing factor. For example, the regularization smoothing factor is 0.3, and ε is to prevent a k If the value is too large, an adjustment factor is introduced. k (I,p) is the covariance of the guide image I and the input image p, var k (I) is the variance of the guide image I, For p in the unit window w k The mean value within is I in the unit window w k The mean within .

[0043] The input image P is used as the guide image, that is, I = p, and after calculation, we can get a k and b k , since point i may be contained in multiple windows, the windows are different, a k and b k The value of is also different, so it is necessary to calculate a within the window centered on point i k and b k Then find the average value of q i The value of ambient atmospheric light I ∞ It can be expressed as:

[0044]

[0045] In the formula, p i is the pixel point in the input image (i.e. grayscale image), i is the pixel index, and w i In pixels p i is the unit window centered, |w| is the filter window w k The total number of pixels in the unit window in , R means filtering each pixel, the filter radius is 30, and the smoothing factor is 0.3.

[0046] Step s4, determining the restoration image based on the preset haze degradation model based on the haze concentration coefficient and the ambient atmospheric light. Specifically, the following is the process:

[0047] The physical model of the original haze degradation is established as shown in the following formula:

[0048] I=I ∞ ρ(x)e -βd(x) +I ∞ (1-e -βd(x) )(2.1)

[0049] In the formula, I ∞ is the ambient atmospheric light, ρ is the standard light radiation intensity, I is the degraded image captured by the camera in foggy weather, β is the influence coefficient of haze concentration, d(x) is the distance function from a point in the environment to the camera, and x is the information of a point in the environment captured by the camera.

[0050] From the radiation principle, we know that E = I ∞ ρ, substituting into formula (2.1) we get:

[0051] E=e βd(x) (II ∞ )+I ∞ (2.2)

[0052] Where E is the restored image.

[0053] It can be assumed that the environmental depth corresponding to each point in the image is a constant value, and e βd(x) Simplified to the fog concentration coefficient f in the current environment β . Formula (2.2) can be simplified to:

[0054] E=f β (II ∞ )+I ∞ (2.3)

[0055] According to the preset degradation model (2.3), the fog concentration coefficient and ambient atmospheric light determined in steps s2 and s3 are substituted into the above formula to obtain the repaired image.

[0056] In some embodiments, the grayscale threshold is 50.

[0057] Specifically, when the grayscale threshold is 50, it can be determined based on empirical values ​​that the value range of p corresponding to the grayscale threshold is [0, 0.035]. Therefore, the value of the constant parameter k2 can be determined to be 200. Therefore, the step of determining the fog concentration coefficient of the grayscale image based on the proportional coefficient also includes:

[0058] According to the value range of the fog concentration coefficient and the proportional coefficient, the corresponding relationship between the proportional coefficient and the fog concentration coefficient is determined:

[0059] f β =8-200*p,

[0060] Where p is the ratio coefficient of pixels below the grayscale threshold in the grayscale image. When p>0.035, f β The value is 1.

[0061] In some embodiments, the step of converting the degraded image into a grayscale image further comprises: converting the degraded image into a grayscale image using an integer method.

[0062] Specifically, the integer method is used to convert the color degraded image into a grayscale image as follows:

[0063] Gray=(R*30+G*59+B*11) / 100,

[0064] Among them, R, G and B are the values ​​of each single channel of the pixel point of the degraded image, and Gray is the grayscale of the gray image. The description of the grayscale image still reflects the distribution and characteristics of the overall and local chromaticity and brightness levels of the entire image, just like the color image.

[0065] In some embodiments, it also includes: calculating the image information entropy and image variance of the degraded image and the restored image respectively; when the image information entropy and image variance of the restored image are both greater than the image information entropy and image variance of the degraded image, judging that the restored image is qualified; when the image information entropy of the restored image is less than or equal to the image information entropy of the degraded image, or the image variance of the restored image is less than or equal to the image variance of the degraded image, judging that the restored image is unqualified.

[0066] Specifically, a degraded image and its corresponding repaired image are selected for evaluation according to a preset time interval.

[0067] The information entropy method is used to evaluate the quality of the restored image. The image information entropy H represents the bit average of the grayscale set and describes the average information content of the image.

[0068]

[0069] In the formula: Calculate the frequency of each pixel in an image, regard the frequency of the pixel as probability, calculate the information entropy on the probability of each pixel, add up the information entropy of each pixel, and get the image information entropy, p i is the probability of pixel i appearing.

[0070] The image variance σ reflects the discreteness of the image grayscale relative to the grayscale average, and its size represents the contrast of the image.

[0071]

[0072] In the above formula, x i is the grayscale of the pixel, N is the number of pixels, and μ is the average grayscale value of the image.

[0073] like Figure 3 The degraded image obtained by the image collector (the color is removed in the figure) is processed by the vehicle-mounted video image defogging method based on guided filtering to obtain the following Figure 4The repaired image shown (the color is removed in the figure). It is calculated that: the image information entropy of the degraded image is 7.287, and the image variance of the degraded image is 0.0082; the image information entropy of the restored image is 7.835, and the image variance of the restored image is 0.0284; at the same time, the image information entropy of the contrast image (not shown) restored by the He algorithm is 7.562, and the image variance of the contrast image is 0.0157. It can be seen that the image information entropy and image variance of the restored image obtained by the vehicle-mounted video image defogging method based on guided filtering are both smaller than the degraded image and the contrast image obtained by the He algorithm, so the method effectively restores the degraded image. On the other hand, a vehicle-mounted video image defogging system based on guided filtering is provided, including: an image collector, the image collector is used to collect degraded images in real time; a controller, the controller is electrically connected to the image collector to receive the degraded image, and the controller outputs the repaired image based on the degraded image using any of the vehicle-mounted video image defogging methods based on guided filtering as described above; a display, the display is connected to the controller, and the display is used to receive and display the repaired image.

[0074] Specifically, the image collector is an image acquisition device including at least one camera, which is used to acquire environmental images or environmental videos. It can be understood by those skilled in the art that the camera directly acquires degraded images affected by environmental fog. The controller includes at least one program unit and a processor. When the program unit is executed by the processor, it implements a vehicle-mounted video image defogging method based on guided filtering, and outputs a repaired image after processing the input degraded image. The display is used to display the repaired image output by the controller.

[0075] On the other hand, a computer-readable storage medium is provided, comprising program instructions, which, when executed by a computer, enable the computer to perform any of the above-mentioned vehicle-mounted video image defogging methods based on guided filtering.

[0076] Specifically, the computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which may store program codes.

[0077] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the present invention.

Claims

1. A vehicle video image defogging method based on guided filtering, characterized in that: include: Acquire a degraded image captured by a camera, and convert the degraded image into a grayscale image; Setting a grayscale threshold, calculating a ratio coefficient of pixel points in the grayscale image that are lower than the grayscale threshold, and determining a fog concentration coefficient of the grayscale image according to the ratio coefficient; The grayscale image is used as a guide image, and a guided filtering method is used to filter each pixel of the grayscale image in turn to determine the ambient atmospheric light; A restored image is determined based on the fog concentration coefficient and the ambient atmospheric light according to a preset haze degradation model.

2. The method according to claim 1, characterized in that The preset haze degradation model adopts: E=f β (I-I ∞ )+I ∞ , Among them, E is the repaired image, f β is the fog concentration coefficient, I is the degraded image, I ∞ is the ambient atmospheric light.

3. The method according to claim 2, characterized in that The step of converting the degraded image into the grayscale image further includes: converting the degraded image into the grayscale image using an integer method.

4. The method according to claim 3, characterized in that The grayscale threshold is 50.

5. The method according to claim 4, characterized in that The step of determining the fog concentration coefficient of the grayscale image according to the proportional coefficient also includes: The corresponding relationship between the proportional coefficient and the fog concentration coefficient is determined according to the value range of the fog concentration coefficient and the proportional coefficient: f β =8-200*p, Wherein, p is the ratio coefficient of the pixel points in the grayscale image that are lower than the grayscale threshold.

6. The method according to claim 5, characterized in that Also includes: Calculating image information entropy and image variance of the degraded image and the restored image respectively; When the image information entropy and image variance of the restored image are both greater than the image information entropy and image variance of the degraded image, the restored image is judged to be qualified; When the image information entropy of the restored image is less than or equal to the image information entropy of the degraded image, or when the image variance of the restored image is less than or equal to the image variance of the degraded image, the restored image is judged to be unqualified.

7. A vehicle-mounted video image defogging system based on guided filtering, characterized in that: include: An image collector, wherein the image collector is used to collect degraded images in real time; A controller, the controller being electrically connected to the image collector to receive the degraded image, and the controller outputting a repaired image based on the degraded image by using the method according to any one of claims 1 to 6; A display is connected to the controller, and is used to receive and display the repaired image.

8. A computer-readable storage medium, characterized in that: The method comprises program instructions, which, when executed by a computer, enable the computer to execute the method according to any one of claims 1 to 6.

Images