Video defogging processing method and device

Abstract

The embodiment of the invention provides a video defogging processing method and device. The method comprises the following steps: determining a dark channel map of each frame of image in a video; determining a global atmospheric light value of each frame of image according to the dark channel map of each frame of image; in combination with edge detection, the transmissivity of each frame of image is determined according to the global atmospheric light value of each frame of image; and based on the atmospheric scattering model, carrying out defogging processing on each frame of image according to the global atmospheric light value and the transmissivity of each frame of image to obtain a fog-free video, thereby solving the problems that a defogging method based on dark channel prior needs to be combined with guided filtering, the guided filtering complexity is relatively high and the implementation cost is relatively high in the prior art, and improving the defogging efficiency. The global atmospheric light value is determined from the dark channel map, the transmissivity is solved in combination with edge detection, the halo phenomenon generated at the edge of the image after defogging is avoided, the method does not depend on a large filtering window, use of guiding filtering is avoided, and the cost is greatly reduced.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to a video dehazing method and apparatus. Background Technology

[0002] Current dehazing techniques generally process RGB (red, green, blue) images and can be mainly divided into three categories: The first category is image enhancement-based methods, which aim to remove image noise and improve image contrast to restore a clear, fog-free image. Representative methods include histogram equalization, Ret Inex algorithm, wavelet transform, and homomorphic filtering. However, these methods are prone to over-enhancing the image, resulting in color distortion, and have poor stability in practical applications. The second category is deep learning-based methods. These methods can directly train an end-to-end dehazing network or design a suitable neural network based on atmospheric scattering models, using datasets of foggy and fog-free images to obtain the parameters of the solution model. Due to the lack of real fog-free images, how to train the model on synthetic data and effectively transfer it to the real world is an important problem. The third category is atmospheric scattering model-based methods. By observing and summarizing a large number of foggy and fog-free images, analyzing the patterns related to color, dynamic range, saturation, and contrast, and proposing reasonable hypotheses and prior conditions, such as dark channel priors, color attenuation priors, color line priors, and fog line priors, the image is then restored in reverse. Such methods rely on the accuracy and versatility of assumptions and prior conditions. Currently, the most commonly used method is the dark channel prior. However, dehazing methods based on the dark channel prior often require the combination of guided filtering. Guided filtering has high complexity and high hardware implementation cost, which limits its practical application.

[0003] There is currently no solution to the problem that dehazing methods based on dark channel priors in related technologies require the combination of guided filtering, which is complex and costly to implement. Summary of the Invention

[0004] This application provides a video dehazing method and apparatus to at least solve the problem that dehazing methods based on dark channel priors in related technologies require the combination of guided filtering, which has high complexity and high implementation cost.

[0005] According to one embodiment of this application, a video dehazing processing method is provided, the method comprising:

[0006] Determine the dark channel map of each frame in the video;

[0007] The global atmospheric light value of each frame is determined based on the dark channel map of each frame.

[0008] By combining edge detection, the transmittance of each frame of the image is determined based on the global atmospheric light value of each frame.

[0009] Based on the atmospheric scattering model, each frame of the image is dehazed according to the global atmospheric light value and the transmittance to obtain a fog-free video.

[0010] According to another embodiment of this application, a video dehazing processing apparatus is provided, the apparatus comprising:

[0011] The first determining module is used to determine the dark channel map of each frame of the video;

[0012] The second determining module is used to determine the global atmospheric light value of each frame image based on the dark channel map of each frame image;

[0013] The third determining module is used to combine edge detection and determine the transmittance of each frame image based on the global atmospheric light value of each frame image;

[0014] The dehazing module is used to perform dehazing processing on each frame of the image based on the atmospheric scattering model, according to the global atmospheric light value and the transmittance of each frame of the image, to obtain a fog-free video.

[0015] According to yet another embodiment of this application, a computer program product is also provided, including computer program instructions, wherein the computer program instructions cause a computer to implement the steps in any of the above method embodiments.

[0016] According to yet another embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0017] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0018] This application embodiment determines the dark channel map of each frame in a video; determines the global atmospheric light value of each frame based on the dark channel map; combines edge detection to determine the transmittance of each frame based on the global atmospheric light value; and performs dehazing processing on each frame based on an atmospheric scattering model, using the global atmospheric light value and transmittance to obtain a fog-free video. This solves the problem in related technologies where dehazing methods based on dark channel priors require guided filtering, which is complex and costly. By determining the global atmospheric light value from the dark channel map and solving for the transmittance using edge detection, halo effects are avoided at image edges after dehazing. This method does not rely on a large filtering window, avoids the use of guided filtering, and significantly reduces costs. Attached Figure Description

[0019] Figure 1 This is a hardware structure block diagram of a computer device for a video dehazing processing method according to an embodiment of this application;

[0020] Figure 2 This is a flowchart of a video dehazing method according to an embodiment of this application;

[0021] Figure 3 This is a flowchart of a video dehazing method according to an optional embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the image dehazing process according to this embodiment;

[0023] Figure 5 This is a flowchart of the image dehazing process according to this embodiment;

[0024] Figure 6 This is a schematic diagram of the directional filtering window according to this embodiment;

[0025] Figure 7 This is a block diagram of a video dehazing processing apparatus according to an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0028] The methods and embodiments provided in this application can be executed in a computer device or similar computing device. Taking running on a computer device as an example, Figure 1This is a hardware structure block diagram of a computer device for the video dehazing processing method according to an embodiment of this application, as shown below. Figure 1 As shown, a computer device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor (MCU) or programmable logic device, etc.) and a memory 104 for storing data are also shown. The computer device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer device described above. For example, the computer device may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0029] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the video dehazing processing method in this embodiment. The processor 102 executes various functional applications and single-board matching by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to computer devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer equipment. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0031] This embodiment provides a video dehazing method running on the aforementioned computer device. Figure 2 This is a flowchart of a video dehazing method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0032] Step S202: Determine the dark channel map of each frame in the video;

[0033] Step S204: Determine the global atmospheric light value of each frame image based on the dark channel map of each frame image;

[0034] Step S206: Combine edge detection to determine the transmittance of each frame of image based on the global atmospheric light value of each frame of image;

[0035] Step S208: Based on the atmospheric scattering model, dehaze each frame of the image is performed according to the global atmospheric light value and transmittance of each frame to obtain a fog-free video.

[0036] In this embodiment, step S202 may specifically include:

[0037] To obtain a single-channel image, find the channel with the smallest value among the three channels for any pixel in each frame of the video.

[0038] Minimum filtering is applied to the single-channel image to obtain the dark channel image of each frame.

[0039] Figure 3 This is a flowchart of a video dehazing method according to an optional embodiment of this application, such as... Figure 3 As shown, step S204 above may specifically include:

[0040] S302, Histogram distribution of the dark channel plot;

[0041] S304, determine whether there is fog in each frame of the image based on the histogram distribution Hist of the dark channel map;

[0042] S306, In the case of fog in each frame of the image, determine the global atmospheric light value of each frame of the image based on the dark channel map of each frame of the image.

[0043] Further, step S304 may include: counting the number of pixels whose dark channel value in the dark channel image is lower than a preset threshold; determining the proportion of the number of pixels in the dark channel image; if the proportion of the number of pixels in the dark channel image is less than a preset value, determining that each frame of the image has fog; if the proportion of the number of pixels in the dark channel image is greater than or equal to the preset value, determining that each frame of the image is fog-free.

[0044] Further, step S306 may include: determining a preset number of pixels with larger brightness values ​​in the dark channel image of each frame as candidate points of each frame. Specifically, the brightness values ​​in the dark channel image of each frame can be sorted from largest to smallest, and the preset number of pixels at the top are determined as candidate points; alternatively, the brightness values ​​in the dark channel image of each frame can be sorted from smallest to largest, and the preset number of pixels at the bottom are determined as candidate points. The dark channel values ​​of the candidate points of each frame are obtained at the corresponding positions in each frame. Candidate points with dark channel values ​​greater than a preset value are determined as target candidate points of each frame. The maximum pixel value in the RGB channel of the target candidate point of each frame is determined to be the atmospheric light value, thus obtaining the global atmospheric light value of each frame.

[0045] In this embodiment, step S206 specifically includes: for each pixel in each frame of the image, performing the following steps to determine the transmittance of each pixel and obtain the transmittance of each frame of the image, wherein the pixel being executed is the current pixel: if the current pixel is an edge point, using 8 filtering windows in different directions to determine the mean square error between the current pixel and the corresponding filtering window in each frame of the image; determining the corresponding window number based on the minimum mean square error, performing minimum value filtering on the window corresponding to the window number to obtain the dark channel value of the current pixel; if the current pixel is not an edge point, performing minimum value filtering on the current pixel to obtain the dark channel value of the current pixel; determining the transmittance of the current pixel based on the dark channel value of the current pixel.

[0046] Furthermore, the above-mentioned determination of the transmittance of the current pixel based on the dark channel value of the current pixel may specifically include: obtaining the neighboring pixels of the current pixel; for each neighboring pixel, determining the minimum dark channel value among the dark channel values ​​of the RGB channels of each neighboring pixel as the first dark channel value; determining the minimum dark channel value from the first dark channel values ​​of the neighboring pixels as the second dark channel value; determining the product of the second dark channel value and the current dehazing intensity of the image where the current pixel is located; and determining the difference between 1 and the product as the transmittance of the current pixel.

[0047] The current dehazing intensity can be determined as follows: In the dark channel image of the image containing the current pixel, determine the proportion of pixels with gray values ​​less than a preset gray value; determine the current dehazing intensity based on this proportion of pixels with gray values ​​less than the preset gray value. In the dark channel image, the proportion of pixels with very low gray values ​​can reflect the current fog concentration to some extent; the lower the proportion, the denser the fog. The proportion of pixels with gray values ​​less than the preset gray value refers to the percentage of pixels with gray values ​​less than the preset gray value among all pixels.

[0048] In this embodiment, step S208 may specifically include: calculating the haze-free image of each frame based on the atmospheric scattering model in the following manner:

[0049]

[0050] Where J(x) is the fog-free image of each frame, t0 is the preset transmittance threshold, A is the global atmospheric light value of each frame, I(x) is each frame, and t(x) is the transmittance of each frame.

[0051] In one embodiment, the method further includes: converting the haze-free image into a YUV image; performing enhancement processing on the YUV image; and converting the enhanced YUV image into an RGB image to obtain a dehazed image.

[0052] In another embodiment, the method further includes: determining the average brightness difference of the dark channel map between the current frame image and the previous frame image in the video; if the average brightness difference is less than or equal to a preset threshold, determining the global atmospheric light value of the current frame image as the global atmospheric light value of the previous frame image, i.e., no change is needed, and the global atmospheric light value of the previous frame can be directly used; if the average brightness difference is greater than the preset threshold, adjusting the global atmospheric light value of the previous frame image according to the average brightness difference to obtain the global atmospheric light value of the current frame image, i.e., it is necessary to use the global atmospheric light value of the previous frame image to update the global atmospheric light value of the current frame image. The specific update method can be A = ((256-luma_diff)*A′+luma_diff*A) / 256; or A = (1-f(luma_diff))*A'+f(luma_diff)*A, where f is a continuous monotonically non-decreasing function of luma_diff, and luma_diff is the average brightness difference of the dark channel map between the current frame image and the previous frame image.

[0053] This embodiment is mainly geared towards hardware implementation and is applied to scenarios such as video recording and photo preview. It is a method and system for real-time dehazing of image signals captured by image sensors. It can be used in a hardware unit of an ISP chip or in digital image processing running on hardware such as DSP, GPU, and CPU in mobile devices.

[0054] This embodiment combines dark channel prior and edge detection for adaptive video dehazing, based on the atmospheric scattering model: I = Jt + A(1-t), where I is the image observed under fog conditions; J is the image observed under fog-free conditions; t is the transmittance, ranging from [0,1], where t = 0 means the fog is completely opaque and obscures the image, and t = 1 means there is no fog at all; A is the global atmospheric light value. First, the image is analyzed using the dark channel histogram statistics to determine the likelihood of fog, and the global atmospheric light value and current fog concentration are further estimated to adaptively determine the dehazing intensity. The calculation process is simple and easy to implement. Second, the atmospheric scattering model is solved based on the dark channel prior to estimate the global atmospheric light value A from the dark channel image, and the transmittance t is solved in conjunction with edge detection. This avoids halo effects at image edges after dehazing, does not rely on a large filtering window, avoids the use of guided filtering, and greatly reduces costs. Furthermore, post-processing can be performed in the YUV domain to further adjust brightness, contrast, and saturation after dehazing to ensure image quality and avoid over-enhancement. Finally, to prevent brightness flickering during video dehazing, temporal smoothing was added to maintain brightness uniformity between video frames after dehazing as much as possible.

[0055] Dehazing is performed in the RGB domain based on the atmospheric scattering model: I = Jt + A(1-t), where I is the image observed under the influence of fog; J is the image observed under fog-free conditions; t is the transmittance, which takes values ​​between [0,1]. t = 0 means that the fog is completely opaque and obscures the image, and t = 1 means that there is no fog at all. It is assumed that the local air is homogeneous, that is, within a certain neighborhood Ω(x) centered at x in the image, the transmittance t is a constant value, and the transmittances of the R, G, and B channels are equal; A is the global atmospheric light value, and the values ​​of the R, G, and B channels may not be equal. Figure 4 This is a schematic diagram of the image dehazing process according to this embodiment, as shown below. Figure 4 As shown, the fog-free image J is solved by obtaining the image transmittance distribution map t(x) and the global atmospheric light value A, where t0 is the set minimum transmittance.

[0056] Figure 5 This is a flowchart of the image dehazing process according to this embodiment, as follows: Figure 5 As shown, it includes:

[0057] Step S501: Calculate the dark channel map. Specifically, for the hazy image, the dark channel map I is obtained in the following manner. dark :

[0058] I dark =min y∈Ω(x) (min c∈{r,g,b} I c (y));

[0059] Where x represents a pixel in the image, Ω(x) represents the neighborhood centered at x, y is the pixel within the neighborhood of x, and I c These represent the different color channels of a color image. This formula means that for any point in the image, the channel with the smallest value among its three channels is first selected to obtain a single-channel image. Then, this single-channel image is subjected to minimum value filtering to obtain a dark channel image, as shown in the figure below. The filtering radius is determined by the window size.

[0060] Step S502, Statistical analysis of dark channel image I dark Histogram distribution;

[0061] Step S503: Determine if there is fog. If the determination result is yes, proceed to step S504. If the determination result is no, proceed to step S510.

[0062] The presence of fog in the current scene can be estimated using the histogram distribution (Hist) of the dark channel image. Given a suitable threshold (hazy_th), the number of pixels with dark channel values ​​below (hazy_th) is counted (hazy_sum), and the proportion (hazy_rate) in the entire image is calculated to determine the probability of fog in the current scene. Assume the image resolution is W*H.

[0063]

[0064]

[0065] Among them, the smaller the hazy_rate, the higher the probability of fog. When hazy_rate is below a certain value, it is considered that there is fog in the current scene.

[0066] Step S504: Estimate global atmospheric light value and defogging intensity;

[0067] In the dark channel image, the proportion of pixels with very low grayscale values ​​can reflect the current fog concentration to some extent; the lower the proportion, the denser the fog. Given a threshold α, where α is a decimal between (0,1), assume i is the minimum value satisfying the following condition:

[0068]

[0069] The value of i can be used to represent the current fog concentration, and a monotonically non-decreasing function f(i) is set to adaptively determine the defogging intensity of the current scene. The larger i is, the denser the current fog is, and the higher the required defogging intensity is.

[0070] Meanwhile, atmospheric light values ​​are generally obtained from the areas with the densest fog in the image. To eliminate interference from white objects in the original image and obtain suitable atmospheric light values, a screening process is needed in the dark channel image. The pixels with the highest brightness values ​​in the dark channel image, representing the top 0.1% (approximately 1 / 1024), are selected as candidate points. Then, the channel value with the highest brightness is found in the corresponding position in the original image. Assuming the image resolution is W*H, sorting the brightness of all points in the dark channel image and selecting the pixels with the highest brightness values ​​in the top 0.1% would be computationally complex. Therefore, a histogram distribution (Hist) is used for approximate calculation. Assuming the number of points in the dark channel image with brightness values ​​greater than dark_th is W*H / 1024, dark_th is calculated as follows:

[0071]

[0072] The atmospheric light value A is calculated using dark_th and the selected candidate points:

[0073] A c =MAX(A c ,I c (x)), if I dark (x)>dark_t(x).

[0074] Where c represents one of the three channels {r, g, b}, I dark (x) represents the dark channel value at candidate point x. As shown in the figure below, the points marked in red on the right are the candidate points obtained from the dark channel plot. The global atmospheric light value A is calculated at these points.

[0075] Step S505: Calculate the transmittance;

[0076] According to the dark channel prior theory, J dark →0, from the derivation of the atmospheric scattering model, we can obtain:

[0077]

[0078] In the above equation, a factor ω is introduced between [0,1] to control the defogging intensity. As mentioned above, ω can be adaptively estimated based on the histogram distribution of the dark channel plot.

[0079]

[0080] Where c represents one of the three channels {r, g, b}, and A is the calculated global atmospheric light value. Clearly, Yes To obtain the dark channel image, we need to find the channel with the lowest value and then perform local minimum filtering.

[0081] To ensure that the prior theory of dark channels is satisfied as much as possible. dark→0, minimum filtering requires a window of a certain size. Due to the windowing effect of minimum filtering, edges with depth-of-field changes in the dark channel image will expand towards higher grayscale values, causing the transmittance calculation near the edges to be overestimated, resulting in a decrease in dehazing effect. Therefore, halo phenomena are prone to appear at edges where depth of field changes occur, and the larger the window, the wider and more obvious the halo. Therefore, in determining... When calculating the dark channel map, edge information of the image is introduced, and eight f-directional filtering windows with different directions are set at the edges of the image. Edge detection can be implemented using the Canny operator or other common methods to calculate the edge map of the image. For non-edge areas of the image, conventional minimum filtering is used to obtain the dark channel, while for edges, side-window filtering is combined to obtain the dark channel. First, the mean square error MSEi between the current point p and the corresponding window is calculated in the original image using eight different filtering windows, i = 0, 1, 2, ..., 7:

[0082]

[0083] In the formula, Ω i (p) represents one of the eight filtering windows centered at the current point p, q represents the pixels in the neighborhood, N represents the number of pixels in the filtering window, and ||·||2 represents the L2 norm. Figure 6 This is a schematic diagram of the directional filtering window according to this embodiment. Assuming the filtering window size is 3*3, then the eight different filtering windows are as follows: Figure 6 As shown, 1 indicates that the image pixel at that location is involved in the calculation, and 0 indicates that it is not involved in the calculation.

[0084] The corresponding window number i = arg min(MSE) is obtained based on the minimum mean square error. i This window is then used for minimum value filtering to obtain the dark channel value of the current edge point p. The transmittance t(p) is then calculated.

[0085] Step S506, Defogging treatment;

[0086] A threshold t0 is set to limit the minimum transmittance, and the haze-free image J is calculated based on the atmospheric scattering model.

[0087]

[0088] The haze-free images obtained through the above calculations often have low brightness. To improve image visibility, further processing is performed on the image's brightness (Light), contrast (Contrast), and saturation (Saturation) after dehazing. The dehazed image is then transformed to the YUV domain and adjusted as follows:

[0089] Y=Y*contrast-a*(contrast-1);

[0090] Y = Y + b*(255-Y);

[0091] U = saturation * (U - 128);

[0092] V = saturation * (V - 128);

[0093] Here, a and b are decimals between [0,1] used to adjust brightness, while contrast and saturation are decimals between [0,2] used to adjust contrast and saturation, respectively. Finally, the transformed YUV image is converted back to an RGB image to obtain the final dehazed image.

[0094] Step S507: Enhancement processing to obtain a dehazed image;

[0095] In video dehazing, it is necessary to consider the brightness consistency between consecutive frames. The average brightness difference of the dark channel map between two consecutive frames, luma_diff, is calculated. If luma_diff is less than the set threshold luma_th, the global atmospheric light value A of the current frame can use the result A' of the previous frame, i.e., A = A'. Otherwise, the atmospheric light value A of the current frame is adjusted according to luma_diff.

[0096] A=((256-luma_diff)*A′+luma_diff*A) / 256; or

[0097] A = (1 - f(luma_diff)) * A' + f(luma_diff) * A, where f is a continuous, monotonically non-decreasing function of luma_diff.

[0098] In this embodiment, the probability of fog in the current scene is determined based on the histogram distribution of the dark channel image, and the required dehazing intensity and global atmospheric light value are estimated. The calculation process is simple, reducing the number of parameters that need to be set, and it can adapt to different foggy scenes, improving the robustness of the dehazing algorithm. By combining edge detection and side window filtering, directional filtering windows are used to avoid edge expansion of the dark channel image, improving the accuracy of the transmittance estimation at the edges and effectively avoiding edge halo phenomenon after dehazing. Furthermore, by adding processing to the image brightness, contrast, and saturation in the YUV domain, the visual effect of the dehazed image can be improved. Traditional dark channel dehazing algorithms require guided filtering to optimize transmittance, which is computationally complex, time-consuming, and has high hardware implementation costs. The dehazing algorithm proposed in this embodiment has low computational load, reducing costs and processing time.

[0099] This application also provides a video dehazing processing device. Figure 7 This is a block diagram of a video dehazing processing apparatus according to an embodiment of this application, such as... Figure 7 As shown, the device includes:

[0100] The first determining module 72 is used to determine the dark channel map of each frame of the video;

[0101] The second determining module 74 is used to determine the global atmospheric light value of each frame image based on the dark channel map of each frame image;

[0102] The third determining module 76 is used to combine edge detection and determine the transmittance of each frame image based on the global atmospheric light value of each frame image;

[0103] The dehazing module 78 is used to perform dehazing processing on each frame of the image based on the global atmospheric light value and the transmittance of each frame of the image, based on the atmospheric scattering model, to obtain a fog-free video.

[0104] In one embodiment, the first determining module 72 is further configured to obtain the channel with the smallest value among the three channels of any pixel point in each frame of the video to obtain a single-channel image; and to perform minimum value filtering on the single-channel image to obtain the dark channel image of each frame of the video.

[0105] In one embodiment, the second determining module 74 includes:

[0106] The statistics submodule is used to calculate the histogram distribution of the dark channel graph;

[0107] The first determining submodule is used to determine whether each frame of the image has fog based on the histogram distribution (Hist) of the dark channel map;

[0108] The second determining submodule is used to determine the global atmospheric light value of each frame image based on the dark channel map of each frame image when each frame image is foggy.

[0109] In one embodiment, the first determining submodule is further configured to count the number of pixels whose dark channel value in the dark channel image is lower than a preset threshold; determine the proportion of the number of pixels in the dark channel image; if the proportion of the number of pixels in the dark channel image is less than a preset value, determine that each frame of the image has fog; if the proportion of the number of pixels in the dark channel image is greater than or equal to the preset value, determine that each frame of the image is fog-free.

[0110] In one embodiment, the second determining submodule is further configured to determine a preset number of pixels with larger brightness values ​​in the dark channel image of each frame image as candidate points of each frame image; obtain the dark channel value of the candidate points of each frame image at the corresponding position of each frame image; determine the candidate points with dark channel values ​​greater than a preset value as target candidate points of each frame image; and determine the maximum pixel value in the RGB channel of the target candidate points of each frame image as the atmospheric light value, thereby obtaining the global atmospheric light value of each frame image.

[0111] In one embodiment, the third determining module 76 is further configured to perform the following steps for each pixel in each frame of the image to determine the transmittance of each pixel and obtain the transmittance of each frame of the image, wherein the pixel being determined is the current pixel:

[0112] If the current pixel is an edge point, the mean square error between the current pixel and the corresponding filter window is determined in each frame of the image using 8 filter windows in different directions; the corresponding window number is determined according to the minimum mean square error, and the window corresponding to the window number is subjected to minimum value filtering to obtain the dark channel value of the current pixel;

[0113] If the current pixel is a non-edge point, perform minimum value filtering on the current pixel to obtain the dark channel value of the current pixel;

[0114] The transmittance of the current pixel is determined based on the dark channel value of the current pixel.

[0115] In one embodiment, the third determining module 76 includes:

[0116] The acquisition submodule is used to acquire the neighboring pixels of the current pixel.

[0117] The third determining submodule is used to determine, for the neighboring pixels, the minimum dark channel value among the dark channel values ​​of the RGB channels of each neighboring pixel is the first dark channel value.

[0118] The fourth determining submodule is used to determine the minimum dark channel value from the first dark channel values ​​of the neighboring pixels as the second dark channel value;

[0119] The fifth determining submodule is used to determine the product of the second dark channel value and the current dehazing intensity of the image where the current pixel is located;

[0120] The sixth determining submodule is used to determine the difference between 1 and the product as the transmittance of the current pixel.

[0121] In one embodiment, the device further includes:

[0122] The seventh determining submodule is used to determine the proportion of pixels with gray values ​​less than a preset gray value in the dark channel image of the image where the current pixel is located; and to determine the current dehazing intensity based on the proportion of pixels with gray values ​​less than the preset gray value.

[0123] In one embodiment, the dehazing module is further configured to calculate the haze-free image of each frame of the image based on the atmospheric scattering model in the following manner:

[0124]

[0125] Wherein, J(x) is the fog-free image of each frame, t0 is the preset transmittance threshold, A is the global atmospheric light value of each frame, I(x) is each frame, and t(x) is the transmittance of each frame.

[0126] In one embodiment, the device further includes:

[0127] The transformation module is used to transform haze-free images into YUV images;

[0128] An enhancement processing module is used to enhance the YUV image;

[0129] The conversion module is used to convert the enhanced YUV image into an RGB image to obtain a dehazed image.

[0130] In one embodiment, the device further includes:

[0131] The fourth determining module is used to determine the average brightness difference of the dark channel map between the current frame image and the previous frame image in the video;

[0132] The fifth determining module is used to determine the global atmospheric light value of the current frame image as the global atmospheric light value of the previous frame image if the average brightness difference is less than or equal to a preset threshold.

[0133] An adjustment module is used to adjust the global atmospheric light value of the previous frame image according to the average brightness difference if the average brightness difference is greater than the preset threshold, so as to obtain the global atmospheric light value of the current frame image.

[0134] This application also provides a computer program product, including computer program instructions, wherein the computer program instructions cause a computer to implement the steps in any of the above method embodiments.

[0135] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0136] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0137] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0138] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0139] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0140] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0141] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A video dehazing method, characterized in that, The method includes: Determine the dark channel map of each frame in the video; The global atmospheric light value of each frame is determined based on the dark channel map of each frame. By combining edge detection, the transmittance of each frame of the image is determined based on the global atmospheric light value of each frame. Based on the atmospheric scattering model, each frame of the image is dehazed according to the global atmospheric light value and the transmittance to obtain a fog-free video.

2. The method according to claim 1, characterized in that, Determining the dark channel map of each frame in the video includes: Obtain the channel with the smallest value among the three channels of any pixel in each frame of the video to obtain a single-channel image; Minimum filtering is applied to the single-channel image to obtain the dark channel image of each frame.

3. The method according to claim 1, characterized in that, Determining the global atmospheric light value of each frame of the image based on the dark channel map of each frame includes: Statistically analyze the histogram distribution of the dark channel plot; The presence of fog in each frame of the image is determined based on the histogram distribution (Hist) of the dark channel map. In the case of fog in each frame of the image, the global atmospheric light value of each frame of the image is determined based on the dark channel map of each frame of the image.

4. The method according to claim 3, characterized in that, Determining whether each frame of an image has fog based on the histogram distribution of the dark channel map includes: The number of pixels whose dark channel values ​​in the dark channel image are lower than a preset threshold is counted. Determine the proportion of the number of pixels in the dark channel image; If the proportion of the number of pixels in the dark channel image is less than a preset value, it is determined that each frame of the image has fog. If the proportion of the number of pixels in the dark channel image is greater than or equal to the preset value, then each frame of the image is determined to be fog-free.

5. The method according to claim 3, characterized in that, Determining the global atmospheric light value of each frame of the image based on the dark channel map of each frame includes: A predetermined number of pixels with larger brightness values ​​in the dark channel image of each frame are determined as candidate points for each frame. Obtain the dark channel value of the candidate point in each frame of the image at the corresponding position; Candidate points whose dark channel values ​​are greater than a preset value are determined as target candidate points for each frame of the image. The maximum pixel value in the RGB channel of the target candidate point in each frame of the image is determined to be the atmospheric light value, thereby obtaining the global atmospheric light value of each frame of the image.

6. The method according to claim 1, characterized in that, Determining the transmittance of each frame of an image based on the global atmospheric light value, in conjunction with edge detection, includes: For each pixel in each frame of the image, the following steps are performed to determine the transmittance of each pixel, thereby obtaining the transmittance of each frame of the image, wherein the pixel being processed is the current pixel: If the current pixel is an edge point, the mean square error between the current pixel and the corresponding filter window is determined in each frame of the image using 8 filter windows in different directions; the corresponding window number is determined according to the minimum mean square error, and the window corresponding to the window number is subjected to minimum value filtering to obtain the dark channel value of the current pixel; If the current pixel is a non-edge point, perform minimum value filtering on the current pixel to obtain the dark channel value of the current pixel; The transmittance of the current pixel is determined based on the dark channel value of the current pixel.

7. The method according to claim 6, characterized in that, Determining the transmittance of the current pixel based on the dark channel value of the current pixel includes: Obtain the neighboring pixels of the current pixel; For the neighboring pixels, the minimum dark channel value among the dark channel values ​​of the RGB channels of each neighboring pixel is determined to be the first dark channel value; The minimum dark channel value is determined from the first dark channel values ​​of the neighboring pixels to be the second dark channel value; Determine the product of the second dark channel value and the current dehazing intensity of the image where the current pixel is located; The difference between 1 and the product is determined as the transmittance of the current pixel.

8. The method according to claim 7, characterized in that, The method further includes: In the dark channel image of the image where the current pixel is located, determine the proportion of pixels with gray values ​​less than a preset gray value; The current dehazing intensity is determined based on the proportion of pixels whose grayscale value is less than the preset grayscale value.

9. The method according to claim 1, characterized in that, Based on the atmospheric scattering model, the dehazing process for each frame of image, according to the global atmospheric light value and the transmittance, includes: The haze-free image of each frame is calculated based on the atmospheric scattering model in the following manner: Wherein, J(x) is the fog-free image of each frame, t0 is the preset transmittance threshold, A is the global atmospheric light value of each frame, I(x) is each frame, and t(x) is the transmittance of each frame.

10. The method according to claim 1, characterized in that, The method further includes: Convert the haze-free image to a YUV image; The YUV image is enhanced. The enhanced YUV image is converted to an RGB image to obtain a dehazed image.

11. The method according to claim 1, further comprising: Determine the average brightness difference of the dark channel map between the current frame and the previous frame in the video; If the average brightness difference is less than or equal to a preset threshold, the global atmospheric light value of the current frame image is determined to be the global atmospheric light value of the previous frame image. If the average brightness difference is greater than the preset threshold, the global atmospheric light value of the previous frame image is adjusted according to the average brightness difference to obtain the global atmospheric light value of the current frame image.

12. A video dehazing processing device, characterized in that, The device includes: The first determining module is used to determine the dark channel map of each frame of the video; The second determining module is used to determine the global atmospheric light value of each frame image based on the dark channel map of each frame image; The third determining module is used to combine edge detection and determine the transmittance of each frame image based on the global atmospheric light value of each frame image; The dehazing module is used to perform dehazing processing on each frame of the image based on the atmospheric scattering model, according to the global atmospheric light value and the transmittance of each frame of the image, to obtain a fog-free video.

13. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the method described in any one of claims 1 to 11 when it is run.

14. An electronic device, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the method described in any one of claims 1 to 11.

15. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1 to 11.

Images