Rain and fog removing image processing method

Abstract

The invention relates to the technical field of image processing, in particular to a rain and fog removal image processing method which comprises the following steps: S1, reading a video to be subjected to rain and fog removal, and then acquiring a rain and fog video frame image from the video to be subjected to rain and fog removal; s2, calculating transmissivity of a plurality of local matrix windows of a dark channel image corresponding to the rain and fog video frame image, and obtaining distribution transmissivity through a weighted ratio; s3, calculating the transmissivity of the rain and fog video frame image by using a method different from S2 to obtain the image transmissivity; s4, obtaining a final transmittance according to a weighted ratio of the distribution transmittance and the image transmittance; s5, performing image restoration on the rain and fog video frame image according to the final transmissivity to obtain a restored rain and fog removed video frame image; and then, combining the restored rain and fog-removed video frame image set with the rain and fog-free video frame image set to obtain a rain and fog-removed video. According to the invention, rain and fog in the video image can be rapidly and accurately removed, and the driving safety is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a rain and fog removal image processing method. Background Art

[0002] At present, in order to present video functions in real time and provide a more scientific basis for the handling and positioning of traffic accidents, several on-board cameras can be installed on vehicles for driving recording. However, it is inevitable to encounter low visibility such as rainy days and haze weather during driving, which will make the images taken by the on-board camera not clear enough and not conducive to recording. The prior art has an image processing method that can perform image defogging, which can remove the effects of interference factors such as haze, rain and fog in the image through an algorithm, so that the clarity of the image can be restored to a certain extent. However, some existing defogging algorithms usually require a large amount of computing resources and complex parameter settings, which will be difficult to implement in practical applications. There are also some defogging algorithms, although the required computing resources or parameter settings are relatively simple, but their scope of application has certain limitations. For images in complex scenes or more severe weather, their restoration effect is poor and the processing effect is unstable, which is not conducive to the rapid defogging of images. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art image defogging, such as limited applicability and unstable processing effect, and to provide a rain and fog removal image processing method, which can quickly and accurately remove rain and fog in video images and enhance driving safety.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] A rain and fog removal image processing method is provided, comprising the following steps:

[0006] S1. Read the rain and fog video to be removed, and then obtain the rain and fog video frame image from the rain and fog video to be removed;

[0007] S2. Calculate the transmittance of multiple local matrix windows of the dark channel image corresponding to the rainy fog video frame image, and obtain the distribution transmittance through the weighted ratio;

[0008] S3. Calculate the transmittance of the rainy and foggy video frame image using a method different from that in step S2 to obtain an image transmittance;

[0009] S4. The distribution transmittance and the image transmittance are added according to a weighted ratio to obtain a final transmittance;

[0010] S5. Perform image restoration on the rain and fog video frame images according to the final transmittance to obtain restored rain and fog-free video frame images; then, combine the restored rain and fog-free video frame image set with the rain and fog-free video frame image set to obtain the rain and fog-free video.

[0011] The present invention discloses a rain and fog removal image processing method, which calculates the transmittance of multiple local matrix windows of a corresponding dark channel image and obtains a distributed transmittance, and obtains an image transmittance according to different methods, and then obtains a final transmittance according to the distributed transmittance and the image transmittance, and then restores the rain and fog video frame image according to the final transmittance, and finally obtains a rain and fog removal video; the present invention can realize the rapid and accurate removal of rain and fog in video images, and enhance driving safety.

[0012] Furthermore, the step S1 comprises:

[0013] S11. Read the video to be de-rained and obtain each corresponding frame image from the video to be de-rained;

[0014] S12. Number each frame of image and divide it into a set of video frames with rain and fog and a set of video frames without rain and fog according to an image judgment method; wherein the image judgment method is:

[0015] Obtain a dark channel image of the original image, then regionalize and count the pixels of the dark channel image, and obtain a pixel threshold; if the pixel of the corresponding dark channel image is less than or equal to the pixel threshold, it is considered to be a rainy and foggy image, otherwise it is considered to be a rainy and foggy image;

[0016] S13. Extract the rainy and foggy video frame image set, and extract any one rainy and foggy video frame image from the rainy and foggy video frame image set.

[0017] Furthermore, the step S2 comprises the following steps:

[0018] S21. Obtain dark channel images of a plurality of matrix windows of different sizes corresponding to the rain and fog video frame image, and calculate the transmittance of the dark channel images respectively;

[0019] S22. The transmittance of the plurality of dark channel images obtained in step S21 is calculated according to a certain weighted ratio to obtain the distributed transmittance;

[0020] The step S3 comprises: estimating the transmittance of the rainy and foggy video frame image by using an SSR image enhancement algorithm to obtain an SSR transmittance;

[0021] The step S4 includes: obtaining the final transmittance by adding the distribution transmittance and the SSR transmittance according to a certain weighted ratio.

[0022] Furthermore, in step S21, the method for calculating the dark channel image is:

[0023] J dark (x) = min y∈Ω(x) [min c∈{r，g,b} I c (x)];

[0024] In the formula, J dark (x) represents the dark channel image, y∈Ω(x) represents a matrix centered on x, c∈{r,g,b} represents the values ​​of the three channels R, G, and B of the image, and I C (x) represents the original image;

[0025] Since the attenuation model of rain and fog image is:

[0026] I(x)=J(x)·t(x)+A·(1-t(x));

[0027] Transforming the above formula, we can get:

[0028]

[0029] Performing two minimum filtering operations on both sides of the above equation simultaneously yields:

[0030]

[0031] Since the dark channel value of the haze-free image is 0, J c (x) represents the haze-free image, A c It represents the atmospheric light value obtained from the previous frame of rainy and foggy video frame image and is a constant value, and we can get:

[0032]

[0033] Therefore, we can get:

[0034]

[0035] Finally, the formula for calculating the transmittance of the dark channel image is:

[0036]

[0037] In the formula, ω represents a constant whose value range is [0,1]; I c (x) represents a foggy image, A c Indicates the atmospheric light value obtained from the previous video frame with rain and fog.

[0038] Further, in step S21, the transmittance of the dark channel image of the n×n matrix window, the transmittance of the dark channel image of the (2n-1)×(2n-1) matrix window, and the transmittance of the dark channel image of the (2n+1)×(2n+1) matrix window are estimated, where n≥2.

[0039] Further, in step S22, the distributed transmittance is:

[0040] t dcpfinal (x) = ω1t dcp1 (x)+ω2t dcp2 (x)+ω3t dcp3 (x);

[0041] ω1+ω2+ω3=1;

[0042] Where, t dcp1 (x) represents the transmittance of the dark channel image of the n×n window, t dcp2 (x) represents the transmittance of the dark channel image of the (2n-1)×(2n-1) window, t dcp3 (x) represents the transmittance of the dark channel image in the (2n+1)×(2n+1) window; ω1, ω2, and ω3 are all weighting coefficients, and the value of each weighting coefficient depends on the similarity of the transmittance graphs of the dark channel images under the matrix windows of different sizes.

[0043] Further, in step S22, regarding the values ​​of the weighting coefficients of ω1, ω2, and ω3, if the transmittance maps of the dark channel images under different matrix windows are not very different, then ω1=ω2=ω3=1 / 3 is taken; if the transmittance maps of the dark channel images under different matrix windows are very different, then the corresponding weight of the most significant transmittance map is selected as 1, and the rest are 0; wherein:

[0044] The difference judgment method of the transmittance graphs of the dark channel images under different matrix windows is as follows: by calculating the histogram of each transmittance graph, and then evaluating the difference size of each transmittance graph by the histogram intersection method;

[0045] If the similarity of the histograms is 90% or above, the transmittance graphs are considered to be not very different;

[0046] If the similarity of the histogram is between 80% and 90%, then ω1>ω2=ω3, or ω1>ω2>ω3;

[0047] If the similarity of the histograms is less than 80%, it is considered that the transmittance maps are quite different.

[0048] Further, in step S3, the transmittance calculation formula of the SSR image enhancement algorithm is:

[0049]

[0050] Where, t ssr (x) represents the SSR transmittance, when t ssr When (x) is a constant, it is written as I c (x) represents a video frame image with rain and fog, A c It represents the atmospheric light value obtained from the previous frame of rainy and foggy video frame image, and G(x) represents the Gaussian center surrounding function.

[0051] Furthermore, in step S4, the calculation formula of the final transmittance is:

[0052] t z (x) = ω a t dcpfinal (x)+ω b t ssr (x);

[0053] ω a +ω b =1;

[0054] In the formula, ω a ,ω b are weighted coefficients, t dcpfinal (x) represents the transmittance obtained in step S22, t ssr (x) represents the transmittance obtained in step S3.

[0055] Furthermore, in step S5, the image restoration formula is:

[0056]

[0057] Where J(x) represents the obtained rain-fog-free video frame image, I(x) represents the rain-fog video frame image to be processed, A represents the atmospheric light value obtained from the previous rain-fog video frame image, and t z (x) represents the final transmittance, and t0 represents the set threshold;

[0058] according to:

[0059]

[0060] We can get:

[0061]

[0062] Where, t z Indicates the corresponding t z (x) is the global constant transmittance.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] 1. By calculating the transmittance of multiple local matrix windows of the corresponding dark channel image, and obtaining the image transmittance according to different methods, and then obtaining the final transmittance, the image frame image with rain and fog is restored by the final transmittance, and finally the rain and fog removal video is obtained; the present invention can realize the rapid and accurate removal of rain and fog in the video image, and enhance driving safety;

[0065] 2. Calculate the transmittance of the dark channel image according to the dark channel images of multiple matrix windows of different sizes, and then obtain the setting of the distribution transmittance according to a certain weighted ratio, which can improve the defogging effect, processing accuracy and processing stability;

[0066] 3. The transmittance of the rainy and foggy video frame image is estimated by the SSR image enhancement algorithm to obtain the SSR transmittance. Then, the final transmittance is set according to the distribution transmittance and the SSR transmittance, which can more accurately capture the rain and fog information in the image and improve the defogging effect and processing stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A flowchart of a rain and fog removal image processing method of the present invention;

[0068] Figure 2 A connection diagram of an image processing system based on a rain and fog removal image processing method according to the present invention;

[0069] Figure 3 This is a flow chart of step S12 of the present invention. DETAILED DESCRIPTION

[0070] The present invention is further described below in conjunction with specific implementation methods. In order to enable those skilled in the art to better understand the solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0071] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatuses.

[0073] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0074] Embodiment 1

[0075] like Figure 1 The first embodiment of a rain and fog removal image processing method of the present invention is shown, comprising the following steps:

[0076] S1. Read the video to be de-rained and then obtain the video frame image with rain and fog from the video to be de-rained;

[0077] S2. Calculate the transmittance of multiple local matrix windows of the dark channel image corresponding to the rainy fog video frame image, and obtain the distribution transmittance through the weighted ratio;

[0078] S3. Calculate the transmittance of the rainy and foggy video frame image using a method different from step S2 to obtain the image transmittance;

[0079] S4. The distribution transmittance and the image transmittance are added according to a weighted ratio to obtain a final transmittance;

[0080] S5. Perform image restoration on the rain and fog video frame images according to the final transmittance to obtain restored rain and fog-free video frame images; then, combine the restored rain and fog-free video frame image set with the rain and fog-free video frame image set to obtain the rain and fog-free video.

[0081] The present invention calculates the transmittances of multiple local matrix windows of the corresponding dark channel image and obtains the distributed transmittance, and obtains the image transmittance according to different methods, and then obtains the final transmittance according to the distributed transmittance and the image transmittance, and then restores the rain and fog video frame image according to the final transmittance, and finally obtains the rain and fog removed video; the present invention can realize the rapid and accurate removal of rain and fog in the video image, and enhance driving safety.

[0082] Embodiment 2

[0083] This embodiment is a second embodiment of a rain and fog removal image processing method of the present invention. This embodiment is similar to the first embodiment, except that: Figure 2As shown, the present invention also includes an image processing system based on the rain and fog removal image processing method, the image processing system includes a vehicle-mounted camera, an image recognition module connected to the vehicle-mounted camera, an image processing module connected to the image recognition module, a display connected to the image processing module, and also includes a first DDR memory connected to the image recognition module, and a second DDR memory connected to the image processing module. In this embodiment, the display can be a TFT display.

[0084] Step S1 includes:

[0085] S11. Read the rain and fog video to be removed, and obtain each corresponding frame image from the rain and fog video to be removed; specifically, the vehicle-mounted camera collects the rain and fog video to be removed, and sends the rain and fog video to the image recognition module to obtain each corresponding frame image;

[0086] S12. Number each frame image, and divide it into a rain fog video frame image set and a rain fog free video frame image set according to an image judgment method; specifically, the image recognition module numbers each frame image, and then judges whether each frame image belongs to a rain fog video frame image or a rain fog free video frame image according to the image judgment method; wherein the image judgment method is:

[0087] Obtain the dark channel image of the original image, then traverse the local area of ​​the dark channel image, and count the pixels in the region at the same time to obtain the pixel threshold; if the pixel of the corresponding dark channel image is less than or equal to the pixel threshold, it is considered to be a rainy and foggy video frame image, otherwise it is considered to be a rainy and foggy video frame image;

[0088] like Figure 3 As shown, for a 1920*1536 image, its corresponding dark channel image includes 2949120 pixels; these dark channel pixels are distributed in the number of pixels from 0 to 15 for statistical calculation, so as to determine whether the current image is a rain fog video frame image. The starting position and the ending position of the traversal of the pixel points can be controlled by the row scan counter and the field scan counter of the display screen. When the first pixel of each frame image is processed, a flag signal is generated to refresh the count value of sum; when the pixel points of the current frame image are traversed, the sum value is compared with the set threshold value 442368. If it is greater than 442368, it is considered to be a rain fog video frame image, otherwise it is a rain fog video frame image; the rain fog video frame image and the rain fog video frame image are stored in the first DDR memory.

[0089] S13. Extract a set of video frames with rain and fog, and extract any one video frame with rain and fog from the set of video frames with rain and fog.

[0090] For a video frame image with rain and fog, the image processing module executes steps S2 to S5 to perform defogging, and the image after defogging is stored in the second DDR memory, and the image after defogging is output to the display screen; for an image without rain and fog, it is directly output to the display screen. Specifically, step S2 includes the following steps:

[0091] S21. Obtain dark channel images of a plurality of matrix windows of different sizes corresponding to the rain and fog video frame image, and calculate the transmittance of the dark channel images respectively; wherein:

[0092] The method for calculating the dark channel image is:

[0093] J dark (x) = min y∈Ω(x) [min c∈{r，g,b} I c (x)];

[0094] In the formula, J dark (x) represents the dark channel image, y∈Ω(x) represents a matrix centered on x, c∈{r,g,b} represents the values ​​of the three channels R, G, and B of the image, and I C (x) represents the original image;

[0095] Since the attenuation model of rain and fog image is:

[0096] I(x)=J(x)·t(x)+A·(1-t(x));

[0097] Transforming the above formula, we can get:

[0098]

[0099] Under the condition that the matrix window is constant, two minimum value filtering processes are performed on both sides of the above equation at the same time; the first minimum value filtering is used to obtain the minimum value from the three channels of R, G, and B, and then the second minimum value filtering is performed to find the minimum value of all pixels in the matrix window for each pixel, which can ensure the smoothness of subsequent transmittance estimation; it can be obtained:

[0100]

[0101] Since the dark channel value of the haze-free image is 0, J c (x) represents the haze-free image, A c It represents the atmospheric light value obtained from the previous frame of rainy and foggy video frame image and is a constant value, and we can get:

[0102]

[0103] Therefore, we can get:

[0104]

[0105] Finally, the formula for calculating the transmittance of the dark channel image is:

[0106]

[0107] Wherein, ω represents a constant, and its value range is [0,1]. In this embodiment, the value is 0..95; I c (x) represents a foggy image, A c Indicates the atmospheric light value obtained from the previous video frame with rain and fog.

[0108] Next, according to the above method, the transmittance of the dark channel image of the n×n matrix window, the transmittance of the dark channel image of the (2n-1)×(2n-1) matrix window, and the transmittance of the dark channel image of the (2n+1)×(2n+1) matrix window are estimated, where n≥2; in this embodiment, n is 3, and the transmittance of the dark channel images of the 3×3 matrix window, 5×5 matrix window, and 7×7 matrix window are calculated.

[0109] S22. The transmittances of the multiple dark channel images obtained in step S21 are calculated according to a certain weighted ratio to obtain a distribution transmittance; wherein the calculation formula of the distribution transmittance is:

[0110] t dcpfinal (x) = ω1t dcp1 (x)+ω2t dcp2 (x)+ω3t dcp3 (x);

[0111] ω1+ω2+ω3=1;

[0112] Where, t dcp1 (x) represents the transmittance of the dark channel image of the n×n matrix window, t dcp2 (x) represents the transmittance of the dark channel image of the (2n-1)×(2n-1) matrix window, t dcp3 (x) represents the transmittance of the dark channel image in the (2n+1)×(2n+1) matrix window; ω1, ω2, and ω3 are all weighting coefficients, and the value of each weighting coefficient depends on the similarity of the transmittance maps of the dark channel images under matrix windows of different sizes.

[0113] Specifically, step S3 includes: estimating the transmittance of the rainy and foggy video frame image by using the SSR image enhancement algorithm to obtain the SSR transmittance; wherein:

[0114] The transmittance calculation formula of the SSR image enhancement algorithm is:

[0115]

[0116] Where, tssr (x) represents the SSR transmittance, when t ssr When (x) is a constant, it is written as I c (x) represents a video frame image with rain and fog, A c It represents the atmospheric light value obtained from the previous frame of rainy and foggy video frame image, and G(x) represents the Gaussian center surrounding function.

[0117] Specifically, step S4 includes: obtaining a final transmittance by adding the distribution transmittance and the SSR transmittance according to a certain weighted ratio; wherein:

[0118] The final transmittance is calculated as:

[0119] t z (x) = ω a t dcpfinal (x)+ω b t ssr (x);

[0120] ω a +ω b =1;

[0121] In the formula, ω a ,ω b are weighted coefficients, t dcpfinal (x) represents the transmittance obtained in step S22, t ssr (x) represents the transmittance obtained in step S3.

[0122] Specifically, in step S5, the image restoration formula is:

[0123]

[0124] Where J(x) represents the obtained rain-fog-free video frame image, I(x) represents the rain-fog video frame image to be processed, A represents the atmospheric light value obtained from the previous rain-fog video frame image, and t z (x) represents the final transmittance, and t0 represents the set threshold;

[0125] according to:

[0126]

[0127] We can get:

[0128]

[0129] Where, t z Indicates the corresponding t z (x) is the global constant transmittance.

[0130] Embodiment 3

[0131] This embodiment is a third embodiment of a rain and fog removal image processing method of the present invention. This embodiment is similar to the first or second embodiment, except that, in step S22, regarding the values ​​of the weighting coefficients of ω1, ω2, and ω3, if the transmittance maps of the dark channel images under different matrix windows are not very different, then ω1=ω2=ω3=1 / 3 is taken; if the transmittance maps of the dark channel images under different matrix windows are very different, then the corresponding weight of the most significant transmittance map is selected as 1, and the rest are 0; wherein:

[0132] The method for judging the difference of the transmittance graphs of the dark channel images under different matrix windows is: by calculating the histogram of each transmittance graph, and then evaluating the difference of each transmittance graph by the histogram intersection method; specifically:

[0133] If the similarity between the histograms is 90% or above, the transmittance graphs are considered to be not very different, and ω1 = ω2 = ω3 = 1 / 3 is taken;

[0134] If the similarity between the histograms is between 80% and 90%, then ω1>ω2=ω3, or ω1>ω2>ω3, and the weight of the n×n matrix window is kept slightly higher to highlight its leading role. Specifically, ω1, ω2, and ω3 can be 0.4, 0.3, and 0.3, or 0.5, 0.25, and 0.25, respectively; or, 0.45, 0.3, and 0.25 can be used to appropriately give weights to windows with slightly larger differences.

[0135] If the similarity between the histograms is less than 80%, it is considered that the transmittance maps are quite different, and the corresponding weight of the most significant transmittance map is 1, and the others are 0.

[0136] In the specific contents of the above-mentioned specific implementation methods, the various technical features can be combined in any non-contradictory manner. In order to make the description concise, not all possible combinations of the above-mentioned technical features are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A rain and fog removal image processing method, characterized in that: The steps include: S1. Read the rain and fog video to be removed, and then obtain the rain and fog video frame image from the rain and fog video to be removed; S2. Calculate the transmittance of multiple local matrix windows of the dark channel image corresponding to the rainy fog video frame image, and obtain the distribution transmittance through the weighted ratio; S3. Calculate the transmittance of the rainy and foggy video frame image using a method different from that in step S2 to obtain an image transmittance; S4. The distribution transmittance and the image transmittance are added according to a weighted ratio to obtain a final transmittance; S5. Perform image restoration on the rain and fog video frame images according to the final transmittance to obtain restored rain and fog-free video frame images; then, combine the restored rain and fog-free video frame image set with the rain and fog-free video frame image set to obtain the rain and fog-free video.

2. The rain and fog removal image processing method according to claim 1, characterized in that: The step S1 comprises: S11. Read the video to be de-rained and obtain each corresponding frame image from the video to be de-rained; S12. Number each frame of image and divide it into a set of video frames with rain and fog and a set of video frames without rain and fog according to an image judgment method; wherein the image judgment method is: Obtain a dark channel image of the original image, then regionalize and count the pixels of the dark channel image, and obtain a pixel threshold; if the pixel of the corresponding dark channel image is less than or equal to the pixel threshold, it is considered to be a rainy and foggy image, otherwise it is considered to be a rainy and foggy image; S13. Extract the rainy and foggy video frame image set, and extract any one rainy and foggy video frame image from the rainy and foggy video frame image set.

3. The rain and fog removal image processing method according to claim 1, characterized in that: The step S2 comprises the following steps: S21. Obtain dark channel images of a plurality of matrix windows of different sizes corresponding to the rain and fog video frame image, and calculate the transmittance of the dark channel images respectively; S22. The transmittance of the plurality of dark channel images obtained in step S21 is calculated according to a certain weighted ratio to obtain the distributed transmittance; The step S3 comprises: estimating the transmittance of the rainy and foggy video frame image by using an SSR image enhancement algorithm to obtain an SSR transmittance; The step S4 includes: obtaining the final transmittance by adding the distribution transmittance and the SSR transmittance according to a certain weighted ratio.

4. The rain and fog removal image processing method according to claim 3, characterized in that: In step S21, the method for calculating the dark channel image is: J dark (x)=min y∈Ω(x) [my c∈{r，g，b} IN c (x)]; In the formula, J dark (x) represents the dark channel image, y∈Ω(x) represents a matrix centered on x, c∈{r,g,b} represents the values ​​of the three channels R, G, and B of the image, and I C (x) represents the original image; Since the attenuation model of rain and fog image is: I(x)=J(x)·t(x)+A·(1-t(x)); Transforming the above formula, we can get: Performing two minimum filtering operations on both sides of the above equation simultaneously yields: Since the dark channel value of the haze-free image is 0, J c (x) represents the haze-free image, A c It represents the atmospheric light value obtained from the previous frame of rainy and foggy video frame image and is a constant value, and we can get: Therefore, we can get: Finally, the formula for calculating the transmittance of the dark channel image is: In the formula, ω represents a constant whose value range is [0,1]; I c (x) represents the foggy image, A c Indicates the atmospheric light value obtained from the previous video frame with rain and fog.

5. The rain and fog removal image processing method according to claim 3 or 4, characterized in that: In step S21, the transmittance of the dark channel image of the n×n matrix window, the transmittance of the dark channel image of the (2n-1)×(2n-1) matrix window, and the transmittance of the dark channel image of the (2n+1)×(2n+1) matrix window are estimated, where n≥2.

6. The rain and fog removal image processing method according to claim 5, characterized in that: In step S22, the distributed transmittance is: t dcpfinal (x)=ω1t dcp1 (x)+ω2t dcp2 (x)+ω3t dcp3 (x);; ω1+ω2+ω3=1; Where, t dcp1 (x) represents the transmittance of the dark channel image of the n×n window, t dcp2 (x) represents the transmittance of the dark channel image of the (2n-1)×(2n-1) window, t dcp3 (x) represents the transmittance of the dark channel image in the (2n+1)×(2n+1) window; ω1, ω2, and ω3 are all weighting coefficients, and the value of each weighting coefficient depends on the similarity of the transmittance maps of the dark channel images under matrix windows of different sizes.

7. The rain and fog removal image processing method according to claim 6, characterized in that: In step S22, regarding the values ​​of the weighting coefficients of ω1, ω2, and ω3, if the transmittance maps of the dark channel images under different matrix windows are not very different, then ω1=ω2=ω3=1 / 3 is taken; if the transmittance maps of the dark channel images under different matrix windows are very different, then the corresponding weight of the most significant transmittance map is selected as 1, and the rest are 0; wherein: The difference judgment method of the transmittance graphs of the dark channel images under different matrix windows is as follows: by calculating the histogram of each transmittance graph, and then evaluating the difference size of each transmittance graph by the histogram intersection method; If the similarity of the histograms is 90% or above, the transmittance graphs are considered to be not very different; If the similarity of the histogram is between 80% and 90%, then ω1>ω2=ω3, or ω1>ω2>ω3; If the similarity of the histograms is less than 80%, it is considered that the transmittance maps are quite different.

8. The rain and fog removal image processing method according to claim 3, characterized in that: In step S3, the transmittance calculation formula of the SSR image enhancement algorithm is: Where, t ssr (x) represents the SSR transmittance, when t ssr When (x) is a constant, it is written as I c (x) represents a video frame image with rain and fog, A c It represents the atmospheric light value obtained from the previous frame of rainy and foggy video frame image, and G(x) represents the Gaussian center surrounding function.

9. The rain and fog removal image processing method according to claim 3, characterized in that: In step S4, the final transmittance is calculated as: t z (x)=ω a t dcpfinal (x)+ω b t ssr (x); oh a +oh b =1; In the formula, ω a ,ω b are weighted coefficients, t dcpfinal (x) represents the transmittance obtained in step S22, t ssr (x) represents the transmittance obtained in step S3.

10. The rain and fog removal image processing method according to claim 9, characterized in that: In step S5, the image restoration formula is: Where J(x) represents the obtained rain-fog-free video frame image, I(x) represents the rain-fog video frame image to be processed, A represents the atmospheric light value obtained from the previous rain-fog video frame image, and t z (x) represents the final transmittance, and t0 represents the set threshold; according to: We can get: In the formula, t z Indicates the corresponding t z (x) is the global constant transmittance.

Images