An image dehazing method based on region line prior

Abstract

The invention discloses an image defogging method based on region line prior, belonging to the technical field of image defogging processing. The method comprises the following steps: firstly, an original image I is read through an image input module; then, the original image I is defogged through an image processing module; finally, a defogged image I' corresponding to the original image I is output through an image output module, wherein the image processing module comprises a prior constraint module, a joint optimization module and a scene restoration module, and a foggy image is defogged through the modules to obtain a fog-free image. The method of the invention has high robustness, so that the defogged image is accurate and effective, and more real and effective information can be obtained therefrom, and the time complexity of the method is low.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an image defogging method based on region line priors. Background Art

[0002] Haze is a common natural phenomenon in the real world. For information collection industries such as image information collection, the appearance of haze will seriously reduce the contrast between image information and real information and change the inherent color of the image. Not only haze, but any weather that can affect the quality of collected images can reduce the contrast between image information and real information, especially in outdoor scene shooting under worse weather conditions. Low-contrast images (low-contrast images can be called foggy images) taken under the above conditions usually do not have enough information to ensure the normal operation of the visual system. Therefore, how to restore foggy images to ensure the authenticity and effectiveness of image information extraction is crucial.

[0003] The current image defogging methods can be divided into two categories: image defogging methods based on image enhancement and image defogging methods based on atmospheric scattering models. The former mainly uses traditional image enhancement technology to improve the contrast of foggy images to achieve the defogging effect. Such algorithms include histogram equalization, wavelet transform, retinex, etc. Although this type of method is simple to implement, since it does not take into account the physical degradation model of the image, it may cause partial information loss of the restored image, resulting in image distortion. In order to make up for the above defects, researchers have discovered and proposed an image defogging method based on the atmospheric scattering model (ASM). This method establishes an atmospheric scattering model by studying the scattering effect of atmospheric suspended particles on light. Mathematically, its modeling is as follows:

[0004] I(x,y)=A·ρ(x,y)·t(x,y)+A·(1-t(x,y))

[0005] In the above formula, (x, y) represents the position coordinates of the foggy image I, A represents the atmospheric light value, ρ(x, y) represents the scene reflectivity of the position coordinates (x, y) in the fog-free scene, and t(x, y) represents the atmospheric transmittance of the position coordinates (x, y). When the particles suspended in the atmosphere are uniformly distributed in space, t(x, y) = e -β·d(x,y) , where β represents the scattering coefficient and d(x,y) represents the depth of the scene at position coordinates (x,y).

[0006] Image dehazing methods based on atmospheric scattering models use prior or complex data processing to model transmittance and estimate atmospheric light. Therefore, these methods can be divided into pixel-level processing strategies, block-level processing strategies, neural network learning strategies, and non-local level processing strategies. Among them, the pixel-level processing strategy mainly uses the minimum color value of each pixel to construct the transmittance formula. Its algorithm complexity is low and the processing time is relatively short. However, the minimum channel contains many incorrect textures, which requires the use of additional control factors and subsequent blur operators to compensate. The block-level processing strategy estimates the transmittance by extracting local information from each block. Since the information contained in the block is richer than that contained in a single pixel, the restored effect is more realistic. However, this method requires the use of guiding tools (such as soft matting or bilateral filtering) to eliminate the artifacts produced by dehazing, and it also increases the complexity of the algorithm to a certain extent. The neural network learning strategy benefits from its powerful learning ability. Image dehazing technology based on the atmospheric scattering model can also be achieved through learning strategies. In recent years, DehazeNet was the first to use a neural network model for image dehazing. Subsequently, multi-scale convolutional neural networks (MSCNN) and end-to-end dehazing networks (AOD-NET) were proposed and applied to the field of image dehazing. The above algorithms can achieve good dehazing effects in most cases, but the quality of their training results depends on the selection of data sets, and they have high requirements for computing platform processing power and memory, and have high overhead. Unlike the three local strategies discussed previously, the non-local level processing strategy uses non-local priors to estimate atmospheric light and transmittance by identifying different color clusters in the image. Although very reliable results can be obtained in most cases, the accuracy of color classification may decrease with the increase of haze concentration, which in turn affects the image restoration results.

[0007] In addition to the above defects, all defogging methods based on non-learning strategies of atmospheric scattering models have a common shortcoming, which is to ignore the potential connection between imaging parameters. Once the atmospheric light cannot be accurately predicted, the subsequent transmittance estimation is bound to be disturbed. Therefore, it is particularly important to propose image defogging technology with higher robustness to improve image contrast and increase image authenticity. Summary of the invention

[0008] Purpose of the invention: In order to overcome the defects of the existing image defogging technology in the prior art, such as ignoring the potential relationship between imaging parameters, resulting in unsatisfactory image defogging effect and slow processing time, the present invention provides an image defogging method based on regional line prior, which has high robustness, makes the defogged image accurate and effective, and can obtain more real and effective information from it, and the time complexity of the method is low.

[0009] Technical solution: To achieve the above-mentioned purpose, the image defogging method based on region line prior of the present invention comprises the following steps: The method provides an image defogging system based on region line prior, the system comprises the following modules:

[0010] Image input module: used to read the original image I;

[0011] Image processing module: used to perform defogging on the original image I;

[0012] Image output module: used to output the defogging image I' corresponding to the original image I;

[0013] The above method comprises the following steps:

[0014] S1 first reads the original image I through the image input module;

[0015] S2 then performs dehazing processing on the original image I through an image processing module;

[0016] S3 finally outputs the defogging image I' corresponding to the original image I through the image output module.

[0017] Furthermore, the image processing module includes a region line prior constraint module, and the original image I is processed by the region line prior constraint module, specifically including the following steps:

[0018] Step 1: Given the original image I, the atmospheric scattering model ASM can be expressed as:

[0019] I(x,y)=A·ρ(x,y)·t(x,y)+A·(1-t(x,y)) Formula (1)

[0020] In the above formula, (x, y) represents the position coordinates of the original image I, I(x, y) represents the scene reflectivity of the position coordinates (x, y) in a foggy scene, A represents the atmospheric light value, ρ(x, y) represents the scene reflectivity of the position coordinates (x, y) in a fog-free scene, t(x, y) represents the atmospheric transmittance of the position coordinates (x, y), and when the particles suspended in the atmosphere are uniformly distributed in space, t(x, y) = e -β·d(x,y) , where β represents the scattering coefficient and d(x,y) represents the scene depth at position coordinates (x,y);

[0021] The original image I is divided into n regions, and the relationship between the scene reflectivity of each region and its scene depth can be expressed by a mathematical formula:

[0022]

[0023] In the above formula, m∈(1,2,…,n), represents the average scene reflectivity of the mth area in a fog-free scene, represents the average scene depth of the mth region, and ∝ represents the average scene reflectivity of the mth region in a fog-free scene. and the average scene depth of the mth region In direct proportion, Ω m represents the set of pixels in the mth region, |Ω m | indicates Ω m The number of pixels in the c (x, y) represents the reflectivity of the scene in the c channel at the position coordinate (x, y) in a fog-free scene, c∈{R,G,B} represents the red, green, and blue channels, (x, y)∈Ω m Represents Ω m The position coordinates of the pixel point in (x, y), d(x, y) represents the scene depth with the position coordinates (x, y) in the mth region;

[0024] Step 2: For a given foggy image, the relationship between the pixel average value of each area and its scene depth can be expressed mathematically as follows:

[0025]

[0026] In the above formula, m∈(1,2,…,n), represents the average scene reflectivity of the mth region in a foggy scene, Ω m represents the set of pixels in the mth region, |Ω m | indicates Ω m The number of pixels in (x,y)∈Ω m Represents Ω m The position coordinates of the pixel point in I c (x, y) represents the reflectivity of the scene at the position coordinate (x, y) in the c channel in a foggy scene, c∈{R,G,B} represents the red, green, and blue channels, represents the average scene depth of the mth region, and ∝ represents the average scene reflectivity of the mth region in a foggy scene. and the average scene depth of the mth region Directly proportional relationship;

[0027] Step 3: Combining the above formula (1) and formula (2), the average scene reflectance of the mth area in the fog-free scene is And the average scene reflectivity of the mth area in the foggy scene The relationship between can be expressed mathematically as:

[0028]

[0029] In the above formula, represents the average scene reflectivity of the mth area in a fog-free scene, represents the average scene reflectivity of the mth area in a foggy scene, is the scene reflectance of the area with minimum scene depth, and k represents the slope.

[0030] Furthermore, first, assuming that the atmospheric transmittance is the same in the same area, represents the average atmospheric transmittance of the mth region. The above formula (1) can be further simplified as:

[0031]

[0032] In the above formula, m∈(1,2,…,n), (x,y) represents the position coordinates of the original image I, A represents the atmospheric light value, and ρ(x,y) represents the scene reflectivity of the position coordinates (x,y) in a fog-free scene. represents the average atmospheric transmittance of the mth region, m∈(1,2,…,n);

[0033] Then, take the mean value on both sides of formula (5), and then formula (5) is transformed into:

[0034]

[0035] In the above formula, represents the average scene reflectivity of the mth region in a foggy scene, represents the average scene reflectivity of the mth area in a fog-free scene, is the average atmospheric transmittance of the mth region, represents the average value of atmospheric light;

[0036] Secondly, combining formula (4) and formula (6), the average atmospheric transmittance of the mth region is It can be expressed as:

[0037]

[0038] In the above formula, is the average atmospheric transmittance of the mth region, represents the average scene reflectivity of the mth region in a foggy scene, is the scene reflectance of the area with the minimum scene depth, A represents the atmospheric light value, represents the average value of atmospheric light, and k represents the slope;

[0039] Next, substitute formula (7) into formula (5) to obtain the following formula:

[0040]

[0041] In the above formula, ρ(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a fog-free scene. represents the average scene reflectivity of the mth region in a foggy scene, is the scene reflectance of the area with the minimum scene depth, A represents the atmospheric light value, represents the average value of atmospheric light, k represents the slope, and I(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a foggy scene;

[0042] Then, given I(x,y), we have and It can be calculated by formula (3);

[0043] Finally, the above formula (8) is further rewritten as an expression formula (9) containing only three parameters:

[0044] ρ(x,y)=R m (k,A,I) Formula (9)

[0045] In the above formula, ρ(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a fog-free scene, and R m (·) represents the scene restoration formula for the mth region, k represents the slope, A represents the atmospheric light value, and I represents the scene reflectivity in a foggy scene. By using formula (9), if the slope k and the atmospheric light value A are determined, the final defogging result is obtained.

[0046] Furthermore, the image processing module includes a joint optimization module, through which the optimal slope k and the atmospheric light value A are obtained;

[0047] The joint optimization module includes the first constraint model F 1 and the second constraint model F 2 ;

[0048] First constraint model F 1 The expression is:

[0049]

[0050] In the above formula, Ψ(·) represents the averaging operation, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in the foggy scene, and the average brightness of the image in the fog-free scene approaches a specific value μ, R m (·) represents the scene restoration formula for the mth region, and the minimum value of formula (10) corresponds to the optimal slope k and atmospheric light value A;

[0051] The second constraint model F 2 The expression is:

[0052]

[0053] In the above formula, Φ(·) represents the calculated information loss rate, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in the foggy scene, and R m (·) represents the scene restoration formula of the mth region;

[0054] Combining the above formula (10) and formula (11), the optimal slope k p and the optimal atmospheric light value A p It can be expressed as:

[0055]

[0056] In the above formula, k p is the optimal slope, A p is the optimal atmospheric light value, k represents the slope, and A represents the atmospheric light value.

[0057] Furthermore, the image processing module includes a scene restoration module, and scene restoration is performed by the scene restoration module, specifically including the following steps:

[0058] First, the atmospheric light value A is defined as:

[0059] A=τ·O=τ·[O R ,O G ,O B ] Formula (13)

[0060] In the above formula, τ represents the amplitude of the atmospheric light value A, and O represents the color direction, where O can be calculated using the white point method;

[0061] Then, combining formula (13) and formula (14), the optimal slope k is p and the amplitude τ of the optimal atmospheric light value A p It can be expressed by formula (14):

[0062]

[0063] Finally, the original image I is downsampled, and the first constraint model F 1 The expression is converted to:

[0064]

[0065] In the above formula, Ψ(·) represents the averaging operation, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in a foggy scene, ↓ω represents the downsampling operation with the coefficient ω, and the average brightness of the image in a fog-free scene approaches a specific value μ, R m(·) represents the scene restoration formula of the mth region;

[0066] The second constraint model F 2 The expression is converted to:

[0067]

[0068] In the above formula, Φ(·) represents the information loss rate, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in a foggy scene, ↓ω represents the downsampling operation with the coefficient ω, and R m (·) represents the scene restoration formula for the mth region.

[0069] Furthermore, the coordinate descent method is used to solve the problem, and the optimal values ​​of the slope k and the amplitude τ of the atmospheric light value A are alternately obtained until convergence;

[0070] After J iterations, the solution of formula (14) can be expressed as:

[0071]

[0072]

[0073] The above formulas (17) and (18) are solved by the Fibonacci algorithm.

[0074] Furthermore, initialize k 1 =0.5, and set the stopping criterion to:

[0075] δ(j)=|τ j -τ j-1 |+|k j -k j-1 |≤∈=10 -3 Formula (19)

[0076] In the above formula (19), τ j is the j-th iteration τ, τ j-1 is the τ of the j-1th iteration, k j is the k of the jth iteration, k j-1 is the k of the j-1th iteration. When the iteration is completed, the atmospheric light value A and the slope k can be calculated through τ j and k j Calculate them separately.

[0077] Furthermore, by the optimal value k p and A p , and traverse all regions through formula (8) to obtain the final dehazing result.

[0078] Furthermore, guided filtering is used in the scene restoration module to refine the transmittance, and the final scene restoration model

[0079] It is expressed as:

[0080]

[0081] In the above formula, ρ(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a fog-free scene, and t r is a rough transmittance diagram, CR(·) represents guided filtering, A p represents the optimal atmospheric light value, and I(x,y) represents the scene reflectivity at the position coordinate (x,y) in a foggy scene;

[0082] The scene restoration module processes the original image I through the above steps to obtain the mathematical model ρ(x, y) corresponding to the defogged image I', thereby obtaining the defogged image I' of the original image I, which is finally output by the image output module.

[0083] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0084] 1. Existing image defogging methods rely on specific scene areas such as the sky for estimating atmospheric light. However, when the sky is not visible, these methods will fail to estimate atmospheric light. The method of the present invention uses the information of the entire image instead of a single pixel value to search for atmospheric light based on a joint optimization strategy, thereby improving the accuracy of atmospheric light and further improving the robustness of the method of the present invention.

[0085] 2. The existing image defogging methods will produce color cast problems after defogging. The method of the present invention can effectively restore the true color of the scene, and the calculated transmittance is consistent with the intuitive perception of the human eye;

[0086] 3. The existing image defogging methods may result in over-enhancement, over-saturation and other phenomena after defogging, and there will still be residual fog. The method of the present invention can effectively suppress these phenomena, so that the bright scenes in the image look real and natural after restoration, and some details obscured by fog can be well enhanced;

[0087] 4. The existing image defogging methods have complex calculation processes and long processing time. The method of the present invention can significantly simplify the defogging process and reduce the processing time. At the same time, the correlation between parameters can be used to improve the accuracy between atmospheric light and transmission rate, thereby ensuring the high efficiency and strong robustness of the method of the present invention.

[0088] 5. The existing methods have relatively high time complexity and low processing efficiency. The method of the present invention has low time complexity and high processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 It is a system structure diagram provided by the method of the present invention.

[0090] Figure 2 It is a flow chart of the steps of the method of the present invention.

[0091] Figure 3 It is the original image I used in the embodiment of the method of the present invention.

[0092] Figure 4 The DEFADE method is used to Figure 3 The defogging effect image after defogging processing.

[0093] Figure 5 The NLD method is used to Figure 3 The defogging effect image after defogging processing.

[0094] Figure 6 Is to use IDE method to Figure 3 The defogging effect image after defogging processing.

[0095] Figure 7 The MSCNN method is used to Figure 3 The defogging effect image after defogging processing.

[0096] Figure 8 The ProNext method is used to Figure 3 The defogging effect image after defogging processing.

[0097] Fig. 9 The EPDN method is used to Figure 3 The defogging effect image after defogging processing.

[0098] Fig.10 The MSBDN method is used to Figure 3 The defogging effect image after defogging processing.

[0099] Fig.11 The method of the present invention is used to Figure 3 The defogging effect image after defogging processing. DETAILED DESCRIPTION

[0100] The present invention will be further described below in conjunction with the accompanying drawings.

[0101] Embodiment 1:

[0102] The present embodiment provides an image defogging method based on region line priors. The method provides an image defogging system based on region line priors. Figure 1 The system includes the following modules:

[0103] Image input module: used to read the original image I, which refers to the image information obtained by collecting outdoor scene information under haze weather conditions or worse bad weather conditions, and the original image I is not limited to foggy images;

[0104] Image processing module: used for performing defogging processing on the original image I to obtain a defogging image I';

[0105] Image output module: used to output the defogging image I' corresponding to the original image I;

[0106] Based on the above-mentioned image defogging system with area line prior, the method of this embodiment includes the following steps: first, the original image I is read through the image input module; then the image input module sends the input original image I to the image processing module, and the image processing module performs defogging on the original image I to obtain the defogged image I'; finally, the image processing module sends the defogged image I' to the image output module, and the image output module outputs the defogged image I' corresponding to the original image I.

[0107] Embodiment 2:

[0108] The present embodiment provides an image defogging method based on region line priors. Based on Embodiment 1, the original image I is defogged by an image processing module to obtain a defogged image I', wherein the image processing module includes a region line prior constraint module.

[0109] The above-mentioned region line prior constraint module is based on dividing an image into multiple regions with the same scene depth. The average radiation intensity of each region is proportional to its scene depth. The farther the scene depth, the higher the average brightness value of the region.

[0110] It is known that for the original image I, the atmospheric scattering model ASM can be expressed as:

[0111] I(x,y)=A·ρ(x,y)·t(x,y)+A·(1-t(x,y)) Formula (1)

[0112] In the above formula, (x, y) represents the position coordinates of the original image I, I(x, y) represents the scene reflectivity of the position coordinates (x, y) in a foggy scene, A represents the atmospheric light value, ρ(x, y) represents the scene reflectivity of the position coordinates (x, y) in a fog-free scene, t(x, y) represents the atmospheric transmittance of the position coordinates (x, y), and when the particles suspended in the atmosphere are uniformly distributed in space, t(x, y) = e -β·d(x,y) , where β represents the scattering coefficient and d(x,y) represents the depth of the scene at position coordinates (x,y).

[0113] The original image I is processed by the region line prior constraint module, which specifically includes the following steps:

[0114] Step 1: Given the original image I, divide it into n regions, each with the same scene depth, and the n regions are n non-overlapping regions (region line prior constraint module divides the foggy image into n non-overlapping regions, each with the same depth of field, but lacks specific depth information, and it is difficult to perform region segmentation directly on the depth map. Since the distribution of the blue channel is very similar to the scene depth, the K-means method is used to perform region segmentation on the blue channel of the foggy image, that is, scenes with the same depth of field are divided into the same region, then the mth region can be expressed as (x, y) ∈ Ω m ), the relationship between the scene reflectivity of each area and its scene depth can be expressed by a mathematical formula:

[0115]

[0116] In the above formula, m∈(1,2,…,n), represents the average scene reflectivity of the mth area in a fog-free scene, represents the average scene depth of the mth region, and ∝ represents the average scene reflectivity of the mth region in a fog-free scene. and the average scene depth of the mth region In direct proportion, Ω m represents the set of pixels in the mth region, |Ω m | indicates Ω m The number of pixels in the c (x, y) represents the reflectivity of the scene in the c channel at the position coordinate (x, y) in a fog-free scene, c∈{R,G,B} represents the red, green, and blue channels, (x, y)∈Ω m Represents Ω m The position coordinates of the pixel point in the mth region are (x, y), and d(x, y) represents the scene depth with the position coordinates (x, y) in the mth region.

[0117] Step 2: It is known that due to haze interference, for a given foggy image, the relationship between the pixel average value of each area and its scene depth can be expressed by a mathematical formula:

[0118]

[0119] In the above formula, m∈(1,2,…,n), represents the average scene reflectivity of the mth region in a foggy scene, Ω m represents the set of pixels in the mth region, |Ω m | indicates Ω mThe number of pixels in (x,y)∈Ω m Represents Ω m The position coordinates of the pixel point in I c (x, y) represents the reflectivity of the scene at the position coordinate (x, y) in the c channel in a foggy scene, c∈{R,G,B} represents the red, green, and blue channels, represents the average scene depth of the mth region, and ∝ represents the average scene reflectivity of the mth region in a foggy scene. and the average scene depth of the mth region Directly proportional relationship.

[0120] Step 3: Combining the above formula (1) and formula (2), the average scene reflectance of the mth area in the fog-free scene is And the average scene reflectivity of the mth area in the foggy scene The relationship between can be expressed mathematically as:

[0121]

[0122] In the above formula, represents the average scene reflectivity of the mth area in a fog-free scene, represents the average scene reflectivity of the mth area in a foggy scene, is the scene reflectivity of the area with the minimum scene depth, k represents the slope, and the average scene reflectivity of the mth area in the fog-free scene And the average scene reflectivity of the mth area in the foggy scene It is a quasi-linear relationship, which is called the regional line prior.

[0123] Embodiment 3:

[0124] This embodiment provides an image defogging method based on region line priors, based on Embodiment 2.

[0125] First, the original image I is divided into n regions. Assuming that the atmospheric transmittance in the same region is the same, then represents the average atmospheric transmittance of the mth region. The above formula (1) can be further simplified as:

[0126]

[0127] In the above formula, m∈(1,2,…,n), (x,y) represents the position coordinates of the original image I, A represents the atmospheric light value, and ρ(x,y) represents the scene reflectivity of the position coordinates (x,y) in a fog-free scene. represents the average atmospheric transmittance of the mth region, m∈(1,2,…,n).

[0128] Then, take the mean value on both sides of formula (5), and then formula (5) is transformed into:

[0129]

[0130] In the above formula, represents the average scene reflectivity of the mth region in a foggy scene, represents the average scene reflectivity of the mth area in a fog-free scene, is the average atmospheric transmittance of the mth region, Represents the average value of atmospheric light.

[0131] Secondly, combining formula (4) and formula (6), the average atmospheric transmittance of the mth region is It can be expressed as:

[0132]

[0133] In the above formula, is the average atmospheric transmittance of the mth region, represents the average scene reflectivity of the mth region in a foggy scene, is the scene reflectance of the area with the minimum scene depth, A represents the atmospheric light value, represents the average value of atmospheric light, and k represents the slope.

[0134] Next, substitute formula (7) into formula (5) to obtain the following formula:

[0135]

[0136] In the above formula, ρ(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a fog-free scene. represents the average scene reflectivity of the mth region in a foggy scene, is the scene reflectance of the area with the minimum scene depth, A represents the atmospheric light value, represents the average value of atmospheric light, k represents the slope, and I(x,y) represents the reflectivity of the scene at position coordinates (x,y) in a foggy scene.

[0137] Then, given I(x,y), we have and It can be calculated by formula (3).

[0138] Then, the above formula (8) is further rewritten as an expression formula (9) containing only three parameters:

[0139] ρ(x,y)=R m (k,A,I) Formula (9)

[0140] In the above formula, ρ(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a fog-free scene, and R m (·) represents the scene restoration formula for the mth region, k represents the slope, A represents the atmospheric light value, and I represents the scene reflectivity in a foggy scene. By using formula (9), if the slope k and the atmospheric light value A are determined, the final defogging result is obtained.

[0141] Embodiment 4:

[0142] The present embodiment provides an image defogging method based on regional line priors. Based on Embodiment 3, the original image I is defogged by an image processing module to obtain a defogged image I', wherein the image processing module includes a joint optimization module. The present embodiment obtains the optimal slope k and atmospheric light value A by the joint optimization module.

[0143] The joint optimization module includes two constraint models. The first constraint model F 1 and the second constraint model F 2 .

[0144] First constraint model F 1 The expression is:

[0145]

[0146] In the above formula, Ψ(·) represents the averaging operation, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in the foggy scene, and the average brightness of the image in the fog-free scene approaches a specific value μ, R m (·) represents the scene restoration formula for the mth region, and the minimum value of formula (10) corresponds to the optimal slope k and atmospheric light value A.

[0147] However, only through the first constraint model F 1 It is not enough to adjust the overall brightness of the image, because this may make some pixels in the image completely black or white, resulting in information loss. Therefore, a second constraint model F is needed. 2 To ensure that information loss is minimized.

[0148] The second constraint model F 2 The expression is:

[0149]

[0150] In the above formula, Φ(·) represents the calculated information loss rate, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in the foggy scene, and R m (·) represents the scene restoration formula for the mth region.

[0151] Combining the above formula (10) and formula (11), the optimal slope k p and the optimal atmospheric light value A p It can be expressed as:

[0152]

[0153] In the above formula, k p is the optimal slope, A p is the optimal atmospheric light value, k represents the slope, and A represents the atmospheric light value.

[0154] Embodiment 5:

[0155] In this embodiment, an image defogging method based on region line prior is based on Embodiment 4, wherein the image processing module includes a scene restoration module, and scene restoration is performed by the scene restoration module;

[0156] In the above formula (12), since the atmospheric light value A has three components, A = [A R ,A G ,A B ], which greatly increases the complexity of the algorithm. In order to improve the operating efficiency of the algorithm, this embodiment adopts a more concise form to define the atmospheric light value, and its mathematical expression is:

[0157] A=τ·O=τ·[O R ,O G ,O B ] Formula (13)

[0158] In the above formula, τ represents the amplitude of the atmospheric light value A, O represents the color direction, and O can be calculated using the white point method.

[0159] Combining formula (13) and formula (14), the optimal slope k p and the amplitude τ of the optimal atmospheric light value A p It can be expressed by formula (14):

[0160]

[0161] In order to further improve the computational efficiency, the original image I is downsampled. Since the downsampling operation still has the original characteristics of the image, it does not affect the accuracy of parameter estimation. The first constraint model F 1 The expression is converted to:

[0162]

[0163] In the above formula, Ψ(·) represents the averaging operation, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in a foggy scene, ↓ω represents the downsampling operation with the coefficient ω, and the average brightness of the image in a fog-free scene approaches a specific value μ, R m (·) represents the scene restoration formula for the mth region.

[0164] The second constraint model F 2 The expression is converted to:

[0165]

[0166] In the above formula, Φ(·) represents the information loss rate, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in a foggy scene, ↓ω represents the downsampling operation with the coefficient ω, and R m (·) represents the scene restoration formula for the mth region.

[0167] Embodiment 6:

[0168] This embodiment is an image dehazing method based on regional line prior. Based on Embodiment 5, in order to solve the problem of solving the above formula (14), this embodiment adopts the coordinate descent method for solution. The key point of the coordinate descent method is to alternately obtain the optimal values ​​of the slope k and the amplitude τ of the atmospheric light value A until convergence.

[0169] After J iterations, the solution of formula (14) can be expressed as:

[0170]

[0171]

[0172] The above formula (17) and formula (18) are both related to one-dimensional search problems and can be solved by the Fibonacci algorithm. In this embodiment, initialize k 1 =0.5, and set the stopping criterion to:

[0173] δ(j)=|τ j -τ j-1 |+|k j -k j-1 |≤∈=10 -3 Formula (19)

[0174] In the above formula (19), τ j is the j-th iteration τ, τ j-1 is the τ of the j-1th iteration, k j is the k of the jth iteration, k j-1 is the k of the j-1th iteration. When the iteration is completed, the atmospheric light value A and the slope k can be calculated through τj and k j Calculate them separately.

[0175] Embodiment 7:

[0176] This embodiment is an image defogging method based on region line priors. Based on Embodiment 6, once the optimal value k is obtained, p and A p , then we can traverse all regions through formula (8) to get the final dehazing result. Since the region segmentation is implemented on the blue channel, there is a certain gap with the real scene depth map, so the result may have some deviations. In order to solve the above defects, guided filtering is used in the scene restoration module to refine the transmittance, and the final scene restoration model is expressed as:

[0177]

[0178] In the above formula, ρ(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a fog-free scene, and t r is a rough transmittance diagram, CR(·) represents guided filtering, A p represents the optimal atmospheric light value, and I(x,y) represents the reflectivity of the scene at position coordinates (x,y) in a foggy scene.

[0179] The scene restoration module of this embodiment processes the original image I through the above steps to obtain the mathematical model ρ(x, y) corresponding to the defogged image I', thereby obtaining the defogged image I' of the original image I, which is finally output by the image output module.

[0180] In summary, the method steps of the image defogging method based on region line prior refer to Figure 2 shown.

[0181] Embodiment 8:

[0182] This embodiment is an image defogging method based on regional line prior. Based on Example 7, assume that the original image I is as shown in Figure 3. This example uses the DEFADE method, NLD method, IDE method, MSCNN method, ProxNet method, EPDN method, MSBDN method and the method of the present invention (abbreviated as IDRLP) to defog the image. Figure 3 The corresponding original image I is dehazed. Figure 4 The DEFADE method is used to Figure 3 The defogging effect after defogging processing. Figure 4 middle, Figure 3 (a) The corresponding defogging effect picture is Figure 4 (a) Figure 3 (b) The corresponding defogging effect is Figure 4 (b) Figure 3 (c) The corresponding defogging effect is Figure 4 (c) Figure 3 (d) The corresponding defogging effect picture is Figure 4 (d);

[0183] Figure 5 The NLD method is used to Figure 3 The defogging effect after defogging processing. Figure 3 (a) The corresponding defogging effect picture is Figure 5 (a) Figure 3 (b) The corresponding defogging effect is Figure 5 (b) Figure 3 (c) The corresponding defogging effect is Figure 5 (c) Figure 3 (d) The corresponding defogging effect picture is Figure 5 (d);

[0184] Figure 6 Is to use IDE method to Figure 3 The defogging effect after defogging processing. Figure 3 (a) The corresponding defogging effect picture is Figure 6 (a) Figure 3 (b) The corresponding defogging effect is Figure 6 (b) Figure 3 (c) The corresponding defogging effect picture is Figure 6 (c) Figure 3 (d) The corresponding defogging effect picture is Figure 6 (d);

[0185] Figure 7 The MSCNN method is used to Figure 3 The defogging effect after defogging processing. Figure 3 (a) The corresponding defogging effect picture is Figure 7 (a) Figure 3 (b) The corresponding defogging effect is Figure 7 (b) Figure 3 (c) The corresponding defogging effect picture is Figure 7 (c) Figure 3 (d) The corresponding defogging effect is Figure 7 (d);

[0186] Figure 8 The ProNext method is used to Figure 3 The defogging effect after defogging processing. Figure 3 (a) The corresponding defogging effect picture is Figure 8 (a) Figure 3 (b) The corresponding defogging effect is Figure 8 (b) Figure 3 (c) The corresponding defogging effect picture is Figure 8 (c) Figure 3 (d) The corresponding defogging effect is Figure 8 (d);

[0187] Fig. 9 The EPDN method is used to Figure 3 The defogging effect after defogging processing. Figure 3 (a) The corresponding defogging effect picture is Fig. 9 (a) Figure 3 (b) The corresponding defogging effect is Fig. 9 (b) Figure 3 (c) The corresponding defogging effect picture is Fig. 9 (c) Figure 3 (d) The corresponding defogging effect is Fig. 9 (d);

[0188] Fig.10 The MSBDN method is used to Figure 3 The defogging effect after defogging processing. Figure 3 (a) The corresponding defogging effect picture is Fig.10 (a) Figure 3 (b) The corresponding defogging effect is Fig.10 (b) Figure 3 (c) The corresponding defogging effect picture is Fig.10 (c) Figure 3 (d) The corresponding defogging effect is Fig.10 (d);

[0189] Fig.11 The method of the present invention is used to Figure 3 The defogging effect after defogging processing. Figure 3 (a) The corresponding defogging effect picture is Fig.11 (a) Figure 3 (b) The corresponding defogging effect is Fig.11 (b) Figure 3 (c) The corresponding defogging effect picture is Fig.11 (c) Figure 3 (d) The corresponding defogging effect is Fig.11 (d).

[0190] It can be seen that the DEFADE method, MSCNN method, ProNext method and MSBDN method are not ideal for defogging dense fog, the NLD method will show over-enhancement phenomenon for the sky area, IDE will produce overexposure phenomenon in some areas, and the defogging result of EPDN will produce too dark phenomenon. The image defogging processing effect of the method proposed in the present invention is better than that of the other 8 methods.

[0191] The following uses a qualitative method to compare the above eight methods, namely, the DEFADE method, the NLD method, the IDE method, the MSCNN method, the ProxNet method, the EPDN method, the MSBDN method and the method of the present invention (IDRLP).

[0192] Table 1 shows the processing of 8 methods including DEFADE method, NLD method, IDE method, MSCNN method, ProxNet method, EPDN method, MSBDN method and the method of the present invention (IDRLP) Figure 3 Two common indicators are used for quantitative comparison of defogging effects, namely Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM). Generally speaking, the larger the PSNR value, the stronger the defogging ability of the algorithm; the larger the SSIM value, the more similar the defogging image is to the original non-fogging image. As can be seen from Table 1, the method of the present invention has a good effect on processing Figure 3 Compared with other methods, the pictures in the figure have obtained the best PSNR value and SSIM value, which shows the strong robustness of the method of the present invention.

[0193] Table 2 shows the comparison of the eight methods including DEFADE method, NLD method, IDE method, MSCNN method, ProxNet method, EPDN method, MSBDN method and the method of the present invention (IDRLP). Figure 3 (e) in Table 2 uses different resolutions to compare the time required, where "-" in Table 2 indicates insufficient memory. Compared with other algorithms, the speed of defogging processing of the present invention has a significant advantage. Experiments have shown that compared with existing methods, the present invention has stronger robustness and is superior to most new technologies in terms of image quality restoration and processing efficiency.

[0194] Table 1

[0195]

[0196] Table 2

[0197]

[0198] Embodiment 9:

[0199] The image defogging method based on regional line prior in this embodiment is based on Embodiment 8, the simulation language is matlab (R2016b), the operating environment is Windows 10, and the computer configuration is Intel (R) Core (TM) i5-7200U CPU @ 2.50GHz 16GB RAM.

[0200] The above is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An image defogging method based on region line prior, characterized in that: The method comprises the following steps: The method provides an image defogging system based on region line priors, the system comprising the following modules: Image input module: used to read the original image I; Image processing module: used to perform defogging on the original image I; Image output module: used to output the defogging image I corresponding to the original image I ‘ ; The above method comprises the following steps: S1 first reads the original image I through the image input module; S2 then performs dehazing processing on the original image I through an image processing module; S3 finally outputs the defogging image I corresponding to the original image I through the image output module ‘ ; The image processing module includes a region line prior constraint module, and the original image I is processed by the region line prior constraint module, specifically including the following steps: Step 1: Given the original image I, the atmospheric scattering model ASM can be expressed as: I(x,y)=A·ρ(x,y)·t(x,y)+A·(1-t(x,y)) Formula (1) In the above formula, (x, y) represents the position coordinates of the original image I, I(x, y) represents the scene reflectivity of the position coordinates (x, y) in a foggy scene, A represents the atmospheric light value, ρ(x, y) represents the scene reflectivity of the position coordinates (x, y) in a fog-free scene, t(x, y) represents the atmospheric transmittance of the position coordinates (x, y), and when the particles suspended in the atmosphere are uniformly distributed in space, t(x, y) = e -β·d(x,y) , where β represents the scattering coefficient and d(x,y) represents the scene depth at position coordinates (x,y); The original image I is divided into n regions, and the relationship between the scene reflectivity of each region and its scene depth can be expressed by a mathematical formula: In the above formula, m∈(1,2,…,n), represents the average scene reflectivity of the mth area in a fog-free scene, represents the average scene depth of the mth region, and ∝ represents the average scene reflectivity of the mth region in a fog-free scene. and the average scene depth of the mth region In direct proportion, Ω m represents the set of pixels in the mth region, |Ω m | indicates Ω m The number of pixels in the c (x, y) represents the reflectivity of the scene in the c channel at the position coordinate (x, y) in a fog-free scene, c∈{R,G,B} represents the red, green, and blue channels, (x, y)∈Ω m Represents Ω m The position coordinates of the pixel point in (x, y), d(x, y) represents the scene depth with the position coordinates (x, y) in the mth area; Step 2: For a given foggy image, the relationship between the pixel average value of each area and its scene depth can be expressed mathematically as follows: In the above formula, m∈(1,2,…,n), represents the average scene reflectivity of the mth region in a foggy scene, Ω m represents the set of pixels in the mth region, |Ω m | indicates Ω m The number of pixels in (x,y)∈Ω m Represents Ω m The position coordinates of the pixel point in I c (x, y) represents the reflectivity of the scene at the position coordinate (x, y) in the c channel in a foggy scene, c∈{R,G,B} represents the red, green, and blue channels, represents the average scene depth of the mth region, and ∝ represents the average scene reflectivity of the mth region in a foggy scene. and the average scene depth of the mth region Directly proportional relationship; Step 3: Combining the above formula (1) and formula (2), the average scene reflectance of the mth area in the fog-free scene is And the average scene reflectivity of the mth area in the foggy scene The relationship between can be expressed mathematically as: In the above formula, represents the average scene reflectivity of the mth area in a fog-free scene, represents the average scene reflectivity of the mth area in a foggy scene, is the scene reflectance of the area with minimum scene depth, and k represents the slope.

2. The image defogging method based on region line prior according to claim 1, characterized in that: First, assuming that the atmospheric transmittance is the same in the same area, represents the average atmospheric transmittance of the mth region. The above formula (1) can be further simplified as: In the above formula, m∈(1,2,…,n), (x,y) represents the position coordinates of the original image I, A represents the atmospheric light value, and ρ(x,y) represents the scene reflectivity of the position coordinates (x,y) in a fog-free scene. represents the average atmospheric transmittance of the mth region, m∈(1,2,…,n); Then, take the mean value on both sides of formula (5), and then formula (5) is transformed into: In the above formula, represents the average scene reflectivity of the mth region in a foggy scene, represents the average scene reflectivity of the mth area in a fog-free scene, is the average atmospheric transmittance of the mth region, represents the average value of atmospheric light; Secondly, combining formula (4) and formula (6), the average atmospheric transmittance of the mth region is It can be expressed as: In the above formula, is the average atmospheric transmittance of the mth region, represents the average scene reflectivity of the mth region in a foggy scene, is the scene reflectance of the area with the minimum scene depth, A represents the atmospheric light value, represents the average value of atmospheric light, and k represents the slope; Next, substitute formula (7) into formula (5) to obtain the following formula: In the above formula, ρ(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a fog-free scene. represents the average scene reflectivity of the mth region in a foggy scene, is the scene reflectance of the area with the minimum scene depth, A represents the atmospheric light value, represents the average value of atmospheric light, k represents the slope, and I(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a foggy scene; Then, given I(x,y), we have and It can be calculated by formula (3); Finally, the above formula (8) is further rewritten as an expression formula (9) containing only three parameters: ρ(x,y) = R m (k, A, I) Equation (9) In the above formula, ρ(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a fog-free scene, and R m (·) represents the scene restoration formula for the mth region, k represents the slope, A represents the atmospheric light value, and I represents the scene reflectivity in a foggy scene. By using formula (9), if the slope k and the atmospheric light value A are determined, the final defogging result is obtained.

3. The image defogging method based on region line prior according to claim 2, characterized in that: The image processing module includes a joint optimization module, through which the optimal slope k and the atmospheric light value A are obtained; The joint optimization module includes a first constraint model F1 and a second constraint model F2; The expression of the first constraint model F1 is: In the above formula, Ψ(·) represents the averaging operation, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in the foggy scene, and the average brightness of the image in the fog-free scene approaches a specific value μ, R m (·) represents the scene restoration formula for the mth region, and the minimum value of formula (10) corresponds to the optimal slope k and atmospheric light value A; The expression of the second constraint model F2 is: In the above formula, Φ(·) represents the calculated information loss rate, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in the foggy scene, and R m (·) represents the scene restoration formula of the mth region; Combining the above formula (10) and formula (11), the optimal slope k p and the optimal atmospheric light value A p It can be expressed as: In the above formula, k p is the optimal slope, A p is the optimal atmospheric light value, k represents the slope, and A represents the atmospheric light value.

4. The image defogging method based on region line prior according to claim 3, characterized in that: The image processing module includes a scene restoration module, and scene restoration is performed by the scene restoration module, specifically including the following steps: First, the atmospheric light value A is defined as: A = τ·Ο = τ·[O R , O G , O B Formula (13) In the above formula, τ represents the amplitude of the atmospheric light value A, and Ο represents the color direction, where Ο can be calculated using the white point method; Then, combining formula (13) and formula (14), the optimal slope k is p and the amplitude τ of the optimal atmospheric light value A p It can be expressed by formula (14): Finally, the original image I is downsampled, and the expression of the first constraint model F1 is converted to: In the above formula, Ψ(·) represents the averaging operation, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in a foggy scene, ↓ω represents the downsampling operation with the coefficient ω, and the average brightness of the image in a fog-free scene approaches a specific value μ, R m (·) represents the scene restoration formula of the mth region; The expression of the second constraint model F2 is converted to: In the above formula, Φ(·) represents the information loss rate, k represents the slope, A represents the atmospheric light value, I represents the scene reflectivity in a foggy scene, ↓ω represents the downsampling operation with the coefficient ω, and R m (·) represents the scene restoration formula for the mth region.

5. The image defogging method based on region line prior according to claim 4, characterized in that: The coordinate descent method is used to solve the problem, and the optimal values ​​of the slope k and the amplitude τ of the atmospheric light value A are obtained alternately until convergence; After J iterations, the solution of formula (14) can be expressed as: The above formulas (17) and (18) are solved by the Fibonacci algorithm.

6. The image defogging method based on region line prior according to claim 5, characterized in that: Initialize k1=0.5 and set the stopping criterion to: δ(j)=|τ j -t j-1 |+|k j -k j-1 |≤∈=10 -3 official(19) In the above formula (19), τ j is the j-th iteration τ, τ j-1 is the τ of the j-1th iteration, k j is the k of the jth iteration, k j-1 is the k of the j-1th iteration. When the iteration is completed, the atmospheric light value A and the slope k can be calculated through τ j and k j Calculate them separately.

7. The image defogging method based on region line prior according to claim 6, characterized in that: By the optimal value k p and A p , and traverse all regions through formula (8) to obtain the final dehazing result.

8. The image defogging method based on region line prior according to claim 7, characterized in that: In the scene restoration module, guided filtering is used to refine the transmittance, and the final scene restoration model is expressed as: In the above formula, ρ(x,y) represents the reflectivity of the scene at the position coordinate (x,y) in a fog-free scene, and t r is a rough transmittance diagram, CR(·) represents guided filtering, A p represents the optimal atmospheric light value, and I(x,y) represents the scene reflectivity at the position coordinate (x,y) in a foggy scene; The scene restoration module processes the original image I through the above steps to obtain the defogging image I ‘ The corresponding mathematical model ρ(x,y) is used to obtain the defogging image I of the original image I ‘ , and finally output by the image output module.

Images