Single-frame Image Dehazing Method and Device Based on Saturation Line Prior

Abstract

The present invention provides a single-frame image defogging method and device based on saturation line prior. The saturation line prior is derived through an atmospheric scattering model, a local consistency assumption of transmittance, and a local monochromatic surface assumption of natural images. The saturation line prior is used to characterize an approximate linear relationship between the saturation component and the reciprocal of the luminance component of pixels in a local image block of a haze image normalized by atmospheric light. Based on the saturation line prior, saturation lines of each local image block in the single-frame image to be processed are constructed. The transmittance map of the single-frame image is determined according to the saturation lines of each local image block. The pixel value of the farthest region in the single-frame image is used as a reference value for global atmospheric light, and the single-frame image is scene-reconstructed in combination with the transmittance map. The present invention constructs an efficient single-frame image defogging framework using the saturation line prior, improves the performance of transmittance estimation, and promotes the generation of high-quality haze-free images.

Description

Technical Field

[0001] The present invention relates to the technical field of image dehazing, and more specifically, to a single-frame image dehazing method and device based on saturation line prior. Background Art

[0002] Due to the presence of suspended particles in the atmosphere, the reflected light of an object will be scattered and mixed with the atmospheric light before reaching the camera, causing image blurring and thus reducing the performance of subsequent tasks. Early dehazing methods were achieved by utilizing additional information from multiple images. Although these methods can obtain good dehazing effects, they require high information acquisition costs. Therefore, the dehazing problem for single-frame images has received extensive attention.

[0003] In the single-frame image dehazing task, a fast method is to directly increase the contrast of the image. However, due to the lack of constraints of the haze imaging model, the dehazing performance of this method is limited. The fusion-based method can effectively alleviate this problem. With the help of a good fusion framework, different enhanced images can be effectively fused to promote haze removal. However, this method is unstable in dark or thick haze areas, resulting in a significant decline in dehazing performance.

[0004] In order to achieve high-quality image restoration, the mainstream image dehazing methods can be mainly divided into two categories: dehazing methods based on image prior and dehazing methods based on convolutional neural network.

[0005] The methods based on image prior effectively alleviate the lack of information in the single-frame image input mode by introducing reasonable assumptions or priors. For example, the effectiveness of DCP (Dark Channel Prior) has been fully verified in most scenarios, with strong robustness, and with the help of the guided filter, the consumption of computing resources is also greatly reduced. CAP (Color Attenuation Prior) designs a trainable linear model to estimate the scene depth using the brightness and saturation of pixels in the blurred image. SBTE proposes three intensity functions to directly enhance the saturation value of each pixel. These methods jointly promote the continuous improvement of dehazing technology and alleviate the interference of haze on image information. However, the local correlation of pixels in haze images has not been fully emphasized and exploited, resulting in most methods having low color protection ability or weak detail restoration ability.

[0006] The powerful learning ability of convolutional neural networks, combined with a carefully designed network structure, has given rise to a series of dehazing techniques based on convolutional neural networks. For example, DehazeNet utilizes prior knowledge to enhance the network's ability to extract haze-related features. The MSCNN (Multi-scale Convolutional Neural Networks) network improves the estimation ability of the transmittance by using features from different scales and achieves excellent dehazing results. The AOD-Net (All-in-One Dehazing Network) integrates different parameters into an estimation formula, reducing the reconstruction error. These network-based dehazing techniques have obtained excellent dehazing results. However, the dehazing performance of these methods largely depends on the data characteristics of the training images, resulting in low generalization ability. Summary of the Invention

[0007] In view of this, to solve the above problems, the present invention provides a single-frame image dehazing method and device based on saturation line prior, and the technical solutions are as follows:

[0008] A single-frame image dehazing method based on saturation line prior, the method comprising:

[0009] Deriving a saturation line prior through an atmospheric scattering model, a transmittance local consistency assumption, and a natural image local monochromatic surface assumption, where the saturation line prior is used to characterize an approximate linear relationship between the saturation component and the reciprocal of the luminance component of pixels in a local image patch of a haze image normalized by atmospheric light;

[0010] Constructing saturation lines for each local image patch in the single-frame image to be processed based on the saturation line prior;

[0011] Determining the transmittance map of the single-frame image according to the saturation lines of each local image patch;

[0012] Taking the pixel value of the farthest region in the single-frame image as a reference value for global atmospheric light, and reconstructing the scene of the single-frame image in combination with the transmittance map.

[0013] Preferably, constructing saturation lines for each local image patch in the single-frame image to be processed based on the saturation line prior includes:

[0014] Screening the pixels of each local image patch by constraining the slope between pixels;

[0015] Calculating the saturation lines of each local image patch by using the least squares method based on the screened pixels.

[0016] Preferably, constructing the saturation lines of each local image block in the single-frame image to be processed based on the saturation line prior further includes:

[0017] Screening the saturation lines of each local image block by constraining the saturation line length and the number of selected pixels.

[0018] Preferably, determining the transmittance map of the single-frame image according to the saturation lines of each local image block includes:

[0019] Determining different local image blocks to which each pixel in the single-frame image belongs by dividing the single-frame image in different ways;

[0020] Determining the transmittance of each pixel according to the saturation lines of different local image blocks to which each pixel belongs.

[0021] Preferably, the method further includes:

[0022] Optimizing the transmittance map through boundary constraints and an adaptive transmittance lower limit.

[0023] A single-frame image defogging device based on saturation line prior, the device includes:

[0024] A derivation module, configured to derive a saturation line prior through an atmospheric scattering model, a transmittance local consistency assumption, and a natural image local monochromatic surface assumption, where the saturation line prior is used to characterize an approximate linear relationship between the saturation component and the reciprocal of the luminance component of pixels in a local image block of a hazy picture normalized by atmospheric light;

[0025] A construction module, configured to construct the saturation lines of each local image block in the single-frame image to be processed based on the saturation line prior; and determine the transmittance map of the single-frame image according to the saturation lines of each local image block;

[0026] A reconstruction module, configured to use the pixel value of the farthest region in the single-frame image as a reference value of the global atmospheric light, and perform scene reconstruction on the single-frame image in combination with the transmittance map.

[0027] Preferably, the construction module for constructing the saturation lines of each local image block in the single-frame image to be processed based on the saturation line prior is specifically configured to:

[0028] Screen the pixels of each local image block by constraining the slope between pixels; and calculate the saturation line of each local image block by using the least squares method based on the screened pixels.

[0029] Preferably, the construction module for constructing the saturation lines of each local image block in the single-frame image to be processed based on the saturation line prior is further configured to:

[0030] The saturation lines of each local image patch are screened by restricting the length of the saturation line and the number of selected pixels.

[0031] Preferably, the construction module for determining the transmittance map of the single-frame image based on the saturation lines of each local image patch is specifically configured to:

[0032] By dividing the single-frame image in different ways, determine different local image patches to which each pixel in the single-frame image belongs; and determine the transmittance of each pixel according to the saturation lines of different local image patches to which each pixel belongs.

[0033] Preferably, the construction module is further configured to:

[0034] Optimize the transmittance map through boundary constraints and an adaptive transmittance lower limit.

[0035] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0036] The present invention provides a single-frame image dehazing method and device based on saturation line prior. The saturation line prior is derived through an atmospheric scattering model, a transmittance local consistency assumption, and a natural image local monochromatic surface assumption. The saturation line prior is used to characterize an approximate linear relationship between the saturation component and the reciprocal of the luminance component of pixels in a local image patch of a haze image normalized by atmospheric light. Based on the saturation line prior, the saturation lines of each local image patch in the single-frame image to be processed are constructed. The transmittance map of the single-frame image is determined according to the saturation lines of each local image patch. The pixel value in the farthest area of the single-frame image is used as a reference value for the global atmospheric light, and the single-frame image is scene-reconstructed in combination with the transmittance map. The present invention constructs an efficient single-frame image dehazing framework using the saturation line prior, estimates the overall transmittance map of the single-frame image by constructing saturation lines for each local image patch, improves the performance of transmittance estimation, and promotes the generation of high-quality haze-free images. Description of the Drawings

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0038] Figure 1 It is the flowchart of the method for the single-frame image dehazing method based on saturation line prior provided by the embodiment of the present invention;

[0039] Figure 2Schematic diagram of the saturation line structure provided by the embodiments of the present invention;

[0040] Figure 3 Schematic diagram of the saturation line structure in complex scenarios provided by the embodiments of the present invention;

[0041] Figure 4 Schematic diagram of different partitioning methods of local image blocks provided by the embodiments of the present invention;

[0042] Figure 5 Qualitative comparison diagram of the sky region noise suppression effect provided by the embodiments of the present invention;

[0043] Figure 6 Comparison diagram of defogging results when the size r of the local image block takes different values provided by the embodiments of the present invention;

[0044] Figure 7 Defogging result diagram of the saturation line prior in different types of haze scenarios provided by the embodiments of the present invention;

[0045] Figure 8 Qualitative comparison diagram of the saturation line prior method and other saturation-based defogging methods provided by the embodiments of the present invention;

[0046] Figure 9 Qualitative comparison diagram of the saturation line prior method and other defogging methods on real images provided by the embodiments of the present invention;

[0047] Figure 10 Qualitative comparison diagram of the saturation line prior and other defogging methods on synthetic images provided by the embodiments of the present invention;

[0048] Figure 11 Schematic diagram of the structure of the single-frame image defogging device based on the saturation line prior provided by the embodiments of the present invention. Detailed implementation manners

[0049] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0050] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0051] The inventors have found through research that the saturation information in an image can help achieve high-quality image dehazing. However, existing saturation-based dehazing methods only focus on the saturation information of each pixel itself, while the advanced distribution characteristics of saturation between pixels remain to be developed and utilized.

[0052] In this regard, in the present invention, the inventors observed that for local pixels with the same surface reflectance coefficient in a local image block of a haze-free image, there is a linear relationship between their saturation component and the inverse of their brightness component in the corresponding haze image normalized by atmospheric light, and the intercept of the function image corresponding to this linear relationship on the saturation axis is exactly the saturation value of these pixels in the haze-free image. This feature is called the saturation line prior (SLP). Based on the saturation line prior, the present invention proposes a new single-frame image defogging method, which utilizes the intrinsic correlation between local pixels to achieve reliable construction of the saturation line, significantly improves the estimation performance of the transmission rate, and thus promotes color protection and detail restoration of haze scenes. A large number of qualitative and quantitative experiments show that this method is superior to various existing image defogging methods.

[0053] See also Figure 1 , Figure 1 The flowchart of the method for single-frame image defogging based on saturation line prior provided by the embodiment of the present invention. Figure 1 As shown in FIG, the single-frame image defogging method based on saturation line prior includes the following steps:

[0054] S10, a saturation line prior is derived through the atmospheric scattering model, the local consistency assumption of transmittance, and the local monochromatic surface assumption of natural images. The saturation line prior is used to characterize the approximate linear relationship between the saturation component and the inverse of the brightness component of the pixel in the local image block of the haze image normalized by atmospheric light.

[0055] In this embodiment of the present invention, the mathematical basis for the saturation line prior is derived through analysis and integration of a haze imaging model and a saturation line model. Specifically, based on the haze imaging model and combined with the assumption of local transmittance consistency and the assumption of local monochromatic surfaces in natural images, it is deduced that for local pixels with the same surface reflectance in a local image block of a haze-free image, a linear relationship exists between the saturation component and the inverse of their brightness component in the corresponding haze image normalized by atmospheric light.

[0056] In the specific implementation process, based on the haze imaging model, the saturation component and other formulas are substituted and normalization and other operations are performed. It can be deduced that for pixels with the same surface reflection coefficient in the local image block of the haze-free image, there is a linear relationship between the saturation component and the inverse of its brightness component in the corresponding haze image normalized by atmospheric light.

[0057] According to the widely used atmospheric scattering model currently, the haze image model can be expressed in the following form:

[0058] I(x,y) = J(x,y)·t(x,y) + A·(1 - t(x,y)) (1)

[0059] Wherein, I, J, A, t, (x,y) respectively represent the observed haze image, the clear image without haze, the background atmospheric light, the medium transmittance, and the pixel position. Among them, the medium transmittance can be modeled as:

[0060] t(x,y) = e -β·d(x,y) (2)

[0061] Wherein, β and d respectively represent the medium transmittance coefficient and the scene depth.

[0062] Based on the relevant research on natural image modeling and following the local monochromatic surface assumption of natural images, there is the following formula:

[0063]

[0064] Wherein, R, l(x,y), respectively represent the three-dimensional vector of the surface reflection coefficient, the one-dimensional vector describing the radiation intensity, and the surface radiation (a three-dimensional vector constant within a given local image patch).

[0065] For a given image K(x,y), its saturation component is:

[0066]

[0067] Wherein, K c (x,y) represents the component of K(x,y) on the color channel c, and c ∈ {r, g, b} indicates that the color channel c can take red, green, and blue.

[0068] Using the background atmospheric light A to normalize formula (1), and using I N , J N to represent the normalized haze image and the haze-free image respectively, we get:

[0069] I N (x,y) = J N (x,y)·t(x,y) + (1 - t(x,y)) (5)

[0070] For the normalized J N , within the local image patch Ω mentioned in formula (3), its saturation component is constant, that is, there is the following formula:

[0071]

[0072] in, Indicates J N The color saturation at the pixel location (x,y), Represents surface radiation The component on color channel c, A c Represents the component of background atmospheric light A on color channel c;

[0073] According to the above formula, after simplified derivation, the following formula can be obtained:

[0074]

[0075] in, represents I normalized by the background atmospheric light A N The color saturation at the pixel location (x,y), Indicates that in J N The constant saturation in the local image patch, represents the locally uniform transmittance, Indicates I N The value of the component in color channel c at pixel position (x,y).

[0076] According to formula (7), we can get Using this prior, the transmittance estimation can be transformed into the construction of the saturation line.

[0077] S20, constructing a saturation line of each local image block in the single-frame image to be processed based on the saturation line prior.

[0078] Based on the S10 step The linear relationship between them is used to construct the saturation line. Figure 2 , Figure 2 Schematic diagram of the saturation line structure provided by an embodiment of the present invention, where (a) is the input image, (b) to (e) are the saturation lines of the local image block, and (f) is the transmittance t sl , (g) is the defogging result. Figure 2 As shown in the figure, (a) shows two haze pictures taken, and some rectangular local areas are selected as local image blocks in the picture, and (b) to (e) show the pixel counts in each corresponding rectangular local area. and It can be seen that the image pixels and There is an approximate linear relationship between them, which verifies the conclusion in formula (7).

[0079] However, the inventors have also observed that not all pixels are located on a straight line, and there are pixel points deviating from the saturation line. This is because there are slight color differences between some pixels within the selected local image patch, resulting in the invalidity of the local monochromatic surface assumption for these pixels. In addition, the change in scene depth within the local image patch will also affect the local consistency of the transmittance, thereby compromising the robustness of the local consistency assumption of the transmittance. See Figure 3 , Figure 3 which is a schematic diagram of constructing the saturation line for complex scenes provided by an embodiment of the present invention. Among them, (a) is the input image, and (b) - (c) are the saturation lines of the local image patches. Although these effects are controllable in most cases, in some extreme cases, the interference caused by pixel deviation may deteriorate, as shown in (b) - (c) of Figure 3 . To ensure the smooth construction of the saturation line, we need to screen the pixels before constructing the saturation line.

[0080] In this regard, step S20, "Construct the saturation lines of each local image patch in the single-frame image to be processed based on the saturation line prior", can adopt the following steps:

[0081] Screen the pixels of each local image patch by constraining the slope between pixels;

[0082] Based on the screened pixels, calculate the saturation lines of each local image patch using the least squares method.

[0083] Since takes values in the range of [0, 1], and ranges in (0, 1), according to formula (7), the slope of the saturation line is between (-1, 0). Therefore, for any two pixels within a given local image patch, if they are both located on the saturation line, the slope of the saturation line formed by them must be between (-1, 0). For a given local image patch, taking any one pixel in this area as a reference pixel, there are two cases for the slope of the line formed with any other pixel: within the range of (-1, 0) or outside the range of (-1, 0). According to the proportion of the former type of pixels in the total number of pixels, it is possible to effectively distinguish whether the reference pixel is on the saturation line. Specifically, the higher the proportion, the more reliable it is to select this reference pixel to construct the saturation line. The specific method is as follows:

[0084] For the pixel (x i , y i ) in the local image patch Ω with a given size of r, the proportion of the number of other pixels that meet the slope requirement can be obtained through the following formula:

[0085]

[0086] where P(xi , y i ), which represents the proportion of pixels meeting the requirements to the total number of pixels; N is the total number of pixels in the local image block Ω, and F(·) is a function that returns 1 for input values in the range of (-1, 0) and 0 for other input values. The selection criterion for pixels is defined as:

[0087]

[0088] Among them, H is the set of selected pixels, and p is a preset parameter. In the present invention, p = 0.5 is taken to ensure that the slope of the line formed by the selected pixels and at least half of the pixels meets the requirements of the saturation line.

[0089] Finally, the present invention uses the least squares method to obtain the parameters k(x, y) and b(x, y) of the saturation line. As Figure 2 shown in (b) to (e) of Figure 3 and (b) to (c) of

[0090] above, the above process effectively eliminates the interference of most deviation pixels, which helps to achieve a more reliable construction of the saturation line.

[0091] Screen the saturation lines of each local image block by restricting the length of the saturation line and the number of selected pixels.

[0092] Specifically, the total number of selected pixels N H and the length L of the saturation line must meet the following requirements:

[0093]

[0094] Among them, and are two preset parameters. In the present invention, and are set, that is, the saturation lines with less than 10 pixel points or a length less than 0.1 are discarded. The calculation method of L is as follows:

[0095]

[0096] Finally, substituting the slope k of the saturation line constructed using the saturation line prior and its intercept b on the saturation axis into formula (7) can obtain the corresponding transmittance t sl = 1 + k / b.

[0097] S30 , determining a transmittance map of a single-frame image according to the saturation lines of each local image block.

[0098] In the embodiment of the present invention, for each local image block, after obtaining the slope k of the saturation line in the local image block and its intercept b on the saturation axis, the corresponding transmittance t can be obtained by substituting them into formula (7): sl =1+k / b, that is, the transmittance of each pixel in the local image block is obtained. Furthermore, the transmittance of each pixel in all local image blocks constitutes the transmittance map of a single frame image.

[0099] In addition, the present invention uses different methods to divide local image blocks to increase the diversity of local pixel combinations. In the specific implementation process, step S30 "determining the transmittance map of a single frame image based on the saturation lines of each local image block" can be implemented as follows:

[0100] Determining different local image blocks to which each pixel in the single-frame image belongs by dividing the single-frame image in different ways;

[0101] The transmittance of each pixel is determined according to the saturation lines of different local image blocks to which each pixel belongs.

[0102] See also Figure 4 , Figure 4 Schematic diagram of different division methods of local image blocks provided by an embodiment of the present invention, wherein (a) is the input image, (b) is the division method of different local image blocks, and (c) is the transmittance t sl , (d) is the defogging result. Figure 4 As shown, for Figure 4 The same rectangular area in (a) Figure 4 (b) uses two local image block partitioning methods: upper and lower. Consider any pixel in the rectangular area of 4(a). It will be divided into different local image blocks in the upper and lower partitioning methods of 4(b), thereby generating different local pixel combinations. Therefore, for each pixel, its transmittance can be taken as the average of the transmittances calculated twice in the two local image blocks.

[0103] In addition, the inventors found that the saturation line prior method proposed in the present invention is applicable to most scenarios, but in some extreme cases, calculation failures may still occur. Failures mainly occur in two types of areas:

[0104] 1) Sky areas with saturation values close to zero;

[0105] 2) Non-sky areas with small color changes, such as river surfaces.

[0106] To perform effective defogging in these regions, the present invention introduces boundary constraints to provide necessary transmittance values for these regions. Specifically, the embodiments of the present invention further include the following steps:

[0107] Optimize the transmittance map through boundary constraints and an adaptive lower limit of transmittance.

[0108] The specific formula is as follows:

[0109]

[0110] where t b (x,y) represents the value of the transmittance obtained by using boundary constraints at the pixel position (x,y), and I c (x,y) represents the value of the component of the image I in color channel c at the pixel position (x,y). and are the color channels of the lower boundary B0 and the upper boundary B1 respectively. The boundary values adopted in the present invention are B0 = [20, 20, 20] and B1 = [300, 300, 300]. Meanwhile, in order to be consistent with step S2, using the local consistency constraint formula (12) of transmittance, there is the following formula:

[0111]

[0112] where Ω(x,y) represents the local image block centered at the pixel position (x,y). represents the maximum value of t b (x,y) in the local image block Ω(x,y).

[0113] Therefore, the transmittance value obtained through the saturation line prior and boundary constraints can be expressed by the following formula:

[0114]

[0115] where t sl (x,y) represents the value of the transmittance obtained by using the saturation line prior at the pixel position (x,y), ifEq.10ismet represents if formula (10) holds at the pixel position (x,y), and t f (x,y) represents the value of the transmittance obtained by combining the saturation line prior and boundary constraints at the pixel position (x,y).

[0116] The introduction of boundary constraints ensures the robustness of image defogging in each region and improves the reliability of the method. In addition, considering that extremely low transmittance values in the sky region are likely to cause serious noise amplification, the present invention limits the lower limit of the obtained transmittance value, which can be expressed by the following formula:

[0117] tr (x, y) = max(t f (x, y), t min ) (15)

[0118] where t min is the average value of the transmittance values t sl for the lowest ε%. t r (x, y) is the value of the optimized transmittance at the pixel position (x, y). Since t sl mainly corresponds to the transmittance values of most non-sky regions, the minimum value of t sl usually corresponds to the transmittance value of the farthest non-sky region. Setting these values as the lower limit helps to avoid noise amplification in the sky region while avoiding harm to the defogging performance of non-sky regions. In the present invention, we take ε = 5, that is, take the average value of the lowest 5% of the values of t sl as t min . Compared with the traditional method of specifying a fixed transmittance lower limit for all images, the t min of the present invention depends on the information of the image itself, and each image has a different lower limit. Therefore, this adaptive t min helps to obtain a more appropriate lower limit value for each image. See Figure 5 , Figure 5 is a qualitative comparison diagram of the sky region noise suppression effect provided by the embodiment of the present invention. Among them, (a) is the input image; (b) is the defogging result based on the traditional fixed transmittance lower limit; (c) is the defogging result of the adaptive transmittance lower limit proposed by the present invention. As Figure 5 shown, the method of the present invention can restore a defogging result with higher quality.

[0119] That is to say, in order to ensure the effective implementation of defogging in the sky region with saturation close to zero and non-sky regions with small color changes, the present invention introduces boundary constraints and an adaptive transmittance value lower limit to optimize the transmittance map, thereby suppressing noise amplification in the sky region. Finally, scene restoration is performed based on the atmospheric scattering model.

[0120] S40. Use the pixel value of the farthest region in the single-frame image as the reference value of the global atmospheric light, and combine the transmittance map to perform scene reconstruction on the single-frame image.

[0121] In the embodiment of the present invention, during scene restoration, the pixel value of the farthest region can be used as the reference value of the global atmospheric light, combined with the obtained transmittance map, and scene restoration is performed according to formula (1). The specific formula is as follows:

[0122]

[0123] where is to use guided filtering for t r(x, y)-processed refined transmittance map.

[0124] The single-frame image defogging method based on saturation line prior provided by the embodiments of the present invention has the following advantages:

[0125] The present invention proposes a simple but effective saturation line prior (SLP), which reveals the linear relationship between the saturation component and the reciprocal of the luminance component in the local image patches of the haze image normalized by the atmospheric light.

[0126] The present invention proposes a single-frame image defogging framework based on saturation line prior. This framework can more effectively utilize the intrinsic correlation between local pixel points during the estimation of transmittance.

[0127] The present invention proves that the proposed method can achieve excellent defogging effects in scenes with different haze concentrations, and its defogging performance on real and synthetic images is better than that of the current state-of-the-art defogging methods.

[0128] Result analysis: Here, the defogging performance of the method proposed by the present invention will be comprehensively evaluated.

[0129] Determination of local image patch size: The construction of the saturation line is based on the pixels in the local image patch. For the size r of the image patch, on the one hand, the larger r is, the more pixel points there are in the image patch, which helps to utilize more correlation information between pixels. On the other hand, when r is too large, the validity of formula (3) in step S10 may be damaged, and the local consistency of the transmittance cannot be guaranteed, thus endangering the robustness of the saturation line prior. To find the most suitable image patch size, defogging experiments are carried out with different values of r, and FADE is used as the evaluation of the defogging result recovery quality. The smaller the FADE value, the better the recovery quality. See Figure 6 , Figure 6 is the comparison diagram of defogging results with different values of the local image patch size r provided by the embodiments of the present invention. Among them, (b) to (f) show the defogging results with different values of r, and (f) shows the corresponding FADE results. It can be seen that although r = 15 does not achieve the optimal recovery effect in all images, in most scenarios, r = 15 is the optimal choice. Therefore, the size of the local image patch is selected as r = 15 in the present invention.

[0130] Preliminary evaluation of the saturation line prior method (i.e., the single-frame image defogging method based on saturation line prior of the present invention): To verify the performance of the saturation line prior, see Figure 7 , Figure 7 is the defogging result diagram of the saturation line prior in different types of haze scenarios provided by the embodiments of the present invention. Among them, the first row is the haze image, the second row is the obtained transmittance map t sl , and the third row is the optimized transmittance map The fourth row shows the defogging result (exposure processing to reduce local dark effects). It can be seen that t sl In most non-sky regions, the transmittance consistent with objective reality is restored, and with the optimized The potential structure and hidden details in the haze image can be reliably restored, while effectively avoiding noise amplification and color distortion. The high-quality defogging result proves the effectiveness and reliability of the saturation line prior method.

[0131] Since the saturation line prior mainly uses saturation information to perform haze removal, see Figure 8 , Figure 8 This is a qualitative comparison chart of the saturation line prior method provided by the embodiments of the present invention and other saturation-based defogging methods (CAP, SBTE). As Figure 8 (a) shows, two haze pictures rich in structural and color information are used as inputs, and (b) - (c) respectively show the restoration results of the CAP, SBTE, and saturation line prior methods. It can be seen that both the CAP and SBTE methods fail to well restore the picture details in the far-distance region and generate more haze residues, while the saturation line prior method can better remove haze and restore the true colors, which shows the superiority of the saturation line prior method in defogging performance.

[0132] Comprehensive evaluation on real haze pictures: To comprehensively compare the defogging performance among different methods, see Figure 9 , Figure 9 This is a qualitative comparison chart of the saturation line prior method provided by the embodiments of the present invention and other defogging methods on real images. Among them, (a) shows 6 degraded pictures with different haze concentrations in different scenes, and (b) - (i) show the defogging results of different methods, including BCCR, CAP, DEFADE, DehazeNet, HL, SBTE, and IDE methods. As Figure 9 (b) shows, the BCCR method can effectively remove haze in most scenes, but it will cause color distortion; Figure 9 (c) shows, CAP cannot restore the details in the thick fog scene; as Figure 9 (d) shows, DEFADE improves the detail restoration ability, but when the haze concentration increases, the defogging performance drops sharply; as Figure 9 (e) shows, although DehazeNet produces results with relatively high color fidelity, the defogging effect is not good; as Figure 9 (f) shows, there is a problem of over-saturated color in the defogging result of HL; as Figure 9 (g) shows, the defogging effect of SBTE on the thick fog scene is limited; as Figure 9 (h) shows, the defogging result of IDE is excellent, but there is a problem of overexposure; as Figure 9(i) As shown, the saturation line prior method proposed by the present invention can clearly restore texture details under different haze concentrations and has high color fidelity. To facilitate quantitative analysis, two metrics, FADE and NIQE (the lower the better), were used as evaluation criteria to quantitatively compare the above method. The results are shown in Table 1. It can be seen that the method of the present invention obtained the best average scores in both criteria, indicating its advantage in defogging performance.

[0133] Evaluation on the synthetic haze image dataset: Comparisons were made on three synthetic datasets, SOTS, D-HAZY, and O-HAZE. The STOS indoor dataset includes 500 indoor haze images and their corresponding ground-truth pictures. The Middlebury part of the D-HAZY dataset has 23 pairs of high-quality indoor haze images and ground-truth images. The O-HAZE dataset consists of 45 pairs of outdoor haze images and their ground-truth. PSNR and SSIM were used as evaluation criteria (the higher the better). Table 2 shows the average values of the evaluation results of various defogging methods on these three datasets. According to Table 2, although DehazeNet, BCCR, CAP, HL, and SBTE can obtain high scores in the single evaluation criteria of specific datasets, they cannot maintain their advantages on other datasets. In contrast, the method of the present invention obtained the optimal or sub-optimal values in the vast majority of metrics of the three datasets. In addition, for qualitative comparison, see Figure 10 , Figure 10 which is the qualitative comparison diagram of the saturation line prior of the embodiment of the present invention and other defogging methods on synthetic images. As Figure 10 shown, an example of each dataset and the defogging results of each method are given. Through qualitative and quantitative comparisons, the method of the present invention can obtain satisfactory defogging effects in all haze scenarios, demonstrating its effectiveness and robustness.

[0134] Table 1

[0135]

[0136]

[0137] Table 2

[0138]

[0139] Based on the single-frame image defogging method based on saturation line prior provided in the above embodiment, the embodiment of the present invention correspondingly provides a device for executing the above single-frame image defogging method based on saturation line prior. The structural schematic diagram of the device is as Figure 11 shown, including:

[0140] A derivation module 10 is configured to derive a saturation line prior by using an atmospheric scattering model, a local consistency assumption of transmittance, and a local monochromatic surface assumption of natural images. The saturation line prior is used to characterize an approximate linear relationship between the saturation component and the reciprocal of the luminance component of pixels in a local image patch of a haze image normalized by atmospheric light.

[0141] A construction module 20 is configured to construct saturation lines for each local image patch in a single-frame image to be processed based on the saturation line prior; and determine a transmittance map of the single-frame image according to the saturation lines of each local image patch.

[0142] A reconstruction module 30 is configured to use the pixel values in the farthest region of the single-frame image as a reference value of global atmospheric light, and perform scene reconstruction on the single-frame image in combination with the transmittance map.

[0143] Optionally, the construction module 20 for constructing saturation lines for each local image patch in a single-frame image to be processed based on the saturation line prior is specifically configured to:

[0144] Screen the pixels of each local image patch by constraining the slope between pixels; and calculate the saturation line of each local image patch by using the least squares method based on the screened pixels.

[0145] Optionally, the construction module 20 for constructing saturation lines for each local image patch in a single-frame image to be processed based on the saturation line prior is further configured to:

[0146] Screen the saturation lines of each local image patch by constraining the saturation line length and the number of selected pixels.

[0147] Optionally, the construction module 20 for determining the transmittance map of the single-frame image according to the saturation lines of each local image patch is specifically configured to:

[0148] Determine different local image patches to which each pixel in the single-frame image belongs by performing different partitioning methods on the single-frame image; and determine the transmittance of each pixel according to the saturation lines of the different local image patches to which each pixel belongs.

[0149] Optionally, the construction module 20 is further configured to:

[0150] Optimize the transmittance map by boundary constraints and an adaptive transmittance lower limit.

[0151] It should be noted that for the refined functions of each module in the embodiments of the present invention, reference may be made to the corresponding disclosed parts in the embodiments of the single-frame image defogging method based on the saturation line prior, which will not be elaborated herein.

[0152] Based on the single-frame image dehazing method using saturation line prior provided in the above embodiments, an embodiment of the present invention provides an electronic device, which includes: at least one memory and at least one processor; the memory stores an application program, and the processor calls the application program stored in the memory, and the application program is used to implement the single-frame image dehazing method using saturation line prior.

[0153] Based on the single-frame image dehazing method using saturation line prior provided in the above embodiments, an embodiment of the present invention provides a storage medium, which stores computer program code, and when the computer program code is executed, it implements the single-frame image dehazing method using saturation line prior.

[0154] The above has introduced in detail a single-frame image dehazing method and device provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

[0155] It should be noted that the various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0156] It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements inherent to the process, method, article or device, but also other elements inherent to these process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0157] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A single-frame image dehazing method based on the prior of the saturation line, characterized in that The method includes: Deriving a saturation line prior through an atmospheric scattering model, a local consistency assumption of transmittance, and a local monochromatic surface assumption of natural images, where the saturation line prior is used to characterize an approximate linear relationship between the saturation component and the reciprocal of the luminance component of pixels in a local image patch of a haze image normalized by atmospheric light; Constructing saturation lines for each local image patch in a single-frame image to be processed based on the saturation line prior; Determining a transmittance map of the single-frame image according to the saturation lines of each local image patch; Using the pixel values in the farthest region of the single-frame image as a reference value for global atmospheric light, and performing scene reconstruction on the single-frame image in combination with the transmittance map.

2. The method according to claim 1, characterized in that, The constructing the saturation lines for each local image patch in a single-frame image to be processed based on the saturation line prior includes: Screening the pixels of each local image patch by constraining the slope between pixels; Calculating the saturation lines of each local image patch using the least squares method based on the screened pixels.

3. The method according to claim 2, wherein The constructing the saturation lines for each local image patch in a single-frame image to be processed based on the saturation line prior further includes: Screening the saturation lines of each local image patch by constraining the length of the saturation line and the number of selected pixels.

4. The method according to claim 1, characterized in that The determining the transmittance map of the single-frame image according to the saturation lines of each local image patch includes: Determining different local image patches to which each pixel in the single-frame image belongs by dividing the single-frame image in different ways; Determining the transmittance of each pixel according to the saturation lines of different local image patches to which each pixel belongs.

5. The method according to claim 1, characterized in that The method further includes: Optimizing the transmittance map through boundary constraints and an adaptive transmittance lower limit.

6. A single-frame image defogging device based on a saturation line prior, characterized in that The apparatus includes: A derivation module for deriving a saturation line prior through an atmospheric scattering model, a local consistency assumption of transmittance, and a local monochromatic surface assumption of natural images, where the saturation line prior is used to characterize an approximate linear relationship between the saturation component and the reciprocal of the luminance component of pixels in a local image patch of a haze image normalized by atmospheric light; A construction module for constructing saturation lines for each local image patch in a single-frame image to be processed based on the saturation line prior; and determining the transmittance map of the single-frame image according to the saturation lines of each local image patch; A reconstruction module for using the pixel values in the farthest region of the single-frame image as a reference value for global atmospheric light, and performing scene reconstruction on the single-frame image in combination with the transmittance map.

7. The device according to claim 6, characterized in that, The construction module for constructing the saturation lines for each local image patch in a single-frame image to be processed based on the saturation line prior is specifically configured to: Screen the pixels of each local image patch by constraining the slope between pixels; and calculate the saturation lines of each local image patch using the least squares method based on the screened pixels.

8. The device according to claim 7, characterized in that The construction module for constructing the saturation lines for each local image patch in a single-frame image to be processed based on the saturation line prior is further configured to: Screen the saturation lines of each local image patch by constraining the length of the saturation line and the number of selected pixels.

9. The device according to claim 6, characterized in that, The construction module for determining the transmittance map of the single-frame image according to the saturation lines of each local image block is specifically configured to: Determine different local image blocks to which each pixel in the single-frame image belongs by dividing the single-frame image in different ways; determine the transmittance of each pixel according to the saturation lines of the different local image blocks to which each pixel belongs.

10. The device according to claim 6, characterized in that The construction module is further configured to: Optimize the transmittance map through boundary constraints and an adaptive transmittance lower limit.

Images