Polarization dehazing method for atmospheric light based on frequency domain feature separation and iterative optimization

Abstract

The application discloses a polarization defogging method for atmospheric light based on frequency domain feature separation and iterative optimization, and relates to the technical field of image defogging. The method comprises the following steps: acquiring a non-polarization image and multiple polarization images with different polarization angles under the same target scene; acquiring a low-frequency image of the polarization image through multi-scale transformation; acquiring an atmospheric scattering light image after median filtering of the low-frequency image; constructing a Stokes vector by using the atmospheric scattering light image, and acquiring total light intensity of the atmospheric scattering light; constructing a physical model of fog imaging, and acquiring a reconstructed defogging image; performing NSP decomposition on the reconstructed defogging image to obtain a high-frequency image, and performing normalization processing on the high-frequency image; constructing a constraint function between the total light intensity of the atmospheric scattering light and the normalized high-frequency image, and acquiring an optimized defogging image by iteratively optimizing the total light intensity of the atmospheric scattering light. The high-frequency image of the defogging image is used to iteratively optimize the atmospheric light, and the polarization defogging effect is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image defogging, and more particularly to a polarization defogging method based on frequency domain feature separation and iterative optimization of atmospheric light. Background Art

[0002] Defogging technology, which enhances the contrast of target imaging in foggy weather, improves image quality, and enhances visibility, has very important and long-term application value in many fields, including outdoor video surveillance, daily photo processing, aerial photography and underwater image processing, as well as existing safety assisted driving systems for automobiles and ships.

[0003] After long-term development, three major categories of image dehazing methods have emerged: those based on image enhancement, those based on physical models, and those based on deep learning. Image enhancement methods are limited in effectiveness because they do not consider the underlying causes of image degradation, while emerging deep learning-based methods are limited by training sets, and their robustness in real-world applications needs to be improved. Physical model-based dehazing methods include single-image dehazing methods based on priors, such as the dark channel dehazing method proposed by He et al., and dehazing methods based on multiple polarization images, such as the differential polarization dehazing method proposed by Schechner et al. Compared to conventional imaging techniques, polarization imaging offers significant advantages because it can capture additional spectral information.

[0004] However, existing polarization dehazing methods ignore the target's polarization characteristics, so they all require manual adjustment of parameters such as the polarization correction factor to obtain the best dehazing effect under different haze concentrations. Relying solely on the spatial domain information of the polarization image, it is difficult to accurately separate the atmospheric scattered light and the target reflected light.

[0005] Therefore, how to improve the stability and effectiveness of polarization defogging and improve the quality of defogging images is an urgent problem that needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a polarization defogging method based on frequency domain feature separation and iterative optimization of atmospheric light to enhance the contrast of target imaging in foggy and hazy weather, improve visibility, and improve the quality of defogging images.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A polarization dehazing method for atmospheric light based on frequency domain feature separation and iterative optimization includes:

[0009] Acquire a non-polarized image and multiple polarized images at different polarization angles under the same target scene;

[0010] Obtain the low-frequency image corresponding to each polarization image through multi-scale transformation;

[0011] Perform median filtering on each low-frequency image to obtain the atmospheric scattered light image corresponding to each low-frequency image;

[0012] The atmospheric scattered light image corresponding to each low-frequency image is used to construct the Stokes vector of the atmospheric scattered light, and the total intensity of the atmospheric scattered light is calculated based on the Stokes vector of the atmospheric scattered light.

[0013] A physical model of foggy imaging is constructed based on the unpolarized image and the total intensity of atmospheric scattered light, and a reconstructed defogging image is obtained based on the physical model of foggy imaging.

[0014] Perform NSP decomposition on the reconstructed dehazed image to obtain the corresponding high-frequency image, and normalize the high-frequency image to obtain a normalized high-frequency image;

[0015] A constraint function between the total intensity of atmospheric scattered light and the normalized high-frequency image is constructed, and the optimized dehazed image is obtained by iteratively optimizing the total intensity of atmospheric scattered light.

[0016] Preferably, the multiple polarization images with different polarization angles include polarization images I0, I1 at three polarization angles of 0°, 60°, and 120° under the same target scene. 60 , I 120 .

[0017] Preferably, obtaining the corresponding low-frequency image of each polarization image through multi-scale transformation specifically includes:

[0018] The NSP transformation is used to decompose each polarization image, with the number of decomposition layers being four, to obtain the corresponding low-frequency image after decomposition of each polarization image.

[0019] Preferably, the atmospheric scattered light image corresponding to each low-frequency image is used to construct the Stokes vector of the atmospheric scattered light. The specific expression includes:

[0020]

[0021] Where S A Stokes vector representing atmospheric scattered light; S AI Indicates the total intensity of atmospheric scattered light; S AQ 、S AU Represents the intensity difference of two linear polarizations of atmospheric scattered light; A0, A 60 、A 120 These represent the atmospheric scattered light images after median filtering of the low-frequency images of the 0°, 60°, and 120° polarization images.

[0022] Preferably, a physical model of foggy imaging is constructed based on the unpolarized image and the total intensity of atmospheric scattered light, and a reconstructed defogged image is obtained based on the physical model of foggy imaging, specifically including:

[0023]

[0024] Where J represents the reconstructed defogging image, I represents the non-polarized image of the target scene, and S AI Indicates the total intensity of atmospheric scattered light, A ∞ Represents the intensity of atmospheric scattered light at infinity.

[0025] Prior to this, the normalization formula for normalizing high-frequency images specifically includes:

[0026]

[0027] H represents the normalized high-frequency image; norm() represents the normalization function; H'(i) represents a pixel i in the high-frequency image; min(H') represents the minimum value of the pixel in the high-frequency image; max(H') represents the maximum value of the pixel in the high-frequency image.

[0028] Preferably, a constraint function is constructed between the total intensity of the atmospheric scattered light and the normalized high-frequency image:

[0029]

[0030] Where H represents the normalized high-frequency image, S AI represents the total intensity of atmospheric scattered light, Ω represents the edge area of ​​the total intensity of atmospheric scattered light image, x1 and y1 represent two pixel points in the edge area respectively, W(S AI ,H) is the weight function;

[0031] Using the high-frequency image H after the i-th normalization i Redefine the total intensity of the atmospheric scattered light (S AI ) i+1 :

[0032]

[0033] Where l represents the Lagrange multiplier, and the gradient descent method is used to solve the total intensity of atmospheric scattered light (S AI ) i+1 , the convergence condition is ||(S AI ) i+1 -(S AI ) i ||<ε, ε represents the convergence threshold.

[0034] Preferably, the weight function expression is:

[0035]

[0036] Where σ represents the variance of the pixels in the edge region of the normalized high-frequency image H, and x2 and y2 represent two pixels in the edge region of the normalized high-frequency image H.

[0037] The above technical solution shows that the present invention provides a polarization dehazing method based on frequency domain feature separation and iterative optimization of atmospheric scattered light. This method optimizes the polarization dehazing parameter estimation process by fully utilizing frequency domain information without ignoring the target polarization characteristics but avoiding detailed discussion of them. Compared with the existing technology, it has the following advantages:

[0038] (1) The present invention utilizes the high-frequency image of the defogging image to iteratively optimize the total intensity of atmospheric scattered light, thereby improving the polarization defogging effect, improving the image quality, and facilitating information processing of images in hazy weather by various complex computer vision systems.

[0039] (2) The present invention uses the low-frequency image of the polarization image to obtain the total intensity of the atmospheric scattered light. Compared with the previous method of directly using spatial filtering to obtain the total intensity of the atmospheric scattered light, it effectively avoids the generation of the halo effect.

[0040] (3) The present invention realizes polarization defogging by directly constructing the Stokes parameters of atmospheric scattered light, avoiding the discussion of complex target polarization characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0042] Figure 1 A schematic diagram of a method flow chart provided in an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the NSP decomposition principle provided by an embodiment of the present invention;

[0044] Figure 3 A flowchart of the method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0046] The embodiment of the present invention discloses a polarization defogging method based on frequency domain feature separation and iterative optimization of atmospheric light, such as Figure 1-Figure 3 , which includes the following steps:

[0047] Step 1: Use polarization imaging technology to image the target scene and obtain three polarization images of the same scene. The polarization camera used (such as a focal plane polarization camera) is required to obtain foggy polarization images under three polarization states of 0°, 60°, and 120°, which are recorded as I0, I 60 , I 120 , and can simultaneously acquire the non-polarized image I.

[0048] Step 2: Obtain the low-frequency image of the polarization image through multi-scale transformation. The multi-scale transformation method required to be used is non-subsampling pyramid transformation (NSP). The decomposition layer number j = 4 is taken from I0, I 60 , I 120 The low-frequency part of the acquired polarization image is recorded as

[0049] Step 3: Perform spatial domain filtering on the low-frequency image to obtain three polarization state atmospheric scattered light images. The atmospheric scattered light image obtained by median filtering is recorded as A0, A 60 ,A 120 .

[0050] Here we follow the objective constraints of atmospheric scattered light, including:

[0051] ① Non-negativity. That is, every pixel in the atmospheric scattered light image is greater than or equal to 0.

[0052] ② The total intensity of the atmospheric scattered light must not exceed the total intensity of light received by the camera. That is, the atmospheric scattered light image at each point must not exceed the pixel value in the corresponding polarization image.

[0053] ③ Local consistency: The atmospheric scattered light images are approximately equal in areas with similar depth of field.

[0054] Step 4: Use the three atmospheric scattered light images to construct the Stokes vector of the atmospheric scattered light, and estimate the atmospheric light value A at infinity based on the definition of the polarization component of the atmospheric scattered light. ∞ The expression of the constructed Stokes vector is:

[0055]

[0056] Among them, S A Stokes vector representing atmospheric scattered light; S AI Indicates the total intensity of atmospheric scattered light; S AQ 、S AU Represents the intensity difference of two linear polarizations of atmospheric scattered light; A0, A 60 、A 120 These represent the atmospheric scattered light images after median filtering of the low-frequency images of the 0°, 60°, and 120° polarization images.

[0057] The median filter is a nonlinear smoothing technique that sets the intensity value of each image pixel to the median of all pixels within a certain neighborhood window. Specifically, a moving region (n×n) with an odd number of points is used, and the value at the center of the region is replaced by the median of all points within the window. The specific size of the region varies depending on the size of the polarization image being captured, so there is no standard requirement. In this example, a median filter window size of 199×199 is selected.

[0058] Calculate the polarization angle θ of atmospheric scattered light using the Stokes vector of atmospheric light A and polarization degree P A :

[0059]

[0060]

[0061] Polarization component of atmospheric light A P The definition of is:

[0062]

[0063] Get an A ∞ Matrix, find the pixel value in the matrix whose ratio to the unpolarized image I is greater than or equal to 0.95 and take the average value to get the atmospheric light intensity A at infinity ∞ .

[0064] Step 5: Construct a physical model of foggy imaging based on the unpolarized image and the total intensity of atmospheric scattered light to obtain the reconstructed defogging image J:

[0065]

[0066] Step 6: Perform NSP decomposition on the reconstructed dehazed image J to obtain a normalized high-frequency image, which is denoted as H. The normalization formula is:

[0067]

[0068] H represents the normalized high-frequency image; norm() represents the normalization function; H'(i) represents a pixel i in the high-frequency image; min(H') represents the minimum value of the pixel in the high-frequency image; max(H') represents the maximum value of the pixel in the high-frequency image. After normalization, each pixel of each high-frequency image is transformed into a value in [0,1].

[0069] Step 7: Construct a constraint function between the total intensity of atmospheric scattered light and the normalized high-frequency image. Obtain a more accurate reconstructed image by iteratively optimizing the total intensity of atmospheric scattered light until the iterative results converge. The final reconstructed image is obtained. The constraint function between the normalized high-frequency image H and the total intensity of atmospheric scattered light is:

[0070]

[0071] Where Ω is an edge region of the total intensity image of atmospheric scattered light, and x1 and y1 represent two pixel points in the edge region of the total intensity image of atmospheric scattered light. AI ,H) is the weight function, and its expression is:

[0072]

[0073] σ represents the variance of the pixels in the edge region of the normalized high-frequency image H, and x2 and y2 represent two pixels in the edge region of the normalized high-frequency image H.

[0074] Using the high-frequency image H after the i-th normalization i Redefine the total intensity of the atmospheric scattered light (S AI ) i+1 :

[0075]

[0076] l represents the Lagrange multiplier, and the gradient descent method can be used to solve the total intensity of the atmospheric scattered light (S AI ) i+1 , the convergence condition is ||(S AI ) i+1 -(S AI ) i ||<ε. Here, ε represents the set convergence threshold.

[0077] The edge region division in the present invention can be detected by some commonly used edge detection algorithms, but the specific region size varies according to the size of the polarization image taken. The present invention does not make a unified requirement. In this example, the size of Ω is 200×200.

[0078] In the present invention, the total intensity of atmospheric scattered light S AI The image represented and the normalized high-frequency image H are exactly the same size. The selected edge region Ω has the same coordinate position relative to the two images. However, the pixel amplitude of the coordinate point in H is different from that of S. AI It’s different.

[0079] In the embodiments of the present invention, "atmospheric scattered light" is also referred to as "atmospheric light". All processing objects in the present invention are matrices, such as polarized images, non-polarized images, high-frequency images, low-frequency images, total intensity of atmospheric scattered light, etc., all of which are processed and calculated on matrices.

[0080] Once the total atmospheric light intensity S AI Convergence means S AI It will not change much with the iteration. At this time, the defogging image obtained by using formula (11) will also not change much. Therefore, the convergence of atmospheric light intensity and the convergence of defogging image are equivalent. The defogging image obtained at this time is the final optimized defogging image.

[0081] In this embodiment of the present invention, the polarization angles of the three polarization images are 0°, 60°, and 120°. In other embodiments, the polarization angles of the three polarization images may also be other polarization angles, for example, 0°, 45°, and 90°, respectively. In this way, the corresponding parameters in the formula for calculating the Stokes vector of atmospheric scattered light in step 4 will also change accordingly. Theoretically, the polarization angles of the three polarization images can be any three non-repeating polarization angles within the range of [0, 180]. However, it is recommended to select from a range of 0°, 45°, 60°, 90°, 120°, and 135°, which can directly calculate trigonometric functions. Using a split-focal plane polarization camera can simultaneously capture polarization images at three of these angles.

[0082] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0083] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A polarization dehazing method for atmospheric light based on frequency domain feature separation and iterative optimization, characterized in that: The method comprises the following steps: Acquire a non-polarized image and multiple polarized images at different polarization angles under the same target scene; Obtain the low-frequency image corresponding to each polarization image through multi-scale transformation; Perform median filtering on each low-frequency image to obtain the atmospheric scattered light image corresponding to each low-frequency image; The atmospheric scattered light image corresponding to each low-frequency image is used to construct the Stokes vector of the atmospheric scattered light, and the total intensity of the atmospheric scattered light is calculated based on the Stokes vector of the atmospheric scattered light. A physical model of foggy imaging is constructed based on the unpolarized image and the total intensity of atmospheric scattered light, and a reconstructed defogging image is obtained based on the physical model of foggy imaging. Perform NSP decomposition on the reconstructed dehazed image to obtain the corresponding high-frequency image, and normalize the high-frequency image to obtain a normalized high-frequency image; Construct a constraint function between the total intensity of atmospheric scattered light and the normalized high-frequency image, and obtain the optimized defogging image by iteratively optimizing the total intensity of atmospheric scattered light; specifically, Construct a constraint function between the total intensity of atmospheric scattered light and the normalized high-frequency image: Where H represents the normalized high-frequency image, S AI represents the total intensity of atmospheric scattered light, Ω represents the edge area of ​​the total intensity of atmospheric scattered light image, x1 and y1 represent two pixel points in the edge area Ω respectively, W(S AI ,H) is the weight function; Using the high-frequency image H after the i-th normalization i Redefine the total intensity of the atmospheric scattered light (S AI ) i+1 : Where l represents the Lagrange multiplier, and the gradient descent method is used to solve the total intensity of atmospheric scattered light (S AI ) i+1 , the convergence condition is ||(S AI ) i+1 -(S AI ) i ||<ε, ε represents the convergence threshold.

2. The polarization defogging method according to claim 1, characterized in that: The multiple polarization images at different polarization angles include polarization images at three polarization angles of 0°, 60°, and 120° under the same target scene.

3. The polarization defogging method according to claim 1, wherein: The corresponding low-frequency image of each polarization image is obtained through multi-scale transformation, specifically including: The NSP transformation is used to decompose each polarization image, with the number of decomposition layers being four, to obtain the corresponding low-frequency image after decomposition of each polarization image.

4. The polarization defogging method according to claim 2, wherein: The Stokes vector of atmospheric scattered light is constructed using the atmospheric scattered light image corresponding to each low-frequency image. The specific expressions include: Where S A Stokes vector representing atmospheric scattered light; S AI Indicates the total intensity of atmospheric scattered light; S AQ 、S AU Represents the intensity difference of two linear polarizations of atmospheric scattered light; A0, A 60 、A 120 These represent the atmospheric scattered light images after median filtering of the low-frequency images of the 0°, 60°, and 120° polarization images.

5. The polarization defogging method according to claim 1, characterized in that: A physical model of foggy imaging is constructed based on the unpolarized image and the total intensity of atmospheric scattered light. The reconstructed defogging image is obtained based on the physical model of foggy imaging. Specifically, the following steps are involved: Where J represents the reconstructed defogging image, I represents the non-polarized image of the target scene, and S AI Indicates the total intensity of atmospheric scattered light, A ∞ Represents the intensity of atmospheric scattered light at infinity.

6. The polarization defogging method according to claim 1, characterized in that: The normalization formula for normalizing high-frequency images specifically includes: H represents the normalized high-frequency image; norm() represents the normalization function; H'(i) represents a pixel i in the high-frequency image; min(H') represents the minimum value of the pixel in the high-frequency image; max(H') represents the maximum value of the pixel in the high-frequency image.

7. The polarization defogging method according to claim 1, characterized in that: The weight function expression is: Where σ represents the variance of the pixels in the edge region of the normalized high-frequency image H, and x2 and y2 represent two pixels in the edge region of the normalized high-frequency image H.

Images