Polarization image defogging method based on double iteration regular separation

Abstract

The invention discloses a polarization image defogging method based on double iteration regular separation, and relates to the technical field of image defogging, and the method comprises the following steps: S1, obtaining a polarization image; s2, constructing a representation function of the fog concentration; s3, indirectly solving an intermediate variable T; s4, formula decomposition; s5, judging whether convergence occurs or not; and S6, finally defogging. When the polarization image defogging method based on double-iteration regular separation is used, non-uniformly distributed concentration characteristic differences are analyzed through a large number of polarization images in a non-uniform haze scene, on this basis, a dynamic fog concentration function in the image is effectively constructed in combination with polarization characteristics, estimation of an atmospheric scattering coefficient is achieved, and the defogging effect is improved. Meanwhile, a double-iteration separation algorithm is provided based on the atmospheric scattering coefficient, the optimal transmittance is solved, image sharpening restoration under the non-uniform haze scene is achieved, haze residues are further reduced, and the image restoration quality is also objectively improved.

Description

Technical Field

[0001] The invention relates to the technical field of image defogging, and in particular to a polarization image defogging method based on double-iteration regularization separation. Background Art

[0002] Haze particles absorb and scatter light, which can cause low contrast and visibility in outdoor imaging, which seriously affects human identification of outdoor targets and causes trouble for many visual activities. Therefore, it is of great research significance to study the restoration algorithm of foggy images to improve the image clarity and solve the problems caused by haze to urban transportation, military reconnaissance and other visual activities.

[0003] At present, there are three categories of defogging algorithms: image enhancement algorithms, deep learning algorithms, and image restoration algorithms. Image enhancement algorithms directly perform noise suppression and contrast enhancement on foggy images, such as histogram equalization, Retinex theory, wavelet transform, and homomorphic filtering. These algorithms are fast in defogging speed, but usually do not fully consider the essential mechanism of image blur caused by haze, thus failing to effectively retain important information in the image; deep learning algorithms use neural networks to defog images, build effective end-to-end defogging networks, and achieve image defogging. However, due to the lack of a large number of real public data sets, they cannot be well applied to actual scenes. The generalization ability of such algorithms in actual scenes is unstable, and their performance often depends on specific data sets; image restoration algorithms start from the mechanism level of image blur, establish an atmospheric scattering model, and combine some prior knowledge and assumptions to perform image defogging. For example, He et al. obtained a dark channel prior by studying several clear images, that is, in most non-sky areas, some pixels reach a very low intensity value in a certain color channel, approaching zero, and use this prior to solve the transmission map, and finally perform defogging. Dana et al. proposed non-local prior knowledge, and various colors in a clear image can form clusters in the color space, and use the fog lines formed by these clusters in the fog map to defog the image. With the development of polarization optical technology, image defogging combined with polarization information has gradually developed. For example, Liang et al. used polarization angle to automatically estimate the polarization degree of atmospheric light, thereby avoiding manual interference for defogging; Zhang et al. used the infinite atmospheric light value to perform polarization orthogonal blind separation, and indirectly constructed atmospheric light for image defogging.

[0004] Therefore, the current image restoration algorithms lack relevant research on the dynamic removal of non-uniform concentration distribution. Most of them perform image dehazing from the perspective of uniform fog concentration. However, haze is actually mostly non-uniformly distributed. At the same scene depth, there will be concentration differences in haze, which leads to the problem of residual haze in the current algorithms when dehazing in non-uniform scenes. Summary of the invention

[0005] The purpose of the present invention is to provide a polarization image defogging method based on double-iteration regularized separation to solve the problem of residual haze in defogging of non-uniform scenes in the current defogging algorithm proposed in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a polarization image defogging method based on double iterative regularization separation, comprising the following steps:

[0007] S1: Use polarization imaging technology to image the target scene and obtain three polarization images of the same scene;

[0008] S2: From a visual perspective, a function representing fog density is constructed based on image features of brightness, saturation and texture.

[0009] S3: Use the dual-iteration separation algorithm to indirectly obtain the local optimal scene depth, thereby obtaining the optimized transmittance;

[0010] S4: The optimal solution to split S3 into two sub-problems;

[0011] S5: Introduce auxiliary variables in S4 to transform the problem into a series of sub-problems, and combine semi-quadratic separation and FFT iterative optimization to find the optimal solution;

[0012] S6: Take the average value of the light intensity corresponding to the intersection position.

[0013] Preferably, in S1, when a beam of linearly polarized light with a Stokes vector of S=(I, Q, U) passes through a polarizer with a polarization angle of α, according to the relationship between the Stokes vector and the Mueller matrix, the polarized image light intensity I represented by the Stokes vector can be obtained. θ The expression is:

[0014]

[0015] Thus, the polarization image intensity I can be obtained θ The numerical value of .

[0016] Preferably, the polarization camera used in S1 can obtain foggy images under three polarization states of 0°, 60°, and 120°, which are denoted as I0, I 60 and I 120 , the calculated Stokes vectors are:

[0017]

[0018] Where I is the total intensity information of the incident light, Q and U are the polarization information of the incident light,

[0019] Then use the Stokes vector to calculate the polarization degree P and polarization angle θ. The calculation formulas are:

[0020]

[0021] The values ​​of the polarization degree P and the polarization angle θ can be calculated by the above formula.

[0022] Preferably, after constructing the fog density characterization function C according to the image features of brightness, saturation and texture in S2, the image I is converted into an HSV channel, the second and third channels are taken as the image brightness and saturation, respectively, denoted as w1 and w2, and the polarization angle is used as the texture feature, denoted as a2 and a3 are the weight coefficients of w1, w2, and w3, and then according to the fog concentration:

[0023] β=exp(c)=exp(a1w1+a2w2+a3w3)

[0024] Calculate the dynamic atmospheric scattering coefficient β.

[0025] Preferably, before using the double iterative separation algorithm to indirectly obtain the local optimal scene depth in S3, an atmospheric scattering model can be used, which can be expressed as:

[0026] I(x)=D+A=Jt+A ∞ (1-t)

[0027] Where x represents a pixel, I represents the foggy image captured by the camera, J represents the scene reflected light, and is the image to be restored based on the atmospheric scattering model. ∞ represents the atmospheric light intensity at infinity, and t represents the transmission rate, which can be further expressed as:

[0028] t(x)=e -βd

[0029] For uniform haze scenes, β can be regarded as a constant related to the haze concentration, and d represents the distance from the scene point to the camera, which is called the depth of field and satisfies the non-negative property. Jt is considered to be the part of the scene reflected light that successfully reaches the camera without scattering, which can also be called direct transmission, denoted as D, and the second term represents the part of the global atmospheric light that reaches the camera after being scattered by the medium, denoted as A. The defogging goal based on this model is to estimate A from the obtained I based on the atmospheric scattering model formula. ∞ and t, whose expression is:

[0030]

[0031] Thus recovering J.

[0032] Preferably, in S3, T is set equal to βd, and then the double iterative separation algorithm can be used to indirectly obtain the local optimal scene depth d, thereby obtaining the optimal intermediate variable T, that is, indirectly solving T under the condition of known atmospheric scattering coefficient β and unknown scene depth d, thereby obtaining the optimized transmittance t, and constructing a regularization function to solve the optimal T and d, so as to achieve indirect acquisition of T from the atmospheric scattering coefficient, which is expressed as:

[0033]

[0034] Thus, the intermediate variable T can be indirectly obtained.

[0035] Preferably, the regularization function benefits from the exploration of the inherent boundary constraints of the transfer function, which is combined with the context regularization of the weighted L1 norm and is modeled as an optimization problem to estimate the transmission rate of unknown scenes. It is proposed to find the optimal t, which can be obtained by minimizing the following objective function, expressed as:

[0036]

[0037] in, is a data item, minimizing this item ensures that t and As close as possible, the second item, is the context constraint minimization of t, which can reduce the artifacts at the edge of the image. λ is the regularization parameter that balances these two terms. j is the adaptive weight matrix, D j is a high-order difference operator.

[0038] Preferably, the optimal solution of dividing the expression in S3 into two sub-problems in S4 is expressed as:

[0039]

[0040] Used to quickly find the optimal solution for T.

[0041] Preferably, S5 introduces auxiliary variables into the two groups of expressions decomposed by S4 to transform the problem into a series of sub-problems, and combines semi-quadratic separation and FFT iterative optimization to find the optimal solution of this series of sub-problems, so as to converge to the optimal solution of the original problem, that is, the T value can be obtained.

[0042] Preferably, in S6, I and The intersection of the coordinate sets of the top 5% of the concentration is regarded as the atmospheric light A at infinity. ∞ The position of A is optimal. ∞ The estimation of the value is more accurate, and finally the average value of the light intensity corresponding to the intersection position is taken as A ∞ , expressed as:

[0043]

[0044] Thus, the image can be dehazed.

[0045] Compared with the prior art, the beneficial effect of the present invention is that when the polarization image defogging method based on double iterative regularized separation is used, a large number of polarization images in non-uniform haze scenes are used to analyze the concentration characteristic differences of non-uniform distribution. On this basis, the dynamic fog concentration function in the image is effectively constructed in combination with the polarization characteristics, and the estimation of the atmospheric scattering coefficient is realized. At the same time, a double iterative separation algorithm is proposed based on the atmospheric scattering coefficient to solve the optimal transmittance, thereby realizing image clarity restoration in non-uniform haze scenes, further reducing the residual degree of haze, and being able to effectively clarify and restore fog images in non-uniform scenes, achieving better defogging effects, further reducing the residual haze, and objectively improving the quality of image restoration. This is the use feature of the polarization image defogging method based on double iterative regularized separation. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the process structure of the present invention;

[0047] Figure 2 It is a schematic diagram of the structure of the atmospheric scattering model of the present invention;

[0048] Figure 3 It is a schematic diagram of the structure of a block diagram of an implementation case of the present invention;

[0049] Figure 4 This is a schematic diagram of polarization 0° in an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of a polarization of 60° in an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of polarization at 120° according to an embodiment of the present invention;

[0052] Figure 7 This is a schematic diagram of the final defogging process of an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] See also Figure 1The present invention provides a technical solution: a polarization image defogging method based on double-iteration regularization separation, comprising the following steps:

[0055] S1: Use polarization imaging technology to image the target scene, obtain foggy images of the same scene at three polarization states of 0°, 60°, and 120°, and use the formula to calculate the polarization degree P;

[0056] S2: From a visual perspective, a function representing fog concentration is constructed based on the image features of brightness, saturation and texture. The image I is converted into an HSV channel, and the second and third channels are taken as image brightness and saturation respectively. The polarization angle is used as a texture feature, and the dynamic atmospheric scattering coefficient β can be obtained by matching the corresponding expression.

[0057] S3: Then use the double iterative separation algorithm to indirectly obtain the local optimal scene depth, that is, indirectly solve T when the atmospheric scattering coefficient β is known and the scene depth d is unknown, so as to obtain the optimized transmittance t;

[0058] S4: To facilitate the solution, the formula in S3 is divided into the optimal solutions of two sub-problems;

[0059] S5: Introduce auxiliary variables into the two sets of formulas in S4 to transform the problem into a series of sub-problems, and combine semi-quadratic separation and FFT iterative optimization to find the optimal solution;

[0060] S6: Take the average value of the light intensity corresponding to the intersection position according to the formula to obtain the defogging image.

[0061] Example

[0062] First, when a beam of linearly polarized light with a Stokes vector of S = (I, Q, U) passes through a polarizer with a polarization angle of α, according to the relationship between the Stokes vector and the Mueller matrix, the polarization image intensity I represented by the Stokes vector can be obtained. θ The expression is:

[0063]

[0064] Then, the target scene is imaged using polarization imaging technology to obtain three polarization images of the same scene. The polarization camera used is required to obtain foggy images under three polarization states of 0°, 60°, and 120°, which are denoted as I0, I 60 and I 120 , and the Stokes vectors calculated according to the above formula are:

[0065]

[0066] Where I is the total intensity information of the incident light, Q and U are the polarization information of the incident light,

[0067] The degree of polarization P is then calculated using the formula as the polarization angle θ, where the calculation formula for the degree of polarization P is:

[0068]

[0069] The calculation formula for the polarization angle θ is:

[0070]

[0071] From a visual perspective, the fog density representation function c is constructed based on the image features of brightness, saturation and texture. After converting the image I into HSV channels, the second and third channels are taken as image brightness and saturation, respectively, denoted as w1 and w2, and the polarization angle is used as the texture feature, denoted as a2 and a3 are the weight coefficients of w1, w2, and w3, and the dynamic atmospheric scattering coefficient β is obtained according to the fog concentration:

[0072] β=exp(c)=exp(a1w1+a2w2+a3w3)

[0073] Then according to the expression of the atmospheric scattering model:

[0074] I(x)=D+A=Jt+A ∞ (1-t)

[0075] Where x represents a pixel, I represents the foggy image captured by the camera, J represents the scene reflected light, and is the image to be restored based on the atmospheric scattering model. ∞ represents the atmospheric light intensity at infinity, and t represents the transmission rate, which can be further expressed as:

[0076] t(x)=e -βd

[0077] And based on the atmospheric scattering model formula, estimate A from the obtained I ∞ and t, thereby recovering J, whose expression is:

[0078]

[0079] Let T equal to βd, and then the double iterative separation algorithm can be used to indirectly obtain the local optimal scene depth d, thereby obtaining the optimal intermediate variable T, that is, indirectly solving T under the condition of known atmospheric scattering coefficient β and unknown scene depth d, so as to obtain the optimized transmittance t, and construct a regularization function to solve the optimal T and d. The expression of the regularization function is:

[0080]

[0081] in, is a data term. Minimizing this term ensures that t and ^t are as close as possible. The second term, is the context constraint minimization of t, which can reduce the artifacts at the edge of the image. λ is the regularization parameter that balances these two terms. j is the adaptive weight matrix, D j It is a high-order difference calculation.

[0082] Through the atmospheric scattering model and regularization function, T can be indirectly obtained from the atmospheric scattering coefficient, the expression is:

[0083]

[0084] In order to obtain the optimal solution of T, the above expression can be divided into the optimal solutions of two sub-problems, which are respectively expressed as:

[0085]

[0086] as well as:

[0087]

[0088] Auxiliary variables are introduced into the two sets of expressions decomposed above to transform the problem into a series of sub-problems. Combined with semi-quadratic separation and FFT iterative optimization, the optimal solution of this series of sub-problems is found. In this way, the optimal solution of the original problem is obtained by convergence, that is, the T value can be obtained.

[0089] Finally, select I and The intersection of the coordinate sets of the top 5% of the concentration is regarded as the atmospheric light A at infinity. ∞ The position of A is optimal. ∞ The estimation of the value is more accurate, and finally the average value of the light intensity corresponding to the intersection position is taken as A ∞ , expressed as:

[0090]

[0091] This achieves image clarity restoration in non-uniform haze scenes and further reduces the residual degree of haze.

[0092] To sum up, the polarization image defogging method based on double iterative regularized separation mainly includes two modules: concentration estimation and double iterative separation algorithm optimization. First, the dynamic fog concentration function in the image is constructed by using the polarization characteristics. Secondly, the atmospheric scattering coefficient is obtained based on the fog concentration function. Then, a double iterative separation algorithm is proposed through the atmospheric scattering coefficient to solve the optimal transmittance. Finally, the image is restored by combining the atmospheric light value at infinity estimated by the polarization difference image. This is the usage feature of the polarization image defogging method based on double iterative regularized separation. The content not described in detail in this description belongs to the existing technology known to professional and technical personnel in this field.

[0093] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A polarization image defogging method based on double iterative regularization separation, characterized in that: The steps include: S1: Use polarization imaging technology to image the target scene and obtain three polarization images of the same scene; S2: From a visual perspective, a function representing fog density is constructed based on image features of brightness, saturation and texture. S3: Use the dual-iteration separation algorithm to indirectly obtain the local optimal scene depth, thereby obtaining the optimized transmittance; S4: The optimal solution to split S3 into two sub-problems; S5: Introduce auxiliary variables in S4 to transform the problem into a series of sub-problems, and combine semi-quadratic separation and FFT iterative optimization to find the optimal solution; S6: Take the average value of the light intensity corresponding to the intersection position.

2. The polarization image defogging method based on double iterative regularization separation according to claim 1, characterized in that: In S1, when a beam of linearly polarized light with a Stokes vector of S = (I, Q, U) passes through a polarizer with a polarization angle of α, according to the relationship between the Stokes vector and the Mueller matrix, the polarization image light intensity I represented by the Stokes vector can be obtained. θ The expression is: Thus, the polarization image intensity I can be obtained θ The numerical value of .

3. The polarization image defogging method based on double iterative regularization separation according to claim 1, characterized in that: The polarization camera used in S1 can obtain foggy images under three polarization states of 0°, 60°, and 120°, denoted as I0, I 60 and I 120 , the calculated Stokes vectors are: Where I is the total intensity information of the incident light, Q and U are the polarization information of the incident light, Then use the Stokes vector to calculate the polarization degree P and polarization angle θ. The calculation formulas are: The values ​​of the polarization degree P and the polarization angle θ can be calculated by the above formula.

4. The polarization image defogging method based on double iterative regularization separation according to claim 1, characterized in that: After constructing the fog density characterization function c based on the image features of brightness, saturation and texture in S2, the image I is converted into HSV channels, and the second and third channels are taken as image brightness and saturation, respectively, denoted as w1 and w2, and the polarization angle is used as the texture feature, denoted as w 1° a1, a2 and a3 are the weight coefficients of w1, w2 and w3, and then according to the fog concentration: β=exp(c)=exp(a1w1+a2w2+a3w3) Calculate the dynamic atmospheric scattering coefficient β.

5. The polarization image defogging method based on double iterative regularization separation according to claim 2, characterized in that: Before using the double iterative separation algorithm to indirectly obtain the local optimal scene depth in S3, an atmospheric scattering model can be used, which can be expressed as: I(x)=D+A=Jt+A ∞ (1-t) Where x represents a pixel, I represents the foggy image captured by the camera, J represents the scene reflected light, and is the image to be restored based on the atmospheric scattering model. ∞ represents the atmospheric light intensity at infinity, and t represents the transmission rate, which can be further expressed as: t(x)=e -βd For uniform haze scenes, β can be regarded as a constant related to the haze concentration, and d represents the distance from the scene point to the camera, which is called the depth of field and satisfies the non-negative property. Jt is considered to be the part of the scene reflected light that successfully reaches the camera without scattering, which can also be called direct transmission, denoted as D, and the second term represents the part of the global atmospheric light that reaches the camera after being scattered by the medium, denoted as A. The defogging goal based on this model is to estimate A from the obtained I based on the atmospheric scattering model formula. ∞ and t, whose expression is: Thus recovering J.

6. The polarization image defogging method based on double iterative regularization separation according to claim 1, characterized in that: In S3, T is set equal to βd, and then the double iterative separation algorithm can be used to indirectly obtain the local optimal scene depth d, thereby obtaining the optimal intermediate variable T, that is, indirectly solving T under the condition of known atmospheric scattering coefficient β and unknown scene depth d, thereby obtaining the optimized transmittance t, and constructing a regularization function to solve the optimal T and d, so as to indirectly obtain T from the atmospheric scattering coefficient, which is expressed as: Thus, the intermediate variable T can be indirectly obtained.

7. The polarization image defogging method based on double iterative regularization separation according to claim 6, characterized in that: The regularization function benefits from the exploration of the inherent boundary constraints of the transfer function. This constraint is combined with the context regularization of the weighted L1 norm and is modeled as an optimization problem to estimate the transmission rate of unknown scenes. It is proposed to find the optimal t, which can be obtained by minimizing the following objective function, expressed as: in, is a data item, minimizing this item ensures that t and As close as possible, the second item, is the context constraint minimization of t, which can reduce the artifacts at the edge of the image. λ is the regularization parameter that balances these two terms. j is the adaptive weight matrix, D j is a high-order difference operator.

8. The polarization image defogging method based on double iterative regularization separation according to claim 1, characterized in that: In S4, the expression in S3 is divided into the optimal solution of two sub-problems, which is expressed as: Used to quickly find the optimal solution for T.

9. The polarization image defogging method based on double iterative regularization separation according to claim 1, characterized in that: The S5 introduces auxiliary variables into the two groups of expressions decomposed by S4 to transform the problem into a series of sub-problems, and combines semi-quadratic separation and FFT iterative optimization to find the optimal solution of this series of sub-problems. In this way, the optimal solution of the original problem is obtained by convergence, that is, the T value can be obtained.

10. The polarization image defogging method based on double iterative regularization separation according to claim 1, characterized in that: In S6, I and The intersection of the coordinate sets of the top 5% of the concentration is regarded as the atmospheric light A at infinity. ∞ The position of A is optimal. ∞ The estimation of the value is more accurate, and finally the average value of the light intensity corresponding to the intersection position is taken as A ∞ , expressed as: Thus, the image can be dehazed.

Images