A global polarization image restoration method without background prior

Abstract

The application discloses a global polarization image recovery method without background prior, comprising the following steps: S1, different angle polarization images are obtained by rotating a polarization mirror in front of a camera; S2, a polarization imaging proxy model is established by fusing backscattering polarization degree and a polarization angle; S3, under the constraint of EME, an original image is segmented and sampled, and local parameter optimal values of the model are solved; S4, a robust polynomial curved surface fitting method is designed based on a least square method, and global parameters of variables in the model are obtained; S5, according to the relationship between model parameters and backscattering, a method for solving an infinite far atmospheric light value based on model parameters is proposed; and S6, based on the polarization imaging proxy model, the global model parameters obtained in S4 and the infinite far atmospheric light value solved in S6 are substituted into the proxy model in S2, and a clear non-fog image is inversed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a global polarized image restoration method without background prior. BACKGROUND

[0002] At present, the polarized fog removal technology has been paid more and more attention by researchers, and plays an important role in the field of image enhancement and restoration. Polarized fog removal is a new technology in the field of image processing based on polarized optics. Its basic principle is to use multiple polarized images to restore fogged images. According to the principle of polarized optics, light beams with different polarization states will produce different polarization effects when passing through the atmosphere, thereby affecting the clarity of the image. Polarized fog removal technology uses this principle to extract the relevant information of fog from multiple polarized images, thereby restoring a clearer image. Compared with traditional fog removal methods, polarized fog removal technology has better fog removal effect and can more accurately restore the clarity and details of the image. SUMMARY

[0003] According to the problems existing in the prior art, the present application discloses a global polarized image restoration method without background prior, which specifically comprises the following steps:

[0004] Different angle polarized images are obtained based on a polarizer, and the polarization degree and polarization angle of the image are calculated using Stokes vector;

[0005] A polarized imaging proxy model is established by fusing the backscattering polarization degree and the polarization angle;

[0006] The original image is segmented and sampled under the constraint of EME, and the local parameter optimal value of the polarized imaging proxy model is solved;

[0007] A robust polynomial surface fitting method is designed based on the least square method to obtain the global parameters of the variables in the polarized imaging proxy model;

[0008] According to the inverse relationship between the polarized imaging proxy model parameter K and the backscattering A, a method for calculating the infinite atmospheric light value is designed and the infinite atmospheric light value is calculated;

[0009] The global model parameters and the infinite atmospheric light value are substituted into the polarized imaging proxy model to inverse a clear fog-free image.

[0010] Further, when calculating the image polarization degree and polarization angle:

[0011] The obtained 0° polarized image and 90° polarized image are summed to obtain the image intensity I(x,y):

[0012] I(x,y) = I0(x,y) + I 90 (x,y)

[0013] The intensity difference Q(x,y) between the horizontal and vertical polarization components of the image intensity is obtained using the acquired 0° and 90° polarization images:

[0014] Q(x,y) = I0(x,y) - I 90 (x,y)

[0015] The intensity difference U(x,y) between the 45° and -45° polarization components is obtained using the acquired 45° polarization component and the image intensity I(x,y):

[0016] U(x,y) = I 45 (x,y) - I -45 (x,y)

[0017] = 2·I 45 (x,y) - I(x,y)

[0018] The image degree of polarization p(x,y) is calculated from the Stokes vector parameters λ (x,y):

[0019]

[0020] The polarization angle Θ(x,y) of the image intensity is calculated from the Stokes vector parameters λ (x,y):

[0021]

[0022] Further, when building a polarization imaging proxy model:

[0023] Let the mathematical expression of the irradiance I(x,y) received by the detector be:

[0024]

[0025] D and A represent the direct transmission light and atmospheric light received by the detector, respectively, also known as target reflected light and backscattered light, (x,y) represents the position of the pixel, L represents the undamped target reflected light, A ∞ represents the atmospheric light intensity value at infinity, β represents the attenuation coefficient, d represents the optical path distance between the imaging target and the camera, and t is the transmittance;

[0026] Then the undamped clear target signal is represented as

[0027]

[0028] In polarization imaging, it is assumed that the polarization effect is generated by backscattering and is partially linearly polarized light, L represents the undamped target reflected light, A n represents the non-polarization part of the backscattered light, Ap Let represent the polarization portion of the backscattered light. Then, the total light intensity ultimately received by the detector is expressed as:

[0029] I(x,y)=D(x,y)+A(x,y)

[0030] =L(x,y)·t(x,y)+A n (x,y)+A p (x,y)

[0031] Since backscattered light is partially polarized, the degree of polarization of backscattered light is expressed as:

[0032]

[0033] Define the horizontal direction of the camera as the x-axis and the vertical direction as the y-axis. According to Malus's law, the components of the backscattered polarized light intensity along the x-axis and y-axis are respectively A. px A py , The backscattering polarization angle is:

[0034]

[0035]

[0036] Since backscattered light is partially polarized, the intensity along the x-axis is expressed as:

[0037]

[0038] The backscattered light A is represented as

[0039]

[0040] If we want to solve for A, we need to obtain P. A and At this time, Because of P A (x,y)∈[0,1], Therefore, K∈[0,1], the backscattering is represented as

[0041]

[0042] The directly reflected light is

[0043]

[0044] Multiply the backscattering polarization degree by a bias coefficient ε (1≤ε≤1 / P) A (x,y)), then the clear, fog-free image L is represented as:

[0045]

[0046] Furthermore, assuming that all pixels within a small local region of the image have equal backscattering polarization degree and polarization angle, the image is segmented and sampled at intervals. The optimal value of K for the sampled region is then determined using an optimization method. A robust polynomial surface fitting method is designed to obtain the global variable K. The local target signal is then represented as:

[0047]

[0048]

[0049] An Image Detail Enhancement (EME) metric is introduced, and parameter K is solved using an optimization method. The stronger the local grayscale variation, the stronger the image detail, and thus the larger the EME value. When solving for parameter K...

[0050]

[0051] For K i If the value of the local region is ∈[0,1], and the step size is set to 0.01 for traversal, the EME value of each traversal of the local region is calculated, and finally the K value corresponding to the maximum EME value is determined to be the optimal K value.

[0052] Repeat the above steps until the optimal parameter K value for each local region after segmentation and sampling in the image is determined.

[0053] Furthermore, a robust polynomial surface fitting method is designed based on the polynomial least squares method. The polynomial surface fitting model is expressed as:

[0054] f(x,y)=p 00 +p 10 x+p 01 y+p 20 x 2 +p 11 xy+p 02 y 2

[0055] The Bisquare function is introduced into this polynomial. The Bisquare function describes the residuals of the fitted model and is expressed as follows:

[0056]

[0057] Where z represents the distance to outliers, and k represents a specified parameter used to measure the degree of outliers. When z is less than k, the value of the Bisquare function is close to 1, indicating that there are no outliers; when z is greater than k, the value of the Bisquare function gradually decreases, indicating that outliers are becoming more and more obvious.

[0058] Furthermore, it is known The change in the magnitude of K indicates that the degree of polarization of the backscattering is proportional:

[0059]

[0060]

[0061] Since A is inversely proportional to t and also inversely proportional to K, the smaller the value of K, the farther the distance. Therefore, we take Ω... i (x,y)=argmin(K i The average value of A represents ∞

[0062]

[0063] Where Ω represents a local region, and N represents the total number of pixels in that region.

[0064] By adopting the above technical solution, a global polarization image restoration method without background prior is proposed. Addressing the issue that traditional polarization imaging relies on estimating parameters from the background region, we construct a polarization imaging surrogate model starting from the backscattered polarization degree and polarization angle. To solve for the spatial variables of parameter K in the model, a strategy of segmentation, sampling, optimization, and fitting is used to obtain the optimal fused values ​​of the backscattered polarization degree and polarization angle. Finally, a clear, haze-free image is restored using the surrogate model. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a schematic diagram of an atmospheric scattering model;

[0067] Figure 2 This is a structural diagram of the method in this invention;

[0068] Figure 3 The figure shows the experimental results of this invention. Detailed Implementation

[0069] This invention discloses a global polarization image restoration method without background prior knowledge, comprising the following steps:

[0070] S1: Obtain polarized images at different angles (0°, 45°, 90°) by rotating the polarizing filter in front of the camera, and calculate the degree of polarization and polarization angle of the images using Stokes vectors;

[0071] S2: By fusing backscattering polarization degree and polarization angle, a polarization imaging proxy model was established;

[0072] S3: Under EME constraints, the original image is segmented and sampled to solve for the optimal values ​​of local parameters of the model;

[0073] S4: Based on the least squares method, a robust polynomial surface fitting method was designed to obtain the global parameters of the variables in the model.

[0074] S5: Based on the inverse relationship between the polarization imaging surrogate model parameter K and the backscattering A, a method for calculating the atmospheric light value at infinity is proposed.

[0075] S6: Based on the polarization imaging proxy model, the global model parameters obtained in S4 and the atmospheric light value at infinity obtained in S5 are substituted into the proxy model proposed in S2 to generate a clear, fog-free image.

[0076] S1 specifically adopts the following method:

[0077] S11: The image intensity I(x,y) is obtained by summing the 0° polarization image and the 90° polarization image.

[0078] I(x,y)=I0(x,y)+I 90 (x,y)

[0079] S12: Utilize the obtained 0° polarization image and 90° polarization image to obtain the intensity difference Q(x,y) between the horizontal and vertical polarization components of the image intensity:

[0080] Q(x,y)=I0(x,y)-I 90 (x,y)

[0081] S13: Using the obtained 45° polarization component and the image intensity I(x,y), the intensity difference U(x,y) between the 45° polarization component and the -45° polarization component is obtained:

[0082] U(x,y)=I 45 (x,y)-I -45 (x,y)

[0083] =2·I 45 (x,y)-I(x,y)

[0084] S14: Using the Stokes vector parameters calculated in S11, S12, and S13, the polarization degree p of the image intensity can be obtained. λ(x,y):

[0085]

[0086] S15: Using the Stokes vector parameters calculated in S11, S12, and S13, the polarization angle θ of the image intensity can be obtained. λ (x,y):

[0087]

[0088] The specific method used in S2 is as follows:

[0089] S21: The atmospheric scattering model is a model used to describe the scattering of light by tiny particles or droplets in the atmosphere, such as... Figure 1 As shown. When light passes through tiny particles in the atmosphere (such as water droplets), it changes its original direction of propagation, resulting in a polarization effect as it passes through the atmosphere. Furthermore, the model assumes that these tiny particles propagate different wavelengths of light and exhibit different scattering rates in different polarization directions. D and A represent the directly transmitted light and atmospheric light received by the detector, also known as the target reflected light and backscattered light, respectively, and (x,y) represents the pixel position. In foggy conditions, the mathematical expression for the irradiance I(x,y) received by the detector is:

[0090]

[0091] Where L represents the unattenuated target reflected light, and A ∞ β represents the atmospheric light intensity at infinity, d represents the attenuation coefficient, d represents the optical path distance between the imaging target and the camera, and t represents the transmittance.

[0092] S22: According to equation S21, the target signal that is not attenuated and clearly defined can be expressed as:

[0093]

[0094] S23: In polarization imaging, it is assumed that the polarization effect is caused by backscattering and is partially linearly polarized light. L represents the unattenuated target reflected light, A n A represents the unpolarized portion of the backscattered light. p Let represent the polarization component of the backscattered light. Then the total light intensity ultimately received by the detector can be expressed as:

[0095] I(x,y)=D(x,y)+A(x,y)

[0096] =L(x,y)·t(x,y)+A n (x,y)+A p (x,y)

[0097] S24: Since backscattered light is partially polarized, the degree of polarization of backscattered light can be expressed as:

[0098]

[0099] S25: We define the horizontal direction of the camera as the x-axis and the vertical direction as the y-axis. According to Malus's law, the components of the backscattered polarized light intensity along the x-axis and y-axis are respectively A... px A py , For the backscattering polarization angle, they can be expressed as:

[0100]

[0101]

[0102] S26: Since backscattered light is partially polarized, its intensity on the x-axis can be expressed as:

[0103]

[0104] S27: From S24 and S26, we can obtain that the backscattered light A can be expressed as...

[0105]

[0106] S28: To solve for A in S27, we need to obtain P. A and At this time, we ordered Because of P A (x,y)∈[0,1], Therefore, K∈[0,1]. Backscattering can then be expressed as...

[0107]

[0108] S29: By combining S21 and S28, the directly reflected light can be obtained as...

[0109]

[0110] S30: To preserve the depth of field of the recovered image, the backscattering polarization degree needs to be multiplied by a bias coefficient ε (1≤ε≤1 / P). A A clear, fog-free image L (x, y) can be represented as:

[0111]

[0112] The specific method used in S3 is as follows:

[0113] S31: Since the changes between local pixels are quite weak, assume that within a small local region of the image, all pixels have equal backscattering polarization degree and polarization angle. Then, we segment the image and perform interval sampling, and for the sampled region, we use an optimization method to solve for the optimal value of K in that region. Finally, a robust polynomial surface fitting method is designed to obtain the global variable K. From S29, we can obtain that the local target signal can be expressed as:

[0114]

[0115]

[0116] S32: We use an optimization method to solve for parameter K, introducing the Image Detail Enhancement Evaluation (EME) metric. EME represents the degree of local gray-level changes in an image; the stronger the local gray-level changes, the stronger the details displayed in the image, and the larger the EME value. The value of K can be expressed as:

[0117]

[0118] S33: For K i If the value of the local region is ∈[0,1], and the step size is set to 0.01 for traversal, the EME value of each traversal of the local region is calculated, and finally the K value corresponding to the maximum EME value is determined to be the optimal K value.

[0119] S34: Repeat the above steps until the globally optimal parameter K value in the image is determined.

[0120] The specific method used in S4 is as follows:

[0121] S41: After obtaining the local optimum of K through S3, we designed a robust polynomial surface fitting method based on polynomial least squares. First, the polynomial surface fitting model is expressed as:

[0122] f(x,y)=p 00 +p 10 x+p 01 y+p 20 x 2 +p 11 xy+p 02 y 2

[0123] S42: To avoid the uncertainty caused by random disturbances, a Bisquare function is introduced into this polynomial. Bisquare is a robust fitting method that can be used to offset the influence of outliers in the data. It is based on a loss function called the Bisquare function, which improves the fitting results by minimizing the influence of outliers. Basically, the Bisquare function can be used to describe the residuals of the fitted model; it is a finite, smooth function, expressed as...

[0124]

[0125] Here, z represents the distance to outliers, and k represents a specified parameter used to measure the degree of outliers. When z is less than k, the value of the Bisquare function is close to 1, indicating that there are no outliers; when z is greater than k, the value of the Bisquare function gradually decreases, indicating that outliers are becoming more and more obvious.

[0126] The specific approach used in S5 is as follows:

[0127] S51: Known The change in the magnitude of K indicates that it is proportional to the degree of polarization of the backscattering, as can be seen from equation S21.

[0128]

[0129] S51: As can be seen from equation S28

[0130]

[0131] Since A is inversely proportional to t and also inversely proportional to K, the smaller K is, the farther the distance. Therefore, we take Ω as... i (x,y)=argmin(K i The average value of A represents ∞ .

[0132]

[0133] Where Ω represents a local region, and N represents the total number of pixels in that region.

Claims

1. A global polarization image restoration method without background prior knowledge, characterized in that: Includes the following steps: Polarized images at different angles are obtained using a polarizing mirror, and the degree of polarization and polarization angle of the images are calculated using Stokes vectors. A polarization imaging proxy model is established by fusing backscattering polarization degree and polarization angle; Under EME constraints, the original image is segmented and sampled to solve for the optimal local parameters of the polarization imaging surrogate model. A robust polynomial surface fitting method based on the least squares approach is designed to obtain the global parameters of variables in the polarization imaging surrogate model. Based on the inverse relationship between the polarization imaging surrogate model parameter K and the backscattering A, a method for calculating the atmospheric light value at infinity is designed and the atmospheric light value at infinity is calculated. Substitute the global model parameters and the atmospheric light value at infinity into the polarization imaging proxy model to derive a clear, fog-free image; When establishing a polarization imaging surrogate model: Let the mathematical expression for the irradiance I(x,y) received by the detector be: D and A represent the directly transmitted light and atmospheric light received by the detector, also known as the target reflected light and backscattered light, respectively. (x, y) represents the pixel position, L represents the unattenuated target reflected light, and A ∞ β represents the atmospheric light intensity at infinity, d represents the attenuation coefficient, d represents the optical path distance between the imaging target and the camera, and t represents the transmittance. The target signal that is not attenuated and clearly defined is represented as: In polarization imaging, it is assumed that the polarization effect is caused by backscattering and is partially linearly polarized light, where L represents the unattenuated target reflected light, and A... n A represents the unpolarized portion of the backscattered light. p Let represent the polarization portion of the backscattered light. Then, the total light intensity ultimately received by the detector is expressed as: I(x,y)=D(x,y)+A(x,y) =L(x,y)·t(x,y)+A n (x,y)+A p (x,y) Since backscattered light is partially polarized, the degree of polarization of backscattered light is expressed as: Define the horizontal direction of the camera as the x-axis and the vertical direction as the y-axis. According to Malus's law, the components of the backscattered polarized light intensity along the x-axis and y-axis are respectively A. px A py , The backscattering polarization angle is: Since backscattered light is partially polarized, the intensity along the x-axis is expressed as: The backscattered light A is represented as If we want to solve for A, we need to obtain P. A and At this time, Because of P A (x,y)∈[0,1], Therefore, K∈[0,1], the backscattering is represented as The directly reflected light is Multiply the backscattering polarization degree by a bias coefficient ε (1≤ε≤1 / P) A (x,y)), then the clear, fog-free image L is represented as:

2. The global polarization image restoration method without background prior as described in claim 1, characterized in that: When calculating the degree of polarization and polarization angle of an image: The image intensity I(x,y) is obtained by summing the 0° polarization image and the 90° polarization image: I(x,y)=I0(x,y)+I 90 (x,y) The intensity difference Q(x,y) between the horizontal and vertical polarization components of the image intensity is obtained by using the 0° polarization image and the 90° polarization image: Q(x,y)=I0(x,y)-I 90 (x,y) The intensity difference U(x,y) between the 45° polarization component and the -45° polarization component is obtained using the acquired 45° polarization component and the image intensity I(x,y): U(x,y)=I 45 (x,y)-I -45 (x,y) =2·I 45 (x,y)-I(x,y) Calculate the image polarization degree p using Stokes vector parameters. λ (x,y): The polarization angle θ of the image intensity is calculated using the Stokes vector parameters. λ (x,y):

3. The global polarization image restoration method without background prior as described in claim 1, characterized in that: Assuming that all pixels within a small local region of an image have equal backscattering polarization degree and polarization angle, the image is segmented and sampled at intervals. The optimal value of K for the sampled region is then determined using an optimization method. A robust polynomial surface fitting method is designed to obtain the global variable K. The local target signal is then represented as: An Image Detail Enhancement (EME) metric is introduced, and parameter K is solved using an optimization method. The stronger the local grayscale variation, the stronger the image detail, and thus the larger the EME value. When solving for parameter K... For K i If the value of the local region is ∈[0,1], and the step size is set to 0.01 for traversal, the EME value of each traversal of the local region is calculated, and finally the K value corresponding to the maximum EME value is determined to be the optimal K value. Repeat the above steps until the optimal parameter K value for each local region after segmentation and sampling in the image is determined.

4. The global polarization image restoration method without background prior as described in claim 1, characterized in that: A robust polynomial surface fitting method is designed based on the polynomial least squares method. The polynomial surface fitting model is expressed as: f(x,y)=p 00 +p 10 x+p 01 y+p 20 x 2 +p 11 xy+p 02 y 2 The Bisquare function is introduced into this polynomial. The Bisquare function describes the residuals of the fitted model and is expressed as follows: Where z represents the distance to outliers, and k represents a specified parameter used to measure the degree of outliers. When z is less than k, the value of the Bisquare function is close to 1, indicating that there are no outliers; when z is greater than k, the value of the Bisquare function gradually decreases, indicating that outliers are becoming more and more obvious.

5. The global polarization image restoration method without background prior as described in claim 1, characterized in that: Known The change in the magnitude of K indicates that the degree of polarization of the backscattering is proportional: Since A is inversely proportional to t and also inversely proportional to K, the smaller the value of K, the farther the distance. Therefore, we take Ω... i (x,y)=argmin(K i The average value of A represents ∞ Where Ω represents a local region, and N represents the total number of pixels in that region.

Images