A novel underwater scattering method based on a polarization scattering model

Abstract

This invention relates to an underwater scattering method based on a novel polarization scattering model. A polarization state variable is designed to simultaneously represent the degree of polarization and the polarization angle. A novel polarization scattering model is established, which unifies and generalizes existing polarization scattering models, making it more universal and applicable to atmospheric polarization scattering models. First, the polarization states of existing turbid and clear underwater images are analyzed, and the polarization state of the background light is determined using the mode of the polarization states. Second, to address the problem of the large polarization state space, polarization quantization is proposed to reduce the number of polarization states, thereby reducing computational complexity. Finally, based on polarization quantization, polarization line constraints are proposed to describe the behavior of polarization states in turbid conditions, and these constraints are used to calculate related physical quantities such as polarization transmittance to achieve underwater scattering. Experimental results show that this method performs well in underwater scattering tasks, and is particularly advantageous in descattering low-light underwater images.

Description

Technical Field

[0001] This invention relates to an image restoration method, specifically an underwater scattering method based on a novel polarization scattering model. Background Technology

[0002] In recent years, with the development of imaging technology, instantaneous polarization imaging has become possible. However, images acquired under scattering environments suffer from color distortion and blurring, severely impacting the performance of advanced visual tasks such as target recognition. Therefore, scattering removal is essential, and utilizing polarization information is an effective method for underwater scattering. Current polarization descattering methods are based on polarization imaging models with certain limitations. These models cannot cover all cases of polarization imaging, such as the degree of polarization of a sharp image and the polarization angle of all images. This limits the recovery capability of existing methods in these situations, and also restricts their ability to recover the polarization state of a sharp image. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides an underwater scattering method based on a novel polarization scattering model. 1) A novel polarization imaging model is proposed, theoretically applicable to the description of underwater scattering and fog imaging processes. 2) Polarization quantization is proposed, allowing all polarization information of an image to be expressed using a small amount of polarization information. 3) Polarization line constraints are proposed, describing the characteristics of polarization states in the complex plane, enabling effective calculation of polarization transmittance, and thus obtaining the descattered image and its polarization information.

[0004] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0005] An underwater scattering method based on a novel polarization scattering model includes the following steps:

[0006] An imaging model is established on the complex plane to provide polarization information between murky and clear images, incorporating both the degree of polarization and the polarization angle into the imaging model.

[0007] Polarization quantization is defined to address the vast polarization state space. Polarization quantization is the process of representing a large number of original polarization states with a small number of polarization states by clustering polarization states that are close in position on the complex plane and using the cluster center to represent the same polarization state.

[0008] A polarization line constraint is defined to describe the behavior of polarization states when the image is turbid. This constraint is used to calculate polarization-related physical quantities, enabling underwater scattering to obtain a clear image. The polarization line constraint is the distribution law of polarization states on the complex plane: on the complex plane, the polarization states of the turbid image corresponding to the points of the same cluster of polarization states in the clear image are distributed on the same complex line.

[0009] The polarization imaging model has the following two representations:

[0010] m I I = m D D+m B B

[0011] Where I represents a murky image, D represents directly transmitted light, and B represents background light;

[0012] m I =m D r+m B (1-r)

[0013] Where, m D This represents the polarization state of directly transmitted light, which is also the polarization state of a clear image; r represents the polarization transmittance.

[0014] The method specifically includes the following steps:

[0015] Step 1. Convert multiple images captured after different polarization filters into intensity and polarization state representations; the polarization state is a complex variable represented by the degree of polarization and the polarization angle;

[0016] Step 2. Analyze the distribution of polarization states of the turbid image in the complex plane, and use the mode of the polarization states to determine the polarization state of the background light;

[0017] Step 3. Calculate the polarization transmittance using polarization line constraints;

[0018] Step 4. Substitute the polarization transmittance and background light polarization state into the model to determine the polarization state of the clear image;

[0019] Step 5. Use the Quadtree-Subdivision method to find the background light at infinity;

[0020] Step 6. Substitute the polarized transmittance and the background light at infinity into the model to calculate the transmittance;

[0021] Step 7. Substitute the polarization transmittance and transmittance into the model to obtain a clear image.

[0022] The polarization state is defined by the following formula:

[0023]

[0024] Where m represents the polarization state, and S0, S1, and S2 are the first, second, and third Stokes parameters, respectively. It represents the imaginary unit.

[0025] The background light polarization state is represented by the following formula:

[0026] m B=ηmode(m I )

[0027] Where, m B Indicates the background light polarization state, m I The polarization state of the turbid image is represented by η, which is a parameter greater than 1, and mode(·) represents the function for finding the mode.

[0028] The polarization transmittance is defined by the following formula:

[0029]

[0030] Obtained through polarization line constraints:

[0031] a. Define polarization difference m IB =m I -m B m DB =m D -m B ;

[0032] b. Convert the polarization difference to polar coordinates m IB =d IB e iψ =r(m D -m B ) = rm DB =rd DB e iψ , where d IB It is m IB The model, d DB It is m DB The model, m IB and m DB They have the same principal argument value ψ;

[0033] c. Thus, we obtain r = d IB / d DB ;

[0034] d. The polarization difference m IB Clustering of the principal argument value ψ and accelerating the process using nearest neighbor search on the KD-Tree yields a series of polarization lines;

[0035] e. For each polarization line, calculate d separately. IB maximum value As d DB The estimate;

[0036] f. For each pixel position u, calculate the coarse polarization transmittance.

[0037] g. To estimate the lower limit r2 of polarized transmittance, first use the following formula to find the intersection point of the polarization line and the unit circle on the complex plane:

[0038] m X1 =m p1 +m p2

[0039] m X2 =m p1 -m p2

[0040] in,

[0041]

[0042]

[0043] In the formula, m * Denotes the conjugate of the complex number m;

[0044] Secondly, the two intersection points m X1 and m X2 Substituting these values ​​into the definition of polarization transmittance, we obtain a lower limit r2 for polarization transmittance:

[0045]

[0046] h. Estimating the lower limit r3 of polarization transmittance:

[0047]

[0048] Where t is transmittance, I represents the first Stokes parameter of the turbid image, and B... ∞ The background light at infinity corresponding to I;

[0049] i. A preliminary estimate of polarization transmittance is obtained using the following formula.

[0050]

[0051] j. Optimize the preliminary estimated polarization transmittance to obtain a refined polarization transmittance:

[0052]

[0053] Where I represents the first Stokes parameter of the murky image, u and v represent pixel positions, μ is the weighting coefficient between the data term and the smoothing term, and N4(u) represents the four neighborhood of pixel position u.

[0054] The polarization state of the clear image is represented by the following formula:

[0055]

[0056] The transmittance is expressed by the following formula:

[0057]

[0058] Where t is transmittance, I represents the turbid image, and is also its first Stokes parameter, B ∞ The background light at infinity corresponds to I.

[0059] The analytical expression of the clear image is represented by the following formula:

[0060]

[0061] Where L represents the intensity of the sharp image.

[0062] Based on the above analytical modification, the calculation formula for the clear image is expressed as follows:

[0063]

[0064] Here, α is a water-related coefficient, and t0 is a positive number close to 0 to prevent division by zero errors.

[0065] The method also includes using complex-valued peak signal-to-noise ratio to measure the similarity between polarization states;

[0066]

[0067]

[0068] Where CVMSE represents the complex mean square error, CVPSNR represents the complex peak signal-to-noise ratio, N represents the number of pixels, and m I For the polarization state to be compared, m R Represents the reference polarization state, ||·|| F Let f(m) denote the Frobenius norm, which has max|m| for polarization states. R | 2 =1. Generally, m I and m R It can also represent complex-valued matrices of the same size.

[0069] The present invention has the following beneficial effects and advantages:

[0070] 1. This invention establishes a novel polarization imaging model. Unlike existing polarization imaging models, this model is built on the complex plane and simultaneously considers the degree of polarization and the polarization angle. This allows the model to recover the polarization state while restoring a clear image, thus enabling further utilization of polarization information after descattering, such as polarization target identification. This model can be used to describe polarization scattering imaging, such as underwater polarization imaging and fog polarization imaging.

[0071] 2. This invention proposes polarization quantization, a simplified way of representing polarization states. It can represent all polarization states of an image with a small number of polarization states, which can greatly reduce computational complexity and can also be used as a way of compressing polarization states.

[0072] 3. The method of the present invention is superior to existing methods in underwater scattering, and exhibits good stability, especially in low-light underwater scattering environments. Attached Figure Description

[0073] Figure 1 This is the overall flowchart of the present invention;

[0074] Figure 2 It is a comparison diagram of turbid and clear images and their polarization states;

[0075] Figure 3 This is a diagram illustrating polarization line constraints and pixel position distribution;

[0076] Figure 4 This is an intermediate step in the method of the present invention;

[0077] Figure 5 This is a comparison chart of the processing effects of the method of the present invention. Detailed Implementation

[0078] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0079] 1. Polarization Imaging Model and Task Decomposition

[0080] Most current underwater scattering methods are based on atmospheric scattering models, which are expressed by the following equation:

[0081] I = D + B = Lt + B ∞ (1-t) (1)

[0082] Where I represents a cloudy image, D represents direct transmitted light, B represents background light, and L represents a clear image. ∞ Let represent the background light at infinity, and t represent the transmittance. The purpose of underwater scattering is to recover a clear image from a murky one. Utilizing polarization information is an effective method for recovering a clear image.

[0083] The polarization imaging model of this method differs from existing polarization imaging models; one form of it is represented by the following equation:

[0084] m I I = m D D+m B B (2)

[0085] Where, m I Indicates the collected polarization state, m D and mB Let D and B represent the polarization states of the directly transmitted light and the background light, respectively. This model describes the influence of the scattering medium on the polarization state of the directly transmitted light. The model takes into account both the degree of polarization and the polarization angle, because the polarization state contains polarization information in both the degree of polarization and the polarization angle.

[0086] The definition of polarization state depends on the Stokes parameter. The Stokes parameter considering only linear polarization is generally expressed as:

[0087] S = [S0, S1, S2] T (3)

[0088] in,

[0089] S0 = I 0° +I 90° =I 45° +I 135°

[0090] S1=I 0° -I 90° (4)

[0091] S2=I 45° -I 135°

[0092] Considering the definition of the Stokes parameter, we acquired and captured four turbid images I after filtering by polarization filter angles θ∈{0°, 45°, 90°, 135°}. θ This allows us to obtain accurate Stokes parameters.

[0093] The polarization state is defined as:

[0094]

[0095] Where m represents the polarization state, and S0, S1, and S2 correspond to the components in the Stokes parameters. It represents the imaginary unit.

[0096] Based on the definitions of polarization state and degree of polarization and polarization angle, we have:

[0097] m=ρe i2φ

[0098] ρ=|m| (6)

[0099]

[0100] Where ρ represents the degree of polarization, φ represents the polarization angle, |·| represents the absolute value, and arg is used to obtain the principal value of the argument. Formula (6) shows that the polarization state includes both the degree of polarization and the polarization angle, and all polarization information can be well expressed by a single parameter. For ease of expression, unless otherwise specified, it is in the form of X Y The symbols represent information X of an object Y, where X includes m, ρ, φ, S, S1, S2, etc., and Y includes I, D, B, L, B ∞ etc., such as m D and m B These represent the polarization states of the directly transmitted light D and the background light B, respectively.

[0101] Starting from the definitions of polarization state and Stokes parameter, we have the following relationship:

[0102]

[0103]

[0104] Formulas (7) and (8) show that m D =m L , To simplify the expression, the following content will directly use m. D To represent the polarization state of a sharp image, use m B This represents the polarization state of the background light at infinity. Solving equations (1) and (2) simultaneously yields:

[0105] D=rI (9)

[0106] B=(1-r)I (10)

[0107] The polarization transmittance r is defined as:

[0108]

[0109] Solving equations (1) and (7) simultaneously yields:

[0110]

[0111] Equation (12) shows that the key to solving for a clear image lies in estimating the polarization transmittance r and transmittance t. Therefore, the overall process of the method can be divided into two parts: the estimation of polarization transmittance and the recovery of the clear polarization state, and the estimation of transmittance and the recovery of the clear image. Figure 2 This demonstrates a typical distribution of clear and murky images and their polarization states.

[0112] 2. Estimation of polarization transmittance and recovery of sharp polarization state

[0113] Another representation of the polarization imaging model of this method is expressed by the following equation:

[0114] m I =m D r+m B (1-r) (13)

[0115] This formula is obtained by combining formulas (2), (9) and (10).

[0116] To estimate the polarization transmittance from formula (13), we must first estimate the polarization state m of the background light at infinity. B It is obtained by the following formula:

[0117]

[0118] in, It is a parameter, where mode represents the mode operator.

[0119] like Figure 3 As shown, multiple polarization lines intersect at a single point, which is the background light polarization state m. B .

[0120] Polarization line constraint is proposed based on polarization quantization and combined with a polarization imaging model. It describes a distribution law of polarization states in the complex plane, that is, in the complex plane, the polarization states of murky images corresponding to points in the same cluster of polarization states in clear images are distributed along the same complex straight line.

[0121] When m B When the polarization transmittance is known, it can be obtained using polarization line constraints. The specific steps are as follows:

[0122] a. Define polarization difference:

[0123] m IB =m I -m B (15)

[0124] m DB =m D -m B (16)

[0125] b. Convert the polarization difference to polar coordinates:

[0126] m IB =d IB e iψ =r(m D -m B ) = rm DB =rd DB e iψ (17)

[0127] Where, d IB It is m IBThe model, d DB It is m DB The model, m IB and m DB They have the same principal argument value ψ;

[0128] c. Thus we obtain:

[0129] r = d IB / d DB (18)

[0130] d. The polarization difference m IB Clustering of the principal argument value ψ and accelerating the process using nearest neighbor search on the KD-Tree yields a series of polarization lines;

[0131] e. For each polarization line, calculate d separately. IB maximum value As d DB The estimate;

[0132] f. For each pixel position u, calculate the coarse polarization transmittance:

[0133]

[0134] g. To estimate the lower limit r2 of polarized transmittance, first use the following formula to find the intersection point of the polarization line and the unit circle on the complex plane:

[0135] m X1 =m p1 +m p2 (20)

[0136] m X2 =m p1 -m p2 (twenty one)

[0137] in,

[0138]

[0139]

[0140] In the formula, m * It represents the conjugate of the complex number m.

[0141] Secondly, substituting the two intersection points into the definition of polarization transmittance (11), we obtain a lower limit r2 for polarization transmittance:

[0142]

[0143] Where u represents the pixel position.

[0144] h. Another lower limit for estimating polarization transmittance r3:

[0145]

[0146] Where t is transmittance, I represents the first Stokes parameter of the turbid image, and B... ∞ The background light at infinity corresponds to I.

[0147] i. A preliminary estimate of polarization transmittance is obtained using the following formula.

[0148]

[0149] j. The preliminary estimated polarization transmittance is optimized using weighted least squares filtering to obtain a refined polarization transmittance:

[0150]

[0151] Where u and v represent pixel positions, μ is the weighting coefficient between the data term and the smoothing term, and N4(u) represents the four neighborhood of pixel position u.

[0152] Substituting the polarization transmittance and the background light polarization state into the model formula (13), the polarization state of the recovered sharp image is expressed by the following formula:

[0153]

[0154] 4. Transmittance estimation and sharp image restoration

[0155] Combining formulas (1) and (10), we get:

[0156]

[0157] We use existing methods, such as Quadtree-Subdivision, to estimate the background light B at infinity. ∞ This allows for the estimation of transmittance t.

[0158] Finally, we obtained a clear image of the recovered image:

[0159]

[0160] Here, α is a water-related coefficient. t0 is set to 0.0001 to prevent division by zero errors.

[0161] Some intermediate steps of the algorithm and clear actual diagrams are shown below. Figure 4 As shown, the overall algorithm flowchart is as follows: Figure 1 As shown, and also given in Algorithm 1.

[0162]

[0163] 5. Experimental Results and Analysis

[0164] We compared our proposed algorithm with state-of-the-art algorithms on our own collected dataset, including the Bayesian Retinex-based underwater image enhancement algorithm BRU and the dual-color-space underwater image enhancement network algorithm UIEC. 2 -Net and the Minimum Color Loss and Locally Adaptive Contrast Enhancement Algorithm MLLE. Regarding evaluation metrics, Peak Signal-to-Noise Ratio (PSNR), Multi-Scale Structural Similarity Index (MS-SSIM), and our proposed Complex Value PSNR are used for full-reference evaluation. Naturalness Image Quality Evaluation (NIQE), Blind / No-Reference Image Spatial Quality Evaluation (BRISQUE), Underwater Color Image Quality Evaluation (UCIQE), and CCF (Color Visibility, Contrast, and Fog Density) are used for no-reference evaluation. The latter two metrics are designed for underwater images. CCF is an underwater image quality metric that considers color visibility, contrast, and fog density.

[0165] Table 1 shows the comparative experimental results of this method with other methods on parametric datasets, and Table 2 shows the comparative experimental results of this method with other methods on non-parametric datasets. It can be seen that the present invention has the superiority in improving image vision.

[0166] Table 1 Comparison of different algorithms on parametric datasets.

[0167]

[0168] Table 2 Comparison of different algorithms on parametric datasets.

[0169]

[0170] In this context, double-underlined data represents the optimal result, while single-underlined data represents the second-best result.

[0171] like Figure 5 As shown, the experimental results comparing this method with other methods demonstrate the superiority of this invention in improving image vision.

[0172] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should be considered within the scope of protection of the present invention.

Claims

1. An underwater scattering method based on a novel polarization scattering model, characterized in that, Includes the following steps: An imaging model is established on the complex plane to provide polarization information between murky and clear images, incorporating both the degree of polarization and the polarization angle into the imaging model. Polarization quantization is defined to address the vast polarization state space. Polarization quantization is the process of representing a large number of original polarization states with a small number of polarization states by clustering polarization states that are close in position on the complex plane and using the cluster center to represent the same polarization state. A polarization line constraint is defined to describe the behavior of polarization states when the image is turbid. This constraint is used to calculate polarization-related physical quantities, enabling underwater scattering to obtain a clear image. The polarization line constraint is the distribution law of polarization states in the complex plane: in the complex plane, the polarization states of the turbid image corresponding to the points of the same cluster of polarization states in the clear image are distributed on the same complex line. The polarization imaging model can be represented in the following two ways: m I I=m D D+m B B Where I represents a murky image, D represents directly transmitted light, and B represents background light; m I =m D r+m B (1-r) Where, m D This represents the polarization state of directly transmitted light, which is also the polarization state of a clear image; r represents the polarization transmittance. The method is specifically executed as follows: Step 1. Convert multiple images captured after different polarization filters into intensity and polarization state representations; the polarization state is a complex variable represented by the degree of polarization and the polarization angle; Step 2. Analyze the distribution of polarization states of the turbid image in the complex plane, and use the mode of the polarization states to determine the polarization state of the background light; Step 3. Calculate the polarization transmittance using polarization line constraints; Step 4. Substitute the polarization transmittance and background light polarization state into the model to determine the polarization state of the clear image; Step 5. Use the Quadtree-Subdivision method to find the background light at infinity; Step 6. Substitute the polarized transmittance and the background light at infinity into the model to calculate the transmittance; Step 7. Substitute the polarization transmittance and transmittance into the model to obtain a clear image.

2. The underwater scattering method based on a novel polarization scattering model according to claim 1, characterized in that, The polarization state is defined by the following formula: Where m represents the polarization state, and S0, S1, and S2 are the first, second, and third Stokes parameters, respectively. It represents the imaginary unit.

3. The underwater scattering method based on a novel polarization scattering model according to claim 1, characterized in that, The background light polarization state is represented by the following formula: m B =ηmode(m I ) Where, m B Indicates the background light polarization state, m I The polarization state of the turbid image is represented by η, which is a parameter greater than 1, and mode(·) represents the function for finding the mode.

4. The underwater scattering method based on a novel polarization scattering model according to claim 1, characterized in that, The polarization transmittance is defined by the following formula: Obtained through polarization line constraints: a. Define polarization difference m IB =m I -m B m DB =m D -m B ; b. Convert the polarization difference to polar coordinates m IB =d IB e iψ =r(m D -m B ) = rm DB =rd DB e iψ , where d IB It is m IB The model, d DB It is m DB The model, m IB and m DB They have the same principal argument value ψ; c. Thus, we obtain r = d IB / d DB ; d. The polarization difference m IB Clustering of the principal argument value ψ and accelerating the process using nearest neighbor search on the KD-Tree yields a series of polarization lines; e. For each polarization line, calculate d separately. IB maximum value As d DB The estimate; f. For each pixel position u, calculate the coarse polarization transmittance. g. To estimate the lower limit r2 of polarized transmittance, first use the following formula to find the intersection point of the polarization line and the unit circle on the complex plane: m X1 =m p1 +m p2 m X2 =m p1 -m p2 in, In the formula, m * Denotes the conjugate of the complex number m; Secondly, the two intersection points m X1 and m X2 Substituting these values ​​into the definition of polarization transmittance, we obtain a lower limit r2 for polarization transmittance: h. Estimating the lower limit r3 of polarization transmittance: Where t is transmittance, I represents the first Stokes parameter of the turbid image, and B... ∞ The background light at infinity corresponding to I; i. A preliminary estimate of polarization transmittance is obtained using the following formula. j. Optimize the preliminary estimated polarization transmittance to obtain a refined polarization transmittance: Where I represents the first Stokes parameter of the murky image, u,v represent pixel positions, μ is the weighting coefficient between the data term and the smoothing term, and N4(u) represents the four neighborhood of pixel position u.

5. The underwater scattering method based on a novel polarization scattering model according to claim 1, characterized in that, The polarization state of the clear image is represented by the following formula:

6. The underwater scattering method based on a novel polarization scattering model according to claim 1, characterized in that, The transmittance is expressed by the following formula: Where t is transmittance, I represents the turbid image, and is also its first Stokes parameter, B ∞ The background light at infinity corresponds to I.

7. The underwater scattering method based on a novel polarization scattering model according to claim 1, characterized in that, The analytical expression of the clear image is represented by the following formula: Where L represents the intensity of the sharp image; Based on the above analytical modification, the calculation formula for the clear image is expressed as follows: Here, α is a water-related coefficient, and t0 is a positive number close to 0 to prevent division by zero errors.

8. The underwater scattering method based on a novel polarization scattering model according to claim 1, characterized in that, The method also includes using complex-valued peak signal-to-noise ratio to measure the similarity between polarization states; Where CVMSE represents the complex mean square error, CVPSNR represents the complex peak signal-to-noise ratio, N represents the number of pixels, and m I For the polarization state to be compared, m R Represents the reference polarization state, ‖·‖ F Let f(m) denote the Frobenius norm, which has max|m| for polarization states. R | 2 =1,m I and m R It can also represent complex-valued matrices of the same size.

Images