Fast Underwater Image Enhancement Method Incorporating Polarization Information

Abstract

The present invention discloses a fast underwater image enhancement method integrating polarization information, and the steps are as follows: Step 1: Calculate the global parameters of the entire image; Step 2: Use the parameters to solve the optimal difference image of the image; Step 3: Solve the polarization degree of the backscattered light; Step 4: Use the optimal difference image and the polarization degree of the backscattered light to solve the polarization degree of the reflected light of the target object; Step 5: Estimate the enhanced image. The present invention accurately calculates the orthogonal polarization image by using the Stokes vector, replacing the tediousness of manually rotating the polarization device; the present invention integrates the optimal polarization difference component to achieve the common-mode suppression of the backscattered light, reducing the algorithm operation time by about 70% compared with the traditional method; when estimating the polarization degree and polarization angle of the backscattered light, the present invention gets rid of the requirements of the original image for the empty area in the scene, improving the applicability of the de-scattering imaging.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and more specifically to a fast underwater image enhancement method that fuses polarization information. Background Art

[0002] Polarization imaging is a means to achieve underwater image de-scattering. The working principle of a polarization imaging system is to separate the polarization states of the target reflection signal and the background scattered light signal by rotating a polarizer, and then implement common-mode suppression on the background scattered light to extract useful target reflection light information therefrom. Therefore, polarization is widely used to remove the scattering effect and enhance the image contrast. The difficulty of a polarization imaging system lies in detecting weak targets and obtaining their accurate time delay and Doppler information, and improving the weak target resolution ability in any "time delay-Doppler" domain is the fundamental to obtaining accurate information of weak targets.

[0003] In recent years, scholars at home and abroad have done a lot of research work on using polarization imaging technology for de-scattering. Representatively, for example, Schechner removed the backscattering component in "Advanced visibility improvement based on polarization-filtered images" by using the polarization state difference between the backscattered light and the target reflection light, but this algorithm requires manual selection of the "best" and "worst" images, which has a certain subjectivity. Mudge improved the Schechner model in "Real time polarimetric dehazing". Using the Stokes vector to characterize polarization information, after obtaining each component of the Stokes vector, the "best" polarization image and the "worst" polarization image are obtained through conversion equations, and then the clear scene image is restored, improving the applicability of the algorithm. However, there are still problems such as noise introduced by multiple CCD snapshots, color bias, and a decrease in accuracy caused by ignoring the polarization characteristics of objects with a high degree of depolarization, resulting in the inability to effectively separate the target object from the scattered light.

[0004] The above two representative polarization imaging technologies both have certain limitations, which make it difficult to be applied in engineering practice. Summary of the Invention

[0005] Aiming at the deficiencies of the existing methods, the present invention proposes a fast underwater image enhancement method that fuses polarization information, mainly solving the problems that the existing image enhancement methods have a large amount of calculation, poor real-time performance, and are difficult to be applied in engineering.

[0006] This method uses the Stokes vector to transform the original problem of the optimal difference component that is difficult to solve into several easy-to-solve first-order optimization problems, and obtains the enhanced image that meets the conditions through iteration.

[0007] The detailed steps adopted by the present invention to solve the technical problem are as follows:

[0008] Step 1: Calculate the global parameters of the entire image;

[0009] Step 2: Use the parameters to solve the optimal difference image of the image;

[0010] Step 3: Calculate the polarization degree of the backscattered light;

[0011] Step 4: Using the optimal difference image and the polarization degree of the backscattered light, the polarization degree of the reflected light of the target object is solved;

[0012] Step 5: Estimate the enhanced image.

[0013] Furthermore, a fast underwater image enhancement method integrating polarization information is provided, wherein the specific method is as follows:

[0014] Step 1: Calculate the global parameters of the entire image

[0015] The polarization imaging method based on the Stokes vector usually takes an arbitrary direction as the 0° reference direction, and then rotates the polarizer 45°, 90° and 135° according to the reference direction to obtain three other polarization azimuth images, whose intensities are recorded as I(0), I(45), I(90) and I(135) respectively. According to the definition of the Stokes vector and the set of simultaneous equations, the Stokes parameters can be solved as:

[0016] S0=I(0)+I(90) (25)

[0017] Q=I(0)-I(90) (26)

[0018] U=I(45)-I(135) (27)

[0019] Among them, S0 is the total light intensity of the scene, which is equal to the overall intensity of the image, that is, S0 = I; Q is the intensity difference between the horizontal and vertical directions; U is the intensity difference between the 45° and 135° directions. The expressions for polarization degree and polarization angle can be expressed as:

[0020]

[0021]

[0022] Step 2: Use parameters to find the optimal difference image of the image

[0023] According to Malus' law, when the polarization direction of the backscattered light and the transmission direction of the mutually orthogonal analyzer are both 45°, polarization difference imaging can filter out the backscattered light through the common-mode suppression effect of the optical analyzer. At this time, two images can be obtained from the analyzer: when the transmission axis is parallel to the θ direction, the captured image is I || (θ), and when the transmission axis is perpendicular to the θ direction, the captured image is I ⊥ (θ). For convenience, it will be abbreviated as I || and I ⊥ subsequently. According to the physical meaning of the Stokes vector, after the light is modulated by the mutually orthogonal analyzers, its light intensities become respectively:

[0024] I || (i) = I i - sin(2θ A )·Q i + cos(2θ A )·U i (30)

[0025] I ⊥ (i) = I i + sin(2θ A )·Q i - cos(2θ A )·U i (31)

[0026] where i represents the backscattered light (A) and the reflected light from the target object (T). From the principle of polarization difference, we have:

[0027] I pd (A) = I || (A) - I ⊥ (A) = Q A ·sin(2θ A ) - U A ·cos(2θ A ) (32)

[0028] I pd (T) = I || (T) - I ⊥ (T) = 2(cos(2θ A )·U T - sin(2θ A )·Q T ) (33)

[0029] where I pd (A) and I pd (T) represent the differential components of the backscattered light and the target reflected light respectively. Since tan(2θ A ) = UA / Q A , substituting into Equation (8) gives I pd (A) = 0. After filtering out the backscattered light, the only remaining differential component is the one reflected by the target, i.e.:

[0030] I pd = I || - I ⊥ = I pd (A) + I pd (T) = 2(cos(2θ A )·U T - sin(2θ A )·Q T ) (34)

[0031] In image enhancement technology, the light reflected by the target is usually considered as useful information, while the backscattered light should be filtered out as stray light. Therefore, the polarization component I pd of the light reflected by the target is the optimal polarization differential component.

[0032] Step 3: Solve the degree of polarization of the backscattered light

[0033] Decompose the backscattered light I(A) in two mutually perpendicular directions. Taking the 0° direction as the reference direction, the background light component in this direction is I0(A). As can be seen from the figure, the component parallel to the vibration direction of the background light can be expressed as

[0034]

[0035] The component perpendicular to the vibration direction of the backscattered light can be written as

[0036]

[0037] According to the definition of the polarization image, the polarization differential component of the backscattered light can be obtained from the following formula:[[]]

[0038] I pd (A) = I(A || ) - I(A ⊥ ) (37)

[0039] Let I0(A) = εI(A), where ε is a variable between 0 and 1. Combining with the definition of the degree of polarization, we can get:[[]]

[0040]

[0041] where p A represents the degree of polarization of the backscattered light. According to the relationship between the degree of polarization and the Stokes vector, we can obtain:[[]]

[0042] |Q(A)| / I(A) ≤ pA (39)

[0043] Then we have:

[0044] |2ε - 1| ≤ p A (40)

[0045] As can be seen from Equation (16)

[0046]

[0047] Combining Equation (14) and Equation (17), the polarization degree p of the backscattered light can be obtained A range of

[0048]

[0049] Step 4: Solve the polarization degree of the reflected light of the target object by using the optimal difference image and the polarization degree of the backscattered light

[0050] According to the definition of the polarization degree, the polarization component of the backscattered light can be obtained

[0051]

[0052] The image imaging model can be written as:

[0053] I = A ∞ +(J(x) - A ∞ )·t(x) (44)

[0054] Rewrite both the backscattered light and the target reflected light as the sum of the polarized part and the unpolarized part. Taking I(0) as an example, we have:

[0055]

[0056] where p A and p D represent the polarization degrees of the backscattered light and the target reflected light respectively, and J px represents the component of the polarization component of the target reflected light on the X-axis. In a certain direction, the polarization amount of light is only the polarization amounts of the target reflected light and the backscattered light. Then we have

[0057]

[0058] J px

[0059] J px = I(0)sin 2 θ A + cos 2 θ A [J p-I(90)] (47)

[0060] Substituting Equation (23) into Equation (21), the degree of polarization p of the target reflected light can be obtained. D 。

[0061] Step 5: Estimate the enhanced image.

[0062] Using the optimal polarization difference component (the polarization component of the target reflected light) and the degree of polarization of the target reflected light, the image after de-scattering can be obtained:

[0063]

[0064] Compared with the existing methods, the present invention has the following advantages:

[0065] (1) The present invention accurately calculates the orthogonal polarization image by using the Stokes vector, replacing the cumbersome manual rotation of the polarization device.

[0066] (2) The present invention fuses the optimal polarization difference component to achieve common-mode suppression of the backscattered light, reducing the algorithm operation time by about 70% compared with the traditional method.

[0067] When estimating the degree of polarization and polarization angle of the backscattered light, the present invention gets rid of the requirement of the original image for the empty area in the scene, improving the applicability of de-scattering imaging. Description of the Drawings

[0068] Figure 1 Schematic diagram of the polarization enhancement process of the exposed object;

[0069] Figure 2 Schematic diagram of the polarization enhancement process of the dark image. Detailed Embodiment

[0070] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0071] The present invention will be described in detail below with reference to the drawings, specific implementation manners and embodiments.

[0072] The present invention is a fast underwater image enhancement method that fuses polarization information. By applying the maximum-minimum method, the original difficult-to-solve quartic optimization problem is transformed into several easy-to-solve linear problems, and the detection waveform that meets the conditions is obtained through an iterative method.

[0073] The specific implementation process is as follows:

[0074] Step 1: Calculate the global parameters of the entire image

[0075] The polarization imaging method based on the Stokes vector usually takes an arbitrary direction as the 0° reference direction, and then rotates the polarizer 45°, 90° and 135° according to the reference direction to obtain three other polarization azimuth images, whose intensities are recorded as I(0), I(45), I(90) and I(135) respectively. According to the definition of the Stokes vector and the set of simultaneous equations, the Stokes parameters can be solved as:

[0076] S0=I(0)+I(90) (49)

[0077] Q=I(0)-I(90) (50)

[0078] U=I(45)-I(135) (51)

[0079] Among them, S0 is the total light intensity of the scene, which is equal to the overall intensity of the image, that is, S0 = I; Q is the intensity difference between the horizontal and vertical directions; U is the intensity difference between the 45° and 135° directions. The expressions for polarization degree and polarization angle can be expressed as:

[0080]

[0081]

[0082] Step 2: Use parameters to find the optimal difference image of the image

[0083] According to Malus's law, when the polarization direction of the backscattered light is 45° to the mutually orthogonal analyzer transmission direction, polarization differential imaging can filter out the backscattered light through the common mode suppression effect of the optical analyzer. At this time, two images can be obtained from the analyzer: when the transmission axis is parallel to the θ direction, the image is I || (θ), and when the transmission axis is perpendicular to the θ direction, the image captured is I ⊥ (θ). For convenience, it is abbreviated as I || and I ⊥ According to the physical meaning of the Stokes vector, after the light is modulated by mutually orthogonal analyzers, its light intensity becomes:

[0084] I || (i)=I i -sin(2θ A )·Q i +cos(2θ A )·U i (54)

[0085] I ⊥ (i)=I i+sin(2θ A )·Q i -cos(2θ A )·U i (55)

[0086] wherein, i represents the backscattered light (A) and the light reflected by the target object (T). According to the polarization difference principle, we have:

[0087] I pd (A) = I || (A) - I ⊥ (A) = Q A ·sin(2θ A ) - U A ·cos(2θ A ) (56)

[0088] I pd (T) = I || (T) - I ⊥ (T) = 2(cos(2θ A )·U T -sin(2θ A )·Q T ) (57)

[0089] wherein, I pd (A) and I pd (T) respectively represent the differential components of the backscattered light and the target reflected light. Since tan(2θ A ) = U A / Q A , substituting it into Equation (56) gives I pd (A) = 0. After filtering out the backscattered light, only the differential component reflected by the target remains, that is:

[0090] I pd = I|| - I ⊥ = I pd (A) + I pd (T) = 2(cos(2θ A )·U T -sin(2θ A )·Q T ) (58)

[0091] In image enhancement technology, it is usually considered that the light reflected by the target is useful information, while the backscattered light should be filtered out as stray light. Therefore, the polarization component I pd of the light reflected by the target is the optimal polarization difference component.

[0092] Step 3: Solve the degree of polarization of the backscattered light

[0093] The backscattered light I(A) is decomposed in two mutually perpendicular directions. Taking the 0° direction as the reference direction, the background light component in this direction is I0(A). As can be seen from the figure, the component parallel to the vibration direction of the background light can be expressed as

[0094]

[0095] The component perpendicular to the vibration direction of the backscattered light can be written as

[0096]

[0097] According to the definition of the polarization image, the polarization difference component of the backscattered light can be obtained from the following formula:

[0098] I pd (A) = I(A || ) - I(A ⊥ ) (61)

[0099] Let I0(A) = εI(A), where ε is a variable between 0 and 1. Combining with the definition of the degree of polarization, we can get:

[0100]

[0101] where p A represents the degree of polarization of the backscattered light. According to the relationship between the degree of polarization and the Stokes vector, we can obtain:

[0102] |Q(A)| / I(A) ≤ p A (63)

[0103] So we have:

[0104] |2ε - 1| ≤ p A (64)

[0105] From equation (64), we know that

[0106]

[0107] Combining equation (62) and equation (65), we can obtain the range of the degree of polarization p A of the backscattered light

[0108]

[0109] Step 4: Solve the degree of polarization of the reflected light of the target object by using the optimal difference image and the degree of polarization of the backscattered light

[0110] According to the definition of the degree of polarization, the polarization component of the backscattered light can be obtained

[0111]

[0112] The image imaging model can also be written as:

[0113] I = A ∞ +(J(x) - A ∞ )·t(x) (68)

[0114] Rewrite the backscattered light and the target reflected light as the sum of the polarized part and the unpolarized part. Taking I(0) as an example, we can get:

[0115]

[0116] Among them, p A and p D respectively represent the polarization degrees of the backscattered light and the target reflected light, and J px represents the component of the polarized component of the target reflected light on the X-axis. In a certain direction, the polarized amount of light is only the polarized amounts of the target reflected light and the backscattered light. So there is

[0117]

[0118] We can obtain J px

[0119] J px = I(0)sin 2 θ A + cos 2 θ A [J p - I(90)] (71)

[0120] Substitute Equation (71) into Equation (69) to obtain the polarization degree p D of the target reflected light.

[0121] Step 5: Estimate the enhanced image.

[0122] Using the optimal polarization difference component (the polarized component of the target reflected light) and the polarization degree of the target reflected light, the image after removing scattering can be obtained:

[0123]

[0124] To verify the effectiveness of the method of the present invention, it will be described in detail below in combination with experimental simulations.

[0125] A challenging problem in the underwater image enhancement method is to deal with the situation of insufficient illumination in a dark environment. At this time, artificial light sources are usually used to improve it. However, different from uniform natural light, artificial light sources have two situations:

[0126] (1) The light source is too strong, resulting in a strong exposure phenomenon for objects with a low degree of depolarization. The outline of the object can hardly be seen. To observe the performance of the algorithm when dealing with large-area exposure of objects, this paper increases the light intensity and uses a white small ball with a large light reflectivity as the observed object. As Figure 1 shown, (a) is the original image, and the small ball after exposure can hardly be seen in its original form; (b - e) are the polarization maps at 0°, 45°, 90°, and 135° respectively. The polarizer can filter out some stray light, and the result after being processed by the method of the present invention is shown in (f). Obviously, compared with the original image, the result after processing has greatly improved the problem of object exposure, and the outline and color of the small ball can be clearly seen. This is because the FUFP method not only utilizes the light filtering property of the polarizer but also takes into account the polarization state of the light reflected by the object in the method, and finally achieves positive results.

[0127] (2) Another situation is insufficient light, which will cause the energy of light not to be transmitted to a relatively long distance, making it impossible to effectively detect the dark part of the image. As Figure 2 shown, Figure 2 is the polarization enhancement process of the dark image. (A) are the polarization maps at 0°, 45°, 90°, and 135° respectively. The polarizer can filter out some stray light, and the result after being processed by the FUFP method is shown in (F). It can be seen from the figure that the clarity of the processed image has been significantly improved. At the same time, due to compensating for the energy of light attenuation, the brightness of the obtained image has also been significantly enhanced. From the result obtained by processing with the method of the present invention, it can be seen that the method of the present invention can effectively improve the image in the case of insufficient light. When the outline of the white ball can hardly be seen in a dark environment, the enhanced image can effectively retain information such as the color texture details of the image.

[0128] Compare the running time of the FUFP method with the running time of the image processed by the NLR method, as shown in Table 1. Since the NLR method needs to repeatedly calculate information such as transmittance and depth of field for each image, this greatly reduces the algorithm efficiency. On the contrary, for the FUFP method, there is no need to calculate the depth of field and transmittance additionally, nor is there a need for a human - machine interaction process, which better improves the average operation time of the underwater image enhancement method, increases the practicability in fields such as target recognition, and significantly improves the method efficiency.

[0129] Table 1 Comparison of running times

[0130]

[0131] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A fast underwater image enhancement method integrating polarization information, characterized in that: The Stokes vector is used to transform the original optimal differential component problem which is difficult to solve into several easy-to-solve primary optimization problems, and the enhanced image satisfying the conditions is obtained through iteration; the steps adopted are as follows: Step 1: Calculate the global parameters of the entire image; Step 2: Use the parameters to solve the optimal difference image of the image; Step 3: Calculate the polarization degree of the backscattered light; Step 4: Using the optimal difference image and the polarization degree of the backscattered light, the polarization degree of the reflected light of the target object is solved; Step 5: Estimate the enhanced image; In step 1, the specific method for calculating the global parameters of the entire image is as follows: The polarization imaging method based on the Stokes vector usually takes any direction as the reference direction of 0°, and then rotates the polarizer 45°, 90° and 135° according to the reference direction to obtain three other polarization azimuth images, whose intensities are recorded as I(0), I(45), I(90) and I(135) respectively; According to the definition of the Stokes vector and the simultaneous equations, the Stokes parameters can be solved as: S0=I(0)+I(90) (1) Q=I(0)-I(90) (2) U=I(45)-I(135) (3) Among them, S0 is the total light intensity of the scene, which is equal to the overall intensity of the image, that is, S0 = I; Q is the intensity difference between the horizontal and vertical directions; U is the intensity difference between the 45° and 135° directions; the expressions of polarization degree and polarization angle can be expressed as: In step 4, the specific method of solving the polarization degree of the reflected light of the target object by using the optimal difference image and the polarization degree of the backscattered light is as follows: According to the definition of polarization degree, the polarization component of the backscattered light can be calculated The image imaging model can be written as: I = A ∞ +(J(x) - A ∞ )·t(x) (7) Rewrite the backscattered light and the target reflected light into the sum of the polarized part and the unpolarized part. Taking I(0) as an example, we can get: where p A and p D represent the degrees of polarization of the backscattered light and the target reflected light respectively, and J px represents the component of the polarization component of the target reflected light on the X-axis; in a certain direction, the polarization amount of light only has the polarization amounts of the target reflected light and the backscattered light, so there is J can be obtained px J px = I(0)sin 2 θ A + cos 2 θ A [J p - I(90)] (10) Substituting equation (10) into equation (8), the degree of polarization p of the target reflected light can be obtained D .

2. The fast underwater image enhancement method integrating polarization information according to claim 1, wherein: In step 2, the specific method of using parameters to solve the optimal difference image of the image is as follows: According to Malus' law, when the polarization direction of the backscattered light and the transmission direction of the mutually orthogonal analyzer are both 45°, the polarization difference imaging can filter out the backscattered light through the common-mode suppression effect of the optical analyzer; at this time, two images can be obtained from the analyzer: when the transmission axis is parallel to the θ direction, the captured image is I || (θ), and when the transmission axis is perpendicular to the θ direction, the captured image is I ⊥ (θ), which will be abbreviated as I || and I ⊥ ; According to the physical meaning of the Stokes vector, after the light is modulated by the mutually orthogonal analyzers, its light intensities become respectively: I || (i) = I i -sin(2θ A )·Q i +cos(2θ A )·U i (11) I ⊥ I(i) = i + sin(2θ A ) · Q i - cos(2θ A ) · U i (12) Where i represents the backscattered light (A) and the reflected light from the target object (T); according to the polarization difference principle: I pd (A) = I || (A) - I ⊥ (A) = Q A ·sin(2θ A ) - U A ·cos(2θ A ) (13) I pd (T) = I || (T) - I ⊥ (T) = 2(cos(2θ A )·U T -sin(2θ A )·Q T ) (14) where, I pd (A) and I pd (T) respectively represent the differential components of the backscattered light and the target reflected light; Since tan(2θ A ) = U A / Q A , substituting into Equation (13) gives I pd (A) = 0. After filtering out the backscattered light, only the differential component reflected by the target remains, that is: I pd = I || -I ⊥ = I pd (A) + I pd (T) = 2(cos(2θ A ) · U T -sin(2θ A ) · Q T ) (15) In image enhancement technology, it is generally considered that the target reflected light is useful information, while the backscattered light should be filtered out as stray light; therefore, the polarization component I of the target reflected light pd is the optimal polarization difference component.

3. The fast underwater image enhancement method for fusing polarization information according to claim 2, characterized in that: In step 3, the specific method for solving the polarization degree of the backscattered light is as follows: The backscattered light I(A) is decomposed into two mutually perpendicular directions, with the 0° direction as the reference direction. The background light component in this direction is I0(A). As can be seen from the figure, the component parallel to the vibration direction of the background light can be expressed as The component perpendicular to the vibration direction of the backscattered light can be written as According to the definition of polarization image, the polarization difference component of backscattered light can be obtained as follows: I pd (A) = I(A || ) - I(A ⊥ ) (18) Let I0(A) = εI(A), where ε is a variable between 0 and 1. Combined with the definition of the degree of polarization, we can obtain: Among them, p A represents the degree of polarization of the backscattered light; according to the relationship between the degree of polarization and the Stokes vector, it can be obtained that: |Q(A)| / I(A) ≤ p A (20) So we have: |2ε - 1| ≤ p A (21) From formula (21), we can know Combining Equation (19) and Equation (22), the polarization degree p of the backscattered light can be obtained A range 4. The fast underwater image enhancement method for fusing polarization information according to claim 3, wherein: In step 5, the specific method for estimating the enhanced image is as follows: The de-scattered image can be obtained by using the optimal polarization difference component and the polarization degree of the target reflected light:

Images