Complex underwater environment scattering removal imaging method based on Mueller matrix

Abstract

The invention discloses a complex underwater environment de-scattering imaging method based on a Mueller matrix, and the method comprises the steps: measuring a Mueller matrix M of a target scene and illumination information of the target scene, and the illumination information comprises active illumination information and passive illumination information; when it is judged that the target scene only has active illumination through the illumination information, transmittance graph parameters are optimized by calculating the average linear polarization degree and the average circular polarization degree of the scene based on an image imaging model so as to recover a target clear image; when it is further judged that the target image is in the natural light and non-uniform illumination state, an image contrast enhancement algorithm is adopted to recover the target clear image; otherwise, a maximized backscattered light polarization degree algorithm or an enhanced image contrast algorithm can be adopted to recover the target clear image. The method is simple in calculation and easy to implement, does not need complex parameter estimation or configuration adjustment of a measuring instrument, and is suitable for real-time processing.

Description

Technical Field

[0001] The invention relates to the technical field of optical imaging, and in particular to a descattering imaging method for a complex underwater environment based on a Mueller matrix. Background Art

[0002] In an underwater environment, light is scattered and absorbed by suspended particles in the water during propagation, resulting in a significant decrease in imaging quality. This situation is particularly serious in turbid water bodies, which seriously affects the identification and detection of underwater targets. In order to solve this problem, descattering techniques based on polarization imaging are proposed in the prior art. These methods remove background scattered light and restore clear target images by detecting the polarization information of light. Among various underwater descattering methods, polarization underwater imaging has attracted widespread attention due to its simple equipment, ability to effectively remove background scattered light, and good balance between cost and effect. Descattering techniques based on physical models can be used to reconstruct blurred images. Schechner et al. proposed a classic polarization descattering method, which is to obtain a clear image by capturing images in two orthogonal polarization directions and enhance image contrast. In general, methods based on physical scattering models can more effectively restore the original intensity image. However, these methods require determining the degree of polarization (DoP) of the object and background backscattered light, which is challenging in actual operation. Moreover, targets composed of multiple materials make the calculation of DoP complicated, resulting in frequent failures of existing methods. To address this problem, Liu derived the depolarization index (Dep index) from the Mueller matrix, characterized the scattering medium by estimating the transmittance map, and provided an optimization method based on the polarization imaging model.

[0003] The main reason for the degradation of image quality is that the backscattered light received by the camera produces the effect of superimposing a layer of haze on the image, which reduces the contrast when imaging underwater objects. Therefore, reducing backscattered light is crucial for clear underwater imaging. In theory, backscattered light is partially polarized, which makes polarization imaging a feasible method to suppress backscattered light and improve image quality. Wang demonstrated that backscattered light can be effectively removed by synergistically adjusting the states of the polarization generator (PSG) and the polarization analyzer (PSA). However, this approach increases the complexity of the system. In fact, Mueller matrix imaging can reveal more information than traditional optical imaging methods, thereby providing a deeper understanding of the underwater environment.

[0004] The aforementioned methods still have some limitations and are not applicable in all cases. Polarization-based descattering techniques usually assume that the backscattered light is partially polarized and often ignore the polarization characteristics of the target reflected light. In some cases, the target reflected light may be partially polarized, and sometimes even have more obvious polarization characteristics than the backscattered light. In fact, the non-uniform and complex polarization characteristics of the target are also very important and cannot be ignored. In addition, these methods rarely consider the impact of unpolarized sunlight on polarization imaging, which is a common situation when conducting underwater detection in shallow waters.

[0005] It can be seen that the existing polarization descattering technology has the following major defects in practical applications: 1. The background light has a great influence: the background scattered light is generally partially polarized, but in the actual underwater environment, the polarization degree of the background light may be low or uneven, resulting in poor descattering effect. 2. Ignoring the influence of natural light: Most existing methods only consider the polarization effect of active light sources, but ignore the role of non-polarized natural light generated by the sun in underwater imaging, especially in shallow sea areas. The influence of natural light is significant, which may cause distortion of imaging results. 3. Poor adaptability to complex environments: The polarization characteristics of the reflected light of the target object are complex, especially for transparent plastic objects, the image model-based method will directly fail. And for targets composed of a variety of materials, it is difficult to effectively estimate and remove the background light, resulting in the failure of image descattering. Summary of the invention

[0006] The purpose of the present invention is to provide a descattering imaging method for a complex underwater environment based on the Mueller matrix, which can realize effective restoration of underwater scattered images under the combined action of active polarization modulated light and passive non-polarized natural light.

[0007] The technical solution adopted by the present invention is:

[0008] The de-scattering imaging method for a complex underwater environment based on the Mueller matrix comprises the following steps:

[0009] Step 1, measuring and obtaining the Mueller matrix M of the target scene and the lighting information of the target scene, where the lighting information includes active lighting information and passive lighting information;

[0010] Step 2, judging whether the target scene has only active lighting through the lighting information; if so, executing step 3; otherwise, executing step 4;

[0011] Step 3, using a descattering algorithm based on an image imaging model to restore the target image, that is, optimizing the transmittance map parameters by calculating the average linear polarization degree and the average circular polarization degree based on the image imaging model to restore the target clear image, and executing step 5;

[0012] Step 4, judging whether it is natural light or non-uniform lighting state through lighting information; if so, using an enhanced image contrast algorithm to restore a clear image; otherwise, using a maximized backscattered light polarization algorithm or an enhanced image contrast algorithm to restore a clear image;

[0013] Among them, the algorithm for maximizing the degree of polarization of backscattered light is based on the 4×4 Mueller matrix polarization image. First, the Stokes vector input when the degree of polarization of the background scattered light is maximum is obtained through a nonlinear optimization algorithm, and a global calculation is performed to suppress the background scattered light to achieve clear underwater imaging; the algorithm for enhancing image contrast uses the weighted calculation of the Stokes vector to restore a clear image;

[0014] Step 5, output the restored clear underwater image and end.

[0015] Furthermore, the method for restoring a clear image of an underwater target based on the average linear polarization degree in step 3 specifically includes the following steps:

[0016] Step 3-1-1, combine the measured Mueller matrix M and the four Stokes vectors distributed on the equator of the Poincare sphere and Calculate the average degree of linear polarization AverageDoLP. The specific formula is as follows:

[0017]

[0018] in, Respectively represent the corresponding four linear polarization Stokes vectors; for a Stokes vector s = [IQ UV] T , the calculation formula of polarization degree DoP is

[0019] Step 3-1-2, calculating the transmittance graph t in the optical imaging physical model based on the average linear polarization degree AverageDoLP;

[0020]

[0021] Among them, m ΔL AverageDoLP 2 ;

[0022] Step 3-1-3, by calculating the total light intensity I Total The average intensity of the brightest 0.1% of pixels in (x,y) gives the scattered intensity at infinity. I Total (x, y) represents the intensity image captured by the camera to be processed;

[0023] Step 3-1-4, calculate the clear image I after descattering Obj (x, y), the specific calculation expression of the optical imaging physical model is:

[0024]

[0025] Where (x, y) represents the pixel position in the image.

[0026] Specifically, the clear image of underwater target restored based on the average linear polarization degree is expressed as This method is referred to as "Method 1L" in the present invention.

[0027] Furthermore, the method for restoring a clear image of an underwater target based on the average circular polarization degree in step 3 specifically includes the following steps:

[0028] Step 3-2-1, combine the measured Mueller matrix M and the two Stokes vectors distributed at the north and south poles of the Poincare sphere and Calculate the average degree of circular polarization AverageDoCP; the specific formula is as follows:

[0029]

[0030] in,

[0031] Step 3-2-2, calculating the transmittance graph t in the optical imaging physical model based on the average degree of circular polarization AverageDoCP;

[0032]

[0033] Among them, m ΔC AverageDoCP 2 ;

[0034] Step 3-2-3, by calculating the average I Total The intensity of the brightest 0.1% of pixels in (x,y) gives the scattering intensity at infinity. I Total (x, y) represents the intensity image captured by the camera to be processed;

[0035] Specifically, in traditional methods, the average pixel grayscale value of the background area of ​​the image is usually used to replace I Scat (x, y). However, for images with specular objects in the experiment, this method is not suitable. Instead, it is better to average I Total The intensity of the brightest 0.1% pixels at (x, y) is used to calculate I Scat (x,y).

[0036] Step 3-2-4: Calculate the clear underwater image I after removing scattering Obj (x, y), and the specific calculation expression is:

[0037]

[0038] where (x, y) represents the pixel position in the image.

[0039] Substitute Formula (1) and Formula (2) into Formula (3) to obtain This method is called "Method 1L" in the present invention. Similarly, substitute Formula (4) and Formula (5) into Formula (3) to obtain

[0040] Specifically, represent the restoration of the target clear image based on the average degree of circular polarization as This method is called "Method 1C" in the present invention.

[0041] In summary, the value ranges of AverageDoLP and AverageDoCP are between 0 and 1. Theoretically, different scattering mechanisms will result in different depolarization effects. When AverageDoLP > AverageDoCP, it indicates Rayleigh scattering. On the contrary, when AverageDoLP < AverageDoCP, it indicates Mie scattering. If the two values are equal, it indicates that the scattering is in the Rayleigh-Mie scattering transition region.

[0042] Furthermore, the specific steps of using the algorithm of maximizing the polarization degree of backscattered light to restore the clear image in Step 4 are as follows:

[0043] Step 4-1-1: Select the background area without objects from the measured Mueller matrix M image, and average all the pixels in the selected area to obtain the Mueller matrix M of the backscattered light b ;

[0044] Step 4-1-2: By calculating Furthermore, obtain the polarization degree of and apply the constrained nonlinear optimization algorithm to calculate the incident Stokes vector when the background polarization degree is the maximum where the initial incident Stokes vector is The corresponding constraint conditions are

[0045] Specifically, through the above steps, the problem of calculating the maximum DoP of the backscattered light is transformed into finding the minimum value of . For example, as an initial input Stokes vector, it is usually set to The corresponding constraints are The subscripts 0-3 represent the number of Stokes vectors. Then, a nonlinear optimization algorithm is applied to optimize the current initial vector to obtain another one with the maximum DoP that provides the background.

[0046] Step 4-1-3, calculate and obtain the clear image after descattering The specific expression is:

[0047]

[0048] Where M is the entire Mueller matrix polarization image including background and target; is with Orthogonal Stokes vectors.

[0049] Furthermore, the specific steps of restoring a clear image using the enhanced image contrast algorithm in step 4 are as follows:

[0050] Step 4-2-1, select N Stokes vectors S with a number of not less than 4 from the Poincare sphere i ;

[0051] Step 4-2-2, calculate each Stokes vector S i The corresponding weight i ;

[0052] weight i =max(1-DoP(MS i ))-min(1-DoP(MS i ))

[0053] Step 4-2-3, calculate and obtain the clear image I after descattering Obj , expressed as

[0054]

[0055] Furthermore, the Stokes vector S in step 4-2-1 i The number N is 4, 6, 8, 12 or 20.

[0056] Further, eight Stokes vectors S are selected on the Poincare sphere i , as follows:

[0057]

[0058] Eight Stokes vectors S i It has a good balance between computational efficiency and recovery results.

[0059] The present invention adopts the above technical solution and has the following technical advantages: 1. Improve the descattering effect under the influence of background light, and can effectively remove background scattered light under different types of background light conditions, significantly improving the imaging quality of underwater targets, especially under low polarization or uneven polarization background light conditions. It can still effectively remove background scattered light. 2. Enhance the imaging effect under non-polarized natural lighting conditions, effectively consider and process the influence of non-polarized natural light on polarization imaging in underwater environments, especially in shallow water areas, and still achieve effective descattering imaging effects to ensure the authenticity and accuracy of image restoration. 3. Improve the imaging adaptability in complex environments, and be able to handle the descattering imaging method with complex polarization characteristics of transparent plastics and multiple target objects, ensuring the reliability and accuracy of image restoration. The present invention has the ability to handle complex target objects without assuming the polarization characteristics of the target reflected light. It can handle complex targets composed of transparent plastics and multiple objects, ensuring the imaging effect in complex environments.

[0060] In summary, the methods proposed in the present invention do not require any assumptions about the polarization degree of the target, thus overcoming the limitation of some methods that assume the polarization degree of the target reflected light is zero, while assuming the polarization degree is zero may cause the image restoration of highly polarized objects to fail. The present invention is simple to calculate, easy to implement, does not require complex parameter estimation or configuration adjustment of the measuring instrument, and is suitable for real-time processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments;

[0062] Figure 1 It is a physical model of underwater polarization imaging under the influence of both underwater active illumination and passive illumination;

[0063] FIG2( a ) is a schematic diagram of the principle of an underwater imaging device based on the Mueller matrix;

[0064] Figure 2(b) shows the underwater imaging experimental setup based on the Mueller matrix;

[0065] Figure 3 Normalized Mueller matrix measurement images of different objects in turbid water under active polarization modulated light illumination;

[0066] Figure 4 Normalized Mueller matrix measurement images of different objects in turbid water illuminated by active polarization modulated light and natural light;

[0067] Figure 5 This is a schematic diagram comparing the recovery results of underwater targets under different scattering coefficients using three methods under active polarization modulated light illumination;

[0068] Figure 6Schematic diagram of intensity distribution along the red dashed line between the three methods under active polarization modulated light illumination;

[0069] FIG7( a ) is a schematic diagram of the contrast C of the three methods at different scattering coefficients under active polarization modulated light illumination;

[0070] FIG7( b ) is a schematic diagram of the standard deviation STD of the three methods at different scattering coefficients under active polarization modulated light illumination;

[0071] Figure 8 A schematic diagram comparing the recovery results of underwater targets at different scattering levels using three methods under active polarization modulated light and natural light illumination;

[0072] Fig. 9 Schematic diagram of intensity distribution along the red dashed line for the three methods under active polarization modulated light and natural light illumination;

[0073] FIG10( a ) is a schematic diagram of the contrast ratio C of the three methods at different scattering coefficients under active polarization modulated light and natural light illumination;

[0074] FIG10( b ) is a schematic diagram of the standard deviation STD of the three methods at different scattering coefficients under active polarization modulated light and natural light illumination. Implementation

[0076] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0077] like Figure 1 to 1 As shown in FIG. 0 , the present invention discloses a descattering imaging method for a complex underwater environment based on a Mueller matrix, which comprises the following steps:

[0078] Step 1, measuring and obtaining the Mueller matrix M of the target scene and the lighting information of the target scene, where the lighting information includes active lighting information and passive lighting information;

[0079] Step 2, judging whether the target scene has only active lighting through the lighting information; if so, executing step 3; otherwise, executing step 4;

[0080] Step 3, using a descattering algorithm based on an image imaging model to restore the target image, that is, optimizing the transmittance map parameters by calculating the average linear polarization degree and the average circular polarization degree based on the image imaging model to restore the target clear image, and executing step 5;

[0081] Step 4, judging whether it is natural light or non-uniform lighting state through lighting information; if so, using an enhanced image contrast algorithm to restore a clear image; otherwise, using a maximized backscattered light polarization algorithm or an enhanced image contrast algorithm to restore a clear image;

[0082] Among them, the algorithm for maximizing the degree of polarization of backscattered light is based on the 4×4 Mueller matrix polarization image. First, the Stokes vector input when the degree of polarization of the background scattered light is maximum is obtained through a nonlinear optimization algorithm, and a global calculation is performed to suppress the background scattered light to achieve clear underwater imaging; the algorithm for enhancing image contrast uses the weighted calculation of the Stokes vector to restore a clear image;

[0083] Step 5, output the restored clear underwater image and end.

[0084] Furthermore, in step 3, restoring a clear image of an underwater target based on the average linear polarization degree specifically includes the following steps:

[0085] Step 3-1-1, combine the measured Mueller matrix M and the four Stokes vectors distributed on the equator of the Poincare sphere and Calculate the average degree of linear polarization AverageDoLP;

[0086]

[0087] in, Respectively represent the corresponding four linear polarization Stokes vectors; for a Stokes vector S = [IQ UV] T , the calculation formula of polarization degree DoP is

[0088] Step 3-1-2, calculating the transmittance graph t in the optical imaging physical model based on the average linear polarization degree AverageDoLP;

[0089]

[0090] Among them, m ΔL AverageDoLP 2 ;

[0091] Step 3-1-3, by calculating the total light intensity I Total The average intensity of the brightest 0.1% of pixels in (x,y) gives the scattered intensity at infinity. I Total (x, y) represents the intensity image captured by the camera to be processed;

[0092] Step 3-1-4, calculate the clear image I after descatteringObj (x, y), the specific calculation expression of the optical imaging physical model is:

[0093]

[0094] Where (x, y) represents the pixel position in the image.

[0095] Specifically, the clear image of underwater target restored based on the average linear polarization degree is expressed as This method is referred to as "Method 1L" in the present invention.

[0096] Furthermore, in step 3, restoring a clear image of an underwater target based on the average circular polarization degree specifically includes the following steps:

[0097] Step 3-2-1, combine the measured Mueller matrix M and the two Stokes vectors distributed at the north and south poles of the Poincare sphere and Calculate the average degree of circular polarization AverageDoCP; the specific formula is as follows:

[0098]

[0099] in,

[0100] Step 3-2-2, calculating the transmittance graph t in the optical imaging physical model based on the average degree of circular polarization AverageDoCP;

[0101]

[0102] Among them, m ΔC AverageDoCP 2 ;

[0103] Step 3-2-3, by calculating the average I Total The intensity of the brightest 0.1% of pixels in (x,y) gives the scattering intensity at infinity. I Total (x, y) represents the intensity image captured by the camera to be processed;

[0104] Specifically, in traditional methods, the average pixel grayscale value of the background area of ​​the image is usually used to replace I Scat (x, y). However, for images with specular objects in the experiment, this method is not suitable. Instead, it is better to average I Total The intensity of the brightest 0.1% pixels at (x, y) is used to calculate I Scat (x,y).

[0105] Step 3-2-4, calculate and obtain the clear underwater image I after descatteringObj (x,y), the specific calculation expression is:

[0106]

[0107] Where (x, y) represents the pixel position in the image.

[0108] Substituting formula (1) and formula (2) into formula (3), we can obtain This method is referred to as "Method 1L" in the present invention. Similarly, substituting formula (4) and formula (5) into formula (3) yields

[0109] Specifically, the clear image of underwater target restored based on the average circular polarization degree is expressed as This method is referred to as "Method 1C" in the present invention.

[0110] In summary, the values ​​of AverageDoLP and AverageDoCP range from 0 to 1. In theory, different scattering mechanisms will lead to different depolarization effects. When AverageDoLP>AverageDoCP, it indicates Rayleigh scattering. On the contrary, when AverageDoCP>AverageDoLP, it indicates Mie scattering. If the two values ​​are equal, it indicates that the scattering is in the Rayleigh-Mie transition region.

[0111] Furthermore, the specific steps of restoring a clear image using the algorithm for maximizing the degree of polarization of backscattered light in step 4 are as follows:

[0112] Step 4-1-1, select a background area without an object from the measured Mueller matrix M image, and average all pixels in the selected area to obtain the Mueller matrix M of the backscattered light. b ;

[0113] Step 4-1-2, by calculating Then get Degree of polarization The incident Stokes vector when the background polarization degree is maximum is calculated by using a constrained nonlinear optimization algorithm. The initial incident Stokes vector is The corresponding constraints are

[0114] Specifically, the above steps are used to transform the problem of calculating the maximum DoP of backscattered light into obtaining For example, as an initial input Stokes vector, it is usually set to The corresponding constraints are Subscripts 0-3 represent the number of Stokes vectors. Then, a nonlinear optimization algorithm is applied to transform the current first guess vector into another one with the maximum DoP that provides the background.

[0115] Step 4-1-3, calculate and obtain the clear image after descattering The specific expression is:

[0116]

[0117] Where M is the entire Mueller matrix polarization image including background and target; is with Orthogonal Stokes vectors.

[0118] Furthermore, the specific steps of restoring a clear image using the enhanced image contrast algorithm in step 4 are as follows:

[0119] Step 4-2-1, select N Stokes vectors S with a number of not less than 4 from the Poincare sphere i ;

[0120] Step 4-2-2, calculate each Stokes vector S i The corresponding weight i ;

[0121] weight i =max(1-DoP(MS i ))-min(1-DoP(MS i ))

[0122] Step 4-2-3, calculate and obtain the clear image I after descattering Obj , expressed as

[0123]

[0124] Furthermore, the Stokes vector S in step 4-2-1 i The number N is 4, 6, 8, 12 or 20.

[0125] Further, eight Stokes vectors S are selected on the Poincare sphere i , as follows:

[0126]

[0127] Eight Stokes vectors S i It has a good balance between computational efficiency and recovery results.

[0128] The present invention develops three polarization descattering methods by using Mueller matrix analysis: optimizing image model, suppressing background light and enhancing image contrast.

[0129] The present invention studies the influence of backscattering and polarization of target signals on underwater imaging. Importantly, the present invention verifies the influence of non-polarized light on underwater Mueller matrix measurement through experiments, providing certain guidance for actual underwater measurement.

[0130] Descattering optimization method based on imaging model: 1. Use Mueller matrix to obtain polarization information of underwater images. 2. Calculate the average degree of linear polarization (AverageDoLP) and average degree of circular polarization (AverageDoCP) of the scene, and restore the target image by optimizing the transmittance map parameters. 3. Perform image restoration under various scattering conditions to verify the descattering effect of the imaging model optimization method.

[0131] Method to maximize background light polarization: 1. Select the background area without objects in the image and obtain its Mueller matrix M b 2. Determine the incident Stokes vector that maximizes the background light polarization through a nonlinear optimization algorithm 3. Perform descattering processing by optical means to obtain a clear image of the target after descattering the light.

[0132] Image contrast enhancement method: 1. Select 8 (or 4, 6, 12, 20) Stokes vectors distributed on the Poincare sphere. 2. Calculate the weight corresponding to each Stokes vector and apply it to underwater image restoration, especially for descattering imaging under the influence of non-polarized natural light. 3. Verify its effectiveness under complex lighting conditions through experiments to ensure significant improvement of image contrast and clarity.

[0133] The specific principle of the present invention is described in detail below:

[0134] Description of underwater scattering physical model: Based on the Jaffe–McGlamery imaging model, the imaging model in a typical scattering medium shows that the light captured by the camera I Total (x, y) consists of two parts. The first part is the direct transmitted light, which represents the object signal I weakened by absorption and scattering. Obj (x,y). The second part is the scattered light I Scat (x, y), which includes forward scattered light (usually negligible) and backscattered light resulting from multiple scattering interactions between particles in the water. The image captured by the camera can be represented as:

[0135] I Total (x,y)=I Obj (x,y)+I Scat(x,y)(9)

[0136] Different from the existing studies which only focus on polarization imaging under active or passive illumination, the present invention considers both active and passive illumination. Figure 1 The schematic diagram in Figure 1 shows the transmission of light in active and passive underwater imaging. In the active underwater polarization imaging model, the fully polarized light emitted by the active polarization light source becomes partially polarized light due to scattering and absorption in water. Obj (x,y) and I Scat Both (x,y) components are partially polarized, but their vibration directions are different. In the passive underwater imaging model, the illumination mainly comes from unpolarized natural light. Background scattered light I Scat The scattered light from seawater contains some polarization information, while the reflected target light usually lacks polarization information. Therefore, a significant difference between active polarization imaging and passive polarization imaging is that in active polarization imaging, the polarization of the target information light cannot be ignored.

[0137] Depending on the geometry and physical properties of the scattering particles, the polarization of light changes when it passes through a scattering medium. The complete polarization state can be expressed as the Stokes vector S = [I, Q, U, V] T Indicates, where I is the total light intensity, Q and U represent linear polarized light in two directions, and V represents circular polarized light. The degree of polarization DoP is a key parameter in the polarization-based descattering model, and the calculation formula is as follows:

[0138]

[0139] 1. De-scattering solution based on imaging model:

[0140] According to the imaging model, an ideal clear underwater image should only include I Obj , and can be expressed as:

[0141]

[0142] By obtaining t and Finally, a clear image of the underwater target can be reconstructed. Obj (x,y). Where (x,y) represents the pixel position in the image, and is the backscattering intensity at infinity. In traditional methods, the average pixel grayscale value of the background area of ​​the image is usually used instead of I Scat (x, y). However, for images with specular objects in the experiment, this method is not suitable. Instead, it is better to average I Total The intensity of the brightest 0.1% pixels at (x, y) is used to calculate I Scat (x,y).t represents the medium transmission diagram, which quantifies the attenuation of light during propagation in water and has been discussed in many studies.

[0143] The present invention estimates t based on the average linear polarization degree (AverageDoLP) and the average circular polarization degree (AverageDoCP) of the entire image. In order to define AverageDoLP, the present invention combines the measured Mueller matrix M and the four Stokes vectors distributed on the equator of the Poincare sphere and Calculate the average degree of linear polarization AverageDoLP. The specific expression is as follows:

[0144]

[0145] in, Respectively represent the corresponding four Stokes vectors;

[0146] Similarly, the present invention further defines AverageDoCP to evaluate the degree of depolarization relative to circularly polarized light, and the two Stokes vectors distributed on the north and south poles of the Poincare sphere are and The specific formula is as follows:

[0147]

[0148] in,

[0149] The calculation formula of the transmission map t is as follows:

[0150]

[0151] Where mΔ is defined as AverageDoLP 2 or AverageDoCP 2 By using the max function, it is ensured that the value range of the transmission map t is between 0 and 1. In addition, in order to avoid pixel saturation in the result image, the present invention corrects the transmission map by adding 0.1 to the denominator.

[0152] Substituting formula (1) and formula (11) into formula (3), we can obtain This method is referred to as "Method 1L" in the present invention. Similarly, substituting formula (4) and formula (11) into formula (3) yields This method is referred to as "Method 1C" in the present invention. The value range of AverageDoLP and AverageDoCP is between 0 and 1. In theory, different scattering mechanisms will lead to different depolarization effects. When AverageDoLP>AverageDoCP, it indicates Rayleigh scattering. On the contrary, when AverageDoCP>AverageDoLP, it indicates Mie scattering. If the two values ​​are equal, it indicates that the scattering is in the Rayleigh-Mie transition region.

[0153] 2. Maximizing the degree of polarization of backscattered light DoP:

[0154] In order to suppress the backscattered light, Wang maximized the DoP of the backscattered light polarization degree by cooperatively adjusting the states of PSG and PSA. Inspired by this method, the present invention tested a method to obtain the DoP of the backscattered light. To this end, a background area without an object is selected from the measured Mueller matrix M image, and all pixels in the area are averaged to obtain the Mueller matrix (M) contributed by the backscattered light. b ). Then, the background polarization degree is calculated to provide The incident Stokes vector at maximum This can be done using a constrained nonlinear optimization algorithm to transform the problem of calculating the maximum DoP of the backscattered light into the problem of finding The minimum value of . Among them, the initial incident Stokes vector is The constraints are

[0155] Specifically, the above steps are used to transform the problem of calculating the maximum DoP of backscattered light into obtaining For example, as an initial input Stokes vector, it is usually set to The corresponding constraints are Subscripts 0-3 represent the order of the Stokes vector. Then, a nonlinear optimization algorithm is applied to convert the current Stokes vector Another maximum background polarization degree is calculated

[0156] Finally, the underwater target image after descattering It is expressed as:

[0157]

[0158] Where M is the Mueller matrix of the target scene, which contains the entire Mueller matrix image of the background and the target. is with Orthogonal Stokes vectors. The biggest advantage of “Method 2” is that it is a purely optical process, does not require complex instrument configuration, and ensures maximum background polarization.

[0159] 3. Solution to enhance image contrast:

[0160] Since the light intensity reaching the CCD is significantly weakened after passing through the polarization device, the contrast of the Mueller matrix polarization image is not high, which affects the imaging quality. Therefore, when applying the polarization-based descattering method, it is necessary to enhance the image contrast. The present invention proposes a descattering imaging method, called "Method 3", which does not require optimized configuration and calculation, but focuses on enhancing image contrast. Descattering target image I Obj It is expressed as:

[0161]

[0162] Where, with each Stokes vector S i The corresponding weight i By weight i =max(1-DoP(MS i ))-min(1-DoP(MS i )) is calculated. Stokes vector S i The number of can be selected from several options, such as 4, 6, 8, 12 or 20. In the experiment, eight Stokes vectors S on the Poincare sphere were selected i , which have a good balance between computational efficiency and recovery results, as follows:

[0163]

[0164] Theoretically, for scenes with only one type of target, the results obtained by "Method 3" and "Method 2" will be similar. However, when unpolarized natural light simultaneously illuminates the water body, especially at a 90° scattering angle, the polarization degree of the background will be significantly affected. In this case, "Method 3" has a clear advantage in recovering the target. Overall, none of the three proposed schemes requires any assumptions about the polarization degree of the target. This overcomes the limitation of some methods that assumes that the polarization degree of the target reflected light is zero, which may lead to failure in image recovery of highly polarized objects.

[0165] Effect experiment:

[0166] Experimental setup and materials: Although natural light (such as the sun) is unpolarized, it usually has polarization properties in underwater environments. This polarization comes from molecular and aerosol scattering in the atmosphere, molecular and water scattering in the ocean, and Fresnel refraction through the air-sea interface. Therefore, in actual measurements, the influence of natural light on underwater Mueller matrix imaging cannot be ignored. In the experiment, the present invention simulates natural sunlight by halogen lamps, and only considers the scattering effect in water.

[0167] The experimental setup is shown in Figure 2(a). The active illumination Mueller matrix (MM) imaging instrument used adopts a backscattering configuration, where the light is reflected from the sample at a very small incident angle θ (about 10°). The detector and active LED illumination device are placed in the xy plane, perpendicular to the incident plane of natural light illumination. The light moves in the xz plane and changes its incident angle β with the z axis. This setup can simulate the sun rising from the east and setting in the west, while the underwater camera observes in the north-south direction.

[0168] At the same detection distance, red light in visible light scatters less than other wavelengths. Therefore, in Figure 2 (b), an LED (Thorlabs M660L4) with a central wavelength of 660nm was selected as the underwater active light source. The imaging device is a 12-bit digital monochrome CCD camera (Point Grey Research, Grasshopper3 GS3-U3-32S4M). Both PSG and PSA are composed of a compensator and a polarizer (Edmund Optics NIR polarizing film). The system has a unique design of compensator, and calibration is essential to compensate for small offsets. The present invention uses a mirror as a system calibration.

[0169] To simulate the underwater environment, a transparent PMMA box (20cm×20cm×20cm) filled with water was used. The distance between the target and the detector was about 35cm. In the experiment, the box needed to be tilted at a certain angle to avoid reflection from the surface of the box. In order to demonstrate the image recovery capability under different polarization and scattering properties, the present invention selected three objects: transparent plastic with birefringence, metal with specular reflection, and plastic coins with diffuse surfaces.

[0170] In order to simulate different degrees of scattering in an underwater environment, the present invention uses different proportions of semi-skimmed milk. Considering that semi-skimmed milk contains spherical particles such as casein molecules (diameter 0.04-0.3μm) and fat globules (diameter 1-20μm), it is generally believed that it forms a Mie scattering system. However, fat particles usually tend to float on the water surface. If the detection is not performed from above the water surface, the observed water scattering is more likely to be attributed to smaller casein molecules, resulting in Rayleigh scattering. This phenomenon can be confirmed by calculating AverageDoLP>AverageDoCP, as shown in Table 1. Theoretically, the scattering coefficient (μ s ) is related to the particle radius, particle size distribution function and scattering efficiency factor. The present invention obtains the scattering coefficient μs of the solution by applying the Beer-Lambert law and assuming that the attenuation of light is caused by scattering. The present invention obtains a scattering coefficient of 0.06 cm for seawater at a wavelength of 660 nm. -1Therefore, the present invention adjusts the concentration of milk to achieve 0.12cm -1 、0.21cm -2 and 0.30cm -1 The scattering coefficient.

[0171] Measurement results of Mueller matrix image: In the Mueller matrix polarization measurement experiment, the polarized light S is actively modulated. in2 The Stokes vector S arriving at the detector out1 Given by the following equation:

[0172] S out1 =M PSA M1M PSG S in1 (12)

[0173] Among them, S in1 is the Stokes vector of the incident light. M1, M PSA and M PSG The Mueller matrices of the sample, PSA and PSG are respectively. Using the experimental setup shown in FIG2 , the present invention measures the Mueller matrices of three objects in turbid water under active modulated illumination. The results are shown in FIG2 . Figure 3 As shown, the image size is 1400 × 836 pixels, corresponding to a scattering coefficient of 0.21 cm for semi-skimmed milk in water. -1 The larger values ​​of the diagonal elements of the Mueller matrix compared to other elements indicate that depolarization caused by scattering in turbid water is the main mechanism of underwater imaging. 23 and m 32 The element is not zero because of the refraction effect caused by the tilted box walls. The tilt of the box is to prevent the specular reflection of the box walls from reaching the detector.

[0174] From unpolarized natural light S in2 The Stokes vector S out2 Unlike actively modulated polarized illumination:

[0175] S out2 =M PsA M2S in2 (13)

[0176] Among them, S in2 =[1 0 0 0] T ; The difference between M2 and M1 is mainly due to the different scattering angles. In addition, since there is no PSG used to generate modulated light, M2 cannot be fully measured and only the first column of M2 can be measured.

[0177] When polarized light I P and natural light I U When irradiating the samples simultaneously, the light intensity ratio between them is assumed to be I P / I U =w, then the final Mueller matrix measured can be written as:

[0178] M final =w·M1+(1-w)M ′ 2(14)

[0179] Where M1 is the Mueller matrix corresponding to active illumination; M ′ 2 is the Mueller matrix corresponding to passive illumination, of which only the first column is non-zero.

[0180] From formula (14), it can be seen that natural light only affects M final As in most experimental backscattering cases, the first column of M1 (m 00 ) will be zero, so it can be assumed that m 10 、m 20 and m 30 The element will only be contributed by passive lighting. The incident angle β of the natural light source can be estimated by the following formula:

[0181]

[0182] Under active modulation and lighting, the present invention measures the Mueller matrix of three objects in turbid water. The results are as follows Figure 4 As shown. The scattering coefficient is 0.21cm -1 , the incident angle of the light is β = 18°. In this case, the intensity of natural light is significantly greater than that of modulated light.

[0183] Results and Discussion: (1) When only active illumination is used, the three technical solutions of the present invention are applied to obtain descattered target images using active modulated illumination at different scattering levels, such as Figure 5 As shown. 00 The element represents the intensity of the original image. It can be seen that in the intensity images with different scattering levels, it is difficult to distinguish the details of the transparent plastic based on the original light intensity.

[0184] In "Method 1", the original grayscale features of different objects are better preserved, and the clear outline of transparent plastic is effectively restored at different scattering levels. The present invention calculates the AverageDoLP and AverageDoCP of the entire image, as shown in Table 1 (left). The results show that in the three concentrations, the linear polarization degree is slightly higher than the circular polarization degree, and the difference between the two is between 0.04 and 0.06, indicating Rayleigh scattering. Due to this slight difference, the descattering effect achieved using linear polarization information and circular polarization information is similar.

[0185] Figure 1: AverageDoLP and AverageDoCP data measured under different scattering coefficients

[0186]

[0187] “Method 2” not only restores the transparent plastics, but also highlights their birefringent properties more clearly. The maximum DoP of the background was obtained to be 0.57, 0.55 and 0.53 at three scattering levels. Figure 5 The red box in the middle marks the selected background area. The results obtained by "Method 3" are very similar to those of "Method 2", both of which improve the uniformity of light intensity and are effective in detecting underwater targets with birefringence.

[0188] In order to quantify the clarity of the reconstructed image, the present invention analyzes the pixel intensity distribution along the white solid line and the red dashed line ( Figure 5 ) to assess the uniformity of the background and the distinguishability of different objects, such as Figure 6 As shown in Figure 2, the background light intensity shows a distribution pattern where the center area is brighter than the sides, which can be attributed to the light source's approximate Gaussian beam distribution. As the scattering increases, this light inhomogeneity significantly affects the light intensity distribution in the original image, making it difficult to distinguish the three objects at the same time.

[0189] like Figure 5 As shown in the figure, the position of the red dashed line passing through the positions of "Plastic Coin", "Metal", and "Transparent Plastic" effectively shows the image details. Figure 6 As shown in the figure, under low scattering conditions, all three methods provide excellent edge differentiation for objects, showing a clear "step" between the transparent plastic and the background, thereby enhancing target recognition. However, as the scattering coefficient increases, "Method 1" becomes more susceptible to uneven lighting and gradually loses the ability to distinguish the edges of objects. In contrast, the other two methods maintain the ability to distinguish the edges of the three objects. They rely less on the original light intensity image and more on the polarization information of the scene, thereby better compensating for uneven lighting.

[0190] As shown in Figure 7, in order to quantitatively evaluate the image quality, two non-reference image indicators are used: contrast (C) and standard deviation (STD). Contrast reflects the magnitude of the grayscale contrast of the image. The larger the value, the higher the contrast. The standard deviation describes the degree of dispersion of the grayscale values ​​of the image pixels relative to the mean. The larger the standard deviation, the better the image quality.

[0191] "Method 1" has the lowest contrast and standard deviation, and using linear polarization information (Method 1L) performs better than using circular polarization information (Method 1C) because the average linear polarization degree is larger than the average circular polarization degree. "Method 2" shows the best performance in terms of contrast, while "Method 3" performs well in terms of standard deviation. These metrics help evaluate image quality by quantifying contrast and standard deviation, providing an objective measure for evaluating and comparing the effectiveness of different descattering methods.

[0192] (2) Influence of natural light: Using active modulation and natural light illumination, descattered target images are obtained at different scattering levels, such as Figure 8 In this case, “Method 1” fails to restore the image of the transparent plastic at high scattering levels to make it consistent with the original intensity image m 00 The data in Table 1 (right) show that the overall polarization degree of the detection scene decreases due to the introduction of unpolarized light and the scattering effect of water particles, and in this case the average circular polarization degree is close to the average linear polarization degree.

[0193] Compared to the descattering effect under active illumination alone, the effect of "Method 2" is weakened, making it difficult to clearly image metal and plastic coins; only the birefringence caused by transparent plastic is still visible. The maximum polarization degree of the background is 0.36, 0.30, and 0.26 at the three scattering levels. In contrast, "Method 3" effectively restores the target image at different scattering levels and significantly corrects the illumination inhomogeneity caused by the simultaneous influence of two light sources.

[0194] Likewise, if Fig. 9 As shown in Figure 1, the intensity distribution of the target and background under active polarization modulation illumination and natural light illumination is obtained. Here, the light inhomogeneity is affected by the two light sources and is greatly affected by the incident angle of natural light.

[0195] “Method 1” can effectively distinguish the three target types at low scattering coefficients. However, as the degree of scattering increases, the change in light intensity becomes almost the same as the original image, resulting in a loss of sensitivity to the edge of the target. “Method 2” shows the worst de-scattering effect and is almost drowned by noise. This can be seen by its complete reliance on the polarization information of the scene without taking advantage of the Mueller matrix m 00 elements (i.e., the original light intensity image). As the degree of depolarization increases, m 00 The weight of the elements becomes more important. When the scene is fully deflected, all relevant information about the target will be included in m 00In the element, the polarization information is masked by noise. In comparison, the descattering performance of “method 3” is significantly better than the other two methods. It achieves effective detection of the edges between the three targets and between the target and the background, except for the high scattering conditions.

[0196] As shown in Figure 10, the quantitative evaluation results of the three schemes under active modulation and natural light illumination. The values ​​of the average linear polarization degree and the average circular polarization degree are almost the same, so the image restoration of “Method 1L” and “Method 1C” is consistent. Obviously, “Method 3” shows the best results for both evaluation metrics. This scheme shows better image quality, which is consistent with the results of visual evaluation and intensity distribution, further demonstrating the effectiveness and superiority of the proposed scheme.

[0197] Based on visual images and evaluation analysis, each of the proposed descattering methods has its own advantages. In deep-sea environments without natural light illumination, active illumination schemes are required, and image model-based methods are usually sufficient to obtain linear or circular polarization information. There is no need to obtain the complete Mueller matrix information of the scene, which greatly simplifies the complexity of the instrument. The method of maximizing background polarization is complicated by the need to modulate the instrument or manually select the background, so it is not suitable for real-time processing. By using the degree of polarization and calculating the corresponding weights to enhance the image contrast, the visibility and imaging quality of underwater targets can be improved. It can not only accurately clarify the target information, but also completely remove the backscattered light in the scene, thereby greatly improving the image contrast and clarity. This method is simple to calculate and can provide good descattering results in both active and natural light environments. Especially in the case of uneven natural light illumination, it shows significant advantages, which is very beneficial for target detection in shallow sea areas during the day.

[0198] In summary, the present invention proposes three methods based on Mueller matrix descattering imaging to improve the image quality of underwater targets. The present invention analyzes the applicable conditions of each method, taking into account the different polarization degrees of the sample, different scattering levels, and the presence of natural light. Under active modulated illumination, the present invention tends to use the image model-based method, but pays attention to the influence of light inhomogeneity. For underwater detection affected by natural light, the present invention prefers to use the proposed enhanced image contrast method. The method effectively alleviates the image inhomogeneity and does not require the estimation of the polarization degrees of backscattered light and target light, thereby avoiding inaccuracies in image restoration. In addition, it eliminates the need for human-computer interaction and bypasses the image quality change problem caused by parameter adjustment in "Method 2". Experimental results show that the proposed method has significant advantages in scattering suppression, detail recovery, and noise reduction in turbid water with different scattering levels. Clear underwater images are essential for reliable image recognition. The method proposed in the present invention will further enhance the effect of underwater image restoration and target recognition when combined with data fusion or deep learning technology.

[0199] The present invention adopts the above technical solution and has the following technical advantages: 1. Improve the descattering effect under the influence of background light, and can effectively remove background scattered light under different types of background light conditions, significantly improving the imaging quality of underwater targets, especially under low polarization or uneven polarization background light conditions. It can still effectively remove background scattered light. 2. Enhance the imaging effect under non-polarized natural lighting conditions, effectively consider and process the influence of non-polarized natural light on polarization imaging in underwater environments, especially in shallow water areas, and still achieve effective descattering imaging effects to ensure the authenticity and accuracy of image restoration. 3. Improve the imaging adaptability in complex environments, and be able to handle the descattering imaging method with complex polarization characteristics of transparent plastics and multiple target objects, ensuring the reliability and accuracy of image restoration. The present invention has the ability to handle complex target objects without assuming the polarization characteristics of the target reflected light. It can handle complex targets composed of transparent plastics and multiple objects, ensuring the imaging effect in complex environments.

[0200] In summary, the methods proposed in the present invention do not require any assumptions about the polarization degree of the target, thus overcoming the limitation of some methods that assume the polarization degree of the target reflected light is zero, which may lead to failure in image restoration of highly polarized objects. The present invention is simple to calculate, easy to implement, does not require complex parameter estimation or configuration adjustment of the measuring instrument, and is suitable for real-time processing.

[0201] Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians of the art without making creative work are within the scope of protection of the present application.

Claims

1. A de-scattering imaging method for complex underwater environments based on the Mueller matrix, characterized by: It includes the following steps: Step 1: Measure and obtain the Mueller matrix M of the target scene and the lighting information of the target scene, where the lighting information includes active lighting information and passive lighting information; Step 2: Determine whether the target scene has only active lighting based on the lighting information; If yes, go to step 3; Otherwise, go to step 4; Step 3: Use a descattering algorithm based on the image imaging model to restore the target image, that is, optimize the transmittance map parameters by calculating the average linear polarization degree and the average circular polarization degree based on the image imaging model to restore the target clear image, and execute step 5; Step 4: Determine whether the lighting condition is natural light or non-uniform lighting based on the lighting information; if so, restore the image to a clear state using an image contrast enhancement algorithm; otherwise, restore the image to a clear state using an algorithm that maximizes the degree of polarization of backscattered light or an image contrast enhancement algorithm; The algorithm for maximizing the degree of polarization of backscattered light is based on a 4×4 Mueller matrix polarization image. It first uses a nonlinear optimization algorithm to obtain the Stokes vector input when the degree of polarization of the background scattered light is maximized. A global calculation is then performed to suppress the background scattered light to achieve clear underwater imaging. The algorithm for enhancing image contrast uses a weighted calculation of the Stokes vector to restore a clear image. Step 5: Output the restored clear underwater image and end.

2. The descattering imaging method for complex underwater environments based on the Mueller matrix according to claim 1, characterized in that: The method for restoring a clear image of an underwater target based on the average linear polarization degree in step 3 specifically includes the following steps: Step 3-1-1, combine the measured Mueller matrix M and the four Stokes vectors distributed on the equator of the Poincare sphere and Calculate the average degree of linear polarization AverageDoLP. The specific formula is as follows: in, Represent the corresponding four linearly polarized Stokes vectors; for one Stokes vector S = [IQUV] T , the calculation formula of polarization degree DoP is Step 3-1-2, calculating the transmittance map parameter t in the optical imaging physical model based on the average linear polarization degree AverageDoLP; Among them, m ΔL AverageDoLP 2 ; Step 3-1-3, calculate the total light intensity I Tot2l The average intensity of the brightest 0.1% pixels at (x,y) gives the scattered intensity at infinity. I Total (x,y) represents the intensity image captured by the camera to be processed; Step 3-1-4, calculate the clear image I after de-scattering Obj (x,y), the specific calculation expression of the optical imaging physical model is: Here, (x,y) represents the pixel position in the image.

3. The descattering imaging method for complex underwater environments based on the Mueller matrix according to claim 1, characterized in that: The method for restoring a clear image of an underwater target based on the average circular polarization degree in step 3 specifically includes the following steps: Step 3-2-1, combine the measured Mueller matrix M and the two Stokes vectors distributed at the north and south poles of the Poincare sphere and Calculate the average degree of circular polarization AverageDoCP; the specific formula is as follows: in, Step 3-2-2, calculating the transmittance map parameter t in the optical imaging physical model based on the average degree of circular polarization AverageDoCP; Among them, m ΔC AverageDoCP 2 ; Step 3-2-3, calculate the total light intensity I Total The average intensity of the brightest 0.1% pixels at (x,y) gives the scattered intensity at infinity. I Total (x,y) represents the intensity image captured by the camera to be processed; Step 3-2-4, calculate and obtain the clear underwater image I after de-scattering Obj (x,y), the specific calculation expression is: Here, (x,y) represents the pixel position in the image.

4. The descattering imaging method for complex underwater environments based on the Mueller matrix according to claim 1, characterized in that: The specific steps for restoring a clear image using the algorithm for maximizing the degree of polarization of backscattered light in step 4 are as follows: Step 4-1-1, select a background area without objects from the measured Mueller matrix M image, and average all pixels in the selected area to obtain the Mueller matrix M of the backscattered light. b ; Step 4-1-2, by calculating Then get Degree of polarization The incident Stokes vector when the background polarization degree is maximum is calculated by applying the constrained nonlinear optimization algorithm. Among them, the initial incident Stokes vector is The corresponding constraints are Step 4-1-3, calculate and obtain the clear image after de-scattering The specific expression is: Where M is the entire Mueller matrix polarization image including background and target; is with Orthogonal Stokes vectors.

5. The descattering imaging method for complex underwater environments based on the Mueller matrix according to claim 1, characterized in that: The specific steps of using the image contrast enhancement algorithm to restore a clear image in step 4 are as follows: Step 4-2-1, select N Stokes vectors S with a number of not less than 4 from the Poincare sphere i ; Step 4-2-2, calculate each Stokes vector S i Corresponding weight weight i ; weight i =max(1-DoP(MS i ))-min(1-DoP(MS i )) Step 4-2-3, calculate the clear image I after de-scattering Obj , expressed as 6. The descattering imaging method for complex underwater environments based on the Mueller matrix according to claim 5, characterized in that: Stokes vector S in step 4-2-1 i The number N is 4, 6, 8, 12 or 20.

7. The descattering imaging method for a complex underwater environment based on the Mueller matrix according to claim 5 or 6, characterized in that: Choose eight Stokes vectors S on the Poincare sphere i , as follows: Eight Stokes vectors S i It has a good balance between computational efficiency and recovery results.

Images