Marine target depth map recovery method based on polarization image and regularization technology

Abstract

The invention discloses a sea target depth map recovery method based on a polarization image and a regularization technology, and belongs to the field of polarization image depth recovery, and the method comprises the following steps: obtaining an original polarization image of a target; based on the four-channel polarization image, according to an illumination estimation model and a Stokes parameter under mirror reflection and diffuse mixing illumination conditions, respectively obtaining zenith angle and azimuth angle images of the target; carrying out polarization information calculation by adopting an analog polarization method to obtain a non-light-intensity polarization image of the target; determining a target surface normal by referring to the polar coordinate system of the target surface normal and using a light source dot product principle, and further obtaining a normal vector field image; gradient information of a normal vector field image is converted into a large discretization sparse matrix under a Poisson equation, on the basis of solving the Poisson equation through a biconjugate gradient method, an adaptive fusion regularization formula is proposed to stably solve the Poisson equation, and depth image recovery of the marine target is completed.

Description

Technical Field

[0001] The present invention relates to the field of polarization image depth restoration, and is particularly concerned with a method for recovering depth maps of marine targets based on polarization images and regularization technology. Specifically, the invention relates to a new polarization image depth restoration method. Background Art

[0002] With the continuous breakthroughs in camera technology, obtaining polarization information has become more accurate and convenient. In the fields of image analysis and three-dimensional reconstruction, polarization information is closely related to the target shape, material composition and surface characteristics. Specifically, the geometric shape of the target and the optical properties of the surface material (such as the refractive index) will affect the polarization characteristics of its reflected light. Based on this, researchers can infer the normal direction of the target surface and complete the target depth information recovery. This method is called polarization information depth recovery method (SFP). This method has many excellent features and can accurately estimate the target surface normal in various harsh environments. Therefore, many scholars have conducted extensive research on target depth information recovery technology based on monocular polarization imaging.

[0003] Regularization techniques are widely used in image processing, particularly in noise suppression and edge preservation. By introducing prior knowledge or constraints, this technique transforms raw image data into an optimization problem, effectively mitigating the instability caused by noise interference in the image. Common regularization methods in image processing include derivative regularization and total variation regularization (TV). Total variation regularization is widely adopted because it can better preserve image edges while smoothing noise. Overall, regularization constraints can effectively suppress noise while preserving image details and edge information.

[0004] Despite the rapid development of SFP, the current estimation of target surface normals remains inaccurate, often resulting in loss of target edge information and shape distortion, which affects the accuracy and visualization of 3D shapes and urgently requires further improvement. While regularization techniques offer a variety of solutions for image processing, they have not yet been applied to recovering target depth maps from polarization images. Therefore, applying the advantages of regularization techniques in image processing to the field of monocular polarization 3D reconstruction and proposing a simple, easy-to-use, and widely applicable method for recovering depth from polarization images has become a pressing technical challenge. Summary of the Invention

[0005] In order to solve the above problems, the technical solution adopted by the present invention is: a method for recovering the depth map of marine targets based on polarization image and regularization technology, comprising the following steps: The method for recovering the depth map of marine targets based on polarization images and regularization technology includes the following steps: Obtain the original polarization image of the target; According to the Stokes parameters, four-channel polarization images of the target original polarization image are obtained at four polarization directions: 0°, 45°, 90°, and 135°; Based on the four-channel polarization image, the zenith angle and azimuth angle images of the target are obtained respectively according to the illumination estimation model and Stokes parameters under the conditions of specular reflection and diffuse mixed illumination. Based on the original polarization image, the adaptive threshold processing method is used to extract and calibrate the edge information of the target contour to obtain a binary mask image of the target; the simulated polarization method is used to calculate the polarization information to obtain a non-light intensity polarization image of the target; According to the non-light intensity polarization image, binary mask image, zenith angle image and azimuth angle image, the target surface normal is determined using the light source dot product principle with reference to the polar coordinate system of the target surface normal, and then the normal vector field image is obtained; According to the discrete form of the Poisson equation, the gradient information of the normal vector field image is converted into a large discrete sparse matrix. Based on the double conjugate gradient method to solve the Poisson equation, an adaptive fusion regularization formula is used to stably solve the Poisson equation and complete the depth image restoration.

[0006] Furthermore, the process of obtaining the zenith angle and azimuth angle images of the target based on the four-channel polarization image and the illumination estimation model and Stokes parameters under the conditions of specular reflection and diffuse mixed illumination is as follows: Based on the electromagnetic theory of light, the relationship between the amplitude and phase of the incident light, reflected light, and refracted light of the target is derived from Fresnel's law and Snell's law, thereby obtaining the zenith angle image under mixed illumination of specular reflection and diffuse reflection; Based on the four-channel polarization image information, the azimuth image information is obtained using the Stokes parameters.

[0007] Furthermore, according to the discrete form of the Poisson equation, the gradient information of the normal vector field image is converted into a large discrete sparse matrix. Based on the biconjugate gradient method for solving the Poisson equation, an adaptive fusion regularization formula is used to stably solve the Poisson equation to complete the process of depth image recovery as follows: S61: After obtaining the normal vector image, perform depth recovery on the normal vector image; S62: According to the discretized form of Poisson's equation, the gradient information of the normal vector field image is converted into a large discretized sparse matrix; S63: Discretize total variation regularization and derivative regularization under the Poisson equation; S64: According to the properties of the polarization image and the target global surface normal vector field information, two weight selection factors are given based on the consistency of the normal vector field; S65: Convert the unknown quantity in the two weight selection factors into an unknown parameter. S66: Based on the four regularization methods, the advantages of total variation regularization in preserving image edge information and the advantages of derivative regularization in suppressing noise are integrated to obtain a weighted formula; S67: To reduce the calculation error of the weighted formula, perform square addition and integration operations on the total variation regularization and derivative regularization to obtain an adaptive fusion regularization formula; S68: When solving the Poisson equation for target depth information, an adaptive fusion regularization formula is added to retain target edge information and stabilize the depth numerical solution, thereby completing depth image recovery.

[0008] Furthermore, the process of converting the gradient information of the normal vector field image into a large discrete sparse matrix according to the discretized form of the Poisson equation is as follows: The discretized form of Poisson's equation is used for deep numerical solution. The specific form of Poisson's equation is as follows: (12) in: represents the Laplace operator, and is a real or complex-valued equation, for a depth function in a continuous domain , Represents the pixel index in the depth image; : represents the horizontal coordinate value of the pixel index in the depth image; : represents the vertical coordinate value of the pixel index in the depth image; Formula (12) can be written as follows: (13) in, Represents the horizontal gradient information in the normal vector field image , longitudinal gradient information The one-dimensional vector of , Represents the pixel index of the normal field image, : represents the horizontal coordinate value of the pixel index of the normal vector field image; : The vertical coordinate value of the pixel index representing the normal vector field image; According to the five-point difference method, the discrete equation is: (14) in: Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column; Then the discretized form of the normal vector field image information under the Poisson equation is , is the coefficient matrix discretized using the five-point difference format, A one-dimensional vector representing the target depth information. The specific discretization matrix is ​​as follows: (15) For each discretized equation, :Indicates that in the first discrete equation, the first unknown The previous coefficient; :Indicates that In a discrete equation, the first unknown The previous coefficients, Indicates that in the first discrete equation, unknown number The previous coefficients, Indicates in In the discrete equations, unknown number The previous coefficients, : Represents the pixel index in the normal vector field image The sum of the corresponding transverse gradient and longitudinal gradient data; Represents the pixel index in the normal vector field image as The corresponding depth value; Represents the pixel index in the normal vector field image as The sum of the corresponding transverse gradient and longitudinal gradient data.

[0009] Furthermore, the process of expressing the total variation regularization and derivative regularization in a discretized form under the Poisson equation is as follows: Discretization form of total variation regularization under Poisson equation for: (16) Derivative regularization discretizes the Poisson equation in the form The formula is: (17) in: Represents the normal vector field image Rank The pixel index of the column.

[0010] Furthermore, according to the properties of the polarization image and the target global surface normal vector field information, two weight selection factors are given based on the consistency of the normal vector field, and the unknown quantities in the two weight selection factors are converted into an unknown parameter. According to four regularization methods, the advantages of total variation regularization in preserving image edge information and the advantages of derivative regularization in suppressing noise are integrated to obtain the weighted formula as follows: According to the properties of the polarization image and the target global surface normal vector field information, the first weight selection factor is given based on the consistency of the normal vector field. and the second weight selection factor : (18) (19) To obtain the target surface normal vector field information, is the rate of change of the normal vector, converting the two unknown variables into one unknown parameter , let it be: (20) According to the four regularization methods, the advantages of total variation regularization in preserving image edge information and the advantages of derivative regularization in suppressing noise are integrated, and the total variation regularization and derivative regularization methods are weighted and integrated. The specific weighted formula is for: (twenty one) Furthermore: the specific formula of the adaptive fusion regularization formula is: (twenty two) The discretized form of formula (22) under the Poisson equation is: (twenty three) Furthermore, the formula for adding the adaptive fusion regularization formula to retain the target edge information and stabilize the depth numerical solution is as follows: (twenty four) in: is the regularization coefficient, is the coefficient matrix after adding regularization; is the coefficient matrix discretized using the five-point difference format, Discretize the sum-of-squares and integral operations for total variation and derivative regularization under the Poisson equation.

[0011] The present invention proposes a method for recovering the depth map of marine targets based on polarization images and regularization technology. According to the advantages of regularization technology in image processing, it is applied to the field of monocular polarization 3D reconstruction. Based on the numerical solution of the Poisson equation, a new adaptive weight selection regularization formula is proposed to complete the target depth image recovery.

[0012] The present invention addresses the problems of detail loss, shape distortion and concavity that occur when performing depth recovery on current polarization information, and proposes a method for recovering the depth map of a polarization image of a maritime target based on regularization technology. Specifically, it includes: firstly, obtaining the zenith angle and azimuth angle of the target under mixed illumination of specular reflection and diffuse reflection from the polarization image through the Fresnel law and Stokes parameters respectively, then extracting the contour information of the target in the scene, and combining it with the light intensity information and polarization information for constraint, and then judging and determining the unique surface normal, and then proposing an adaptive fusion regularization formula based on the numerical solution of the Poisson equation to complete the target depth map recovery. At the same time, the experimental results in different scenarios show that the proposed method is suitable for depth map recovery of targets with uniform albedo, diffuse reflection and specular reflection, and provides a new solution for polarization three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0014] Figure 1 is a flow chart of the method of the present invention; Figure 2 This is the flowchart of polarization 3D reconstruction; Figure 3 is a schematic diagram of the surface normal polar coordinates; Figure 4 This is a partial enlarged image of the depth restored image of the indoor ship model according to the method of this application; Figure 5 This is a partial enlarged image of the depth restored image of the indoor ship model using the FC algorithm; Figure 6 The 3D reconstruction results of the long-distance offshore target using the method of the present application; (a) the original image of the long-distance offshore target, (b) the front view of the method proposed in the present application, and (c) the side view of the method proposed in the present application; Figure 73D reconstruction results of distant sea targets using the FC method; (a) original image of distant sea targets, (b) front view using the FC method, and (c) front view using the FC method; Figure 8 This is a partial enlarged image of the three-dimensional reconstruction result of a long-distance marine target using the method of this application. Figure 9 This is a partially enlarged view of the 3D reconstruction results of long-distance maritime targets using the FC algorithm. DETAILED DESCRIPTION

[0015] It should be noted that, unless there is any conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0016] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0017] Figure 1 is a flow chart of the method of the present invention; The method for recovering the depth map of marine targets based on polarization images and regularization technology includes the following steps: S1: Acquire the original polarization image of the target; S2: According to the Stokes parameters, four-channel polarization images of the target original polarization image are obtained at four polarization directions: 0°, 45°, 90° and 135°; S3: Based on the four-channel polarization image, the zenith angle and azimuth angle images of the target are obtained respectively according to the illumination estimation model and Stokes parameters under the conditions of specular reflection and diffuse mixed illumination; S4: using the original polarization image of the target, adopting relevant morphological processing methods such as adaptive threshold processing to extract and calibrate the edge information of the target outline in the reconstructed scene to obtain a binary mask image of the target, and using a simulated polarization method to calculate the polarization information to obtain a non-light intensity polarization image of the target; S5: According to the non-light intensity polarization image, the binary mask image, the zenith angle image and the azimuth angle image, the target surface normal is determined using the light source dot product principle with reference to the polar coordinate system of the target surface normal, thereby obtaining a normal vector field image; S6: According to the discrete form of the Poisson equation, the gradient information of the normal vector field image is converted into a large discrete sparse matrix. Based on the double conjugate gradient method for solving the Poisson equation, an adaptive fusion regularization formula is proposed to stably solve the Poisson equation and complete the depth image restoration.

[0018] Steps S1 / S2 / S3 are executed sequentially, while step S1 is executed after step S4, and finally S5 and S6 are executed; The experimental system primarily consists of a solar light source, a polarization camera, the target to be reconstructed, and a computer. The polarization camera captures the reflected light from the target surface, and the resulting digital image is then transmitted to the computer for processing. The polarization camera is mounted on a stable platform and secured with a tripod and gimbal to ensure high-quality image acquisition. Figure 1 The overall flow chart of the method in this paper is shown in Figure 1. According to the Stokes principle, the polarization image is calculated to obtain four polarization images with polarization directions of 0°, 45°, 90°, and 135, which are represented as S0, S1, S2, and S3 respectively in Figure 1; Figure 2 This is the flow chart of polarization 3D reconstruction. The principle is that the plane light wave enters the polarization acquisition system after being reflected on the target surface. From the propagation direction of the plane reflected light, it can be seen that the reflected light is parallel to the Z axis, so the zenith angle of the target is , the azimuth is , the surface normal of the target is ; To determine the normal direction of the target surface, first calculate the polarization degree according to the Stokes parameter. The specific solution formula is: (1) Among them, S0, S1, and S2 represent the four-channel polarization images with polarization directions of 0°, 45°, and 90° respectively, which are solved based on the original polarization image. Figure 1 They are represented as: S0, S1, S2; then solve the zenith angle under the mixed lighting of mirror reflection and diffuse reflection , specular reflection light and diffuse light Zenith angle relative to the target surface and refractive index The degree of polarization is expressed as follows: (2) (3) Then, based on the electromagnetic theory of light, the relationship between the amplitude and phase of the incident, reflected and refracted light of the target is derived from Fresnel's law and Snell's law, so as to obtain the zenith angle under the mixed illumination of specular reflection and diffuse reflection. for; (4) Then, according to the obtained four-channel polarization images (S0, S1, S2, S3), the azimuth angle is obtained using the Stokes parameters. ; (5) Adopting the adaptive threshold processing algorithm, the edge information of the target outline in the reconstructed scene is extracted and calibrated, and the overall information of the target is extracted and saved as a binary mask image; For non-intensity polarization image acquisition, the polarization information is calculated using the simulated polarization method and then saved as a non-intensity polarization image.

[0019] According to the non-light intensity polarization image, binary mask image, zenith angle image and azimuth angle image, the target surface normal is determined using the light source dot product principle with reference to the polar coordinate system of the target surface normal, and the process of obtaining the normal vector field image is as follows: For any given target in the Cartesian coordinate system, we can establish the target surface height With rectangular coordinates The relationship between Indicates that Represents the image pixel coordinates. The binary mask pixels of the target are represented by the set Assume that the target surface is convex, that is, there is no multiple reflection on the target surface, and the unit normal of the target surface is In the spatial coordinate system, it can be expressed as: (6) Figure 3 express The spatial geometric relationship between rectangular coordinates and polar coordinates. and azimuth , we can uniquely determine the unit vector of the target surface normal. and azimuth Substituting into formula (6), we can get the following formula: (7) (8) Assume function is integrable, For Direction partial derivative, For Directional partial derivatives; gradients in normal vector images The relationship is expressed as: (9) (10) The propagation theorem of light reflected from a plane, according to the direction of the light source , to determine the unique normal of the target surface. The specific formula is: (11) in is the normal With light source direction The angle between and are the directional modulus of the normal and the light source respectively. If the angle between the normal and the light source direction is less than 90°, it means that the normal direction is consistent with the light source direction, and then the normal vector image that best matches the light source direction is determined.

[0020] According to the discrete form of the Poisson equation, the gradient information of the normal vector field image is converted into a large discrete sparse matrix. Based on the biconjugate gradient method to solve the Poisson equation, the adaptive fusion regularization formula is used to stably solve the Poisson equation to complete the process of depth image recovery as follows: S61: After obtaining the normal vector image, perform depth recovery on the normal vector image; S62: According to the discretized form of Poisson's equation, the gradient information of the normal vector field image is converted into a large discretized sparse matrix; S63: Discretize total variation regularization and derivative regularization under the Poisson equation; S64: According to the properties of the polarization image and the target global surface normal vector field information, two weight selection factors are given based on the consistency of the normal vector field; S65: Convert the unknown quantity in the two weight selection factors into an unknown parameter. S66: Based on the four regularization methods, the advantages of total variation regularization in preserving image edge information and the advantages of derivative regularization in suppressing noise are integrated to obtain a weighted formula; S67: To reduce the calculation error of the weighted formula, perform square addition and integration operations on the total variation regularization and derivative regularization to obtain an adaptive fusion regularization formula; S68: When solving the Poisson equation for target depth information, an adaptive fusion regularization formula is added to retain target edge information and stabilize the depth numerical solution, thereby completing depth image recovery.

[0021] The process of converting the gradient information of the normal vector field image into a large discretized sparse matrix according to the discretized form of the Poisson equation is as follows: The discretized form of Poisson's equation is used for deep numerical solution. The specific form of Poisson's equation is as follows: (12) represents the Laplace operator, and is a real or complex valued equation. For a deep function in a continuous domain , Represents the pixel index in the depth image; : represents the horizontal coordinate value of the pixel index in the depth image; : represents the vertical coordinate value of the pixel index in the depth image; Formula (12) is generally written as: (13) in, Represents a one-dimensional vector containing the horizontal gradient and vertical gradient information in the normal vector field image. The specific expression is , represents the pixel index in the normal field image, : represents the horizontal coordinate value of the pixel index of the normal vector field image; : The vertical coordinate value of the pixel index representing the normal vector field image; According to the five-point difference method, the discrete equation is: (14) in: Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column; Then the discretized form of the normal vector field image information under the Poisson equation is , is the coefficient matrix discretized using the five-point difference format, A one-dimensional vector representing the target depth information. The specific discretization matrix is ​​shown below.

[0022] (15) For each discretized equation, :Indicates that in the first discrete equation, the first unknown The previous coefficient; :Indicates that In a discrete equation, the first unknown The previous coefficients, Indicates that in the first discrete equation, unknown number The previous coefficients, Indicates in In the discrete equations, unknown number The previous coefficients, : Indicates that the pixel index in the normal vector field image is The sum of the corresponding transverse gradient and longitudinal gradient data; Represents the pixel index in the normal vector field image as The corresponding depth value; Represents the pixel index in the normal vector field image as The sum of the corresponding transverse gradient and longitudinal gradient data.

[0023] Numerical solutions to the Poisson equation include: sparse matrix dimensionality reduction (SVD), algebraic multigrid (AMG), conjugate gradient (CG), biconjugate gradient (BiCG), and generalized minimum residual (GMRES). However, when reproducing the problem, it was found that the above five methods all performed poorly. In order to improve the 3D reconstruction effect of the target to be reconstructed, a regularization formula is introduced to improve the reconstruction accuracy of complex-shaped targets. A comparison of commonly used regularization methods is shown in Table 1 below: Table 1 Comparison of regularization methods

[0024] According to the advantages and disadvantages of the above four regularization methods, this paper integrates the total variation regularization and derivative regularization methods; The process of fusing the advantages of total variation regularization in preserving image edge information and derivative regularization in suppressing noise is as follows: (16) The discretization formula of derivative regularization under Poisson equation is: (17) in: Represents the normal vector field image Rank The pixel index of the column.

[0025] According to the properties of the polarization image and the target global surface normal vector field information, two weight selection factors are given based on the consistency of the normal vector field. The unknown quantities in the two weight selection factors are converted into an unknown parameter. According to four regularization methods, the advantages of total variation regularization in preserving image edge information and the advantages of derivative regularization in suppressing noise are integrated to obtain the weighted formula as follows: According to the properties of the polarization image and the target global surface normal vector field information, a weight selection factor based on the normal vector field consistency is given. and : (18) (19) To obtain the target surface normal vector field information, is the rate of change of the normal vector, converting the two unknown variables into one unknown parameter , let it be: (20) when Larger, When is small, derivative regularization plays a dominant role. It is relatively large and is mainly used to smooth noise, so that the weight ratio of the two formulas of total variation regularization and derivative regularization can be made according to the rate of change of the normal vector.

[0026] According to the four regularization methods, the advantages of total variation regularization in preserving image edge information and the advantages of derivative regularization in suppressing noise are integrated, and the total variation regularization and derivative regularization methods are weighted and integrated. The specific weighted formula is for: (twenty one) In order to reduce the calculation error of the fusion formula, the total variation regularization and derivative regularization are squared and integrated. The specific formula is: (twenty two) The discretized form of formula (22) under the Poisson equation is: (twenty three) The formula for adding the adaptive fusion regularization formula to retain the target edge information and stabilize the depth numerical solution is as follows: (twenty four) in: is the regularization coefficient, is the coefficient matrix after adding regularization; is the coefficient matrix discretized using the five-point difference format, Discretize the sum-of-squares and integral operations for total variation and derivative regularization under the Poisson equation.

[0027] We took a polarization image of a ship model in a dark indoor environment, and used the image to carry out a close-range indoor target depth information recovery experiment. During the experiment, the environmental interference factors were strictly controlled: reducing scene noise, ensuring that there was no mirror reflection in the surrounding environment, and selecting targets and surrounding environments that were only made of dielectric materials. These measures reduced the interference of the external environment on polarization imaging. This application then compared the proposed method with target depth map recovery methods, such as: traditional monocular polarization 3D reconstruction method (FC method), point cloud method, integral method and Poisson equation method. The comparison of the results of different methods is shown in the figure below. Figure 4 、 Figure 5 As shown; Figure 4 This is a partial enlarged image of the depth restored image of the indoor ship model according to the method of this application; Figure 5 A partial enlarged image of the depth recovery image reconstructed by the FC algorithm for the indoor ship model; To further verify the applicability of the proposed method in other scenarios, an experiment was conducted to restore the depth map of coastal buildings. First, the light source direction was determined based on the geometric relationship between the polarization camera shooting angle, the solar altitude angle at the time of shooting, and the solar azimuth angle. Then, the boundary mask information of the coastal buildings to be reconstructed was extracted using the adaptive threshold method. After that, the target depth information of the coastal buildings was restored. The specific process is shown in the figure below. Figure 6 — Figure 9 As shown; Figure 6 The three-dimensional reconstruction results of the long-distance offshore target by the method of the present application; wherein (a) the original image of the long-distance offshore target, (b) the front view of the method proposed in the present application, and (c) the side view of the method proposed in the present application; Figure 7 The 3D reconstruction results of the long-distance sea target using the FC method; (a) the original image of the long-distance sea target, (b) the front view using the FC method, and (c) the front view using the FC method; Figure 8 This is a partial enlarged image of the three-dimensional reconstruction result of the long-distance sea target using the proposed method. Figure 9 This is a partially enlarged view of the 3D reconstruction results of long-distance maritime targets using the FC method.

[0028] As can be seen from the figure, the conventional polarization 3D reconstruction method (FC method) distorts the target shape when restoring it, and the texture detail information is not obvious. The proposed method avoids the above problems, the target reconstruction details are clearer, and the overall visual observation effect is better.

[0029] This application addresses the problems of detail loss and shape distortion and concavity that typically occur when performing depth recovery using polarization information. A regularization-based depth recovery method for marine polarization images is proposed. Specifically, the method involves first obtaining the zenith angle and azimuth of a target under mixed specular / diffuse illumination from the polarization image using the Fresnel law and Stokes parameters. The method then extracts contour information from the target in the scene, constraining it with light intensity and polarization information to determine a unique surface normal. An adaptive fusion regularization formula is then proposed based on the numerical solution of the Poisson equation to restore target depth information. Experimental results in various scenarios demonstrate that the proposed method is applicable to both indoor and outdoor scenes with uniform albedo, diffuse reflection, and specular reflection. Compared with traditional monocular polarization 3D reconstruction methods (FC methods), this method offers greater ease of implementation, improved applicability, and higher recovery accuracy.

[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above examples, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the various technical solutions implemented in the present invention.

Claims

1. A method for recovering depth maps of marine targets based on polarization images and regularization technology, characterized by: The following steps are involved: Obtain the original polarization image of the target; According to the Stokes parameters, four-channel polarization images of the target original polarization image are obtained at four polarization directions: 0°, 45°, 90°, and 135°; Based on the four-channel polarization image, the zenith angle and azimuth angle images of the target are obtained respectively according to the illumination estimation model and Stokes parameters under the conditions of specular reflection and diffuse mixed illumination. Based on the original polarization image, the adaptive threshold processing method is used to extract and calibrate the edge information of the target contour to obtain a binary mask image of the target; the simulated polarization method is used to calculate the polarization information to obtain a non-light intensity polarization image of the target; According to the non-light intensity polarization image, binary mask image, zenith angle image and azimuth angle image, the target surface normal is determined by using the light source dot product principle with reference to the polar coordinate system of the target surface normal, and then the normal vector field image is obtained; According to the discrete form of the Poisson equation, the gradient information of the normal vector field image is converted into a large discrete sparse matrix. Based on the double conjugate gradient method to solve the Poisson equation, an adaptive fusion regularization formula is used to stably solve the Poisson equation and complete the depth image restoration.

2. The method for recovering marine target depth maps based on polarization images and regularization technology according to claim 1, characterized in that: The process of obtaining the zenith angle and azimuth angle images of the target based on the four-channel polarization image and the illumination estimation model and Stokes parameters under the conditions of specular reflection and diffuse mixed illumination is as follows: Based on the electromagnetic theory of light, the relationship between the amplitude and phase of the incident, reflected, and refracted light of the target is derived from Fresnel's law and Snell's law, thereby obtaining the zenith angle image under mixed illumination of specular reflection and diffuse reflection. Based on the four-channel polarization image information, the azimuth image information is obtained using the Stokes parameters.

3. The method for recovering marine target depth maps based on polarization images and regularization technology according to claim 1, characterized in that: According to the discrete form of the Poisson equation, the gradient information of the normal vector field image is converted into a large discrete sparse matrix. Based on the biconjugate gradient method to solve the Poisson equation, the adaptive fusion regularization formula is used to stably solve the Poisson equation to complete the process of depth image recovery as follows: S61: After obtaining the normal vector image, perform depth recovery on the normal vector image; S62: According to the discretized form of Poisson's equation, the gradient information of the normal vector field image is converted into a large discretized sparse matrix; S63: Discretize total variation regularization and derivative regularization under the Poisson equation; S64: According to the properties of the polarization image and the target global surface normal vector field information, two weight selection factors are given based on the consistency of the normal vector field; S65: Convert the unknown quantity in the two weight selection factors into an unknown parameter. S66: Based on the four regularization methods, the advantages of total variation regularization in preserving image edge information and the advantages of derivative regularization in suppressing noise are integrated to obtain a weighted formula; S67: To reduce the calculation error of the weighted formula, perform square addition and integration operations on the total variation regularization and derivative regularization to obtain an adaptive fusion regularization formula; S68: When solving the Poisson equation for target depth information, an adaptive fusion regularization formula is added to retain target edge information and stabilize the depth numerical solution, thereby completing depth image recovery.

4. The method for recovering marine target depth maps based on polarization images and regularization technology according to claim 3 is characterized in that: The process of converting the gradient information of the normal vector field image into a large discretized sparse matrix according to the discretized form of the Poisson equation is as follows: The discretized form of Poisson's equation is used for deep numerical solution. The specific form of Poisson's equation is as follows: (12) in: represents the Laplace operator, and is a real or complex-valued equation, for a depth function in a continuous domain , Represents the pixel index in the depth image; : represents the horizontal coordinate value of the pixel index in the depth image; : represents the vertical coordinate value of the pixel index in the depth image; Formula (12) can be written as follows: (13) in, Represents the horizontal gradient information in the normal vector field image , longitudinal gradient information The one-dimensional vector of , Represents the pixel index of the normal field image, : represents the horizontal coordinate value of the pixel index of the normal vector field image; : The vertical coordinate value of the pixel index representing the normal vector field image; According to the five-point difference method, the discrete equation is: (14) in: :Represents the first Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column, Represents the normal vector field image Rank The pixel index of the column; Then the discretized form of the normal vector field image information under the Poisson equation is , is the coefficient matrix discretized using the five-point difference format, A one-dimensional vector representing the target depth information. The specific discretization matrix is ​​as follows: (15) For each discretized equation, :Indicates that in the first discrete equation, the first unknown The previous coefficient; :Indicates that In a discrete equation, the first unknown The previous coefficients, Indicates that in the first discrete equation, unknown number The previous coefficients, Indicates in In the discrete equations, unknown number The previous coefficients, : Represents the pixel index in the normal vector field image The sum of the corresponding transverse gradient and longitudinal gradient data; Represents the pixel index in the normal vector field image as The corresponding depth value; Represents the pixel index in the normal vector field image as The sum of the corresponding transverse gradient and longitudinal gradient data.

5. The method for recovering marine target depth maps based on polarization images and regularization technology according to claim 3, characterized in that: The process of discretizing total variation regularization and derivative regularization under the Poisson equation is as follows: Discretization form of total variation regularization under Poisson equation for: (16) Derivative regularization discretizes the Poisson equation in the form The formula is: (17) in: Represents the normal vector field image Rank The pixel index of the column.

6. The method for recovering marine target depth maps based on polarization images and regularization technology according to claim 3, characterized in that: According to the properties of the polarization image and the target global surface normal vector field information, two weight selection factors are given based on the consistency of the normal vector field. The unknown quantities in the two weight selection factors are converted into an unknown parameter. According to four regularization methods, the advantages of total variation regularization in preserving image edge information and the advantages of derivative regularization in suppressing noise are integrated to obtain the weighted formula as follows: According to the properties of the polarization image and the target global surface normal vector field information, the first weight selection factor is given based on the consistency of the normal vector field. and the second weight selection factor : (18) (19) To obtain the target surface normal vector field information, is the rate of change of the normal vector, converting the two unknown variables into one unknown parameter , let it be: (20) According to the four regularization methods, the advantages of total variation regularization in preserving image edge information and the advantages of derivative regularization in suppressing noise are integrated, and the total variation regularization and derivative regularization methods are weighted and integrated. The specific weighted formula is for: (21)。 7. The method for recovering marine target depth maps based on polarization images and regularization technology according to claim 3, characterized in that: The specific formula of the adaptive fusion regularization formula is: (22) The discretized form of formula (22) under the Poisson equation is: (23)。 8. The method for recovering marine target depth maps based on polarization images and regularization technology according to claim 3, characterized in that: The formula for adding the adaptive fusion regularization formula to retain the target edge information and stabilize the depth numerical solution is as follows: (24) in: is the regularization coefficient, is the coefficient matrix after adding regularization; is the coefficient matrix discretized using the five-point difference format, Discretize the sum-of-squares and integral operations for total variation and derivative regularization under the Poisson equation.

Images