Polarization three-dimensional reconstruction method for solving zenith angle based on mixed reflected light

Abstract

The invention relates to a polarization three-dimensional reconstruction method for solving zenith angle based on mixed reflected light. The method comprises the following steps: obtaining polarization images of a target in four directions of 0 degree, 45 degrees, 90 degrees and 135 degrees; respectively calculating the polarization degrees of the polarization images; obtaining a hybrid model of target surface reflected light, solving a zenith angle and an azimuth angle of the target, and obtaining a normal vector field of the target; based on the normal vector field of the target, the Frankot-Chellappa algorithm is used for integrating the normal vector field to obtain the three-dimensional depth map of the target, the method is suitable for outdoor natural light conditions, especially for three-dimensional reconstruction of offshore and long-distance targets, the limitation of a traditional method in a severe environment is overcome, and through efficient processing of the multi-angle polarization image, the depth map of the target is obtained. Dependence on a single light source and strict experiment conditions is reduced, and higher operation convenience and engineering practical value are achieved.

Description

Technical Field

[0001] The present invention belongs to the field of fully automated products and relates to a method for polarization three-dimensional reconstruction by solving the zenith angle based on mixed reflected light. Background Art

[0002] Traditional imaging records a three-dimensional scene as a two-dimensional image, losing the information of a depth dimension in this process. Traditional three-dimensional imaging includes structured light three-dimensional imaging, binocular stereovision, holographic three-dimensional imaging, lidar, and time-of-flight technology. Structured light three-dimensional imaging is sensitive to ambient light, has a complex hardware system, is complex and costly, and has a limited scope of application; binocular stereovision requires a large amount of image matching and calculation, and real-time processing is difficult. For low-texture areas or repetitive texture areas, parallax calculation is inaccurate, resulting in large reconstruction errors. The selection of the baseline distance affects depth accuracy, and both too large and too small baselines will affect the results; holographic three-dimensional imaging has a large amount of data, a complex optical system, requires high-precision optical components and a stable laser, and is complex and expensive. The resolution and sensitivity of the recording medium limit the quality of the hologram; lidar is costly, and high-performance lidar equipment is expensive and not suitable for low-cost applications. Affected by the weather, in harsh weather conditions such as fog, rain, and snow, the laser propagation is limited, the measurement accuracy decreases, and it is not suitable for particularly long-distance measurements; for time-of-flight technology, the ToF accuracy decreases significantly for long-distance measurements. In a complex environment, signal multipath reflection will interfere with the measurement results. High-frequency signals and complex electronic components are required, resulting in high power consumption.

[0003] Currently, most traditional three-dimensional reconstruction technologies based on polarization information are limited to imaging under indoor conditions, with strict controllable laboratory lighting conditions and a single darkroom scene. The light source is usually a point light source, the target object is of moderate size, has a uniform material, the distance between the target and the camera is relatively small, and there are no interference factors in the field of view. These conditions ensure the stability and availability of polarization information and lay a foundation for accurate three-dimensional reconstruction. Compared with the indoor experimental environment, the outdoor sea surface long-distance imaging environment is extremely complex. There are large-scale dynamic interferences in the outdoor sea surface background, including the movement of sea waves, light scattering and reflection, etc. The sea surface has high specular reflection characteristics, and the polarization information in the reflected light is often affected by sea wave fluctuations and lighting conditions, such as the sun angle and cloud changes, and local highlights or strong specular reflections may occur. In this way, the light reflected onto the detector is the mixed reflected light from the target, that is, the superposition of specular reflection and diffuse reflection light. At this time, it is no longer reliable to simply use specular reflection or diffuse reflection to calculate the zenith angle of the target reflected light, and many detailed information of the target will be lost. Summary of the Invention

[0004] To solve the above problems, the technical solution adopted by the present invention is: a method for polarization three-dimensional reconstruction for solving the zenith angle based on mixed reflected light, comprising the following steps:

[0005] Obtain the polarization images of the target in four directions of 0°, 45°, 90° and 135°;

[0006] Calculate the degree of polarization of the polarization images respectively;

[0007] Based on the mixed model of the reflected light on the target surface, solve the zenith angle and azimuth angle of the target to obtain the normal vector field of the target;

[0008] Based on the normal vector field of the target, use the Frankot-Chellappa algorithm to integrate the normal vector field to obtain the three-dimensional depth map of the target.

[0009] Furthermore: the method for obtaining the degree of polarization calculation formula is as follows:

[0010] The polarization of light is usually described by the Stokes vector method. The Stokes vector includes the intensity I, the difference Q between the horizontal and vertical polarization components, and the difference U parameter between the polarization components in the 45° and 135° directions, which is used to represent the intensity and polarization information of light. The Stokes vector can be obtained from the light intensity information of 3 or more polarization angles:

[0011]

[0012] Where: I 0 is the light intensity of the target at 0°, I 45 is the light intensity of the target at 45°, I 90 is the light intensity of the target at 90°, I 135 is the light intensity of the target at 135°;

[0013] According to the definition of the degree of polarization, in the total intensity of the light wave, the proportion of completely polarized light, so the degree of polarization is calculated by the following formula.

[0014]

[0015] Furthermore: the process of the mixed model of the reflected light on the target surface is as follows:

[0016] The light received by the detector is the mixed reflected light of specular reflection light and diffuse reflection light. S m represents the light intensity of the mixed light received by the detector, P m is the degree of polarization of the mixed light, S r represents the light intensity of the specular reflection light, P r represents the degree of polarization of the specular reflection light, S t represents the light intensity of the diffuse reflection light, P tDenote the degree of polarization of the diffuse reflected light, then there is:

[0017] S m P m = S r P r + S t P t

[0018] S m = S r + S t

[0019] Furthermore, the mixed model of the reflected light on the target surface is obtained as follows:

[0020]

[0021] Where: θ is the zenith angle, θ ∈ [0°, 90°], where: θ is the zenith angle, θ ∈ [0°, 90°], P t is the degree of polarization of the diffuse reflected light, P r is the degree of polarization of the specular reflected light, P t , P r The relationship with the zenith angle is shown as follows.

[0022]

[0023] Furthermore: The solution formula of the azimuth angle is as follows:

[0024]

[0025] Furthermore: The expression of the normal vector field of the target is as follows:

[0026]

[0027] Let p and q represent the gradient fields of the surface normal in the x and y directions respectively.

[0028] Furthermore: Based on the normal vector field of the target, using the Frankot-Chellappa algorithm to integrate the normal vector field, the formula for obtaining the three-dimensional depth map of the target is as follows:

[0029]

[0030] Combining the zenith angle θ and the azimuth angle to obtain the normal vector, and using the integration algorithm to calculate the surface elevation function Z(μ) of the target, where M and N represent the number of pixels, representing the imaging resolution of the target, F -1 and F respectively represent the inverse Fourier transform and the Fourier transform.

[0031] A method for polarization three-dimensional reconstruction based on solving the zenith angle from mixed reflected light, comprising:

[0032] An acquisition module: used to acquire polarization images of the target in four directions of 0°, 45°, 90°, and 135°;

[0033] A calculation module: used to calculate the degree of polarization of the polarization images respectively;

[0034] A solution module: used to solve the zenith angle and azimuth angle of the target based on the mixed model of the reflected light on the target surface, and obtain the normal vector field of the target;

[0035] A obtaining module: used to integrate the normal vector field based on the normal vector field of the target by using the Frankot-Chellappa algorithm to obtain the three-dimensional depth map of the target.

[0036] A readable storage medium stores program modules, and the program modules can be run in a processor to implement the method described in any one of the above.

[0037] A method for polarization three-dimensional reconstruction based on solving the zenith angle from mixed reflected light provided by the present invention. The present invention collects polarization images in four directions of 0°, 45°, 90°, and 135° through a polarization camera, and obtains the mixed model of the reflected light on the target surface through the calculation of the degree of polarization, so as to solve the zenith angle and azimuth angle, and finally obtain the normal vector field of the target. The Frankot-Chellappa algorithm is used to integrate the normal vector field to obtain the three-dimensional depth map of the target. This method can effectively handle the problem of the mixture of specular reflection and diffuse reflection under complex lighting conditions, and improve the accuracy and detail retention of three-dimensional reconstruction.

[0038] Combined with a polarization camera, multi-angle polarization image acquisition, and Fresnel's law and Snell's law, the zenith angle is solved by simultaneously considering the effects of specular reflection and diffuse reflection to achieve higher-precision three-dimensional shape reconstruction. Compared with existing methods, the present invention is particularly suitable for long-distance scenes under outdoor natural light conditions, such as three-dimensional reconstruction of targets such as ships and lighthouses at sea.

[0039] Has the following advantages:

[0040] Improvement of reconstruction accuracy: By introducing the mixed reflected light model, the present invention can better retain the detail information on the target surface in complex lighting environments such as at sea. Compared with traditional polarization three-dimensional reconstruction methods, the reconstruction results are more accurate, especially in high-brightness regions and surfaces with complex materials.

[0041] Wide applicability: The present invention is applicable to outdoor natural light conditions, especially for three-dimensional reconstruction of targets at sea and at a long distance, overcoming the limitations of traditional methods in harsh environments.

[0042] Improvement in computational efficiency: Through the efficient processing of multi-angle polarization images, the dependence on a single light source and strict experimental conditions is reduced, and it has higher operational convenience and engineering practical value. Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0044] Figure 1 is the flowchart of this method;

[0045] Figure 2 are the light intensities of the lighthouse at different angles. (a) is 0°, (b) is 45°, (c) is 90°, and (d) is 135°;

[0046] Figure 3 are the light intensities of the cylindrical building by the sea at different angles. (a) is 0°, (b) is 45°, (c) is 90°, and (d) is 135°. Detailed Embodiments

[0047] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The following will refer to the drawings and combine the embodiments to detail the present invention.

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. The following description of at least one exemplary embodiment is actually only illustrative and in no way restricts the present invention and its application or use. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0049] Figure 1 is the flowchart of this method;

[0050] A method for polarization three-dimensional reconstruction based on solving the zenith angle from mixed reflected light, comprising the following steps:

[0051] Obtain the polarization images of the target in four directions of 0°, 45°, 90°, and 135°;

[0052] Calculate the degrees of polarization of the polarization images respectively;

[0053] Based on the hybrid model of the reflected light from the target surface, solve the zenith angle and azimuth angle of the target to obtain the normal vector field of the target;

[0054] Based on the normal vector field of the target, use the Frankot-Chellappa algorithm to integrate the normal vector field to obtain the three-dimensional depth map of the target.

[0055] Furthermore: The method for obtaining the polarization degree calculation formula is as follows:

[0056] The polarization of light is usually described by the Stokes vector method. The Stokes vector includes parameters such as intensity I, the difference Q between the horizontal and vertical polarization components, and the difference U between the polarization components in the 45° and 135° directions, which are used to represent the intensity and polarization information of light. The Stokes vector can be obtained from the light intensity information at 3 or more polarization angles:

[0057]

[0058] Where: I 0 is the light intensity at 0° of the target, I 45 is the light intensity at 45° of the target, I 90 is the light intensity at 90° of the target, I 135 is the light intensity at 135° of the target;

[0059] According to the definition of polarization degree, in the total intensity of light waves, the proportion of completely polarized light is calculated by the following formula for polarization degree.

[0060]

[0061] Furthermore: The process of the hybrid model of the reflected light from the target surface is as follows:

[0062] The light received by the detector is the mixed reflected light of specular reflection light and diffuse reflection light. S m represents the light intensity of the mixed light received by the detector, P m is the polarization degree of the mixed light, S r represents the light intensity of the specular reflection light, P r represents the polarization degree of the specular reflection light, S t represents the light intensity of the diffuse reflection light, P t represents the polarization degree of the diffuse reflection light. Then there is:

[0063] S m P m = S r P r + S t P t

[0064] S m = Sr +S t

[0065] A hybrid model is derived based on Fresnel's law and Snell's law, which can jointly solve the zenith angle for specular reflection and diffuse reflection, and then obtain the reflected light on the target surface, as follows:

[0066]

[0067] Where: θ is the zenith angle, θ ∈ [0°, 90°], where: θ is the zenith angle, θ ∈ [0°, 90°], P t is the degree of polarization of diffuse reflected light, P r is the degree of polarization of specular reflected light, P t 、P r The relationship with the zenith angle is as follows:

[0068]

[0069] By adopting the hybrid reflected light model, the limitation of the difficulty in separating specular reflection and diffuse reflection in traditional methods is broken through; it is applicable to outdoor natural light conditions, especially complex marine environments, and solves the problem of low accuracy of traditional polarization three-dimensional reconstruction under strong light conditions; an efficient method for calculating the degree of polarization is proposed. By introducing multi-angle polarization imaging, the ambiguity in traditional methods is reduced, and the accuracy of zenith angle solution is improved.

[0070] Furthermore: The solution formula for the azimuth angle is as follows:

[0071]

[0072] A vector field is a collection of vectors at each point in space and can represent various physical fields, such as electric fields, magnetic fields, flow velocity fields, etc.

[0073] A gradient field is a special vector field obtained by taking the gradient from a scalar field, which describes the rate of change and direction of the scalar field in space.

[0074] What is mentioned and applied in the FC algorithm is the gradient field. In the process of polarization three-dimensional reconstruction, the surface normal vector field is the core intermediate variable in polarization three-dimensional reconstruction, and it is directly related to polarization information. The gradient field is the projection of the normal vector field, providing the rate of change of surface height and ultimately being used for surface reconstruction.

[0075] Furthermore: The expression of the normal vector field of the target is as follows:

[0076]

[0077] Let p and q represent the gradient fields of the surface normal in the x direction and y direction respectively.

[0078] The Frankot-Chellappa algorithm is a classic method for reconstructing the surface depth map from the gradient map, especially suitable for the depth reconstruction problem when dealing with the light intensity gradient information. This algorithm integrates the surface gradient to solve the height information of a surface, and is widely used in 3D reconstruction in the fields of computer vision and image processing.

[0079] Furthermore: Based on the normal vector field of the target, the Frankot-Chellappa algorithm is used to integrate the normal vector field, and the formula for obtaining the three-dimensional depth map of the target is as follows:

[0080]

[0081] The zenith angle θ and the azimuth angle to obtain the normal vector, and the integral algorithm is used to calculate the surface elevation function Z(μ) of the target, where M and N represent the number of pixels, representing the imaging resolution of the target, and F -1 and F respectively represent the inverse Fourier transform and the Fourier transform.

[0082] A method for polarization three-dimensional reconstruction based on solving the zenith angle of mixed reflected light, including:

[0083] An acquisition module: used to acquire the polarization images of the target in four directions of 0°, 45°, 90°, and 135°;

[0084] A calculation module: used to calculate the polarization degrees of the polarization images respectively;

[0085] A solution module: used to solve the zenith angle and the azimuth angle of the target based on the mixed model of the reflected light on the target surface, and obtain the normal vector field of the target;

[0086] A obtaining module: used to integrate the normal vector field based on the normal vector field of the target by using the Frankot-Chellappa algorithm to obtain the three-dimensional depth map of the target.

[0087] A readable storage medium stores program modules, and the program modules can be run in a processor to implement the method described in any one of the above.

[0088] This application collects images at four polarization angles of 0°, 45°, 90°, and 135° through a polarization camera, calculates the polarization degree using the Stokes parameter method, and solves the zenith angle and the azimuth angle through the mixed reflected light model to obtain the normal vector field of the target. The Frankot-Chellappa algorithm is used for integration to reconstruct the three-dimensional shape of the target.

[0089] The polarization three-dimensional imaging experiments of the present invention were carried out at the seaside and in the laboratory respectively. In the seaside experiment, the sun was used as the natural light source to image distant targets, verifying the applicability of the present invention under complex lighting conditions. Under laboratory conditions, an artificial light source was used to image close-range targets such as a bus model to verify the effectiveness of the method under controlled conditions.

[0090] Figure 2 are the light intensities of the lighthouse at different angles, (a) is 0°, (b) is 45°, (c) is 90°, and (d) is 135°;

[0091] Figure 3 are the light intensities of the cylindrical building at the seaside at different angles, (a) is 0°, (b) is 45°, (c) is 90°, and (d) is 135°.

[0092] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for polarization three-dimensional reconstruction of zenith angle based on mixed reflected light, characterized in that: The following steps are involved: Obtain polarization images of the target at four directions: 0°, 45°, 90° and 135°; Calculate the polarization degree of the polarization image respectively; Based on the hybrid model of the reflected light on the target surface, the zenith angle and azimuth of the target are solved to obtain the normal vector field of the target; Based on the normal vector field of the target, the Frankot-Chellappa algorithm is used to integrate the normal vector field to obtain the three-dimensional depth map of the target.

2. The method for polarization three-dimensional reconstruction based on mixed reflected light for solving zenith angle according to claim 1, characterized in that: The polarization degree calculation formula is derived as follows: The polarization of light is usually described by the Stokes vector method. The Stokes vector includes the intensity I, the difference Q between the horizontal and vertical polarization components, and the difference U between the polarization components in the 45° and 135° directions. It is used to represent the intensity and polarization information of light. The Stokes vector is obtained from the light intensity information of 3 or more polarization angles: Where: I0 is the light intensity at 0°, I 45 is the light intensity at 45° to the target, I 90 is the light intensity at 90° to the target, I 135 is the light intensity at 135° of the target; According to the definition of polarization degree, the proportion of completely polarized light in the total intensity of the light wave is calculated using the following formula.

3. The method for polarization three-dimensional reconstruction based on mixed reflected light for solving zenith angle according to claim 1, characterized in that: The process of the mixed model of the target surface reflected light is as follows: The light received by the detector is a mixture of specular reflection light and diffuse reflection light. m Indicates the intensity of the mixed light received by the detector, P m is the polarization degree of the mixed light, S r Indicates the intensity of specular reflected light, P r Indicates the polarization degree of specular reflected light, S t Represents the intensity of diffuse reflected light, P t represents the polarization degree of diffuse reflected light, then: S m P m =S r P r +S t P t S m =S r +S t Then the mixed model of the reflected light on the target surface is obtained, as follows: Where: θ is the zenith angle, θ∈[0°,90°], P t is the polarization degree of diffuse reflected light, P r is the polarization degree of the specularly reflected light, P t , P r The relationship with the zenith angle is shown below.

4. The method for polarization three-dimensional reconstruction based on mixed reflected light for solving zenith angle according to claim 1, characterized in that: The azimuth The solution formula is as follows:

5. The method for polarization three-dimensional reconstruction based on mixed reflected light for solving zenith angle according to claim 1, characterized in that: The expression of the normal vector field of the target is as follows: Let p and q denote the gradient field of the surface normal in the x-direction and y-direction respectively.

6. The method for polarization three-dimensional reconstruction based on mixed reflected light for solving zenith angle according to claim 1, characterized in that: Based on the normal vector field of the target, the Frankot-Chellappa algorithm is used to integrate the normal vector field to obtain the three-dimensional depth map of the target. The formula used is as follows: The zenith angle θ and the azimuth angle Get the normal vector and use the integral algorithm to calculate the surface elevation function Z(μ) of the target, where M and N represent the number of pixels, F represents the imaging resolution of the target, -1 and F denote inverse Fourier transform and Fourier transform, respectively.

7. A method for polarization three-dimensional reconstruction of zenith angle based on mixed reflected light, characterized in that: include: Acquisition module: used to acquire polarization images of the target at four directions: 0°, 45°, 90° and 135°; Calculation module: used to calculate the polarization degree of the polarization image respectively; Solution module: used to solve the zenith angle and azimuth angle of the target based on the hybrid model of the reflected light on the target surface, and obtain the normal vector field of the target; Obtaining module: It is used to integrate the normal vector field based on the target using the Frankot-Chellappa algorithm to obtain the three-dimensional depth map of the target.

8. A readable storage medium storing a program module, characterized in that: The program module is executed in a processor to implement the method according to any one of claims 1 to 6.

Images