Three-dimensional shape reconstruction method, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to the field of machine vision, and discloses a three-dimensional morphology reconstruction method, electronic equipment and a storage medium, and the method comprises the steps: employing a polarization image collection system to shoot speckle images and infrared images of a target surface in a plurality of polarization states; obtaining a depth map for the speckle patterns in the plurality of polarization states by using a depth recovery algorithm, and converting the depth map into a normal graph; obtaining a polarization degree image and a polarization angle image according to the infrared images in each polarization state, calculating a zenith angle of the target surface based on the polarization degree image, calculating two ambiguous azimuth angles of the target surface based on the polarization angle image, further obtaining a corresponding polarization normal direction, and taking a normal image obtained after conversion of the depth image as normal prior; one polarization normal direction is selected from the two polarization normal directions with ambiguity to serve as the final polarization normal direction of the target surface, and a polarization normal vector field is formed; and reconstructing the three-dimensional morphology of the target surface based on the polarization normal vector field, thereby eliminating polarization angle ambiguity and ensuring that the reconstructed three-dimensional morphology is more accurate.

Description

Technical Field

[0001] The present invention relates to the field of machine vision, and in particular to a three-dimensional shape reconstruction method, electronic equipment and storage medium. Background Art

[0002] At present, the most important technical branch in the field of machine vision is 3D measurement technology. Commonly used optical 3D imaging technologies include time of flight (TOF), laser radar three-dimensional measurement, speckle structured light 3D imaging, and binocular 3D imaging. However, TOF is often affected by thermal power and has a low resolution; laser radar obtains sparse point clouds and is expensive; speckle structured light 3D imaging actively projects modulated scattered spots and uses the principle of triangulation to calculate the 3D shape of the target. Although it has many advantages over binocular 3D imaging, such as higher accuracy and no reliance on texture and ambient lighting, the accuracy of triangulation still decreases rapidly at a distance as the square of the distance decreases.

[0003] Compared with the above-mentioned optical measurement technologies that commonly use light intensity information, polarization 3D imaging can utilize another characteristic of light—polarization. Using polarization 3D imaging, the relationship between light intensity, degree of polarization, polarization angle, and three-dimensional surface morphology information of the target can be established by solving the polarization characteristic information in the target reflected light, thereby obtaining the target surface information from the polarization information. Polarization information changes slowly with distance, so the long-distance accuracy is better, and it can reflect the normal information of the target. It is very sensitive to the subtle fluctuations of the constantly changing normal information and can perceive small changes in depth. However, reconstructing the normal vector field from polarization information usually requires resolving the problem of polarization angle ambiguity. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a three-dimensional morphology reconstruction method, electronic device and storage medium, which can effectively eliminate polarization angle ambiguity in the process of reconstructing a normal vector field from polarization information, thereby improving the 3D measurement accuracy of the entire system, and thereby ensuring that the reconstructed three-dimensional morphology is more accurate.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a three-dimensional shape reconstruction method, comprising:

[0006] A polarization image acquisition system is used to capture speckle patterns and infrared images of a target surface under multiple polarization states, wherein the polarization image acquisition system includes a polarization camera and a speckle projector arranged side by side facing the target surface;

[0007] Using a depth recovery algorithm to obtain a depth map of the speckle patterns under the multiple polarization states, and converting the depth map into a normal map;

[0008] Based on the infrared images in each polarization state, a degree of polarization map and a polarization angle map are obtained. The zenith angle of the target surface is calculated based on the degree of polarization map, and two ambiguous azimuth angles of the target surface are calculated based on the polarization angle map;

[0009] Two ambiguous polarization normals are calculated based on the zenith angle and the two ambiguous azimuth angles of the target surface. Using the normal map converted from the depth map as a normal prior, one polarization normal is selected from the two ambiguous polarization normals as the final polarization normal of the target surface, forming a polarization normal vector field;

[0010] Based on the polarization normal vector field, the three-dimensional topography of the target surface is reconstructed.

[0011] An embodiment of the present invention also provides an electronic device, including:

[0012] At least one processor; and,

[0013] A memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the three-dimensional topography reconstruction method as described above.

[0015] An embodiment of the present invention also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the three-dimensional topography reconstruction method as described above is implemented.

[0016] In the embodiments of the present invention, compared with the prior art, speckle images and infrared images of a target surface are captured in multiple polarization states by using a polarization image acquisition system, and the polarization image acquisition system includes a polarization camera and a speckle projector arranged side by side facing the target surface; a depth map is obtained from the speckle images in multiple polarization states by using a depth recovery algorithm, and the depth map is converted into a normal map; a polarization degree map and a polarization angle map are obtained from the infrared images in each polarization state, the zenith angle of the target surface is calculated based on the polarization degree map, and two ambiguous azimuth angles of the target surface are calculated based on the polarization angle map; two ambiguous polarization normals are calculated based on the zenith angle and the two ambiguous azimuth angles of the target surface, and the normal map converted from the depth map is used as a normal prior to select one polarization normal from the two ambiguous polarization normals as the final polarization normal of the target surface, thereby forming a polarization normal vector field; the three-dimensional shape of the target surface is reconstructed based on the polarization normal vector field. In the process of reconstructing the normal vector field from polarization information, this solution combines polarization information with a speckle structured light system, and uses the normal map generated by converting the depth map as a prior for the polarization normal reconstructed based on polarization information, thereby eliminating the ambiguity of the polarization angle, improving the 3D measurement accuracy of the entire system, and further ensuring that the reconstructed three-dimensional shape is more accurate. Description of the Drawings

[0017] Figure 1 is a specific flowchart of a three-dimensional shape reconstruction method according to an embodiment of the present invention;

[0018] Figure 2 is a schematic structural diagram of a polarization image acquisition system according to an embodiment of the present invention;

[0019] Figure 3 is a schematic diagram of the polar coordinate representation of the zenith angle and the azimuth angle according to an embodiment of the present invention;

[0020] Figure 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. Detailed Embodiments

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that in the embodiments of the present invention, many technical details are provided to help readers better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions required to be protected by the present application can still be implemented.

[0022] An embodiment of the present invention relates to a three-dimensional shape reconstruction method, which can be used to reconstruct the 3D information of a target surface. The target surface can be the surface of a real target object such as a person or an object. For example Figure 1As shown in the figure, the three-dimensional topography reconstruction method provided in this embodiment includes the following steps.

[0023] Step 101: Use a polarization image acquisition system to capture speckle images and infrared images of the target surface under multiple polarization states. The polarization image acquisition system includes a polarization camera and a speckle projector arranged side by side facing the target surface.

[0024] As Figure 2 shown, the polarization image acquisition system provided in this embodiment includes a polarization camera and a speckle projector, which are arranged side by side facing the target surface. The polarization camera can capture images in several different polarization directions. For example, the split focal plane camera shown in the figure can obtain a four-way image. The polarization camera and the speckle projector use light of the same wavelength. For example, if the speckle projector projects common infrared light of 940 nm, the polarization camera should be equipped with a 940 nm filter. The working mode of the speckle projector is pulsed, that is, during the pulse emission, the polarization camera captures the scene object to form a speckle image, and during the non-emission period, it captures the scene object to form an infrared image. That is to say, the captured speckle images and infrared images both have images under multiple polarization states, and the polarization directions corresponding to each polarization state are different.

[0025] Step 102: Use a depth recovery algorithm to obtain a depth map from the speckle images under multiple polarization states, and convert the depth map into a normal map.

[0026] Specifically, a reference map for depth recovery can be preset in the polarization camera. After the polarization camera captures the speckle images under multiple polarization states, the reference map can be used to perform depth recovery on the speckle images to obtain a depth map, and then the normal vectors of each pixel point in the depth map are calculated to generate a normal map.

[0027] For example, the execution process of this step may include steps ① to ②.

[0028] Step ①: Stitch and add the obtained speckle images under each polarization state to obtain a total speckle image, and use the reference map built in the image system to perform depth recovery on the total speckle image to obtain the depth map.

[0029] Specifically, compared with the speckle images obtained by an ordinary monocular speckle structured light system, the images obtained by the polarization image acquisition system in this embodiment are speckle images of multiple polarization states. Only by adding the speckle images of multiple polarization states can a total speckle image be obtained, and then, like the speckle Figure 1 of the monocular speckle structured light, the depth recovery algorithm can be used to recover the depth map. The type of depth recovery algorithm used to recover the depth map in this embodiment is not limited.

[0030] Step ②: Convert the depth map into a point cloud map, and for each point cloud, determine the normal vector of the point cloud based on the normal vectors of all its adjacent facets, and form a normal map based on the normal vectors of each point cloud in the point cloud map.

[0031] Specifically, there are many algorithms for converting a depth map into a normal map, which are not limited in this embodiment. For example, first convert the depth map into a point cloud map in space; then for each point cloud, calculate the normal vectors of all the surface elements adjacent to the point cloud in the point cloud map; take the average of these normal vectors as the normal vector of the point cloud, so that the normal of each pixel point in the point cloud map can be obtained, forming a normal map. This normal map can be used as a normal prior.

[0032] Step 103: Obtain a degree of polarization map and a polarization angle map based on the infrared maps in each polarization state, and calculate the zenith angle of the target surface based on the degree of polarization map, and calculate two ambiguous azimuth angles of the target surface based on the polarization angle map.

[0033] Specifically, after obtaining the infrared maps in multiple polarization states by using a polarization camera, two parameters, namely the degree of polarization and the polarization angle in Malus' law, can be solved based on the sampling values of the light intensity and the polarization direction reflected in the infrared maps in each polarization state, so as to construct and form a degree of polarization map and a polarization angle map.

[0034] For example, according to the variation law of the light intensity I(φ) of a linear polarizer at different directions φ in Malus' law:

[0035] I(φ) = I / 2 * (1 + ρ * cos(2 * (ψ - φ))) …………………………………………(1)

[0036] Construct a Stokes vector by using the light intensity sampling values and the polarization direction sampling values in the infrared maps in each polarization state, and solve the degree of polarization map and the polarization angle map from the Stokes vector; where, I represents the total light intensity without a linear polarizer, ρ represents the degree of polarization, and ψ represents the polarization angle.

[0037] For example, in formula (1), there are 3 unknowns I, ρ, and ψ. Therefore, at least three polarization directions and the corresponding light intensities are required for solution. After obtaining the Stokes vector through the following formula group (2) for the commonly used four-direction polarization map, the degree of polarization ρ and the polarization angle ψ can be solved, so as to form a degree of polarization map and a polarization angle map. Among them, S1 is the unknown I.

[0038] S1 = I(0°) + I(90°) + I(45°) + I(-45°)

[0039] S2 = I(0°) - I(90°)

[0040] S3 = I(45°) - I(-45°)

[0041]

[0042]

[0043] After obtaining the degree of polarization map and the polarization angle map, the zenith angle of the target surface can be calculated based on the degree of polarization map, and two ambiguous azimuth angles of the target surface can be calculated based on the polarization angle map. For example, the solution can be carried out according to the corresponding spatial geometric relationship between these parameters.

[0044] Assume that the normal vector of the object surface is expressed as The light projected onto the xoy plane where the detector (such as the polarization camera in this application) is located is expressed as The angle with the x-axis is This angle is called the incident azimuth angle (abbreviation: "azimuth angle"). The reflected light after reflection from the object surface and the normal vector of the object surface The included angle is θ, and this included angle is called the incident zenith angle (abbreviation: "zenith angle"). As Figure 3 shown, the polar coordinate representation of the normal vector of the object surface, the zenith angle θ and the azimuth angle is given.

[0045] The reflected light formed by reflection from the target surface mainly includes specular reflection and diffuse reflection. Specular reflection is a single reflection occurring on the target surface, and diffuse reflection is mainly multiple refractions that occur after light enters the surface of the object's microstructure. The zenith angle under reflection and refraction and the degree of polarization, and the azimuth angle and the polarization angle have different mapping relationships.

[0046] The relationship between the zenith angle θ and the degree of polarization P s under specular reflection:

[0047]

[0048] The relationship between the zenith angle θ and the degree of polarization P t under diffuse reflection is a bijective function:

[0049]

[0050] Among them, the above-mentioned n is the refractive index, which is generally defaulted to 1.5.

[0051] The relationship between the azimuth angle and the polarization angle ψ under specular reflection: Or

[0052] The relationship between the azimuth angle and the polarization angle ψ under diffuse reflection: Or

[0053] In this embodiment, the mapping relationships between the zenith angle and the degree of polarization, and between the azimuth angle and the polarization angle under diffuse reflection are selected to calculate and obtain the zenith angle and the azimuth angle. Therefore, it is important to separate specular reflection from diffuse reflection in the reflected light. This embodiment uses, but is not limited to, the independent component analysis (ICA) method to extract the diffuse reflection component from all the reflected light, so that the mapping relationship model under diffuse reflection is used for solving both the zenith angle and the azimuth angle.

[0054] For example, in the process of solving the degree of polarization and the polarization angle based on the light intensity sampling value and the polarization direction sampling value in the infrared image mentioned above, the light intensity in the infrared image here is the diffuse reflection component extracted from the total light intensity of the infrared image, and the extraction method can be extracted by the ICA method.

[0055] Correspondingly, in the process of calculating the zenith angle of the target surface based on the degree-of-polarization map, the zenith angle θ and the degree of polarization P under diffuse reflection t The relationship function, that is, formula (4), is used to calculate the zenith angle.

[0056] Similarly, in the process of calculating the azimuth angle of the target surface based on the polarization angle map, the azimuth angle under diffuse reflection And the relationship with the polarization angle ψ: Or To obtain two ambiguous azimuth angles of the target surface.

[0057] Step 104: Calculate two ambiguous polarization normal vectors based on the zenith angle and the two ambiguous azimuth angles of the target surface, and use the normal vector map converted from the depth map as the normal prior, and select one polarization normal vector from the two ambiguous polarization normal vectors as the final polarization normal vector of the target surface to form a polarization normal vector field.

[0058] In step 103, two normal vectors can be obtained through the two ambiguous azimuth angles obtained. To distinguish from the normal vectors in the normal vector map obtained based on the depth map mentioned above, the normal vectors here are called polarization normal vectors. To eliminate the polarization angle ambiguity, this embodiment uses the normal vectors in the normal vector map converted from the depth map mentioned above as a prior to select one from the two ambiguous polarization normal vectors here, and determine one polarization normal vector as the final polarization normal vector of the target surface, and form a polarization normal vector field.

[0059] The specific process may include the following steps ① to ③.

[0060] Step ①: According to the polarization normal formula: Determine two ambiguous polarization normal vectors And Among them, Corresponding to the azimuth angle Corresponding azimuth angle

[0061] Specifically, after obtaining the zenith angle θ and azimuth angle of the target surface two polarization normal directions can be obtained based on the polarization normal formula and Corresponding azimuth angle Corresponding azimuth angle That is, each pixel point in the image of the target surface has two temporarily ambiguous polarization normal directions

[0062] Step ②: In the normal map after depth map conversion, calculate the sum of the cost values between the normal directions of each pixel point in the unit window centered on each pixel point and the corresponding polarization normal directions and the sum of the cost values between the normal directions of each pixel point in the unit window centered on each pixel point and the corresponding polarization normal directions and the sum of the cost values between the normal directions of each pixel point in the unit window centered on each pixel point and the corresponding polarization normal directions

[0063] For example, traverse the valid pixel points (non-hole points after depth denoising) of the depth map, select the unit window centered on this pixel point, such as the unit window size is 2*6 + 1 (take 6 pixel points on both sides along the horizontal direction or vertical direction of this pixel point, plus its own pixel point). Denote the normal direction in the normal map based on the depth map conversion as the prior normal direction Calculate the cost function of all pixel points (u, v) in the internal area Ω of this window

[0064]

[0065] Obtain the sum of cost values C 1 The sum of cost values C 2 and

[0066] where C 1 is the sum of the cost values between the normal directions of each pixel point in the normal map and the corresponding polarization normal directions and C 2 is the sum of the cost values between the normal directions of each pixel point in the normal map and the corresponding polarization normal directions and the sum of the cost values between the normal directions of each pixel point in the normal map and the corresponding polarization normal directions

[0067] Step ③: If the sum of the cost values corresponding to the polarization normal direction is smaller, record that the number of times the polarization normal direction of all pixel points in the current unit window takes is incremented by 1; if the sum of the cost values corresponding to the polarization normal direction is smaller, record that the number of times the polarization normal direction of all pixel points in the current unit window takes is incremented by 1; after traversing all pixel points, determine the polarization normal direction with a larger number of takes corresponding to each pixel point Or the polarization normal direction As the final polarization normal direction of the target surface corresponding to this pixel, a polarization normal vector field is formed.

[0068] For example, create a zero array M with the same resolution as the above normal map to store the results. If C 1 <C 2 , that is, the sum of the cost values corresponding to the polarization normal direction is smaller, then it is considered that the azimuth angle corresponding to this unit window should take ψ + π / 2. At this time, M adds 1 to all pixels in this window, indicating that the number of times the polarization normal direction of all pixel points in the current unit window takes is increased by 1; if C 2 <C 1 , that is, the sum of the cost values corresponding to the polarization normal direction is smaller, then it is considered that the azimuth angle corresponding to this unit window should take ψ - π / 2. At this time, M subtracts 1 from all pixels in this window, indicating that the number of times the polarization normal direction of all pixel points in the current unit window takes is increased by 1. After traversing all pixel points, finally, for the pixel points where the array M is greater than 0, the corresponding normal vector takes more times. Therefore, the normal vector can be used as the final polarization normal direction of the target surface corresponding to this pixel point; for the pixel points where the array M is less than 0, the corresponding normal vector takes more times. Therefore, the normal vector can be used as the final polarization normal direction of the target surface corresponding to this pixel point. Based on this, the polarization angle ambiguity is eliminated, the final polarization normal direction of the target surface is obtained, and a polarization normal vector field is formed.

[0069] Step 105: Reconstruct the three-dimensional topography of the target surface based on the polarization normal vector field.

[0070] After obtaining the polarization normal vector field, the three-dimensional topography of the target surface can be reconstructed from the polarization normal vector field using existing methods. The specific process of reconstructing the three-dimensional topography from the polarization normal vector field in this embodiment is not limited.

[0071] In one example, the reconstruction process of the polarization normal vector field can be completed through the following steps ① to ②.

[0072] Step ①: Based on the target surface height function expressed by the polarization normal vector field, obtain the gradient field of the target surface height function, and obtain the height value of the target surface through global or local integration to form the relative surface coordinates of the target surface.

[0073] Specifically, when reconstructing from the polarization normal vector field, assuming that the target surface is continuously integrable and the target surface height function is z(x, y), then the normal vector field of the object surface can also be expressed as:

[0074]

[0075] Among them, the normal vector The rectangular coordinate components of are expressed in polar coordinates as:

[0076]

[0077] In this way, after the zenith angle and azimuth angle are known, the gradient field (p, q) of z(x, y) is known, and then z(x, y) is obtained by global or local integration. The z(x, y), that is, the target surface height function value, can be regarded as the relative surface coordinates of the target surface.

[0078] Step ②: Optimize the translation amount and size scaling factor between the coordinates converted from the depth map to the point cloud and the relative surface coordinates, and translate and scale the relative surface coordinates with the obtained optimal translation amount and size scaling factor to obtain the three-dimensional topography of the target surface.

[0079] Specifically, under the condition that the gradient field ((p, q)) satisfies the integrable condition, for seeking the optimal surface z(x, y), it can be expressed as a process of solving an optimal solution of the following cost function, that is, to minimize the error between the partial derivatives of the given surface function z(x, y) at each point and the true gradient field (p, q). The cost function is expressed as follows:

[0080] d{(p,q),(z x ,z y )}=∫∫|z x -p| 2 +|z y -q| 2 dxdy………………………………………(8)

[0081] There are currently many mature mathematical methods for solving the optimal solution of the above cost function, such as the Frankot Chellappa algorithm, which is not restricted here.

[0082] The above reconstructed result only retains the relative shape information, and the absolute distance information needs to be fused using the depth map obtained by depth perception. The point cloud recording the relative shape, that is, the relative surface coordinates of the above target surface, is (x p ,y p ,z p ), and the point cloud coordinates converted from the depth map are (x d ,y d ,zd ), there are only translation and scale changes between them. Therefore, the translation amount and scale factor can be optimized based on the coordinates converted from the depth map to the point cloud and the relative surface coordinates, and the relative surface coordinates can be translated and scaled with the optimal translation amount and scale factor to obtain the three-dimensional topography of the target surface.

[0083] For example, the optimal translation amount and scale factor can be obtained by minimizing E in the following function:

[0084] E = ∑ A ||(x d , y d , z d ) - s(x p , y p , z p ) - D|| 2 ……………………………………(9)

[0085] where A is the coordinates of all point clouds, (x d , y d , z d ) are the coordinates converted from the depth map to the point cloud, (x p , y p , z p ) are the relative surface coordinates, S is the scale factor, and D is the translation amount.

[0086] After obtaining s and D by minimizing E through formula (9), high-precision 3D information can finally be obtained, that is, the three-dimensional topography of the reconstructed target surface is:

[0087] (x l , y l , z l ) = s * (x p , y p , z p ) + D,

[0088] where (x l , y l , z l ) are the coordinates of the three-dimensional topography of the reconstructed target surface.

[0089] Compared with the related art, in this embodiment, speckle images and infrared images of a target surface are captured by using a polarization image acquisition system. The polarization image acquisition system includes a polarization camera and a speckle projector arranged side by side facing the target surface; the depth map is obtained from the speckle images in multiple polarization states by using a depth recovery algorithm, and the depth map is converted into a normal map; the degree of polarization map and the polarization angle map are obtained from the infrared images in each polarization state, the zenith angle of the target surface is calculated based on the degree of polarization map, and two ambiguous azimuth angles of the target surface are calculated based on the polarization angle map; two ambiguous polarization normals are calculated based on the zenith angle and the two ambiguous azimuth angles of the target surface, and the normal map converted from the depth map is used as the normal prior to select one polarization normal from the two ambiguous polarization normals as the final polarization normal of the target surface, forming a polarization normal vector field; the three-dimensional topography of the target surface is reconstructed based on the polarization normal vector field. In the process of reconstructing the normal vector field from polarization information, this solution combines polarization information with a speckle structured light system, and uses the normal map generated by converting the depth map as the prior for the polarization normal reconstructed based on polarization information, thereby eliminating the polarization angle ambiguity, improving the 3D measurement accuracy of the entire system, and further ensuring that the reconstructed three-dimensional topography is more accurate.

[0090] Another embodiment of the present invention relates to an electronic device, such as Figure 4 shown, including at least one processor 302; and a memory 301 communicatively connected to the at least one processor 302; wherein, the memory 301 stores instructions executable by the at least one processor 302, and the instructions are executed by the at least one processor 302 to enable the at least one processor 302 to execute any of the above method embodiments.

[0091] Wherein, the memory 301 and the processor 302 are connected in a bus manner. The bus can include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors 302 and the memory 301 together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art, so they will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, and provides a unit for communicating with various other devices on the transmission medium. The data processed by the processor 302 is transmitted on the wireless medium through the antenna. Further, the antenna also receives data and transmits the data to the processor 302.

[0092] The processor 302 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. The memory 301 can be used to store the data used by the processor 302 when performing operations.

[0093] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, any of the above method embodiments is implemented.

[0094] That is, those skilled in the art can understand that all or part of the steps in implementing the above method embodiments can be completed by instructing relevant hardware through a program. The program is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0095] Those of ordinary skill in the art can understand that the above embodiments are specific embodiments for implementing the present invention, and in practical applications, various changes can be made in form and details without departing from the spirit and scope of the present invention.

Claims

1. A three-dimensional topography reconstruction method, characterized in that, it includes: using a polarization image acquisition system to capture speckle images and infrared images of the target surface under multiple polarization states, where the polarization image acquisition system includes a polarization camera and a speckle projector arranged side by side facing the target surface; obtaining a depth map from the speckle images under the multiple polarization states using a depth recovery algorithm, and converting the depth map into a normal map; obtaining a degree of polarization map and a polarization angle map based on the infrared images under each polarization state, calculating the zenith angle of the target surface based on the degree of polarization map, and calculating two ambiguous azimuth angles of the target surface based on the polarization angle map; calculating two ambiguous polarization normals based on the zenith angle and the two ambiguous azimuth angles of the target surface, and using the normal map converted from the depth map as a normal prior to select one polarization normal from the two ambiguous polarization normals as the final polarization normal of the target surface to form a polarization normal vector field; reconstructing the three-dimensional topography of the target surface based on the polarization normal vector field.

2. The method according to claim 1, characterized in that, obtaining a depth map from the speckle images under the multiple polarization states using a depth recovery algorithm, and converting the depth map into a normal map, including: stitching and adding the obtained speckle images under each polarization state to obtain a total speckle image, and performing depth recovery on the total speckle image using the reference image built in the image system to obtain the depth map; converting the depth map into a point cloud map, and for each point cloud, determining the normal vector of the point cloud based on the normal vectors of all its adjacent facets, and forming the normal map based on the normal vectors of the point clouds in the point cloud map.

3. The method according to claim 1, characterized in that, obtaining the degree of polarization map and the polarization angle map based on the infrared images under each polarization state, including: According to the light intensity I(φ) variation law of a linear polarizer at different directions φ in Malus' law: I(φ) = I / 2 * (1 + ρ * cos(2 * (ψ - φ))), constructing a Stokes vector using the light intensity sampling values and polarization direction sampling values in the infrared images under each polarization state, and solving the degree of polarization map and the polarization angle map from the Stokes vector; where I represents the total light intensity without the linear polarizer, ρ represents the degree of polarization, and ψ represents the polarization angle.

4. The method according to claim 3, characterized in that, the light intensity sampling value in the infrared image is the diffuse reflection component extracted from the total light intensity of the infrared image; calculating the zenith angle of the target surface based on the degree of polarization map, including: Through the relationship function between the zenith angle θ and the degree of polarization P under diffuse reflection t : calculating the zenith angle; where n is the refractive index.

5. The method according to claim 4, characterized in that, the light intensity sampling value in the infrared image is the diffuse reflection component extracted from the total light intensity of the infrared image; calculating the two ambiguous azimuth angles of the target surface based on the polarization angle map, including: The azimuth angle of the polarization angle map under diffuse reflection or As two ambiguous azimuth angles of the target surface.

6. The method according to claim 5, characterized in that, Calculating two ambiguous polarization normals according to the zenith angle of the target surface and two ambiguous azimuth angles, and using the normal map after the conversion of the depth map as a normal prior, selecting one of the two ambiguous polarization normals as the final polarization normal of the target surface to form a polarization normal vector field, including: According to the polarization normal formula: Determine two ambiguous polarization normals and wherein corresponding azimuth angle corresponding azimuth angle In the normal map after the depth map conversion, calculate, within a unit window centered on each pixel point, the sum of the cost values between the normal of each pixel point in the normal map and the corresponding polarization normal and the sum of the cost values between the corresponding polarization normal and; If the polarization normal direction has a relatively small sum of cost values, then record that the number of times the polarization normal direction of all pixel points within the current unit window takes is incremented by 1; if the polarization normal direction has a relatively small sum of cost values, then record that the number of times the polarization normal direction of all pixel points within the current unit window takes is incremented by 1; after traversing all pixel points, determine the polarization normal direction or the polarization normal direction with a relatively larger number of occurrences corresponding to each pixel point as the final polarization normal direction of the target surface corresponding to this pixel point, and form a polarization normal vector field.

7. The method according to claim 6, wherein, reconstructing the three-dimensional topography of the target surface based on the polarization normal vector field, including: Obtaining the gradient field of the target surface height function based on the target surface height function expressed by the polarization normal vector field, and obtaining the height value of the target surface through global or local integration to form the relative surface coordinates of the target surface; Optimizing the translation amount and the size scaling factor between the coordinates of the point cloud converted from the depth map and the relative surface coordinates, and translating and scaling the relative surface coordinates with the obtained optimal translation amount and size scaling factor to obtain the three-dimensional topography of the target surface.

8. The method according to claim 7, wherein, optimizing the translation amount and the size scaling factor between the coordinates of the point cloud converted from the depth map and the relative surface coordinates, including: Obtaining the optimal translation amount and the size scaling factor by minimizing E in the following function: E = ∑ A ||(x d , y d , z d ) - s(x p , y p , z p ) - D|| 2 Among them, A is the coordinates of all point clouds, (x d , y d , z d ) are the coordinates of the depth map converted into point clouds, (x p , y p , z p ) are the relative surface coordinates, S is the size scaling factor, and D is the translation amount.

9. An electronic device, wherein, including: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the three-dimensional topography reconstruction method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, wherein, the computer program, when executed by a processor, implements the three-dimensional topography reconstruction method according to any one of claims 1 to 8.

Images