A Three-Dimensional Imaging Method of Polarization-Domain Spatial Structured Light

Abstract

A three-dimensional imaging method of polarization-domain spatial structured light, comprising the following steps: Step S1: Build a three-dimensional imaging system of polarization-domain spatial structured light, perform image acquisition; and finally perform calibration to calculate the system parameters; Step S2: The three-dimensional imaging system of polarization-domain spatial structured light projects polarized light that conforms to the deflection coding of the polarized light encoder onto the surface of the object to be measured; after the camera system acquires the polarization images, align the images with different polarization states to obtain an image group; Step S3: Demodulate the phase of the image group obtained in Step S2 to obtain the information on the normal distribution of the object surface; Step S4: Perform depth map reconstruction and complete three-dimensional information reconstruction based on the normal information; The present invention uses the polarization direction as the carrier of structured light information, fuses the normal information and the structured light information for three-dimensional reconstruction, can perform long-distance imaging, and effectively improves the working range of the system.

Description

Technical Field

[0001] The present invention relates to the technical fields of optoelectronic measurement and three-dimensional vision, and particularly relates to a three-dimensional imaging method for polarized domain spatial structured light. Background Art

[0002] Structured light three-dimensional imaging technology is widely applied to fields such as three-dimensional modeling, robot navigation, and topography measurement. A classic structured light three-dimensional imaging system consists of a structured light projector and a camera with an optical system. The structured light projector projects a pre-designed structured light image or image sequence onto the surface of an object, and the camera with the optical system captures the structured light pattern on the object surface from another angle. Due to the difference in viewing angles between the two, the camera captures a deformed structured light pattern. The deformed structured light pattern is related to the height distribution of the object surface. By designing corresponding calibration and demodulation algorithms, the height distribution of the object surface can be calculated, and then three-dimensional information can be reconstructed. The "structure" of this structured light occurs in the spatial intensity distribution, that is, features are artificially created by modulating the brightness and darkness of the light field. The brightness and darkness of the light field are captured by the camera after being reflected by the object surface, so as to restore three-dimensional information using the collected images.

[0003] When the object surface has a diffuse reflection surface, the above method is very effective. However, when the object surface is not an ideal diffuse reflection surface: when the reflectivity of the object surface is very low, such as a black object; when the reflectivity is very high, such as a near-specular object; when the transmittance is very high and the reflectivity is very low for a transparent object; or when the surface is complex such as a black shiny object or a metal surface, it is very difficult to perform effective three-dimensional imaging on the object using the above method.

[0004] Different from the reflection property of light, in addition to having good adaptability to diffuse reflection surfaces, for non-diffuse reflection surfaces, a large amount of effective information can still be obtained from the polarization state of light. Therefore, using the polarization information of light for three-dimensional reconstruction is an effective means to solve the above problems. In the past, a method for three-dimensional topography recovery by analyzing graphic information in different polarization states, "Polarized 3d: High-quality depth sensing with polarization cues", has been proposed. This method calculates the normal information of the object surface and then integrates it to obtain the three-dimensional shape. However, it has problems such as lack of reference information and difficulty in unwrapping, and can only be used in special scenarios or in combination with other methods, with great limitations.

[0005] The emergence of optical metasurfaces has provided a means to manipulate the polarization state of light. A metasurface is a two-dimensional planar structure composed of artificial atoms with special electromagnetic properties arranged in a certain pattern, which can flexibly control the amplitude, phase, polarization, etc. of incident light. The metasurface not only breaks through the electromagnetic properties of traditional materials, but its two-dimensional planar structure also overcomes the problems such as the difficult processing of the three-dimensional structure of metamaterials, providing convenience for the integration and miniaturization of nano-optical devices. Therefore, using metasurfaces to control the polarization state of light and encoding the light field in the polarization domain will become a potential method to solve three-dimensional imaging of non-diffuse reflection surfaces. Summary of the Invention

[0006] In order to overcome the defect that the existing structured light technology has poor applicability to the surface characteristics of objects, the purpose of the present invention is to provide a three-dimensional imaging method of polarization-domain spatial structured light. By encoding the spatial distribution of the polarization state of the projected light, then collecting images under different polarization states, and further recovering three-dimensional information, three-dimensional imaging of high-reflectivity, low-reflectivity, transparent, and complex surfaces is realized, with the characteristics of high precision, flexible control, and wide working range.

[0007] In order to achieve the above purpose, the technical solution of the present invention is as follows:

[0008] A three-dimensional imaging method of polarization-domain spatial structured light, comprising the following steps:

[0009] Step S1: Build a three-dimensional imaging system of polarization-domain spatial structured light, use natural light containing polarized light in all directions as the light source for calibration, irradiate the light source to form images of different regions respectively, and finally perform calibration to calculate the system parameters;

[0010] Step S2: The three-dimensional imaging system of polarization-domain spatial structured light projects the polarized light that conforms to the deflection encoding of the polarized light projector onto the surface of the object to be measured; after the camera system collects the polarization images, the images of different polarization states are aligned to obtain an image group;

[0011] Step S3: Demodulate the phase of the image group obtained in Step S2 to obtain the information on the normal distribution of the object surface;

[0012] Step S4: Perform depth map reconstruction and complete three-dimensional information reconstruction based on the normal information.

[0013] In the three-dimensional imaging system of polarization-domain spatial structured light in Step S1, it is arranged above the object to be measured and at least includes a polarized light encoder projector and a camera system for collecting polarized light information;

[0014] The polarization light encoding projector includes a semiconductor laser as a light source. The linearly polarized light beam emitted by the semiconductor laser passes through a deflector and then is further focused by a collimating lens to obtain a collimated linearly polarized light beam. After passing through a polarization light encoder, a polarization domain encoded light beam with a spatial polarization distribution is obtained, and the polarization domain encoded light beam is finally irradiated on the object to be measured.

[0015] The deflector is an electro-optic or magneto-optic device that changes the polarization direction of the linearly polarized light by regulating the electric field or magnetic field.

[0016] The camera system is a conventional CCD or CMOS camera, or a multi-channel polarization camera with different polarization state filters in front of each pixel. After receiving the reflected light beam after the polarization domain encoded light beam is reflected by the object to be measured, the light beam reaches the imaging device through the filtering of a polarization beam splitter lens, and a polarization image carrying polarization information is obtained.

[0017] The polarization light encoder of the polarization light encoding projector is a metasurface flat optical system for polarizing and encoding incident light. The surface of the polarization light encoder is provided with micro-nano structures, which are manufactured by photolithography, electron beam processing or nanoimprinting. The micro-nano structures are composed of discrete nano-unit structure materials. Ge or Si is used as the unit structure material in the infrared band; TiO2 or GaN is selected as the unit structure material in the visible light band; HfO2 or AlN is used as the unit structure material in the ultraviolet band.

[0018] The semiconductor laser is selected from a multimode edge-emitting laser (EEL) or a vertical cavity surface-emitting laser array (VCSELs).

[0019] The polarization light encoding projector performs the following deflection encoding on the polarization state of the light:

[0020] (Equation 1-1)

[0021] Where represents the polarization argument phase of a point (x, y) in the x-y plane perpendicular to the projection optical axis in space. X represents the maximum range in the x direction of the area covered by the projection light field, and X is determined by the field of view angle and the z-direction distance from the projection system. N is the number of encoding periods.

[0022] The phase transformation of the incident light wave by the polarization light encoder needs to satisfy the following phase distribution:

[0023] (Equation 1-13)

[0024] The phase transformation of the incident light wave by the polarization beam splitter lens needs to satisfy the following phase distribution to focus the incident light:

[0025] (Equation 1-2)

[0026] Wherein, is the wavelength of the incident light, f is the focal length; x and y are the position coordinates of the beam cross-section.

[0027] The formation of images in different regions in the step S1 is specifically as follows:

[0028] Change the angle of the calibration plate, take multiple groups of pictures, image the reflected light with different polarization states to form images in different regions, and obtain a picture group , ,…, .

[0029] The calibration and calculation of the system parameters in the step S1 are specifically as follows:

[0030] Camera coordinate system and the projection device coordinate system The transformation relationship to the world coordinate system is expressed as follows:

[0031] (Equation 1-3)

[0032] Wherein, M c = [ R c , t c ] and M p = [ R p , t p ] , R and t are the rotation and translation matrices respectively; X c = [ x c , y c , z c ] T , X p = [ x p , y p , z p ] T , X w = [ x w , y w , z w ] T are the coordinates of the corresponding points in the world coordinate system; and there is:

[0033] { X i c [ u c , v c , 1 ] = A c X c X i p [ u p , v p , 1 ] = A p X p (Equation 1-4)

[0034] Wherein, X i c [ u c , v c , 1 ] are the pixel coordinates of the corresponding points in the camera coordinate system, X i p [ u p , v p , 1 ] are the pixel coordinates of the corresponding points in the projection device coordinate system, and are the transformation matrices in the pinhole model, and their form is:

[0035] A = [ α γ u 0 0 β v 0 0 0 1 ] (Equation 1-5)

[0036] Wherein, is the position of the principal point, and is the focal length along the u-axis and v-axis of the image plane, is a parameter describing the skewness of two image axes; thus, there is:

[0037] { X i c [ u c , v c , 1 ] = A c M c X w X i p [ u p , v p , 1 ] = A p M p X w (Equation 1-6)

[0038] The transformation matrix is obtained through calibration , .

[0039] The specific method for obtaining the pixel-aligned image group by clipping in step S2 is as follows:

[0040] A group of images is clipped according to the principal point position in the pinhole model transformation parameters and the resolution;

[0041] Among them, the image of the first area collected for the i-th time is:

[0042] I i 1 = I i [ u 0 1 − U 2 : u 0 1 + U 2 , v 0 1 − V 2 : v 0 1 + V 2 ] (Equation 1-7)

[0043] Here, U and V are the effective resolutions of the camera system 001 in the u and v directions.

[0044] The specific method for step S3 is as follows:

[0045] Step S3-1: Calculate the phase distribution of the polarization state of the light wave reflected from the object surface, that is, the angular space distribution, by collecting images; specifically, the phase shift is achieved by adjusting the electric field or magnetic field intensity of the deflector, and then the angular space distribution after the phase shift is collected; the calculation method is:

[0046] (Equation 1-8)

[0047] Among them, and are the components of the argument in two orthogonal directions, which are directly read out by the camera system;

[0048] Step S3-2: Calculate the phase distribution of the incident light polarization state;

[0049] Assume that the angular phase shift before the two incident polarized light encoders is , since is fixed and can be calibrated in advance. According to the Fresnel reflection law, we have:

[0050] (Equation 1-9)

[0051] Among them, n is the relative refractive index and i is the incident angle;

[0052] After phase shift,

[0053] (Equation 1-10)

[0054] Since , and are all known, the phase distribution of the polarization state of the light wave when it enters the surface of the incident object can be calculated ;

[0055] Step S3-3: Calculate the normal distribution on the object surface;

[0056] Through Equation (1-9) and Equation (1-10), the incident angle i is further calculated, and the normal vector on the reflecting surface is:

[0057] n ⇀ p = [ tan i cos θ 1 ,tan i sin θ 1 , 1 ] (Equation 1-11).

[0058] The said Step S4 includes the following specific steps:

[0059] Step S4-1: According to the calibration parameters, perform epipolar correction on the polarization argument phase of the projected image and the incident light argument phase map corresponding to the acquired image and perform interpolation and clipping processing to make the resolutions of the images the same; after epipolar correction, the phase matching points are row-aligned;

[0060] Step S4-2: When the polarization state of the light is deflected and encoded by the polarization light encoding projector, if the number of encoding cycles N≥2, for the polarization argument phase , the incident light argument phase map needs to be phase-unwrapped to obtain the absolute phase distribution;

[0061] Step S4-3: and are searched, interpolated, and matched row by row to obtain the disparity of the phase matching points, calculate the disparity map, and then use the calibration parameter matrix to reconstruct the depth map to obtain ;

[0062] Step S4-4: Use to calculate the normal information ;

[0063] Step S4-5: Use and , optimize to obtain the optimized depth map ;

[0064] (Equation 1-12);

[0065] Step S4-6: Use the camera coordinate system and the projection device coordinate system to solve for the three-dimensional information using the transformation relationship to the world coordinate system, obtaining X w = [ x w , y w , z w ] T , thus completing the three-dimensional information reconstruction.

[0066] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0067] 1. The present invention uses the polarization angle of light waves as the spatial coding information. In step 1, the design, implementation method, and calibration method of polarization structured light are given. In step 2, the acquisition and processing method of polarization domain structured light images are given. In step 3, the method for solving the phase of polarization domain structured light is given, and in step 4, the reconstruction method is given. Therefore, the present invention combines the ideas of three-dimensional imaging of spatial structured light and polarization three-dimensional imaging. Using the polarization direction as the carrier of structured light information, it can achieve three-dimensional imaging of high-reflectivity, low-reflectivity, transparent, and complex surfaces, with the characteristics of high precision, flexible regulation, and wide working range.

[0068] 2. In step 4, the present invention combines normal information and structured light information for three-dimensional reconstruction. This method is very sensitive to areas with large curvature changes on the object surface and can effectively retain local details, greatly improving the precision. For traditional intensity-based coding information, at long distances, the intensity of light decreases exponentially, resulting in poor long-distance imaging effects for traditional structured light. However, since polarization information is not particularly sensitive to the intensity of light, the imaging method of the present invention can perform long-distance imaging, effectively expanding the working range of the system.

[0069] 3. Since the present invention encodes in the polarization domain, different from traditional structured light encoding in the intensity domain, it improves the adaptability to materials. It is effective not only for diffuse reflection surfaces but also when the object surface has a non-ideal diffuse reflection surface: when the reflectivity of the object surface is very low, such as a black object; when the surface reflectivity is very high, such as a near-specular object; when the transmittance is very high and the reflectivity is very low for a transparent object; or for complex surfaces such as black shiny objects and metal surfaces. The above method can still effectively image, greatly enhancing the adaptability to materials and solving the problem that traditional methods are not applicable to special reflectivity materials.

[0070] In summary, the present invention encodes the spatial distribution of the polarization state of the projected light, then acquires images under different polarization states, and further performs the recovery of three-dimensional information, ultimately achieving three-dimensional imaging of high-reflectivity, low-reflectivity, transparent, and complex surfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 is a structural diagram of a three-dimensional imaging system.

[0072] Figure 2 It is a schematic structural diagram of a spatial structured light projector in the polarization domain.

[0073] Figure 3 It is a schematic structural diagram of polarization domain spatial structured light encoding.

[0074] Figure 4 It is a schematic structural diagram of a polarization domain structured light acquisition system.

[0075] In the figure: 001 camera system, 002 polarized light encoding projector, 003 object to be measured; 101 reflected light beam, 102 polarization beam splitter lens, 103 imaging device, 104 polarization image; 1021 nanostructure; 201 semiconductor laser, 202 polarization encoder; 203 polarization domain encoded light beam, 204 linearly polarized light beam, 205 collimating lens, 206 collimated linearly polarized light beam, 207 deflector, 2021 micro-nanostructure. Specific embodiments

[0076] The present invention will be described in detail below with reference to the accompanying drawings.

[0077] A three-dimensional imaging method for spatial structured light in the polarization domain includes the following steps:

[0078] Step S1: Build a three-dimensional imaging system for spatial structured light in the polarization domain and tighten it to avoid relative position changes of the system under external interference; use natural light containing polarized light in all directions as the calibration light source for all light sources, then irradiate the calibration plate with the light source for image acquisition; finally, perform calibration calculations on the system parameters;

[0079] Step S2: The three-dimensional imaging system for spatial structured light in the polarization domain projects polarized light that conforms to the deflection encoding of the polarized light encoding projector 002 onto the surface of the object to be measured 003; after the camera system 001 acquires the polarization image, the images in different regions represent image information of different polarization states, and the images are cropped to obtain a set of pixel-aligned images; after the camera acquires the polarization image from another angle, the images in different regions represent image information of different polarization states. As Figure 4 shown, it is preferable to divide the camera system 001 into 4 regions for imaging. The corresponding angular arguments are 0°, 45°, 90°, and 135° respectively; by adjusting the deflector, the polarization angle of the incident light is changed to achieve phase shift;

[0080] Step S3: Perform phase demodulation on the set of images to obtain the information on the normal distribution of the object surface;

[0081] Step S4: Perform depth map reconstruction and complete three-dimensional information reconstruction based on the normal information.

[0082] As Figure 1As shown, in the polarization domain spatial structured light three-dimensional imaging system in step S1, it is arranged above the object to be measured 003 and at least includes a polarization light encoding projector 002 and a camera system 001 for collecting polarization light information;

[0083] As Figure 2 shown, the polarization light encoding projector 002 includes a semiconductor laser 201 as a light source. After the linearly polarized light beam 204 emitted by the semiconductor laser 201 passes through a deflector 207, it is further focused by a collimating lens 205 to obtain a collimated linearly polarized light beam 206. After the collimated linearly polarized light beam 206 passes through a polarization light encoder 202, a polarization domain encoded light beam 203 with a specific distribution of spatial polarization states is obtained. As Figure 3 shown, the polarization domain encoded light beam 203 is finally irradiated on the object to be measured 003.

[0084] The deflector 207 is an electro-optical or magneto-optical device, and the polarization direction of the linearly polarized light can be changed by adjusting the electric field or magnetic field.

[0085] As Figure 4 shown, the camera system 001 is a multi-channel polarization camera that uses different polarization state filters in front of each pixel to simultaneously collect a group of images with different polarization states; or it is a conventional CCD / cmos camera. After receiving the reflected light beam 101 after the polarization domain encoded light beam 203 is reflected by the object to be measured 003, it passes through a polarization beam splitter lens 102. The surface of the polarization beam splitter lens 102 is provided with micro-nano structures 1021. After being filtered by the polarization beam splitter lens 102, the light beam reaches the imaging device 103 to obtain a polarization image 104 carrying polarization information; on the polarization image 104, the images in different regions represent the information of different polarization states, so as to obtain a group of images with different polarization states.

[0086] The polarization light encoder 202 of the polarization light encoding projector 002 is a metasurface flat optical system for encoding the incident linearly polarized light; the surface of the polarization light encoder 202 is provided with micro-nano structures 2021, which are manufactured by photolithography or electron beam processing or nanoimprinting. The micro-nano structures 2021 are composed of discrete nano-unit structure materials. Ge or Si is used as the unit structure material in the infrared band; TiO2 or GaN is selected as the unit structure material in the visible light band; HfO2 or AlN is used as the unit structure material in the ultraviolet band.

[0087] The semiconductor laser 201 is selected from a multimode edge-emitting laser (EEL) or a vertical cavity surface-emitting laser array (VCSELs).

[0088] The polarization light encoding projector 002 performs the following deflection encoding on the polarization state of the light:

[0089] (Equation 1-1)

[0090] Wherein represents the polarization angle phase of a point (x, y) in the x-y plane perpendicular to the optical axis of the projected light in space, X represents the maximum range in the x direction of the area covered by the projected light field, X is determined by the field of view angle and the z-direction distance from the projection system, and N is the number of coding periods. As Figure 4 shown, the larger the value of N, the finer the coding and the higher the accuracy, but the higher the difficulty of unwrapping. The preferred value range of N is [1, 3].

[0091] The phase transformation of the incident light wave by the polarized light encoding projector 002 needs to satisfy the following phase distribution:

[0092] (Equation 1-13)

[0093] The phase transformation of the incident light wave by the polarization beam splitter lens 102 needs to satisfy the following phase distribution to focus the incident light:

[0094] (Equation 1-2)

[0095] Wherein is the wavelength of the incident light, f is the focal length; x, y are the position coordinates of the beam cross-section.

[0096] The polarization beam splitter lens 102 is an optical lens with a linear polarizer. By rotating the angle of the polarizer, multiple images containing different polarization states of image information are obtained. As long as images with different polarization states can be obtained, it does not affect the implementation effect of the present invention.

[0097] In the step S1, forming images of different regions specifically includes:

[0098] Respectively imaging the reflected light of different polarization states to form images of different regions. At this time, it is regarded as multiple cameras, and each imaging region is treated as a camera. The angle of the calibration plate is changed, and multiple groups of pictures are taken to obtain a picture group , ,…, ;

[0099] Or by rotating the angle of the polarizer, multiple images containing different polarization states of image information are obtained. At this time, multiple pictures are taken at each angle for calibration.

[0100] In the step S1, calibrating and calculating the system parameters specifically includes:

[0101] Camera coordinate system and the transformation relationship between the projection device coordinate system to the world coordinate system is expressed as follows:

[0102] (Equation 1-3)

[0103] Wherein, M c = [ R c , t c ] and M p = [ R p , t p ] , R and t are the rotation and translation matrices respectively; X c = [ x c , y c , z c ] T , X p = [ x p , y p , z p ] T , X w = [ x w , y w , z w ] T are the coordinates of the corresponding points in the world coordinate system; and there is:

[0104] { X i c [ u c , v c , 1 ] = A c X c X i p [ u p , v p , 1 ] = A p X p (Equation 1-4)

[0105] Wherein, X i c [ u c , v c , 1 ] are the pixel coordinates of the corresponding points in the camera coordinate system, X i p [ u p , v p , 1 ] are the pixel coordinates of the corresponding points in the projection device coordinate system, and are the transformation matrices in the pinhole model, and their forms are:

[0106] A = [ α γ u 0 0 β v 0 0 0 1 ] (Equation 1-5)

[0107] Wherein, is the position of the principal point, and are the focal lengths along the u-axis and v-axis of the image plane, is the parameter describing the skewness of the two image axes;

[0108] Then:

[0109] { X i c [ u c , v c , 1 ] = A c M c X w X i p [ u p , v p , 1 ] = A p M p X w (Equation 1-6)

[0110] The transformation matrix , is obtained through calibration. The specific calibration method can refer to the mature solutions in the industry, such as "Zhang S, Huang P S. Novel method for structured light system calibration[J]. Optical Engineering, 2006, 45(8): 083601-083601-8."

[0111] The camera system 001 is a monocular system with only one camera. At this time, the optical axis of the camera system 001 is parallel to the polarization light encoding projector 002, or there is a certain angle between the optical axis of the camera system 001 and the polarization light encoding projector 002 within the plane formed by the optical center connection line and the optical axis, and the value range of the angle is [5°, 45°].

[0112] As an alternative, the camera system 001 includes two cameras and a projection device, forming a binocular system. At this time, the two cameras can be directly calibrated to obtain the internal and external parameters of the two cameras. Specifically, the Zhang Z. calibration method can be referred to (Zhang Z. A flexible new technique for camera calibration[J]. IEEETransactions on pattern analysis and machine intelligence, 2000, 22(11):1330-1334.).

[0113] As an alternative, an optical system is used to separately image the reflected light of different polarization states to form images of different regions. At this time, there are multiple sets of camera parameters .

[0114] The specific method for obtaining the pixel-aligned image group by clipping in step S2 is as follows:

[0115] Clip a set of images according to the principal point position in the pinhole model transformation parameters and the resolution;

[0116] Among them, the image of the first region collected for the i-th time is:

[0117] I i 1 = I i [ u 0 1 − U 2 : u 0 1 + U 2 , v 0 1 − V 2 : v 0 1 + V 2 ] (Equation 1-7)

[0118] Here, U and V are the effective resolutions of the camera system 001 in the u and v directions.

[0119] The specific method of step S3 is as follows:

[0120] Step S3-1: Calculate the phase distribution, i.e., the argument spatial distribution, of the polarization state of the reflected light wave on the object surface by collecting images; adjust the electric and magnetic field intensities of the deflector 207, so as to deflect the polarization angles of the incident linearly polarized light, and further realize the phase shift of the polarization domain encoding. Further, calculate the phase distribution, i.e., the argument spatial distribution, of the polarization state of the reflected light wave on the object surface after the phase shift by collecting images. The calculation method is:

[0121] (Equation 1-8)

[0122] Among them, and are the components of the argument in two orthogonal directions and are directly read out by the camera system 001;

[0123] Step S3-2: Calculate the phase distribution of the polarization state of the incident light;

[0124] Assume that the phase shift of the argument before the two incident polarization light encoders 202 is , since is fixed and can be calibrated in advance,

[0125] According to the Fresnel reflection law:

[0126] (Equation 1-9)

[0127] where n is the relative refractive index and i is the incident angle;

[0128] After the phase shift,

[0129] (Equation 1-10)

[0130] Since , and are all known, the phase distribution of the polarization state of the light wave when it enters the surface of the object can be calculated ;

[0131] Step S3-3: Calculate the normal distribution of the object surface;

[0132] Through Equation (1-9) and Equation (1-10), the incident angle i is further calculated, and the normal vector of the reflection surface is:

[0133] n ⇀ p = [ tan i cos θ 1 ,tan i sin θ 1 , 1 ] (Equation 1-11)

[0134] The said step S4 includes the following specific steps:

[0135] Step S4-1: According to the calibration parameters, perform epipolar correction on the polarization argument phase of the projected image and the incident light argument phase diagram corresponding to the acquired image, and perform interpolation and clipping processing to make the resolutions of the images the same; after epipolar correction, the phase matching points are row-aligned;

[0136] Step S4-2: When the polarization state of the light is deflected and encoded by the polarization light encoding projector 002, as shown in Equation (1-1), if the number of encoding cycles N≥2, for the polarization argument phase , the incident light argument phase diagram Phase unwrapping is required to obtain the absolute phase distribution; the larger the N, the higher the accuracy of obtaining the absolute phase distribution, but the more likely it is to cause wrapping ambiguity and it is difficult to obtain the absolute phase;

[0137] Step S4-3: and Search, interpolate, and match row by row to obtain the disparity of the phase matching points, calculate the disparity map, and then use the calibration parameter matrix to reconstruct the depth map to obtain ;

[0138] Step S4-4: Use to calculate the normal information ;

[0139] Step S4-5: Use and , optimize to obtain the optimized depth map ;

[0140] (Equation 1-12);

[0141] Step S4-6: Use the transformation relationship between the camera coordinate system and the projection device coordinate system to the world coordinate system, as shown in (Equation 1-3), and solve for the three-dimensional information to obtain X w = [ x w , y w , z w ] T , that is, the three-dimensional information reconstruction is completed.

[0142] The present invention integrates the ideas of spatial structured light three-dimensional imaging and polarization three-dimensional imaging, uses the polarization direction as the carrier of structured light information, and integrates normal information and structured light information for three-dimensional reconstruction, enabling long-distance imaging and effectively expanding the working range of the system.

Claims

1. A polarization domain spatial structured light three-dimensional imaging method, characterized in that: The following steps are involved: Step S1: construct a polarization domain spatial structured light three-dimensional imaging system, use natural light containing polarized light in various directions as the light source for calibration, use the light source to illuminate and image to form images of different areas, and finally calibrate and calculate system parameters; Step S2: The polarization domain spatial structured light three-dimensional imaging system projects polarized light that complies with the deflection coding of the polarization light coding projector (002) onto the surface of the object to be measured (003); after the camera system (001) collects the polarization image, the images of different polarization states are aligned to obtain an image group; Step S3: performing phase demodulation on the image group obtained in step S2 to obtain the surface normal distribution information of the object; Step S4: reconstructing the depth map and completing the three-dimensional information reconstruction based on the normal information; The polarization domain spatial structured light three-dimensional imaging system in step S1 is arranged above the object to be measured (003), and comprises at least one polarization light encoding projector (002) and a camera system (001) for collecting polarization light information; The polarization light encoding projector (002) includes a semiconductor laser (201) as a light source. The linear polarized light beam (204) emitted by the semiconductor laser (201) passes through a deflector (207) and is further focused by a collimating lens (205) to obtain a collimated linear polarized light beam (206). The collimated linear polarized light beam (206) passes through a polarization light encoder (202) to obtain a polarization domain encoding light beam (203) with spatial polarization distribution. The polarization domain encoding light beam (203) is finally irradiated on the object to be measured (003). The step S4 comprises the following specific steps: Step S4-1: According to the calibration parameters, the polarization angle phase θ of the projected image is calibrated. p (x, y) and the incident light angular phase map θ1(u, v) corresponding to the collected image are subjected to epipolar correction and interpolation and cropping processing to make the image resolution the same; after epipolar correction, the phase matching points are line-aligned; Step S4-2: When the polarization light coding projector (002) deflects and codes the polarization state of the light, if the number of coding cycles N ≥ 2, the polarization angle phase θ p (x, y), the incident light angular phase diagram θ1(u, v) needs to be phase-unwrapped to obtain the absolute phase distribution; Step S4-3: θ p (x, y) and θ1(u, v) are searched, interpolated, and matched row by row to obtain the disparity of the phase matching point, calculate the disparity map, and then use the calibration parameter matrix to reconstruct the depth map to obtain D(u, v); Step S4-4: Use D(u,v) to calculate the normal information Step S4-5: Use and Optimize D(u,v) to obtain the optimized depth map D′(u,v); Represents the normal vector of the reflective surface; Step S4-6: Use the camera coordinate system X c and the projection device coordinate system X p The transformation relationship from the world coordinate system is used to solve the three-dimensional information and obtain X w =[x w ,y w ,z w ] T , that is, the three-dimensional information reconstruction is completed.

2. The polarization domain spatial structured light three-dimensional imaging method according to claim 1, characterized in that: The deflector (207) is an electro-optical or magneto-optical device, which changes the polarization direction of linearly polarized light by regulating the electric field or magnetic field; The camera system (001) is a conventional CCD, CMOS camera, or a multi-channel polarization camera using different polarization state filters in front of each pixel. After receiving the reflected light beam (101) after the polarization domain coded light beam (203) is reflected by the measured object (003), the light beam is filtered by the polarization splitting lens (102) and reaches the imaging device (103), thereby obtaining a polarization image (104) carrying polarization information; The polarization light encoder (202) of the polarization light encoding projector (002) is a metasurface flat optical system, which is used to perform polarization encoding on incident light; a micro-nano structure (2021) is arranged on the surface of the polarization light encoder (202), which is manufactured by photolithography, electron beam processing or nano-imprinting, and the micro-nano structure (2021) is composed of discrete nano unit structure materials, and Ge or Si is used as the unit structure material in the infrared band; Select TiO2 or GaN as the unit structure material in the visible light band; In the ultraviolet band, HfO2 or AlN is selected as the unit structure material; The semiconductor laser (201) is selected from a multi-mode edge emitting laser (EEL) or a vertical cavity surface emitting laser array (VCSELs).

3. The polarization domain spatial structured light three-dimensional imaging method according to claim 2, characterized in that: The phase transformation of the incident light wave by the polarization encoder (002) must satisfy the following phase distribution: The phase transformation of the incident light wave of the polarization beam splitting lens (102) needs to satisfy the following phase distribution, so as to focus the incident light: Among them, λ is the wavelength of the incident light, f is the focal length; x, y are the position coordinates of the beam cross section.

4. The polarization domain spatial structured light three-dimensional imaging method according to claim 1, characterized in that: The polarized light coding projector (002) performs the following deflection coding on the polarization state of the light: where θ p (x,y) represents the polarization angular phase of a point (x,y) in the xy plane perpendicular to the projection optical axis in space. X represents the maximum range of the area covered by the projection light field in the x direction. X is determined by the field of view angle and the z-direction distance from the projection system. N is the number of encoding cycles.

5. The polarization domain spatial structured light three-dimensional imaging method according to claim 1, characterized in that: The images of different regions are formed in step S1, specifically: By changing the angle of the calibration plate, taking multiple sets of pictures, the reflected light of different polarization states is imaged to form images of different areas, and the picture sets are obtained.

6. The polarization domain spatial structured light three-dimensional imaging method according to claim 1, characterized in that: The calibration calculation system parameters in step S1 are specifically as follows: Camera coordinate system X c and the projection device coordinate system X p The transformation relationship to the world coordinate system is expressed as follows: Among them, M c =[R c ,t c ] and M p =[R p ,t p ], R and t are rotation and translation matrices respectively; X c =[x c ,y c ,z c ] T , X p =[x p ,y p ,z p ] T , X w =[x w ,y w ,z w ] T are the coordinates of the corresponding points in the world coordinate system; and: in, is the pixel coordinate of the corresponding point in the camera coordinate system, is the pixel coordinate of the corresponding point in the projection device coordinate system, A c and A p is the transformation matrix under the pinhole model, and its form is: Where (u0, v0) is the position of the principal point, α and β are the focal lengths along the u-axis and v-axis of the image plane, and γ is a parameter describing the skewness of the two image axes; thus: The transformation matrix A is obtained by calibration c M c , A p M p .

7. The polarization domain spatial structured light three-dimensional imaging method according to claim 1, characterized in that: The specific method of cutting out the pixel-aligned image group in step S2 is as follows: A set of images The principal point position in the pinhole model transformation parameters and resolution for cropping; Among them, the image of the first area collected for the i-th time is: Here, U, V is the effective resolution of the camera system (001) in the u and v directions.

8. The polarization domain spatial structured light three-dimensional imaging method according to claim 1, characterized in that: The specific method of step S3 is as follows: Step S3-1: Calculate the phase distribution of the polarization state of the light wave reflected from the surface of the object, i.e., the spatial distribution of the angular polarization, by collecting images; specifically, the phase shift is achieved by adjusting the electric field or magnetic field intensity of the deflector (207), and further collect images to calculate the spatial distribution of the angular polarization after the phase shift; the calculation method is: Among them, θ p and θ s are the components of the argument in two orthogonal directions, which are directly read out by the camera system (001); Step S3-2: Calculate the phase distribution of the polarization state of the incident light; Assuming that the angular phase shift before the two incident polarization light encoders (202) is α, since α is fixed and can be calibrated in advance, according to the Fresnel reflection law, we get: Where n is the relative refractive index and i is the incident angle; After phase shift, Since θ2, θ2′ and α are all known, the phase distribution θ1 of the polarization state of the light wave on the surface of the incident object can be calculated; Step S3-3: Calculate the surface normal distribution of the object; Through equations (1-9) and (1-10), the incident angle i is further calculated, and the normal vector of the reflecting surface is:

Images