Three-dimensional imaging method based on structured light and structured light imaging equipment, and storage medium

Abstract

The invention discloses a three-dimensional imaging method based on structured light and structured light imaging equipment and a storage medium, and the three-dimensional imaging method comprises the steps: obtaining a first polarization structured light image and a second polarization structured light image of an imaged object, which are shot when two kinds of polarized light with different polarization directions and the same incident angle are subjected to structured light projection, performing three-dimensional reconstruction by using the first height image and the second height image respectively to obtain a first height image and a second height image; calculating a reflectivity difference chart according to the first polarization structure light image and the second polarization structure light image; and judging whether multiple reflections exist on the surface of the imaged object corresponding to each pixel position according to the reflectivity difference chart, if so, setting the height value of the pixel position as an invalid pixel value, and if not, determining the final height value of the pixel position according to the height values of the first height image and the second height image at the pixel position. Therefore, the influence of multiple reflection on the surface of the object on the reconstruction precision and integrity is weakened.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a three-dimensional imaging method and a storage medium based on structured light and a structured light imaging device. Background Art

[0002] With the increasing application of 3D detection technology in the field of industrial automation, the use of 3D reconstruction technology based on structured light 3D cameras, which is based on the principle of triangulation, is becoming more and more popular. Structured light 3D reconstruction systems often use a projector combined with a camera, with the projector and the camera's main optical axis at a certain angle. The projector projects structured light on the object, and the camera captures the structured light image of the object, which is then reconstructed in 3D to obtain the height image of the object in order to reconstruct the surface morphology of the object.

[0003] In the structured light 3D reconstruction system, the imaging unit performs 3D reconstruction by collecting diffuse reflection images of the object surface. 3D reconstruction uses the geometric constraints of reflected light. Its essence is that the illuminating light and the diffuse reflection light form a certain angle, forming a triangular constraint. When the surface morphology of an object is complex, the illumination light is incident on the surface of the object and causes multiple reflections after passing through the complex surface. The image collected by the imaging unit is not the diffuse reflection image of the actual object surface, but a diffuse reflection image containing multiple surface information. Even if the reconstruction can be solved, the obtained morphological information cannot represent the real surface information of the object. Figure 1 As shown, the incident light and the normal diffuse reflection light form a triangular constraint, but the geometric relationship between the multiple reflection light and the incident light is uncertain and does not satisfy the triangular constraint. At this time, the geometric constraint of 3D reconstruction does not hold, resulting in incomplete and unrealistic reconstructed data. Summary of the invention

[0004] The present invention provides a three-dimensional imaging method based on structured light, a three-dimensional imaging method based on structured light imaging equipment and a storage medium, which mainly solve the technical problem of multiple reflections of light caused by complex surfaces in three-dimensional reconstruction.

[0005] According to the first aspect, an embodiment provides a three-dimensional imaging method based on structured light, including:

[0006] Acquire a first polarized structured light image and a second polarized structured light image of an imaged object; wherein the first polarized structured light image is an image of the imaged object taken when structured light projection is performed on the imaged object using a first polarized light, and the second polarized structured light image is an image of the imaged object taken when structured light projection is performed on the imaged object using a second polarized light, the first polarized light and the second polarized light have different polarization directions, and the first polarized light and the second polarized light are incident on the surface of the imaged object at the same incident angle;

[0007] Performing three-dimensional reconstruction using the first polarized structured light image to obtain a first height image of the imaged object; performing three-dimensional reconstruction using the second polarized structured light image to obtain a second height image of the imaged object;

[0008] A reflectivity difference map is calculated based on the first polarized structured light image and the second polarized structured light image, wherein the pixel value of each pixel position in the reflectivity difference map represents the reflectivity difference degree of the pixel position, and the reflectivity difference degree represents the difference degree of the reflectivity of the surface of the imaged object corresponding to the pixel position to the first polarized light and the reflectivity of the second polarized light;

[0009] According to the reflectivity difference map, it is determined whether there are multiple reflections on the surface of the imaged object corresponding to each pixel position. If so, the height value of the pixel position is set to an invalid pixel value. If not, the final height value of the pixel position is determined based on the height values ​​of the first height image and the second height image at the pixel position to obtain the final height image of the imaged object.

[0010] According to the second aspect, an embodiment provides a three-dimensional imaging method based on a structured light imaging device, wherein the structured light imaging device includes an imaging device and two projection lighting devices, and the two projection lighting devices are symmetrically arranged with the imaging device as the center, and the projection lighting device can perform structured light projection with a first polarized light and a second polarized light, and the polarization directions of the first polarized light and the second polarized light are different, and the three-dimensional imaging method includes:

[0011] Controlling one of the two projection lighting devices to perform structured light projection on the imaged object with the first polarized light, triggering the imaging device to photograph the imaged object, and obtaining a first polarized structured light image; controlling the projection lighting device to perform structured light projection on the imaged object with the second polarized light, triggering the imaging device to photograph the imaged object, and obtaining a second polarized structured light image;

[0012] According to the three-dimensional imaging method described in the first aspect above, three-dimensional imaging is performed using the first polarized structured light image and the second polarized structured light image to obtain a first projection direction height image;

[0013] Controlling the other of the two projection lighting devices to perform structured light projection on the imaged object with the first polarized light, triggering the imaging device to photograph the imaged object, and obtaining a third polarized structured light image; controlling the projection lighting device to perform structured light projection on the imaged object with the second polarized light, triggering the imaging device to photograph the imaged object, and obtaining a fourth polarized structured light image;

[0014] According to the three-dimensional imaging method described in the first aspect above, three-dimensional imaging is performed using the third polarized structured light image and the fourth polarized structured light image to obtain a second projection direction height image;

[0015] The first projection direction height image and the second projection direction height image are fused to obtain a height image of the imaged object.

[0016] According to the third aspect, an embodiment provides a computer-readable storage medium, on which a program is stored, and the program can be executed by a processor to implement the three-dimensional imaging method described in the first aspect or the second aspect.

[0017] According to the three-dimensional imaging method of the above embodiment, the principle that polarized light with different polarization directions has different reflectivity (i.e., different reflection intensity) on the surface of an object when incident at the same angle is utilized. Since the reflection intensity after multiple surface reflections is different, whether multiple reflections occur can be determined by the degree of difference in reflectivity. In the three-dimensional imaging method of the present application, firstly, a first polarized structured light image and a second polarized structured light image are obtained when structured light projection is performed with two polarized lights with different polarization directions and the same incident angle, and they are respectively used for three-dimensional reconstruction to obtain a first height image and a second height image, and then a reflectivity difference map is calculated based on the first polarized structured light image and the second polarized structured light image, and whether the surface of the imaged object corresponding to each pixel position has multiple reflections is determined based on the reflectivity difference map, and if so, the height value of the pixel position is set to an invalid pixel value, and if not, the final height value of the pixel position is determined based on the height values ​​of the first height image and the second height image at the pixel position, thereby reducing the influence of multiple reflections on the object surface on the reconstruction accuracy and integrity, improving the integrity and authenticity of the three-dimensional reconstruction, and effectively improving the material adaptability and scene adaptability of the three-dimensional imaging system and method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of multiple reflections of light on the surface of an object;

[0019] Figure 2 A flowchart of a three-dimensional imaging method based on structured light according to an embodiment;

[0020] Figure 3 This is a comparison chart of the effects of the three-dimensional imaging method of the present application and the traditional reconstruction method;

[0021] Figure 4 A schematic diagram of the structure of a structured light imaging device according to an embodiment;

[0022] Figure 5A schematic diagram of the polarization directions of the first polarizer and the second polarizer and the analyzing direction of the analyzer in an embodiment;

[0023] Figure 6 A schematic diagram of the structure of a structured light imaging device according to another embodiment;

[0024] Figure 7 Schematic diagram of the relationship between the light transmittance of the polarizer and the incident angle of the light;

[0025] Figure 8 A schematic diagram of the geometric relationship between the projection image plane of the spatial light modulator, the principal plane of the projection lens, and the projection object plane;

[0026] Fig. 9 Schematic diagram of the optical path of S-polarized light and P-polarized light after passing through the light combining device;

[0027] Fig.10 A schematic diagram of the process of using S-polarized light and P-polarized light to perform three-dimensional reconstruction with multiple reflection suppression;

[0028] Fig.11 A schematic diagram of the structure of a structured light imaging device according to another embodiment;

[0029] Fig.12 A schematic diagram of the structure of a structured light imaging device according to another embodiment;

[0030] Fig.13 is a structural schematic diagram of a projection control module according to an embodiment;

[0031] Fig.14 Schematic diagram of the blind area of ​​the three-dimensional reconstruction;

[0032] Fig.15 The present invention is a flowchart of a three-dimensional imaging method based on a structured light imaging device according to an embodiment.

[0033] Reference numerals:

[0034] 100, projection lighting device; 101, first light source; 102, second light source; 103, first polarizer; 104, second polarizer; 105, light combining device; 106, spatial light modulator; 107, projection lens group; 108, first collimating lens group; 109, second collimating lens group; 110, reflector; 111, light homogenizing device; 112, first relay lens; 113, second relay lens; 114, compound prism; 200, imaging device; 201, analyzer; 202, image sensor; 203, imaging lens group; 300, control device; 400, imaged object. DETAILED DESCRIPTION

[0035] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0036] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0037] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0038] Currently, the 3D reconstruction method based on structured light still has the following problems when dealing with complex surfaces (high reflectivity, multiple reflections, etc.):

[0039] 1. Structured light 3D camera is a diffuse reflection imaging system. However, the surface materials of some objects will cause specular reflection on the surface of the object, which has high reflective properties. As a result, the images collected by the camera are overexposed or too dark, making it difficult to solve and reconstruct the actual surface morphology of the object.

[0040] 2. When the surface of an object has a complex morphology, and the illumination light is incident on the surface of the object and forms multiple reflections through the complex surface, the image collected by the camera is not a diffuse reflection image of the actual object surface, but a diffuse reflection image containing multiple surface information. Even if the reconstruction can be solved, the obtained morphological information cannot represent the real surface information of the object;

[0041] 3. Due to the angle, the structured light 3D camera will have two blind spots: first, the area that the projector can illuminate but the camera cannot capture; second, the area that the camera can capture but the projector cannot illuminate; these blind spots will cause incomplete reconstruction data.

[0042] In response to the difficulties of the above-mentioned structured light-based 3D reconstruction methods, existing methods are often optimized by adopting high-cost cameras with low practicality or multiple exposure methods that are not real-time, in order to improve the reconstruction integrity of structured light 3D reconstruction.

[0043] Patent application CN202310024635 uses a four-directional polarization camera combined with projected polarized light to solve the high reflection problem. The four sets of polarization images of the polarization camera are fused to reconstruct the object shape information, thereby improving the robustness and reconstruction integrity of the system. Although the four-directional polarization camera can provide reconstruction data in four polarization directions, the camera resolution is sacrificed. The system resolution is only one-fourth of that of an ordinary camera, and the camera cost is relatively high.

[0044] Patent application CN202210831038 uses a modified error energy function to remove abnormal invalid points. The error energy function combines the wrapped phase of the fringe image, the background light intensity, and the modulated light intensity related information. This patent application removes abnormal points from a purely algorithmic perspective and does not solve the problem of high reflection and multiple reflections from an optical perspective.

[0045] Patent application CN202110372335 uses multiple shooting methods to fuse reconstruction data under different projection intensities to solve the problem of high reflection. The image pixel features are used to estimate the optimal projection brightness, and the corresponding number of stripe patterns are projected according to the number, and the stripe image of the object is captured. The stripe images of each group are fused to obtain a high-dynamic stripe image, and then more complete data is reconstructed. This solution trades time for integrity, is not real-time, and does not solve the problems of high reflection and multiple reflections from an optical perspective.

[0046] Patent application CN202110369458 uses a liquid crystal display to project linearly polarized light, and places a polarizer in front of the imaging lens. When the camera is shooting, the angle of the polarizer in front of the camera is rotated four times, the corresponding coded image is collected, the Stokes parameters are calculated based on the four images, and the functional relationship between the linear polarization degree and the incident angle is determined. Finally, the normal information of each point on the surface of the object is calculated according to the incident angle constraint condition to obtain the morphological information of the surface of the object. This solution uses the polarization characteristics of light to solve the problem of mirror reflection, but it needs to mechanically rotate the polarizer in front of the lens and ensure the accuracy of the polarization angle, so it is not very practical.

[0047] In addition, the method of using polarizers to solve the high reflection problem in structured light 3D reconstruction has been disclosed. Although it can reduce or eliminate the high reflection phenomenon in certain areas of the image according to the difference between the polarization direction of the high reflection light and the imaging polarization direction, it also reduces the illumination intensity of other normal diffuse reflection areas, causing insufficient lighting and the image is too dark to be completely reconstructed. This trade-off has caused new problems and has not improved the practicality of structured light 3D reconstruction.

[0048] In response to the problem of multiple reflections, the applicant realized that when polarized light with different polarization directions, such as linear polarized light, is incident at the same angle, the reflectivity of the object surface is different (the reflection intensity is different). After multiple surface reflections, the reflection intensity is quite different. This can be used by means of image algorithms to estimate the degree of difference in reflectivity to determine whether multiple reflections have occurred, thereby improving the reconstruction integrity.

[0049] Based on this, the applicant proposed a three-dimensional imaging method based on structured light, which uses polarized light with different polarization directions to enter the imaged object at the same incident angle for structured light projection. At this time, the reflectivity of the object surface is different, and the polarized light reflected from a certain point on the object surface enters the corresponding pixel position of the image sensor with different light intensities, resulting in different intensities of the projected image of the imaged object collected. The multiple reflection area is identified based on the difference in the intensity of the projected image, and the reconstructed height image is corrected. The method of the present application analyzes the difference between multiple reflections and normal reflections from the optical essence, and solves the difficult problem of multiple reflections in three-dimensional reconstruction of complex surface objects by combining polarization optical technology and image processing algorithms.

[0050] Please refer to Figure 2 In some embodiments, the structured light-based three-dimensional imaging method includes steps 10 to 40, which are described in detail below.

[0051] Step 10: Acquire a first polarized structured light image and a second polarized structured light image of the imaged object.

[0052] The first polarized structured light image is an image of the imaged object captured when the imaged object is subjected to structured light projection using the first polarized light, and the second polarized structured light image is an image of the imaged object captured when the imaged object is subjected to structured light projection using the second polarized light, the first polarized light and the second polarized light have different polarization directions, and the first polarized light and the second polarized light are incident on the surface of the imaged object at the same incident angle. Preferably, the polarization directions of the first polarized light and the second polarized light are perpendicular.

[0053] This application does not limit the projection lighting device for structured light projection, as long as it can use polarized light with two different polarization directions to be incident at the same incident angle. For example, the same projection lighting device can be used for structured light projection in the same posture, and a polarizer with a different polarization direction is installed in front of the projection lens for each projection; or a projection lighting device with a switchable two-channel polarized light source is used to perform two polarized light projections.

[0054] Step 20: Perform three-dimensional reconstruction using the first polarized structured light image to obtain a first height image of the imaged object; perform three-dimensional reconstruction using the second polarized structured light image to obtain a second height image of the imaged object.

[0055] The three-dimensional reconstruction here may adopt any three-dimensional reconstruction method, and this application does not limit this.

[0056] Step 30: Calculate a reflectivity difference map based on the first polarized structured light image and the second polarized structured light image

[0057] The pixel value of each pixel position in the reflectivity difference map represents the reflectivity difference degree of the pixel position, and the reflectivity difference degree represents the difference degree between the reflectivity of the surface of the imaged object corresponding to the pixel position to the first polarized light and the reflectivity to the second polarized light.

[0058] Step 40: Fusing the first height image and the second height image according to the reflectivity difference map to obtain a final height image of the imaged object.

[0059] Specifically, it is determined based on the reflectivity difference map whether there are multiple reflections on the surface of the imaged object corresponding to each pixel position. If so, the height value of the pixel position is set to an invalid pixel value. If not, the final height value of the pixel position is determined based on the height values ​​of the first height image and the second height image at the pixel position to obtain the final height image of the imaged object.

[0060] The applicant has conducted research on the projection of fringe patterns in structured light projection. The general N-step cosine phase shift method is used for three-dimensional reconstruction. The projection lighting device is controlled to project a standard N-step cosine fringe pattern onto the imaged object. The imaging device is used to collect N fringe projection images. The phase image is obtained by the phase resolution method. The height image is reconstructed by combining the pre-calibrated geometric constraint parameters of the projection lighting device and the imaging device. Where N is an integer not less than 2.

[0061] The N-step phase shift method can be divided into two parts: encoding and decoding. First, the fringe pattern is encoded according to certain rules and sent to the projection lighting device for projection. The encoding formula of the N-step fringe projection pattern is as follows:

[0062]

[0063] Where I is the ideal projection intensity, I(x,y) represents the ideal projection intensity at the pixel position (x,y), represents the phase of the pixel position (x, y), A is the coded background light intensity, B is the coded modulated light intensity, both are ideal coded values, and k is the number of steps.

[0064] After the fringe pattern is projected onto the object, the object is imaged and decoded or phase-decoded according to the N fringe projection images taken to obtain the phase, decoded background light intensity, and decoded modulated light intensity. The decoding formula is as follows:

[0065]

[0066] Wherein, I' is the intensity (grayscale value) of the captured fringe projection image, I'(x, y) represents the intensity at the pixel position (x, y), A' is the decoded background light intensity, and B' is the decoded modulated light intensity.

[0067] Due to the influence of ambient light, differences in light incident angles, differences in optical lenses, nonlinear responses of image sensors, and differences in object surface reflectivity, the relationship between the ideal projection intensity and the intensity of the actual captured fringe projection image can be simply expressed as:

[0068] I'(x,y)=I(x,y)·F(x,y)·ξ(x,y)+μ(x,y), (3)

[0069] Among them, μ(x,y) is the influence of the background ambient light on the pixel position (x,y), that is, the background light intensity collected by the imaging device when the projection lighting device is turned off; F(x,y) is the reflectivity of the surface of the imaged object corresponding to the pixel position (x,y); ξ(x,y) is the collective expression of other nonlinear influencing factors. Substituting formulas (1) and (2) into (3) to simplify, we can get:

[0070]

[0071] The above formula can be approximated as follows:

[0072] A'=A·F(x,y)·ξ(x,y)+μ(x,y),

[0073] B'=B·F(x,y)·ξ(x,y),

[0074] The above two equations show that the difference in reflectivity of the object surface can be reflected in the two indicators of decoding background light intensity and decoding modulated light intensity.

[0075] In the process of using two polarized light sources with different polarization directions for projection reconstruction, the currents of the two polarized light sources can be controlled to be consistent, so as to ensure that the brightness of the polarized light sources of the two channels is strictly consistent; and the two polarized light sources can be set to share optical devices, and the image sensor can also be shared during the shooting process (for example, using Figure 4 The structured light imaging device shown in the figure), therefore, the nonlinear influencing factors ξ(x, y) of the two projection reconstructions can be considered to be consistent; the ambient background light μ(x, y) is also consistent. Therefore, the first background light intensity image and the first modulated light intensity image obtained by projecting the fringe pattern using the first polarized light, and the second background light intensity image and the second modulated light intensity image obtained by projecting the fringe pattern using the second polarized light can be expressed as:

[0076] A1'(x,y)=A(x,y)·F1(x,y)·ξ(x,y)+μ(x,y),

[0077] B1'(x,y)=B(x,y)·F1(x,y)·ξ(x,y),

[0078] A'2(x,y)=A(x,y)·F2(x,y)·ξ(x,y)+μ(x,y),

[0079] B'2(x,y)=B(x,y)·F2(x,y)·ξ(x,y),

[0080] Among them, A1' represents the first background light intensity image, A1'(x, y) represents the first background light intensity at the pixel position (x, y), A'2 represents the second background light intensity image, A'2(x, y) represents the second background light intensity at the pixel position (x, y), B1' represents the first modulated light intensity image, B1'(x, y) represents the first modulated light intensity at the pixel position (x, y), B'2 represents the second modulated light intensity image, B'2(x, y) represents the second modulated light intensity at the pixel position (x, y).

[0081] By subtracting, we get:

[0082] ΔA'=A1'-A'2=[F1(x,y)-F2(x,y)]·A·ξ(x,y),

[0083] ΔB'=B1'-B'2=[F1(x,y)-F2(x,y)]·B·ξ(x,y),

[0084] ΔA′ represents the difference between the first background light intensity image and the second background light intensity image, and ΔB′ represents the difference between the first modulated light intensity image and the second modulated light intensity image.

[0085] Therefore, the reflectivity difference between the two polarized light projection reconstructions can be quantified by the numerical difference between the two reconstructed decoded background light intensities or decoded modulated light intensities. Therefore, the reflectivity difference map ΔF can be determined by the following formula:

[0086] ΔF=ΔA'=A1'-A'2, (4)

[0087] or,

[0088] ΔF=ΔB'=B1'-B'2, (5)

[0089] Although both of the above can represent the reflectivity difference, when the surface of the object is translucent, the contrast of the black and white stripe image is too low, resulting in the decoded modulated light intensity being too low, and unable to accurately reflect the reflectivity. In this case, it is more appropriate to use the decoded background light intensity to represent the reflectivity difference; and when the surface of the object is a complex texture surface, the decoded background light intensity cannot accurately reflect the reflectivity, the black and white stripe image has a high contrast, and the decoded modulated light intensity is high. It is more appropriate to use the decoded modulated light intensity to represent the reflectivity difference. In order to improve the stability and applicability of the system, this application preferably uses a linear weighted method to combine and represent the reflectivity difference:

[0090] ΔF=α·ΔA'+(1-α)·ΔB', (6)

[0091] Where α is the preset weight value, which is set by the user according to the usage scenario and ranges from [0.0, 1.0].

[0092] When fringe pattern projection is used for three-dimensional reconstruction, the acquired first polarized structured light image includes N first fringe projection images obtained when the fringe pattern is projected on the imaged object using the first polarized light according to the N-step phase shift method, and the second polarized structured light image includes N second fringe projection images obtained when the fringe pattern is projected on the imaged object using the second polarized light according to the N-step phase shift method.

[0093] Based on the above analysis, in steps 20 to 30, the first polarized structured light image is used for three-dimensional reconstruction to obtain a first height image of the imaged object, the second polarized structured light image is used for three-dimensional reconstruction to obtain a second height image of the imaged object, and the reflectivity difference map is calculated based on the first polarized structured light image and the second polarized structured light image, including:

[0094] Performing phase deconvolution on N first fringe projection images to obtain a first phase image, a first background light intensity image, and a first modulated light intensity image; performing phase deconvolution on N second fringe projection images to obtain a second phase image, a second background light intensity image, and a second modulated light intensity image; the phase deconvolution process here can adopt any phase deconvolution method, which is not limited in the present application;

[0095] Perform three-dimensional reconstruction based on the first phase image to obtain a first height image; perform three-dimensional reconstruction based on the second phase image to obtain a second height image; the three-dimensional reconstruction here can adopt any three-dimensional reconstruction method, and this application does not limit this;

[0096] The reflectivity difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, or the reflectivity difference map is calculated based on the difference between the first modulated light intensity image and the second modulated light intensity image, or the reflectivity difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image and the difference between the first modulated light intensity image and the second modulated light intensity image. For the specific formula for determining the reflectivity difference map, please refer to the above formulas (4), (5), and (6).

[0097] Regarding the final height image of the imaged object, it is essentially obtained by fusing the first height image and the second height image using the reflectivity difference map. In some embodiments, the final height image of the imaged object obtained by fusion is determined by the following formula:

[0098]

[0099] Wherein, H(x,y) represents the height value at the pixel position (x,y) in the final height image of the imaged object, H1(x,y) represents the height value at the pixel position (x,y) in the first height image, H2(x,y) represents the height value at the pixel position (x,y) in the second height image, ΔF(x,y) represents the degree of reflectivity difference at the pixel position (x,y) in the reflectivity difference map, T is the preset reflectivity difference degree threshold, and h0 represents an invalid pixel value.

[0100] Please refer to Figure 3 It can be seen that when facing a multi-reflective surface, the surface reconstructed by the traditional 3D reconstruction method (Figure (b)) is jagged and inconsistent with the actual situation, while the method of the present application can suppress multiple reflections and correctly reconstruct the multi-reflective surface (Figure (c)), thereby reducing the impact of multiple reflections on the object surface on the reconstruction accuracy and integrity, improving the integrity and authenticity of the 3D reconstruction, and effectively improving the material adaptability and scene adaptability of the 3D imaging system and method.

[0101] In view of the difficulties of structured light 3D reconstruction mentioned above, existing methods usually improve and optimize around optical or algorithmic solutions. Some embodiments of the present application also provide a highly practical, low-cost, and highly real-time structured light imaging device, which is designed based on polarized light imaging technology and has a dual-channel polarized light source. It can easily implement the above 3D imaging method and solve the technical problem of high reflectivity on the surface of objects.

[0102] Please refer to Figure 4 In some embodiments, the structured light imaging device includes a projection lighting device 100 and an imaging device 200, wherein the projection lighting device 100 adopts a two-channel lighting light source design, including a first light source 101, a first polarizer 102, a second light source 103, a second polarizer 104, a light combining device 105 and a spatial light modulator 106; the imaging device 200 includes a polarizer 201 and an image sensor 202.

[0103] The first light source 101 and the second light source 103 may be LED light sources, etc. The first polarizer 102 is disposed on the optical path of the first light beam emitted by the first light source 101, and is used to polarize the first light beam into a first polarized light in a first polarization direction; the second polarizer 104 is disposed on the optical path of the second light beam emitted by the second light source 102, and is used to polarize the second light beam into a second polarized light in a second polarization direction; wherein the first polarized light and the second polarized light may be linearly polarized light, and the first polarization direction is different from the second polarization direction.

[0104] The first light source 101 and the second light source 102 can be turned on or off at the same time, or one of them can be turned on and the other off.

[0105] The light combining device 105 is disposed at the intersection of the optical path of the first polarized light and the optical path of the second polarized light, and is used to combine the first polarized light and the second polarized light into the same optical path and emit the light to the spatial light modulator 106 when both the first light source 101 and the second light source 102 are turned on, and emit the first polarized light or the second polarized light to the spatial light modulator 106 when only the first light source 101 or only the second light source 102 is turned on. The light combining device 105 combines the first polarized light and the second polarized light into the same optical path, which makes the optical path structure compact, which is beneficial to reducing the volume of the projection lighting device 100, and allows the first polarized light and the second polarized light to share optical devices, receive the same optical processing, and be incident at the same angle.

[0106] like Figure 4 As shown, in some embodiments, the optical path of the first polarized light is perpendicular to the optical path of the second polarized light, and the light combining device 105 is a beam splitter (also called a semi-transparent and semi-reflective mirror), the beam splitter and the optical path of the first polarized light and the optical path of the second polarized light are at a 45° angle, and the first polarized light and the second polarized light are incident on the beam splitter from both sides of the beam splitter, respectively. One polarized light is transmitted through the beam splitter, and the other polarized light is reflected, so as to be combined into the same optical path.

[0107] The spatial light modulator 106 is used to modulate the light beam incident therein (e.g., the outgoing light of the light combining device 105) into a specific structured light, so as to project the structured light onto the imaged object 400, thereby realizing structured light projection. In some embodiments, the structured light is used to project a stripe pattern, for example, the stripe pattern is projected by an N-step phase shift method, and the imaging device 200 obtains N stripe projection images of the imaged object 400 for three-dimensional reconstruction. The user can set the projection pattern according to actual needs, and input the structured light imaging device to control the spatial light modulator 106 to modulate the light beam into a structured light that projects the pattern. The spatial light modulator 106 can be a digital micromirror device DMD, LCD, or LCOS, etc.

[0108] The imaged object may be an industrial product or workpiece, a mechanical part, an electronic component, etc., and this application does not limit this.

[0109] The analyzer 201 is used to receive structured reflected light formed by the structured light reflected by the imaging object 400, analyze the structured reflected light, and then project it onto the image sensor 202 for imaging. The image sensor 202 may be a CMOS sensor or a CCD sensor.

[0110] In the structured light imaging device of the present application, since the projection lighting device 100 uses polarized light for projection, and the polarization state of the reflected light changes to a certain extent after the polarized light is reflected by the surface of the object, the analyzer 201 can filter out high reflection and glare light in a specific polarization direction during imaging, thereby improving the three-dimensional reconstruction performance of highly reflective objects, and enhancing the reconstruction integrity and material adaptability. Preferably, the polarization direction of the analyzer 201 is different from both the first polarization direction and the second polarization direction, so that high reflection and glare light can be better filtered out when two polarized lights are used for projection.

[0111] In addition, considering that in the existing method of adding a polarizer in front of the imaging lens to solve the high reflection problem, there is still a problem that the high reflection area is weakened and eliminated, but other diffuse reflection areas are also weakened and insufficient light occurs, in the present application, a light combining device 105 is provided, and the light combining device 105 can combine the first polarized light and the second polarized light into one polarized light. The control device 300 can control the first light source 101 and the second light source 102 to work simultaneously, so that the two polarized lights can be used to simultaneously project the imaged object 400. Since the polarization direction of the first polarized light is different from the polarization direction of the second polarized light, it is almost impossible for them to be completely filtered out by the polarizer 201 at the same time. This not only weakens the high reflection area, but also complements the light intensity of the diffuse reflection area, thereby improving the final three-dimensional reconstruction integrity.

[0112] Preferably, the first polarization direction is perpendicular to the second polarization direction. For example, the first polarizer 103 is polarized in the horizontal direction (abbreviated as S polarization), and the second polarizer 104 is polarized in the vertical direction (abbreviated as P polarization). This is used as an example for explanation below. Further, the polarization direction of the analyzer 201 is at a 45° angle to both the first polarization direction and the second polarization direction. Figure 5 As shown. The analysis angle of the analyzer 201 can eliminate the high reflection and glare caused by the first polarized light (S polarized light) light source irradiating the surface of the object to a certain extent, and can also eliminate the high reflection and glare caused by the second polarized light (P polarized light) light source irradiating the surface of the object. At the same time, even in extreme cases, when the first polarized light (S polarized light) light source irradiates the surface of the object and reflects, the polarization direction of the light changes, which is completely perpendicular to the analysis direction of the analyzer 201 of the imaging device 200. The light cannot pass through the analyzer 201, and the first polarized light (S polarized light) cannot be imaged at all. However, after the second polarized light (P polarized light) irradiates the surface of the object and reflects, it is almost impossible for the reflected light to be perpendicular to the analysis direction of the analyzer 201, and it can still pass through the analyzer 201 to achieve illumination imaging. Under normal circumstances, the first polarized light (S polarized light) light source and the second polarized light (P polarized light) light source form complementary illumination in the diffuse reflection area. It should be noted that after the analyzer in the present application is assembled in the system assembly stage, there is no need to rotate the analysis angle again during subsequent use.

[0113] Please refer to Figure 6 In some embodiments, the projection lighting device 100 further includes a first collimating lens group 108 and a second collimating lens group 109. The first collimating lens group 108 is disposed between the first light source 101 and the first polarizer 103, and is used to collimate the first light beam and project it onto the first polarizer 103; the second collimating lens group 109 is disposed between the second light source 102 and the second polarizer 104, and is used to collimate the second light beam and project it onto the second polarizer 104.

[0114] Generally, a collimating lens is arranged behind a light source to collimate the light beam emitted by the light source. As for the polarizer, theoretically, it can achieve the effect of polarizing light by placing it at any position on the light path of the projection lighting device 100. However, if it is placed at different positions, the transmittance of light will be different, and the light transmission efficiency of the entire projection lighting device 100 will be greatly different. Figure 7As shown, if the incident angles are all summarized in the range of 0° to 90°, the transmittance of the polarizer decreases as the incident angle of the light decreases, and the transmittance is highest when the incident angle is 90°. Currently, most of the disclosed methods place the polarizer in front of the projection lens, and the projection lens is a FA lens. The light has a fixed field angle. In the edge field of view, the incident angle is small and the light transmittance is low, which affects the final light transmission efficiency and uniformity. The present application places the polarizer after the collimating lens group of the light source. The light passing through the collimating lens group can ensure good collimation. The incident light angles of each field of view are close to 90°, and the light transmittance passing through the polarizer is close to the highest, which greatly improves the light transmission efficiency and projection uniformity.

[0115] Please refer to Figure 4 and Figure 6 In some embodiments, the projection lighting device 100 further includes a projection lens group 107 (also called a projection lens), which is disposed on the optical path of the structured light and is used to project the structured light onto the imaged object 400. The projection lens group 107 adopts a common FA design or a telecentric design.

[0116] Usually, the projection object plane is not perpendicular to the main optical axis of the projection lens 107, but forms a certain angle with the main optical axis of the projection lens 107, for example, a 60° angle, that is, the main optical axis of the projection lens 107 forms a 30° angle with the main optical axis of the imaging device 200 (of course, it can also be other angles, usually in the range of [27°, 30°]). In order to ensure the clarity of the projection object plane, in the optical path design, the spatial light modulator 106 is placed at a certain angle so that the projection image plane of the spatial light modulator 106, the main plane of the projection lens 107, and the projection object plane satisfy Sham's law. Figure 8 As shown, the extension line of the projection image plane, the extension line of the projection lens principal plane and the extension line of the projection object plane intersect at one point, so that even if the projection lens 107 is installed at an angle, a clear projection pattern can still be obtained on the object plane.

[0117] Please refer to Figure 4 and Figure 6 In some embodiments, the imaging device 200 further includes an imaging lens group 203 (also referred to as an imaging lens), and the polarizer 201 is disposed between the imaging lens group 203 and the image sensor 202. The imaging lens group 203 is used to receive structured reflected light formed by the structured light being reflected by the imaging object 400, and converge and project it onto the polarizer 201.

[0118] The imaging lens 203 preferably adopts an image-side telecentric design or a double telecentric design. The analyzer 201 is placed between the imaging lens 203 and the image sensor 202, and the imaging lens 203 adopts an image-side telecentric lens or a double telecentric lens, so as to ensure that the angle of the incident light passing through the analyzer 201 is as close to the 90° incident angle as possible, so that a high light transmittance and imaging uniformity can also be ensured on the imaging side. Under the same lighting brightness conditions, the exposure time of the image sensor 202 is shorter, the shooting speed of the structured light three-dimensional imaging device is faster, and it has higher real-time performance.

[0119] Please refer to Figure 6 In some embodiments, the projection lighting device 100 further includes a reflector 110, which is disposed on the optical path of the outgoing light of the light combining device 105 and is used to reflect the outgoing light of the light combining device 105 to the spatial light modulator 106. The value range of the angle θ between the normal of the reflection surface of the reflector 110 and the outgoing light of the light combining device 105 is 0°<θ<90°, so that the optical path of the outgoing light of the light combining device 105 is turned.

[0120] Please refer to Figure 6 In some embodiments, the projection lighting device 100 further includes a light homogenizing device 111, which is disposed on the optical path of the output light of the light combining device 105 incident on the reflector 110, and is used to homogenize the output light of the light combining device 105 and then project it to the reflector 110. The light homogenizing device 111 may be a double-sided microlens array. The provision of the light homogenizing device 111 makes the illumination more uniform.

[0121] Please refer to Figure 6 In some embodiments, the projection lighting device 100 further includes a first relay lens 112, which is disposed on the optical path of the output light of the light homogenizing device 111 incident on the reflector 110, and is used to converge the output light of the light homogenizing device 111 and then project it to the reflector 110. Since the light beam is relatively divergent after passing through the light homogenizing device 111, the first relay lens 112 is provided in this embodiment to converge the light beam, for example, the output light of the light combining device 105 is cut into light spots by the double-sided microlens array, and then converged by the first relay lens 112, so that the light is more concentrated.

[0122] Please refer to Figure 6 In some embodiments, the projection lighting device 100 further includes a second relay lens 113, which is disposed on the optical path of the reflected light from the reflector 110 incident on the spatial light modulator 106, and is used to converge the reflected light from the reflector 110 and then project it to the spatial light modulator 106. Here, the light beam is converged again to improve the convergence effect.

[0123] from Figure 6As can be seen in the figure, the reflector 110 bends the originally straight light path, so that the optical components can be arranged more centrally and compactly in the projection lighting device 100.

[0124] Please refer to Figure 6 In some embodiments, the projection lighting device 100 further includes a compound prism 114, which is disposed in front of the spatial light modulator 106, so that the output light of the second relay lens 113 is incident on the spatial light modulator 106 through the compound prism 114, and the structured light output from the spatial light modulator 106 is reflected by the compound prism 114 onto the imaged object 400.

[0125] The propagation path of light in the composite prism 114 is as follows: Figure 6 In some embodiments, the positions of the spatial light modulator 106 and the compound prism 114 are set to cooperate with the reflector 110 so that the direction of the structured light emitted by the compound prism 114 is opposite to the direction of the output light of the light combining device 105, so that the overall light path of the projection lighting device 100 is in a folded form, making the structure of the projection lighting device 100 more compact and miniaturized. It should be noted that at this time, the projection image plane is not on the spatial light modulator 106, but on the reflection surface of the structured light on the compound prism 114.

[0126] According to the on-off state of the first light source 101 and the second light source 102, the projection lighting device 100 can realize at least three projection modes: single polarization projection mode: a single-channel illumination light source is lit, that is, only one of the first light source 101 and the second light source 102 is turned on; dual polarization projection mode: dual-channel illumination light sources are lit at the same time, that is, both the first light source 101 and the second light source 102 are turned on, and the first polarized light and the second polarized light are used for simultaneous projection. This mode can realize the complementarity of the dual polarized light sources; alternating projection mode: the dual-channel illumination light sources are lit in sequence, that is, the projection lighting device 100 performs two structured light projections, one of which is when the first light source 101 is turned on and the second light source 102 is turned off, and the first polarized light is used for projection, and the other time is when the first light source 101 is turned off and the second light source 102 is turned on, and the second polarized light is used for projection. This mode can realize the above-mentioned three-dimensional imaging method and suppress multiple reflections.

[0127] Based on the structured light imaging device of the present application, the first polarized structured light image and the second polarized structured light image of the imaged object are obtained in the above step 10, including: controlling the projection lighting device 100 to perform structured light projection on the imaged object in an alternating projection mode, in which the projection lighting device performs two structured light projections, and triggers the imaging device 200 to shoot the imaged object each time, thereby obtaining the first polarized structured light image and the second polarized structured light image.

[0128] The user can input instructions to select the projection mode, which correspond to the single polarization projection mode, the dual polarization projection mode and the alternating projection mode, namely the first imaging instruction, the second imaging instruction and the third imaging instruction.

[0129] In some embodiments, the three-dimensional imaging method of the present application further includes:

[0130] When receiving the first imaging instruction input by the user, the projection lighting device 100 is controlled to perform structured light projection on the imaged object in the single polarization projection mode, and the imaging device 200 is triggered to shoot the imaged object to obtain a fifth polarization structured light image of the imaged object, and the fifth polarization structured light image is used to perform three-dimensional reconstruction to obtain a height image of the imaged object; the three-dimensional reconstruction here can adopt any three-dimensional reconstruction method, and the present application is not limited to this;

[0131] When receiving the second imaging instruction input by the user, the projection lighting device 100 is controlled to perform structured light projection on the imaged object in the dual polarization projection mode, and the imaging device 200 is triggered to shoot the imaged object to obtain a sixth polarization structured light image of the imaged object, and the sixth polarization structured light image is used to perform three-dimensional reconstruction to obtain a height image of the imaged object; the three-dimensional reconstruction here can adopt any three-dimensional reconstruction method, and the present application is not limited to this;

[0132] When the third imaging instruction input by the user is received, the above steps 10 to 40 are executed to obtain a height image of the imaged object.

[0133] Users can select the projection mode according to the needs of the actual scene. When dealing with scenes with high real-time requirements, you can choose the single-polarization projection mode that only turns on the single-channel polarized light source; when dealing with scenes with multiple reflections, you can choose the alternating projection mode; when dealing with high-reflection and glare scenes, you can choose the dual-polarization projection mode that turns on the dual-channel polarized light source at the same time.

[0134] The following uses the first polarized light being S polarized light and the second polarized light being P polarized light as an example to illustrate the steps of performing the three-dimensional imaging method of the present application by using the structured light imaging device of the present application to project a stripe pattern in an alternating projection mode. Fig. 9 As shown, the polarization directions of the S polarized light and the P polarized light are perpendicular, and they propagate along the same optical path after passing through the optical combining device 105. Fig. 9 For the convenience of display, both S polarization and P polarization are shown, but it does not mean that S polarization and P polarization must be turned on at the same time. Fig.10 , the specific steps include:

[0135] 1. Turn on the first light source 101 (i.e., turn on the S-polarized light source), turn off the second light source 102 (i.e., turn off the P-polarized light source) to perform fringe pattern projection, collect N first fringe projection images obtained by S-polarized light projection through the image sensor 202, perform phase resolution processing, and obtain a first phase image, a first background light intensity image, and a first modulated light intensity image;

[0136] 2. Turn on the second light source 102 (i.e., turn on the P polarized light source), turn off the first light source 101 (i.e., turn off the S polarized light source) to perform fringe pattern projection, collect N second fringe projection images obtained by P polarized light projection through the image sensor 202, perform phase resolution processing, and obtain a second phase image, a second background light intensity image, and a second modulated light intensity image;

[0137] 3. Reconstruct the first phase image and the second phase image in three dimensions respectively through the pre-calibrated geometric constraint relationship to obtain the first height image and the second height image;

[0138] 4. According to the difference between the first background light intensity image and the second background light intensity image, and the difference between the first modulated light intensity image and the second modulated light intensity image, calculate the polarized light reflectivity difference to obtain a reflectivity difference map, see formula (6) for details;

[0139] 5. Perform threshold comparison based on the reflectivity difference map to determine the pixel area with multiple reflections, set the corresponding height value as an invalid pixel value, thereby eliminating abnormal data caused by multiple reflections, and finally fuse the height image that suppresses multiple reflections. For details, see formula (7).

[0140] Please refer to Fig.11In some embodiments, the structured light imaging device may also be provided with a control device 300, which may include an embedded control system, which is composed of an MCU (Microcontroller Unit), a SOC (System on Chips), a DSP (Digital Signal Processing) and / or an FPGA (Field Programmable Gate Array), and is mainly responsible for synchronization signal control, image processing and data transmission functions. The three-dimensional imaging method of each embodiment of the present application can be applied to the control device 300. The control device 300 can control the opening and closing of the dual-channel light source and the pattern projected by the spatial light modulator 106. The user can control the structured light imaging device by sending instructions to the control device 300 through the host computer, and the control device 300 starts the projection lighting device 100 and the imaging device 200 for projection and shooting according to the signal sent by the host computer. The shooting signal sent by the control device 300 to the imaging device 200 and the exposure time of the imaging device 200 need to be synchronized with the projection signal sent to the projection lighting device 100 and the projection time of the projection lighting device 100. After a series of structured light pattern projections and photographing, the control device 300 collects images formed by the image sensor 200, starts an image algorithm for reconstruction and fusion, and finally obtains a height image of the imaged object 400, and then transmits the height image to the host computer.

[0141] Please refer to Fig.11 and Fig.12 In some embodiments, the control device 300 includes a projection control module for controlling the projection lighting device 100. The projection control module receives a control signal from the control device 300 to control the two-channel light source (the first light source 101 and the second light source 102) and the spatial light modulator 106. When there are multiple projection lighting devices 100, each projection lighting device 100 corresponds to a projection control module for control. Fig.12 One projection lighting device 100 is controlled by a first projection control module, and the other projection lighting device 100 is controlled by a second projection control module. Fig.13 In some embodiments, the projection control module includes a first light source control module, a second light source control module and a light modulation control module. The first light source control module and the second light source control module respectively control the on / off enable and current size of the two-channel light sources, and change the projection brightness of the projection lighting device 100 by adjusting the current size of the light source. The light modulation control module can control the pattern projected by the spatial light modulator 106.

[0142] To address the problem of blind spots in the field of vision, the structured light imaging device of some embodiments of the present application uses a dual-projection lighting device to illuminate the imaged object from two different incident angles, such as Fig.12 As shown; and on this basis, another three-dimensional imaging method is proposed to remove some blind spots in the field of vision through image fusion. Specifically, two projection lighting devices are symmetrically arranged with the imaging device as the center. The two projection lighting devices can implement projection lighting on the imaged object from two different incident angles, thereby solving the reconstruction blind spot problem caused by the blind spot of the lighting. Please refer to Fig.14 When a single projection lighting device is used for reconstruction, area A in the figure is a blind spot for camera shooting. Even if it can be illuminated, it cannot be photographed, resulting in failure to reconstruct; area B in the figure is a lighting blind spot. The camera can capture it, but it cannot be illuminated, resulting in failure to reconstruct. This embodiment uses dual projection lighting devices for projection lighting, and performs two projection reconstructions through projection lighting devices placed in symmetrical directions, and then performs fusion, which can remove the lighting blind spot of area B in the figure.

[0143] Please refer to Fig.15 The three-dimensional imaging method based on the structured light imaging device includes steps 100 to 500.

[0144] Step 100: Control one of the two projection lighting devices to perform structured light projection on the imaged object with a first polarized light, trigger the imaging device to shoot the imaged object, and obtain a first polarized structured light image; control the projection lighting device to perform structured light projection on the imaged object with a second polarized light, trigger the imaging device to shoot the imaged object, and obtain a second polarized structured light image.

[0145] That is, one of the two projection illumination devices is controlled to perform structured light projection in an alternating projection mode, and an imaging device is used to form an image to obtain a first polarized structured light image and a second polarized structured light image.

[0146] Step 200: Perform three-dimensional imaging using the first polarized structured light image and the second polarized structured light image to obtain a first projection direction height image.

[0147] The three-dimensional imaging here can be achieved according to the three-dimensional imaging method based on structured light in any of the foregoing embodiments of the present application.

[0148] Step 300: Control the other of the two projection lighting devices to perform structured light projection on the imaged object with the first polarized light, trigger the imaging device to shoot the imaged object, and obtain a third polarized structured light image; control the projection lighting device to perform structured light projection on the imaged object with the second polarized light, trigger the imaging device to shoot the imaged object, and obtain a fourth polarized structured light image.

[0149] That is, the other projection lighting device of the two projection lighting devices is controlled to perform structured light projection in an alternating projection mode, and an imaging device is used to form an image to obtain a third polarized structured light image and a fourth polarized structured light image.

[0150] Step 400: Perform three-dimensional imaging using the third polarized structured light image and the fourth polarized structured light image to obtain a second projection direction height image.

[0151] The three-dimensional imaging here can be achieved according to the three-dimensional imaging method based on structured light in any of the foregoing embodiments of the present application.

[0152] Step 500: Fusing the first projection direction height image and the second projection direction height image to obtain a height image of the imaged object.

[0153] In some embodiments, the height image of the imaged object is determined by the following formula:

[0154]

[0155] Among them, I h (x, y) represents the height value of the pixel position (x, y) in the height image of the imaged object, H 1d (x, y) represents the height value at the pixel position (x, y) in the height image in the first projection direction, H 2d (x, y) represents the height value at the pixel position (x, y) in the second projection direction height image.

[0156] Of course, the projection lighting device used in this embodiment is not limited to Fig.12 The projection lighting device shown in the figure can be used as long as the projection lighting device can perform structured light projection with the first polarized light and the second polarized light, and the polarization directions of the first polarized light and the second polarized light are different and the incident angles are the same. Similarly, the imaging device can also use other imaging devices, not limited to Fig.12 The imaging device shown.

[0157] In this embodiment, the structured light imaging device eliminates the lighting blind spot by setting up two projection lighting devices that are symmetrical with the imaging device as the center to illuminate the imaged object from two different incident angles, and also provides a method for fusing the height images obtained by projecting and reconstructing through the two projection lighting devices, thereby further improving the reconstruction integrity.

[0158] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0159] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A three-dimensional imaging method based on structured light, characterized in that: include: Acquire a first polarized structured light image and a second polarized structured light image of an imaged object; wherein the first polarized structured light image is an image of the imaged object taken when structured light projection is performed on the imaged object using a first polarized light, and the second polarized structured light image is an image of the imaged object taken when structured light projection is performed on the imaged object using a second polarized light, the first polarized light and the second polarized light have different polarization directions, and the first polarized light and the second polarized light are incident on the surface of the imaged object at the same incident angle; Performing three-dimensional reconstruction using the first polarized structured light image to obtain a first height image of the imaged object; performing three-dimensional reconstruction using the second polarized structured light image to obtain a second height image of the imaged object; A reflectivity difference map is calculated based on the first polarized structured light image and the second polarized structured light image, wherein the pixel value of each pixel position in the reflectivity difference map represents the reflectivity difference degree of the pixel position, and the reflectivity difference degree represents the difference degree of the reflectivity of the surface of the imaged object corresponding to the pixel position to the first polarized light and the reflectivity of the second polarized light; According to the reflectivity difference map, it is determined whether there are multiple reflections on the surface of the imaged object corresponding to each pixel position. If so, the height value of the pixel position is set to an invalid pixel value. If not, the final height value of the pixel position is determined based on the height values ​​of the first height image and the second height image at the pixel position to obtain the final height image of the imaged object.

2. The three-dimensional imaging method according to claim 1, characterized in that: The first polarized structured light image includes N first fringe projection images obtained when the first polarized light is used to project a fringe pattern onto the imaged object according to an N-step phase shift method, and the second polarized structured light image includes N second fringe projection images obtained when the second polarized light is used to project a fringe pattern onto the imaged object according to an N-step phase shift method, wherein N is an integer not less than 2; The method of performing three-dimensional reconstruction using the first polarized structured light image to obtain a first height image of the imaged object, performing three-dimensional reconstruction using the second polarized structured light image to obtain a second height image of the imaged object, and calculating a reflectivity difference map based on the first polarized structured light image and the second polarized structured light image includes: Performing phase deconvolution on the N first fringe projection images to obtain a first phase image, a first background light intensity image, and a first modulated light intensity image; performing phase deconvolution on the N second fringe projection images to obtain a second phase image, a second background light intensity image, and a second modulated light intensity image; Performing three-dimensional reconstruction based on the first phase image to obtain the first height image; performing three-dimensional reconstruction based on the second phase image to obtain the second height image; The reflectivity difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, or the reflectivity difference map is calculated based on the difference between the first modulated light intensity image and the second modulated light intensity image, or the reflectivity difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image and the difference between the first modulated light intensity image and the second modulated light intensity image.

3. The three-dimensional imaging method according to claim 2, characterized in that: The reflectivity difference map is determined by the following formula: ΔF=ΔA'=A1'-A'2, Or, ΔF = ΔB' = B'1 - B'2, Or, ΔF=αΔA'+(1-α)·ΔB'; Among them, ΔF represents the reflectivity difference map, ΔA' represents the difference between the first background light intensity image and the second background light intensity image, ΔB' represents the difference between the first modulated light intensity image and the second modulated light intensity image, A1' represents the first background light intensity image, A'2 represents the second background light intensity image, B1' represents the first modulated light intensity image, B'2 represents the second modulated light intensity image, and α is a preset weight value.

4. The three-dimensional imaging method according to claim 3, characterized in that: The value range of α is [0.0,1.0].

5. The three-dimensional imaging method according to any one of claims 1 to 4, characterized in that: The final height image of the imaged object is determined by the following formula: Among them, H(x,y) represents the height value at the pixel position (x,y) in the final height image of the imaged object, H1(x,y) represents the height value at the pixel position (x,y) in the first height image, H2(x,y) represents the height value at the pixel position (x,y) in the second height image, ΔF(x,y) represents the degree of reflectivity difference at the pixel position (x,y) in the reflectivity difference map, T is the preset reflectivity difference degree threshold, and h0 represents an invalid pixel value.

6. The three-dimensional imaging method according to claim 1, characterized in that: The polarization directions of the first polarized light and the second polarized light are perpendicular to each other.

7. The three-dimensional imaging method according to claim 1, characterized in that: The three-dimensional imaging method is implemented based on a structured light imaging device, which includes a projection lighting device and an imaging device. The projection lighting device includes a first light source, a first polarizer, a second light source, a second polarizer, a light combining device and a spatial light modulator. The imaging device includes an analyzer and an image sensor. The first polarizer is disposed on the optical path of the first light beam emitted by the first light source, and is used for polarizing the first light beam into the first polarized light in the first polarization direction; The second polarizer is disposed on the optical path of the second light beam emitted by the second light source, and is used to polarize the second light beam into the second polarized light in a second polarization direction; wherein the first polarization direction is different from the second polarization direction; The first light source and the second light source can be turned on or off at the same time, or one of them can be turned on and the other can be turned off; The light combining device is disposed at the intersection of the optical path of the first polarized light and the optical path of the second polarized light, and is used to combine the first polarized light and the second polarized light into the same optical path and emit them to the spatial light modulator when both the first light source and the second light source are turned on, and to emit the first polarized light or the second polarized light to the spatial light modulator when only the first light source is turned on or only the second light source is turned on; The spatial light modulator is used to modulate the light beam incident thereinto into a specific structured light, so as to project the structured light toward the imaged object; The polarizer is used to receive structured reflected light formed by the structured light reflected by the imaging object, analyze the structured reflected light and then project it onto the image sensor for imaging; The step of acquiring a first polarized structured light image and a second polarized structured light image of the imaged object comprises: The projection lighting device is controlled to perform structured light projection on the imaged object in an alternating projection mode. In the alternating projection mode, the projection lighting device performs structured light projection twice, one time with the first light source turned on and the second light source turned off, and another time with the first light source turned off and the second light source turned on. During each projection, the imaging device is triggered to shoot the imaged object, thereby obtaining the first polarized structured light image and the second polarized structured light image.

8. A three-dimensional imaging method based on a structured light imaging device, characterized in that: The structured light imaging device includes an imaging device and two projection lighting devices, and the two projection lighting devices are symmetrically arranged with the imaging device as the center, the projection lighting device can perform structured light projection with a first polarized light and a second polarized light, and the polarization directions of the first polarized light and the second polarized light are different. The three-dimensional imaging method includes: Controlling one of the two projection lighting devices to perform structured light projection on the imaged object with the first polarized light, triggering the imaging device to photograph the imaged object, and obtaining a first polarized structured light image; controlling the projection lighting device to perform structured light projection on the imaged object with the second polarized light, triggering the imaging device to photograph the imaged object, and obtaining a second polarized structured light image; The three-dimensional imaging method according to any one of claims 1 to 6, using the first polarized structured light image and the second polarized structured light image to perform three-dimensional imaging to obtain a first projection direction height image; Controlling the other of the two projection lighting devices to perform structured light projection on the imaged object with the first polarized light, triggering the imaging device to photograph the imaged object, and obtaining a third polarized structured light image; controlling the projection lighting device to perform structured light projection on the imaged object with the second polarized light, triggering the imaging device to photograph the imaged object, and obtaining a fourth polarized structured light image; The three-dimensional imaging method according to any one of claims 1 to 6, using the third polarized structured light image and the fourth polarized structured light image to perform three-dimensional imaging to obtain a second projection direction height image; The first projection direction height image and the second projection direction height image are fused to obtain a height image of the imaged object.

9. The three-dimensional imaging method according to claim 8, characterized in that: The height image of the imaged object is determined by the following formula: Among them, I h (x, y) represents the height value at the pixel position (x, y) in the height image of the imaged object, H 1d (x, y) represents the height value at the pixel position (x, y) in the height image in the first projection direction, H 2d (x, y) represents the height value at the pixel position (x, y) in the second projection direction height image, and h0 represents an invalid pixel value.

10. A computer-readable storage medium, characterized in that: The medium stores a program, which can be executed by a processor to implement the three-dimensional imaging method according to any one of claims 1 to 9.

Images