Structured light imaging device, three-dimensional imaging method therefor, and storage medium

Abstract

A structured light imaging device, a three-dimensional imaging method therefor, and a storage medium. The structured light imaging device comprises two polarizers which generate polarized light, a light-combining device, and a polarization analyzer, and can suppress high reflectivity and improve the accuracy and integrity of three-dimensional reconstruction. The three-dimensional imaging method comprises: using two rays of polarized light to perform structured light projection so as to obtain a first polarized structured light image and a second polarized structured light image of an imaged object, and separately using the images to perform three-dimensional reconstruction so as to obtain a first height image and a second height image; on the basis of the first polarized structured light image and the second polarized structured light image, determining whether there are multiple reflections on the imaged object surface corresponding to a pixel position, and if not, then determining a final height value of the pixel position on the basis of the height values of the first height image and the second height image at said pixel position, thereby reducing the impact which multiple reflections on the object surface have on reconstruction accuracy and integrity.

Description

A structured light imaging device, its three-dimensional imaging method, and storage medium Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, specifically to a structured light imaging device, a three-dimensional imaging method based on structured light and the structured light imaging device, and a storage medium. Background Technology

[0002] With the increasingly widespread application of 3D inspection technology in the field of industrial automation, the use of 3D reconstruction technology based on structured light 3D cameras, with triangulation as its core principle, is also becoming more and more popular. Structured light 3D reconstruction systems often use a projector combined with a camera, with the projector and camera's principal optical axes at a certain angle. The projector projects structured light onto the object, and the camera captures the structured light image of the object. Then, 3D reconstruction is performed to obtain the object's height image, thereby reconstructing the object's surface morphology.

[0003] In a structured light 3D reconstruction system, the imaging unit performs 3D reconstruction by acquiring diffuse reflection images of the object's surface. 3D reconstruction utilizes the geometric constraints of reflected light rays, essentially a triangular constraint formed by the angle between the incident light ray and the diffuse reflection ray. When the object's surface has a complex shape, the incident light ray, after passing through the complex surface, causes multiple reflections. The image acquired by the imaging unit is not the actual diffuse reflection image of the object's surface, but rather a diffuse reflection image containing information from multiple surfaces. Even if reconstruction can be achieved, the obtained shape information cannot represent the true surface information of the object. As shown in Figure 1, the incident light ray and the normal diffuse reflection ray form a triangular constraint, but the geometric relationship between the multiple reflected rays and the incident light ray is uncertain and does not satisfy the triangular constraint. In this case, the geometric constraints of 3D reconstruction are not valid, leading to incomplete and inaccurate reconstructed data. Summary of the Invention

[0004] This invention provides a structured light-based three-dimensional imaging method, a structured light imaging device, and a storage medium, which respectively solve the technical problems of multiple reflections of light caused by complex surfaces and the problem of high reflectivity in three-dimensional reconstruction.

[0005] According to a first aspect, one embodiment provides a structured light-based three-dimensional imaging method, comprising:

[0006] Acquire a first polarized structured light image and a second polarized structured light image of the object being imaged; wherein, the first polarized structured light image is an image captured on the object being imaged when structured light projection is performed on the object using first polarized light, and the second polarized structured light image is an image captured on the object being imaged when structured light projection is performed on the object using second polarized light, wherein the polarization directions of the first polarized light and the second polarized light are different, and the first polarized light and the second polarized light are incident on the surface of the object being imaged at the same incident angle when structured light projection is performed on the object being imaged;

[0007] A first height image of the imaged object is obtained by performing three-dimensional reconstruction using the first polarized structured light image; a second height image of the imaged object is obtained by performing three-dimensional reconstruction using the second polarized structured light image.

[0008] A reflectance difference map is calculated based on the first polarized structured light image and the second polarized structured light image. The pixel value of each pixel position in the reflectance difference map represents the degree of reflectance difference at that pixel position. The degree of reflectance difference represents the degree of difference between the reflectance of the surface of the imaged object corresponding to that pixel position to the first polarized light and the reflectance to the second polarized light.

[0009] Based on the reflectivity difference map, it is determined whether there is multiple reflection on the surface of the object being imaged corresponding to each pixel position. If so, the height value of the pixel position is set as an invalid pixel value. If not, the final height value of the pixel position is determined based on the height values ​​of the first height image and the second height image at that pixel position, so as to obtain the final height image of the object being imaged.

[0010] According to a second aspect, one embodiment provides a three-dimensional imaging method based on a structured light imaging device. The structured light imaging device includes an imaging unit and two projection illumination units, which are symmetrically arranged around the imaging unit. The projection illumination units are capable of projecting structured light using first polarized light and second polarized light, wherein the polarization directions of the first polarized light and the second polarized light are different. The three-dimensional imaging method includes:

[0011] One of the two projection lighting devices is controlled to project structured light onto the object being imaged using the first polarized light, triggering the imaging device to capture the object being imaged, thereby obtaining a first polarized structured light image; the same projection lighting device is then controlled to project structured light onto the object being imaged using the second polarized light, triggering the imaging device to capture the object being imaged, thereby obtaining a second polarized structured light image.

[0012] According to the three-dimensional imaging method described in the first aspect above, three-dimensional imaging is performed using the first polarized structured light image and the second polarized structured light image to obtain a first projection direction height image;

[0013] Control the other of the two projection lighting devices to project structured light onto the object being imaged with the first polarized light, trigger the imaging device to capture the object being imaged, and obtain a third polarized structured light image; control the projection lighting device to project structured light onto the object being imaged with the second polarized light, trigger the imaging device to capture the object being imaged, and obtain a fourth polarized structured light image.

[0014] According to the three-dimensional imaging method described in the first aspect above, three-dimensional imaging is performed using the third polarization structured light image and the fourth polarization structured light image to obtain a second projection direction height image;

[0015] The height image of the first projection direction and the height image of the second projection direction are fused together to obtain the height image of the imaged object.

[0016] According to a third aspect, one embodiment provides a computer-readable storage medium storing a program that can be executed by a processor to implement the three-dimensional imaging method described in the first or second aspect above.

[0017] According to the fourth aspect, one embodiment provides a structured light imaging device, including a projection illumination device, an imaging device, and a control device;

[0018] The projection lighting device includes a first light source, a first polarizer, a second light source, a second polarizer, a light combining device, and a spatial light modulator;

[0019] The first polarizer is disposed in the optical path of the first beam emitted by the first light source, and is used to polarize the first beam into first polarized light with a first polarization direction.

[0020] The second polarizer is disposed in the optical path of the second beam emitted by the second light source, and is used to polarize the second beam into second polarized light with a second polarization direction; wherein the first polarization direction is different from the second polarization direction.

[0021] The control device is used to control the opening and closing of the first light source and the second light source;

[0022] The light combining device is located at the intersection of the optical paths of the first polarized light and the second polarized light, and is used to make the first polarized light and the second polarized light incident at the same angle. When both the first light source and the second light source are turned on, the light combining device combines the first polarized light and the second polarized light into the same optical path and emits them to the spatial light modulator. When only the first light source or only the second light source is turned on, the first polarized light or the second polarized light is emitted to the spatial light modulator.

[0023] The spatial light modulator is used to modulate the incident light beam into a specific structured light to project the structured light onto the object being imaged;

[0024] The imaging device includes an analyzer and an image sensor;

[0025] The analyzer is used to receive the structured reflected light formed by the structured light reflected from the object being imaged, and to analyze the polarization of the structured reflected light before projecting it onto the image sensor for imaging, so as to obtain a polarized structured light image.

[0026] The three-dimensional imaging method according to the above embodiments utilizes the principle that polarized light with different polarization directions has different reflectivities (i.e., different reflection intensities) on the surface of an object when incident at the same angle. Since the difference in reflection intensity after multiple surface reflections is large, the degree of difference in reflectivity can be used to determine whether multiple reflections have occurred. In the three-dimensional imaging method of this application, a first polarized structured light image and a second polarized structured light image are first obtained by structured light projection with two polarized lights of different polarization directions but the same incident angle. Three-dimensional reconstruction is performed using them respectively to obtain a first height image and a second height image. A reflectivity difference map is then calculated based on the first polarized structured light image and the second polarized structured light image. Based on the reflectivity difference map, it is determined whether there are multiple reflections on the surface of the imaged object corresponding to each pixel position. If so, the height value of that pixel position is set as an invalid pixel value. If not, the final height value of that pixel position is determined based on the height values ​​of the first height image and the second height image at that pixel position. This reduces the impact of multiple reflections on the object surface on the reconstruction accuracy and integrity, improves the integrity and realism of the three-dimensional reconstruction, and effectively enhances the material adaptability and scene adaptability of the three-dimensional imaging system and method.

[0027] The structured light imaging device according to the above embodiments includes a projection illumination device, an imaging device, and a control device. The projection illumination device includes a first light source, a first polarizer, a second light source, a second polarizer, a light combiner, and a spatial light modulator. The first light source and the first polarizer generate first polarized light, and the second light source and the second polarizer generate second polarized light, enabling the projection illumination device to perform structured light projection using one or both types of polarized light. The imaging device is equipped with a polarizer capable of analyzing the polarization of the structured light reflected from the object being imaged. Because the projection illumination device can use polarized light for projection, and the polarization state of the reflected light changes to some extent after reflection from the object's surface, the polarizer can filter out highly reflective light and glare in specific polarization directions during imaging, thereby improving the 3D reconstruction performance of highly reflective objects. Furthermore, the light combining device can combine the first polarized light and the second polarized light into a single polarized light and project it simultaneously. Since the polarization directions of the first polarized light and the second polarized light are different, it is almost impossible for them to be completely filtered out by the analyzer at the same time. When one polarized light is filtered out, the other polarized light can play a complementary role, which can suppress high reflectivity while ensuring illumination brightness. Attached Figure Description

[0028] Figure 1 is a schematic diagram of multiple reflections of light on the surface of an object;

[0029] Figure 2 is a flowchart of a structured light-based three-dimensional imaging method according to an embodiment;

[0030] Figure 3 is a comparison of the effects of the three-dimensional imaging method of this application and the traditional reconstruction method;

[0031] Figure 4 is a schematic diagram of the structure of a structured light imaging device according to an embodiment;

[0032] Figure 5 is a schematic diagram of the polarization direction of the first polarizer and the second polarizer and the polarization detection direction of the detector in one embodiment.

[0033] Figure 6 is a schematic diagram of the structure of another embodiment of the structured light imaging device;

[0034] Figure 7 is a schematic diagram showing the relationship between the light transmittance of the polarizer and the incident angle of the light;

[0035] Figure 8 is a schematic diagram of the geometric relationship between the projection image plane, the main plane of the projection lens, and the projection object plane of the spatial light modulator;

[0036] Figure 9 is a schematic diagram of the optical path of S-polarized light and P-polarized light after passing through the optical combining device.

[0037] Figure 10 is a schematic diagram of the process of three-dimensional reconstruction using S-polarized light and P-polarized light to suppress multiple reflections;

[0038] Figure 11 is a schematic diagram of the structure of a structured light imaging device according to another embodiment;

[0039] Figure 12 is a schematic diagram of the structure of another embodiment of the structured light imaging device;

[0040] Figure 13 is a schematic diagram of the projection control module according to one embodiment;

[0041] Figure 14 is a schematic diagram of the blind spots in the 3D reconstruction.

[0042] Figure 15 is a flowchart of a three-dimensional imaging method based on a structured light imaging device according to an embodiment.

[0043] Reference numerals: 100, projection illumination device; 101, first light source; 102, second light source; 103, first polarizer; 104, second polarizer; 105, light combining device; 106, spatial light modulator; 107, projection lens group; 108, first collimating lens group; 109, second collimating lens group; 110, mirror; 111, light homogenizing device; 112, first relay lens; 113, second relay lens; 114, compound prism; 200, imaging device; 201, analyzer; 202, image sensor; 203, imaging lens group; 300, control device; 400, object being imaged. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0045] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0046] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this application, unless otherwise specified, include both direct and indirect connections (linkages).

[0047] Currently, structured light-based 3D reconstruction methods still face the following challenges when dealing with complex surfaces (high reflectivity, multiple reflections, etc.):

[0048] 1. Structured light 3D cameras are a type of diffuse reflection imaging system. However, the materials of some object surfaces can cause specular reflection, resulting in high reflectivity. This makes it difficult to reconstruct the actual surface morphology of the object due to overexposure or underexposure of the images captured by the camera.

[0049] 2. When the surface of an object is complex, and the illumination light is incident on the object's surface, multiple reflections are formed after passing through the complex surface. The image captured by the camera is not a diffuse reflection image of the actual object's surface, but a diffuse reflection image containing multiple surface information. Even if it can be solved and reconstructed, the obtained shape information cannot represent the true object's surface information.

[0050] 3. Due to the angle, structured light 3D cameras will have two blind spots: first, areas that the projector can illuminate but the camera cannot capture; second, areas that the camera can capture but the projector cannot illuminate. These blind spots will lead to incomplete reconstruction data.

[0051] To address the challenges of structured light-based 3D reconstruction methods, existing approaches often employ high-cost cameras with limited practicality or multiple exposures that lack real-time capability for optimization, aiming to improve the reconstruction integrity of structured light 3D reconstruction.

[0052] Patent application CN202310024635 addresses the high reflectivity problem by combining a four-way polarization camera with projected polarized light. It reconstructs the object's shape information by fusing four sets of polarized images from the polarization camera, improving the system's robustness and reconstruction integrity. While the four-way polarization camera can provide reconstruction data in four polarization directions, it sacrifices camera resolution; the system resolution is only one-quarter that of a conventional camera, and the camera cost is relatively high.

[0053] Patent application CN202210831038 uses a modified error energy function to remove outlier points. This error energy function combines information related to the wrapping phase of the fringe image, background light intensity, and modulated light intensity. However, this patent application removes outliers purely from an algorithmic perspective and does not address the optical issues of high reflectivity and multiple reflections.

[0054] Patent application CN202110372335 addresses the high reflectivity problem by fusing reconstructed data from different projection intensities using multiple imaging techniques. It estimates the optimal projection brightness based on image pixel features, projects a corresponding number of stripe patterns based on this value, and captures stripe images of the object. These stripe images are then fused to obtain a high dynamic range stripe image, thereby reconstructing more complete data. However, this approach trades time for completeness, lacks real-time performance, and does not address the high reflectivity and multiple reflection issues from an optical perspective.

[0055] Patent application CN202110369458 uses a liquid crystal display to project linearly polarized light and places a polarizer in front of the imaging lens. During camera shooting, the polarizer in front of the camera is rotated four times, and corresponding coded images are acquired. Stokes parameters are calculated based on the four images to determine the functional relationship between the degree of linear polarization and the incident angle. Finally, the normal information of each point on the object's surface is calculated based on the incident angle constraint to obtain the object's surface morphology information. This scheme utilizes the polarization characteristics of light to solve the specular reflection problem, but it requires mechanically rotating the polarizer in front of the lens and ensuring the accuracy of the polarization angle, making it impractical.

[0056] Furthermore, while the publicly disclosed method of using polarizers to address the high reflectivity problem in structured light 3D reconstruction can reduce or eliminate high reflectivity in certain areas of the image by utilizing the difference between the polarization direction of the highly reflective rays and the imaging analysis direction, it also reduces the illumination intensity in other normally diffuse reflective areas, resulting in insufficient illumination and an overly dark image that cannot be fully reconstructed. This trade-off creates new problems and does not improve the practicality of structured light 3D reconstruction.

[0057] Regarding the issue of multiple reflections, the applicant recognized that polarized light with different polarization directions, such as linearly polarized light, will have different reflectivities (different reflection intensities) on the surface of an object when incident at the same angle. After multiple surface reflections, the difference in reflection intensity can be significant. This can be used to determine whether multiple reflections have occurred by estimating the degree of difference in reflectivity through image algorithms, thereby improving the integrity of the reconstruction.

[0058] Based on this, the applicant proposes a structured light-based 3D imaging method. This method uses polarized light with different polarization directions incident on the object being imaged at the same incident angle for structured light projection. Since the reflectivity of the object surface varies, the intensity of the polarized light reflected from a certain point on the object surface and projected onto the corresponding pixel position of the image sensor differs, resulting in varying intensities of the projected images of the object. Multiple reflection regions are identified based on these differences in projection image intensity, correcting the reconstructed height image. This application's method analyzes the difference between multiple reflections and normal reflections from an optical perspective, and combines polarization optics technology and image processing algorithms to solve the challenging problem of multiple reflections in the 3D reconstruction of complex surface objects.

[0059] This application also provides a highly practical, low-cost, and real-time structured light imaging device, which mainly addresses the technical problems that existing structured light 3D cameras and 3D reconstruction methods cannot effectively solve due to multiple reflections, high reflectivity, and blind spots on the object surface, which prevent the reconstruction of complete data. Multiple embodiments are provided to solve one or more of the above-mentioned technical problems.

[0060] In some embodiments, an algorithm for suppressing multiple reflections is proposed, which combines polarization optics technology with the algorithm for suppressing multiple reflections. Starting from multiple dimensions of optical principles and algorithm principles, the characteristics of high reflectivity and abnormal reflections are analyzed from the optical essence. A technical solution that conforms to theoretical analysis is proposed, which solves the difficult problems of poor reconstruction integrity and poor material adaptability caused by multiple reflections, high reflectivity and blind spots on the object surface in structured light 3D reconstruction.

[0061] The concept of this application includes:

[0062] 1. When polarized light, such as linearly polarized light, is incident on the surface of an object and reflected, the polarization state of the reflected light will change to a certain extent. The analyzer can filter out highly reflective light and glare light in a specific polarization direction.

[0063] 2. When linearly polarized light with different polarization directions is incident at the same angle, the reflectivity of the object surface is different (the reflection intensity is different). The applicant realized that the reflection intensity is significantly different after multiple surface reflections. This can be used to determine whether multiple reflections have occurred by estimating the degree of difference in reflectivity through image algorithms, thereby improving the integrity of the reconstruction.

[0064] 3. Multi-angle projection lighting can eliminate areas that can be photographed by the camera but cannot be illuminated by the projector.

[0065] Based on the above research and concepts, the applicant has proposed structured light imaging devices and three-dimensional imaging methods according to various embodiments of this application, which are described in detail below.

[0066] Please refer to Figure 2. Some embodiments of the structured light-based 3D imaging method include steps 10 to 13, which are described in detail below.

[0067] Step 10: Acquire the first polarization structured light image and the second polarization structured light image of the imaged object.

[0068] The first polarized structured light image is an image captured on the object being imaged when structured light projection is performed using first polarized light, and the second polarized structured light image is an image captured on the object being imaged when structured light projection is performed using second polarized light. The first and second polarized lights have different polarization directions, and both are incident on the surface of the object being imaged at the same incident angle. Preferably, the polarization directions of the first and second polarized lights are perpendicular.

[0069] This application does not limit the projection lighting device for structured light projection, as long as it can use two polarized lights with different polarization directions incident at the same angle. For example, the same projection lighting device can be used to perform structured light projection in the same orientation, with a polarizer with a different polarization direction added in front of the projection lens for each projection; or a projection lighting device with a switchable two-channel polarized light source can be used to perform two polarized light projections.

[0070] Step 11: Perform 3D reconstruction using the first polarized structured light image to obtain the first height image of the imaged object; perform 3D reconstruction using the second polarized structured light image to obtain the second height image of the imaged object.

[0071] The three-dimensional reconstruction here can use any three-dimensional reconstruction method, and this application does not limit it.

[0072] Step 12: Calculate the reflectivity difference map based on the first polarization structured light image and the second polarization structured light image.

[0073] The pixel value of each pixel in the reflectance difference map represents the degree of reflectance difference at that pixel location. The degree of reflectance difference indicates the difference between the reflectance of the surface of the imaged object corresponding to that pixel location for the first polarized light and the reflectance for the second polarized light.

[0074] Step 13: Fuse the first height image and the second height image according to the reflectance difference map to obtain the final height image of the imaged object.

[0075] Specifically, based on the reflectivity difference map, it is determined whether there is multiple reflection on the surface of the object being imaged corresponding to each pixel position. If so, the height value of that pixel position is set as an invalid pixel value. If not, the final height value of that pixel position is determined based on the height values ​​of the first height image and the second height image at that pixel position, so as to obtain the final height image of the object being imaged.

[0076] The applicant investigated fringe pattern projection in structured light projection. A general N-step cosine phase-shifting method was employed for 3D reconstruction. A standard N-step cosine fringe pattern was projected onto the object being imaged by controlling the projection illumination device. N fringe projection images were acquired using an imaging device, and the phase image was obtained through a phase resolution method. The height image was then reconstructed by combining the pre-calibrated geometric constraint parameters of the projection illumination device and the imaging device. Here, N is an integer not less than 2.

[0077] The N-step phase-shifting method can be divided into two parts: encoding and decoding. First, the stripe pattern is encoded according to certain rules and then sent to the projection lighting device for projection. The encoding formula for the N-step stripe projection pattern is as follows:

[0078] Where I is the ideal projection intensity, and I(x,y) represents the ideal projection intensity at pixel position (x,y). The phase of pixel position (x,y) is represented by A, the coded background light intensity and B, the coded modulated light intensity, both of which are ideal coded values, and k is the number of steps.

[0079] After the stripe pattern is projected onto the object, an image is formed on the object. Based on the N stripe projection images captured, decoding or phase decoding is performed to obtain the phase, decoded background light intensity, and decoded modulated light intensity. The decoding formula is as follows:

[0080] Where I' is the intensity (grayscale value) of the captured striped projection image, I'(x,y) represents the intensity at pixel position (x,y), A' is the intensity of the decoded background light, and B' is the intensity of the decoded modulated light.

[0081] Due to the influence of ambient light, differences in the incident angle of light rays, differences in optical lenses, nonlinear response of image sensors, and differences in the reflectivity of object surfaces, the relationship between the ideal projection intensity and the intensity of the actual captured stripe projection image can be simply expressed as: I'(x,y)=I(x,y)·F(x,y)·ξ(x,y)+μ(x,y), (3)

[0082] Where μ(x,y) represents the influence of ambient light on pixel position (x,y), i.e., the intensity of background light collected by the imaging device when the projection illumination device is off; F(x,y) represents the reflectivity of the surface of the object being imaged corresponding to pixel position (x,y); and ξ(x,y) represents the set of other nonlinear influencing factors. Substituting equations (1) and (2) into equation (3) and simplifying, we get:

[0083] From the above formula, we can approximate the following: A'=A·F(x,y)·ξ(x,y)+μ(x,y), B'=B·F(x,y)·ξ(x,y).

[0084] The above two equations demonstrate that the difference in the reflectivity of an object's surface can be reflected in two indicators: the intensity of the decoded background light and the intensity of the decoded modulated light.

[0085] During projection reconstruction using two polarized light sources with different polarization directions, the current of the two polarized light sources can be controlled to be kept consistent, thereby ensuring that the brightness of the polarized light sources in the two channels is strictly consistent. Furthermore, the two polarized light sources can be configured to share optical devices, and the image sensor can also be shared during the shooting process (for example, using the structured light imaging device shown in Figure 4). Therefore, the nonlinear influencing factors ξ(x,y) of the two projection reconstructions can be considered to be consistent, and the ambient background light μ(x,y) is also consistent.

[0086] Therefore, the first background light intensity image and the first modulated light intensity image obtained by projecting the fringe pattern using the first polarized light, and the second background light intensity image and the second modulated light intensity image obtained by projecting the fringe pattern using the second polarized light, can be expressed as: A1'(x,y)=A(x,y)·F1(x,y)·ξ(x,y)+μ(x,y), B1'(x,y)=B(x,y)·F1(x,y)·ξ(x,y), A'2(x,y)=A(x,y)·F2(x,y)·ξ(x,y)+μ(x,y), B'2(x,y)=B(x,y)·F2(x,y)·ξ(x,y),

[0087] Wherein, A1' represents the first background light intensity image, A1'(x,y) represents the first background light intensity at pixel position (x,y), A'2 represents the second background light intensity image, A'2(x,y) represents the second background light intensity at pixel position (x,y), B1' represents the first modulated light intensity image, B1'(x,y) represents the first modulated light intensity at pixel position (x,y), B'2 represents the second modulated light intensity image, B'2(x,y) represents the second modulated light intensity at pixel position (x,y).

[0088] By subtraction, we can obtain: ΔA'=A1'-A'2=[F1(x,y)-F2(x,y)]·A·ξ(x,y), ΔB'=B1'-B'2=[F1(x,y)-F2(x,y)]·B·ξ(x,y),

[0089] Wherein, ΔA' represents the difference between the first background light intensity image and the second background light intensity image, and ΔB' represents the difference between the first modulated light intensity image and the second modulated light intensity image.

[0090] Therefore, the reflectance difference between the two polarized light projection reconstructions can be quantified by the numerical difference between the decoded background light intensity or the decoded modulated light intensity of the two reconstructions. Thus, the reflectance difference map ΔF can be determined by the following formula: ΔF=ΔA'=A1'-A'2, (4)

[0091] Alternatively, ΔF = ΔB' = B1' - B'2, (5)

[0092] Although both of the above can represent reflectivity differences, when the object surface is semi-transparent, the contrast of the black and white stripe image is too low, resulting in insufficient decoding modulation light intensity to accurately reflect reflectivity. In this case, using the decoding background light intensity to represent reflectivity differences is more appropriate. However, when the object surface has a complex texture, the decoding background light intensity cannot accurately reflect reflectivity, the black and white stripe image has high contrast, and the decoding modulation light intensity is high. Therefore, using the decoding modulation light intensity to represent reflectivity differences is more appropriate. To improve system stability and applicability, this application preferably uses a linear weighted method to combine and represent reflectivity differences: ΔF=α·ΔA'+(1-α)·ΔB', (6)

[0093] Where α is a preset weight value, which is set by the user according to the usage scenario, and the value range is [0.0, 1.0].

[0094] When using fringe pattern projection for 3D reconstruction, the first polarized structured light image obtained includes N first fringe projection images obtained when the object being imaged is projected using the first polarized light in an N-step phase-shifting method, and the second polarized structured light image includes N second fringe projection images obtained when the object being imaged is projected using the second polarized light in an N-step phase-shifting method.

[0095] Based on the above analysis, in steps 11-12, three-dimensional reconstruction is performed using the first polarized structured light image to obtain the first height image of the imaged object, and three-dimensional reconstruction is performed using the second polarized structured light image to obtain the second height image of the imaged object. A reflectance difference map is calculated based on the first and second polarized structured light images, including:

[0096] Phase-de-phase processing is performed on N first fringe projection images to obtain a first phase image, a first background light intensity image, and a first modulated light intensity image; phase-de-phase processing is performed on N second fringe projection images to obtain a second phase image, a second background light intensity image, and a second modulated light intensity image; any phase-de-phase processing method can be used here, and this application does not limit it;

[0097] A first height image is obtained by performing three-dimensional reconstruction based on the first phase image; a second height image is obtained by performing three-dimensional reconstruction based on the second phase image; any three-dimensional reconstruction method can be used here, and this application does not limit it;

[0098] The reflectance difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, or based on the difference between the first modulated light intensity image and the second modulated light intensity image, or based on the difference between the first background light intensity image and the second background light intensity image, and the difference between the first modulated light intensity image and the second modulated light intensity image. For the specific formulas for determining the reflectance difference map, please refer to formulas (4), (5), and (6) above.

[0099] The final height image of the imaged object is essentially obtained by fusing the first height image and the second height image using a reflectance difference map. In some embodiments, the final height image of the imaged object obtained by fusing is determined by the following formula:

[0100] Where H(x,y) represents the height value at pixel position (x,y) in the final height image of the imaged object, H1(x,y) represents the height value at pixel position (x,y) in the first height image, H2(x,y) represents the height value at pixel position (x,y) in the second height image, ΔF(x,y) represents the degree of reflectance difference at pixel position (x,y) in the reflectance difference image, T is the preset threshold for the degree of reflectance difference, and h0 represents invalid pixel values.

[0101] Please refer to Figure 3. As can be seen, when facing a multi-reflective surface, the surface reconstructed by the traditional 3D reconstruction method (Figure (b)) is jagged and does not match reality. However, the method of this application can suppress multiple reflections and correctly reconstruct the multi-reflective surface (Figure (c)). This reduces the impact of multiple reflections on the object surface on the reconstruction accuracy and integrity, improves the integrity and realism of 3D reconstruction, and effectively enhances the material adaptability and scene adaptability of the 3D imaging system and method.

[0102] To address the challenges of structured light 3D reconstruction mentioned above, existing methods typically focus solely on improving or optimizing optical or algorithmic solutions. Some embodiments of this application also provide a highly practical, low-cost, and real-time structured light imaging device. This device is designed based on polarized light imaging technology, features a dual-channel polarized light source, and can easily implement the aforementioned 3D imaging method while also solving the technical problem of high reflectivity on object surfaces.

[0103] Please refer to Figure 4. In some embodiments, the structured light imaging device includes a projection illumination device 100 and an imaging device 200. The projection illumination device 100 adopts a two-channel illumination source design, including a first light source 101, a first polarizer 102, a second light source 103, a second polarizer 104, a light combining device 105, and a spatial light modulator 106. The imaging device 200 includes an analyzer 201 and an image sensor 202.

[0104] The first light source 101 and the second light source 103 can be LED light sources, etc. The first polarizer 102 is disposed in the optical path of the first beam emitted by the first light source 101 and is used to polarize the first beam into first polarized light with a first polarization direction; the second polarizer 104 is disposed in the optical path of the second beam emitted by the second light source 102 and is used to polarize the second beam into second polarized light with a second polarization direction; wherein, the first polarized light and the second polarized light can be linearly polarized light, and the first polarization direction is different from the second polarization direction.

[0105] The first light source 101 and the second light source 102 can be turned on and / or turned off simultaneously, or one can be turned on and the other off.

[0106] As shown in Figure 4, it may also include a control device 300, which is used to control the opening and closing of the first light source 101 and the second light source 102. The first light source 101 and the second light source 102 are controlled by the control device 300, and can be both turned on or only one can be turned on.

[0107] The light combining device 105 is located at the intersection of the optical paths of the first polarized light and the second polarized light. When both the first light source 101 and the second light source 102 are turned on, it combines the first polarized light and the second polarized light into the same optical path and outputs them to the spatial light modulator 106. When only the first light source 101 or only the second light source 102 is turned on, it outputs either the first polarized light or the second polarized light to the spatial light modulator 106. The light combining device 105 combines the first polarized light and the second polarized light into the same optical path, which on the one hand makes the optical path structure compact, thus reducing the size of the projection lighting device 100; on the other hand, it allows the first polarized light and the second polarized light to share optical components, receive the same optical processing, and be incident at the same angle.

[0108] As shown in Figure 4, in some embodiments, the optical paths of the first polarized light and the second polarized light are perpendicular. The light combining device 105 is a beam splitter (also called a semi-transparent and semi-reflective mirror). The beam splitter forms a 45° angle with both the optical paths of the first and second polarized light, and the first and second polarized light are incident on the beam splitter from opposite sides. One path of polarized light is transmitted through the beam splitter, while the other path is reflected, thus converging into the same optical path.

[0109] The spatial light modulator 106 modulates an incident light beam (e.g., the outgoing light from the light combiner 105) into a specific structured light beam, which is then projected onto the imaged object 400 to achieve structured light projection. In some embodiments, the structured light is used to project a fringe pattern, such as by using an N-step phase-shifting method. The imaging device 200 acquires N fringe projection images of the imaged object 400 for 3D reconstruction. The user can set the projection pattern according to actual needs and input it into the structured light imaging device to control the spatial light modulator 106 to modulate the light beam into structured light that projects the pattern. The spatial light modulator 106 can be a digital micromirror device (DMD), an LCD, or an LCOS, etc.

[0110] The object being imaged can be a product or workpiece from industrial production, a mechanical part, an electronic component, etc., and this application does not limit it.

[0111] The analyzer 201 receives the structure-reflected light formed by the structured light reflected from the object 400 being imaged, and after polarizing the structure-reflected light, projects it onto the image sensor 202 for imaging. The image sensor 202 can be a CMOS sensor or a CCD sensor, etc.

[0112] The structured light imaging device of this application uses polarized light for projection through the projection illumination device 100. Since the polarization state of the reflected light changes to some extent after reflection from the object surface, the analyzer 201 can filter out highly reflective and glare light in specific polarization directions during imaging. This improves the 3D reconstruction performance of highly reflective objects and enhances reconstruction integrity and material adaptability. Preferably, the polarization direction of the analyzer 201 is different from both the first and second polarization directions, so that highly reflective and glare light can be effectively filtered out when using both types of polarized light for projection.

[0113] Furthermore, considering that existing methods of adding an analyzer in front of the imaging lens to solve the high reflectivity problem still suffer from insufficient illumination because while the high reflectivity area is reduced or eliminated, other diffuse reflection areas are also reduced. In this application, a light combining device 105 is provided. The light combining device 105 can combine the first polarized light and the second polarized light into a single polarized light. The control device 300 can control the first light source 101 and the second light source 102 to work simultaneously, so that the two polarized lights can be used simultaneously to project onto the imaged object 400. Since the polarization direction of the first polarized light is different from that of the second polarized light, it is almost impossible for them to be completely filtered out by the analyzer 201 at the same time. This reduces the high reflectivity area and complements the illumination intensity of the diffuse reflection area, thereby improving the completeness of the final three-dimensional reconstruction.

[0114] Preferably, the first polarization direction is perpendicular to the second polarization direction. For example, the first polarizer 103 polarizes horizontally (referred to as S-polarization), and the second polarizer 104 polarizes vertically (referred to as P-polarization), which will be used as an example in the following description. Further, the polarization direction of the analyzer 201 is at a 45° angle to both the first and second polarization directions, as shown in Figure 5. The polarization angle of the analyzer 201 can, to a certain extent, eliminate the high reflectivity and glare caused by the first polarized light (S-polarized light) source illuminating the surface of the object, and can also eliminate the high reflectivity and glare caused by the second polarized light (P-polarized light) source illuminating the surface of the object. Meanwhile, even in extreme cases, when the first polarized light (S-polarized light) is reflected from the object surface, the polarization direction of the light changes, becoming completely perpendicular to the polarization direction of the analyzer 201 of the imaging device 200. The light cannot pass through the analyzer 201, and the first polarized light (S-polarized light) cannot form an image at all. However, when the second polarized light (P-polarized light) is reflected from the object surface, the reflected light is almost impossible to be perpendicular to the polarization direction of the analyzer 201, and can still pass through the analyzer 201, achieving illumination imaging. Under normal circumstances, the first polarized light (S-polarized light) and the second polarized light (P-polarized light) form complementary illumination in the diffuse reflection region. It should be noted that after the analyzer in this application is assembled during the system assembly stage, the polarization angle does not need to be rotated again during subsequent use.

[0115] Referring to Figure 6, in some embodiments, the projection illumination device 100 further includes a first collimating lens group 108 and a second collimating lens group 109. The first collimating lens group 108 is disposed between the first light source 101 and the first polarizer 103, and is used to collimate the first light beam and project it onto the first polarizer 103; the second collimating lens group 109 is disposed between the second light source 102 and the second polarizer 104, and is used to collimate the second light beam and project it onto the second polarizer 104.

[0116] Typically, a collimating lens placed behind the light source can collimate the light beam emitted by the source. Theoretically, a polarizer can achieve light polarization by being placed anywhere in the optical path of the projection lighting device 100. However, the transmittance varies depending on its placement, resulting in significant differences in the overall light transmission efficiency of the projection lighting device 100. As shown in Figure 7, if the incident angles are all categorized within the range of 0° to 90°, the transmittance of the polarizer decreases as the incident angle decreases, reaching its highest at 90°. Most currently disclosed methods place the polarizer in front of the projection lens, which is an FA lens. The light has a fixed field of view, and at the edges of the field of view, the incident angle is small, resulting in lower transmittance and affecting the final light transmission efficiency and uniformity. This application places the polarizer after the collimating lens group of the light source. The light passing through the collimating lens group ensures excellent collimation, with the incident light angles in each field of view approaching 90°. The transmittance of the light passing through the polarizer is also close to its maximum, greatly improving the light transmission efficiency and projection uniformity.

[0117] Please refer to Figures 4 and 6. In some embodiments, the projection lighting device 100 further includes a projection lens group 107 (also called a projection lens). The projection lens group 107 is disposed in the optical path of the structured light and is used to project the structured light onto the object 400 being imaged. The projection lens group 107 adopts a conventional FA design or a telecentric design.

[0118] Typically, the projection surface is not perpendicular to the principal optical axis of the projection lens 107, but rather forms an angle with it, such as 60°, meaning the principal optical axis of the projection lens 107 forms a 30° angle with the principal optical axis of the imaging device 200 (other angles are also possible, typically ranging from [27° to 30°]). To ensure the clarity of the projection surface, the spatial light modulator 106 is positioned at a certain angle in the optical path design, ensuring that the projection image plane of the spatial light modulator 106, the principal plane of the projection lens 107, and the projection surface satisfy Scherrer's Law. As shown in Figure 8, the extension lines of the projection image plane, the principal plane of the projection lens, and the projection surface intersect at a single point, ensuring that even if the projection lens 107 is installed at an angle, a clear projection pattern can still be obtained on the object surface.

[0119] Please refer to Figures 4 and 6. In some embodiments, the imaging device 200 also includes an imaging lens group 203 (also called an imaging lens). The analyzer 201 is disposed between the imaging lens group 203 and the image sensor 202. The imaging lens group 203 is used to receive the structured light reflected by the object being imaged 400 and project it onto the analyzer 201.

[0120] The imaging lens 203 preferably adopts an image-side telecentric design or a dual telecentric design. The analyzer 201 is placed between the imaging lens 203 and the image sensor 202, and the imaging lens 203 adopts an image-side telecentric lens or a dual telecentric lens. This ensures that the angle of incident light passing through the analyzer 201 is as close as possible to a 90° angle of incidence. This also ensures high light transmittance and imaging uniformity on the imaging side. Under the same illumination conditions, the exposure time of the image sensor 202 is shorter, and the structured light 3D imaging device has a faster shooting speed and higher real-time performance.

[0121] Referring to Figure 6, in some embodiments, the projection lighting device 100 further includes a reflector 110. The reflector 110 is disposed in the optical path of the light emitted from the light combining device 105 and is used to reflect the light emitted from the light combining device 105 to the spatial light modulator 106. The angle θ between the normal of the reflecting surface of the reflector 110 and the light emitted from the light combining device 105 is in the range of 0° < θ < 90°, thereby causing a reversal in the optical path of the light emitted from the light combining device 105.

[0122] Referring to Figure 6, in some embodiments, the projection lighting device 100 further includes a light homogenizing device 111. The light homogenizing device 111 is disposed on the optical path from the light combining device 105 to the reflector 110, and is used to homogenize the light emitted from the light combining device 105 before projecting it onto the reflector 110. The light homogenizing device 111 can be a double-sided microlens array. The arrangement of the light homogenizing device 111 makes the illumination more uniform.

[0123] Referring to Figure 6, in some embodiments, the projection illumination device 100 further includes a first relay lens 112. The first relay lens 112 is disposed on the optical path from the light emitted by the light homogenizing device 111 to the reflector 110, and is used to converge the light emitted by the light homogenizing device 111 before projecting it onto the reflector 110. Since the light beam is relatively divergent after passing through the light homogenizing device 111, in this embodiment, the first relay lens 112 is used to converge the light beam. For example, the light emitted by the light combining device 105 is cut into a spot by a double-sided microlens array and then converged by the first relay lens 112, making the light more concentrated.

[0124] Referring to Figure 6, in some embodiments, the projection lighting device 100 further includes a second relay lens 113. The second relay lens 113 is disposed on the optical path from the reflected light of the reflector 110 to the spatial light modulator 106, and is used to converge the reflected light of the reflector 110 before projecting it onto the spatial light modulator 106. Here, the light beam is converged again to improve the converging effect.

[0125] As can be seen from Figure 6, the reflector 110 bends the originally straight light path, so that the optical devices can be arranged more compactly in the projection lighting device 100.

[0126] Please refer to Figure 6. In some embodiments, the projection lighting device 100 also includes a composite prism 114. The composite prism 114 is disposed in front of the spatial light modulator 106 so that the light emitted from the second relay lens 113 is incident on the spatial light modulator 106 through the composite prism 114, and the structured light emitted from the spatial light modulator 106 is reflected onto the imaged object 400 through the composite prism 114.

[0127] The propagation path of light in the composite prism 114 is shown in Figure 6. In some embodiments, the positions of the spatial light modulator 106 and the composite prism 114 are configured in conjunction with the reflector 110 so that the direction of the structured light emitted by the composite prism 114 is opposite to the direction of the light emitted by the light combining device 105. This results in the overall optical path of the projection illumination device 100 being in a refracted form, making the structure of the projection illumination device 100 more compact and miniaturized. It should be noted that the projection image surface is not on the spatial light modulator 106 at this time, but is located on the reflection surface of the structured light in the composite prism 114.

[0128] To address the challenges of structured light 3D reconstruction mentioned above, existing methods typically focus solely on improving or optimizing optical or algorithmic solutions. This application, combined with the proposed structured light imaging device, provides a reconstruction algorithm that suppresses multiple reflections. It employs polarized light with different polarization directions incident on the object being imaged at the same incident angle for structured light projection. Since the reflectivity of the object surface varies, the intensity of the polarized light reflected from a certain point on the object surface entering the corresponding pixel position of the image sensor 220 differs, resulting in varying intensities of the projected images of the object. Multiple reflection regions are identified based on these differences in projection image intensity, correcting the reconstructed height image. This application analyzes the difference between multiple reflections and normal reflections from an optical perspective, combining polarization optics technology and image processing algorithms to solve the challenge of multiple reflections in 3D reconstruction of complex surface objects. Specifically, using polarized light with different polarization directions to project onto the object at the same incident angle can be achieved using the structured light imaging device of this application. The control device 300 controls the dual-channel illumination source to be lit sequentially and individually, thereby obtaining two polarized structured light images of the object with different polarization directions.

[0129] Therefore, under the control of the control device 300, based on the on / off state of the first light source 101 and the second light source 102, the projection lighting device 100 can realize at least three projection modes: single-polarization projection mode: a single-channel lighting source is lit, that is, only one of the first light source 101 and the second light source 102 is turned on; dual-polarization projection mode: both channels of lighting are lit simultaneously, that is, both the first light source 101 and the second light source 102 are turned on, and the first polarized light and the second polarized light are used for projection simultaneously. This mode can realize the complementarity of the dual polarized light sources; alternating projection mode: the dual channels of lighting are lit sequentially, that is, the projection lighting device 100 performs two structured light projections, in one where the first light source 101 is turned on and the second light source 102 is turned off, and the first polarized light is used for projection, and in the other where the first light source 101 is turned off and the second light source 102 is turned on, and the second polarized light is used for projection. This mode can realize the above-mentioned three-dimensional imaging method and suppress multiple reflections.

[0130] Based on the structured light imaging device of this application, the acquisition of the first polarized structured light image and the second polarized structured light image of the imaged object in step 10 above includes: controlling the projection illumination device 100 to project structured light onto the imaged object in an alternating projection mode. In the alternating projection mode, the projection illumination device performs two structured light projections. Each time the projection is performed, the imaging device 200 is triggered to take a picture of the imaged object, thereby obtaining the first polarized structured light image and the second polarized structured light image.

[0131] Users can input commands to select the projection mode, corresponding to single-polarization projection mode, dual-polarization projection mode, and alternating projection mode, which are respectively the first imaging command, the second imaging command, and the third imaging command.

[0132] In some embodiments, the three-dimensional imaging method of this application further includes:

[0133] When the first imaging command input by the user is received, the control device 300 controls the projection illumination device 100 to project structured light onto the object 400 in a single polarization projection mode, triggering the imaging device 200 to take a picture of the object 400, thereby obtaining a fifth polarization structured light image of the object 400. The fifth polarization structured light image is then used for three-dimensional reconstruction to obtain the height image of the object. Any three-dimensional reconstruction method can be used here, and this application does not limit it.

[0134] When the second imaging command input by the user is received, the projection lighting device 100 is controlled to project structured light onto the object to be imaged in a dual polarization projection mode, triggering the imaging device 200 to take a picture of the object to be imaged, thereby obtaining a sixth polarization structured light image of the object to be imaged, and using the sixth polarization structured light image to perform three-dimensional reconstruction, thereby obtaining a height image of the object to be imaged; the three-dimensional reconstruction here can adopt any three-dimensional reconstruction method, and this application does not limit it.

[0135] When a third imaging command is received from the user, steps 10 to 13 above are executed to obtain the height image of the object being imaged.

[0136] Users can choose the projection mode according to the needs of the actual scenario. When dealing with scenarios with high real-time requirements, they can choose the single polarization projection mode with only a single channel polarized light source enabled; when dealing with scenarios with multiple reflections, they can choose the alternating projection mode; when dealing with scenarios with high reflectivity and glare, they can choose the dual polarization projection mode with both channels polarized light sources enabled.

[0137] The following example uses S-polarized light as the first polarized light and P-polarized light as the second polarized light to illustrate the steps of using the structured light imaging device of this application to project a stripe pattern in an alternating projection mode, thus executing the three-dimensional imaging method of this application. As shown in Figure 9, the polarization directions of S-polarized light and P-polarized light are perpendicular. After passing through the light combining device 105, they propagate along the same optical path. For ease of illustration, Figure 9 shows both S-polarized light and P-polarized light simultaneously, but this does not mean that S-polarized light and P-polarized light must be turned on at the same time. Please refer to Figure 10 for the specific steps, which include:

[0138] 1. Turn on the first light source 101 (i.e., turn on the S-polarized light source) and turn off the second light source 102 (i.e., turn off the P-polarized light source) to project a stripe pattern. The image sensor 202 collects N first stripe projection images obtained by S-polarized light projection, performs phase de-phase processing, and obtains the first phase image, the first background light intensity image, and the first modulated light intensity image.

[0139] 2. Turn on the second light source 102 (i.e., turn on the P-polarized light source) and turn off the first light source 101 (i.e., turn off the S-polarized light source) to project the stripe pattern. The image sensor 202 collects N second stripe projection images obtained by P-polarized light projection, performs phase de-phase processing, and obtains the second phase image, the second background light intensity image, and the second modulation light intensity image.

[0140] 3. Using the pre-defined geometric constraints, perform three-dimensional reconstruction on the first phase image and the second phase image respectively to obtain the first height image and the second height image;

[0141] 4. Based on the difference between the first background light intensity image and the second background light intensity image, and the difference between the first modulation light intensity image and the second modulation light intensity image, calculate the difference in polarized light reflectance to obtain the reflectance difference map, see formula (6) for details;

[0142] 5. Based on the reflectivity difference map, threshold comparison is performed to determine the pixel areas with multiple reflections. The corresponding height values ​​are set as invalid pixel values ​​to remove abnormal data caused by multiple reflections. Finally, the height image with suppressed multiple reflections is obtained by fusion. See formula (7) for details.

[0143] Referring to Figure 11, in some embodiments, the structured light imaging device may also include a control device 300. The control device 300 may include an embedded control system, composed of an MCU (Microcontroller Unit), a SOC (System on Chips), a DSP (Digital Signal Processor), and / or an FPGA (Field Programmable Gate Array), primarily responsible for synchronization signal control, image processing, and data transmission functions. The three-dimensional imaging methods of the various embodiments of this application can be applied to the control device 300. The control device 300 can control the on / off state of the dual-channel light source and the pattern projected by the spatial light modulator 106. Users can send commands to the control device 300 via a host computer to control the structured light imaging device. The control device 300 activates the projection illumination device 100 and the imaging device 200 for projection and imaging based on the signals sent by the host computer. The imaging signal sent by the control device 300 to the imaging device 200 and the exposure time of the imaging device 200 need to be synchronized with the projection signal sent to the projection illumination device 100 and the projection time of the projection illumination device 100. After a series of structured light patterns are projected and photographed, the control device 300 acquires the image captured by the image sensor 200, starts the image algorithm for reconstruction and fusion, and finally obtains the height image of the imaged object 400, and then transmits the height image to the host computer.

[0144] Referring to Figures 11 and 12, in some embodiments, the control device 300 includes a projection control module for controlling the projection lighting device 100. The projection control module receives control signals from the control device 300 and controls the two-channel light sources (first light source 101 and second light source 102) and the spatial light modulator 106. When there are multiple projection lighting devices 100, each projection lighting device 100 corresponds to one projection control module for control, as shown in Figure 12, where one projection lighting device 100 is controlled by the first projection control module and the other by the second projection control module. Referring to Figure 13, in some embodiments, the projection control module includes a first light source control module, a second light source control module, and a light modulation control module. The first light source control module and the second light source control module respectively control the on / off enable and current magnitude of the two-channel light sources, changing the projection brightness of the projection lighting device 100 by adjusting the current magnitude of the light sources. The light modulation control module can control the pattern projected by the spatial light modulator 106.

[0145] To address the issue of blind spots in the field of view, some embodiments of the structured light imaging device in this application employ dual projection illumination devices to illuminate the object being imaged from two different incident angles, as shown in Figure 12. Based on this, another three-dimensional imaging method is proposed, which removes some blind spots through image fusion. Specifically, two projection illumination devices are symmetrically arranged around the imaging device. These two devices can illuminate the object being imaged from two different incident angles, thereby solving the reconstruction blind spot problem caused by illumination blind spots. Referring to Figure 14, when reconstructing using a single projection illumination device, area A in the figure is a camera blind spot; even if illumination is available, it cannot be captured, resulting in no reconstruction. Area B in the figure is an illumination blind spot; the camera can capture the image, but illumination is unavailable, leading to no reconstruction. This embodiment uses dual projection illumination devices for projection illumination. By performing two projection reconstructions using symmetrically placed projection illumination devices, followed by fusion, the illumination blind spot in area B in the figure can be removed.

[0146] Please refer to Figure 15. The three-dimensional imaging method based on this structured light imaging device includes steps 20 to 24.

[0147] Step 20: Control one of the two projection lighting devices to project structured light onto the object being imaged with first polarized light, trigger the imaging device to capture the image of the object being imaged, and obtain a first polarized structured light image; control the projection lighting device to project structured light onto the object being imaged with second polarized light, trigger the imaging device to capture the image of the object being imaged, and obtain a second polarized structured light image.

[0148] That is, one of the two projection lighting devices is controlled to perform structured light projection in an alternating projection mode, and the image is formed by the imaging device to obtain a first polarized structured light image and a second polarized structured light image.

[0149] Step 21: Use the first polarization structured light image and the second polarization structured light image to perform three-dimensional imaging to obtain the first projection direction height image.

[0150] The three-dimensional imaging here can be achieved according to the structured light-based three-dimensional imaging method in any of the preceding embodiments of this application.

[0151] Step 22: Control the other of the two projection lighting devices to project structured light onto the object being imaged with first polarized light, trigger the imaging device to capture the object being imaged, and obtain a third polarized structured light image; control the projection lighting device to project structured light onto the object being imaged with second polarized light, trigger the imaging device to capture the object being imaged, and obtain a fourth polarized structured light image.

[0152] That is, controlling one of the two projection lighting devices to perform structured light projection in an alternating projection mode, and then imaging it through an imaging device to obtain a third polarization structured light image and a fourth polarization structured light image.

[0153] Step 23: Use the third polarization structured light image and the fourth polarization structured light image to perform three-dimensional imaging to obtain the second projection direction height image.

[0154] The three-dimensional imaging here can be achieved according to the structured light-based three-dimensional imaging method in any of the preceding embodiments of this application.

[0155] Step 24: Fuse the height image in the first projection direction and the height image in the second projection direction to obtain the height image of the imaged object.

[0156] In some embodiments, the height image of the imaged object is determined by the following formula:

[0157] Among them, I h (x,y) represents the height value at pixel position (x,y) in the image of the object being imaged. 1d (x,y) represents the height value at pixel position (x,y) in the height image of the first projection direction, H 2d (x,y) represents the height value at pixel position (x,y) in the height image of the second projection direction.

[0158] Of course, the projection lighting device used in this embodiment is not limited to the projection lighting device shown in Figure 12, as long as the projection lighting device can perform structured light projection with first polarized light and second polarized light, and the polarization directions of the first polarized light and the second polarized light are different and the incident angles are the same. Similarly, other imaging devices can also be used, not limited to the imaging device shown in Figure 12.

[0159] In this embodiment, the structured light imaging device illuminates the object being imaged from two different incident angles by setting up two symmetrical projection illumination devices centered on the imaging device, thereby eliminating the illumination blind spots. Furthermore, it provides a way to fuse the height images obtained by projection reconstruction through the two projection illumination devices, further improving the reconstruction integrity.

[0160] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0161] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A three-dimensional imaging method based on structured light, characterized in that, include: Acquire a first polarized structured light image and a second polarized structured light image of the object being imaged; wherein, the first polarized structured light image is an image captured on the object being imaged when structured light projection is performed on the object using first polarized light, and the second polarized structured light image is an image captured on the object being imaged when structured light projection is performed on the object using second polarized light, wherein the polarization directions of the first polarized light and the second polarized light are different, and the first polarized light and the second polarized light are incident on the surface of the object being imaged at the same incident angle when structured light projection is performed on the object being imaged; A first height image of the imaged object is obtained by performing three-dimensional reconstruction using the first polarized structured light image; a second height image of the imaged object is obtained by performing three-dimensional reconstruction using the second polarized structured light image. A reflectance difference map is calculated based on the first polarized structured light image and the second polarized structured light image. The pixel value of each pixel position in the reflectance difference map represents the degree of reflectance difference at that pixel position. The degree of reflectance difference represents the degree of difference between the reflectance of the surface of the imaged object corresponding to that pixel position to the first polarized light and the reflectance to the second polarized light. Based on the reflectivity difference map, it is determined whether there is multiple reflection on the surface of the object being imaged corresponding to each pixel position. If so, the height value of the pixel position is set as an invalid pixel value. If not, the final height value of the pixel position is determined based on the height values ​​of the first height image and the second height image at that pixel position, so as to obtain the final height image of the object being imaged.

2. The three-dimensional imaging method as described in claim 1, characterized in that, The first polarized structured light image includes N first fringe projection images obtained by projecting a fringe pattern onto the imaged object using the first polarized light in an N-step phase-shifting method, and the second polarized structured light image includes N second fringe projection images obtained by projecting a fringe pattern onto the imaged object using the second polarized light in an N-step phase-shifting method, where N is an integer not less than 2. The process involves performing 3D reconstruction using the first polarized structured light image to obtain a first height image of the imaged object, performing 3D reconstruction using the second polarized structured light image to obtain a second height image of the imaged object, and calculating a reflectance difference map based on the first and second polarized structured light images, including: Phase-de-phase processing is performed on N first fringe projection images to obtain a first phase image, a first background light intensity image, and a first modulated light intensity image; phase-de-phase processing is performed on N second fringe projection images to obtain a second phase image, a second background light intensity image, and a second modulated light intensity image; Three-dimensional reconstruction is performed based on the first phase image to obtain the first height image; three-dimensional reconstruction is performed based on the second phase image to obtain the second height image; The reflectance difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, or the reflectance difference map is calculated based on the difference between the first modulated light intensity image and the second modulated light intensity image, or the reflectance difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, and the difference between the first modulated light intensity image and the second modulated light intensity image.

3. The three-dimensional imaging method as described in claim 2, characterized in that, The reflectance difference map is determined by the following formula: ΔF=ΔA′=A′1-A′2, Alternatively, ΔF = ΔB′ = B′1 - B′2, Or, ΔF=α·ΔA′+(1-α)·ΔB′; Wherein, ΔF represents the reflectance difference map, ΔA′ represents the difference between the first background light intensity image and the second background light intensity image, ΔB′ represents the difference between the first modulated light intensity image and the second modulated light intensity image, A′1 represents the first background light intensity image, A′2 represents the second background light intensity image, B′1 represents the first modulated light intensity image, B′2 represents the second modulated light intensity image, and α is a preset weight value.

4. The three-dimensional imaging method as described in claim 3, characterized in that, The value of α ranges from [0.0, 1.0].

5. The three-dimensional imaging method according to any one of claims 1 to 4, characterized in that, The final height image of the imaged object is determined by the following formula: Wherein, H(x,y) represents the height value at pixel position (x,y) in the final height image of the imaged object, H1(x,y) represents the height value at pixel position (x,y) in the first height image, H2(x,y) represents the height value at pixel position (x,y) in the second height image, ΔF(x,y) represents the degree of reflectance difference at pixel position (x,y) in the reflectance difference image, T is a preset threshold for the degree of reflectance difference, and h0 represents an invalid pixel value.

6. The three-dimensional imaging method as described in claim 1, characterized in that, The polarization directions of the first polarized light and the second polarized light are perpendicular.

7. The three-dimensional imaging method as described in claim 1, characterized in that, The three-dimensional imaging method is implemented based on a structured light imaging device, which includes a projection illumination device and an imaging device. The projection illumination device includes a first light source, a first polarizer, a second light source, a second polarizer, a light combining device, and a spatial light modulator. The imaging device includes an analyzer and an image sensor. The first polarizer is disposed in the optical path of the first beam emitted by the first light source, and is used to polarize the first beam into the first polarized light with the first polarization direction. The second polarizer is disposed in the optical path of the second beam emitted by the second light source, and is used to polarize the second beam into second polarized light with a second polarization direction; wherein the first polarization direction is different from the second polarization direction. The first light source and the second light source can be turned on and / or turned off simultaneously, or one can be turned on and the other off. The light combining device is located at the intersection of the optical path of the first polarized light and the optical path of the second polarized light. When both the first light source and the second light source are turned on, it combines the first polarized light and the second polarized light into the same optical path and emits them to the spatial light modulator. When only the first light source or only the second light source is turned on, it emits the first polarized light or the second polarized light to the spatial light modulator. The spatial light modulator is used to modulate the incident light beam into a specific structured light to project the structured light onto the imaged object; The analyzer is used to receive the structure-reflected light formed by the structure light being reflected by the object being imaged, and to perform polarization analysis on the structure-reflected light before projecting it onto the image sensor for imaging; The acquisition of the first polarization structured light image and the second polarization structured light image of the imaged object includes: The projection illumination device is controlled to project structured light onto the object being imaged in an alternating projection mode. In the alternating projection mode, the projection illumination device performs two structured light projections, one in which the first light source is turned on and the second light source is turned off, and the other in which the first light source is turned off and the second light source is turned on. Each time a projection occurs, the imaging device is triggered to take a picture of the object being imaged, thereby obtaining the first polarized structured light image and the second polarized structured light image.

8. A three-dimensional imaging method based on a structured light imaging device, characterized in that, The structured light imaging device includes an imaging unit and two projection illumination units, which are symmetrically arranged around the imaging unit. The projection illumination units are capable of projecting structured light using first polarized light and second polarized light, wherein the polarization directions of the first polarized light and the second polarized light are different. The three-dimensional imaging method includes: One of the two projection lighting devices is controlled to project structured light onto the object being imaged using the first polarized light, triggering the imaging device to capture the object being imaged, thereby obtaining a first polarized structured light image; the same projection lighting device is then controlled to project structured light onto the object being imaged using the second polarized light, triggering the imaging device to capture the object being imaged, thereby obtaining a second polarized structured light image. Using the first polarized structured light image and the second polarized structured light image, three-dimensional imaging is performed using the three-dimensional imaging method according to any one of claims 1 to 6 to obtain a first projection direction height image; Control the other of the two projection lighting devices to project structured light onto the object being imaged with the first polarized light, trigger the imaging device to capture the object being imaged, and obtain a third polarized structured light image; control the projection lighting device to project structured light onto the object being imaged with the second polarized light, trigger the imaging device to capture the object being imaged, and obtain a fourth polarized structured light image. Using the third polarization structured light image and the fourth polarization structured light image, three-dimensional imaging is performed by the three-dimensional imaging method according to any one of claims 1 to 6 to obtain a second projection direction height image; The height image of the first projection direction and the height image of the second projection direction are fused together to obtain the height image of the imaged object.

9. The three-dimensional imaging method as described in claim 8, characterized in that, The height image of the imaged object is determined by the following formula: Among them, I h (x,y) represents the height value at pixel position (x,y) in the height image of the imaged object, H 1d (x,y) represents the height value at pixel position (x,y) in the first projection direction height image, H 2d (x,y) represents the height value at pixel position (x,y) in the second projection direction height image, and h0 represents an invalid pixel value.

10. A computer-readable storage medium, characterized in that, The medium stores a program that can be executed by a processor to implement the three-dimensional imaging method as described in any one of claims 1 to 9.

11. A structured light imaging device, characterized in that, Includes projection lighting devices, imaging devices, and control devices; The projection lighting device includes a first light source, a first polarizer, a second light source, a second polarizer, a light combining device, and a spatial light modulator; The first polarizer is disposed in the optical path of the first beam emitted by the first light source, and is used to polarize the first beam into first polarized light with a first polarization direction. The second polarizer is disposed in the optical path of the second beam emitted by the second light source, and is used to polarize the second beam into second polarized light with a second polarization direction; wherein the first polarization direction is different from the second polarization direction. The control device is used to control the opening and closing of the first light source and the second light source; The light combining device is located at the intersection of the optical paths of the first polarized light and the second polarized light, and is used to make the first polarized light and the second polarized light incident at the same angle. When both the first light source and the second light source are turned on, the light combining device combines the first polarized light and the second polarized light into the same optical path and emits them to the spatial light modulator. When only the first light source or only the second light source is turned on, the first polarized light or the second polarized light is emitted to the spatial light modulator. The spatial light modulator is used to modulate the incident light beam into a specific structured light to project the structured light onto the object being imaged; The imaging device includes an analyzer and an image sensor; The analyzer is used to receive the structured reflected light formed by the structured light reflected from the object being imaged, and to analyze the polarization of the structured reflected light before projecting it onto the image sensor for imaging, so as to obtain a polarized structured light image.

12. The structured light imaging device as described in claim 11, characterized in that, The polarization direction of the analyzer is different from both the first polarization direction and the second polarization direction.

13. The structured light imaging device as described in claim 11, characterized in that, The first polarization direction is perpendicular to the second polarization direction.

14. The structured light imaging device as described in claim 13, characterized in that, The polarization direction of the analyzer is at a 45° angle to both the first polarization direction and the second polarization direction.

15. The structured light imaging device as described in claim 11, characterized in that, The projection lighting device further includes a first collimating lens group and a second collimating lens group; The first collimating lens group is disposed between the first light source and the first polarizer, and is used to collimate the first beam and project it onto the first polarizer; The second collimating lens group is disposed between the second light source and the second polarizer, and is used to collimate the second beam and project it onto the second polarizer.

16. The structured light imaging device as described in claim 11, characterized in that, The optical paths of the first polarized light and the second polarized light are perpendicular, and the light combining device is a beam splitter; The beam splitter is at a 45° angle to both the optical path of the first polarized light and the optical path of the second polarized light, and the first polarized light and the second polarized light are incident on the beam splitter from both sides of the beam splitter.

17. The structured light imaging device as described in claim 11, characterized in that, The projection lighting device further includes a reflector, which is disposed in the optical path of the light output of the light combining device and is used to reflect the light output of the light combining device to the spatial light modulator; the angle θ between the normal of the reflector surface and the light output of the light combining device is in the range of 0° < θ < 90°.

18. The structured light imaging device as described in claim 17, characterized in that, The projection lighting device further includes a light homogenizing device, which is disposed on the optical path from the light combining device to the reflector, and is used to homogenize the light emitted from the light combining device before projecting it onto the reflector.

19. The structured light imaging device as described in claim 18, characterized in that, The projection lighting device further includes a first relay lens, which is disposed on the optical path from the light output of the light homogenizing device to the reflector, and is used to converge the light output of the light homogenizing device and project it onto the reflector.

20. The structured light imaging device as described in claim 18 or 19, characterized in that, The projection lighting device further includes a second relay lens, which is disposed on the optical path from the reflected light of the reflector to the spatial light modulator, and is used to converge the reflected light of the reflector and project it onto the spatial light modulator.

21. The structured light imaging device as described in claim 20, characterized in that, The projection lighting device further includes a composite prism, which is disposed in front of the spatial light modulator so that the light emitted from the second relay lens is incident on the spatial light modulator through the composite prism, and the structured light emitted from the spatial light modulator is reflected onto the object being imaged by the composite prism.

22. The structured light imaging device as described in claim 11, characterized in that, The projection lighting device further includes a projection lens group, which is disposed in the optical path of the structured light and is used to project the structured light onto the object being imaged. The projection image plane of the spatial light modulator, the principal plane of the projection lens group, and the projection object plane satisfy Schahm's law.

23. The structured light imaging device as described in claim 11, characterized in that, The imaging device further includes an imaging lens group, and the analyzer is disposed between the imaging lens group and the image sensor; The imaging lens group is used to receive the structured light reflected by the object being imaged, and to converge and project it onto the analyzer.

24. The structured light imaging device as described in claim 23, characterized in that, The imaging lens group is an image-side telecentric lens or a double telecentric lens.

25. The structured light imaging device as described in claim 11, characterized in that, The control device is used for: When the first imaging command input by the user is received, the projection lighting device is controlled to project structured light onto the object to be imaged in a single polarization projection mode, triggering the imaging device to take a picture of the object to be imaged, thereby obtaining a first polarization structured light image of the object to be imaged, and using the first polarization structured light image to perform three-dimensional reconstruction to obtain a height image of the object to be imaged; in the single polarization projection mode, only one of the first light source and the second light source is turned on. When a second imaging command is received from the user, the projection lighting device is controlled to project structured light onto the object being imaged in a dual-polarization projection mode, triggering the imaging device to capture an image of the object being imaged, thereby obtaining a second-polarization structured light image of the object being imaged. The second-polarization structured light image is then used for three-dimensional reconstruction to obtain a height image of the object being imaged. In the dual-polarization projection mode, both the first light source and the second light source are turned on. When a third imaging command is received from the user, the projection illumination device is controlled to project structured light onto the object being imaged in an alternating projection mode. In this alternating projection mode, the projection illumination device performs two structured light projections: once with the first light source on and the second light source off, and again with the first light source off and the second light source on. Each projection triggers the imaging device to capture an image of the object being imaged, resulting in a third-polarization structured light image and a fourth-polarization structured light image of the object. The third-polarization structured light image and the fourth-polarization structured light image are then used to perform three-dimensional reconstruction to obtain the height image of the object being imaged. The step of using the third polarization structured light image and the fourth polarization structured light image to perform three-dimensional reconstruction to obtain the height image of the imaged object includes: The first height image of the imaged object is obtained by performing three-dimensional reconstruction using the third polarization structured light image; the second height image of the imaged object is obtained by performing three-dimensional reconstruction using the fourth polarization structured light image. A reflectance difference map is calculated based on the third polarization structured light image and the fourth polarization structured light image. The pixel value of each pixel position in the reflectance difference map represents the degree of reflectance difference at that pixel position. The degree of reflectance difference represents the degree of difference between the reflectance of the surface of the imaged object corresponding to that pixel position to the first polarized light and the reflectance to the second polarized light. Based on the reflectivity difference map, it is determined whether there is multiple reflection on the surface of the object being imaged corresponding to each pixel position. If so, the height value of the pixel position is set as an invalid pixel value. If not, the final height value of the pixel position is determined based on the height values ​​of the first height image and the second height image at that pixel position, so as to obtain the height image of the object being imaged.

26. The structured light imaging device as described in claim 25, characterized in that, The step of controlling the projection illumination device to project structured light onto the object being imaged in an alternating projection mode includes: controlling the projection illumination device to project a stripe pattern onto the object being imaged in an alternating projection mode using an N-step phase-shifting method; where N is an integer not less than 2. The third polarization structured light image includes N first fringe projection images obtained when the first polarized light is used to project a fringe pattern onto the imaged object using an N-step phase-shifting method; the fourth polarization structured light image includes N second fringe projection images obtained when the second polarized light is used to project a fringe pattern onto the imaged object using an N-step phase-shifting method. The process involves using the third polarization structured light image for 3D reconstruction to obtain a first height image of the imaged object, using the fourth polarization structured light image for 3D reconstruction to obtain a second height image of the imaged object, and calculating a reflectance difference map based on the third and fourth polarization structured light images, including: Phase-de-phase processing is performed on N first fringe projection images to obtain a first phase image, a first background light intensity image, and a first modulated light intensity image; phase-de-phase processing is performed on N second fringe projection images to obtain a second phase image, a second background light intensity image, and a second modulated light intensity image; Three-dimensional reconstruction is performed based on the first phase image to obtain the first height image; three-dimensional reconstruction is performed based on the second phase image to obtain the second height image; The reflectance difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, or the reflectance difference map is calculated based on the difference between the first modulated light intensity image and the second modulated light intensity image, or the reflectance difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, and the difference between the first modulated light intensity image and the second modulated light intensity image.

27. The structured light imaging device as described in claim 26, characterized in that, The reflectance difference map is determined by the following formula: ΔF=ΔA′=A′1-A′2, Alternatively, ΔF = ΔB′ = B′1 - B′2, Or, ΔF=α·ΔA′+(1-α)·ΔB′; Wherein, ΔF represents the reflectance difference map, ΔA′ represents the difference between the first background light intensity image and the second background light intensity image, ΔB′ represents the difference between the first modulated light intensity image and the second modulated light intensity image, A′1 represents the first background light intensity image, A′2 represents the second background light intensity image, B′1 represents the first modulated light intensity image, B′2 represents the second modulated light intensity image, and α is a preset weight value.

28. The structured light imaging device as described in claim 25, characterized in that, It includes two of the aforementioned projection lighting devices, and the two projection lighting devices are symmetrically arranged with the imaging device as the center; The control device is also used to: when receiving an imaging command input by the user: Control one of the two projection lighting devices to perform structured light projection on the object being imaged in a projection mode corresponding to the imaging command, so as to perform three-dimensional reconstruction and obtain a first projection direction height image; Control the other of the two projection lighting devices to perform structured light projection on the object being imaged in a projection mode corresponding to the imaging command, so as to perform three-dimensional reconstruction and obtain a second projection direction height image; The height image of the first projection direction and the height image of the second projection direction are fused together to obtain the height image of the imaged object.

29. The structured light imaging device as described in claim 11, characterized in that, It includes at least one projection lighting combination, each projection lighting combination including two projection lighting devices, and the two projection lighting devices in each projection lighting combination are symmetrically arranged with respect to the imaging device.

Images