Structured light imaging equipment

Abstract

Structured light imaging equipment comprises a projection lighting device, an imaging device and a control device, the projection lighting device comprises a first light source, a first polarizer, a second light source, a second polarizer, a light combining device and a spatial light modulator, the first light source and the first polarizer can generate first polarized light, and the second light source and the second polarizer can generate second polarized light. The light combining device combines the first polarized light or the second polarized light or combines the first polarized light and the second polarized light into one path of polarized light and emits the polarized light to the spatial light modulator, and the spatial light modulator modulates light beams entering the spatial light modulator into specific structured light so as to project the structured light to an imaged object; the imaging device comprises a polarization analyzer and an image sensor. The polarization analyzer carries out polarization analysis on structure reflected light reflected by an imaged object and then projects the structure reflected light to the image sensor for imaging. The control device controls on and off of the first light source and the second light source. The structured light imaging equipment can restrain high light reflection and improve the precision and integrity of three-dimensional reconstruction of a high-light-reflection object.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a structured light imaging device. Background Art

[0002] With the increasing application of 3D detection technology in the field of industrial automation, the use of 3D reconstruction technology based on structured light 3D cameras, which is based on the principle of triangulation, is becoming more and more popular. Structured light cameras often use a projector combined with a camera, with the projector and the camera's main optical axis at a certain angle. The projector projects structured light on the object, and the camera captures the structured light image of the object, which is then reconstructed in 3D to obtain the height image of the object in order to reconstruct the surface morphology of the object.

[0003] The structured light 3D camera is a diffuse reflection imaging system. However, the materials on the surface of some objects will cause specular reflection on the surface of the objects, which has high reflective properties. As a result, the images collected by the camera are overexposed or too dark, making it difficult to solve and reconstruct the actual surface morphology of the object. Summary of the invention

[0004] The present invention mainly solves the technical problem that it is difficult to effectively perform three-dimensional reconstruction on highly reflective objects.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a structured light imaging device, including a projection lighting device, an imaging device and a control device;

[0006] The projection lighting device comprises a first light source, a first polarizer, a second light source, a second polarizer, a light combining device and a spatial light modulator;

[0007] The first polarizer is disposed on the light path of the first light beam emitted by the first light source, and is used to polarize the first light beam into a first polarized light in a first polarization direction;

[0008] The second polarizer is disposed on the optical path of the second light beam emitted by the second light source, and is used to polarize the second light beam into a second polarized light with a second polarization direction; wherein the first polarization direction is different from the second polarization direction;

[0009] The control device is used to control the turning on and off of the first light source and the second light source;

[0010] The light combining device is disposed at the intersection of the optical path of the first polarized light and the optical path of the second polarized light, and is used to combine the first polarized light and the second polarized light into the same optical path and emit them to the spatial light modulator when both the first light source and the second light source are turned on, and to emit the first polarized light or the second polarized light to the spatial light modulator when only the first light source is turned on or only the second light source is turned on;

[0011] The spatial light modulator is used to modulate the light beam incident thereinto into a specific structured light, so as to project the structured light toward the imaged object;

[0012] The imaging device comprises a polarizer and an image sensor;

[0013] The polarizer is used to receive structured reflected light formed by the structured light reflected by the imaging object, analyze the structured reflected light, and then project it onto the image sensor for imaging.

[0014] The structured light imaging device according to the above embodiment includes a projection lighting device, an imaging device and a control device. The projection lighting device includes a first light source, a first polarizer, a second light source, a second polarizer, a light combining device and a spatial light modulator. The first light source and the first polarizer can generate a first polarized light, and the second light source and the second polarizer can generate a second polarized light, so that the projection lighting device can perform structured light projection with one or two polarized lights; the imaging device is provided with a polarizer, which can analyze the structured reflected light reflected by the imaging object. Since the projection lighting device can use polarized light for projection, and the polarization state of the reflected light will change to a certain extent after the polarized light is reflected by the surface of the object, the polarizer can filter out high reflection and glare light in a specific polarization direction during imaging, thereby improving the three-dimensional reconstruction performance of the highly reflective object. Moreover, the light combining device can combine the first polarized light and the second polarized light into one polarized light and project them simultaneously. Since the polarization direction of the first polarized light is different from that of the second polarized light, it is almost impossible for them to be completely filtered out by the analyzer at the same time. When one of the polarized light is filtered out, the other polarized light can play a complementary role, thereby suppressing high reflection and ensuring the lighting brightness. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of multiple reflections of light on the surface of an object;

[0016] Figure 2 A schematic diagram of the structure of a structured light imaging device according to an embodiment;

[0017] Figure 3 A schematic diagram of the polarization directions of the first polarizer and the second polarizer and the analyzing direction of the analyzer in an embodiment;

[0018] Figure 4 A schematic diagram of the structure of a structured light imaging device according to another embodiment;

[0019] Figure 5 Schematic diagram of the relationship between the light transmittance of the polarizer and the incident angle of the light;

[0020] Figure 6A schematic diagram of the geometric relationship between the projection image plane of the spatial light modulator, the principal plane of the projection lens, and the projection object plane;

[0021] Figure 7 A schematic diagram of the structure of a structured light imaging device according to another embodiment;

[0022] Figure 8 is a structural schematic diagram of a projection control module according to an embodiment;

[0023] Fig. 9 Schematic diagram of the optical path of S-polarized light and P-polarized light after passing through the light combining device;

[0024] Fig.10 A schematic diagram of the process of using S-polarized light and P-polarized light to perform three-dimensional reconstruction with multiple reflection suppression;

[0025] Fig.11 A comparison diagram of the effects of the reconstruction method for suppressing multiple reflections proposed in this application and the traditional reconstruction method;

[0026] Fig.12 Schematic diagram of the blind area of ​​3D reconstruction.

[0027] Reference numerals:

[0028] 100, projection lighting device; 101, first light source; 102, second light source; 103, first polarizer; 104, second polarizer; 105, light combining device; 106, spatial light modulator; 107, projection lens group; 108, first collimating lens group; 109, second collimating lens group; 110, reflector; 111, light homogenizing device; 112, first relay lens; 113, second relay lens; 114, compound prism; 200, imaging device; 201, analyzer; 202, image sensor; 203, imaging lens group; 300, control device; 400, imaged object. DETAILED DESCRIPTION

[0029] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0030] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0031] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0032] The existing 3D reconstruction methods based on structured light still have the following problems when dealing with complex surfaces (high reflectivity, multiple reflections, etc.):

[0033] 1. In the structured light 3D reconstruction system, the imaging unit performs 3D reconstruction by collecting diffuse reflection images of the object surface. It uses the geometric constraints of reflected light. Its essence is that the irradiation light and the diffuse reflection light form a certain angle, forming a triangular constraint. When the surface morphology of the object is complex, the illumination light is incident on the surface of the object, and multiple reflections are formed after passing through the complex surface. The image collected by the imaging unit is not the diffuse reflection image of the actual object surface, but a diffuse reflection image containing multiple surface information. Even if the reconstruction can be solved, the obtained morphological information cannot represent the real surface information of the object. Figure 1 As shown, the incident light and the normal diffuse reflection light form a triangular constraint, but the geometric relationship between the multiple reflection light and the incident light is uncertain and does not satisfy the triangular constraint. At this time, the geometric constraint of 3D reconstruction does not hold, resulting in incomplete and unrealistic reconstructed data.

[0034] 2. Due to the angle, the structured light 3D camera will have two blind spots: first, the area that the projector can illuminate but the camera cannot capture; second, the area that the camera can capture but the projector cannot illuminate; these blind spots will cause incomplete reconstruction data.

[0035] In response to the difficulties of the above-mentioned structured light-based 3D reconstruction methods, existing methods are often optimized by adopting high-cost cameras with low practicality or multiple exposure methods that are not real-time, in order to improve the reconstruction integrity of structured light 3D reconstruction.

[0036] Patent application CN202310024635 uses a four-directional polarization camera combined with projected polarized light to solve the high reflection problem. The four sets of polarization images of the polarization camera are fused to reconstruct the object shape information, thereby improving the robustness and reconstruction integrity of the system. Although the four-directional polarization camera can provide reconstruction data in four polarization directions, the camera resolution is sacrificed. The system resolution is only one-fourth of that of an ordinary camera, and the camera cost is relatively high.

[0037] Patent application CN202210831038 uses a modified error energy function to remove abnormal invalid points. The error energy function combines the wrapped phase of the fringe image, the background light intensity, and the modulated light intensity related information. This patent application removes abnormal points from a purely algorithmic perspective and does not solve the problem of high reflection and multiple reflections from an optical perspective.

[0038] Patent application CN202110372335 uses multiple shooting methods to fuse reconstruction data under different projection intensities to solve the problem of high reflection. The image pixel features are used to estimate the optimal projection brightness, and the corresponding number of stripe patterns are projected according to the number, and the stripe image of the object is captured. The stripe images of each group are fused to obtain a high-dynamic stripe image, and then more complete data is reconstructed. This solution trades time for integrity, is not real-time, and does not solve the problems of high reflection and multiple reflections from an optical perspective.

[0039] Patent application CN202110369458 uses a liquid crystal display to project linearly polarized light, and places a polarizer in front of the imaging lens. When the camera is shooting, the angle of the polarizer in front of the camera is rotated four times, the corresponding coded image is collected, the Stokes parameters are calculated based on the four images, and the functional relationship between the linear polarization degree and the incident angle is determined. Finally, the normal information of each point on the surface of the object is calculated according to the incident angle constraint condition to obtain the morphological information of the surface of the object. This solution uses the polarization characteristics of light to solve the problem of mirror reflection, but it needs to mechanically rotate the polarizer in front of the lens and ensure the accuracy of the polarization angle, so it is not very practical.

[0040] In addition, the method of using polarizers to solve the high reflection problem in structured light 3D reconstruction has been disclosed. Although it can reduce or eliminate the high reflection phenomenon in certain areas of the image according to the difference between the polarization direction of the high reflection light and the imaging polarization direction, it also reduces the illumination intensity of other normal diffuse reflection areas, causing insufficient lighting and the image is too dark to be completely reconstructed. This trade-off has caused new problems and has not improved the practicality of structured light 3D reconstruction.

[0041] The present application provides a highly practical, low-cost, and highly real-time structured light imaging device, which is mainly aimed at the technical problem that existing structured light 3D cameras and three-dimensional reconstruction methods are unable to effectively solve the technical problem that multiple reflections on the surface of objects, high reflectivity, and blind spots in the field of view make it impossible to reconstruct complete data. Multiple embodiments are provided to solve one or more of the above-mentioned technical problems respectively.

[0042] In some embodiments, an algorithm for suppressing multiple reflections is proposed, which combines polarization optical technology with the algorithm for suppressing multiple reflections. Starting from the multi-dimensional aspects of optical principles and algorithm principles, the characteristics of high reflection and abnormal reflection are analyzed from the optical essence, and a technical solution that conforms to theoretical analysis is proposed, which solves the difficult problems of poor reconstruction integrity and poor material adaptability caused by multiple reflections on the surface of objects, high reflections and blind spots in the field of view in structured light three-dimensional reconstruction.

[0043] The concepts of this application include:

[0044] 1. When polarized light, such as linear polarized light, is incident on the surface of an object and reflected, the polarization state of the reflected light will change to a certain extent. The analyzer can filter out high reflection and glare light in a specific polarization direction;

[0045] 2. When linear polarized light of different polarization directions is incident at the same angle, the reflectivity of the surface of the object is different (the reflection intensity is different). The applicant realizes that the reflection intensity is quite different after multiple surface reflections. This can be used to estimate the difference in reflectivity by means of image algorithms to determine whether multiple reflections have occurred, thereby improving the reconstruction integrity;

[0046] 3. Multi-angle projection lighting can eliminate areas that can be photographed by the camera but cannot be illuminated by the projector.

[0047] Based on the above concept, the applicant has proposed the structured light imaging device of each embodiment of the present application, which is described in detail below. Figure 2 In some embodiments, the structured light imaging device includes a projection lighting device 100, an imaging device 200, and a control device 300. The projection lighting device 100 adopts a two-channel illumination light source design, including a first light source 101, a first polarizer 102, a second light source 103, a second polarizer 104, a light combining device 105, and a spatial light modulator 106; the imaging device 200 includes an analyzer 201 and an image sensor 202.

[0048] The first light source 101 and the second light source 103 may be LED light sources, etc. The first polarizer 102 is disposed on the optical path of the first light beam emitted by the first light source 101, and is used to polarize the first light beam into a first polarized light in a first polarization direction; the second polarizer 104 is disposed on the optical path of the second light beam emitted by the second light source 102, and is used to polarize the second light beam into a second polarized light in a second polarization direction; wherein the first polarized light and the second polarized light may be linearly polarized light, and the first polarization direction is different from the second polarization direction.

[0049] The control device 300 is used to control the on and off of the first light source 101 and the second light source 102. The first light source 101 and the second light source 102 are controlled by the control device 300, and can be both turned on or only one of them can be turned on.

[0050] The light combining device 105 is disposed at the intersection of the optical path of the first polarized light and the optical path of the second polarized light, and is used to combine the first polarized light and the second polarized light into the same optical path and emit the light to the spatial light modulator 106 when both the first light source 101 and the second light source 102 are turned on, and emit the first polarized light or the second polarized light to the spatial light modulator 106 when only the first light source 101 or only the second light source 102 is turned on. The light combining device 105 combines the first polarized light and the second polarized light into the same optical path, which makes the optical path structure compact, which is beneficial to reducing the volume of the projection lighting device 100, and allows the first polarized light and the second polarized light to share optical devices, receive the same optical processing, and be incident at the same angle.

[0051] like Figure 2 As shown, in some embodiments, the optical path of the first polarized light is perpendicular to the optical path of the second polarized light, and the light combining device 105 is a beam splitter (also called a semi-transparent and semi-reflective mirror), the beam splitter and the optical path of the first polarized light and the optical path of the second polarized light are at a 45° angle, and the first polarized light and the second polarized light are incident on the beam splitter from both sides of the beam splitter, respectively. One polarized light is transmitted through the beam splitter, and the other polarized light is reflected, so as to be combined into the same optical path.

[0052] The spatial light modulator 106 is used to modulate the light beam incident therein (e.g., the outgoing light of the light combining device 105) into a specific structured light, so as to project the structured light onto the imaged object 400, thereby realizing structured light projection. In some embodiments, the structured light is used to project a stripe pattern, for example, the stripe pattern is projected according to the N-step phase shift method, and the imaging device 200 obtains N stripe projection images of the imaged object 400, and transmits them to the control device 300 for three-dimensional reconstruction. The control device 300 is also used to receive a projection pattern set by a user, and control the spatial light modulator 106 to modulate the light beam into a structured light that projects the pattern. The spatial light modulator 106 can be a digital micromirror device DMD, LCD, or LCOS, etc.

[0053] The imaged object may be an industrial product or workpiece, a mechanical part, an electronic component, etc., and this application does not limit this.

[0054] The analyzer 201 is used to receive structured reflected light formed by the structured light reflected by the imaging object 400, analyze the structured reflected light, and then project it onto the image sensor 202 for imaging. The image sensor 202 may be a CMOS sensor or a CCD sensor.

[0055] In the structured light imaging device of the present application, since the projection lighting device 100 uses polarized light for projection, and the polarization state of the reflected light changes to a certain extent after the polarized light is reflected by the surface of the object, the analyzer 201 can filter out high reflection and glare light in a specific polarization direction during imaging, thereby improving the three-dimensional reconstruction performance of highly reflective objects, and enhancing the reconstruction integrity and material adaptability. Preferably, the polarization direction of the analyzer 201 is different from both the first polarization direction and the second polarization direction, so that high reflection and glare light can be better filtered out when two polarized lights are used for projection.

[0056] In addition, considering that in the existing method of adding a polarizer in front of the imaging lens to solve the high reflection problem, there is still a problem that the high reflection area is weakened and eliminated, but other diffuse reflection areas are also weakened and insufficient light occurs, in the present application, a light combining device 105 is provided, and the light combining device 105 can combine the first polarized light and the second polarized light into one polarized light. The control device 300 can control the first light source 101 and the second light source 102 to work simultaneously, so that the two polarized lights can be used to simultaneously project the imaged object 400. Since the polarization direction of the first polarized light is different from the polarization direction of the second polarized light, it is almost impossible for them to be completely filtered out by the polarizer 201 at the same time. This not only weakens the high reflection area, but also complements the light intensity of the diffuse reflection area, thereby improving the final three-dimensional reconstruction integrity.

[0057] Preferably, the first polarization direction is perpendicular to the second polarization direction. For example, the first polarizer 103 is polarized in the horizontal direction (abbreviated as S polarization), and the second polarizer 104 is polarized in the vertical direction (abbreviated as P polarization). This is used as an example for explanation below. Further, the polarization direction of the analyzer 201 is at a 45° angle to both the first polarization direction and the second polarization direction. Figure 3As shown. The analysis angle of the analyzer 201 can eliminate the high reflection and glare caused by the first polarized light (S polarized light) light source irradiating the surface of the object to a certain extent, and can also eliminate the high reflection and glare caused by the second polarized light (P polarized light) light source irradiating the surface of the object. At the same time, even in extreme cases, when the first polarized light (S polarized light) light source irradiates the surface of the object and reflects, the polarization direction of the light changes, which is completely perpendicular to the analysis direction of the analyzer 201 of the imaging device 200. The light cannot pass through the analyzer 201, and the first polarized light (S polarized light) cannot be imaged at all. However, after the second polarized light (P polarized light) irradiates the surface of the object and reflects, it is almost impossible for the reflected light to be perpendicular to the analysis direction of the analyzer 201, and it can still pass through the analyzer 201 to achieve illumination imaging. Under normal circumstances, the first polarized light (S polarized light) light source and the second polarized light (P polarized light) light source form complementary illumination in the diffuse reflection area. It should be noted that after the analyzer in the present application is assembled in the system assembly stage, there is no need to rotate the analysis angle again during subsequent use.

[0058] Please refer to Figure 4 In some embodiments, the projection lighting device 100 further includes a first collimating lens group 108 and a second collimating lens group 109. The first collimating lens group 108 is disposed between the first light source 101 and the first polarizer 103, and is used to collimate the first light beam and project it onto the first polarizer 103; the second collimating lens group 109 is disposed between the second light source 102 and the second polarizer 104, and is used to collimate the second light beam and project it onto the second polarizer 104.

[0059] Generally, a collimating lens is arranged behind a light source to collimate the light beam emitted by the light source. As for the polarizer, theoretically, it can achieve the effect of polarizing light by placing it at any position on the light path of the projection lighting device 100. However, if it is placed at different positions, the transmittance of light will be different, and the light transmission efficiency of the entire projection lighting device 100 will be greatly different. Figure 5 As shown, if the incident angles are all summarized in the range of 0° to 90°, the transmittance of the polarizer decreases as the incident angle of the light decreases, and the transmittance is highest when the incident angle is 90°. Currently, most of the disclosed methods place the polarizer in front of the projection lens, and the projection lens is a FA lens. The light has a fixed field angle. In the edge field of view, the incident angle is small and the light transmittance is low, which affects the final light transmission efficiency and uniformity. The present application places the polarizer after the collimating lens group of the light source. The light passing through the collimating lens group can ensure good collimation. The incident light angles of each field of view are close to 90°, and the light transmittance passing through the polarizer is close to the highest, which greatly improves the light transmission efficiency and projection uniformity.

[0060] Please refer to Figure 2 and Figure 4In some embodiments, the projection lighting device 100 further includes a projection lens group 107 (also called a projection lens), which is disposed on the optical path of the structured light and is used to project the structured light onto the imaged object 400. The projection lens group 107 adopts a common FA design or a telecentric design.

[0061] Usually, the projection object plane is not perpendicular to the main optical axis of the projection lens 107, but forms a certain angle with the main optical axis of the projection lens 107, for example, a 60° angle, that is, the main optical axis of the projection lens 107 forms a 30° angle with the main optical axis of the imaging device 200 (of course, it can also be other angles, usually in the range of [27°, 30°]). In order to ensure the clarity of the projection object plane, in the optical path design, the spatial light modulator 106 is placed at a certain angle so that the projection image plane of the spatial light modulator 106, the main plane of the projection lens 107, and the projection object plane satisfy Sham's law. Figure 6 As shown, the extension line of the projection image plane, the extension line of the projection lens principal plane and the extension line of the projection object plane intersect at one point, so that even if the projection lens 107 is installed at an angle, a clear projection pattern can still be obtained on the object plane.

[0062] Please refer to Figure 2 and Figure 4 In some embodiments, the imaging device 200 further includes an imaging lens group 203 (also referred to as an imaging lens), and the polarizer 201 is disposed between the imaging lens group 203 and the image sensor 202. The imaging lens group 203 is used to receive structured reflected light formed by the structured light being reflected by the imaging object 400, and converge and project it onto the polarizer 201.

[0063] The imaging lens 203 preferably adopts an image-side telecentric design or a double telecentric design. The analyzer 201 is placed between the imaging lens 203 and the image sensor 202, and the imaging lens 203 adopts an image-side telecentric lens or a double telecentric lens, so as to ensure that the angle of the incident light passing through the analyzer 201 is as close to the 90° incident angle as possible, so that a high light transmittance and imaging uniformity can also be ensured on the imaging side. Under the same lighting brightness conditions, the exposure time of the image sensor 202 is shorter, the shooting speed of the structured light three-dimensional imaging device is faster, and it has higher real-time performance.

[0064] Please refer to Figure 4 In some embodiments, the projection lighting device 100 further includes a reflector 110, which is disposed on the optical path of the outgoing light of the light combining device 105 and is used to reflect the outgoing light of the light combining device 105 to the spatial light modulator 106. The value range of the angle θ between the normal of the reflection surface of the reflector 110 and the outgoing light of the light combining device 105 is 0°<θ<90°, so that the optical path of the outgoing light of the light combining device 105 is turned.

[0065] Please refer to Figure 4In some embodiments, the projection lighting device 100 further includes a light homogenizing device 111, which is disposed on the optical path of the output light of the light combining device 105 incident on the reflector 110, and is used to homogenize the output light of the light combining device 105 and then project it to the reflector 110. The light homogenizing device 111 may be a double-sided microlens array. The provision of the light homogenizing device 111 makes the illumination more uniform.

[0066] Please refer to Figure 4 In some embodiments, the projection lighting device 100 further includes a first relay lens 112, which is disposed on the optical path of the output light of the light homogenizing device 111 incident on the reflector 110, and is used to converge the output light of the light homogenizing device 111 and then project it to the reflector 110. Since the light beam is relatively divergent after passing through the light homogenizing device 111, the first relay lens 112 is provided in this embodiment to converge the light beam, for example, the output light of the light combining device 105 is cut into light spots by the double-sided microlens array, and then converged by the first relay lens 112, so that the light is more concentrated.

[0067] Please refer to Figure 4 In some embodiments, the projection lighting device 100 further includes a second relay lens 113, which is disposed on the optical path of the reflected light from the reflector 110 incident on the spatial light modulator 106, and is used to converge the reflected light from the reflector 110 and then project it to the spatial light modulator 106. Here, the light beam is converged again to improve the convergence effect.

[0068] from Figure 4 As can be seen in the figure, the reflector 110 bends the originally straight light path, so that the optical components can be arranged more centrally and compactly in the projection lighting device 100.

[0069] Please refer to Figure 4 In some embodiments, the projection lighting device 100 further includes a compound prism 114, which is disposed in front of the spatial light modulator 106, so that the output light of the second relay lens 113 is incident on the spatial light modulator 106 through the compound prism 114, and the structured light output from the spatial light modulator 106 is reflected by the compound prism 114 onto the imaged object 400.

[0070] The propagation path of light in the composite prism 114 is as follows: Figure 4In some embodiments, the positions of the spatial light modulator 106 and the compound prism 114 are set to cooperate with the reflector 110 so that the direction of the structured light emitted by the compound prism 114 is opposite to the direction of the output light of the light combining device 105, so that the overall light path of the projection lighting device 100 is in a folded form, making the structure of the projection lighting device 100 more compact and miniaturized. It should be noted that at this time, the projection image plane is not on the spatial light modulator 106, but on the reflection surface of the structured light on the compound prism 114.

[0071] The control device 300 may include an embedded control system, which is composed of an MCU (Microcontroller Unit), a SOC (System on Chips), a DSP (Digital Signal Processing) and / or an FPGA (Field Programmable Gate Array), and is mainly responsible for synchronization signal control, image processing and data transmission functions. The user can send instructions to the control device 300 through the host computer to control the structured light imaging device, and the control device 300 starts the projection lighting device 100 and the imaging device 200 for projection and shooting according to the signal sent by the host computer. The shooting signal sent by the control device 300 to the imaging device 200 and the exposure time of the imaging device 200 need to be synchronized with the projection signal sent to the projection lighting device 100 and the projection time of the projection lighting device 100. After a series of structured light pattern projection and photography, the control device 300 collects the image imaged by the image sensor 200, starts the image algorithm for reconstruction and fusion, and finally obtains the height image of the imaged object 400, and then transmits the height image to the host computer.

[0072] Please refer to Figure 4 and Figure 7 In some embodiments, the control device 300 includes a projection control module for controlling the projection lighting device 100. The projection control module receives a control signal from the control device 300 to control the two-channel light source (the first light source 101 and the second light source 102) and the spatial light modulator 106. When there are multiple projection lighting devices 100, each projection lighting device 100 corresponds to a projection control module for control. Figure 7 One projection lighting device 100 is controlled by a first projection control module, and the other projection lighting device 100 is controlled by a second projection control module. Figure 8In some embodiments, the projection control module includes a first light source control module, a second light source control module and a light modulation control module. The first light source control module and the second light source control module respectively control the on / off enable and current size of the two-channel light sources, and change the projection brightness of the projection lighting device 100 by adjusting the current size of the light source. The light modulation control module can control the pattern projected by the spatial light modulator 106.

[0073] In view of the difficulties of structured light three-dimensional reconstruction mentioned above, the existing methods usually improve and optimize around the optical or algorithmic solutions. In combination with the proposed structured light imaging device, this application provides a reconstruction algorithm for suppressing multiple reflections, and uses polarized light with different polarization directions to project onto the imaged object at the same incident angle for structured light projection. At this time, the reflectivity of the object surface is different, and the polarized light reflected from a certain point on the object surface enters the image sensor 220 at a different light intensity for imaging at the corresponding pixel position, resulting in different intensities of the collected projection images of the imaged object. The multiple reflection area is identified based on the difference in the intensity of the projection image, and the reconstructed height image is corrected. This application analyzes the difference between multiple reflections and normal reflections from the optical essence, and combines polarization optical technology and image processing algorithms to solve the difficult problem of multiple reflections in three-dimensional reconstruction of complex surface objects. Among them, using polarized light with different polarization directions to project structured light onto the imaged object at the same incident angle can be achieved by the structured light imaging device of this application, and the control device 300 controls the dual-channel illumination light source to light up separately in sequence, thereby obtaining two polarized structured light images of the imaged object with different polarization directions.

[0074] Therefore, under the control of the control device 300, the projection lighting device 100 can realize at least three projection modes: single polarization projection mode: a single-channel illumination light source is lit, that is, only one of the first light source 101 and the second light source 102 is turned on; dual polarization projection mode: dual-channel illumination light sources are lit at the same time, that is, both the first light source 101 and the second light source 102 are turned on, and the first polarized light and the second polarized light are used for simultaneous projection, and this mode can realize the complementarity of the dual polarized light sources; alternating projection mode: the dual-channel illumination light sources are lit in sequence, that is, the projection lighting device 100 performs two structured light projections, one of which is the first light source 101 turned on and the second light source 102 turned off, and the first polarized light is used for projection, and the other is the first light source 101 turned off and the second light source 102 turned on, and the second polarized light is used for projection, and this mode can realize the suppression of multiple reflections. The user can input instructions to select the projection mode, which correspond to the single polarization projection mode, the dual polarization projection mode and the alternating projection mode, respectively, as the first imaging instruction, the second imaging instruction and the third imaging instruction.

[0075] When receiving the first imaging instruction input by the user, the control device 300 controls the projection lighting device 100 to perform structured light projection on the imaged object 400 in a single polarization projection mode, triggers the imaging device 200 to shoot the imaged object 400, obtains a first polarization structured light image of the imaged object 400, and performs three-dimensional reconstruction using the first polarization structured light image to obtain a height image of the imaged object 400. The three-dimensional reconstruction here can adopt any three-dimensional reconstruction method, and this application is not limited to this.

[0076] When receiving the second imaging instruction input by the user, the control device 300 controls the projection lighting device 100 to perform structured light projection on the imaged object 400 in a dual polarization projection mode, triggers the imaging device 200 to shoot the imaged object 400, obtains a second polarization structured light image of the imaged object 400, and performs three-dimensional reconstruction using the second polarization structured light image to obtain a height image of the imaged object 400. The three-dimensional reconstruction here can adopt any three-dimensional reconstruction method, and this application is not limited to this.

[0077] When receiving the third imaging instruction input by the user, the control device 300 controls the projection lighting device 100 to perform structured light projection on the imaged object 400 in an alternating projection mode. This mode performs two projections. During each projection, the control device 300 triggers the imaging device 200 to shoot the imaged object 400 to obtain a third polarized structured light image and a fourth polarized structured light image of the imaged object 400; the third polarized structured light image and the fourth polarized structured light image are used to perform three-dimensional reconstruction to obtain a height image of the imaged object 400.

[0078] The three-dimensional reconstruction is performed using the third polarized structured light image and the fourth polarized structured light image to obtain a height image of the imaged object 400, including:

[0079] The third polarized structured light image is used for three-dimensional reconstruction to obtain a first height image of the imaged object 400; the fourth polarized structured light image is used for three-dimensional reconstruction to obtain a second height image of the imaged object 400; the three-dimensional reconstruction here can adopt any three-dimensional reconstruction method, which is not limited in the present application;

[0080] A reflectivity difference map is calculated based on the third polarized structured light image and the fourth polarized structured light image, wherein the pixel value of each pixel position in the reflectivity difference map represents the reflectivity difference degree of the pixel position, and the reflectivity difference degree represents the difference degree between the reflectivity of the surface of the imaged object 400 corresponding to the pixel position to the first polarized light and the reflectivity of the second polarized light;

[0081] According to the reflectivity difference map, it is determined whether there are multiple reflections on the surface of the imaged object 400 corresponding to each pixel position. If so, the height value of the pixel position is set to an invalid pixel value. If not, the final height value of the pixel position is determined based on the height values ​​of the first height image and the second height image at the pixel position, so as to finally obtain a height image of the imaged object 400.

[0082] The applicant has conducted research on the projection of fringe patterns in structured light projection. The general N-step cosine phase shift method is used for three-dimensional reconstruction. By controlling the projection lighting device 100 to project a standard N-step cosine fringe pattern onto the imaged object, the imaging device 200 collects N fringe projection images, and the phase image is obtained by the phase resolution method. The height image is reconstructed by combining the pre-calibrated geometric constraint parameters of the projection lighting device 100 and the imaging device 200. Where N is an integer not less than 2.

[0083] The N-step phase shift method can be divided into two parts: encoding and decoding. First, the fringe pattern is encoded according to a certain rule and sent to the spatial light modulator 106 for projection. The encoding formula of the N-step fringe projection pattern is as follows:

[0084]

[0085] Where I is the ideal projection intensity, I(x,y) represents the ideal projection intensity at the pixel position (x,y), represents the phase of the pixel position (x, y), A is the coded background light intensity, B is the coded modulated light intensity, both are ideal coded values, and k is the number of steps.

[0086] After the fringe pattern is projected onto the object, the imaging device 200 performs imaging, and performs decoding or phase processing according to the N fringe projection images taken to obtain the phase, decoded background light intensity, and decoded modulated light intensity. The decoding formula is as follows:

[0087]

[0088] Wherein, I' is the intensity (grayscale value) of the captured fringe projection image, I'(x, y) represents the intensity at the pixel position (x, y), A' is the decoded background light intensity, and B' is the decoded modulated light intensity.

[0089] Due to the influence of ambient light, differences in light incident angles, differences in optical lenses, nonlinear responses of image sensors, and differences in object surface reflectivity, the relationship between the ideal projection intensity and the intensity of the actual captured fringe projection image can be simply expressed as:

[0090] I'(x,y)=I(x,y)·F(x,y)·ξ(x,y)+μ(x,y), (3)

[0091] Wherein, μ(x, y) is the influence of the background ambient light on the pixel position (x, y), that is, the background light intensity collected by the imaging device 200 when the projection lighting device 100 is turned off; F(x, y) is the reflectivity of the surface of the imaged object corresponding to the pixel position (x, y); ξ(x, y) is the collective expression of other nonlinear influencing factors. Substituting formulas (1) and (2) into (3) for simplification, we can obtain:

[0092]

[0093] The above formula can be approximated as follows:

[0094] A'=A·F(x,y)·ξ(x,y)+μ(x,y),

[0095] B'=B·F(x,y)·ξ(x,y),

[0096] The above two equations show that the difference in reflectivity of the object surface can be reflected in the two indicators of decoding background light intensity and decoding modulated light intensity.

[0097] When using the structured light imaging device of the present application, in the process of projection reconstruction using two polarized light sources (first polarized light and second polarized light) with different polarization directions, the current of the first light source 101 and the second light source 102 can be controlled by the control device 300 to be consistent, so as to ensure that the brightness of the light sources of the two channels is strictly consistent; and because the light sources of the two channels share optical devices and the shooting process also shares image sensors, the nonlinear influencing factors ξ(x, y) of the two projection reconstructions can be considered to be consistent; the ambient background light μ(x, y) is also consistent. Therefore, the first background light intensity image and the first modulated light intensity image obtained by using the first polarized light to project the stripe pattern, and the second background light intensity image and the second modulated light intensity image obtained by using the second polarized light to project the stripe pattern can be expressed as:

[0098] A1'(x,y)=A(x,y)·F1(x,y)·ξ(x,y)+μ(x,y),

[0099] B1'(x,y)=B(x,y)·F1(x,y)·ξ(x,y),

[0100] A'2(x,y)=A(x,y)·F2(x,y)·ξ(x,y)+μ(x,y),

[0101] B'2(x,y)=B(x,y)·F2(x,y)·ξ(x,y),

[0102] Among them, A1' represents the first background light intensity image, A1'(x, y) represents the first background light intensity at the pixel position (x, y), A'2 represents the second background light intensity image, A'2(x, y) represents the second background light intensity at the pixel position (x, y), B1' represents the first modulated light intensity image, B1'(x, y) represents the first modulated light intensity at the pixel position (x, y), B'2 represents the second modulated light intensity image, B'2(x, y) represents the second modulated light intensity at the pixel position (x, y).

[0103] By subtracting, we get:

[0104] ΔA'=A1'-A'2=[F1(x,y)-F2(x,y)]·A·ξ(x,y),

[0105] ΔB'=B1'-B'2=[F1(x,y)-F2(x,y)]·B·ξ(x,y),

[0106] ΔA′ represents the difference between the first background light intensity image and the second background light intensity image, and ΔB′ represents the difference between the first modulated light intensity image and the second modulated light intensity image.

[0107] Therefore, the reflectivity difference between the two polarized light projection reconstructions can be quantified by the numerical difference between the two reconstructed decoded background light intensities or decoded modulated light intensities. Therefore, the reflectivity difference map ΔF can be determined by the following formula:

[0108] ΔF=ΔA'=A1'-A'2, (4)

[0109] or,

[0110] ΔF=ΔB'=B1'-B'2, (5)

[0111] Although both of the above can represent the reflectivity difference, when the surface of the object is translucent, the contrast of the black and white stripe image is too low, resulting in the decoded modulated light intensity being too low, and unable to accurately reflect the reflectivity. In this case, it is more appropriate to use the decoded background light intensity to represent the reflectivity difference; and when the surface of the object is a complex texture surface, the decoded background light intensity cannot accurately reflect the reflectivity, the black and white stripe image has a high contrast, and the decoded modulated light intensity is high. It is more appropriate to use the decoded modulated light intensity to represent the reflectivity difference. In order to improve the stability and applicability of the system, this application preferably uses a linear weighted method to combine and represent the reflectivity difference:

[0112] ΔF=α·ΔA'+(1-α)·ΔB', (6)

[0113] Where α is the preset weight value, which is set by the user according to the usage scenario and ranges from [0.0, 1.0].

[0114] Based on the above analysis, when fringe pattern projection is used for three-dimensional reconstruction, the control device 300 controls the projection lighting device 100 to perform structured light projection on the imaged object 400 in an alternating projection mode, including: controlling the projection lighting device 100 to perform fringe pattern projection on the imaged object 400 in an alternating projection mode according to an N-step phase shift method. Then the third polarized structured light image includes N first fringe projection images obtained when the fringe pattern is projected on the imaged object by the first polarized light according to the N-step phase shift method, and the fourth polarized structured light image includes N second fringe projection images obtained when the fringe pattern is projected on the imaged object by the second polarized light according to the N-step phase shift method.

[0115] The control device 300 performs three-dimensional reconstruction using the third polarized structured light image to obtain a first height image of the imaged object 400, performs three-dimensional reconstruction using the fourth polarized structured light image to obtain a second height image of the imaged object 400, and calculates a reflectivity difference map based on the third polarized structured light image and the fourth polarized structured light image, including:

[0116] Performing phase deconvolution on N first fringe projection images to obtain a first phase image, a first background light intensity image, and a first modulated light intensity image; performing phase deconvolution on N second fringe projection images to obtain a second phase image, a second background light intensity image, and a second modulated light intensity image; the phase deconvolution process here can adopt any phase deconvolution method, which is not limited in the present application;

[0117] Perform three-dimensional reconstruction based on the first phase image to obtain a first height image; perform three-dimensional reconstruction based on the second phase image to obtain a second height image; the three-dimensional reconstruction here can adopt any three-dimensional reconstruction method, and this application does not limit this;

[0118] The reflectivity difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, or the reflectivity difference map is calculated based on the difference between the first modulated light intensity image and the second modulated light intensity image, or the reflectivity difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image and the difference between the first modulated light intensity image and the second modulated light intensity image. For the specific formula for determining the reflectivity difference map, please refer to the above formulas (4), (5), and (6).

[0119] The height image of the imaged object 400 obtained in the end is essentially obtained by fusing the first height image and the second height image using the reflectivity difference map. In some embodiments, the fused height image of the imaged object 400 is determined by the following formula:

[0120]

[0121] Among them, H(x,y) represents the height value at the pixel position (x,y) in the height image of the imaged object 400 finally obtained, H1(x,y) represents the height value at the pixel position (x,y) in the first height image, H2(x,y) represents the height value at the pixel position (x,y) in the second height image, ΔF(x,y) represents the degree of reflectivity difference at the pixel position (x,y) in the reflectivity difference map, T is the preset reflectivity difference degree threshold, and Inv represents an invalid pixel value.

[0122] The following uses an example in which the first polarized light is S polarized light and the second polarized light is P polarized light to illustrate the steps of using a dual-polarized light source to suppress multiple reflections for 3D reconstruction. Fig. 9 As shown, the polarization directions of the S polarized light and the P polarized light are perpendicular, and they propagate along the same optical path after passing through the optical combining device 105. Fig. 9 For the convenience of display, both S polarization and P polarization are shown, but it does not mean that S polarization and P polarization must be turned on at the same time. Fig.10 , the specific steps include:

[0123] 1. Turn on the first light source 101 (i.e., turn on the S-polarized light source), turn off the second light source 102 (i.e., turn off the P-polarized light source) to perform fringe pattern projection, collect N first fringe projection images obtained by S-polarized light projection through the image sensor 202, perform phase resolution processing, and obtain a first phase image, a first background light intensity image, and a first modulated light intensity image;

[0124] 2. Turn on the second light source 102 (i.e., turn on the P polarized light source), turn off the first light source 101 (i.e., turn off the S polarized light source) to perform fringe pattern projection, collect N second fringe projection images obtained by P polarized light projection through the image sensor 202, perform phase resolution processing, and obtain a second phase image, a second background light intensity image, and a second modulated light intensity image;

[0125] 3. Reconstruct the first phase image and the second phase image in three dimensions respectively through the pre-calibrated geometric constraint relationship to obtain the first height image and the second height image;

[0126] 4. According to the difference between the first background light intensity image and the second background light intensity image, and the difference between the first modulated light intensity image and the second modulated light intensity image, calculate the polarized light reflectivity difference to obtain a reflectivity difference map, see formula (6) for details;

[0127] 5. Perform threshold comparison based on the reflectivity difference map to determine the pixel area with multiple reflections, set the corresponding height value as an invalid pixel value, thereby eliminating abnormal data caused by multiple reflections, and finally fuse the height image that suppresses multiple reflections. For details, see formula (7).

[0128] Please refer to Fig.11 It can be seen that when faced with a multi-reflection surface, the surface reconstructed by the traditional 3D reconstruction method (Figure (b)) is jagged and inconsistent with reality, while the method of the present application correctly reconstructs the multi-reflection surface (Figure (c)).

[0129] To address the problem of blind spots in the field of vision, the structured light imaging device of some embodiments of the present application uses a dual-projection lighting device to illuminate the imaged object from two different incident angles, and removes part of the blind spots in the field of vision through image fusion. Figure 7 As shown, the structured light imaging device includes two projection lighting devices 100, which are symmetrically arranged with the imaging device 200 as the center. The two projection lighting devices 100 can implement projection lighting on the imaged object from two different incident angles, thereby solving the reconstruction blind area problem caused by the blind area of ​​illumination. Please refer to Fig.12 When a single projection lighting device is used for reconstruction, area A in the figure is a blind spot for camera shooting. Even if it can be illuminated, it cannot be photographed, resulting in failure to reconstruct; area B in the figure is a lighting blind spot. The camera can capture it, but it cannot be illuminated, resulting in failure to reconstruct. This embodiment uses dual projection lighting devices for projection lighting, and performs two projection reconstructions through projection lighting devices placed in symmetrical directions, and then performs fusion, which can remove the lighting blind spot of area B in the figure.

[0130] Based on the dual-projection lighting device, the control device 300 is also used for receiving an imaging instruction input by a user:

[0131] Controlling one of the two projection lighting devices to perform structured light projection on the imaged object in a projection mode corresponding to the imaging instruction to perform three-dimensional reconstruction and obtain a height image in a first projection direction;

[0132] Controlling another projection lighting device of the two projection lighting devices to perform structured light projection on the imaged object in a projection mode corresponding to the imaging instruction to perform three-dimensional reconstruction and obtain a height image in a second projection direction;

[0133] The height image of the first projection direction and the height image of the second projection direction are fused to obtain a height image of the imaged object.

[0134] The user can select the projection mode according to the needs of the actual scene. When dealing with scenes with high real-time requirements, the single polarization projection mode in which only a single-channel polarized light source is turned on can be selected; when dealing with scenes with multiple reflections, the alternating projection mode can be selected; when dealing with scenes with high reflection and glare light, the dual polarization projection mode in which dual-channel polarized light sources are turned on at the same time can be selected. In each projection mode, the control device 300 uses the corresponding three-dimensional reconstruction method for reconstruction.

[0135] In some embodiments, the height image of the imaged object is determined by the following formula:

[0136]

[0137] Among them, I h (x, y) represents the height value of the pixel position (x, y) in the height image of the imaged object, H 1d (x, y) represents the height value at the pixel position (x, y) in the height image in the first projection direction, H 2d (x, y) represents the height value at the pixel position (x, y) in the second projection direction height image.

[0138] Two symmetrically arranged projection lighting devices 100 are defined as a projection lighting combination, that is, a projection lighting combination includes two projection lighting devices 100, and the two projection lighting devices 100 are symmetrically arranged around the imaging device 200. In some embodiments, the structured light imaging device may include at least one projection lighting combination.

[0139] The structured light imaging device provided in the embodiment of the present application, since the projection lighting device can use polarized light for projection, and the polarization state of the reflected light will change to a certain extent after the polarized light is reflected by the surface of the object, the high reflection and glare light in a specific polarization direction can be filtered out through the polarizer during imaging, thereby improving the three-dimensional reconstruction performance of highly reflective objects and enhancing the reconstruction integrity and material adaptability.

[0140] In addition, the light combining device can combine the first polarized light and the second polarized light into one polarized light and project them at the same time. Since the polarization direction of the first polarized light is different from that of the second polarized light, it is almost impossible for them to be completely filtered out by the analyzer at the same time. When one of the polarized light is filtered out, the other polarized light can play a complementary role, thereby suppressing high reflection and ensuring the lighting brightness.

[0141] At the same time, the light combining device also makes the optical path structure compact to a certain extent. In some embodiments, it can be combined with a composite prism to achieve miniaturization of the device.

[0142] In some embodiments, by setting the positions of the polarizer and the analyzer, the structured light imaging device of the present application has high light transmission efficiency and high uniformity of light.

[0143] In one embodiment of the present application, a reconstruction method for suppressing multiple reflections is also proposed, which combines polarization optical technology and image processing algorithms to solve the difficult problem of multiple reflections in three-dimensional reconstruction of complex surface objects and improve the reconstruction integrity.

[0144] The structured light imaging device in one embodiment of the present application also eliminates the lighting blind area and further improves the reconstruction integrity by setting two projection lighting devices symmetrically centered on the imaging device to illuminate the imaged object from two different incident angles.

[0145] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0146] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A structured light imaging device, characterized in that: It includes a projection lighting device, an imaging device and a control device; The projection lighting device comprises a first light source, a first polarizer, a second light source, a second polarizer, a light combining device and a spatial light modulator; The first polarizer is disposed on the optical path of the first light beam emitted by the first light source, and is used to polarize the first light beam into a first polarized light in a first polarization direction; The second polarizer is disposed on the optical path of the second light beam emitted by the second light source, and is used to polarize the second light beam into a second polarized light with a second polarization direction; wherein the first polarization direction is different from the second polarization direction; The control device is used to control the turning on and off of the first light source and the second light source; The light combining device is disposed at the intersection of the optical path of the first polarized light and the optical path of the second polarized light, and is used to combine the first polarized light and the second polarized light into the same optical path and emit them to the spatial light modulator when both the first light source and the second light source are turned on, and to emit the first polarized light or the second polarized light to the spatial light modulator when only the first light source is turned on or only the second light source is turned on; The spatial light modulator is used to modulate the light beam incident thereinto into a specific structured light, so as to project the structured light toward the imaged object; The imaging device comprises a polarizer and an image sensor; The polarizer is used to receive structured reflected light formed by the structured light reflected by the imaging object, analyze the structured reflected light, and then project it onto the image sensor for imaging.

2. The structured light imaging device according to claim 1, characterized in that: The polarization direction of the polarizer is different from both the first polarization direction and the second polarization direction.

3. The structured light imaging device according to claim 1, characterized in that: The first polarization direction is perpendicular to the second polarization direction.

4. The structured light imaging device according to claim 3, characterized in that: The polarization direction of the polarizer forms an angle of 45° with both the first polarization direction and the second polarization direction.

5. The structured light imaging device according to claim 1, characterized in that: The projection lighting device further comprises a first collimating lens group and a second collimating lens group; The first collimating lens group is disposed between the first light source and the first polarizer, and is used for collimating the first light beam and then projecting it onto the first polarizer; The second collimating lens group is arranged between the second light source and the second polarizer, and is used for collimating the second light beam and then projecting it onto the second polarizer.

6. The structured light imaging device according to claim 1, characterized in that: The optical path of the first polarized light is perpendicular to the optical path of the second polarized light, and the light combining device is a beam splitter; The beam splitter forms an angle of 45° with both the optical path of the first polarized light and the optical path of the second polarized light, and the first polarized light and the second polarized light are incident on the beam splitter from both sides of the beam splitter respectively.

7. The structured light imaging device according to claim 1, characterized in that: The projection lighting device also includes a reflector, which is arranged on the optical path of the output light of the light combining device and is used to reflect the output light of the light combining device to the spatial light modulator; the angle θ between the normal of the reflecting surface of the reflector and the output light of the light combining device is in the range of 0°<θ<90°.

8. The structured light imaging device according to claim 7, characterized in that: The projection lighting device also includes a light homogenizing device, which is arranged on the optical path of the output light of the light combining device incident on the reflector, and is used to homogenize the output light of the light combining device and then project it onto the reflector.

9. The structured light imaging device according to claim 8, characterized in that: The projection lighting device further includes a first relay lens, which is disposed on an optical path where the outgoing light of the light homogenizing device is incident on the reflector, and is used for converging the outgoing light of the light homogenizing device and then projecting it onto the reflector.

10. The structured light imaging device according to claim 8 or 9, characterized in that: The projection lighting device further includes a second relay lens, which is disposed on an optical path of the reflected light from the reflector incident on the spatial light modulator, and is used for converging the reflected light from the reflector and then projecting it onto the spatial light modulator.

11. The structured light imaging device according to claim 10, characterized in that: The projection lighting device also includes a composite prism, which is arranged in front of the spatial light modulator, so that the output light of the second relay lens is incident on the spatial light modulator through the composite prism, and the structured light emitted by the spatial light modulator is reflected onto the imaged object through the composite prism.

12. The structured light imaging device according to claim 1, characterized in that: The projection lighting device further comprises a projection lens group, which is arranged on the optical path of the structured light and is used to project the structured light onto the imaged object; The projection image plane of the spatial light modulator, the principal plane of the projection lens group and the projection object plane satisfy Schaum's law.

13. The structured light imaging device according to claim 1, characterized in that: The imaging device further comprises an imaging lens group, and the polarizer is arranged between the imaging lens group and the image sensor; The imaging lens group is used to receive the structured reflected light formed by the structured light reflected by the imaging object, and to converge and project it onto the analyzer.

14. The structured light imaging device according to claim 13, characterized in that: The imaging lens group is an image-side telecentric lens or a bi-telecentric lens.

15. The structured light imaging device according to claim 1, characterized in that: The control device is used to: When receiving a first imaging instruction input by a user, controlling the projection lighting device to perform structured light projection on the imaged object in a single polarization projection mode, triggering the imaging device to shoot the imaged object, obtaining a first polarization structured light image of the imaged object, and performing three-dimensional reconstruction using the first polarization structured light image to obtain a height image of the imaged object; in the single polarization projection mode, only one of the first light source and the second light source is turned on; When receiving a second imaging instruction input by the user, controlling the projection lighting device to perform structured light projection on the imaged object in a dual-polarization projection mode, triggering the imaging device to shoot the imaged object, obtaining a second polarization structured light image of the imaged object, and performing three-dimensional reconstruction using the second polarization structured light image to obtain a height image of the imaged object; in the dual-polarization projection mode, both the first light source and the second light source are turned on; When receiving a third imaging instruction input by the user, the projection lighting device is controlled to perform structured light projection on the imaged object in an alternating projection mode, wherein the projection lighting device performs structured light projection twice, wherein the first light source is turned on and the second light source is turned off once, and the first light source is turned off and the second light source is turned on another time, and the imaging device is triggered to shoot the imaged object during each projection to obtain a third polarized structured light image and a fourth polarized structured light image of the imaged object; and the third polarized structured light image and the fourth polarized structured light image are used to perform three-dimensional reconstruction to obtain a height image of the imaged object; The three-dimensional reconstruction using the third polarized structured light image and the fourth polarized structured light image to obtain the height image of the imaged object includes: Performing three-dimensional reconstruction using the third polarized structured light image to obtain a first height image of the imaged object; performing three-dimensional reconstruction using the fourth polarized structured light image to obtain a second height image of the imaged object; A reflectivity difference map is calculated based on the third polarized structured light image and the fourth polarized structured light image, wherein the pixel value of each pixel position of the reflectivity difference map represents the reflectivity difference degree of the pixel position, and the reflectivity difference degree represents the difference degree of the reflectivity of the surface of the imaged object corresponding to the pixel position to the first polarized light and the reflectivity of the second polarized light; According to the reflectivity difference map, it is determined whether there are multiple reflections on the surface of the imaged object corresponding to each pixel position. If so, the height value of the pixel position is set to an invalid pixel value. If not, the final height value of the pixel position is determined based on the height values ​​of the first height image and the second height image at the pixel position to obtain the height image of the imaged object.

16. The structured light imaging device according to claim 15, characterized in that: The controlling the projection illumination device to perform structured light projection on the imaged object in an alternating projection mode comprises: controlling the projection illumination device to perform stripe pattern projection on the imaged object in an alternating projection mode according to an N-step phase shift method; wherein N is an integer not less than 2; The third polarized structured light image includes N first fringe projection images obtained when the first polarized light is used to project a fringe pattern on the imaged object according to an N-step phase shift method, and the fourth polarized structured light image includes N second fringe projection images obtained when the second polarized light is used to project a fringe pattern on the imaged object according to an N-step phase shift method; The method of performing three-dimensional reconstruction using the third polarized structured light image to obtain a first height image of the imaged object, performing three-dimensional reconstruction using the fourth polarized structured light image to obtain a second height image of the imaged object, and calculating a reflectivity difference map based on the third polarized structured light image and the fourth polarized structured light image includes: Performing phase deconvolution on the N first fringe projection images to obtain a first phase image, a first background light intensity image, and a first modulated light intensity image; performing phase deconvolution on the N second fringe projection images to obtain a second phase image, a second background light intensity image, and a second modulated light intensity image; Performing three-dimensional reconstruction based on the first phase image to obtain the first height image; performing three-dimensional reconstruction based on the second phase image to obtain the second height image; The reflectivity difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, or the reflectivity difference map is calculated based on the difference between the first modulated light intensity image and the second modulated light intensity image, or the reflectivity difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image and the difference between the first modulated light intensity image and the second modulated light intensity image.

17. The structured light imaging device according to claim 16, wherein: The reflectivity difference map is determined by the following formula: ΔF=ΔA'=A1'-A'2, Or, ΔF = ΔB' = B'1 - B'2, Or, ΔF=αΔA'+(1-α)·ΔB'; Among them, ΔF represents the reflectivity difference map, ΔA' represents the difference between the first background light intensity image and the second background light intensity image, ΔB' represents the difference between the first modulated light intensity image and the second modulated light intensity image, A1' represents the first background light intensity image, A'2 represents the second background light intensity image, B1' represents the first modulated light intensity image, B'2 represents the second modulated light intensity image, and α is a preset weight value.

18. The structured light imaging device according to claim 15, characterized in that: It comprises two projection and lighting devices, and the two projection and lighting devices are symmetrically arranged with the imaging device as the center; The control device is also used for, when receiving an imaging instruction input by a user: Controlling one of the two projection lighting devices to perform structured light projection on the imaged object in a projection mode corresponding to the imaging instruction to perform three-dimensional reconstruction and obtain a height image in a first projection direction; Controlling another projection lighting device of the two projection lighting devices to perform structured light projection on the imaged object in a projection mode corresponding to the imaging instruction, so as to perform three-dimensional reconstruction and obtain a height image in a second projection direction; The first projection direction height image and the second projection direction height image are fused to obtain a height image of the imaged object.

19. The structured light imaging device according to claim 1, characterized in that: It comprises at least one projection lighting combination, each of which comprises two projection lighting devices, and the two projection lighting devices in each of which are symmetrically arranged with the imaging device as the center.

Images