Three-dimensional imaging method based on structured light and structured light imaging device, storage medium

Abstract

A three-dimensional imaging method based on structured light and a structured light imaging device, a storage medium, wherein the three-dimensional imaging method comprises: acquiring a first polarized structured light image and a second polarized structured light image of an imaged object photographed when two polarized lights with different polarization directions and the same incident angle are projected, performing three-dimensional reconstruction respectively by using the first polarized structured light image and the second polarized structured light image to obtain a first height image and a second height image; calculating a reflectivity difference map according to the first polarized structured light image and the second polarized structured light image; judging whether the surface of the imaged object corresponding to each pixel position exists multiple reflections according to the reflectivity difference map, if yes, setting the height value of the pixel position as an invalid pixel value, if not, determining the final height value of the pixel position according to the height values of the first height image and the second height image at the pixel position, thereby weakening the influence of the multiple reflections of the surface of the object on the reconstruction accuracy and completeness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional reconstruction, in particular to a three-dimensional imaging method based on structured light and a structured light imaging device and a storage medium. BACKGROUND

[0002] With the increasingly wide application of three-dimensional detection technology in the field of industrial automation, the use of three-dimensional reconstruction technology based on structured light 3D camera with the core of triangulation principle is also becoming more and more popular. The structured light three-dimensional reconstruction system often adopts a projector combined with a camera, and the main optical axis of the projector and the camera forms a certain angle. The structured light is projected on the object by the projector, the structured light image of the object is captured by the camera, and then the three-dimensional reconstruction is carried out to obtain the height image of the object to reconstruct the surface topography of the object.

[0003] In the structured light three-dimensional reconstruction system, the imaging unit carries out three-dimensional reconstruction by collecting the diffuse reflection image of the object surface. The three-dimensional reconstruction utilizes the geometric constraint of reflected light, and its essence is that the incident light and the diffuse reflection light form a certain angle to form a triangular constraint. When the surface topography of the object is complex, the incident light is incident on the object surface, and the complex surface causes multiple reflections, so that the image collected by the imaging unit is not the diffuse reflection image of the actual object surface, but a diffuse reflection image containing multiple surface information. Even if the reconstruction can be solved, the obtained topography information cannot represent the real object surface information. Figure 1 As shown in the figure, the incident light and the normal diffuse reflection light form a triangular constraint, but the geometric relationship between the multiple reflection light and the incident light is uncertain, which does not satisfy the triangular constraint. At this time, the geometric constraint of three-dimensional reconstruction does not hold, resulting in incomplete and unrealistic reconstruction data. SUMMARY

[0004] The present application provides a three-dimensional imaging method based on structured light, a three-dimensional imaging method based on a structured light imaging device and a storage medium, which mainly solves the technical problem of multiple reflections of light caused by complex surfaces in three-dimensional reconstruction.

[0005] According to a first aspect, a three-dimensional imaging method based on structured light is provided in an embodiment, comprising:

[0006] obtaining a first polarized structured light image and a second polarized structured light image of an imaged object; wherein the first polarized structured light image is an image of the imaged object captured when the imaged object is projected with a first polarized light, the second polarized structured light image is an image of the imaged object captured when the imaged object is projected with a second polarized light, the polarization directions of the first polarized light and the second polarized light are different, and the first polarized light and the second polarized light are incident on the surface of the imaged object at the same incident angle;

[0007] performing three-dimensional reconstruction using the first polarized structured light image to obtain a first height image of the imaged object; performing three-dimensional reconstruction using the second polarized structured light image to obtain a second height image of the imaged object;

[0008] calculating a reflectivity difference map according to the first polarized structured light image and the second polarized structured light image, wherein a pixel value of each pixel position of the reflectivity difference map represents a reflectivity difference degree of the pixel position, and the reflectivity difference degree represents a difference degree between a reflectivity of the first polarized light and a reflectivity of the second polarized light of a surface of the imaged object corresponding to the pixel position;

[0009] judging whether a surface of the imaged object corresponding to each pixel position has multiple reflections according to the reflectivity difference map, if yes, setting a height value of the pixel position as an invalid pixel value, and if not, determining a final height value of the pixel position according to height values of the pixel position in the first height image and the second height image to obtain a final height image of the imaged object.

[0010] According to a second aspect, an embodiment provides a three-dimensional imaging method based on a structured light imaging device, the structured light imaging device comprising an imaging apparatus and two projection illuminating apparatuses, and the two projection illuminating apparatuses are symmetrically arranged with the imaging apparatus as the center, the projection illuminating apparatuses are capable of performing structured light projection with first polarized light and second polarized light, the polarization directions of the first polarized light and the second polarized light are different, and the three-dimensional imaging method comprises:

[0011] controlling one of the two projection illuminating apparatuses to perform structured light projection on an imaged object with the first polarized light, triggering the imaging apparatus to capture the imaged object to obtain a first polarized structured light image; and controlling the projection illuminating apparatus to perform structured light projection on the imaged object with the second polarized light, triggering the imaging apparatus to capture the imaged object to obtain a second polarized structured light image;

[0012] The three-dimensional imaging method according to the first aspect, using the first polarized structured light image and the second polarized structured light image to perform three-dimensional imaging to obtain a first projection direction height image;

[0013] controlling the other of the two projection illuminating apparatuses to perform structured light projection on an imaged object with the first polarized light, triggering the imaging apparatus to capture the imaged object to obtain a third polarized structured light image; and controlling the projection illuminating apparatus to perform structured light projection on the imaged object with the second polarized light, triggering the imaging apparatus to capture the imaged object to obtain a fourth polarized structured light image;

[0014] According to the three-dimensional imaging method of the first aspect, the third polarized structured light image and the fourth polarized structured light image are used to perform three-dimensional imaging to obtain a second projection direction height image;

[0015] The first projection direction height image and the second projection direction height image are fused to obtain a height image of the imaged object.

[0016] According to a third aspect, a computer readable storage medium is provided in an embodiment, and the medium stores a program which can be executed by a processor to implement the three-dimensional imaging method of the first aspect or the second aspect.

[0017] According to the three-dimensional imaging method of the above embodiment, the principle that the reflectivity of polarized light with different polarization directions is different (i.e., the reflection intensity is different) when the polarized light is incident at the same angle is used. Since the reflection intensity difference after multiple surface reflections is large, the degree of reflectivity difference can be used to determine whether multiple reflections occur. In the three-dimensional imaging method of the present application, a first polarized structured light image and a second polarized structured light image obtained by projecting structured light with two polarized lights with different polarization directions and the same incident angle are first obtained, and three-dimensional reconstruction is performed using the first polarized structured light image and the second polarized structured light image to obtain a first height image and a second height image. Then, a reflectivity difference image is calculated according to the first polarized structured light image and the second polarized structured light image. According to the reflectivity difference image, it is determined whether the surface of the imaged object corresponding to each pixel position has multiple reflections. If so, the height value of the pixel position is set to an invalid pixel value. If not, the final height value of the pixel position is determined according to the height values of the first height image and the second height image at the pixel position. Thus, the influence of multiple reflections on the surface of the object on the reconstruction accuracy and completeness is weakened, the completeness and authenticity of three-dimensional reconstruction are improved, and the material adaptability and scene adaptability of the three-dimensional imaging system and method are effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A schematic diagram of multiple reflections of light on the surface of an object;

[0019] Figure 2 A flowchart of a structured light-based three-dimensional imaging method of an embodiment;

[0020] Figure 3 A comparison diagram of the effects of the three-dimensional imaging method of the present application and a traditional reconstruction method;

[0021] Figure 4 A structural schematic diagram of a structured light imaging device of an embodiment;

[0022] Figure 5Fig. 1 is a schematic diagram of the polarization direction of the first and second polarizers and the polarization direction of the analyzer in one embodiment;

[0023] Figure 6 Fig. 2 is a schematic diagram of the structure of the structured light imaging device in another embodiment;

[0024] Figure 7 Fig. 3 is a schematic diagram of the relationship between the light transmittance of the polarizer and the incident angle of the light;

[0025] Figure 8 Fig. 4 is a schematic diagram of the geometric relationship between the projection image plane of the spatial light modulator, the main plane of the projection lens, and the projection object plane;

[0026] Figure 9 Fig. 5 is a schematic diagram of the light path of the S-polarized light and the P-polarized light after passing through the light combining device;

[0027] Figure 10 Fig. 6 is a schematic diagram of the process of the three-dimensional reconstruction using the S-polarized light and the P-polarized light to suppress multiple reflections;

[0028] Figure 11 Fig. 7 is a schematic diagram of the structure of the structured light imaging device in yet another embodiment;

[0029] Figure 12 Fig. 8 is a schematic diagram of the structure of the structured light imaging device in yet another embodiment;

[0030] Figure 13 Fig. 9 is a schematic diagram of the structure of the projection control module in one embodiment;

[0031] Figure 14 Fig. 10 is a schematic diagram of the field of view blind area of the three-dimensional reconstruction;

[0032] Figure 15 Fig. 11 is a flowchart of the three-dimensional imaging method based on the structured light imaging device in one embodiment.

[0033] Reference signs:

[0034] 100, projection illuminating device; 101, first light source; 102, second light source; 103, first polarizer; 104, second polarizer; 105, light combining device; 106, spatial light modulator; 107, projection lens group; 108, first collimating lens group; 109, second collimating lens group; 110, reflecting mirror; 111, light homogenizing device; 112, first relay lens; 113, second relay lens; 114, compound prism; 200, imaging device; 201, analyzer; 202, image sensor; 203, imaging lens group; 300, control device; 400, imaged object. DETAILED DESCRIPTION

[0035] The application will be described in further detail below with specific reference being made to the figures. Like elements in different embodiments are denoted by like reference numerals. In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the application. However, it will be apparent to one skilled in the art that the specific features described can not be felt to be critical in every case, and that some features who can be omitted in different cases, or replaced by other elements, materials, methods. In some cases, some operations related to the application are not shown or described in the specification, in order to avoid the core of the application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail according to the description in the specification and the general technical knowledge in the art.

[0036] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various embodiments. At the same time, the steps or actions in the method description can also be sequentially adjusted or adjusted in a manner that can be apparent to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment, and do not mean that the sequence is necessary, unless otherwise stated that a certain sequence must be followed.

[0037] The serial numbers of the components in this paper, such as "first", "second", etc., are only used to distinguish the described objects, and do not have any sequence or technical meaning. The "connection" and "coupling" in this application include direct and indirect connections (couplings) unless otherwise specified.

[0038] Currently, the following problems still exist when the structured light-based three-dimensional reconstruction method deals with complex surfaces (high reflectivity, multiple reflections, etc.):

[0039] 1. The structured light 3D camera is a diffuse reflection imaging system, but the material of some object surfaces can cause mirror reflection on the object surface, which has high reflectivity characteristics, making it difficult to solve the actual surface topography of the reconstructed object due to overexposure or overdarkness of the images collected by the camera;

[0040] 2. When the object surface topography is complex, the illumination light is incident on the object surface, and multiple reflections are formed after the complex surface, the images collected by the camera are not the diffuse reflection images of the actual object surface, but the diffuse reflection images containing multiple surface information. Even if the reconstruction can be solved, the obtained topographic information cannot represent the true object surface information;

[0041] 3. Due to the angle, the structured light 3D camera will have 2 visual field blind areas: one is the area that the projector can illuminate but the camera cannot shoot; the second is the area that the camera can shoot but the projector cannot illuminate; these blind areas will cause the reconstruction data to be incomplete.

[0042] In view of the difficulties of the above-mentioned structured light-based three-dimensional reconstruction method, existing methods often use high-cost cameras with low practicability or multiple exposure methods without real-time performance to optimize and improve the reconstruction integrity of structured light three-dimensional reconstruction.

[0043] Patent application CN202310024635 solves the problem of high reflectivity by using a four-way polarization camera combined with projected polarized light. The four sets of polarization images of the polarization camera are fused to reconstruct the object topography information, improving the robustness and reconstruction integrity of the system. Although the four-way polarization camera can provide reconstruction data of four polarization directions, the resolution of the camera is sacrificed, and the system resolution is only one-fourth of that of a normal camera. Moreover, the cost of the camera is relatively high.

[0044] Patent application CN202210831038 uses a modified error energy function to remove abnormal invalid points. The error energy function combines the wrapped phase of the stripe image, the background light intensity, and the modulation light intensity related information. This patent application removes abnormal points from a purely algorithmic perspective, and does not solve the problems of high reflectivity and multiple reflections from an optical essence.

[0045] Patent application CN202110372335 uses a multi-shot method to fuse reconstruction data under different projection intensities to solve the problem of high reflectivity. The optimal number of projection brightness is estimated based on image pixel features, and the corresponding number of stripe patterns is projected according to the number, and the object's stripe image is obtained by shooting. Fusing each group of stripe images obtains a high dynamic stripe image, and then reconstructs more complete data. This scheme trades time for integrity and does not have real-time performance. It also does not solve the problems of high reflectivity and multiple reflections from an optical perspective.

[0046] Patent application CN202110369458 uses a liquid crystal display to project linearly polarized light and places a polarizing plate in front of the imaging lens. When the camera is shooting, the angle of the polarizing plate in front of the camera is rotated four times, and the corresponding coded images are collected. The Stokes parameters are calculated based on the four images to determine the functional relationship between the linear polarization degree and the incident angle. Finally, the normal information of each point on the object surface is calculated according to the incident angle constraint condition, and the topography information of the object surface is obtained. This scheme uses the polarization characteristics of light to solve the problem of specular reflection, but it needs to mechanically rotate the polarizing plate in front of the lens and ensure the accuracy of the polarization angle, which is not practical.

[0047] In addition, the disclosed method for solving the problem of high light reflection in structured light three-dimensional reconstruction by using a polarizer, although according to the difference between the polarization direction of the high light reflection light and the imaging detection direction, the high light reflection phenomenon in some areas of the image can be weakened or eliminated, but at the same time, the illumination intensity of other normally diffused reflection areas is weakened, causing the problem of insufficient illumination, and the image is too dark to completely reconstruct. This and that, new problems are caused, and the practicability of structured light three-dimensional reconstruction is not improved.

[0048] For the problem of multiple reflections, the applicant realizes that when polarized light with different polarization directions, such as linearly polarized light, is incident at the same angle, the reflectivity of the object surface is different (the reflection intensity is different), and after multiple surface reflections, the reflection intensity difference is large. By using this point, the degree of reflectivity difference can be estimated by image algorithm to determine whether multiple reflections have occurred, and then the reconstruction integrity is improved.

[0049] Accordingly, the applicant proposes a structured light-based three-dimensional imaging method, which uses polarized light with different polarization directions to project structured light on the imaged object at the same incident angle. At this time, the reflectivity of the object surface is different, and the intensity of the polarized light reflected by the object surface at a certain position of the object surface into the corresponding pixel position of the image sensor is different, resulting in different intensities of the collected projection images of the imaged object. According to the difference in the intensity of the projection image, the multiple reflection area is identified, and the height image reconstructed is corrected. The method of the present application analyzes the difference between multiple reflections and normal reflections from the optical essence, and solves the difficult problem of multiple reflections in three-dimensional reconstruction of complex surface objects by combining polarized optical technology and image processing algorithm.

[0050] Please refer to Figure 2 The structured light-based three-dimensional imaging method in some embodiments includes steps 10-40, which are described in detail below.

[0051] Step 10: Obtain a first polarized structured light image and a second polarized structured light image of the imaged object.

[0052] The first polarized structured light image is an image of the imaged object taken when the first polarized light is used to project structured light on the imaged object, and the second polarized structured light image is an image of the imaged object taken when the second polarized light is used to project structured light on the imaged object. The polarization directions of the first polarized light and the second polarized light are different, and the first polarized light and the second polarized light are incident on the surface of the imaged object at the same incident angle. Preferably, the polarization directions of the first polarized light and the second polarized light are perpendicular.

[0053] The present application does not limit the projection illumination device for structured light projection, as long as it can use two different polarization directions of polarized light to be incident at the same incident angle. For example, the same projection illumination device can be used to project structured light in the same posture, and a different polarizer with different polarization directions is added in front of the projection lens each time; or a projection illumination device with two switchable channel polarized light sources is used to project twice.

[0054] Step 20: three-dimensional reconstruction is performed using the first polarized structured light image to obtain a first height image of the imaged object; three-dimensional reconstruction is performed using the second polarized structured light image to obtain a second height image of the imaged object.

[0055] The three-dimensional reconstruction here can use any three-dimensional reconstruction method, which is not limited by the present application.

[0056] Step 30: calculate the reflectivity difference map from the first polarized structured light image and the second polarized structured light image

[0057] The pixel value of each pixel position of the reflectivity difference map represents the reflectivity difference degree of the pixel position, and the reflectivity difference degree represents the difference degree of the reflectivity of the surface of the imaged object corresponding to the pixel position to the first polarized light and the second polarized light.

[0058] Step 40: fuse the first height image and the second height image according to the reflectivity difference map to obtain the final height image of the imaged object.

[0059] Specifically, according to the reflectivity difference map, it is judged whether the surface of the imaged object corresponding to each pixel position has multiple reflections, if yes, the height value of the pixel position is set to an invalid pixel value, if not, the final height value of the pixel position is determined according to the height values of the first height image and the second height image at the pixel position, to obtain the final height image of the imaged object.

[0060] The applicant has explored the stripe pattern projection in structured light projection. The general N-step cosine phase shift method is used for three-dimensional reconstruction. By controlling the projection illumination device to project a standard N-step cosine stripe pattern to the imaged object, N stripe projection images are collected by the imaging device, and the phase image is obtained by solving the phase method, and the height image is reconstructed combined with the geometric constraint parameters of the projection illumination device and the imaging device. Wherein N is an integer not less than 2.

[0061] The N-step phase shift method can be divided into encoding and decoding. First, encode the stripe pattern according to certain rules, and send it to the projection illumination device for projection. The encoding formula of N-step stripe projection pattern is as follows:

[0062]

[0063] where I is the ideal projection intensity, I(x, y) represents the ideal projection intensity at pixel position (x, y), represents the phase of pixel position (x, y), A is the encoded background light intensity, B is the encoded modulation light intensity, both are ideal encoded values, and k is the step number.

[0064] After the fringe pattern is projected onto the object, the object is imaged, and the phase, the decoded background light intensity, and the decoded modulation light intensity are obtained by decoding or phase unwrapping processing according to the N captured fringe projection images, and the decoding formula is as follows:

[0065]

[0066] where I' is the intensity (gray value) of the captured fringe projection image, I'(x, y) represents the intensity at pixel position (x, y), A' is the decoded background light intensity, and B' is the decoded modulation light intensity.

[0067] Due to the influence of environmental light, light incidence angle difference, optical lens difference, nonlinear response of the image sensor, and surface reflectivity difference of the object, the relationship between the ideal projection intensity and the intensity of the actually captured fringe projection image can be briefly expressed as:

[0068] I'(x, y) = I(x, y) · F(x, y) · ξ(x, y) + μ(x, y), (3)

[0069] where μ(x, y) is the influence of the background environmental light on pixel position (x, y), that is, the background light intensity collected by the imaging device when the projection illumination device is turned off; F(x, y) is the reflectivity of the surface of the imaged object corresponding to pixel position (x, y); and ξ(x, y) is a collective expression of other nonlinear influencing factors. After substituting formulas (1) and (2) into (3) and simplifying, we obtain:

[0070]

[0071] From the above formula, we can approximately obtain:

[0072] A' = A · F(x, y) · ξ(x, y) + μ(x, y),

[0073] B' = B · F(x, y) · ξ(x, y),

[0074] The above two formulas show that the difference in surface reflectivity of the object can be reflected in the decoded background light intensity and the decoded modulation light intensity.

[0075] In the process of projection reconstruction using two polarized light sources with different polarization directions, the currents of the two polarized light sources can be controlled to be consistent, so as to ensure that the brightness of the two-channel polarized light sources is strictly consistent; and the two polarized light sources can be set to share optical devices, and the shooting process also shares an image sensor (for example, the structure light imaging device shown in Figure 4 The nonlinear influencing factors ξ(x, y) of the two projection reconstructions can be considered to be consistent, and the environmental background light μ(x, y) also remains consistent. Therefore, the first background light intensity image and the first modulation light intensity image obtained by projecting the fringe pattern using the first polarized light, and the second background light intensity image and the second modulation light intensity image obtained by projecting the fringe pattern using the second polarized light can be represented as:

[0076] A1'(x, y) = A(x, y) · F1(x, y) · ξ(x, y) + μ(x, y),

[0077] B1'(x, y) = B(x, y) · F1(x, y) · ξ(x, y),

[0078] A'2(x, y) = A(x, y) · F2(x, y) · ξ(x, y) + μ(x, y),

[0079] B'2(x, y) = B(x, y) · F2(x, y) · ξ(x, y),

[0080] wherein A1' represents the first background light intensity image, A1'(x, y) represents the first background light intensity at the pixel position (x, y), A'2 represents the second background light intensity image, A'2(x, y) represents the second background light intensity at the pixel position (x, y), B1' represents the first modulation light intensity image, B1'(x, y) represents the first modulation light intensity at the pixel position (x, y), B'2 represents the second modulation light intensity image, and B'2(x, y) represents the second modulation light intensity at the pixel position (x, y).

[0081] By subtraction, we can obtain:

[0082] ΔA' = A1'- A'2= [F1(x, y)- F2(x, y)]·A·ξ(x, y),

[0083] ΔB' = B1'- B'2= [F1(x, y)- F2(x, y)]·B·ξ(x, y),

[0084] wherein ΔA' represents the difference between the first background light intensity image and the second background light intensity image, and ΔB' represents the difference between the first modulation light intensity image and the second modulation light intensity image.

[0085] Therefore, the reflectivity difference of the two times of polarized light projection reconstruction can be quantitatively represented by the numerical difference of the decoded background light intensity or the decoded modulation light intensity of the two times of reconstruction. Therefore, the reflectivity difference map ΔF can be determined by the following formula:

[0086] ΔF = ΔA' = A1' - A'2, (4)

[0087] ΔF = ΔA' = A1' - A'2, (4)

[0088] ΔF = ΔB' = B1' - B'2, (5)

[0089] Although both of the above can represent the reflectivity difference, when the object surface is translucent, the contrast of the black and white stripe image is too low to cause the decoded modulation light intensity to be too low to accurately reflect the reflectivity, at this time, it is more appropriate to use the decoded background light intensity to represent the reflectivity difference; and when the object surface is a complex textured surface, the decoded background light intensity cannot accurately reflect the reflectivity, the contrast of the black and white stripe image is high, the decoded modulation light intensity is high, and it is more appropriate to use the decoded modulation light intensity to represent the reflectivity difference. In order to improve the stability and applicability of the system, the application preferably adopts a linear weighted method to combine the reflectivity difference:

[0090] ΔF = α·ΔA' + (1 - α)·ΔB', (6)

[0091] Wherein, α is a preset weight value, which is set by the user according to the use scene, and the value range is [0.0, 1.0].

[0092] When the stripe pattern projection is used for three-dimensional reconstruction, the first polarized structured light image obtained includes N first stripe projection images obtained by projecting the stripe pattern on the imaged object by the N-step phase shift method using the first polarized light, and the second polarized structured light image includes N second stripe projection images obtained by projecting the stripe pattern on the imaged object by the N-step phase shift method using the second polarized light.

[0093] Based on the above analysis, in steps 20-30, the first polarized structured light image is used for three-dimensional reconstruction to obtain the first height image of the imaged object, the second polarized structured light image is used for three-dimensional reconstruction to obtain the second height image of the imaged object, and the reflectivity difference map is calculated according to the first polarized structured light image and the second polarized structured light image, including:

[0094] The N first stripe projection images are subjected to phase unwrapping processing to obtain a first phase image, a first background light intensity image and a first modulation light intensity image; the N second stripe projection images are subjected to phase unwrapping processing to obtain a second phase image, a second background light intensity image and a second modulation light intensity image; here, the phase unwrapping processing can use any phase unwrapping method, which is not limited by the application;

[0095] According to the first phase image, three-dimensional reconstruction is performed to obtain a first height image; and according to the second phase image, three-dimensional reconstruction is performed to obtain a second height image; the three-dimensional reconstruction can adopt any three-dimensional reconstruction method, which is not limited in the present application;

[0096] The reflectivity difference map is calculated according to the difference between the first background light intensity image and the second background light intensity image, or according to the difference between the first modulation light intensity image and the second modulation light intensity image, or according to the difference between the first background light intensity image and the second background light intensity image and the difference between the first modulation light intensity image and the second modulation light intensity image. The specific determination formula of the reflectivity difference map is shown in the above formula (4), (5) and (6).

[0097] As to the final height image of the imaged object, it is essentially obtained by fusing the first height image and the second height image according to the reflectivity difference map. In some embodiments, the final height image of the imaged object obtained by fusion is determined by the following formula:

[0098]

[0099] Wherein, H(x, y) represents the height value of the pixel position (x, y) in the final height image of the imaged object, H1(x, y) represents the height value of the pixel position (x, y) in the first height image, H2(x, y) represents the height value of the pixel position (x, y) in the second height image, ΔF(x, y) represents the reflectivity difference degree of the pixel position (x, y) in the reflectivity difference map, T is a preset reflectivity difference degree threshold, and h0 represents an invalid pixel value.

[0100] Please refer to Figure 3 It can be seen that, when facing multiple reflection surfaces, the surface reconstructed by the traditional three-dimensional reconstruction method (Fig. (b)) is jagged, which is not consistent with the actual situation, while the method of the present application can suppress multiple reflections and correctly reconstruct the multiple reflection surface (Fig. (c)), thereby reducing the influence of the multiple reflections of the object surface on the reconstruction accuracy and completeness, improving the completeness and authenticity of three-dimensional reconstruction, and effectively improving the material adaptability and scene adaptability of the three-dimensional imaging system and method.

[0101] In view of the difficult problems of the structured light three-dimensional reconstruction described above, the existing methods usually single-sidedly improve and optimize the optical or algorithm scheme. Some embodiments of the present application also provide a high-practicality, low-cost, high-real-time structured light imaging device, which is designed based on the polarization light imaging technology and has a double-channel polarization light source, can conveniently implement the above three-dimensional imaging method, and can also solve the technical problem of high reflection of the object surface.

[0102] Reference is made to Figure 4 The structured light imaging device in some embodiments includes a projection illuminating device 100 and an imaging device 200, wherein the projection illuminating device 100 adopts a two-channel illuminating light source design, including a first light source 101, a first polarizer 102, a second light source 103, a second polarizer 104, a light combining device 105, and a spatial light modulator 106; the imaging device 200 includes a polarizing detector 201 and an image sensor 202.

[0103] The first light source 101 and the second light source 103 can be LED light sources or the like. The first polarizer 102 is arranged on the light path of the first light beam emitted by the first light source 101, and is used to polarize the first light beam into first polarized light of a first polarization direction; the second polarizer 104 is arranged on the light path of the second light beam emitted by the second light source 102, and is used to polarize the second light beam into second polarized light of a second polarization direction; wherein the first polarized light and the second polarized light can be linearly polarized light, and the first polarization direction is different from the second polarization direction.

[0104] The first light source 101 and the second light source 102 can be turned on or turned off at the same time, or one of them is turned on and the other is turned off.

[0105] The light combining device 105 is arranged at the intersection of the light path of the first polarized light and the light path of the second polarized light, and is used to combine the first polarized light and the second polarized light into the same light path and emit to the spatial light modulator 106 when the first light source 101 and the second light source 102 are both turned on, and emit the first polarized light or the second polarized light to the spatial light modulator 106 when only the first light source 101 is turned on or only the second light source 102 is turned on. The light combining device 105 combines the first polarized light and the second polarized light into the same light path, on the one hand, it makes the light path structure compact, which is conducive to reducing the volume of the projection illuminating device 100, and on the other hand, it makes the first polarized light and the second polarized light share the same optical device and receive the same optical processing, and enter at the same angle.

[0106] As shown in Figure 4 some embodiments, the light path of the first polarized light and the light path of the second polarized light are perpendicular to each other, and the light combining device 105 is a beam splitter (also known as a half-transmission half-reflection mirror). The beam splitter is at a 45° angle with the light path of the first polarized light and the light path of the second polarized light, and the first polarized light and the second polarized light respectively enter the beam splitter from two sides of the beam splitter. One of the polarized lights is transmitted on the beam splitter, and the other polarized light is reflected, thereby being combined into the same light path.

[0107] The spatial light modulator 106 is configured to modulate the light beam (e.g. the light emitted by the light combiner 105) incident thereto into a specific structured light to project the structured light to the imaged object 400, so as to realize structured light projection. In some embodiments, the structured light is configured to project a fringe pattern, e.g. a fringe pattern projected by the N-step phase shift method, and the imaging device 200 acquires N fringe projection images of the imaged object 400 for three-dimensional reconstruction. The user can set the projection pattern according to actual needs and input the structured light imaging device to control the spatial light modulator 106 to modulate the light beam into structured light for projecting the pattern. The spatial light modulator 106 can be a digital micromirror device (DMD), an LCD or an LCOS, etc.

[0108] The imaged object can be a product or workpiece in industrial production, a mechanical part, an electronic component, etc., which are not limited in the present application.

[0109] The polarizer 201 is configured to receive the structured reflected light formed by the structured light reflected by the imaged object 400, and project the structured reflected light onto the image sensor 202 after polarization detection for imaging. The image sensor 202 can be a CMOS sensor or a CCD sensor, etc.

[0110] The structured light imaging device of the present application, since the projection illumination device 100 uses polarized light for projection, and the polarization state of the reflected light will change to a certain extent after the polarized light is reflected by the surface of the object, the high reflection light and glare light of a specific polarization direction can be filtered out by the polarizer 201 during imaging, thereby improving the three-dimensional reconstruction performance of the high reflection object and improving the reconstruction integrity and material adaptability. Preferably, the polarization direction of the polarizer 201 is different from the first polarization direction and the second polarization direction, so as to better filter out the high reflection light and glare light when using two kinds of polarized light for projection.

[0111] In addition, considering that in the existing method of adding a polarizer in front of the imaging lens to solve the high reflection problem, the high reflection area is weakened and eliminated, but the other diffuse reflection area is also weakened and the light intensity is insufficient, in the present application, the light combiner 105 is provided, which can combine the first polarized light and the second polarized light into one polarized light, and the control device 300 can control the first light source 101 and the second light source 102 to work simultaneously, so as to use two kinds of polarized light to project the imaged object 400 simultaneously. Since the polarization direction of the first polarized light is different from the polarization direction of the second polarized light, it is almost impossible to be completely filtered out by the polarizer 201 at the same time, so as to weaken the high reflection area and complement the light intensity of the diffuse reflection area, thereby improving the final three-dimensional reconstruction integrity.

[0112] Preferably, the first polarization direction is perpendicular to the second polarization direction. For example, the first polarizer 103 polarizes in the horizontal direction (referred to as S-polarization), and the second polarizer 104 polarizes in the vertical direction (referred to as P-polarization), which will be used as an example in the following description. Further, the analyzing direction of the analyzer 201 forms a 45° angle with both the first and second polarization directions, such as... Figure 5 As shown. The polarization angle of the analyzer 201 can, to a certain extent, eliminate the high reflectivity and glare caused by the first polarized light (S-polarized light) source illuminating the object surface, and also eliminate the high reflectivity and glare caused by the second polarized light (P-polarized light) source illuminating the object surface. Furthermore, even in extreme cases, when the first polarized light (S-polarized light) source reflects off the object surface, the polarization direction changes and becomes completely perpendicular to the polarization direction of the analyzer 201 of the imaging device 200. The light cannot pass through the analyzer 201, and the first polarized light (S-polarized light) cannot form an image at all. However, when the second polarized light (P-polarized light) reflects off the object surface, the reflected light is almost impossible to be perpendicular to the polarization direction of the analyzer 201 and can still pass through the analyzer 201, achieving illumination imaging. Under normal circumstances, the first polarized light (S-polarized light) source and the second polarized light (P-polarized light) source form complementary illumination in the diffuse reflection region. It should be noted that after the analyzer in this application is assembled during the system assembly stage, the polarization angle does not need to be rotated again during subsequent use.

[0113] Please refer to Figure 6 In some embodiments, the projection illumination device 100 further includes a first collimating lens group 108 and a second collimating lens group 109. The first collimating lens group 108 is disposed between the first light source 101 and the first polarizer 103, and is used to collimate the first light beam and project it onto the first polarizer 103; the second collimating lens group 109 is disposed between the second light source 102 and the second polarizer 104, and is used to collimate the second light beam and project it onto the second polarizer 104.

[0114] Typically, a collimating lens placed behind the light source can collimate the light beam emitted by the source. While a polarizer can theoretically achieve light polarization by being placed anywhere in the optical path of the projection lighting device 100, the transmittance varies depending on its placement, resulting in significant differences in the overall light transmission efficiency of the projection lighting device 100. For example... Figure 7As shown, if the incident angles are all summarized in the range of 0° to 90°, the transmittance of the polarizer decreases with the decrease of the incident angle of the light, and the transmittance is the highest when the incident angle is 90°. Most of the currently disclosed methods place the polarizer in front of the projection lens, and the projection lens is a FA lens, and the light has a fixed field of view. At the edge of the field of view, the incident angle is small, and the transmittance of the light is low, which affects the final light efficiency and uniformity. The polarizer is placed behind the collimating lens group of the light source in the present application. The light passing through the collimating lens group can ensure good collimation, the incident light angles of each field of view are close to 90°, the transmittance of the light passing through the polarizer is close to the highest, and the light efficiency and projection uniformity are greatly improved.

[0115] Please refer to Figure 4 and Figure 6 In some embodiments, the projection illumination device 100 further comprises a projection lens group 107 (also referred to as a projection lens), which is arranged on the light path of the structured light and is used to project the structured light onto the imaged object 400. The projection lens group 107 adopts a common FA design or a telecentric design.

[0116] Generally, the projection object plane is not perpendicular to the principal axis of the projection lens 107, but forms a certain angle with the principal axis of the projection lens 107, for example, an angle of 60°, that is, the principal axis of the projection lens 107 forms an angle of 30° with the principal axis of the imaging device 200 (of course, other angles can also be used, and the value range is generally [27°, 30°]). In order to ensure the definition of the projection object plane, the spatial light modulator 106 is placed at a certain angle in the light path design, so that the projection image plane of the spatial light modulator 106, the principal plane of the projection lens 107, and the projection object plane satisfy the Scheimpflug principle. As shown in Figure 8 The projection image plane extension line, the projection lens principal plane extension line, and the projection object plane extension line intersect at a point, so that even if the projection lens 107 is installed at an angle, a clear projection pattern can still be obtained on the object plane.

[0117] Please refer to Figure 4 and Figure 6 In some embodiments, the imaging device 200 further comprises an imaging lens group 203 (also referred to as an imaging lens), and the polarizer 201 is arranged between the imaging lens group 203 and the image sensor 202. The imaging lens group 203 is used to receive the structured reflected light formed by the structured light reflected by the imaged object 400, and converge and project the structured reflected light to the polarizer 201.

[0118] The imaging lens 203 preferably adopts an image-side telecentric design or a double telecentric design. The polarizer 201 is arranged between the imaging lens 203 and the image sensor 202, and the imaging lens 203 adopts an image-side telecentric lens or a double telecentric lens, so as to ensure that the incident light passing through the polarizer 201 is as close to 90° as possible, thereby ensuring high light transmittance and imaging uniformity on the imaging side, and under the same illumination brightness condition, the exposure time of the image sensor 202 is shorter, the photographing speed of the structured light three-dimensional imaging device is faster, and the real-time performance is higher.

[0119] Please refer to Figure 6 The projection illumination device 100 in some embodiments further comprises a reflector 110, which is arranged on the light path of the light emitted by the light combining device 105, and is used for reflecting the light emitted by the light combining device 105 to the spatial light modulator 106. The angle θ between the normal of the reflecting surface of the reflector 110 and the light emitted by the light combining device 105 is in the range of 0°<θ<90°, so that the light path of the light emitted by the light combining device 105 is turned.

[0120] Please refer to Figure 6 The projection illumination device 100 in some embodiments further comprises a light homogenizing device 111, which is arranged on the light path of the light emitted by the light combining device 105 and incident to the reflector 110, and is used for projecting the light emitted by the light combining device 105 to the reflector 110 after homogenizing. The light homogenizing device 111 can adopt a double-sided micro-lens array. The arrangement of the light homogenizing device 111 makes the light more uniform.

[0121] Please refer to Figure 6 The projection illumination device 100 in some embodiments further comprises a first relay lens 112, which is arranged on the light path of the light emitted by the light homogenizing device 111 and incident to the reflector 110, and is used for projecting the light emitted by the light homogenizing device 111 to the reflector 110 after converging. Since the light beam is more divergent after passing through the light homogenizing device 111, the first relay lens 112 is arranged in this embodiment to converge the light beam. For example, the light emitted by the light combining device 105 is cut into spots by the double-sided micro-lens array, and then converged by the first relay lens 112, so that the light is more concentrated.

[0122] Please refer to Figure 6 The projection illumination device 100 in some embodiments further comprises a second relay lens 113, which is arranged on the light path of the light reflected by the reflector 110 and incident to the spatial light modulator 106, and is used for projecting the light reflected by the reflector 110 to the spatial light modulator 106 after converging. The light beam is converged again here to improve the converging effect.

[0123] From Figure 6As can be seen, the mirror 110 turns the originally straight light path, so that the optical devices can be more concentrated and compactly arranged in the projection illumination device 100.

[0124] Please refer to Figure 6 The projection illumination device 100 in some embodiments further comprises a compound prism 114, which is arranged in front of the spatial light modulator 106, so that the light emitted by the second relay lens 113 is incident on the spatial light modulator 106 through the compound prism 114, and the structured light emitted by the spatial light modulator 106 is reflected to the imaged object 400 through the compound prism 114.

[0125] The propagation path of the light in the compound prism 114 is shown in Figure 6 In some embodiments, the positions of the spatial light modulator 106 and the compound prism 114 are arranged in cooperation with the mirror 110, so that the direction of the structured light emitted by the compound prism 114 is opposite to the direction of the light emitted by the light combining device 105, so that the overall light path of the projection illumination device 100 is in a form of turn-back, so that the structure of the projection illumination device 100 is more compact, and miniaturization is achieved. It should be noted that at this time, the projection image plane is not on the spatial light modulator 106, but on the reflection surface of the structured light in the compound prism 114.

[0126] According to the on-off state of the first light source 101 and the second light source 102, the projection illumination device 100 can realize at least three projection modes: single polarization projection mode: single channel illumination light source is turned on, that is, only one of the first light source 101 and the second light source 102 is turned on; double polarization projection mode: double channel illumination light source is turned on at the same time, that is, the first light source 101 and the second light source 102 are turned on, and the first polarized light and the second polarized light are used for projection at the same time, and this mode can realize the complementation of the double polarized light source; alternating projection mode: double channel illumination light source is turned on in turn, that is, the projection illumination device 100 performs two structured light projections, one time the first light source 101 is turned on and the second light source 102 is turned off, and the first polarized light is used for projection, and the other time the first light source 101 is turned off and the second light source 102 is turned on, and the second polarized light is used for projection, and this mode can realize the above-mentioned three-dimensional imaging method and suppress multiple reflections.

[0127] Based on the structured light imaging device of the present application, the first polarized structured light image and the second polarized structured light image of the imaged object obtained in step 10 above include: controlling the projection illumination device 100 to project structured light on the imaged object in the alternating projection mode, and in the alternating projection mode, the projection illumination device performs two structured light projections, and the imaging device 200 captures the imaged object to obtain the first polarized structured light image and the second polarized structured light image.

[0128] The user can input an instruction to select a projection mode, which corresponds to a single-polarization projection mode, a double-polarization projection mode, and an alternating projection mode, and is respectively a first imaging instruction, a second imaging instruction, and a third imaging instruction.

[0129] In some embodiments, the three-dimensional imaging method of the present application further includes:

[0130] When the first imaging instruction input by the user is received, the projection illumination device 100 is controlled to project structured light on the imaged object in the single-polarization projection mode, the imaging device 200 is triggered to capture the imaged object, a fifth polarized structured light image of the imaged object is obtained, three-dimensional reconstruction is performed using the fifth polarized structured light image, and a height image of the imaged object is obtained; the three-dimensional reconstruction here can use any three-dimensional reconstruction method, which is not limited by the present application;

[0131] When the second imaging instruction input by the user is received, the projection illumination device 100 is controlled to project structured light on the imaged object in the double-polarization projection mode, the imaging device 200 is triggered to capture the imaged object, a sixth polarized structured light image of the imaged object is obtained, three-dimensional reconstruction is performed using the sixth polarized structured light image, and a height image of the imaged object is obtained; the three-dimensional reconstruction here can use any three-dimensional reconstruction method, which is not limited by the present application;

[0132] When the third imaging instruction input by the user is received, steps 10-40 are executed to obtain a height image of the imaged object.

[0133] The user can select a projection mode according to the actual scene needs. In a scene with high real-time requirements, the single-polarization projection mode with only a single-channel polarized light source turned on can be selected; in a scene with multiple reflections, the alternating projection mode can be selected; in a high-reflective and dazzling light scene, the double-polarization projection mode with both double-channel polarized light sources turned on can be selected.

[0134] Hereinafter, taking the first polarized light as S-polarized light and the second polarized light as P-polarized light as an example, the steps of projecting a stripe pattern in the alternating projection mode using the structured light imaging device of the present application and performing the three-dimensional imaging method of the present application are described. As shown in Figure 9 The polarization directions of the S-polarized light and the P-polarized light are perpendicular, and after passing through the light combining device 105, they propagate along the same light path, Figure 9 The S-polarized light and the P-polarized light are shown at the same time for convenience of display, but it does not mean that the S-polarized light and the P-polarized light must be turned on at the same time. Please refer to Figure 10 The specific steps include:

[0135] 1. Turn on the first light source 101 (i.e. turn on the S-polarized light source), turn off the second light source 102 (i.e. turn off the P-polarized light source) to project the fringe pattern, and collect N first fringe projection images obtained by S-polarized light projection through the image sensor 202, and then perform phase unwrapping to obtain the first phase image, the first background light intensity image and the first modulation light intensity image;

[0136] 2. Turn on the second light source 102 (i.e. turn on the P-polarized light source), turn off the first light source 101 (i.e. turn off the S-polarized light source) to project the fringe pattern, and collect N second fringe projection images obtained by P-polarized light projection through the image sensor 202, and then perform phase unwrapping to obtain the second phase image, the second background light intensity image and the second modulation light intensity image;

[0137] 3. Perform three-dimensional reconstruction on the first phase image and the second phase image respectively through the geometric constraint relationship obtained by pre-calibration to obtain the first height image and the second height image;

[0138] 4. Calculate the polarization reflectivity difference according to the difference between the first background light intensity image and the second background light intensity image, and the difference between the first modulation light intensity image and the second modulation light intensity image to obtain the reflectivity difference image, see formula (6) for details;

[0139] 5. Perform threshold comparison according to the reflectivity difference image to determine the pixel area with multiple reflections, set the corresponding height value as the invalid pixel value, thereby eliminating the abnormal data caused by multiple reflections, and finally obtain the height image with suppressed multiple reflections by fusion, see formula (7) for details.

[0140] Please refer to Figure 11In some embodiments, the structured light imaging device can further be provided with a control device 300, which can include an embedded control system composed of a MCU (Microcontroller Unit), a SOC (System on Chips), a DSP (Digital Signal Processor) and / or a FPGA (Field Programmable Gate Array), and is mainly responsible for the functions of synchronization signal control, image processing and data transmission. The three-dimensional imaging method of the embodiments of the present application can be applied to the control device 300. The control device 300 can control the opening and closing of the two-channel light source and the pattern projected by the spatial light modulator 106. The user can control the structured light imaging device by sending instructions to the control device 300 through the host computer, and the control device 300 starts the projection and shooting of the projection illumination device 100 and the imaging device 200 according to the signal sent by the host computer. The shooting signal sent by the control device 300 to the imaging device 200 and the exposure time of the imaging device 200 need to be synchronized with the projection signal sent to the projection illumination device 100 and the projection time of the projection illumination device 100. After a series of projection and shooting of structured light patterns, the control device 300 collects the images imaged by the image sensor 200, starts the image algorithm for reconstruction and fusion, and finally obtains the height image of the imaged object 400, and then transmits the height image to the host computer.

[0141] Please refer to Figure 11 and Figure 12 In some embodiments, the control device 300 includes a projection control module for controlling the projection illumination device 100. The projection control module receives the control signal of the control device 300 and controls the two-channel light source (the first light source 101 and the second light source 102) and the spatial light modulator 106. When there are multiple projection illumination devices 100, each projection illumination device 100 is controlled by a corresponding projection control module, for example, Figure 12 one projection illumination device 100 is controlled by a first projection control module, and the other projection illumination device 100 is controlled by a second projection control module. Please refer to Figure 13 In some embodiments, the projection control module includes a first light source control module, a second light source control module and a light modulation control module. The first light source control module and the second light source control module control the on-off enable and current size of the two-channel light source, and change the projection brightness of the projection illumination device 100 by adjusting the current size of the light source. The light modulation control module can control the pattern projected by the spatial light modulator 106.

[0142] For the problem of visual field blind area, the structured light imaging device of some embodiments of the present application adopts double projection illumination devices to irradiate the imaged object from two different incident angles, as shown in Figure 12 Another three-dimensional imaging method is proposed on this basis to remove part of the visual field blind area through image fusion. Specifically, the two projection illumination devices are symmetrically arranged around the imaging device, and the two projection illumination devices can project and illuminate the imaged object from two different incident angles, thereby solving the problem of reconstruction blind area caused by illumination blind area. Please refer to Figure 14 When using a single projection illumination device for reconstruction, the A area in the figure is the camera shooting blind area, which cannot be shot even if it can be illuminated, resulting in the inability to reconstruct; the B area in the figure is the illumination blind area, which can be shot by the camera but cannot be illuminated, resulting in the inability to reconstruct. The embodiment adopts double projection illumination devices for projection and illumination, and through two projection reconstructions by the projection illumination devices placed in symmetric directions, the illumination blind area of the B area in the figure can be removed after fusion.

[0143] Please refer to Figure 15 The three-dimensional imaging method based on the structured light imaging device includes steps 100-500.

[0144] Step 100: Control one of the two projection illumination devices to project structured light on the imaged object with first polarized light, trigger the imaging device to shoot the imaged object, and obtain a first polarized structured light image; control the projection illumination device to project structured light on the imaged object with second polarized light, trigger the imaging device to shoot the imaged object, and obtain a second polarized structured light image.

[0145] That is, control one of the two projection illumination devices to project structured light in an alternating projection mode, and obtain a first polarized structured light image and a second polarized structured light image through the imaging device.

[0146] Step 200: Perform three-dimensional imaging using the first polarized structured light image and the second polarized structured light image to obtain a first projection direction height image.

[0147] The three-dimensional imaging here can be realized according to the three-dimensional imaging method based on structured light in any of the preceding embodiments of the present application.

[0148] Step 300: Control the other of the two projection illumination devices to project structured light on the imaged object with first polarized light, trigger the imaging device to shoot the imaged object, and obtain a third polarized structured light image; control the projection illumination device to project structured light on the imaged object with second polarized light, trigger the imaging device to shoot the imaged object, and obtain a fourth polarized structured light image.

[0149] That is, controlling the other one of the two projection illumination devices to project structured light in an alternating projection mode, and obtaining a third polarized structured light image and a fourth polarized structured light image through imaging by the imaging device.

[0150] Step 400: three-dimensional imaging using the third polarized structured light image and the fourth polarized structured light image to obtain a second projection direction height image.

[0151] The three-dimensional imaging here can be implemented according to the structured light-based three-dimensional imaging method in any of the preceding embodiments of the present application.

[0152] Step 500: fusing the first projection direction height image and the second projection direction height image to obtain a height image of the imaged object.

[0153] In some embodiments, the height image of the imaged object is determined by the following formula:

[0154]

[0155] wherein I h (x,y) represents the height value at pixel position (x,y) in the height image of the imaged object, H 1d (x,y) represents the height value at pixel position (x,y) in the first projection direction height image, H 2d (x,y) represents the height value at pixel position (x,y) in the second projection direction height image.

[0156] Of course, the projection illumination device used in the present embodiment can also not be limited to Figure 12 the projection illumination device shown in the figure, as long as the projection illumination device can project structured light in the first polarized light and the second polarized light, and the polarization directions of the first polarized light and the second polarized light are different and the incident angles are the same. Similarly, other imaging devices can also be used for the imaging device, and the imaging device is not limited to Figure 12 the imaging device shown in the figure.

[0157] In the present embodiment, the structured light imaging device irradiates the imaged object from two different incident angles by setting two projection illumination devices that are relatively symmetrical with the imaging device as the center, removes the illumination blind area, and also provides a way to fuse the height images reconstructed by the two projection illumination devices, further improving the reconstruction integrity.

[0158] Those skilled in the art can understand that all or part of the functions of various methods in the above embodiments can be realized by hardware or by a computer program. When all or part of the functions in the above embodiments are realized by a computer program, the program can be stored in a computer readable storage medium, which can include a read-only memory, a random access memory, a magnetic disk, an optical disk, a hard disk, and the like. The above functions are realized by executing the program by a computer. For example, the program is stored in a memory of a device, and the above functions are realized by executing the program in the memory by a processor. In addition, when all or part of the functions in the above embodiments are realized by a computer program, the program can also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash disk, or a mobile hard disk, and is saved in a memory of a local device by downloading or copying, or the system of the local device is updated, and the above functions are realized by executing the program in the memory by a processor.

[0159] The above application of specific examples to the present application is described, which is only used to help understand the present application and does not limit the present application. According to the idea of the present application, those skilled in the art can make several simple deductions, deformations, or substitutions.

Claims

1. A three-dimensional imaging method based on structured light, characterized in that, include: Acquire a first polarized structured light image and a second polarized structured light image of the object being imaged; wherein, the first polarized structured light image is an image captured on the object being imaged when structured light projection is performed on the object being imaged using first polarized light, and the second polarized structured light image is an image captured on the object being imaged when structured light projection is performed on the object being imaged using second polarized light, wherein the polarization directions of the first polarized light and the second polarized light are different, and the first polarized light and the second polarized light are incident on the surface of the object being imaged at the same incident angle; A first height image of the imaged object is obtained by performing three-dimensional reconstruction using the first polarized structured light image; a second height image of the imaged object is obtained by performing three-dimensional reconstruction using the second polarized structured light image. A reflectance difference map is calculated based on the first polarized structured light image and the second polarized structured light image. The pixel value of each pixel position in the reflectance difference map represents the degree of reflectance difference at that pixel position. The degree of reflectance difference represents the degree of difference between the reflectance of the surface of the imaged object corresponding to that pixel position to the first polarized light and the reflectance to the second polarized light. Based on the reflectivity difference map, it is determined whether there is multiple reflection on the surface of the object being imaged corresponding to each pixel position. If so, the height value of the pixel position is set as an invalid pixel value. If not, the final height value of the pixel position is determined based on the height values ​​of the first height image and the second height image at that pixel position, so as to obtain the final height image of the object being imaged.

2. The three-dimensional imaging method as described in claim 1, characterized in that, The first polarized structured light image includes N first fringe projection images obtained by projecting a fringe pattern onto the imaged object using the first polarized light in an N-step phase-shifting method, and the second polarized structured light image includes N second fringe projection images obtained by projecting a fringe pattern onto the imaged object using the second polarized light in an N-step phase-shifting method, where N is an integer not less than 2. The process involves performing 3D reconstruction using the first polarized structured light image to obtain a first height image of the imaged object, performing 3D reconstruction using the second polarized structured light image to obtain a second height image of the imaged object, and calculating a reflectance difference map based on the first and second polarized structured light images, including: Phase-de-phase processing is performed on N first fringe projection images to obtain a first phase image, a first background light intensity image, and a first modulated light intensity image; phase-de-phase processing is performed on N second fringe projection images to obtain a second phase image, a second background light intensity image, and a second modulated light intensity image; Three-dimensional reconstruction is performed based on the first phase image to obtain the first height image; three-dimensional reconstruction is performed based on the second phase image to obtain the second height image; The reflectance difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, or the reflectance difference map is calculated based on the difference between the first modulated light intensity image and the second modulated light intensity image, or the reflectance difference map is calculated based on the difference between the first background light intensity image and the second background light intensity image, and the difference between the first modulated light intensity image and the second modulated light intensity image.

3. The three-dimensional imaging method as described in claim 2, characterized in that, The reflectance difference map is determined by the following formula: ΔF = ΔA' = A1' - A'2, Alternatively, ΔF = ΔB' = B'1 - B'2. Or, ΔF=αΔA'+(1-α)·ΔB'; Wherein, ΔF represents the reflectance difference map, ΔA' represents the difference between the first background light intensity image and the second background light intensity image, ΔB' represents the difference between the first modulated light intensity image and the second modulated light intensity image, A1' represents the first background light intensity image, A'2 represents the second background light intensity image, B1' represents the first modulated light intensity image, B'2 represents the second modulated light intensity image, and α is a preset weight value.

4. The three-dimensional imaging method as described in claim 3, characterized in that, The value of α ranges from [0.0, 1.0].

5. The three-dimensional imaging method according to any one of claims 1 to 4, characterized in that, The final height image of the imaged object is determined by the following formula: Wherein, H(x,y) represents the height value at pixel position (x,y) in the final height image of the imaged object, H1(x,y) represents the height value at pixel position (x,y) in the first height image, H2(x,y) represents the height value at pixel position (x,y) in the second height image, ΔF(x,y) represents the degree of reflectance difference at pixel position (x,y) in the reflectance difference image, T is a preset threshold for the degree of reflectance difference, and h0 represents an invalid pixel value.

6. The three-dimensional imaging method as described in claim 1, characterized in that, The polarization directions of the first polarized light and the second polarized light are perpendicular.

7. The three-dimensional imaging method as described in claim 1, characterized in that, The three-dimensional imaging method is implemented based on a structured light imaging device, which includes a projection illumination device and an imaging device. The projection illumination device includes a first light source, a first polarizer, a second light source, a second polarizer, a light combining device, and a spatial light modulator. The imaging device includes an analyzer and an image sensor. The first polarizer is disposed in the optical path of the first beam emitted by the first light source, and is used to polarize the first beam into the first polarized light with the first polarization direction. The second polarizer is disposed in the optical path of the second beam emitted by the second light source, and is used to polarize the second beam into second polarized light with a second polarization direction; wherein the first polarization direction is different from the second polarization direction. The first light source and the second light source can be turned on or off simultaneously, or one can be turned on while the other is off. The light combining device is located at the intersection of the optical path of the first polarized light and the optical path of the second polarized light. When both the first light source and the second light source are turned on, it combines the first polarized light and the second polarized light into the same optical path and emits them to the spatial light modulator. When only the first light source or only the second light source is turned on, it emits the first polarized light or the second polarized light to the spatial light modulator. The spatial light modulator is used to modulate the incident light beam into a specific structured light to project the structured light onto the imaged object; The analyzer is used to receive the structure-reflected light formed by the structure light being reflected by the object being imaged, and to perform polarization analysis on the structure-reflected light before projecting it onto the image sensor for imaging; The acquisition of the first polarization structured light image and the second polarization structured light image of the imaged object includes: The projection illumination device is controlled to project structured light onto the object being imaged in an alternating projection mode. In the alternating projection mode, the projection illumination device performs two structured light projections, one in which the first light source is turned on and the second light source is turned off, and the other in which the first light source is turned off and the second light source is turned on. Each time a projection occurs, the imaging device is triggered to take a picture of the object being imaged, thereby obtaining the first polarized structured light image and the second polarized structured light image.

8. A three-dimensional imaging method based on a structured light imaging device, characterized in that, The structured light imaging device includes an imaging unit and two projection illumination units, which are symmetrically arranged around the imaging unit. The projection illumination units are capable of projecting structured light using first polarized light and second polarized light, wherein the polarization directions of the first polarized light and the second polarized light are different. The three-dimensional imaging method includes: One of the two projection lighting devices is controlled to project structured light onto the object being imaged using the first polarized light, triggering the imaging device to capture the object being imaged, thereby obtaining a first polarized structured light image; the same projection lighting device is then controlled to project structured light onto the object being imaged using the second polarized light, triggering the imaging device to capture the object being imaged, thereby obtaining a second polarized structured light image. According to the three-dimensional imaging method as described in any one of claims 1 to 6, three-dimensional imaging is performed using the first polarized structured light image and the second polarized structured light image to obtain a first projection direction height image; Control the other of the two projection lighting devices to project structured light onto the object being imaged with the first polarized light, trigger the imaging device to capture the object being imaged, and obtain a third polarized structured light image; control the projection lighting device to project structured light onto the object being imaged with the second polarized light, trigger the imaging device to capture the object being imaged, and obtain a fourth polarized structured light image. According to the three-dimensional imaging method as described in any one of claims 1 to 6, three-dimensional imaging is performed using the third polarized structured light image and the fourth polarized structured light image to obtain a second projection direction height image; The height image of the first projection direction and the height image of the second projection direction are fused together to obtain the height image of the imaged object.

9. The three-dimensional imaging method as described in claim 8, characterized in that, The height image of the imaged object is determined by the following formula: Among them, I h (x,y) represents the height value at pixel position (x,y) in the height image of the imaged object, H 1d (x,y) represents the height value at pixel position (x,y) in the first projection direction height image, H 2d (x,y) represents the height value at pixel position (x,y) in the second projection direction height image, and h0 represents an invalid pixel value.

10. A computer-readable storage medium, characterized in that, The medium stores a program that can be executed by a processor to implement the three-dimensional imaging method as described in any one of claims 1 to 9.

Images