Speckle-assisted binocular structured light dynamic three-dimensional reconstruction system

Abstract

The invention provides a speckle-assisted binocular structured light dynamic three-dimensional reconstruction system, which is characterized in that a projector projects a designed sinusoidal grating stripe and speckle composite color pattern to a measured object, meanwhile, a left camera and a right camera collect color images modulated by the surface of the measured object, the color images are separated, and the speckle-assisted binocular structured light dynamic three-dimensional reconstruction system is obtained. The method comprises the following steps: carrying out improved variational mode decomposition (VMD) phase extraction operation on a separated fringe pattern to obtain accurate fringe information and background information, and then calculating a wrapped phase through Hilbert transform and an arc tangent formula. Then homonymy points are obtained based on speckle auxiliary matching, the initial depth is obtained through binocular speckle triangulation, the optimal stripe level is calculated through unification of the binocular speckle depth and the monocular stripe structure light depth, and after the stripe levels of all the points are calculated, the final unwrapped phase can be calculated through a phase unwrapping formula; accurate three-dimensional information of the object can be calculated according to the calibration parameters and the unwrapping phase.

Description

Technical Field

[0001] The present invention belongs to the field of computer three-dimensional vision technology, and specifically relates to a speckle-assisted binocular structured light dynamic three-dimensional reconstruction system. Background Art

[0002] Binocular structured light (BINOCULAR STRUCTURED LIGHT) combines traditional binocular vision and structured light projection techniques. It is a key method for structured light 3D measurement, offering advantages such as full-field, high precision, speed, and non-contact performance. This method projects structured light onto a target surface and records its deformation. Through structured light reconstruction and binocular stereo vision algorithms, the 3D shape of the target object is reconstructed. Binocular structured light 3D measurement technology is rapidly developing and is increasingly being applied in intelligent manufacturing, robotic vision, reverse engineering, human-computer interaction, and other fields. Currently, mainstream dynamic 3D measurement methods mostly use multi-frequency heterodyning of multiple fringe patterns for 3D reconstruction, which requires long projection times. The recently proposed Gray code phase shifting technique reduces the number of projected fringe frames but still requires multiple image acquisitions and places high demands on both the camera acquisition and projection frame rates. Fourier transform profilometry or quality-oriented algorithms can use a single fringe pattern for 3D reconstruction, shortening the projection time. However, this approach suffers from lower accuracy and is difficult to apply to complex scenes with multiple isolated objects. Therefore, binocular structured light still faces challenges in dynamic 3D reconstruction, such as accurate reconstruction in complex scenes, interference of moving objects on reconstruction algorithms, and the cost-performance balance of hardware equipment. Solving these difficulties will promote dynamic 3D reconstruction technology to a wider range of applications. Summary of the Invention

[0003] In view of this, the present invention aims to propose a speckle-assisted binocular structured light 3D reconstruction system, which is used to combine binocular vision with structured light, and improve the accuracy and stability of 3D reconstruction by making full use of the stereo information of binocular vision and the depth information of structured light.

[0004] To achieve the above objectives, the present invention proposes a speckle-assisted binocular structured light dynamic 3D reconstruction system, comprising the following steps:

[0005] S1: Build a speckle-assisted binocular structured light dynamic 3D reconstruction system;

[0006] S2: Calibrate the binocular speckle system, camera and projector system and the two system coordinate systems;

[0007] S3: The projector projects a designed sinusoidal grating fringe and speckle composite color pattern onto the object under test, while the left and right cameras capture the color image modulated by the surface of the object under test;

[0008] S4: Separate the color image to obtain the fringe pattern and speckle pattern. Apply the phase extraction algorithm based on the improved variational mode decomposition of the shearlet transform to the fringe pattern to obtain the wrapped phase. Use the speckle-assisted binocular structured light 3D measurement method based on the depth constraint of the monocular and binocular systems to calculate the unwrapped phase and 3D information.

[0009] S5: Calculate the three-dimensional information of the object based on the camera-projector calibration results and the unfolded phase.

[0010] Furthermore, a designed sinusoidal grating fringe and speckle composite color pattern is characterized in that: a single color image includes a sinusoidal grating fringe pattern with a frequency of 64 and a single speckle pattern;

[0011]

[0012] in, Function represents the primitive of speckle, (x i，j ,y i，j ) represents the center position of the speckle, and M×N represents the total number of speckles.

[0013] Furthermore, the color image separation in step S4 is specifically as follows:

[0014] According to the color channel, the red channel in imgL is separated into the stripe part imgL_Fringe, and the blue channel in imgL is separated into the speckle part imgL_Speckle

[0015] Furthermore, the improved variational mode decomposition phase extraction algorithm described in step S4 is specifically:

[0016] Normalize the image imgL_Fringe and take the mean. Then use the variational mode method to extract the intrinsic mode function u from imgL_Fringe. k , where u1 is the background part and u2 is the stripe part. Perform shearlet denoising on the background part u1 to get denResult. The shearlet denoising formula is:

[0017]

[0018] Among them, SH ψ (·)and where are the forward and inverse shearlet transforms, shrink is the threshold shrinkage function, and thr is the threshold parameter. The following formula is used to further optimize the fringe information to obtain more accurate fringe information FinalV.

[0019] FinalV=(u1+u2)-denResult

[0020] The Hilbert transform is used to analyze the phase information in the stripe part FinalV, and then the arc tangent formula is used to extract the wrapped phase information wrapL.

[0021] Furthermore, the speckle-assisted binocular structured light 3D measurement algorithm described in step S4 is specifically as follows:

[0022] The wrapping phase wrapL and wrapR of the left and right cameras are calculated; the epipolar line of a point p1 in the wrapped phase of the left camera in the wrapped phase of the right camera is calculated according to the epipolar line constraint; several points on the epipolar line that are close to the wrapped phase of p1 are found, and these points are stereo matched based on the speckle pattern to find the point p2 with the minimum cost, which is the synonym of p1; the three-dimensional coordinates P(x, y, z) of the spatial point are calculated by triangulation based on p1 and p2; assuming that p1 is located at the kth fringe order, the unwrapped phase of point p1 is obtained according to the phase unwrapping formula; the three-dimensional information is calculated by the unwrapped phase and the camera projector calibration parameters, and the depth constraint is performed with the three-dimensional coordinates P(x, y, z) of the binocular speckle triangulation to find the optimal fringe order k; the unwrapped phase is calculated by the phase unwrapping formula, and the three-dimensional information P′(x, y, z′) of the object is calculated based on the calibration results.

[0023] Compared with the existing technology, the speckle-assisted binocular structured light 3D measurement system described in the present invention has the following advantages:

[0024] (1) The present invention combines binocular vision with structured light, and improves the accuracy and stability of three-dimensional reconstruction by making full use of the stereo information of binocular vision and the surface depth information of structured light.

[0025] (2) The improved variational mode decomposition phase extraction algorithm proposed in the present invention can obtain a more accurate and concise wrapped phase.

[0026] (3) The speckle-assisted binocular structured light 3D measurement algorithm proposed in this paper can achieve fast phase unwrapping with only one wrapped phase and one speckle pattern, and is not limited by camera resolution. It also has good accuracy and stability when performing phase unwrapping on multiple isolated objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which constitute part of the present invention, are provided to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are provided to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0028] Figure 1 This is a flow chart of a speckle-assisted binocular structured light three-dimensional measurement system of the present invention;

[0029] Figure 2 This is a system architecture diagram of the present invention;

[0030] Figure 3 Get a flow chart for the wrap phase;

[0031] Figure 4 It is a color image modulated by the object being measured;

[0032] Figure 5 is the depth map of the object;

[0033] Figure 6 is the 3D point cloud of the reconstructed object; DETAILED DESCRIPTION

[0034] The present invention proposes a speckle-assisted binocular structured light three-dimensional measurement system, which is described in more detail below with reference to the accompanying drawings and specific embodiments.

[0035] In this embodiment, the following steps are included:

[0036] Step 1: Use a computer, two cameras, and a projector to build a speckle-assisted binocular structured light dynamic 3D reconstruction system. Then, use Zhang Zhengyou's calibration method to calibrate the binocular cameras to obtain the transformation relationship between the left and right cameras. At the same time, use the phase method to calibrate the projector and the left and right cameras separately to obtain the transformation relationship between the projector and the left camera and the projector and the right camera.

[0037] Step 2: The computer designs the color coding pattern, and controls the projector to project the color pattern by running the computer trigger acquisition program, and controls the left and right cameras to acquire the pattern synchronously. The left and right cameras will capture the pattern modulated by the object being measured as shown in the attached figure. Figure 4 Transfer to your computer.

[0038] Step 3: As attached Figure 4 The color pattern is separated by computer to obtain fringe pattern and speckle pattern. The fringe pattern is processed by the following method: Figure 3 The improved variational mode decomposition phase extraction algorithm shown in the figure obtains the wrapped phase, and the wrapped phase and the speckle pattern are used to obtain the unwrapped phase according to the speckle-assisted binocular structured light three-dimensional measurement algorithm.

[0039] Step 4: Calculate the 3D information of the object based on the unwrapped phase and camera projector calibration results as shown in the attached figure. Figure 5 The depth map shown in the figure is used to reconstruct the object. Figure 6 The object is shown in the three-dimensional point cloud.

[0040] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. Speckle-assisted binocular structured light dynamic 3D reconstruction system, characterized by: The steps include: S1: Build a speckle-assisted binocular structured light dynamic 3D reconstruction system; S2: Calibrate the binocular speckle system, camera and projector system and the two system coordinate systems; S3: The projector projects a designed sinusoidal grating fringe and speckle composite color pattern onto the object under test, while the left and right cameras capture the color image modulated by the surface of the object under test; S4: Separate the color image to obtain the fringe pattern and the speckle pattern. Apply the improved variational mode decomposition phase extraction algorithm to the fringe pattern to obtain the wrapped phase. Use the single wrapped phase and the speckle pattern to obtain the unwrapped phase according to the speckle-assisted binocular structured light 3D measurement algorithm. S5: Calculate the three-dimensional information of the object based on the camera-projector calibration results and the unfolded phase.

2. The designed sinusoidal grating stripes and speckle composite color pattern according to claim 1, characterized in that: A single color image contains a sinusoidal grating fringe pattern with a frequency of 64 and a single speckle pattern; The speckle pattern is generated by the speckle generation model, which can be expressed as: in, Function represents the primitive of speckle, (x i，j ,y i，j ) represents the center position of the speckle, and M×N represents the total number of speckles.

3. The color image separation according to claim 1, wherein: According to the color channel, the red channel in imgL is separated into the stripe part imgL_Fringe, and the blue channel in imgL is separated into the speckle part imgL_Speckle.

4. The improved variational mode decomposition phase extraction algorithm according to claim 4, characterized in that: Normalize the image imgL_Fringe and take the mean. Then use the variational mode method to extract the intrinsic mode function u from imgL_Fringe. k , where u1 is the background part and u2 is the stripe part. Perform shearlet denoising on the background part u1 to get denResult. The shearlet denoising formula is: Among them, SH ψ (·)and where are the forward and inverse shearlet transforms, shrink is the threshold shrinkage function, and thr is the threshold parameter. The following formula is used to further optimize the fringe information to obtain more accurate fringe information FinalV. FinalV=(u1+u2)-denResult The Hilbert transform is used to analyze the phase information in the stripe part FinalV, and then the arc tangent formula is used to extract the wrapped phase information wrapL.

5. The speckle-assisted binocular structured light 3D measurement based on depth constraints of monocular and binocular systems according to claim 4, characterized in that: By calculating the wrapping phase wrapL and phase wrapR of the left and right cameras; calculating the epipolar line of a point p1 in the wrapped phase of the left camera in the wrapped phase of the right camera according to the epipolar constraint; finding several points on the epipolar line that are close to the wrapped phase of p1, performing stereo matching on these points based on the speckle pattern, and finding the point p2 with the minimum cost, which is the point with the same name as p1; calculating the three-dimensional coordinates P(x, y, z) of the spatial point through triangulation based on p1 and p2; assuming that p1 is at the kth fringe order, the unwrapped phase of point p1 is obtained according to the phase unwrapping formula; calculating the three-dimensional information through the unwrapped phase and the camera projector calibration parameters, and using the three-dimensional coordinates P(x, y, z) of the binocular speckle triangulation to perform depth constraint to find the optimal fringe order k; The unwrapped phase is calculated using the phase unwrapping formula, and the three-dimensional information P′(x, y, z′) of the object is calculated based on the calibration results.

Images