Three-dimensional measurement method based on mixed structured light projection

Abstract

The invention discloses a three-dimensional measurement method based on mixed structured light projection. Firstly, a DLP projector is used for carrying out laser line scanning projection and high-frequency phase shift fringe pattern projection on a measured scene in sequence, a binocular camera synchronously collects stereo images, and calibration parameters of the binocular camera are used for carrying out stereo correction on the collected laser line images and fringe images. And processing the corrected laser line image by using a laser line center extraction method based on adaptive convolution to obtain a dense and complete initial disparity map. And processing the corrected fringe pattern by using a generalized phase shift method to obtain a wrapped phase diagram of the measured scene. And finally, based on a matching position provided by the initial disparity map and wrapped phase information of the left view and the right view, realizing sub-pixel positioning compensation of a matching point by using a linear interpolation method under phase consistency constraint. According to the invention, a laser line center extraction method based on adaptive convolution and sub-pixel positioning compensation based on phase consistency constraint are utilized, so that the measurement precision and spatial resolution of a traditional laser line scanning method are improved, and high-dynamic-range and high-precision three-dimensional shape measurement is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of optical measurement, and particularly relates to a three-dimensional measurement method based on mixed structured light projection. BACKGROUND

[0002] The three-dimensional imaging technology based on structured light projection has a wide application in manufacturing, biomedical diagnosis and industrial measurement due to its high precision and non-contact measurement. However, when facing a complex measurement scene with high dynamic range, it is difficult to obtain a high signal-to-noise ratio structured light pattern due to the uneven reflectivity of the target surface, thereby affecting the quality of three-dimensional reconstruction. In the field of optical measurement, the fringe projection profilometry as the most representative three-dimensional measurement method is easily affected by the reflectivity difference of the measurement scene and the measurement distance, resulting in missing measurement results and precision decline, and unable to complete high-quality three-dimensional measurement with one exposure. On the other hand, the laser line scanning method can complete relatively complete three-dimensional reconstruction due to its robustness, but it is difficult to restore the fine profile of the measurement scene due to the limitation of spatial resolution, and the reconstruction precision is low. SUMMARY

[0003] The application aims to provide a three-dimensional measurement method based on mixed structured light projection to realize high dynamic range and high precision three-dimensional profile measurement of a complex scene.

[0004] The technical scheme for realizing the application is as follows: a three-dimensional measurement method based on mixed structured light projection, comprising the following steps:

[0005] Step 1: using a DLP projector to sequentially project a laser line scanning pattern and a high-frequency phase shift fringe pattern on a measured scene, and using a binocular camera to synchronously collect laser line images and fringe images, and using the calibration parameters of the binocular camera to perform stereo correction on the collected laser line images and fringe images;

[0006] Step 2: using an adaptive convolution-based laser line center extraction method to process the corrected laser line images to obtain a dense and complete initial disparity map;

[0007] Step 3: using a generalized phase shift method to process the corrected fringe pattern to obtain a wrapped phase map of the measured scene;

[0008] Step 4: using a linear interpolation method under phase consistency constraint to realize sub-pixel positioning compensation of the matching points based on the matching positions provided by the initial disparity map and the wrapped phase information of the left and right views.

[0009] Compared with the prior art, the application has the following obvious advantages:

[0010] The application combines the high resolution of the fringe projection profilometry and the high robustness of the line laser scanning, realizes significant improvement on the basis of a single measurement technology, and can be widely applied to high-precision three-dimensional measurement tasks of various complex scenes, and has high detail restoration capability and strong anti-interference performance.

[0011] The application will be further described in detail below with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 It is a flowchart of a three-dimensional measurement method based on mixed structured light projection.

[0013] Figure 2 It is a three-dimensional measurement result of three isolated objects with different reflection characteristics (from left to right, high-reflective metal, white diffuse reflection model and low-reflective metal) using the method. Among them, Figure 2 (a) in the (a) is the real object picture of the measurement scene; Figure 2 (b) in the (b) is the three-dimensional reconstruction result of the initial parallax map obtained by the laser line scanning method; Figure 2 (c) in the (c) is the high-frequency phase shift fringe pattern collected from the measured scene; Figure 2 (d) in the (d) is the three-dimensional reconstruction result obtained by parallax optimization calculation. DETAILED DESCRIPTION

[0014] A three-dimensional measurement method based on mixed structured light projection, comprising the following steps:

[0015] Step 1: using a DLP projector to project laser line scanning and high-frequency phase shift fringe pattern on the measured scene in turn, and using a binocular camera to synchronously collect stereo images, and using the calibration parameters of the binocular camera to perform stereo correction on the collected laser line images and fringe images;

[0016] The system is composed of two monochrome cameras with a resolution of 1280*1024 pixels and a one-dimensional structured light projection module based on a laser scanning galvanometer, and the working process of the projection module is as shown in Figure 2 .

[0017] Step 2: using an adaptive convolution-based laser line center extraction method to process the corrected laser line image to obtain a dense and complete initial parallax map, and the specific method is as follows:

[0018] First, the corrected laser line image is binarized, and the effective line segment with the most pixels in each row is regarded as the line width of the row. Specifically, the binarized image can be represented as:

[0019]

[0020] Wherein For the threshold of binarization, since the laser line has the characteristics of brightness concentration and narrow width, its default value of 50 can meet most scenes. By counting the length of all valid regions in each row, the maximum valid region length is set as the line width of this row:

[0021]

[0022] wherein is the set of valid regions is the th valid region. Select the minimum odd value greater than or equal to the line width as the size of the convolution kernel, to ensure that the convolution kernel can fully cover the cross section of the laser line pattern.

[0023] Select the corresponding size of the convolution kernel from the pre-constructed multi-scale convolution kernels of different sizes;

[0024] Use the selected convolution kernel to perform multi-scale convolution calculation on the laser line image, and use the boundary padding strategy to calculate the gradient response row by row to avoid the influence of convolution boundary effect on the result:

[0025]

[0026] wherein is the half width of the convolution kernel.

[0027] The weights of the convolution kernel are constructed based on the first derivative of the two-dimensional Gaussian function, which has good derivative characteristics and symmetry:

[0028]

[0029] wherein, denotes the size of the convolution kernel, is the standard deviation of the Gaussian function, which is empirically set to .

[0030] Since the laser line presents an approximate Gaussian distribution in the image, its first-order gradient response will have a zero-crossing jump at the center position with the highest brightness. Therefore, by detecting adjacent gradient values that meet the following conditions, the gradient zero-crossing point (i.e. the center position of the laser line) can be located:

[0031]

[0032] When multiple adjacent point pairs that meet the conditions appear, select the point pair with the largest absolute change, and realize sub-pixel level center point extraction through linear interpolation:

[0033]

[0034] By calculating the laser line center coordinates in the left and right images, the disparity value of each point can be obtained .

[0035] Step 3: Process the corrected fringe pattern using the generalized phase-shifting method to obtain the enveloping phase map of the measured scene, specifically:

[0036] The corrected stripe pattern is represented as follows:

[0037]

[0038] in, These are the camera's pixel coordinates. It is the average light intensity. To emphasize the system, The phase distribution of the measured object, This is the bit offset;

[0039] By fitting using the least squares method, the corresponding solution can be directly obtained. The wrapping phase :

[0040]

[0041] Orthogonal phase components Constructed from a linear combination of light intensity values:

[0042]

[0043] Among them, coefficient It can be calculated from the following matrix:

[0044]

[0045] Step 4: Based on the matching position provided by the initial disparity map and the wrapping phase information of the left and right views, sub-pixel localization compensation of the matching point is achieved using a linear interpolation method under phase consistency constraints. Specifically:

[0046] The wrapping phase of the left and right views is obtained. , Then, through the disparity map You can get a certain point in the middle exist Matching points on Due to the spatial resolution limitations of the laser line scanning method, there is still a certain pixel positioning deviation between the matching point and the ideal matching point. Since the system has undergone epipolar correction, the left and right views are basically horizontal. Therefore, the vertical error can be ignored within a short horizontal distance, and only the horizontal coordinate pixel positioning error needs to be corrected.

[0047] First, let's look at the matching points. The nearest integer point corresponding to the x-axis Establish a predetermined radius for the center. The local search window, in which The target disparity to be optimized. Detect neighboring points that satisfy the continuous phase constraint. Make:

[0048]

[0049] in for The corresponding value. Considering that the wrapper phase is only continuous within a certain interval, there may be differences between adjacent points. The phase jump, then the final corrected matching position It is possible

[0050]

[0051] The calculated optimized disparity value is:

[0052]

[0053] After completing the parallax map compensation optimization, the depth information can be calculated based on the camera calibration parameters, thus completing the three-dimensional measurement strategy of the entire hybrid structured light profilometry.

[0054] Figure 2 The system demonstrates the 3D reconstruction results of three isolated objects (approximately 0.9m from the camera) with different reflective properties in the same scene: Figure 2 (a) is a physical image of the measurement scene, from left to right: a highly reflective metal part, a white diffuse reflection model, and a low-reflective metal part; Figure 2 (b) is the 3D reconstruction result of the initial disparity map obtained by the laser line scanning method; Figure 2 (c) is a high-frequency phase-shift fringe pattern of the scene under test; Figure 2 (d) shows the 3D reconstruction result obtained after parallax optimization calculation. The results show that the method can achieve complete and high-precision 3D topography measurement in high dynamic range scenes.

Claims

1. A three-dimensional measurement method based on hybrid structured light projection, characterized in that, Includes the following steps: Step 1: Use a DLP projector to project laser line scan and high-frequency phase-shift fringe patterns onto the scene under test. Use a binocular camera to simultaneously acquire laser line images and fringe images. Use the calibration parameters of the binocular camera to perform stereo correction on the acquired laser line images and fringe images. Step 2: Process the corrected laser line image using an adaptive convolution-based laser line center extraction method to obtain a dense and complete initial disparity map; Step 3: Process the corrected fringe pattern using the generalized phase shift method to obtain the wrap-around phase map of the scene under test; Step 4: Based on the matching position provided by the initial disparity map and the wrapping phase information of the left and right views, sub-pixel positioning compensation of the matching point is achieved by using the linear interpolation method under phase consistency constraints.

2. The three-dimensional measurement method based on hybrid structured light projection according to claim 1, characterized in that, The specific method for processing the corrected laser line image using an adaptive convolution-based laser line center extraction method to obtain a dense and complete initial disparity map is as follows: The corrected laser line image is binarized, and the effective line segment with the most pixels in each row is taken as the line width of each row. Select the smallest odd value greater than or equal to the line width as the convolution kernel size; Select the appropriate size convolution kernel from the pre-constructed multi-scale convolution kernels of different sizes; Multi-scale convolution calculations are performed on the laser line image using the selected convolution kernel, while a boundary padding strategy is employed to calculate the gradient response row by row. ， in The width of the convolution kernel is half its width. Indicates the first The convolution kernel corresponding to the line width of the row; The zero-crossing point of the gradient, i.e., the center position of the laser line, can be located by detecting adjacent gradient values ​​that meet the following conditions: ， When multiple pairs of adjacent points meet the conditions, the pair with the largest absolute change is selected, and sub-pixel level laser line center position extraction is achieved through linear interpolation. ， The disparity value of each point is obtained by calculating the center coordinates of the laser line in the left and right images. This allows us to obtain the initial disparity map.

3. The three-dimensional measurement method based on hybrid structured light projection according to claim 2, characterized in that, The weights of the convolution kernel are constructed based on the first derivative of a two-dimensional Gaussian function: ， in, Indicates the size of the convolution kernel. is the standard deviation of the Gaussian function.

4. The three-dimensional measurement method based on hybrid structured light projection according to claim 1, characterized in that, The specific method for obtaining the enclosed phase map of the measured scene by processing the corrected fringe pattern using the generalized phase-shift method is as follows: The corrected stripe pattern is represented as follows: ， in, These are the camera's pixel coordinates. It is the average light intensity. To emphasize the system, The phase distribution of the measured object, This is the phase offset. The phase distribution of the measured object is obtained by fitting using the least squares method. Corresponding package phase : ， In the formula, These are orthogonal phase components.

5. The three-dimensional measurement method based on hybrid structured light projection according to claim 4, characterized in that, The orthogonal phase components are as follows: ， In the formula, It is a phase-shifted stripe pattern. This represents the phase offset of the fringe pattern. The number of stripes. The correlation coefficient is calculated from the following matrix: 。 6. The three-dimensional measurement method based on hybrid structured light projection according to claim 1, characterized in that, Based on the matching position provided by the initial disparity map and the wrapping phase information of the left and right views, the specific method for sub-pixel localization compensation of the matching point is as follows, using a linear interpolation method under phase consistency constraints: Based on the wrapping phase of the left and right views , and initial disparity map It is possible to determine any point in the phase diagram enclosed by the left view. Matching points on the phase map in the right view ; Matching points The nearest integer point corresponding to the x-coordinate Establish a predetermined radius for the center. A horizontal local search window, in which, For the target parallax to be optimized, They are respectively The vertical and horizontal coordinates; Detecting neighboring points that satisfy continuous phase constraints using a local search window ; The corrected matching position is determined based on adjacent points that satisfy the continuous phase constraint, and the optimized disparity value is determined accordingly. The depth information is calculated based on the camera's calibration parameters, thus completing the three-dimensional measurement of the entire hybrid structured light profilometry technique.

7. The three-dimensional measurement method based on hybrid structured light projection according to claim 6, characterized in that, The continuous phase constraint is specifically as follows: ， in, and The wrap-around phases are the corresponding points in the left and right views, respectively. for The corresponding value.

8. The three-dimensional measurement method based on hybrid structured light projection according to claim 6, characterized in that, The corrected matching positions are as follows: 。 9. The three-dimensional measurement method based on hybrid structured light projection according to claim 6, characterized in that, The optimized disparity values ​​are as follows: ， In the formula, This is the corrected matching position.

Images