Stereoscopic vision-based phase-unwrapping-free segmented phase encoding three-dimensional measurement method

Abstract

The invention discloses a phase-unwrapping-free segmented phase coding three-dimensional measurement method based on stereoscopic vision, and the method comprises the steps: firstly taking a segmented phase coding matching problem as a sequence similarity problem, and carrying out the line-by-line alignment through employing a LCS (Longest Common Subsequence) algorithm, thereby improving the binocular matching efficiency. Then, monotonicity is used for matching the monotonous region of each wrapped phase, and partial parallax is obtained through wrapped phase values; and finally, wrapping the phase by the pi phase shift to obtain the parallax of the jump region. According to the method, measurement can be completed only through three sine phase shift diagrams and two segmented phase coding diagrams, and the measurement efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of photoelectric detection technology, and in particular to a three-dimensional measurement method based on stereo vision with phase-unfolded segmented phase encoding. Background Technology

[0002] Fringe projection profilometry (FPP) is an optical 3D measurement technique widely used in industrial quality inspection, biomedicine, machine vision, navigation, and other fields. It offers advantages such as high speed, high accuracy, and non-contact operation. Active stereo vision measurement methods are less affected by ambient light and can measure objects without obvious texture features.

[0003] Traditional structured light stereo vision measurement methods rely on absolute phase to determine matching points. A common unwrapping method is the multi-frequency heterodyne method, which requires projecting multiple fringes of different frequencies. Therefore, a single measurement is time-consuming, and absolute phase is generally less accurate than wrapped phase. In binocular matching algorithms, it is usually necessary to search for matching points in two dimensions, which is also time-consuming.

[0004] To address the above technical issues, this invention proposes a phase-unfolded segmented phase encoding three-dimensional measurement method based on stereo vision. This method reduces the number of fringe patterns required for measurement and completes binocular matching at a one-dimensional level, enabling flexible, efficient, and accurate fringe projection contour three-dimensional reconstruction. Summary of the Invention

[0005] The purpose of this invention is to solve the technical problems existing in the prior art and to provide a three-dimensional measurement method based on stereo vision with phase-free unfolding and segmented phase encoding.

[0006] To achieve the above objectives, the technical solution provided by the present invention is as follows: The specific steps of the measurement method are as follows:

[0007] (1) The effective projection range is determined by using a binocular structured light measurement system. The binocular structured light measurement system consists of two cameras and a single projector. The projector does not need to be calibrated during the measurement process. The effective projection range of the projector is within the common field of view of the two cameras.

[0008] (2) Project three sinusoidal fringe patterns and two segmented phase coding patterns onto the object under test in sequence to obtain the phase of the object. After the object is modulated, the wrapping phase and segmented coded phase of the object under test are obtained; wherein, the sinusoidal fringe pattern and the segmented phase coding pattern have the same frequency.

[0009] (3) Treat the binocular matching problem of periodic stripes or phases as a similarity problem of left and right epipolar sequences, and use the weighted longest common subsequence algorithm, i.e., the weighted LCS algorithm, to perform line-by-line alignment. That is, perform epipolar correction on the left and right segmented coding phases, then extract the quantized codewords line by line, and perform matching according to the weighted LCS algorithm, thereby realizing the matching of segmented codes and improving the binocular matching efficiency.

[0010] (4) With the assistance of the matched segmented coded phase, the monotonicity of the wrapped phase and the phase value are used to complete the binocular matching of the wrapped phase monotonic region and obtain the disparity. The π-phase-shift wrapping phase is used to handle the periodic misalignment between the segmented coded phase and the wrapping phase without the need to correct the periodic misalignment. The disparity value of the original wrapping phase jump region is obtained from the π-phase-shift wrapping phase. Finally, the three-dimensional reconstruction of the object is completed by combining the dual-camera calibration parameters and disparity using the principle of triangulation.

[0011] Preferably, the projector does not require calibration during the measurement process, thus simplifying the initial calibration process of the entire system; the effective projection range of the projector represents the projector's viewing angle range within the common viewing angle range of the two cameras, or the pattern projected by the projector is reflected by the plane and is only visible in the common area of ​​the two cameras.

[0012] Preferably, in step (3), the weighted longest common subsequence algorithm, i.e., the weighted LCS algorithm, performs row-by-row matching as follows:

[0013]

[0014] In Formula 1, This indicates the position of the segmented coding phase on the corresponding epipolar line, in pixels; and This represents the codeword value at that position. and Indicates weight, matching effect as follows Figure 4 As shown.

[0015] Preferably, in step (4), the traditional Zhang Zhengyou calibration method is used for binocular calibration, and the disparity value of sub-pixel accuracy is obtained by formulas 2 and 3. ;

[0016]

[0017]

[0018] In Formulas 2 and 3, Indicates the first OK, Indicates the first image in the left image Column coordinates, Indicates the first image in the right image Column coordinates, It's parallax; and These are the wrap-around phases on the corresponding polar lines, one on the left and one on the right.

[0019] Beneficial effects of this invention:

[0020] This invention provides a three-dimensional measurement method based on stereo vision with phase-free unfolding and segmented phase encoding. This method requires fewer projected images to complete the measurement and can use high-frequency fringe patterns to achieve higher measurement accuracy. Furthermore, it provides a simple binocular matching algorithm based on segmented phase encoding, which eliminates the need to correct complex periodic misalignment problems, further improving the efficiency of the measurement algorithm. Since the pattern is projected onto the object, it is equivalent to adding texture features to the object. Therefore, this method can measure objects with no obvious surface features, such as whiteboards, cubes, etc., and has a wide range of applications. Attached Figure Description

[0021] The accompanying drawings, which are provided to further illustrate the invention and constitute a part of this invention, are illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention.

[0022] Figure 1 This is a schematic diagram of the same k value in discontinuous regions in this invention;

[0023] Figure 2 This is a schematic diagram of the wrapping phase matching in this invention;

[0024] Figure 3 This is a schematic diagram of a binocular structured light measurement system;

[0025] Figure 4 It is (a) the quantized codeword on the left pole line; (b) the quantized codeword on the right pole line; (c)

[0026] Figure 5 It is a wrap-around phase matching, with the blue line representing the wrap-around phase, the green line representing the π-phase-shifted wrap-around phase, and the red line representing the segmented coded phase.

[0027] Figure 6 This is a schematic diagram of the measurement results of Embodiment 1 of the present invention; (a) the object under test; (b) the segmented phase-coded stripes captured by the left camera; (c) a partial disparity map; (d) the final disparity map; (e) a point cloud map of the object under test; and (f) the point cloud encapsulation result. Detailed Implementation

[0028] This section will describe in detail specific embodiments of the present invention. Preferred embodiments of the present invention are shown in the accompanying drawings. The purpose of the drawings is to supplement the textual description with graphics, so that people can intuitively and vividly understand each technical feature and overall technical solution of the present invention, but they should not be construed as limiting the scope of protection of the present invention.

[0029] Reference Figures 1-6 In a preferred embodiment of the present invention, a three-dimensional measurement method based on stereo vision with phase-free unfolding and segmented phase encoding is provided. This method utilizes a periodic non-globally unique identifier stripe pattern, i.e., periodic segmented phase encoding, to assist in wrapping phase matching, which not only reduces the number of stripes required but also reduces the complexity of binocular matching.

[0030] By using a zoom camera or changing the position of the projector relative to the object being measured, the projection area can be placed within the common field of view of both cameras, thus completing the overall system layout and satisfying its requirements. Figure 3 The conditions are shown in the diagram.

[0031] Three sinusoidal fringe patterns and two piecewise phase-coded patterns are projected sequentially onto the object under test to obtain the object's phase. Formulas four and five are the formulas for generating the desired fringes. The sinusoidal fringe patterns and piecewise phase-coded patterns have the same frequency.

[0032]

[0033]

[0034] In the formula, , Indicates image intensity, Represents pixel coordinates, It is the background intensity. It is the modulation intensity. , These are the wrapping phase and the segmented coding phase, respectively.

[0035] After object modulation, the enveloping phase and segmented coded phase of the measured object can be obtained. The following detailed steps complete binocular matching to obtain a disparity map, and finally, the object's three-dimensional coordinates are obtained by combining the binocular calibration parameters:

[0036] Step 1: After capturing the image of the stripes modulated by the object and obtaining the phase, remove background noise according to formulas six and seven. This is the modulation intensity threshold. Before matching, a median filter needs to be applied to the segmented coded phase to correct for phase step errors. Epipolar correction is performed on the left and right segmented coded phases, then the quantized codewords are extracted line by line, and matching is performed according to Formula 1.

[0037]

[0038]

[0039] Step 2: After matching the segmented codes, the approximate coordinate region of each matched wrapper phase can be obtained. However, to obtain a dense disparity map, the complete coordinates of the corresponding codeword on its respective wrapper phase map are needed. Therefore, post-processing is performed on the start and end coordinates of each aligned codeword to obtain the complete coordinate range. The left and right phase maps are checked separately to determine whether the codeword value of the subsequent coordinates is equal to the initial coordinate. If they are equal, the start coordinate (start) is updated according to Formula 8, and the end coordinate (end) is updated in the same way. When searching for the complete segmented codeword coordinates from a position, the codeword values ​​on both sides of the discontinuous region may be equal, such as... Figure 1 As shown. In this case, if discontinuous regions span multiples of the segmented phase coding period, the start-to-end interval will contain two wrapping phases. Therefore, during the search process, the difference between two coordinates with the same codeword value needs to be limited to a certain threshold, namely the threshold thr_pix in Formula 8, in pixels. Take the midpoint coordinates midL and midR of each aligned codeword (midL represents the midpoint coordinate in the left image, and midR represents the midpoint coordinate in the right image); start coordinates startL and startR; and end coordinates endL and endR. These coordinate values ​​will be used in the next step to assist in wrapping phase matching.

[0040]

[0041] Step 3, as follows Figure 2 As shown, due to the periodic misalignment between the quantized segmented coding phase and the wrapping phase, this paper utilizes monotonicity to match each wrapping phase without needing to correct for the periodic misalignment. After performing segmented coding phase matching, to avoid the influence of inconsistent monotonicity at the edge points of each wrapping phase, it is necessary to search for all coordinates satisfying the wrapping phase monotonicity condition from the midpoint coordinates (midL and midR) towards both ends. The wrapping phase derivative values ​​at the midpoint coordinates midL and midR obtained in step 2 are consistent with the monotonicity of the wrapping phase. During the search process, relying solely on the monotonicity of the wrapping phase is prone to errors in discontinuous regions, for example, Figure 2 It shows that the effective phase is monotonic at the discontinuity, but not in the same wrap phase.

[0042] Therefore, the search needs to be performed using the coordinates obtained in step 2, as shown in Formula 9. Formula 9, like Formula 8, only shows the processing formula for the start endpoint; the processing principle for the end endpoint is the same and is therefore omitted. First, the monotonic region of the wrapping phase is obtained through monotonicity. Then, (wrap_start, wrap_end) is compared with the endpoints (start, end) of the segmented encoded phase. When the difference between the two is large (determined by the thr_x threshold), it indicates that an error occurred due to matching based solely on monotonicity.

[0043] At this point, the coordinate point closest to the endpoints (start, end) of the segmented coding matching region and inconsistent with the monotonicity of the wrapper phase is selected. , ) serves as the matching endpoint of the wrap phase (wrap_start, wrap_end). represent The derivative of the phase at the coordinates.

[0044]

[0045] Step 4: Calculate the π-phase shift wrapped phase using Formula 10. Similar to Step 3, combine the matched segmented encoded phase endpoints and the monotonicity of the π-phase shift wrapped phase to fill in the disparity of the jump region. The jump region refers to the sawtooth data jump region wrapped in the phase.

[0046]

[0047] Step 5: Combining the dual-camera calibration parameters and parallax, the three-dimensional coordinates (X, Y, Z) of the object are obtained using the principle of triangulation:

[0048]

[0049] In Formula 11, Represents the baseline of a dual-camera system. Focal length; The principal point coordinates are the intersection of the camera's optical axis and the image sensor; these parameters are obtained during calibration. For parallax.

[0050] The three-dimensional measurement of an object can be completed by following the five steps described above. This method can use high-frequency fringe patterns for measurement, and only five images need to be projected to achieve high-precision three-dimensional measurement.

[0051] Example 1

[0052] Step 1: Calibrate the stereo camera using conventional methods;

[0053] The imaging process of a camera is perspective projection. Without considering lens distortion, the standard pinhole lens model can be mathematically represented as:

[0054]

[0055] In formula twelve, This is the scaling factor; Camera pixel coordinates; , These are the focal lengths of the camera in different directions; , Principal point coordinates; It is The extrinsic parameter matrix describes the camera's external parameters, where... It is The rotation matrix, It is Translation vector; The world coordinates of a certain point.

[0056] In practical applications, aberrations are unavoidable in imaging lenses, therefore lens distortion should be considered in camera models. Existing epipolar correction techniques all take into account the influence of lens distortion, so distortion correction and epipolar correction can be performed conveniently and quickly by calling the epipolar correction function. Equation thirteen describes the extrinsic parameter relationship between the left and right cameras. Reconstructing the homography matrix of the two cameras can create a virtual camera imaging plane where the two image planes are coplanar and row-aligned. Combining this with the numerical features of the epipolar lines allows binocular matching to be performed only in corresponding rows.

[0057]

[0058] Step 2: Synchronize projection and pattern acquisition;

[0059] After completing the binocular stereo calibration, the projector sequentially projects five 8-bit grayscale images, and the camera achieves synchronous acquisition through the trigger line connected to the projector.

[0060] Step 3: Implement binocular matching using the algorithm described in the invention to obtain disparity. And reconstruct the three-dimensional topography. Specific measurement results are shown in the following example. Figure 6 As shown.

[0061] Without causing conflict, those skilled in the art can freely combine and use the above-mentioned additional technical features.

[0062] The above description is only a preferred embodiment of the present invention. Any technical solution that achieves the purpose of the present invention by essentially the same means is within the protection scope of the present invention.

Claims

1. A three-dimensional measurement method based on stereo vision with phase-unfolded segmented phase encoding, characterized in that: The specific steps of the measurement method are as follows: (1) The effective projection range is determined by using a binocular structured light measurement system. The binocular structured light measurement system consists of two cameras and a single projector. The projector does not need to be calibrated during the measurement process. The effective projection range of the projector is within the common field of view of the two cameras. (2) Project three sinusoidal fringe patterns and two segmented phase coding patterns onto the object under test in sequence to obtain the phase of the object. After the object is modulated, the wrapping phase and segmented coded phase of the object under test are obtained; wherein, the sinusoidal fringe pattern and the segmented phase coding pattern have the same frequency. (3) Treat the binocular matching problem of periodic stripes or phases as a similarity problem of left and right epipolar sequences, and use the weighted longest common subsequence algorithm, i.e., the weighted LCS algorithm, to perform line-by-line alignment. That is, perform epipolar correction on the left and right segmented coding phases, then extract the quantized codewords line by line, and perform matching according to the weighted LCS algorithm, thereby realizing the matching of segmented codes and improving the binocular matching efficiency. (4) With the assistance of the matched segmented coded phase, the monotonicity of the wrapped phase and the phase value are used to complete the binocular matching of the wrapped phase monotonic region and obtain the disparity. The π-phase-shift wrapping phase is used to handle the periodic misalignment between the segmented coded phase and the wrapping phase without the need to correct the periodic misalignment. The disparity value of the original wrapping phase jump region is obtained from the π-phase-shift wrapping phase. Finally, the three-dimensional reconstruction of the object is completed by combining the dual-camera calibration parameters and disparity using the principle of triangulation.

2. The three-dimensional measurement method based on stereo vision with phase-free unfolding and segmented phase encoding according to claim 1, characterized in that: The projector does not require calibration during the measurement process, thus simplifying the initial calibration process of the entire system; The effective projection range of a projector means that the projector's viewing angle is within the common viewing angle of the two cameras, or that the pattern projected by the projector is reflected by the plane and is only visible in the common area of ​​the two cameras.

3. The three-dimensional measurement method based on stereo vision with phase-free unfolding and segmented phase encoding according to claim 1, characterized in that: In step (3), the weighted longest common subsequence algorithm, i.e., the weighted LCS algorithm, performs row-by-row matching as follows: ； In Formula 1, This indicates the position of the segmented coding phase on the corresponding epipolar line, in pixels; and This represents the codeword value at that position. and Indicates the weight.

4. The three-dimensional measurement method based on stereo vision with phase-free unfolding and segmented phase encoding according to claim 1, characterized in that: In step (4), the traditional Zhang Zhengyou calibration method is used for binocular calibration, and the disparity value of sub-pixel accuracy is obtained by formulas 2 and 3. ; ； ； In Formulas 2 and 3, Indicates the first OK, Indicates the first image in the left image Column coordinates, Indicates the first image in the right image Column coordinates, It's parallax; and These are the wrap-around phases on the corresponding polar lines, one on the left and one on the right.

Images