Phase ambiguity reduction method based on speckle-assisted phase tracking

Abstract

The invention discloses a phase ambiguity reduction method based on speckle auxiliary phase tracking, which can accurately correct the motion error in a phase shift image by adding a speckle pattern in a phase shift projection sequence and calculating the sub-pixel projection point displacement between a front frame and a rear frame by using a DIC algorithm, obviously improves the precision of three-dimensional measurement, and improves the accuracy of phase ambiguity reduction. The projection points of the phase shift image are aligned to the same camera coordinate through a remapping technology, so that the motion blur error is effectively reduced, the accuracy of a measurement result is ensured, the projection points are tracked instead of tracking the motion of an object, the motion error with z-direction displacement can be corrected, a wrapped phase diagram is obtained through a phase shift method, and the accuracy of the measurement result is improved. And phase unwrapping is carried out by combining a double-frequency order method and a constant geometric constraint method, so that the accuracy and continuity of phase unwrapping are ensured, and the three-dimensional shape of the object is reconstructed by utilizing a triangulation method.

Description

Technical Field

[0001] The invention relates to an optical three-dimensional measurement method based on digital image correlation (DIC) technology and phase tracking technology, in particular to a phase blur reduction method based on speckle-assisted phase tracking. Background Art

[0002] Optical 3D measurement technology is widely used in industrial measurement, reverse engineering and other fields. Among them, fringe projection profilometry (FPP) has become one of the most popular technologies due to its high precision and full-field measurement advantages. A typical FPP system consists of a camera and a projector. The 3D surface information of the object to be measured can be obtained through phase demodulation and triangulation. Traditional Fourier transform method and phase shift method are often used to obtain the phase of the object. Among them, the Fourier transform method only needs to project a single sinusoidal fringe to obtain the phase map, which is suitable for 3D measurement of dynamic scenes, but this method is difficult to measure complex scenes. The phase shift method requires the projection of multiple fringe patterns to complete the measurement process, but the movement of the object will destroy the phase measurement process, causing errors caused by motion and reducing the measurement accuracy.

[0003] In recent years, many methods have been proposed to compensate for the errors caused by motion, such as motion tracking, FTP and PSP combination, self-resistance method, etc. However, these methods have certain limitations and it is difficult to achieve high-precision measurements in high-speed dynamic scenes.

[0004] The present invention has the following technical problems:

[0005] 1) Existing technologies are difficult to measure complex scenes. The phase shift method requires projecting multiple fringe patterns to complete the measurement process, but the movement of the object will destroy the phase measurement process, causing errors caused by movement and reducing the measurement accuracy.

[0006] 2) From the perspective of stripe design, the existing technology believes that uniformly moving objects will cause additional phase differences. The stripe sequence designed can offset this error without introducing additional complex calculations, which can improve efficiency. However, it is only applicable to small displacement scenarios and has a limited range.

[0007] 3. Existing technologies cannot adapt to the measurement requirements of different resolutions. High-precision measurement and efficient rough measurement cannot be met at the same time, and the scope of application is limited.

[0008] 4 The existing technology uses the SIFT algorithm combined with the region segmentation algorithm to track objects, automatically reduces the motion error of multiple objects, detects the two-dimensional sub-pixel displacement of objects, and realizes phase compensation of non-uniform moving objects. However, it can only realize 2D tracking at the image level and cannot correct motion errors with displacement in the z direction.

[0009] 5 The existing technology estimates the motion of an object through two adjacent 3D frames, and uses an iterative method to obtain the phase shift caused by the object's motion at different positions. However, in the process of estimating the three-dimensional motion of the object, there will be a large error due to the influence of motion noise.

[0010] SUMMARY OF THE INVENTION

[0011] To address the deficiencies of the prior art, the present invention aims to provide a phase ambiguity reduction method based on speckle-assisted phase tracking to achieve high-precision measurement in high-speed dynamic scenes.

[0012] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0013] The present invention discloses a phase blur reduction method based on speckle-assisted phase tracking. The method is implemented by the following devices: a camera, a projector, a motion controller and a computing device. The camera is connected to the projector, and the computing device is connected to the camera, the projector and the motion controller respectively. The method comprises the following steps:

[0014] The camera is synchronously triggered by the projector, and the computing device sends a signal to start acquisition. The camera can accurately acquire an image sequence containing high-period and low-period three-step phase-shift fringe patterns and speckle patterns;

[0015] The DIC algorithm is used to calculate the image sequence of the three-step phase-shift fringe pattern and the speckle pattern to obtain the displacement of the projection point on the u-axis and v-axis caused by the object movement;

[0016] The fringe image is corrected by the displacement of the projection points on the u-axis and v-axis generated by the movement of the object, and the image is remapped to obtain the corrected fringe pattern;

[0017] By calculating the corrected fringe pattern, the wrapped phase image is obtained;

[0018] The wrapped phase image is used to perform the order unwrapping algorithm and the constant geometry constraint algorithm to obtain the final unwrapped phase image;

[0019] The three-dimensional measurement is completed by unwrapping the phase image and performing triangulation calculations to obtain point cloud data.

[0020] As a further improvement, the camera of the present invention is synchronously triggered by the projector, and the computing device sends a start acquisition signal, so that the camera can accurately acquire an image sequence including a high-period and low-period three-step phase-shift fringe pattern and a speckle pattern, specifically:

[0021] The projector triggers the camera synchronously. The computing device connects the projector and the motion controller via the serial port to send a start acquisition signal. The phase blur reduction method uses a 50-120Hz projection fringe pattern for 3D reconstruction. The projection double periods are 128 and 16 pixels respectively. The projector stores 6 8-bit images at a projection speed of 50-120Hz. A 128-cycle fringe is projected in 7-bit format, and the speckle pattern is projected in 1-bit format to obtain an image sequence of a three-step phase-shifted fringe pattern with high and low periods and a speckle pattern.

[0022] As a further improvement, the DIC algorithm is calculated by the image sequence of the three-step phase shift fringe pattern and the speckle pattern in the present invention to obtain the displacement of the u-axis and v-axis projection points caused by the object movement. Specifically, the DIC algorithm searches for matching points between the reference image and the deformed image through the image subset, and uses the zero-mean normalized sum of squares ZNSSD to evaluate the similarity between the reference image and the deformed image:

[0023]

[0024] Where S is a subset of the reference image, f(x, y) and g(x′, y′) are the intensities of the reference image and the deformed image, and The IC-GN algorithm is used to perform sub-pixel matching to obtain the displacement (Δu0, Δv0) of the reference image and the deformed image:

[0025]

[0026] The displacement of the speckle image between frames is small, so a smaller S is set to speed up the matching speed. The modified Ncorr open source software is used to implement the DIC algorithm. The speckle pattern is projected onto the object, which can detect the slight deformation of the projected texture. The reference pattern and the deformed pattern are cross-correlated, and the sub-pixel projection point displacement map of the speckle pattern between the previous and next frames during the motion process is calculated.

[0027] As a further improvement, the present invention corrects the fringe image by the u-axis and v-axis projection point displacements generated by the object movement, and remaps the image to obtain the corrected fringe pattern, specifically: the image is remapped according to the projection point displacement calculated by the DIC method, the phase displacement of the intermediate phase shift image is interpolated and calculated according to Formula 3)

[0028]

[0029] I in formula 3) n The intensity on (u+nΔu, v+nΔv) is mapped to I′ n (u, v), remap() function is an image remapping algorithm, which can quickly implement image correction and reduce motion blur.

[0030] As a further improvement, the present invention obtains the wrapped phase map by calculating the corrected fringe pattern, specifically: the corrected fringe pattern is calculated by projecting the sinusoidal fringe pattern onto the surface of the object to be measured, and the camera collects the distorted fringes, which is described as:

[0031] I n (u, v) = A (u, v) + B (u, v) cos (φ (u, v) + δ n )4)

[0032] Where (u, v) is the coordinate of the camera image target surface, I n is the nth fringe pattern, n∈[0,N), N is the number of phase shift steps3; A and B are the average light intensity and fringe intensity modulation respectively; φ represents the absolute phase; δ n is the phase shift, The wrapped phase is obtained by calculating the phase shift fringes through equation 5):

[0033]

[0034] As a further improvement, the present invention performs calculations using a wrapped phase diagram using a sequential unwrapping algorithm and a constant geometric constraint algorithm to obtain a final unwrapped phase diagram. Specifically, the sequential unwrapping algorithm maps a low-period phase to a high-period phase to obtain a wrapped phase diagram that is highly accurate and easy to unwrap, and the constant geometric constraint algorithm actively limits the measurement depth range of the object to obtain the final unwrapped phase diagram.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1 By adding speckle patterns to the phase-shift projection sequence and using the DIC algorithm to calculate the sub-pixel projection point displacement between the previous and next frames, the motion error in the phase-shift image can be accurately corrected, significantly improving the accuracy of three-dimensional measurement.

[0037] 2 The present invention aligns the projection points of the phase-shifted image to the same camera coordinates through the remapping technology, which effectively reduces the motion blur error and ensures the accuracy of the measurement results.

[0038] 3 The present invention introduces a high-precision DIC algorithm and speckle-assisted technology, which can achieve stable three-dimensional measurement in high-speed displacement scenarios and has high robustness and reliability.

[0039] The projector of the present invention can project sinusoidal fringe patterns of different periods to meet the measurement requirements of different resolutions, thereby flexibly switching between high-precision measurement and efficient rough measurement, further improving the application scope of the system.

[0040] 5 This method tracks the projection points instead of the movement of the object, and can correct the motion error with z-direction displacement.

[0041] 6 The wrapped phase image is obtained by the phase shift method, and the phase unwrapping is performed in combination with the dual-frequency order method and the constant geometric constraint method to ensure the accuracy and continuity of the phase unwrapping, so as to reconstruct the three-dimensional shape of the object using the triangulation method.

[0042] The camera, projector, computing device and motion controller in the present invention work in a coordinated manner through synchronous control, thereby ensuring efficient coordination of various parts of the system and improving the overall performance in the field of dynamic three-dimensional measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 is a schematic diagram of the process of this method;

[0045] Figure 2 Figure 1 is the experimental setup diagram of this method;

[0046] Figure 3 Projection sequence diagram for this method; DETAILED DESCRIPTION

[0047] The present invention discloses a phase ambiguity reduction method based on speckle-assisted phase tracking. The method is implemented by the following devices: a camera, a projector, a motion controller and a computing device. The camera is connected to the projector, and the computing device is respectively connected to the camera, the projector and the motion controller. The sinusoidal fringe pattern is projected onto the object to be measured. After the distorted fringes are received by the camera, the three-dimensional surface information of the object to be measured can be obtained through phase demodulation and triangulation. The flow chart of the method is as follows Figure 1 The experimental setup of the method is shown in Figure 2 As shown, the method comprises the following steps:

[0048] The camera is triggered synchronously by the projector, and the computing device sends a start acquisition signal. The camera can accurately acquire an image sequence containing high-period and low-period three-step phase-shift fringe patterns and speckle patterns; the projector triggers the camera synchronously, and the computing device connects the projector and motion controller via a serial port to send a start acquisition signal. The phase blur reduction method uses a 50-120Hz projected fringe pattern for three-dimensional reconstruction. The projection double periods are 128 and 16 pixels respectively. The projector stores 6 8-bit images at a projection speed of 50-120Hz. A 128-period fringe is projected in a 7-bit format, and the speckle pattern is projected in a 1-bit format, obtaining a high- and low-period three-step phase-shift fringe pattern and a speckle pattern image sequence. The projection sequence is as follows: Figure 3 As shown, the fringe pattern in motion can be described as:

[0049]

[0050] Where I0 is the speckle image of the previous group, I1-I6 are the fringe images of the current pattern, T L ,T H are low and high cycles respectively, I7 is the current speckle image, Δu and Δv are the u-axis and v-axis displacements of the projection point relative to the speckle pattern I0 at different times.

[0051] The DIC algorithm is calculated through the image sequence of the three-step phase-shift fringe pattern and the speckle pattern to obtain the displacement of the u-axis and v-axis projection points caused by the object movement; the DIC algorithm finds the matching points of the reference image and the deformed image through the image subset, and uses the zero-mean normalized sum of squares ZNSSD to evaluate the similarity between the reference image and the deformed image:

[0052] Where S is a subset of the reference image, f(x, y) and g(x′, y′) are the intensities of the reference image and the deformed image, and The IC-GN algorithm is used to perform sub-pixel matching to obtain the displacement (Δu0, Δv0) of the reference image and the deformed image:

[0053] The displacement of the speckle image between frames is small, so a smaller S is set to speed up the matching speed. The modified Ncorr open source software is used to implement the DIC algorithm. The speckle pattern is projected onto the object, which can detect the slight deformation of the projected texture. The reference pattern and the deformed pattern are cross-correlated, and the sub-pixel projection point displacement map of the speckle pattern between the previous and next frames during the motion is calculated. By adding a frame of speckle image to the fringe sequence, the projection points between the speckle images are tracked at the sub-pixel level, and the speckle images of the current frame and the previous frame are used with the DIC algorithm to obtain the u-axis and v-axis projection point displacements (u0, v0) generated by the object motion, combined with formula 9 to obtain:

[0054]

[0055] It is a geometric matching point, but it shows a different neighborhood distribution. The speckle image is projected onto the surface of the object instead of being attached to the object. The movement of the object causes the projection material and the object material to be misaligned. Unlike the target tracking method, the proposed method tracks the projection points but not the movement of the object. It is suitable for the case of z-axis movement. The obtained spot matching results cannot be used for 3D reconstruction.

[0056] The fringe image is corrected by the displacement of the u-axis and v-axis projection points generated by the object movement, and the image is remapped to obtain the corrected fringe pattern. After the fringe pattern displacement is obtained, the fringe pattern displacement is calculated by formula 6.

[0057]

[0058] The real phase can be restored by interpolating the phase shift of the intermediate phase shift map obtained by image remapping according to the projection point displacement calculated by the DIC method and according to Equation 3) I in formula 3) n (u+nΔu, v+nΔv intensity is mapped to I′ n (u, c), the remap() function is an image remapping algorithm that quickly calculates the wrapping phase to achieve image correction and reduce motion blur.

[0059] By calculating the corrected fringe pattern, the wrapped phase image is obtained; the corrected fringe pattern is calculated by projecting the sinusoidal fringe pattern onto the surface of the object to be measured, and the camera collects the distorted fringes, which can be described as:

[0060] I n (u, v) = A (u, v) + B (u, v) cos (φ (u, v) + δ n )4)

[0061] Where (u, v) is the coordinate of the camera image target surface, I nis the nth fringe pattern, n∈[0,N), N is the number of phase shift steps, the larger the number of phase shift steps N is, the higher the measurement accuracy is, and the lower the efficiency is. In the method, N=3 is selected; A and B are the average light intensity and fringe intensity modulation respectively; φ is the absolute phase; δ n is the phase shift,

[0062] The wrapped phase is obtained by calculating the phase shift fringes through equation 5):

[0063]

[0064] The wrapped phase diagram is used to perform calculations using the sequential unwrapping algorithm and the constant geometry constraint algorithm to obtain the final unwrapped phase diagram. The sequential unwrapping algorithm maps the low-period phase to the high-period phase to obtain a wrapped phase diagram that is highly accurate and easy to unwrap. The constant geometry constraint algorithm actively limits the measurement depth range of the object to obtain the final unwrapped phase diagram.

[0065] The three-dimensional measurement is completed by unwrapping the phase image and performing triangulation calculations to obtain point cloud data.

[0066] The above description of the embodiments is to facilitate the understanding and application of the present invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments. Improvements and modifications made by those skilled in the art to the present invention based on the disclosure of the present invention should be within the protection scope of the present invention.

Claims

1. A phase ambiguity reduction method based on speckle-assisted phase tracking, characterized in that: The method is implemented by the following devices: a camera, a projector, a motion controller and a computing device, wherein the camera is connected to the projector, and the computing device is connected to the camera, the projector and the motion controller respectively, and the method comprises the following steps: The camera is synchronously triggered by the projector, and the computing device sends a signal to start acquisition. The camera can accurately acquire an image sequence containing high-period and low-period three-step phase-shift fringe patterns and speckle patterns; The DIC algorithm is used to calculate the image sequence of the three-step phase-shift fringe pattern and the speckle pattern to obtain the displacement of the projection point on the u-axis and v-axis caused by the object movement; The fringe image is corrected by the displacement of the projection points on the u-axis and v-axis generated by the movement of the object, and the image is remapped to obtain the corrected fringe pattern; By calculating the corrected fringe pattern, the wrapped phase image is obtained; The wrapped phase image is used to perform the order unwrapping algorithm and the constant geometry constraint algorithm to obtain the final unwrapped phase image; The three-dimensional measurement is completed by unwrapping the phase image and performing triangulation calculations to obtain point cloud data.

2. The phase ambiguity reduction method based on speckle-assisted phase tracking according to claim 1, characterized in that: The camera is synchronously triggered by the projector, and the computing device sends a start acquisition signal. The camera can accurately acquire an image sequence containing a high-period and low-period three-step phase-shift fringe pattern and a speckle pattern. Specifically, the projector synchronously triggers the camera, and the computing device connects the projector and the motion controller via a serial port to send a start acquisition signal. ， The phase blur reduction method uses 50-120Hz projected fringe patterns for three-dimensional reconstruction. The projection double periods are 128 and 16 pixels respectively. The projector stores 6 8-bit images at a projection speed of 50-120Hz. A 128-period fringe is projected in 7-bit format, and the speckle pattern is projected in 1-bit format to obtain an image sequence of high and low period three-step phase-shifted fringe patterns and speckle patterns.

3. The phase ambiguity reduction method based on speckle-assisted phase tracking according to claim 1, characterized in that: The DIC algorithm is calculated by the image sequence of the three-step phase-shift fringe pattern and the speckle pattern to obtain the displacement of the projection points on the u-axis and v-axis caused by the object movement. Specifically, the DIC algorithm searches for the matching points of the reference image and the deformed image through the image subset, and uses the zero-mean normalized sum of squares ZNSSD to evaluate the similarity between the reference image and the deformed image: Where S is a subset of the reference image, f(x, y) and g(x′, y′) are the intensities of the reference image and the deformed image, and is the average intensity of the corresponding subset, and the IC-GN algorithm is used for sub-pixel matching to obtain the displacement (Δu0, Δv0) of the reference image and the deformed image: The displacement of the speckle image between frames is small, so a smaller S is set to speed up the matching speed. The modified Ncorr open source software is used to implement the DIC algorithm. The speckle pattern is projected onto the object, which can detect the slight deformation of the projected texture. The reference pattern and the deformed pattern are cross-correlated, and the sub-pixel projection point displacement map of the speckle pattern between the previous and next frames during the motion process is calculated.

4. The phase ambiguity reduction method based on speckle-assisted phase tracking according to claim 1, 2 or 3, characterized in that: The fringe image is corrected by the u-axis and v-axis projection point displacements generated by the object movement, and the image is remapped to obtain the corrected fringe pattern. Specifically, the image is remapped according to the projection point displacement calculated by the DIC method, the phase displacement of the intermediate phase shift image is interpolated and calculated according to formula 3) I in formula 3) n The intensity on (u+nΔu, v+nΔv) is mapped to I′ n (u, v), remap() function is an image remapping algorithm, which can quickly implement image correction and reduce motion blur.

5. The phase ambiguity reduction method based on speckle-assisted phase tracking according to claim 4, characterized in that: The wrapped phase image is obtained by calculating the corrected fringe image, specifically: the corrected fringe image is calculated by projecting the sinusoidal fringe image onto the surface of the object to be measured, and the camera collects the distorted fringes, which is described as: AND n (u, v)=A(u, v)+B(u, v)cos(φ(u, v)+δ n ) 4) Where (u, v) is the coordinate of the camera image target surface, I n is the nth fringe pattern, n∈[0,N), N is the number of phase shift steps3; A and B are the average light intensity and fringe intensity modulation respectively; φ represents the absolute phase; δ n is the phase shift, The wrapped phase is obtained by calculating the phase shift fringes through equation 5):

6. The phase ambiguity reduction method based on speckle-assisted phase tracking according to claim 1, 2, 3 or 5, characterized in that: The described method of performing the sequential unwrapping algorithm and the constant geometric constraint algorithm calculations on the wrapped phase diagram to obtain the final unwrapped phase diagram is as follows: the sequential unwrapping algorithm maps the low-period phase to the high-period phase to obtain a wrapped phase diagram that is highly accurate and easy to unwrap; the constant geometric constraint algorithm actively limits the measurement depth range of the object to obtain the final unwrapped phase diagram.

Images