A Phase Ambiguity Reduction Method Based on Speckle-Assisted Phase Tracking

Abstract

The present invention discloses a phase blur reduction method based on speckle-assisted phase tracking. By adding a speckle pattern to a phase-shifted projection sequence and using a DIC algorithm to calculate the sub-pixel projection point displacement between previous and next frames, the method can accurately correct motion errors in the phase-shifted image, significantly improving the accuracy of three-dimensional measurement. The projection points of the phase-shifted image are aligned to the same camera coordinates through a remapping technique, effectively reducing motion blur errors and ensuring the accuracy of the measurement results. By tracking the projection points rather than the movement of the object, motion errors with z-direction displacement can be corrected. A wrapped phase image is obtained through a phase shift method, and phase unwrapping is performed in combination with a dual-frequency order method and a constant geometric constraint method, ensuring the accuracy and continuity of phase unwrapping, thereby reconstructing the three-dimensional shape of the object using a triangulation method.

Description

Technical Field

[0001] The present invention relates to an optical three-dimensional measurement method based on digital image correlation (DIC) technology and phase tracking technology, in particular to a phase ambiguity reduction method based on speckle-assisted phase tracking. Background Art

[0002] Optical three-dimensional measurement technology is widely used in industrial measurement, reverse engineering and other fields. Among them, fringe projection profilometry (FPP) has become one of the most popular technologies due to its high precision and full-field measurement advantages. A typical FPP system consists of a camera and a projector, which can obtain three-dimensional surface information of the object to be measured through phase demodulation and triangulation. Traditional Fourier transform and phase shift methods are often used to obtain the phase of the object. Among them, the Fourier transform method only needs to project a single sinusoidal fringe to obtain the phase map, which is suitable for three-dimensional measurement of dynamic scenes, but this method has difficulty measuring complex scenes. The phase shift method requires the projection of multiple fringe patterns to complete the measurement process, but the movement of the object will destroy the phase measurement process, resulting in motion-induced errors and reduced measurement accuracy.

[0003] In recent years, many methods have been proposed to compensate for motion-induced errors, such as motion tracking, FTP and PSP combination, self-resistance method, etc. However, these methods have certain limitations and it is difficult to achieve high-precision measurements in high-speed dynamic scenes.

[0004] The present invention has the following technical problems:

[0005] 1) Existing technologies have difficulty measuring complex scenes. Phase-shifting methods require projecting multiple fringe patterns to complete the measurement process, but the movement of the object can disrupt the phase measurement process, introducing motion-induced errors and reducing measurement accuracy.

[0006] 2) From the perspective of fringe design, existing technologies believe that uniformly moving objects will cause additional phase differences. Designing a fringe sequence that can offset this error does not require the introduction of additional complex calculations and can improve efficiency. However, it is only applicable to small displacement scenarios and has a limited range.

[0007] 3. Existing technologies cannot adapt to measurement requirements of different resolutions. High-precision measurement and efficient rough measurement cannot be met at the same time, and the scope of application is limited.

[0008] 4 Existing technologies use the SIFT algorithm combined with a region segmentation algorithm to track objects, automatically reduce multi-object motion errors, detect the two-dimensional sub-pixel displacement of objects, and achieve phase compensation for non-uniform moving objects. However, it can only achieve 2D tracking at the image level and cannot correct motion errors with z-direction displacement.

[0009] 5 The existing technology estimates the motion of an object through two adjacent 3D frames and uses an iterative method to obtain the phase shift caused by the object's movement at different positions. However, in the process of estimating the three-dimensional motion of the object, there will be large errors due to the influence of motion noise.

[0010] SUMMARY OF THE INVENTION

[0011] To address the deficiencies of the prior art, the present invention aims to provide a phase ambiguity reduction method based on speckle-assisted phase tracking to achieve high-precision measurement in high-speed dynamic scenarios.

[0012] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0013] The present invention discloses a phase ambiguity reduction method based on speckle-assisted phase tracking. The method is implemented by the following apparatus: a camera, a projector, a motion controller, and a computing device. The camera and the projector are connected, and the computing device is connected to the camera, the projector, and the motion controller respectively. The method comprises the following steps:

[0014] The camera is synchronously triggered by the projector, and the computing device sends a start acquisition signal. The camera can accurately capture an image sequence containing high-period and low-period three-step phase-shift fringe patterns and speckle patterns;

[0015] The DIC algorithm is used to calculate the image sequence of the three-step phase-shifted fringe pattern and the speckle pattern to obtain the displacement of the projection point on the u-axis and v-axis caused by the object motion;

[0016] The fringe image is corrected by the displacement of the u-axis and v-axis projection points generated by the object movement, and the image is remapped to obtain the corrected fringe pattern;

[0017] By calculating the corrected fringe pattern, the wrapped phase image is obtained;

[0018] The wrapped phase image is calculated by performing the level unwrapping algorithm and the constant geometric constraint algorithm to obtain the final unwrapped phase image;

[0019] The three-dimensional measurement is completed by performing triangulation calculations by unwrapping the phase image and obtaining point cloud data.

[0020] As a further improvement, the camera of the present invention is synchronously triggered by the projector, and the computing device sends a start acquisition signal. The camera can accurately capture an image sequence containing high-period and low-period three-step phase-shift fringe patterns and speckle patterns, specifically:

[0021] The projector synchronously triggers the camera, and the computing device connects the projector and motion controller via a serial port to send a start acquisition signal. The phase blur reduction method projects a fringe pattern at 50-120Hz for 3D reconstruction, with dual projection periods of 128 and 16 pixels, respectively. The projector stores six 8-bit images at a projection speed of 50-120Hz, projects a 128-period fringe pattern in 7-bit format, and projects a speckle pattern in 1-bit format, resulting in an image sequence of high- and low-period three-step phase-shifted fringe patterns and speckle patterns.

[0022] As a further improvement, the present invention performs DIC algorithm calculations on image sequences of three-step phase-shifted fringe patterns and speckle patterns to obtain the displacements of projection points on the u-axis and v-axis caused by object motion. Specifically, the DIC algorithm searches for matching points between the reference image and the deformed image through image subsets, and uses the zero-mean normalized sum of squares (ZNSSD) to evaluate the similarity between the reference image and the deformed image:

[0023]

[0024] Where S is a subset of the reference image, f(x, y) and g(x′, y′) are the intensities of the reference image and the deformed image, and The IC-GN algorithm is used to perform sub-pixel matching to obtain the displacement (Δu0, Δv0) between the reference image and the deformed image:

[0025]

[0026] The displacement of speckle images between frames is small, so a smaller S is set to speed up the matching speed. The DIC algorithm is implemented using the modified Ncorr open source software. The speckle pattern is projected onto the object, which can detect slight deformations of the projected texture. The reference pattern and the deformed pattern are cross-correlated, and the sub-pixel projection point displacement map of the speckle pattern between the previous and next frames during the motion process is calculated.

[0027] As a further improvement, the present invention corrects the fringe image by using the u-axis and v-axis projection point displacements generated by the object motion, and remaps the image to obtain a corrected fringe pattern. Specifically, the image is remapped according to the projection point displacement calculated by the DIC method, the phase displacement of the intermediate phase shift image is interpolated, and the phase displacement is calculated according to Formula 3).

[0028]

[0029] I in formula 3) n The intensity on (u+nΔu, v+nΔv) is mapped to I′ n (u, v), remap() function is an image remapping algorithm that quickly implements image correction and reduces motion blur.

[0030] As a further improvement, the present invention obtains a wrapped phase image by calculating the corrected fringe pattern, specifically: the sinusoidal fringe pattern is projected onto the surface of the object to be measured to calculate the corrected fringe pattern, and the camera collects the distorted fringes, which is described as:

[0031] I n (u, v) = A (u, v) + B (u, v) cos (φ (u, v) + δ n )4)

[0032] Where (u, v) is the coordinate of the camera image target surface, I n is the nth fringe pattern, n∈[0,N), N is the number of phase shift steps3; A and B are the average light intensity and fringe intensity modulation respectively; φ represents the absolute phase; δ n is the phase shift, The wrapping phase is obtained by calculating the phase shift fringes using Equation 5):

[0033]

[0034] As a further improvement, the present invention describes the use of a wrapped phase diagram to perform a hierarchical unwrapping algorithm and a constant geometric constraint algorithm to calculate and obtain the final unwrapped phase diagram. Specifically, the hierarchical unwrapping algorithm maps the low-period phase to the high-period to obtain a wrapped phase diagram that is highly accurate and easy to unwrap, and the constant geometric constraint algorithm actively limits the measurement depth range of the object to obtain the final unwrapped phase diagram.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] By adding a speckle pattern to the phase-shift projection sequence and using the DIC algorithm to calculate the sub-pixel projection point displacement between the previous and next frames, the motion error in the phase-shift image can be accurately corrected, significantly improving the accuracy of three-dimensional measurement.

[0037] 2 The present invention aligns the projection points of the phase-shifted image to the same camera coordinates through remapping technology, effectively reducing the motion blur error and thus ensuring the accuracy of the measurement results.

[0038] 3 The present invention introduces a high-precision DIC algorithm and speckle-assisted technology, which can achieve stable three-dimensional measurement in high-speed displacement scenarios and has high robustness and reliability.

[0039] The projector of the present invention can project sinusoidal fringe patterns of different periods to meet the measurement requirements of different resolutions, thereby flexibly switching between high-precision measurement and efficient rough measurement, further improving the application range of the system.

[0040] 5 This method tracks the projection points instead of the object's motion, and can correct the motion error with z-direction displacement.

[0041] 6 The wrapped phase image is obtained by the phase shift method, and the phase unwrapping is performed in combination with the dual-frequency order method and the constant geometric constraint method to ensure the accuracy and continuity of the phase unwrapping, so as to reconstruct the three-dimensional shape of the object using the triangulation method.

[0042] The camera, projector, computing device, and motion controller in the present invention work in a coordinated manner through synchronous control, thereby ensuring efficient collaboration among all parts of the system and improving the overall performance in the field of dynamic three-dimensional measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 Schematic diagram of the process of this method;

[0045] Figure 2 Figure 1 is the experimental setup diagram of this method;

[0046] Figure 3 Projection sequence diagram for this method; DETAILED DESCRIPTION

[0047] The present invention discloses a phase ambiguity reduction method based on speckle-assisted phase tracking. The method is implemented by the following devices: a camera, a projector, a motion controller and a computing device. The camera and the projector are connected, and the computing device is connected to the camera, the projector and the motion controller respectively. A sinusoidal fringe pattern is projected onto the object to be measured. The distorted fringes are received by the camera, and after phase demodulation, three-dimensional surface information of the object to be measured can be obtained by triangulation. The flow chart of the method is shown as follows Figure 1 The experimental setup of the method is shown in Figure 2. Figure 2 As shown, the method includes the following steps:

[0048] The camera is triggered synchronously by the projector, and the computing device sends a start acquisition signal. The camera can accurately capture an image sequence containing high-period and low-period three-step phase-shifted fringe patterns and speckle patterns. The projector triggers the camera synchronously, and the computing device sends a start acquisition signal to the projector and motion controller via a serial port. The phase blur reduction method uses a 50-120Hz projection fringe pattern for three-dimensional reconstruction. The projection double periods are 128 and 16 pixels respectively. The projector stores 6 8-bit images. At a projection speed of 50-120Hz, a 128-period fringe is projected in a 7-bit format, and the speckle pattern is projected in a 1-bit format, obtaining an image sequence of high- and low-period three-step phase-shifted fringe patterns and speckle patterns. The projection sequence is as follows: Figure 3 As shown, the fringe pattern in motion can be described as:

[0049]

[0050] Where I0 is the speckle image of the previous group, I1-I6 are the fringe images of the current pattern, T L ,T H are low and high cycles respectively, I7 is the current speckle image, Δu and Δv are the u-axis and v-axis displacements of the projection point relative to the speckle pattern I0 at different times.

[0051] The DIC algorithm calculates the displacement of the u-axis and v-axis projection points caused by the object motion through the image sequence of the three-step phase-shifted fringe pattern and the speckle pattern. The DIC algorithm finds the matching points between the reference image and the deformed image through the image subset and uses the zero-mean normalized sum of squares (ZNSSD) to evaluate the similarity between the reference image and the deformed image:

[0052] Where S is a subset of the reference image, f(x, y) and g(x′, y′) are the intensities of the reference image and the deformed image, and The IC-GN algorithm is used to perform sub-pixel matching to obtain the displacement (Δu0, Δv0) between the reference image and the deformed image:

[0053] The speckle image displacement between frames is small, so a smaller S is set to speed up the matching speed. The modified Ncorr open source software is used to implement the DIC algorithm. The speckle pattern is projected onto the object, which can detect slight deformations of the projected texture. The reference pattern and the deformed pattern are cross-correlated, and the sub-pixel projection point displacement map of the speckle pattern between the previous and next frames during the motion is calculated. By adding a frame of speckle image to the fringe sequence, the projection points between the speckle images are tracked at the sub-pixel level. The speckle patterns of the current frame and the previous frame are used with the DIC algorithm to obtain the u-axis and v-axis projection point displacements (u0, v0) caused by the object's motion. Combined with Formula 9, we obtain:

[0054]

[0055] While these points are geometrically matched, they exhibit a different neighborhood distribution. The speckle image is projected onto the object's surface rather than attached to it, and object motion causes misalignment between the projected and object materials. Unlike target tracking, the proposed method tracks projected points, not object motion. This makes it suitable for z-axis motion, and the resulting speckle matching results cannot be used for 3D reconstruction.

[0056] The fringe image is corrected by the displacement of the u-axis and v-axis projection points generated by the object movement, and the image is remapped to obtain the corrected fringe pattern; after obtaining the fringe pattern displacement, the fringe pattern displacement is calculated by formula 6.

[0057]

[0058] The true phase can be restored by interpolating the phase shift of the intermediate phase shift map obtained by image remapping according to the projection point displacement calculated by the DIC method and according to Equation 3) I in formula 3) n (The intensity on u+nΔu, v+nΔv is mapped to I′ n (u, c), the remap() function is an image remapping algorithm that quickly calculates the wrapping phase to achieve image correction and reduce motion blur.

[0059] The wrapped phase image is obtained by calculating the corrected fringe pattern. The corrected fringe pattern is calculated by projecting the sinusoidal fringe pattern onto the surface of the object to be measured. The camera collects the distorted fringes, which can be described as:

[0060] I n (u, v) = A (u, v) + B (u, v) cos (φ (u, v) + δ n )4)

[0061] Where (u, v) is the coordinate of the camera image target surface, I nis the nth fringe pattern, n∈[0,N), N is the number of phase shift steps. The larger the number of phase shift steps N, the higher the measurement accuracy and the lower the efficiency. In this method, N=3 is selected; A and B are the average light intensity and fringe intensity modulation respectively; φ represents the absolute phase; δ n is the phase shift,

[0062] The wrapping phase is obtained by calculating the phase shift fringes using Equation 5):

[0063]

[0064] The wrapped phase image is calculated by the hierarchical unwrapping algorithm and the constant geometric constraint algorithm to obtain the final unwrapped phase image; the hierarchical unwrapping algorithm maps the low-period phase to the high-period to obtain a highly accurate and easily unwrapped wrapped phase image, and the constant geometric constraint algorithm actively limits the measurement depth range of the object to obtain the final unwrapped phase image.

[0065] The three-dimensional measurement is completed by performing triangulation calculations by unwrapping the phase image and obtaining point cloud data.

[0066] The above description of the embodiments is intended to facilitate understanding and application of the present invention by those skilled in the art. It is apparent that those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without requiring creative effort. Therefore, the present invention is not limited to the above embodiments. Any improvements or modifications made by those skilled in the art based on the disclosure of the present invention should fall within the scope of protection of the present invention.

Claims

1. A phase ambiguity reduction method based on speckle-assisted phase tracking, characterized in that: The method is implemented by the following devices: a camera, a projector, a motion controller and a computing device, wherein the camera and the projector are connected, and the computing device is connected to the camera, the projector and the motion controller respectively. The method comprises the following steps: The camera is synchronously triggered by the projector, and the computing device sends a start acquisition signal. The camera can accurately capture an image sequence containing high-period and low-period three-step phase-shift fringe patterns and speckle patterns; The DIC algorithm is used to calculate the image sequence of the three-step phase-shifted fringe pattern and the speckle pattern to obtain the displacement of the projection point on the u-axis and v-axis caused by the object motion; The fringe image is corrected by the displacement of the u-axis and v-axis projection points generated by the object motion, and the image is remapped to obtain the corrected fringe pattern; By calculating the corrected fringe pattern, the wrapped phase image is obtained; The wrapped phase image is calculated by performing the level unwrapping algorithm and the constant geometric constraint algorithm to obtain the final unwrapped phase image; The three-dimensional measurement is completed by performing triangulation calculations by unwrapping the phase image and obtaining point cloud data; The fringe image is corrected by the displacement of the projection points on the u-axis and v-axis generated by the object motion, and the image is remapped to obtain the corrected fringe pattern. Specifically, the image is remapped according to the projection point displacement calculated by the DIC method, the phase displacement of the intermediate phase shift image is interpolated and calculated according to formula 3). 3）； In formula 3) The intensity on the ,remap() function is an image remapping algorithm, which quickly implements image correction and reduces motion blur; The wrapped phase image is obtained by calculating the corrected fringe pattern. Specifically, the corrected fringe pattern is calculated by projecting the sinusoidal fringe pattern onto the surface of the object to be measured, and the camera collects the distorted fringes, which can be described as: 4）； Where (u, v) is the coordinate of the camera image target surface, is the nth fringe pattern, , N is the phase shift step number 3; A and B are the average light intensity and fringe intensity modulation respectively; Expressed as absolute phase; is the phase shift, , the wrapping phase is obtained by calculating the phase shift fringe through Equation 5): 5）。 2. The phase ambiguity reduction method based on speckle-assisted phase tracking according to claim 1, characterized in that: The camera is synchronously triggered by the projector, and the computing device sends a start acquisition signal. The camera can accurately capture an image sequence containing a high-period and low-period three-step phase-shifted fringe pattern and a speckle pattern, specifically: The projector synchronously triggers the camera. The computing device connects the projector and motion controller via a serial port to send a start acquisition signal. The phase ambiguity reduction method projects a fringe pattern at 50-120Hz for 3D reconstruction. The projection double period is 128 and 16 pixels, respectively. The projector stores six 8-bit images at a projection speed of 50-120Hz. A 128-period fringe pattern is projected in a 7-bit format, and the speckle pattern is projected in a 1-bit format. This results in an image sequence of high- and low-period three-step phase-shifted fringe patterns and speckle patterns.

3. The phase ambiguity reduction method based on speckle-assisted phase tracking according to claim 1 or 2, characterized in that: The described method of performing the graded unwrapping algorithm and the constant geometric constraint algorithm on the wrapped phase map to obtain the final unwrapped phase map is as follows: the graded unwrapping algorithm maps the low-period phase to the high-period phase to obtain a highly accurate and easily unwrapped wrapped phase map; the constant geometric constraint algorithm actively limits the measurement depth range of the object to obtain the final unwrapped phase map.

Images