A Fourier-assisted phase-shift profilometry method for dynamic scenes

Abstract

The present invention discloses a Fourier-assisted phase-shift profile measurement method for dynamic scenes, comprising: S1, judging the motion state of an object according to a motion mask, and if it is a dynamic object, proceeding to step S2; S2, performing Fourier transform on the stripe distortion map of the dynamic object to obtain a corresponding frequency domain image; S3, filtering the frequency domain image obtained in step S2 using a bandpass filter with preset parameters to remove the DC component and obtain a filtered frequency domain image; S4, obtaining the maximum position of the filtered frequency domain image, and using the maximum position as the filter center, filtering the filtered frequency domain image again using a bandpass filter with preset parameters to obtain a baseband component; S5, performing inverse Fourier transform on the baseband component to obtain a complex image group, and calculating the complex image The wrapped phase is obtained by calculating the ratio of the imaginary and real components of the group; S6, the two projected fringe distortion images are respectively calculated by the Fourier transform profilometry of steps S2 to S5 to obtain the wrapped phase of each fringe distortion image; the wrapped phases of two adjacent fringe distortion images of each group are subtracted to obtain two groups of phase differences; S7, the two groups of phase differences are substituted into the standard N-step phase shift profilometry algorithm to compensate for the phase distortion caused by motion and obtain the compensated wrapped phase; and the uncompensated wrapped phase is substituted into the solution of the dual-frequency fringe order to solve the fringe order; S8, the fringe order obtained in step S7 is corrected for the fringe order; S9, the unwrapped phase is obtained using the fringe order corrected in step S8 and the wrapped phase after compensation in step S7 to obtain the dynamic point cloud.

Description

Technical Field

[0001] The present invention relates to the field of optical measurement technology, and in particular to a Fourier-assisted phase-shift profile measurement method for dynamic scenes. Background Art

[0002] In today's world, the demand for 3D measurement of object morphology and surface geometry continues to rise. Based on the number of modes used, mainstream SL (structured light) technologies can be divided into two categories: multi-lens SL and single-lens SL.

[0003] FTP (Fourier Transform Profilometry) is a single-shot SL scheme. To estimate the motion displacement of each point, FTP must first be performed on each distorted fringe image. Specifically, a sinusoidal fringe pattern is projected onto the object, and the captured fringe image changes horizontally. By applying a 1D Fourier transform along the horizontal direction and then applying a suitably designed bandpass filter in the frequency domain, the positive and complex frequency components can be extracted. FTP is highly regarded for its single-shot projection, but its accuracy is limited by spatial encoding, as a group of pixels is used as a codeword, which can introduce errors. FTP can overcome motion vulnerability, but it also suffers from low accuracy. Spectral filtering is a crucial process in FTP and a significant factor affecting FTP measurement accuracy. FTP requires conversion to the frequency domain using an appropriate bandpass filter. This requires manual selection of the filter function and filter window size. Too small a window will lose high-frequency information representing object details; too large a window will fail to eliminate noise and will be susceptible to noise. The need for appropriate bandpass filtering to convert to the frequency domain reduces the accuracy of the final reconstruction. On the other hand, FTP is a single projection, which requires a group of pixels to be reserved as codewords, which will also reduce the reconstruction effect.

[0004] Multi-lens SL enables accurate and dense 3D reconstruction because the depth map is recovered pixel by pixel using a temporal encoding pattern. However, due to the strict static requirements of temporal encoding, multi-lens SL is not suitable for 3D sensing of dynamic scenes. However, with the development of high-speed digital projection and high-speed image sensors, PSP (phase shifting profilometry) has become a trend in measuring moving objects.

[0005] Standard N-step PSP is one of the most widely used and accurate strategies among many proposed SL methods. The standard N-step phase shift profilometry technique uses a set of phase-shifted sinusoidal waveforms to assign an intensity value at a point (x,y) in the distorted fringe distribution captured by the camera as follows:

[0006] I n (x,y)=A(x,y)+B(x,y)cos(φ(x,y)-2πn / N) (1)

[0007] Where n = 0, 1, 2, ..., N-1 represents the phase shift order; A(x, y), B(x, y), and φ(x, y) are all unknown quantities. A(x, y) represents the average image intensity, B(x, y) is the image intensity modulation, and φ(x, y) is the object's phase to be measured. Therefore, at least three phase-shifted fringe patterns are required to calculate the phase to be measured. φ(x, y) is the corresponding wrapping phase, which can be obtained using Equation (2):

[0008]

[0009] The image average intensity A(x,y) and intensity modulation B(x,y) are calculated by equations (3) and (4) respectively:

[0010]

[0011]

[0012] Like the FTP algorithm, the PSP algorithm only obtains the principal phase value, i.e., the wrapped phase. Phase unwrapping is also required to obtain continuous absolute phase values. The greater the number of phase shift steps, the more difficult it is to estimate the object's motion state, and this also reduces the 3D measurement frame rate.

[0013] PSP can achieve accurate and dense 3D reconstruction because the depth map is restored pixel by pixel using a time-domain coding mode, but it requires multiple projections. Due to the strict static requirements of time-domain coding, multi-lens SL is not suitable for 3D sensing in dynamic scenes. In multi-lens SL scenes, object motion will affect phase unfolding. When projecting multiple pictures, the same pixel point in different pictures will be offset, causing the stripe pattern to shift left and right and deform. Therefore, the corresponding three-dimensional coordinate information is also different, causing errors in three-dimensional measurement. The principle of measurement deviation caused by motion is as follows: Figure 1 As shown. Figure 1 It can be seen that at time T1, the points projected by the camera on the object are O1 and O2, corresponding to the points P1 and P2 in the projector imaging; when the object starts to move, at time T2, the points projected by the camera on the object are O1' and O2', corresponding to the points P1' and P2' in the projector imaging plane. Therefore, the original stripes received by the camera at time T1 are P1P2. Due to the movement of the object, the stripes change to P1'P2', and the corresponding stripes move left and right, resulting in the phase distortion of the main value corresponding to the pixel point.

[0014] From the perspective of real-time performance, when dynamic scenes are involved, the phase error caused by motion will lead to serious artifacts, which cannot be ignored even when using high-speed hardware. This motion error greatly limits the application of PSP. Summary of the Invention

[0015] In view of this, the present invention proposes a Fourier-assisted phase-shift profile measurement method for dynamic scenes, and performs three-dimensional reconstruction of dynamic scenes based on dual-frequency fringe structured light. By introducing a dual-frequency cross-projection scheme, a pixel-by-pixel motion detection algorithm and an improved fringe sequence correction method, the problems of phase error and motion artifacts in moving scenes are solved.

[0016] In order to solve the above problems, the present invention proposes the following technical solutions:

[0017] A Fourier-assisted phase-shift profile measurement method for dynamic scenes comprises: S1, judging the motion state of an object based on a motion mask, and if the object is dynamic, proceeding to step S2; S2, performing Fourier transform on the fringe distortion image of the dynamic object to obtain a corresponding frequency domain image; S3, filtering the frequency domain image obtained in step S2 using a bandpass filter with preset parameters to remove the DC component and obtain a filtered frequency domain image; S4, obtaining the maximum position of the filtered frequency domain image, and using the maximum position as the filter center, filtering the filtered frequency domain image again using the bandpass filter with preset parameters to obtain a fundamental frequency component; S5, performing an inverse Fourier transform on the fundamental frequency component to obtain a complex image group, and calculating the ratio of the imaginary and real components of the complex image group to obtain a wrapped phase;

[0018] S6. Calculate the wrapped phase of each fringe distortion image by Fourier transform profilometry in steps S2 to S5 for each of the two projected fringe distortion images; subtract the wrapped phases of two adjacent fringe distortion images in each group to obtain two sets of phase differences; S7. Substitute the two sets of phase differences into a standard N-step phase shift profilometry algorithm to compensate for phase distortion caused by motion, thereby obtaining a compensated wrapped phase; and substitute the uncompensated wrapped phase into the solution of the dual-frequency fringe order to obtain the fringe order; S8. Perform fringe order correction on the fringe order obtained in step S7;

[0019] S9. Calculate the unwrapping phase using the fringe order corrected in step S8 and the wrapped phase compensated in step S7 to obtain a dynamic point cloud.

[0020] Furthermore, step S1 includes: calculating the motion mask by an improved motion background detection algorithm to determine the motion state of the object; if it is a static object, reconstructing it by using a dual-frequency phase shift profilometry algorithm; if it is a dynamic object, entering step S2.

[0021] Furthermore, step S1 calculates the motion mask through an improved motion background detection algorithm, specifically including: S1.1, performing pixel traversal on the two groups of projected fringe distortion images, respectively, to solve the first average grayscale of the first group of fringe distortion images and the second average grayscale of the second group of fringe distortion images; S1.2, comparing the absolute value of the difference between the first average grayscale and the second average grayscale with a preset threshold, and determining an initial motion mask based on the comparison result; S1.3, performing image processing on the initial motion mask to remove burrs and defects to obtain a smoothed motion mask; S1.4, performing an AND operation on the first average grayscale, the second average grayscale and the smoothed motion mask to obtain a final motion mask; S1.5, judging the motion state of the object based on the final motion mask.

[0022] Furthermore, the preset parameters of the bandpass filter in step S3 and step S4 include: the height and width of the bandpass filter in the spectrum, wherein the values ​​of the height and width are selected according to the application scenario and the desired filtering effect, and the width and height are both set to 5 to 25.

[0023] Furthermore, the two groups of fringe distortion images in step S6 are: the first group includes fringe distortion images I1(x, y), I2(x, y), and I3(x, y); the second group includes fringe distortion images I4(x, y), I5(x, y), and I6(x, y); and the phase differences of the two groups are:

[0024] The first phase difference

[0025] The second phase difference

[0026] Among them, FTP[] represents the Fourier transform profilometry calculation of the image.

[0027] Furthermore, step S7 uses the following formula to calculate the compensated wrapping phase:

[0028]

[0029]

[0030] Among them, I1'(x,y)~I6'(x,y) represent the DC components of the sinusoidal carriers of the six fringe distortion images, I1"(x,y)~I6"(x,y) represent the phases of the scene depth modulation of the six fringe distortion images, and φ1(x,y) and φ2(x,y) are the compensated wrapping phases of the two sets of fringe distortion images.

[0031] Furthermore, step S7 substitutes the uncompensated wrapping phase into the solution of the dual-frequency fringe order to solve the fringe order, specifically including:

[0032]

[0033]

[0034] Among them, K(i,j) represents the stripe order of the i-th row and j-th column, ceil is the function of rounding the floating point number upwards, T H With T L They represent the periods of the high-frequency fringe image and the low-frequency fringe image respectively; phase1(i,j) represents the uncompensated wrapping phase of the high-frequency projection, and phase2(i,j) represents the uncompensated wrapping phase of the low-frequency projection.

[0035] Furthermore, step S8 performs stripe order correction specifically including: S8.1, traversing each pixel point detected as moving in the motion mask, and detecting whether the corresponding pixel point in the stripe order two-dimensional vector is a stripe order transition point; S8.2, storing the horizontal coordinates of the pixel points detected as stripe order transition points in the stripe order two-dimensional vector in an array, and correcting the stripe order transition points so that the stripe order shows a linear change trend in the horizontal direction of the image.

[0036] Furthermore, step S9 calculates the unwrapping phase using the following formula: unwrapping phase = compensated wrapping phase + 2*π*K, where K is the corrected fringe order.

[0037] Furthermore, in step S9, the unwrapped phase is converted into three-dimensional coordinates in the real world by triangulation according to the corresponding conversion relationship between the real-world coordinates and the image point coordinates, so as to obtain the dynamic point cloud.

[0038] Compared with the prior art, the beneficial effects of the technical solution of the present invention are reflected in the following aspects: the present invention introduces dual-frequency cross-projection, and the newly captured two frames of image data can form a new set of six frames with the previous four frames before a new point cloud can be generated. This means that the original six frames of image data that required sequential input can now be realized through dual-frequency cross-projection, requiring only the input of two new frames of image data, which are then combined with the previous four frames of image data to form a new set of six frames of image data, thereby reducing the generation time of the new six images. The projection rate can reach 60 Hz, and the data processing speed can reach 30 frames per second. On this basis, the phase distortion problem caused by motion scenes is solved by applying a pixel-by-pixel motion detection algorithm and an improved fringe order correction method in motion measurement, and motion error compensation is performed using FAPS. On the one hand, FTP (Fourier transform profilometry) is used to overcome motion vulnerability, and on the other hand, the accuracy of PSP (phase shift profilometry) is maintained to minimize motion error and overcome motion vulnerability. At the same time, a motion-related phase shift is introduced to replace the phase shift that should have been a known constant. Finally, fringe order correction is performed to optimize compensation. Ultimately, the method of the present invention significantly reduces phase errors and motion artifacts caused by moving objects, and based on the motion compensation calculation method, it can also achieve good results in scenes with non-uniform motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of measurement deviation caused by object motion.

[0040] Figure 2 This is a flow chart of a Fourier-assisted phase-shift profile measurement method for dynamic scenes provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described below with reference to the accompanying drawings and specific examples and implementation methods. It should be understood that the examples are provided for illustration only and are not intended to limit the scope of protection of the present invention.

[0042] In real-time three-dimensional measurement, in order to pursue effectiveness and to be able to measure moving scenes, it is necessary to minimize the phase distortion caused by motion. The embodiments of the present invention mainly consider the following four aspects: accelerating hardware measurement speed, reducing the number of projected images, phase error compensation, and accelerating data processing. Based on this, the embodiments of the present invention propose a Fourier-assisted phase shift profile measurement method for dynamic scenes. By introducing a dual-frequency cross-projection scheme, a pixel-by-pixel motion detection algorithm, and an improved fringe order correction method, the problems of phase error and motion artifacts in moving scenes are solved, overcoming the low precision of FTP and the motion fragility of PSP (which cannot maintain robustness for moving scenes) defects. FAPS combines the advantages of the two methods to achieve compensation for motion phase shift errors, and on the basis of ensuring accuracy, efficiently and robustly realizes three-dimensional reconstruction of dynamic scenes.

[0043] Please refer to Figure 1 The Fourier-assisted phase-shift profile measurement method for dynamic scenes proposed in an embodiment of the present invention includes the following steps S1 to S9:

[0044] S1. Calculate the object's motion mask using an improved motion background detection algorithm to determine the object's motion state. If the object is static, reconstruct it using a dual-frequency phase-shift profilometry algorithm without motion compensation. If the object is dynamic, proceed to step S2.

[0045] The steps of calculating the motion mask of the object using the improved motion background detection algorithm specifically include S1.1 to S1.5:

[0046] S1.1. Perform pixel traversal on each of the two projected fringe distortion images (a total of six images) to determine the first average grayscale A1 of the first set of fringe distortion images and the second average grayscale A2 of the second set of fringe distortion images, as follows:

[0047] A1=(I t1 +I t2 +I t3 ) / 3

[0048] A2=(I t4 +I t5 +I t6 ) / 3

[0049] Among them, I t1 , I t2 , I t3 , I t4 , I t5 , I t6 These are the grayscale values ​​of the six fringe distortion images. The six fringe distortion images include three high-frequency images and three low-frequency images, replacing the traditional three phase-shift images plus six Gray code images. The low-frequency images assist in high-frequency unwrapping.

[0050] S1.2. Compare the absolute value of the difference between the first average grayscale A1 and the second average grayscale A2 with a preset threshold Th, and determine an initial motion mask A according to the comparison result. mask ; The details are as follows:

[0051]

[0052] Among them, the initial motion mask A mask For solving the motion mask used for motion detection, it is an intermediate variable and belongs to the mask image that has not been processed. Compared with the actual object, the mask image has certain burrs and defects, so it needs to be processed by image processing. Preferably, the threshold Th can be set to 10-15;

[0053] S1.3, initial motion mask A mask Perform image processing to remove burrs and defects and obtain a smoothed motion mask;

[0054] S1.4. Perform an AND operation on the first average grayscale A1, the second average grayscale A2, and the smoothed motion mask to obtain a final motion mask;

[0055] S1.5. Determine the motion state of the object based on the final motion mask obtained in step S1.4. If the final motion mask is greater than or equal to Th, determine that the object is a dynamic object; otherwise, determine that the object is a static object.

[0056] The method of calculating the motion mask of the object through an improved motion background detection algorithm to determine whether the object is dynamic or static is an improved frame difference method. It can detect motion within six frames without causing data lag and shows good detection accuracy.

[0057] S2. Perform Fourier transform on the fringe distortion image of the dynamic object to obtain the corresponding frequency domain image.

[0058] S3. Use a bandpass filter with preset parameters to filter the frequency domain image obtained in step S2, filter out the DC component, and obtain a filtered frequency domain image. Here, the two important parameters of the bandpass filter are its width and height in the spectrum, which determine the frequency range that the bandpass filter "retains" or "cuts" in the frequency domain. Their values ​​are usually based on the understanding of the image content and noise characteristics, and are selected according to the specific application scenario and the desired filtering effect. This usually requires some experiments and adjustments to find the best parameter settings. If possible, you can try to use some adaptive methods to dynamically select these parameters. In this step, the filter parameters used to filter out the DC component in the frequency domain image include height dcHeight and width dcWidth. Experiments have shown that the value range of the two parameters between 5 and 25 can achieve better results; preferably, the width and height are equal, forming a square window.

[0059] S4. Calculate the maximum position of the filtered frequency domain image. Using this maximum position as the filter center, apply a bandpass filter to the filtered frequency domain image again to obtain the fundamental frequency component. Since the filtered frequency domain is bilaterally symmetrical, use the OpenCV library function SinusoidalPatternProfilometry_Impl::findMaxInHalvesTransform to traverse the right or left side of the axis and select the maximum position. A window is then created with this maximum position as the center. The window height (bpHeight) and width (bpWidth) are the filter parameters, again ranging from 5 to 25 in this example.

[0060] S5. Perform an inverse Fourier transform on the fundamental frequency components obtained above, transforming from the frequency domain to the time domain, to obtain a complex image group. This complex image group is converted into a three-dimensional vector group, planes. The real part values ​​are stored in the two-dimensional array, planes[0], and the imaginary part values ​​are stored in the two-dimensional array, planes[1]. Then, the ratio of the imaginary and real components of this complex image group is calculated to obtain the wrapped phase.

[0061] S6. For the two groups of projected fringe distortion images, the Fourier transform profilometry calculations of steps S2 to S5 are respectively performed to obtain the wrapping phase of each fringe distortion image; the wrapping phases of two adjacent fringe distortion images in each group are subtracted to obtain two groups of phase differences. In an embodiment of the present invention, the two groups of fringe distortion images include six fringe distortion images, namely: the first group of fringe distortion images I1(x, y), I2(x, y), I3(x, y), and the second group includes fringe distortion images I4(x, y), I5(x, y), I6(x, y). Each fringe distortion image is subjected to Fourier transform profilometry FTP calculation of the corresponding wrapping phase using the method of steps S2 to S5; then, the wrapping phases of each two adjacent fringe distortion images in each group are subtracted to obtain two groups of phase differences, as follows:

[0062] The first phase difference

[0063] The second phase difference

[0064] Wherein, FTP[] indicates that the Fourier transform profilometry calculation is performed on the image according to the aforementioned steps S2 to S5 to obtain the wrapped phase.

[0065] S7. Substitute the two sets of phase differences obtained above into a standard N-step phase-shift profilometry algorithm to compensate for phase distortion caused by motion and obtain a compensated wrapped phase. Substitute the uncompensated wrapped phase into the dual-frequency fringe order solution to obtain the fringe order. In this embodiment of the present invention, a three-step phase-shift profilometry algorithm is used as an example for illustration.

[0066] In an embodiment of the present invention, the compensated wrapping phase is calculated using the following formula:

[0067]

[0068]

[0069] Among them, I1'(x,y)~I6'(x,y) represent the DC components of the sinusoidal carriers of the six fringe distortion images, I1"(x,y)~I6"(x,y) represent the phases of the scene depth modulation of the six fringe distortion images, and φ1(x,y) and φ2(x,y) are the compensated wrapping phases of the two sets of fringe distortion images.

[0070] Substitute the uncompensated wrapping phase into the solution of the dual-frequency fringe order to solve the fringe order as follows:

[0071]

[0072]

[0073] Among them, K(i,j) represents the stripe order of the i-th row and j-th column, ceil is the function of rounding the floating point number upwards, T H With T L They represent the periods of the high-frequency fringe image and the low-frequency fringe image respectively; phase1(i,j) represents the uncompensated wrapping phase of the high-frequency projection, and phase2(i,j) represents the uncompensated wrapping phase of the low-frequency projection.

[0074] S8. Perform fringe order correction on the fringe order obtained in step S7. Experimental results show that, within a row of data, as x gradually increases, most of the corresponding fringe orders can maintain a linear change. However, if some points do not form a continuous linear change trend with the previous and next points, these points are jump points and need to be corrected. Based on this, the following method is used to perform fringe order correction, specifically including:

[0075] S8.1. For each fringe distortion image, traverse each pixel detected as moving in its motion mask and detect whether the corresponding pixel in the fringe order two-dimensional vector is a fringe order transition point;

[0076] S8.2. Store the horizontal coordinates of the pixel points detected as fringe order transition points in the fringe order two-dimensional vector in an array, and correct the fringe order transition points so that the fringe order shows a linear change trend in the horizontal direction of the image.

[0077] S9. Calculate the unwrapped phase using the fringe order corrected in step S8 and the wrapped phase compensated in step S7, thereby obtaining a dynamic point cloud. The specific formula is: Unwrapped Phase = Compensated Wrapped Phase + 2*π*K, where K is the corrected fringe order. After obtaining the unwrapped phase, the unwrapped phase is converted to real-world 3D coordinates through triangulation using calibrated parameters and the corresponding conversion relationship between real-world coordinates and image point coordinates, thereby obtaining a dynamic point cloud.

[0078] In the aforementioned embodiment of the present invention, the six projected fringe distortion images, including three high-frequency and three low-frequency images, replace the traditional three phase-shifted images plus six Gray codes, with the low-frequency image assisting with high-frequency unwrapping. Furthermore, the encoding scheme employed is dual-frequency projection, and the decoding scheme is dual-frequency heterodyning. The front and back images are also integrated, totaling six images.

[0079] The aforementioned embodiments of the present invention combine motion compensation and fringe order correction with the FTP and PSP algorithms to correct them near jump points in the linear region. The results show a mean absolute error (MAE) improvement of 0.4 to 1.4 mm, highlighting its advantages in scenes with non-uniform motion. However, this does not mean that the present invention is only applicable to scenes with non-uniform motion. The present method is applicable to 3D reconstruction of various dynamic scenes, as well as static objects.

[0080] In addition, in order to further realize high-speed, low-resource consumption three-dimensional measurement, an FPGA architecture for the FAPS compensation algorithm is further proposed. There are two difficulties in transplanting line structured light to FPGA: the implementation of the filter operator and the calculation of floating-point numbers. Another embodiment of the present invention uses the HLS (High-Level Synthesis) tool to convert the description of the filter operator into a hardware description language, and maps it to the RAM of the FPGA. By utilizing the parallelism of the FPGA, the filter operator is operated in the RAM, and efficient filter operator operations are achieved through reasonable data structures and access modes. This design makes full use of the parallelism of the FPGA hardware and the high-bandwidth characteristics of the memory to achieve fast filtering processing of the input data. Based on this, the Fourier-assisted phase shift profile measurement method for dynamic scenes provided by the embodiment of the present invention may further include the following step S10:

[0081] S10. Design the HLS hardware implementation of the Fourier-assisted phase shift profilometry algorithm, and implement the image reading, wrapped phase calculation, unwrapped phase and 3D point cloud coordinate calculation modules through parallel and pipeline.

[0082] Step S10 specifically includes:

[0083] S10.1. For filter design, first read the filter parameters and input data from the input buffer or RAM;

[0084] S10.2. Perform a weighted summation of the input data and the filter parameters according to the convolution operation formula to obtain a filtered output result;

[0085] S10.3. Write the filtered output result into the output buffer or RAM;

[0086] S10.4. Design an appropriate data path structure (incorporating pipelining and parallel processing, primarily in the form of pipelines or dataflows to achieve pipelining and parallelization and increase processing speed) to process input data and filter parameters in parallel to increase the speed of the algorithm.

[0087] S10.5. For functions in the OpenCV library in the code (which do not have the same function in HLS), manually rewrite the functions in HLS according to their specific principles, such as FFT two-dimensional Fourier transform, mask calculation, Gaussian filtering and other functions.

[0088] S10.6. By adding appropriate pipeline design to the HLS code, such as operation-level pipelining and function-level pipelining, unrelated functions can be processed in parallel. For example, the FAPS module and the PSP calculation module can be calculated in parallel.

[0089] The benefit of accelerating code through hardware HLS is that, using HLS (High-Level Synthesis) technology, the algorithm logic is directly mapped to the hardware, achieving highly parallel computing and improving processing speed and throughput. Secondly, the processing module implemented in the hardware can directly access the data, reducing the delay in data transmission and storage, and improving the response speed and real-time performance of the algorithm. In terms of power consumption, FPGA, as a programmable hardware platform, has the characteristics of flexibility and low power consumption. Compared with traditional processors, it can achieve lower power consumption with the same performance. In terms of real-time performance, hardware HLS accelerated processing can achieve hardware acceleration of the algorithm, improve the real-time performance and response speed of the algorithm, and meet the needs of real-time data processing and computing.

[0090] The above content is a further detailed description of the present invention in conjunction with specific embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the art to which the present invention belongs, several equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and the performance or use are the same, and all of them should be considered to fall within the scope of protection of the present invention.

Claims

1. A Fourier-assisted phase-shift profile measurement method for dynamic scenes, characterized in that: The following steps are involved: S1. Determine the object's motion state based on the motion mask. If it is a dynamic object, proceed to step S2. S2. Perform Fourier transform on the fringe distortion image of the dynamic object to obtain the corresponding frequency domain image; S3, filtering the frequency domain image obtained in step S2 using a bandpass filter with preset parameters to remove the DC component and obtain a filtered frequency domain image; S4, obtaining the maximum position of the filtered frequency domain image, and using the maximum position as the filter center, filtering the filtered frequency domain image again using the bandpass filter with the preset parameters to obtain a fundamental frequency component; S5. Performing an inverse Fourier transform on the fundamental frequency component to obtain a complex image group, and calculating a ratio of an imaginary component to a real component of the complex image group to obtain a wrapped phase; S6. Calculate the wrapped phase of each fringe distortion image by Fourier transform profilometry in steps S2 to S5 for each of the two projected fringe distortion images; and subtract the wrapped phases of two adjacent fringe distortion images in each group to obtain two sets of phase differences. S7, substituting the two sets of phase differences into a standard N-step phase shift profilometry algorithm to compensate for phase distortion caused by motion, and obtaining a compensated wrapping phase; The uncompensated wrapped phase is substituted into the solution of the dual-frequency fringe order to solve the fringe order; S8, performing fringe order correction on the fringe order obtained in step S7; S9. Calculate the unwrapping phase using the fringe order corrected in step S8 and the wrapped phase compensated in step S7 to obtain a dynamic point cloud.

2. The Fourier-assisted phase-shift profile measurement method for dynamic scenes according to claim 1, characterized in that: Step S1 includes: calculating the motion mask by an improved motion background detection algorithm to determine the motion state of the object; if it is a static object, reconstructing it by using a dual-frequency phase shift profilometry algorithm; if it is a dynamic object, proceeding to step S2.

3. The Fourier-assisted phase-shift profile measurement method for dynamic scenes according to claim 2, characterized in that: Step S1 calculates the motion mask using an improved motion background detection algorithm, specifically including: S1.

1. Perform pixel traversal on each of the two projected fringe distortion images to determine the first average grayscale of the first fringe distortion image and the second average grayscale of the second fringe distortion image. S1.

2. Compare the absolute value of the difference between the first average grayscale and the second average grayscale with a preset threshold, and determine an initial motion mask based on the comparison result; S1.

3. Performing image processing on the initial motion mask to remove burrs and defects to obtain a smoothed motion mask; S1.

4. Perform an AND operation on the first average grayscale, the second average grayscale, and the smoothed motion mask to obtain a final motion mask; S1.

5. Determine the motion state of the object based on the final motion mask.

4. The Fourier-assisted phase-shift profile measurement method for dynamic scenes according to claim 1, characterized in that: The preset parameters of the bandpass filter in step S3 and step S4 include: the height and width of the bandpass filter in the spectrum, wherein the values ​​of the height and width are selected according to the application scenario and the desired filtering effect, and the width and height are both set to 5 to 25.

5. The Fourier-assisted phase-shift profile measurement method for dynamic scenes according to claim 1, wherein: The two groups of fringe distortion images in step S6 are: The first group includes fringe distortion images I1(x,y), I2(x,y), and I3(x,y); The second group includes fringe distortion images I4(x,y), I5(x,y), and I6(x,y); The two sets of phase differences are: The first phase difference The second phase difference Among them, FTP[] represents the Fourier transform profilometry calculation of the image.

6. The Fourier-assisted phase-shift profile measurement method for dynamic scenes according to claim 5, characterized in that: Step S7 uses the following formula to calculate the compensated wrapping phase: Among them, I1'(x,y)~I6'(x,y) represent the DC components of the sinusoidal carriers of the six fringe distortion images, I1"(x,y)~I6"(x,y) represent the phases of the scene depth modulation of the six fringe distortion images, and φ1(x,y) and φ2(x,y) are the compensated wrapping phases of the two sets of fringe distortion images.

7. The Fourier-assisted phase-shift profile measurement method for dynamic scenes according to claim 1, characterized in that: Step S7 substitutes the uncompensated wrapping phase into the solution of the dual-frequency fringe order to solve the fringe order, specifically including: Among them, K(i,j) represents the stripe order of the i-th row and j-th column, ceil is the function of rounding the floating point number upwards, T H With T L They represent the periods of the high-frequency fringe image and the low-frequency fringe image respectively; phase1(i,j) represents the uncompensated wrapping phase of the high-frequency projection, and phase2(i,j) represents the uncompensated wrapping phase of the low-frequency projection.

8. The Fourier-assisted phase-shift profile measurement method for dynamic scenes according to claim 1, wherein: Step S8 of performing fringe order correction specifically includes: S8.

1. Traverse each pixel detected as moving in the motion mask and detect whether the corresponding pixel in the fringe order two-dimensional vector is a fringe order transition point; S8.

2. Store the horizontal coordinates of the pixel points detected as fringe order transition points in the fringe order two-dimensional vector in an array, and correct the fringe order transition points so that the fringe order shows a linear change trend in the horizontal direction of the image.

9. The Fourier-assisted phase-shift profile measurement method for dynamic scenes according to claim 1, wherein: Step S9 calculates the unwrapping phase using the following formula: Unwrapped phase = compensated wrapped phase + 2*π*K, where K is the corrected fringe order.

10. The Fourier-assisted phase-shift profile measurement method for dynamic scenes according to claim 1, wherein: In step S9, according to the corresponding conversion relationship between the real-world coordinates and the image point coordinates, the unwrapped phase is converted into three-dimensional coordinates in the real world through triangulation, so as to obtain the dynamic point cloud.

Images