A false point correction method for black and shadow areas in structured light three-dimensional measurement and a structured light three-dimensional measurement method

Abstract

The application discloses a false point correction method for black and shadow areas in a structured light three-dimensional measurement and a structured light three-dimensional measurement method, and comprises the following steps: S1, acquiring all phase shift images required for unwrapping phase solution of a measured object; S2, obtaining an absolute phase image of the measured object according to the phase shift images; S3, taking a phase shift image of a frequency from the phase shift images solved in the step S1, and obtaining a mask through a cyclic phase shift difference method, the mask separates the shadow area and the black area of the measured object from a normal light area; and S4, multiplying the absolute phase image of the measured object obtained in the step S2 and the mask obtained in the step S3 to obtain a corrected absolute phase image of the measured object in the black area and the shadow area. The application can obtain a 3D point cloud without stray points and false points without any post-processing operation, can guarantee that a calibration parameter is not disturbed by the false points, can effectively improve measurement accuracy, can save a stray noise post-processing link, and can effectively improve measurement efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of three-dimensional measurement, in particular to a method for correcting false points in black and shadow regions in structured light three-dimensional measurement and a structured light three-dimensional measurement method. BACKGROUND

[0002] Fringe projection profilometry (FPP) is a representative technique for precision three-dimensional measurement. It is widely used in computer vision, industrial defect detection, reverse engineering and other fields for precision measurement due to its high resolution, high precision and non-contact manner. The system is usually composed of a camera, a projector and a microprocessor. In a well-calibrated FPP system, the projector projects a pattern with phase information encoding onto the target object. These patterns are modulated when interacting with the object and then reflected into the camera. After phase unwrapping and phase-to-height mapping, the processor produces a 3D point cloud, presenting the spatial characteristics of the measured object. However, there are two problems in the field of structured light three-dimensional measurement that have not been well solved so far: how to effectively solve the influence of black and shadow regions on structured light three-dimensional measurement. In the structured light three-dimensional measurement technology, due to the straight-line propagation characteristics of light, the convexity of the surface profile of the measured object will block the propagation of light, thereby producing shadows.

[0003] 1. These shadows appear as dark areas in the phase shift images taken by the camera. However, due to the effects of camera defocus, nonlinear noise errors of the camera and projector, random noise, reflectivity of the measured object and ambient light, in the time-series phase unwrapping algorithm, the shadow regions often solve "false" phase values, which are then converted into "false point clouds", affecting the authenticity of three-dimensional reconstruction.

[0004] 2. In addition to traditional algorithms, the correct solution of shadow regions also has an important impact on deep learning strategies that require accurate data sets. In recent years, deep learning technology has been widely used in FPP, especially in the absolute phase recovery stage. In the deep learning prediction system from grating to absolute phase, the accuracy and precision of the true value (absolute phase) play a decisive role in the prediction result. However, in the existing unwrapping algorithm, there is still the problem of "false" phase in the shadow region.

[0005] 3. In the application scene of structured light three-dimensional measurement technology, the reflectivity of black area is relatively low, and the phase recovery is not accurate, so the accurate three-dimensional measurement of black object in structured light technology is very difficult. This phenomenon is particularly prominent in the system calibration method of one-way structured light system, the equal phase plane model, the Han calibration model and other system calibration methods. The common feature of the above calibration methods is that in the system calibration process, the projector projects the phase on the calibration board of different calibration positions in turn, and then the calculation unit calculates the phase. The absolute phase and three-dimensional coordinates are mapped pixel by pixel to complete the calculation of system calibration parameters. The method with the above characteristics is collectively referred to as the pixel-by-pixel phase model mapping calibration method. The calibration board is mainly divided into two types of chessboard and circle, and the commonly used calibration board has only black and white two colors. Finally, due to the small phase signal-to-noise ratio (SNR) in the dark area, the absolute phase of the black area is distorted, which directly affects the result of phase mapping. Therefore, the unwrapping of black objects is not only present in the measured object, but also in the calibration part, which directly determines the three-dimensional measurement result.

[0006] In summary, the distorted points presented by the shadow area and the black area in the unwrapping phase are collectively referred to as false points. For the false points caused by the shadow area of the measured object, the existing solutions are divided into two categories. The first method is to perform denoising post-processing on the point cloud calculated by the false points. However, point cloud segmentation and removal based on height threshold is difficult to completely remove noisy point clouds. The other is to directly identify and remove the shadow area. The energy function method shows that the background pixels are farther away from the ideal sinusoidal curve. In order to filter out these noisy pixels, an adaptive threshold is calculated. However, it considers the pixels in the vicinity, and the calculation is complex. The modulation map method based on the reflectivity of the surface. If the measured object is black, the modulation level will also decrease, and in this case, the background may not be able to distinguish correctly. The prior art also uses a K-means threshold adaptive setting method, but K-means needs to specify the initial K value, otherwise it may not work well. The prior art also uses a generative adversarial network (GAN) to fill in the shadow stripes when given an object image with shadows and a sinusoidal phase map. However, preparing the true value requires additional projection, and the cost of artificial and technical data set generation is high.

[0007] To address the impact of black areas on calibration, existing research approaches fall into two main categories: improving the calibration plate pattern design or using methods such as plane fitting or statistical error removal to address the impact of black areas on phase calculation. To minimize the percentage of black areas, existing calibration methods utilize non-standard calibration plates with circular calibration plates of varying diameters, with the larger one used for coarse calibration and the smaller one for fine calibration. Alternatively, the black dots on the calibration plate are replaced with two circular arcs of varying diameters. This plate is a blue plane with purple dots that can be projected with red light to resemble a standard black and white plate. These methods require redesigning the calibration plate pattern and calibration algorithm and fail to fundamentally address the impact of the black calibration pattern. Plane fitting methods use black areas for polynomial fitting, but using noisy pixel values ​​in these areas can lead to inaccurate plane fitting. Statistical error removal methods categorize pixel errors into five categories based on their pixel error. Removing these error points can effectively improve measurement accuracy, but removing the error point cloud results in missing points, which can affect 3D reconstruction.

[0008] In summary, during the phase solution process, shadowed areas can lead to "false" phase problems, while "black" areas significantly impact the phase solution and system calibration process. The 3D point cloud contains both "false" noise points from the shadowed areas and "flawed point clouds" caused by calibration in the black checkerboard area. Even more challenging, these "fake point clouds" are located around the true 3D point cloud. The similar data makes correction difficult through simple point cloud post-processing. Summary of the Invention

[0009] In order to solve the problem that the existing structured light 3D measurement causes measurement false points in black and shadow areas, which is difficult to effectively resolve, the present invention proposes a false point correction method for black and shadow areas in structured light 3D measurement and a structured light 3D measurement method.

[0010] The technical problem of the present invention is solved by the following technical solutions:

[0011] A method for correcting false points in black and shadow areas in structured light three-dimensional measurement includes the following steps: S1, obtaining all phase shift maps required for unwrapping phase solution of a measured object; S2, obtaining an absolute phase map of the measured object based on the phase shift maps; S3, taking a phase shift map of a frequency from the phase shift map solved in step S1, and obtaining a mask by a cyclic phase shift difference method, wherein the mask separates shadow areas and black areas of the measured object from normally illuminated areas; S4, multiplying the absolute phase map of the measured object obtained in step S2 by the mask obtained in step S3 to obtain an absolute phase map of the measured object with the black and shadow areas corrected.

[0012] In some embodiments, in step S2 , the absolute phase is calculated using a multi-frequency N-step phase shift method or a multi-frequency heterodyne method.

[0013] In some embodiments, in step S3, obtaining a mask by cyclic phase shifting and differencing includes: first setting a mask with an initial value of 0, and then sequentially performing internal and external cyclic differentiation on phase gratings with different initial phases under the same frequency grating. The absolute value of each difference is compared with the mask pixel by pixel, and only pixels with higher illumination intensity are retained as the new mask. After cyclic difference comparison and binarization, a mask that completely removes black and shadow areas is obtained, i.e., the mask. Finally, the absolute phase without the influence of false points is obtained by multiplying the mask with the absolute phase solved by traditional structured light 3D measurement methods.

[0014] In some embodiments, in step S3, the lowest frequency among the multiple frequencies of the multi-frequency phase-shifted projection is selected for cyclic difference operation; alternatively, a cyclic difference operation is performed using stripes of multiple frequencies, and the results obtained from the multiple frequencies are compared pixel by pixel, and the area with the highest light intensity value is retained pixel by pixel.

[0015] In some embodiments,

[0016] In step S3, the initial value of the mask is set to 0. Then, by performing a pixel-by-pixel comparison and retaining the largest pixel, the difference between the different phase-shifted images is processed to obtain a mask that separates black and shadow areas from normal areas with high contrast. The detailed process is as follows: All N-step phase-shift images of a single frequency (not limited to a single frequency; other frequencies are also possible) are collected. A single step Ii is selected as the first minuend (starting with i = 1), and Ij is selected as the first subtrahend (starting with j = 1). A threshold T is set (T = 0.2, for example). Next, we enter the loop iteration process, which includes an outer loop and an inner loop. The outer loop determines whether the number of steps i of the subtrahend Ii is less than (N+1) / 2. At this time, i=1. If yes, we enter the inner loop. If not, we perform binarization based on the threshold T and mask M to obtain the final required mask M1. The inner loop determines whether the number of steps j of the subtrahend Ij is less than (N+1). If so, the first step is to compare the mask M with abs(Ii-Ij) pixel by pixel and retain the maximum value pixel by pixel to obtain the new mask M (abs represents the absolute value). The second step is to subtrahend Ij, and the number of steps j=j+1. Then we determine whether j is less than (N+1). The inner loop continues until j≥N+1, at which point we exit the inner loop and enter the outer loop again with i=i+1.

[0017] The above method is repeated until i ≥ (N+1) / 2. At this point, the outer loop is exited and the mask obtained by subtracting the loop is binarized according to the threshold T, finally obtaining the effective mask M1.

[0018] In some embodiments, the method further includes: performing system calibration using a cyclic phase shift difference method.

[0019] In some embodiments, the system calibration comprises the following steps: T1, obtaining all phase shift maps required for solving the absolute phase of the calibration plate; T2, solving the absolute phase map of the calibration plate according to the phase shift maps; T3, taking a phase shift map of a frequency from the phase shift maps solved in step T1, obtaining a mask by the cyclic phase shift difference method, the mask separates the shadow area and the black area of the calibration plate from the normally illuminated area; T4, multiplying the absolute phase map of the calibration plate obtained in step T2 and the mask obtained in step T3 to obtain the absolute phase map of the calibration plate with the black area and the shadow area removed; T5, completing the global phase correction of the calibration plate by fitting or interpolating the absolute phase map of the calibration plate.

[0020] In some embodiments, step S3 is replaced by the following step: S3', taking N-step phase shift maps of any frequency (not limited to a single frequency, multiple frequencies are also applicable) from the phase shift maps solved in step S1, comparing all N-step phase shift maps pixel by pixel, selecting the maximum value as the pixel value of the high-contrast mask map by comparing the pixel values at the same position, and then performing a binary operation on the high-contrast mask map to generate the final mask.

[0021] The application further provides a structured light three-dimensional measurement method, comprising: using the false point correction method for black and shadow areas in the structured light three-dimensional measurement to obtain a corrected absolute phase map of the measured object; and reconstructing the three-dimensional topography of the measured object based on the corrected absolute phase map.

[0022] The application further provides a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the method.

[0023] The beneficial effects of the application compared with the prior art include:

[0024] The application obtains a mask of the phase shift map of the measured object by the cyclic phase shift difference method, multiplies the mask and the absolute phase map of the measured object to obtain the absolute phase map of the measured object after correction of the black area and the shadow area, i.e., the absolute phase without the influence of false points. In the process of generating the mask, no additional equipment and projection images are required, which can minimize the influence of the change of the intensity of the stripe light on the shadow and the black area of the measured object, and can also increase the contrast difference between the normal area and the black and shadow areas, and has absolute robustness to the ambient light. The method of the application finally solves the problem of the "false points" caused by the shadow area and the black area in the two links of phase solving and system calibration. The application ensures that the calibration parameters are not disturbed by the false points, effectively improves the measurement accuracy, and eliminates the post-processing link of stray noise, effectively improving the measurement efficiency.

[0025] The present application obtains accurate and effective 3D point cloud without stray points (caused by noise) and "false points" without any post-processing and other additional experimental process operations. Compared with shooting additional images without stripe patterns, the present application directly separates the measured object and the shadow by using the raster image, saving the measurement time. The false point correction method solves the inherent challenges and difficulties of the flow calibration method such as the pixel-by-pixel phase three-dimensional height mapping model, the inverse camera model and the ray model. The false point correction method can also be used for data set preparation to improve the performance of deep learning.

[0026] In a preferred example, the present application sets a mask with an initial value of 0 first, and then performs inner-outer loop sequential difference on phase gratings with different initial phases under the same frequency grating. The absolute value of each difference is compared with the mask pixel by pixel, and only the pixels with higher light intensity are retained as the new mask. After the cycle difference comparison and binarization processing, a mask that can completely remove the black and shadow areas, i.e. the mask, is obtained. Finally, by multiplying the mask with the absolute phase solved by the traditional structured light three-dimensional measurement method, the absolute phase without the influence of false points is obtained. In the process of generating the mask, no additional equipment and projection images are needed. Other beneficial effects in the embodiments of the present application will be further described below. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a flow chart of the false point correction method of the black and shadow areas in the structured light three-dimensional measurement in the embodiments of the present application.

[0028] Figure 2 is a flow chart of the cycle phase shift difference method in the embodiments of the present application.

[0029] Figure 3 is a flow chart of the application of the cycle phase shift difference method in the embodiments of the present application in the absolute phase solution of the measured object.

[0030] Figure 4 is a flow chart of the application of the CPSD in a calibration posture position in the system calibration in the embodiments of the present application.

[0031] Figure 5a is a single period grating schematic diagram of the measured object in embodiment 1.

[0032] Figure 5b is an absolute phase diagram of the measured object under the existing traditional method in embodiment 1.

[0033] Figure 5c is a mask of the measured object removing the shadow area in embodiment 1.

[0034] Figure 5d is an absolute phase diagram of the measured object in embodiment 1 after Figure 5b and Figure 5cThe shadow area obtained by multiplying the absolute phase map of the measured object and the mask is shown in Fig. 6.

[0035] Figure 6a Fig. 4 is a schematic diagram of the plane of the calibration plate at a certain calibration posture in Example 2.

[0036] Figure 6b Fig. 5 is an absolute phase map of the calibration plate. Figure 6a

[0037] Figure 6c Fig. 6 is a schematic diagram of the mask removing the black area in Example 2.

[0038] Figure 6d Fig. 7 is an absolute phase map of the measured object obtained by multiplying the absolute phase map of the measured object and the mask. Figure 6b Figure 6c Fig. 8 is a schematic diagram of the absolute phase value of the calibration plane obtained by fitting or interpolation.

[0039] Figure 6e Figure 6d Fig. 9 is a schematic diagram of the chessboard calibration plate in the experimental example of the present application.

[0040] Figure 7a Fig. 10 is a schematic diagram of the circular calibration plate in the experimental example of the present application.

[0041] Figure 7b DETAILED DESCRIPTION The present application will be further described below with reference to the accompanying drawings and in conjunction with the preferred embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0042] It should be noted that the left, right, up, down, top, bottom and other orientation terms in the embodiments are only relative concepts or are referenced to the normal use state of the product, and should not be considered as limiting.

[0043] FPP has become the mainstream method of structured light three-dimensional reconstruction due to its non-contact, high precision, fast speed and other advantages, and researchers have been committed to improving its precision and measurement efficiency. However, the reconstruction process will be affected by shadows and black, affecting the reconstruction accuracy and measurement efficiency. Therefore, the present application provides a cyclic phase shift difference method to segment the black and shadow areas of the measured object. On the basis of the cyclic phase shift difference method, further research is made on the error points generated by the black area of the calibration plate in the calibration process, which effectively eliminates these defects.

[0044] FPP has become the mainstream method of structured light three-dimensional reconstruction due to its non-contact, high precision, fast speed and other advantages, and researchers have been committed to improving its precision and measurement efficiency. However, the reconstruction process will be affected by shadows and black, affecting the reconstruction accuracy and measurement efficiency. Therefore, the present application provides a cyclic phase shift difference method to segment the black and shadow areas of the measured object. On the basis of the cyclic phase shift difference method, further research is made on the error points generated by the black area of the calibration plate in the calibration process, which effectively eliminates these defects.

[0045] ​​​To address the problems caused by "black" and shadowed areas at their source, embodiments of the present invention propose a method for correcting false points in black and shadowed areas in structured light 3D measurement and a structured light 3D measurement method. This method also employs a cyclic phase shift difference (CPSD) method based on a phase-shift grating to remove the influence of black and shadowed areas on structured light 3D measurement. The CPSD method, based on the principle that pixel values ​​in black and shadowed areas theoretically do not change with the intensity of the phase-shifted light, combines a double-layer cyclic method with an n-step phase-shift grating. A mask with an initial value of 0 is first set. Phase gratings with different initial phases are then sequentially differentiated using the same frequency grating. The absolute value of each difference is compared pixel by pixel with the mask, retaining only pixels with higher illumination intensity as the new mask. After cyclic difference comparison and binarization, a mask is obtained that completely removes black and shadowed areas. Finally, by multiplying the mask with the absolute phase solved by the traditional structured light 3D measurement method, the absolute phase without the influence of false points is obtained. In the process of generating the mask, no additional equipment and projection images are required. The cyclic phase shift difference (CPSD) method is simple and has a high degree of generalization. It can minimize the impact of stripes on the shadows and black areas of the measured object. It can also increase the contrast difference between normal areas and black and shadow areas, and is absolutely robust to ambient light. In summary, the CPSD method ultimately solves the "false point" problem caused by shadow areas and black areas in structured light 3D measurement and 3D reconstruction in both phase solution and system calibration.

[0046] Among existing absolute phase determination methods, the mainstream approach is to use a multi-frequency, N-step phase shift method. This involves first determining the wrapped phase using N-step phase shifts, then unwrapping the wrapped phase using a multi-frequency unwrapping method to obtain the absolute phase. The cyclic phase shift difference (CPSD) method proposed in this embodiment of the present invention is a modification of this existing absolute phase determination method.

[0047] The present invention adopts the multi-frequency N-step phase shift method, which is currently the most commonly used method in the structured light 3D measurement process, to solve the absolute phase. The method for correcting the false points in the black and shadow areas in the structured light 3D measurement proposed in the embodiment of the present invention uses the absolute phase solution method including but not limited to the multi-frequency N-step phase shift method. All existing absolute phase solution methods can be used. The method for correcting the false points in the black and shadow areas in the structured light 3D measurement proposed in the embodiment of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0048] S1. Obtain all phase shift diagrams required for unwrapping phase solution of the object under test based on the standard N-step phase shift grating and wrapped phase in the multi-frequency N-step phase shift method;

[0049] The standard N-step phase shift grating formula is described as follows:

[0050]

[0051] In the embodiment of the present invention, the phase shift diagram is an N-step phase shift grating diagram. n (x, y) represents the nth (n∈[1, N]) phase shift grating image captured by the camera, A(x, y) represents the background light intensity, and B(x, y) represents the light intensity modulation determined by the reflectivity of the object being measured. represents the phase determined by the surface height of the object being measured, and N represents the total number of phase shift steps.

[0052] The wrapping phase is obtained by least square fitting and inverse tangent function from the N-step phase shift. The expression for wrapping phase calculation is as follows:

[0053]

[0054] where φ(x,y) represents the wrapped phase obtained by least squares fitting and solving the inverse tangent function for N-step phase shifts.

[0055] S2. Obtain the wrapped phase of the object under test based on the N-step phase shift diagram. The characteristics of the inverse tangent phase indicate that the wrapped phase limits the phase value to [-π, π]. The next step is to unwrap the wrapped phase and calculate its absolute phase value (i.e., the absolute phase diagram). The expression for solving the absolute phase value is as follows:

[0056]

[0057] The absolute phase value includes high-frequency absolute phase and low-frequency absolute phase, Φ h (x,y),Φ l (x,y) represent the high-frequency absolute phase and low-frequency absolute phase, respectively, h (x,y),φ l (x, y) represent the high-frequency wrapping phase and the low-frequency wrapping phase, respectively, k h , k l They represent the high-frequency unwrapping level and the low-frequency unwrapping level respectively.

[0058] Typically, k l is 0, φ l (x,y) is a single-cycle wrapped phase, so at this time only the high-frequency unwrapping level k is required h , high frequency unpacking level k h The solution formula is as follows:

[0059]

[0060] Among them, λl , λ h respectively represent the pixel period of high frequency grating and low frequency grating, and Round represents the rounding operation.

[0061] After the above process, the absolute phase Φ of the measured object can be obtained h (x,y), at this time the absolute phase is not corrected by the shadow area and the black area, and the phase false points caused by these areas are difficult to distinguish with naked eyes.

[0062] S3, a phase shift map of a frequency is taken from the phase shift map solved in step S1, a mask is obtained by a cyclic phase difference method, and the mask separates the shadow area and the black area of the measured object from the normal illumination area;

[0063] The key of the cyclic phase difference method is to obtain a correction mask that clearly separates the shadow area / black area from the normal measured object by using the original phase shift grating map in the traditional method. The principle is that in theory, the light intensity of the black area and the shadow area does not change with the change of the phase shift grating intensity. In an ideal case, while the grating projection phase shift intensity changes, the light intensity of the black area remains constant at 0, and the light intensity of the shadow area remains constant at the background light intensity. Therefore, through cyclic difference, the light intensity contrast between the normal area and the black area and the shadow area can be maximized, thereby obtaining a better binary mask map. Finally, the absolute phase obtained by the traditional method is corrected. The steps of the cyclic phase difference method are as follows:

[0064] 1. Set the initial value of the mask to 0.

[0065] 1. In the loop, the inside and outside loops are subtracted between different phase shift images at the same frequency, and the absolute value of each difference is compared with the mask pixel by pixel, and the larger value is retained as the new mask.

[0066] 2. After the cyclic difference, the mask in the middle process is iterated to obtain a high-contrast mask image with high contrast between the normal area and the black and shadow areas. Through a binary operation, the image is threshold segmented to obtain the final mask.

[0067] 3. Under the processing of the mask on the traditional absolute phase, the absolute phase map of only the non-black and non-shadow area is obtained.

[0068] The advantage of this method is that the difference can not only eliminate the influence of the phase shift projection grating, but also solve the influence of the environmental light intensity, and still has robustness in the case of environmental light intensity.

[0069] In this embodiment of the present invention, the lowest frequency is selected. Here, the lowest frequency refers to the lowest frequency among the multiple frequencies in the multi-frequency phase-shifted projection, typically a single-period fringe. (With a single frequency, there is no limit on the lowest frequency; other frequencies can achieve similar results.) Furthermore, this method can also be extended to use fringe patterns at multiple frequencies. The results obtained at these frequencies are compared pixel by pixel, and the areas with the highest light intensity values ​​are retained pixel by pixel.

[0070] The specific operation of using cyclic phase shifting to perform a difference method to obtain an effective phase mask in the embodiment of the present invention is as follows: Figure 2 As shown,

[0071] First, the initial value of the mask is set to 0, and then the difference between the different phase-shifted images is processed by cyclically comparing each pixel and retaining the maximum pixel, and finally a mask is obtained that separates the black area and shadow area from the normal area with high contrast.

[0072] The detailed process is as follows: All N-step phase shift maps for a single frequency (not limited to a single frequency; other frequencies are also possible) are collected. A random step Ii is selected as the first minuend (starting with i=1, for example) and Ij as the first subtrahend (starting with j=1, for example). A threshold T is set (T=0.2, for example). Next, an iterative loop is entered, consisting of an outer loop and an inner loop. The outer loop determines whether the step number i of the subtrahend Ii is less than (N+1) / 2. If i=1, the inner loop enters. If not, binarization is performed based on the threshold T and mask M, resulting in the final mask M1. The inner loop determines whether the step number j of the subtrahend Ij is less than (N+1). If so, the first step mask M is compared pixel by pixel with abs(Ii-Ij), retaining the maximum pixel value to obtain the new mask M (abs represents the absolute value). The second step is for the subtrahend Ij, with step number j=j+1, and a check is then made to determine whether j is less than (N+1). If the result is "yes", the inner loop is executed until j≥N+1, at which time the inner loop is jumped out, and i=i+1 and then the outer loop is entered.

[0073] The above method is repeated until i ≥ (N+1) / 2. At this point, the outer loop is exited and the mask obtained by subtracting the loop is binarized according to the threshold T, finally obtaining the effective mask M1.

[0074] The cyclic phase shift subtraction method uses the lowest frequency among the multiple frequencies of the multi-frequency phase-shifted projection for the cyclic subtraction operation. Alternatively, the cyclic subtraction operation is performed using fringes with multiple frequencies, and the results obtained from the multiple frequencies are compared pixel by pixel, retaining the area with the highest light intensity. This embodiment uses the subtraction of phase-shifted images Ii and Ij as an example.

[0075] The input data is an N-order phase-shifted image I (i=1…N) with a frequency of 1. Set the initial value of the mask to 0. Select a reference plane Ii As the minuend of the difference operation, where i∈1...(N+1) / 2. Select I j As the subtrahend involved in the difference operation, where j∈1...N. The difference between the phase-shifted images i and j is calculated as follows:

[0076]

[0077]

[0078] Among them, I i (x,y) is the minuend, I j (x,y) is the subtrahend, in I i and I j Do a difference operation between them, I i and I j The difference operation expression between them is as follows:

[0079]

[0080] Where Diff i,j Is the difference between any two phase-shifted grating images (i.e., phase-shifted images). From Formula 7, we can see that since the phase-shifted grating images Ii and Ij are subtracted, the background A(x, y) is eliminated, so the difference image is absolutely robust to ambient light. j An absolute value operation is performed to ensure that the value of the mask is ≥ 0. The purpose of Formula 7 is to filter out black and shadow areas because the grayscale values ​​of these areas do not change with the change of the phase-shifted grating image.

[0081] Next, the cyclic difference processing operation begins, which is detailed as follows: first, the initial value of the mask is set to 0, and then the difference between the different phase-shifted images is processed through cyclic iteration. The cyclic iteration processing includes inner and outer loop differences.

[0082] Outer loop: variable i iterates from 1 to i_max(N).

[0083] Inner loop: variable j iterates from 1 to j_max((N+1) / 2).

[0084] In each inner loop, the element M_ij in the i-th row and j-th column of matrix M is updated to the maximum value between M_ij and abs(Diff_ij). At the end of the algorithm, each element M_ij in matrix M will contain the maximum absolute value of all elements in the original Diff matrix at that position so far. abs stands for absolute value, and max stands for maximum value.

[0085] That is, an intermediate image with high contrast between the normal area, the black area, and the shadow area is obtained. This image is the high-contrast mask M(ij). The above loop iterative processing is as follows:

[0086]

[0087] The last step is to generate a mask by performing a binarization threshold segmentation on the high-contrast middle mask. The formula is as follows:

[0088] M1=Binarize(M,T)(9)

[0089] Here, "binarize" represents a binarization operation on the image, i.e., pixel values ​​greater than a threshold T are set to 1, and pixel values ​​less than the threshold T are set to 0. This creates a binary mask that eliminates the effects of black and shadow areas on the absolute phase. In this embodiment, the threshold is typically between 0.2 and 0.3.

[0090] 3. This mask's processing of the traditional absolute phase yields an absolute phase map consisting solely of non-black and non-shadowed areas. The cyclic phase shift subtraction method proposed in this embodiment of the present invention can be used simultaneously for both absolute phase calculation and calibration of the measured object in structured light 3D measurement and reconstruction. The principles are the same: both utilize a mask to correct existing absolute phase calculation methods.

[0091] The application of the cyclic phase shift difference method in obtaining the absolute phase of the object under test in the embodiment of the present invention is as follows: The cyclic phase shift difference method is applied to the absolute phase solution link of the object under test. Figure 1 and Figure 3 As shown, the following steps are included:

[0092] S1. Obtain all phase shift diagrams required for unwrapping phase solution of the object under test.

[0093] S2. Based on the phase shift diagram, directly solve for absolute phase using multi-frequency N-step phase shifting using traditional methods to obtain the absolute phase diagram of the object under test. Methods for solving absolute phase include, but are not limited to, multi-frequency N-step phase shifting, multi-frequency heterodyning, and all other methods that require a phase shift diagram to solve for absolute phase are equally applicable.

[0094] S3. Based on the phase shift map obtained in step S1, an N-step phase shift map of a frequency is extracted therefrom, and a mask is obtained using a cyclic difference method. The mask separates the shadow area and the black area of ​​the object to be measured from the normally illuminated area.

[0095] S4. Multiply the absolute phase map of the object under test obtained by the conventional method in step S2 and the mask obtained in step S3 to obtain an absolute phase map of the object under test with the black area and the shadow area effectively corrected.

[0096] The application of the cyclic phase shift difference method in the system calibration link in the embodiment of the present invention is as follows:

[0097] The purpose of FPP calibration is to align the phase with the three-dimensional coordinate points of the object being measured. Therefore, in the calibration process, accurate acquisition of the absolute phase is also crucial, which directly affects the corresponding parameters of the pixel-by-pixel phase-to-three-dimensional point cloud mapping. In the cyclic phase shift difference method proposed in the embodiment of the present invention, the equiphase plane model is adopted because it has flexibility, high precision and real-time strategy. It should be noted that the applicable method is not limited to the equiphase plane model. All methods that require phase solution and phase-height mapping of the calibration attitude plane during the system calibration process are applicable.

[0098] For each pixel, a cubic polynomial fitting relationship is set between the phase Φ and the camera coordinate system (Xc, Yc, Zc). The fitting model applicable to the embodiment of the present invention is not limited to cubic fitting, and all fitting methods such as multi-time fitting, reciprocal fitting, and control equation fitting are applicable.

[0099]

[0100] In formula 10, a0, a1, a2, b0, b1, b2, c0, c1, c2 are system calibration parameters. (X c ,Y c ,Z c ) are the 3D coordinates of the object under test in the camera coordinate system. Equation 10 shows that accurate absolute phase measurement is crucial for system calibration. Since current mainstream calibration targets primarily feature black and white contrast patterns, such as black circular arrays and black checkerboard arrays, performing black area correction on the absolute phase of the calibration plane directly impacts the accuracy of the system calibration parameters, thereby determining the 3D reconstruction quality.

[0101] Compared with the application of cyclic phase shift difference method in the absolute phase solution of the measured object, the application of cyclic phase shift difference method in system calibration requires not only the removal of black areas at the absolute phase level, but also the phase fitting of the phase plane with the black calibration area removed to complete the phase plane of the calibration plate, thereby achieving a global correction of the absolute phase of each calibration position plane. In system calibration, the calibration plate often has multiple calibration positions. This example only uses one calibration position to introduce the application of cyclic phase shift difference method (CPSD) in system calibration method. The specific process of the application of cyclic phase shift difference method in system calibration is as follows: Figure 4 As shown, the following steps are included:

[0102] T1. Obtain all phase shift diagrams required for the absolute phase solution of the calibration plate.

[0103] T2, the absolute phase of the calibration plate is solved by a traditional method directly using a multi-frequency N-step phase shift method according to the phase shift diagram, and the absolute phase diagram of the calibration plate is obtained. The method for solving the absolute phase is not limited to the multi-frequency N-step phase shift method, such as multi-frequency heterodyne and all methods that need to solve the absolute phase from the phase shift diagram.

[0104] T3, based on the phase shift diagram obtained in step T1, the N-step phase shift diagram of one frequency is taken out, and the mask is obtained by using the cyclic difference method, which separates the shadow area and the black area of the calibration plate from the normal light area.

[0105] T4, the absolute phase diagram of the calibration plate solved by the traditional method in step T2 is multiplied by the mask solved in step T3, and the absolute phase diagram phi of the calibration plate with the black area and the shadow area effectively removed is obtained.

[0106] T5, the absolute phase diagram phi of the calibration plate is completed based on plane fitting or interpolation, so that the complete global phase correction of the calibration plate is completed. The method for completing the black area is not limited to fitting, but can also use interpolation and other completion methods.

[0107] The above completes the global correction of the phase calculation of the calibration plate, and the corrected absolute phase is used in formula 10 to solve the phase-3D mapping coefficient.

[0108] The embodiment of the application also proposes a structured light three-dimensional measurement method, comprising: using the false point correction method for black and shadow areas in the structured light three-dimensional measurement described above to obtain the corrected absolute phase diagram of the measured object; and based on the corrected absolute phase diagram, reconstructing the three-dimensional topography of the measured object.

[0109] The embodiment of the application also proposes a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the method described above.

[0110] The false point correction method for black and shadow areas in the structured light three-dimensional measurement proposed by the embodiment of the application has the following advantages:

[0111] 1. The embodiment of the application proposes to use the cyclic phase shift difference method to solve the "false" phase and the corresponding "false" 3D point cloud caused by the shadow and black areas. Compared with shooting an additional image without a stripe pattern, it directly separates the measured object and the shadow by using the grating image, saves the measurement time, and improves the light intensity contrast of the object and the shadow, the black area. Without any post-processing, high-precision 3D point cloud without stray points is obtained.

[0112] 2. This embodiment of the present invention employs a cyclic phase shift (CPSD) method to mitigate calibration errors caused by black calibration plates. By applying CPSD to system calibration, these black areas can be identified and filtered out, ensuring that calibration parameters are not disturbed and resulting in a precisely calibrated 3D point cloud. This method addresses the inherent challenges and difficulties of flow calibration methods such as pixel-by-pixel phase 3D height mapping models, inverse camera models, and ray models.

[0113] 3. This method can be used for both the phase resolution and calibration stages of the object being measured.

[0114] Example 1:

[0115] This embodiment applies the cyclic phase shift difference method to the absolute phase acquisition link of the measured object. The specific operations are as follows:

[0116] S1. Obtain all phase shift diagrams required for unwrapping phase solution of the object under test; Figure 5a shown.

[0117] S2. Obtain the absolute phase diagram of the object under test according to the phase shift diagram; specifically, the absolute phase diagram of the object under test obtained by the four-frequency twelve-step phase shift is as follows: Figure 5b shown.

[0118] S3. Take a frequency phase shift diagram from the phase shift diagram solved in step S1, and obtain a mask by cyclic phase shift difference method. The mask obtained by cyclic phase shift difference method to remove the shadow area is as follows: Figure 5c shown.

[0119] S4, multiplying the absolute phase map of the object under test obtained in step S2 and the mask obtained in step S3 to obtain the corrected absolute phase map of the object under test obtained in the black area and the shadow area. The corrected absolute phase map of the object under test is as follows: Figure 5d shown.

[0120] It can be concluded that the problem of "false" points in absolute phase caused by shadows in the field of view caused by protrusions on the surface of the measured object has been fully solved.

[0121] Example 2:

[0122] This embodiment applies the cyclic phase shift difference method to the system calibration link. The specific operations are as follows:

[0123] T1. Obtain all phase shift diagrams required for solving the absolute wrapped phase of the calibration plate; specifically, the calibration plate plane at any calibration posture position is as follows: Figure 6a shown.

[0124] T2, according to the phase shift diagram to obtain the absolute phase diagram of the calibration plate; step T1 calibration plate plane after four-frequency twelve-step phase shift to obtain the absolute phase diagram as shown Figure 6b As shown in the figure, the absolute phase values ​​in the black area are all "false" points.

[0125] T3. Take a frequency phase shift diagram from the phase shift diagram solved in step T1, and obtain a mask for removing the black area by performing the cyclic phase shift difference method. The mask separates the shadow area and the black area of ​​the calibration plate from the normal light area. The mask for removing the black area obtained in this step is as follows: Figure 6c shown.

[0126] T4, multiplying the absolute phase image of the calibration plate obtained in step T2 and the mask obtained in step T3 to obtain the absolute phase image of the calibration plate with the black area and the shadow area removed. The absolute phase image after removal is as follows: Figure 6d As shown in the figure, it can be seen that the problem of “false” points in absolute phase caused by the low reflectivity of the black area has been solved.

[0127] T5. Use plane fitting or interpolation to complete the absolute phase map of the calibration plate obtained in step T4, completing the global phase correction of the calibration plate and obtaining the complete absolute phase value of the calibration plane. At this point, the false point problem caused by the black area of ​​the plane absolute phase value has been compensated.

[0128] Experimental example:

[0129] This experiment uses two classic calibration plates to evaluate the 3D reconstruction accuracy. The specific parameters of the calibration plates are set as follows:

[0130] (a) Checkerboard calibration board Figure 7a As shown, the length is 150mm, the width is 130mm, the unit cell side length is 8mm, the accuracy is ±10um, and the small checkerboard grid is distributed in 11*8.

[0131] (b) Circular calibration plate such as Figure 7b As shown, the length is 150mm, the width is 130mm, the center distance is 8mm, the center diameter is 4.5mm, the accuracy is ±10um, and the small chessboard distribution is 11*8.

[0132] This experimental example evaluates the accuracy improvement effectiveness of the cyclic phase shift difference method (CPSD) using two classic checkerboard patterns. It should be noted that in actual experiments, any checkerboard pattern is applicable, not limited to these two.

[0133] Experiments show that for a checkerboard calibration plate (50% black) and a large circle calibration plate (27.4% black), the in-plane reconstruction error is reduced by 51.7% and 45.2%, respectively, and the standard deviation is reduced to 56.8% and 52.4%, respectively. This cyclic phase shift difference method (CPSD) can also be used to prepare precise absolute phase datasets, improving deep learning performance.

[0134] In summary, the present invention demonstrates the effectiveness of cyclic phase shift difference (CPSD) on two checkerboard patterns (a checkerboard calibration pattern and a circular calibration pattern) based on a pixel-by-pixel phase height mapping model. Experimental results demonstrate that the proposed method for correcting false points in dark and shadowed areas during structured light 3D measurement can produce a clear 3D point cloud free of noise and false points, without requiring any post-processing or other additional operations.

[0135] In some embodiments, step S3 is replaced by the following steps:

[0136] S3', select all phase shift maps of any frequency from the phase shift map solved in step S1, compare all selected phase shift maps pixel by pixel, and select the maximum value as the pixel intensity value of the high contrast map M(x,y) by comparing the pixel intensity values ​​at the same position in all grating maps. Then, perform a binarization operation on the high contrast map to generate mask M1. The formulas for M(x,y) and M1 are as follows:

[0137] M(x,y)=max(I1(x,y),I2(x,y),...I N (x,y)) (11)

[0138] M1=Binarize(M,T) (12)

[0139] In Formula 11, max() represents the maximum value of all the images in the brackets. Binarize() in Formula 12 represents the binarization operation on the image.

[0140] The mask obtained by the method of step S3' has almost the same effect as the mask obtained by the cyclic phase shift difference method.

[0141] The false point correction method for black and shadow areas in structured light three-dimensional measurement proposed in the embodiment of the present invention has great application value in the precision detection industry. It can improve the 3D measurement accuracy of the object to be measured, and can also directly obtain a three-dimensional point cloud of the object to be measured without stray points without post-processing operations. No additional manual projection or acquisition operations are required, and only the images collected by the traditional phase shift method are used for cyclic phase shift difference processing. It can simultaneously solve the two difficult problems of shadow occlusion of the object to be measured and the influence of the black area of ​​the calibration plate on the calibration. This method can simultaneously act on the absolute phase solution accuracy and calibration process of the object to be measured. This method is not only applicable to the multi-frequency phase shift method, but also to various absolute phase solution algorithms such as the Gray code method and the triangulation method.

[0142] In addition to the field of 3D measurement, it can be applied to all fields that require a projector to perform absolute phase solution. The false point correction method for black and shadow areas in structured light three-dimensional measurement is a correction to the existing absolute phase acquisition method. The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art of the technical field to which the present invention belongs, without departing from the concept of the present invention, several equivalent substitutions or obvious variations can be made, and the performance or use are the same, which should be regarded as belonging to the scope of protection of the present invention.

Claims

1. A method for correcting false points in black and shadow areas in structured light 3D measurement, characterized in that: The steps include: S1. Obtain all phase shift diagrams required for unwrapping phase solution of the object under test; S2. Obtaining an absolute phase diagram of the object under test according to the phase shift diagram; S3. Take a frequency phase shift map from the phase shift map solved in step S1, and obtain a mask by a cyclic phase shift difference method, wherein the mask separates the shadow area and black area of ​​the object to be measured from the normally illuminated area; wherein, obtaining the mask by the cyclic phase shift difference method includes: first setting a mask with an initial value of 0, and then performing internal and external cyclic differences on phase gratings with different initial phases under the same frequency grating, and comparing the absolute value of each difference with the mask pixel by pixel, retaining only pixels with higher illumination intensity as new masks, and after cyclic difference comparison and binarization processing, obtaining a mask that can completely remove black areas and shadow areas, i.e., a mask; finally, by multiplying the mask with the absolute phase solved by the traditional structured light three-dimensional measurement method, obtaining an absolute phase without the influence of false points; wherein, the lowest frequency among the multiple frequencies of the multi-frequency phase shift projection is selected for cyclic difference operation; or, using stripes of multiple frequencies for cyclic difference operation, the mask results obtained by the multiple frequencies are compared pixel by pixel, and the area with the highest light intensity value is retained pixel by pixel; Alternatively, step S3 may be replaced by the following step S3': S3', taking an N-step phase shift map of any one frequency or multiple frequencies from the phase shift map solved in step S1, performing pixel-by-pixel comparison on all the N-step phase shift maps, selecting the maximum value as the pixel value of the high-contrast mask map by comparing the pixel values ​​at the same position, and then performing a binarization operation on the high-contrast mask map to generate a final mask; S4. Multiply the absolute phase image of the object under test obtained in step S2 and the mask obtained in step S3 or step S3' to obtain a corrected absolute phase image of the object under test in the black area and the shadow area.

2. The method for correcting false points in black and shadow areas in structured light 3D measurement according to claim 1, wherein: In step S2, the absolute phase is calculated using a multi-frequency N-step phase shift method or a multi-frequency heterodyne method.

3. The method for correcting false points in black and shadow areas in structured light 3D measurement according to claim 1, wherein: In step S3, the initial value of the mask is first set to 0, and then the difference between the different phase-shifted images is processed by cyclically comparing each pixel and retaining the maximum pixel, ultimately obtaining a mask that separates the black area and shadow area from the normal area with high contrast; The specific method includes: collecting all N-step phase shift images of a single frequency or multiple frequencies, randomly selecting a step Ii as the first minuend and Ij as the first subtrahend, setting a threshold T, and then entering a loop iteration process, which includes an outer loop and an inner loop. The outer loop determines whether the step number i of the subtrahend Ii is less than (N+1) / 2. At this time, i=1. If the result is "yes", the inner loop is entered. If not, binarization is performed based on the threshold T and the mask M to obtain the final required mask M1. The inner loop determines whether the step number j of the subtrahend Ij is less than (N+1). If the result is "yes", the mask M is first compared pixel by pixel with abs(Ii-Ij) and the maximum value is retained pixel by pixel to obtain a new mask M, where abs is the absolute value. Next, the step number j of the subtrahend Ij is j=j+1, and then whether j is less than (N+1) is determined. If the result is "yes", the inner loop is entered until j≥N+1, at which time the inner loop is jumped out, and the outer loop is entered again when i=i+1. Iterate the loop according to the above method until i ≥ (N+1) / 2, then jump out of the outer loop, perform binarization on the mask obtained by subtracting the loop according to the threshold T, and finally obtain the effective mask M1; The maximum number of iterations j in the inner loop is determined by the N-step phase shift N, j<N+1, while the maximum number of iterations i in the outer loop is <(N+1) / 2.

4. The method for correcting false points in black and shadow areas in structured light 3D measurement according to any one of claims 1 to 3, wherein: Also includes: The system is calibrated using the cyclic phase shift difference method.

5. The method for correcting false points in black and shadow areas in structured light 3D measurement according to claim 4, wherein: The system calibration comprises the following steps: T1. Taking one calibration point as an example, obtain all phase shift diagrams required for unwrapping phase solution of the calibration plate; T2. Obtaining an absolute phase diagram of the calibration plate according to the phase shift diagram; T3, taking a frequency phase shift map from the phase shift map solved in step T1, and obtaining a mask by performing a cyclic phase shift difference method, wherein the mask separates the shadow area and the black area of ​​the calibration plate from the normally illuminated area; T4, multiplying the absolute phase image of the calibration plate obtained in step T2 and the mask obtained in step T3 to obtain the absolute phase image of the calibration plate with the black area removed; T5. Completing the absolute phase map of the calibration plate by fitting or interpolation to complete the global phase correction of the calibration plate.

6. A structured light three-dimensional measurement method, characterized in that: include: Using the false point correction method for black and shadow areas in structured light three-dimensional measurement according to any one of claims 1 to 5, obtaining a corrected absolute phase map of the object under measurement; And the corrected absolute phase image of the calibration plate is obtained to reconstruct the three-dimensional morphology of the object being measured.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images