A color crosstalk error removal method for chromatic phase measurement profilometry

Abstract

The present application relates to a kind of color crosstalk error removal methods for color phase measurement profilometry.By the analysis of crosstalk model, it is found that in the color separation process of three channels, it will cause the influence to the background light intensity of stripe, modulation and phase shift, lead to not meet the condition of full period wrapped phase calculated by phase shift method.This method can efficiently handle the problem by normalization.For the change of three-channel phase shift caused by color crosstalk, the present application proposes a phase correction matrix, to ensure that the corrected phase shift meets the requirements of full period wrapped phase calculation, and finally, the influence of color crosstalk error on three-dimensional reconstruction result is eliminated.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional reconstruction, specifically a method for removing color crosstalk errors in color phase measurement profilometry. Background Technology

[0002] Color phase-shift profilometry encodes three monochromatic fringe patterns with phase-shift relationships into the red, green, and blue channels of a single color image. A projector then projects the composite color fringe onto the surface of the object being measured. Subsequently, a color camera acquires a distorted color fringe pattern modulated by the surface topography of the object. By separating the red, green, and blue channels of the acquired image, three distorted monochromatic fringe patterns are obtained and used for phase calculation, enabling single-frame 3D reconstruction. This technique combines the high precision of phase-shift methods with high speed, thus attracting widespread research.

[0003] However, both color projectors and color cameras suffer from color crosstalk, which directly affects the accuracy of phase calculation, leading to significant errors in the 3D reconstruction results. To address this, Huang et al. proposed a crosstalk coefficient compensation method. This method requires projecting nine monochrome fringe patterns to obtain the crosstalk matrix, and then calculating the inverse of the matrix to compare the acquired crosstalk fringes. Figure 3 Crosstalk correction for three-channel fringe projection profilometry (see reference: Color phase-shifting technique for three-dimensional shape measurement) is inefficient and inaccurate because it introduces errors in calculating the modulation ratios of the three channels and obtaining the inverse of the matrix, and it requires a large number of projected fringe patterns. Wang et al. proposed Hilbert transform to eliminate phase errors caused by crosstalk (see reference: Hilbert transform-based crosstalk compensation for color fringe projection profilometry), but this method ignores the influence of crosstalk on the phase shift of the three channels in its formula derivation, cannot handle complex fringe patterns with high crosstalk levels, and cannot completely eliminate phase errors in general crosstalked fringe patterns. In summary, commonly used color fringe crosstalk removal algorithms have two drawbacks: first, color crosstalk error elimination is incomplete; second, multiple images need to be projected and acquired, resulting in low efficiency. Developing an efficient and high-precision color crosstalk error removal method is crucial for the development of color fringe projection profilometry.

[0004] The following publicly available patent documents for similar technologies were found through a search:

[0005] A method for reducing the impact of multipath effects in phase measuring profilometry (CN104215201B) relates to how to reduce phase errors caused by multipath effects when performing 3D measurements using phase measuring profilometry. A conventional phase measuring profilometry (PMP) system can be used. The brightness modulation parameters are obtained using the conventional PMP algorithm, and the difference distribution is obtained after difference calculation. From the difference distribution, the parts severely affected by multipath effects are identified. Then, the phase values ​​of the severely affected parts are estimated based on the phase values ​​of other parts. Finally, the 3D data is calculated using the corrected phase values. This invention improves upon the conventional PMP algorithm, avoiding uncertainties and additional effects that may be caused by introducing external models. Furthermore, the method is direct and effective, comprehensively reducing phase errors in all directions and improving the accuracy of 3D measurements.

[0006] A phase-shifting profilometry method, system, device, and storage medium for three-dimensional measurement (CN110360952B) includes the following steps: generating a target image; acquiring the target image; obtaining a color fringe pattern from the target image; performing a correction operation on the color fringe pattern to obtain a corrected fringe pattern; obtaining phase information from the corrected fringe pattern; and obtaining three-dimensional topographic information from the phase information. This invention can solve the phase error problem caused by unstable light sources and changes in ambient light; it can meet the needs of various industrial fields and reduce the economic cost of three-dimensional inspection; the phase error correction method based on color-coded stripes in the phase-shifting profilometry system is simple and has high robustness, flexibility, and accuracy.

[0007] Upon comparison, the technical problem solved by this patent application differs significantly from that in the prior art. Therefore, the specific extraction and calculation methods used are clearly different, and the prior art does not affect the innovativeness of this patent application. Summary of the Invention

[0008] To eliminate errors caused by color crosstalk and reduce the number of projected and acquired images, thereby improving correction efficiency, this application proposes a novel method for removing color crosstalk errors in color phase measurement profilometry. Analysis of the crosstalk model reveals that the color separation process in the three channels affects the background light intensity, modulation, and phase shift of the stripes, causing the conditions for calculating the full-cycle wrap-around phase using the phase-shift method to be unsatisfactory. This method efficiently addresses this issue through normalization. To address the changes in the three-channel phase shift caused by color crosstalk, this application proposes a phase correction matrix to ensure that the corrected phase shift meets the requirements for full-cycle wrap-around phase calculation. Ultimately, the impact of color crosstalk errors on the 3D reconstruction results is eliminated.

[0009] The technical problem solved by this invention is achieved through the following technical solution:

[0010] A method for removing color crosstalk error in color phase measurement profilometry includes:

[0011] S1 establishes the phase correction matrix, and the specific steps are as follows:

[0012] S11 projects a grayscale fringe pattern and a three-step phase-shift color composite fringe pattern sequentially onto a white flat plate;

[0013] S12 collects grayscale stripe images and performs normalization processing;

[0014] S13 performs inverse cosine function calculation on the normalized grayscale fringe pattern to obtain the fundamental frequency component;

[0015] S14 uses a color camera to capture a color stripe image, then performs color channel separation to form a three-channel grayscale image.

[0016] S15 performs channel normalization on the three-channel grayscale image to form the actual three-channel normalized grayscale image.

[0017] S16 establishes a system of quadratic equations in one variable related to the phase offset caused by color crosstalk error. Then, the system of quadratic equations in one variable is solved to obtain the actual phase offset of the three channels.

[0018] Furthermore, using the three-channel normalized grayscale image and the actual phase shift, an ideal three-channel three-step phase-shifted grayscale image is calculated;

[0019] S17 establishes the phase correction matrix A, and uses ideal and actual three-channel three-step phase-shifted grayscale image data to solve for the element values ​​in the phase correction matrix;

[0020] S2 uses the phase correction matrix to remove color crosstalk errors during 3D reconstruction. The specific steps are as follows:

[0021] S21 projects a color composite stripe pattern onto the surface of the object being measured for actual 3D reconstruction;

[0022] S22 uses a color camera to capture color stripe patterns for actual 3D reconstruction, performs color channel separation, and forms a three-channel grayscale image for actual 3D reconstruction.

[0023] S23 normalizes the three-channel grayscale image used for actual 3D reconstruction;

[0024] S24 uses the phase correction matrix A, substitutes the normalized three-channel three-step phase-shift grayscale image used for actual 3D reconstruction, and calculates the ideal three-step phase-shift grayscale image used for actual 3D reconstruction.

[0025] S25 uses a three-step phase-shifting method for phase calculation, and finally performs three-dimensional reconstruction.

[0026] Furthermore, the specific steps of S11 are as follows:

[0027] A grayscale fringe pattern I and a color composite fringe pattern I are projected sequentially onto a white flat panel. m ,

[0028] I m =I r +I g +I b

[0029] Among them, I r I g I b The two images are colored stripes I. m Grayscale fringe patterns on the red, green, and blue channels; the two fringe patterns have the same frequency, the initial phase shift of the grayscale fringe pattern is 0, and the phase difference between the three channels of the color fringe pattern is 2π / 3.

[0030] Furthermore, step S13 involves calculating the inverse cosine function on the normalized grayscale fringe pattern to obtain the fundamental frequency component. The specific steps are as follows:

[0031] Normalized grayscale fringe image I' n The fundamental frequency component is obtained by performing inverse cosine function calculation. and in Let x represent the phase of each pixel in the image, and y represent the horizontal and vertical pixel coordinates of the image.

[0032] Furthermore, the specific steps of S16 are as follows:

[0033] In the normalized three-channel image I crn I cgn I cbn In the data collected, the red, green, and blue channels all exhibit phase shifts, as shown in the following formula.

[0034]

[0035] The phase offsets of the three channels are denoted as θ. r θ g θ b By expanding the fringe intensity formula for the three color channels, we obtain the value expressed in cosθ. r cosθ g cosθ b Solve the quadratic equation with variables in one variable, and then obtain the actual phase offset θ of the red, green, and blue channels. r θ g θ b

[0036] Furthermore, the red channel cosθ r The quadratic equation in one variable is given by the formula:

[0037]

[0038] The fundamental frequency component calculated in step S13 and Substituting these values ​​into the quadratic equation above, we can solve for the red channel cosθ separately. r , and thus obtain θ r .

[0039] Furthermore, the specific steps of S17 are as follows:

[0040] I in S11 r I g I b After normalization, place it on the left side of the equation. The right side of the equation, based on the color crosstalk error rule, is in I... crn I cgn I cbn Add a phase correction matrix A in front,

[0041]

[0042] Calculate the correction factor 'a' for the red channel separately. 12 and a 13 The correction factor a for the green channel 21 and a 23 And the correction factor a for the blue channel. 31 and a 32 Thus, a phase correction matrix is ​​established.

[0043] Furthermore, calculate the correction factor 'a' for the red channel. 12 and a 13 Establish the following equation:

[0044] I r =I crn +a 12 I cgn +a 13 I cbn

[0045] By using the correspondence between the left and right sides of the equation above, we can obtain a. 12 and a 13 .

[0046] A color crosstalk error removal device for color phase measurement profilometry, characterized in that it includes a phase correction matrix module and a color crosstalk error removal module.

[0047] The phase correction matrix module is used to create the phase correction matrix.

[0048] The color crosstalk error removal module is used to remove color crosstalk errors when using the phase correction matrix for 3D reconstruction.

[0049] An electronic device, characterized in that it includes a processor and a memory, the processors being communicatively connected to each other, the memory storing computer instructions, and the processors executing the computer instructions to perform any of the methods described above.

[0050] A computer-readable storage medium, characterized in that the computer-readable storage medium stores instructions that, when executed, perform the method described in any one of the preceding descriptions.

[0051] The advantages and positive effects of this invention are:

[0052] The method of this invention is applicable to color composite fringe patterns based on phase-shift encoding. It only requires projecting one grayscale fringe pattern and one color fringe pattern to eliminate color crosstalk errors in the color stripes. Specifically, it has the following advantages:

[0053] (1) The method of the present invention only requires pre-projecting two stripe patterns to obtain the crosstalk correction matrix.

[0054] (2) The method of the present invention is simple to calculate the phase correction matrix, does not require iterative calculation, and is highly efficient.

[0055] (3) The method of the present invention does not limit the frequency of the color composite stripe pattern used during reconstruction. Even if the frequency of the subsequent projected color composite stripe pattern is different from the frequency when the phase correction matrix is ​​obtained, the obtained correction matrix can still be used to eliminate the color crosstalk error of the color stripe. Attached Figure Description

[0056] Figure 1 This is a flowchart illustrating the establishment of the phase correction matrix in an embodiment of the present invention.

[0057] Figure 2 This is a flowchart illustrating the removal of color crosstalk error during 3D reconstruction in an embodiment of the present invention.

[0058] Figure 3 The figure shows the simulation results of the system phase error elimination in the embodiment of the present invention.

[0059] Figure 4 This is a phase error diagram between the measured unfolded phase diagram of the sphere in this embodiment of the invention and the true phase diagram (the unfolded phase diagram obtained by 20 phase shifts).

[0060] Figure 5This is a diagram illustrating the effect of three-dimensional reconstruction of a measured sphere in an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer and easier to understand, the present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the scope of this application.

[0062] The overall process of a color crosstalk error removal method for color phase measurement profilometry according to an embodiment of the present invention consists of two parts:

[0063] First, the process of establishing the phase correction matrix is ​​described. Second, the process of using the phase correction matrix to remove color crosstalk errors during 3D reconstruction is described.

[0064] Figure 1 The diagram shows a flowchart for establishing the phase correction matrix. Figure 1 The diagram illustrates the process of establishing the phase correction matrix in this scheme. The specific implementation steps are as follows:

[0065] (1) Projecting the characteristic fringe pattern for correction: Project a grayscale fringe pattern I and a color composite fringe pattern I onto the white flat plate in sequence. m I m =I r +I g +I b I r I g I b The two images are colored stripes I. m Grayscale fringe patterns on the red, green, and blue channels; the two fringe patterns have the same frequency, the initial phase shift of the grayscale fringe pattern is 0, and the phase difference between the three channels of the color fringe pattern is 2π / 3.

[0066] (2) Acquire and normalize the grayscale fringe image: Acquire the projected grayscale fringe image I using a color camera. The acquired image is denoted as I'. Perform channel normalization on the acquired image to obtain the normalized grayscale fringe image I'. n .

[0067] (3) Calculate the fundamental frequency component: For the normalized grayscale fringe image I' n The fundamental frequency component is obtained by performing inverse cosine function calculation. and in Let x represent the phase of each pixel in the image, and y represent the horizontal and vertical pixel coordinates of the image.

[0068] (4) Acquire color stripe pattern and separate channels: Acquire projected color stripe pattern I using a color camera.m The acquired image is denoted as I. c I c =I cr +I cg +I cb I cr I cg I cb The two images are colored stripes I. c Grayscale stripes on the red, green, and blue channels.

[0069] (5) Normalization of the separated three-channel image: Normalization of the separated three-channel I cr I cg I cb Perform channel normalization separately, denoted as I. crn I cgn I cbn .

[0070] (6) Solve for the phase offset of the three channels: in I crn I cgn I cbn In the process, due to color crosstalk errors, the acquired red, green, and blue channels all exhibit phase shifts, as shown in the following formula.

[0071]

[0072] The phase offsets of the three channels are denoted as θ. r θ g θ b Due to the influence of color crosstalk, θ r θ g θ b The phase differences from the standard are -2π / 3, 0, and 2π / 3, respectively. By expanding the formula for the fringe intensity of the three color channels, we obtain the expression with cosθ... r cosθ g cosθ b A quadratic equation with variable cosθ as the variable. r Taking a quadratic equation in one variable as an example, the formula is:

[0073]

[0074] The fundamental frequency component calculated in step (3) and Substituting these values ​​into the quadratic equation above, we can solve for the red channel cosθ separately. r This allows us to obtain the actual phase offset θ of the red channel. r Similarly, the actual phase offset θ between the green and blue channels can be calculated.g and θ b .

[0075] (7) Establish the correction matrix A: In step (1), I r I g I b After normalization, place it on the left side of the equation. The right side of the equation, based on the color crosstalk error rule, is in I... crn I cgn I cbn Add a correction matrix A in front,

[0076]

[0077] To calculate the correction factor 'a' for the red channel 12 and a 13 For example, using the matrix relationships described above, we can establish the following equation:

[0078] I r =I crn +a 12 I cgn +a 13 I cbn

[0079] By using the correspondence between the left and right sides of the equation above, we can obtain a. 12 and a 13 Similarly, the correction factor 'a' for the green channel can be obtained. 21 and a 23 And the correction factor a for the blue channel 31 and a 32 Thus, the correction matrix A is established.

[0080] Figure 2 The diagram illustrates the process of removing color crosstalk errors when using the acquired phase correction matrix for 3D reconstruction. Combined with... Figure 2 The diagram illustrates the process of using the phase correction matrix to remove color crosstalk errors during 3D reconstruction, and the specific implementation steps are as follows:

[0081] (1) Project a three-step phase-shift color composite stripe pattern for actual three-dimensional reconstruction.

[0082] (2) Acquire color stripe images for actual 3D reconstruction and separate the channels.

[0083] (3) Normalize the three-channel separated image to form a normalized three-channel three-step phase-shift grayscale image for actual three-dimensional reconstruction.

[0084] (4) Calculate the ideal three-step phase-shift grayscale image: Using the phase correction matrix A, substitute it to form the normalized actual three-channel actual three-step phase-shift grayscale image, and calculate the ideal three-step phase-shift grayscale image for actual three-dimensional reconstruction.

[0085] (5) Phase calculation and three-dimensional reconstruction.

[0086] The process of removing color crosstalk error during 3D reconstruction is a conventional calculation process in this field and will not be described in detail in this paper. The innovation of this method lies in applying the phase correction matrix to the process of removing color crosstalk error during 3D reconstruction.

[0087] Figure 3 The simulation results for eliminating system phase error are shown in the figure. A simulated, undistorted ideal color fringe pattern is used to obtain the true phase image. Color crosstalk error is added to the simulated ideal color fringe pattern to form a crosstalk color fringe pattern. The phase image is directly obtained by processing the crosstalk color fringe pattern, and the phase error ΔΦ between it and the true phase image is calculated. Finally, after normalizing only three channels of the crosstalk color fringe pattern, the phase image is obtained, and the phase error ΔΦ between it and the true phase image is calculated. N After processing the crosstalk color stripe pattern using the method of this invention, a phase diagram is obtained, and the phase error ΔΦ between the phase diagram and the true phase diagram is calculated. A As can be seen from the figure, the phase error ΔΦ of the method of the present invention A Much smaller than ΔΦ and ΔΦ N Theoretically, this can eliminate phase errors caused by color crosstalk.

[0088] Figure 4 The phase error diagram between the unfolded phase diagram of the sphere measured using the method of the present invention and the true phase diagram (the unfolded phase diagram obtained by 20 phase shifts) is shown to be only 0.0821 rad, which proves that the method of the present invention can effectively reduce the phase error caused by color crosstalk in actual measurements.

[0089] Figure 5 The image shows the effect of three-dimensional reconstruction of a measured sphere using the method of the present invention. As can be seen, the reconstruction effect is smooth, proving the effectiveness of the method of the present invention.

[0090] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for removing color crosstalk error in color phase measurement profilometry, characterized in that: include: S1 Establish the phase correction matrix, the specific steps are as follows: S11 projects a grayscale fringe pattern and a three-step phase-shift color composite fringe pattern sequentially onto a white flat plate; S12 collects grayscale stripe images and performs normalization processing; S13 performs inverse cosine function calculation on the normalized grayscale fringe pattern to obtain the fundamental frequency component; S14 uses a color camera to capture a color stripe image, then performs color channel separation to form a three-channel grayscale image. S15 performs channel normalization on the three-channel grayscale image to form the actual three-channel normalized grayscale image; S16 Establish a system of quadratic equations in one variable related to the phase offset caused by color crosstalk error. Then, solve the system of quadratic equations to obtain the actual phase offset of the three channels. Furthermore, using the three-channel normalized grayscale image and the actual phase shift, an ideal three-channel three-step phase-shifted grayscale image is calculated; S17 Establish the phase correction matrix A, and use ideal and actual three-channel three-step phase-shifted grayscale image data to solve for the element values ​​in the phase correction matrix; S2 uses the phase correction matrix to remove color crosstalk errors during 3D reconstruction. The specific steps are as follows: S21 Projects a color composite stripe pattern onto the surface of the object being measured for actual 3D reconstruction; S22 acquires color stripe patterns for actual 3D reconstruction using a color camera, performs color channel separation, and forms a three-channel grayscale image for actual 3D reconstruction. S23 normalizes the three-channel grayscale image used for actual 3D reconstruction; S24 Using the phase correction matrix A, substitute the normalized three-channel three-step phase-shift grayscale image used for actual 3D reconstruction to calculate the ideal three-step phase-shift grayscale image used for actual 3D reconstruction. S25 uses a three-step phase-shifting method for phase calculation, and finally performs three-dimensional reconstruction.

2. The method for removing color crosstalk error in color phase measurement profilometry according to claim 1, characterized in that: The specific steps of S11 are as follows: A striped grayscale image is projected sequentially onto a white flat surface. I and a color composite striped pattern , ， in, , , These are colored striped patterns. Grayscale fringe patterns on the red, green, and blue channels; the two fringe patterns have the same frequency, the initial phase shift of the grayscale fringe pattern is 0, and the phase difference between the three channels of the color fringe pattern is 2π / 3.

3. The method for removing color crosstalk error in color phase measurement profilometry according to claim 1, characterized in that: S13 performs inverse cosine function calculation on the normalized grayscale fringe image to obtain the fundamental frequency component. The specific steps are as follows: Normalized grayscale stripe pattern The fundamental frequency component is obtained by performing inverse cosine function calculation. and ,in For the phase of each pixel in the image, x and y These are the horizontal and vertical pixel coordinates of the image.

4. The method for removing color crosstalk error in color phase measurement profilometry according to claim 3, characterized in that: The specific steps of S16 are as follows: In normalized three-channel images , , In the data collected, the red, green, and blue channels all exhibit phase shifts, as shown in the following formula. ， The phase offsets of the three channels are denoted as follows: θ r , θ g , θ b By expanding the fringe intensity formula for the three color channels, we obtain... , , Solve the quadratic equation with variables in one variable, and then obtain the actual phase offsets for the red, green, and blue channels. , , .

5. The method for removing color crosstalk error in color phase measurement profilometry according to claim 4, characterized in that: Red Channel The quadratic equation in one variable is given by the formula: ， The fundamental frequency component calculated in step S13 and Substituting these values ​​into the quadratic equation above, we can solve for the red channel separately. And thus obtain .

6. The method for removing color crosstalk error in color phase measurement profilometry according to claim 1, characterized in that: The specific steps of S17 are as follows: S11 , , After normalization, place it on the left side of the equation; the right side of the equation is determined according to the color crosstalk error rule. , , Add a phase correction matrix A in front, ， Calculate the correction factor for the red channel separately. a 12 and a 13 Correction factor for green channel a 21 and a 23 And the correction factor for the blue channel. a 31 and a 32 Thus, the phase correction matrix A is established.

7. The method for removing color crosstalk error in color phase measurement profilometry according to claim 6, characterized in that: Calculate the correction factor for the red channel. a 12 and a 13 Establish the following equation: ， By using the correspondence between the left and right sides of the equation above, we can obtain... a 12 and a 13 .

8. A device for removing color crosstalk error in color phase measurement profilometry, characterized in that: The method for performing the method as described in any one of claims 1-7 includes a phase correction matrix module and a color crosstalk error removal module. The phase correction matrix module is used to create the phase correction matrix. The color crosstalk error removal module is used to remove color crosstalk errors when using the phase correction matrix for 3D reconstruction.

9. An electronic device, characterized in that: The device includes a processor and a memory, wherein the processors are interconnected and the memory stores computer instructions, and the processors execute the computer instructions to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-7.

Images