An interleaved phase unwrapping method to avoid periodic errors

Abstract

The present invention discloses an interlaced phase unwrapping method that avoids periodic errors, comprising the following steps: Step 1: Obtain an N-step phase-shifted grating deformed fringe pattern modulated by an object, calculate the wrapped phase using a phase shift formula, and then construct an additional wrapped phase from the current wrapped phase; Step 2: Obtain a traditional binary Gray code deformed fringe pattern modulated by the object, establish a lookup table after image processing to determine the phase order; Step 3: Detect phase truncation points, use phase jump points to divide the forward phase unwrapping region, and combine the phase orders with the unwrapped region to perform forward and reverse phase unwrapping; Step 4: Combine the forward and reverse phase unwrapping with the phase order information to obtain the final absolute phase. Since the unstable jump region of the wrapped phase and erroneous edge order information are not used, the influence of edge order and wrapped phase jump errors can be avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional shape measurement, and in particular to an interleaved phase unwrapping method for avoiding periodic errors. Background Art

[0002] With modern advances in computational methods, optics, and graphics computing, research in the field of computer vision has rapidly grown, achieving widespread research and application over the past few decades. To meet the demands of production and daily life, the development and application of computer vision technology has gradually evolved from two-dimensional images to three-dimensional space. Enabling machines to acquire high-quality spatial stereoscopic perception information has become a key research focus in computer vision today. Structured light measurement, due to its non-contact, high-precision, and robust nature, has become one of the most practical technologies in 3D reconstruction.

[0003] Phase-shift fringe measurement involves projecting a grating pattern carrying phase information. A camera captures the image, modulated by the scene being measured. The wrapped phase is calculated using a phase-shifting method, then unwrapped into absolute phase using a phase unwrapping algorithm. Finally, the three-dimensional point cloud of the object's surface is calculated using the system's calibration information. In this process, the three-dimensional surface information is hidden within the wrapped phase. Because the wrapped phase is calculated using the inverse tangent function, the phase value is truncated within the range (-π, π]) and is not globally unique. Unwrapping the wrapped phase into absolute phase is necessary to uniquely determine the point cloud in space using this phase information combined with the system's calibration information.

[0004] To unwrap the wrapped phase into the absolute phase, additional coded structured light is projected to assist in phase unwrapping. Phase order information is obtained using the additional coded structured light, and this information is then used to assist in phase unwrapping of the wrapped phase to obtain the absolute phase. Gray code is a commonly used method for calculating phase order, and it is known to be resistant to interference. However, it also has significant drawbacks. Projector defocus and camera discrete sampling can cause errors in the binary boundaries of the Gray code, misaligning the binary boundaries with the phase wrapping boundaries. This leads to periodic phase errors during the phase unwrapping process, which directly affect the measurement results. Furthermore, in actual measurement environments, the use of the inverse tangent function for phase wrapping calculation can cause multiple unstable phase jumps around the boundaries of the wrapped phase, which can also affect the measurement results during phase unwrapping. Both of these errors exhibit periodicity, making them difficult to remove through post-processing of the point cloud.

[0005] To eliminate cycle-jump errors, researchers at home and abroad have proposed several measures with promising results. For example, the article "3-D shape measurement based on complementary Gray-code light" proposes a method using complementary Gray codes. This method constructs two sets of complementary phase orders by projecting an additional Gray code pattern, and then combines this with phase wrapping to perform phase unwrapping to avoid errors. However, this method requires projecting an additional Gray code pattern, which reduces system measurement efficiency.

[0006] Therefore, it is necessary to design an interleaved phase unwrapping method that avoids periodic errors. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to propose an interleaved phase unwrapping method that avoids periodic errors. Since the unstable jump area of ​​the wrapped phase is not used, and the erroneous edge order information is not used, the influence of the edge order and wrapped phase jump errors can be avoided.

[0008] The technical solutions of the invention are as follows:

[0009] A staggered phase unwrapping method for avoiding periodic errors comprises the following steps:

[0010] Step 1: Obtain the deformed fringe pattern of the N-step phase-shifted grating modulated by the object in the test scene, calculate the wrapping phase using the phase shift formula, and then construct additional wrapping phases from the current wrapping phase;

[0011] Step 2: Obtain the traditional binary Gray code deformed fringe pattern modulated by the object, and after image processing, establish a lookup table to determine the phase order;

[0012] Step 3: Detect the phase truncation point, use the phase jump point to divide the forward phase unwrapping area, and combine the phase order with the unwrapping area to perform forward and reverse phase unwrapping;

[0013] Step 4: Unwrap the forward and reverse phases and combine them with the phase order information to obtain the final absolute phase.

[0014] The wrapped phase in step 1 is obtained as follows:

[0015] Step 1.1: A projector projects an N-step phase-shifted fringe pattern onto the scene to be measured, and an industrial camera simultaneously captures the deformed fringe pattern to obtain the initial wrapping phase.

[0016] Step 1.2: Using the initial wrapped phase, construct three additional wrapped phases through offset calculation, so that the initial phase difference between any two adjacent wrapped phases is π / 2;

[0017] Among them, the deformation phase shift fringe pattern obtained in step 1.1 is The calculation formula for the initial wrapping phase is:

[0018]

[0019] Where, the superscript c represents the pattern captured by the camera, N is the number of phase-shifted patterns, (u, v) represents the pixel coordinates, and the pixel coordinates are retrieved from zero to the maximum value. φ1(u, v) is the initial wrapping phase.

[0020] The formula for calculating the misalignment in step 1.2 is:

[0021] No matter what N is, there are always 4 phases;

[0022] Among them, wrap[φ n (u,v)]=φ n (u,v)+2π,φ n (u, v)<π, φ2(u, v), φ3(u, v) and φ4(u, v) are the wrapping phases obtained by misalignment calculation.

[0023] The phase order in step 2 is obtained as follows:

[0024] Step 2.1: Use a DLP projector to project multiple traditional binary Gray code fringe patterns onto the test scene, where the minimum grating pitch of the Gray code is twice the period of the phase-shifted fringe. An industrial camera is used to simultaneously capture the deformed fringe patterns.

[0025] Step 2.2, perform binary segmentation, opening operation and rounding operation on the image;

[0026] Step 2.3: Establish a Gray code-phase order lookup table to convert multiple binary images into decimal images and phase order information.

[0027] The forward and reverse phase unwrapping results in step 3 are obtained as follows:

[0028] Step 3.1, using phase value constraints between multiple phases to detect the truncation edge of the wrapped phase;

[0029] Step 3.2: Use the truncated edge information combined with the marker to divide the phase unwrapping area;

[0030] Step 3.3, use the phase unwrapping region combined with the wrapped phase to perform segmented phase unwrapping;

[0031] The formula for piecewise forward phase unwrapping is:

[0032]

[0033] Among them, k mode1 Indicates the mode of the level information in the current A1(u,v) area, Φ1(u,v) is the positive absolute phase result; the retrieval direction of the pixel coordinate is from zero to the maximum value;

[0034] The formula for piecewise reverse phase unwrapping is:

[0035]

[0036] Among them, k mode2 Indicates the mode of the level information in the current A3(u,v) area, Φ2(u,v) is the inverse absolute phase result; the retrieval direction of the pixel coordinate is from the maximum value to the zero point;

[0037] 5. The interleaved phase unwrapping method for avoiding periodic errors according to claim 1, wherein the final absolute phase in step 4 is:

[0038]

[0039] in,

[0040] (1)Φ merge (u,v) represents the absolute phase after fusion;

[0041]

[0042] (u,v) errors The coordinate value representing the error phase, (u,v) errors ∈{|Φ merge (u,v)-Φ medfilt (u,v)|≥Th}(2)

[0043] Wherein Th represents a threshold value; the value of the threshold value Th is selected according to user requirements, and is pai / 2 in Example 1;

[0044] Φ medfilt Represents Φ after median filtering merge ;

[0045] Φ medfilt =medfilt[Φ merge ] a×b .

[0046] Beneficial effects:

[0047] 1. This invention proposes an interleaved phase unwrapping method that avoids periodic errors. By limiting the phase value, it can accurately locate the truncation edge of the wrapped phase, divide the phase unwrapping region by the truncation edge, and then combine the phase order information for phase unwrapping. This method not only avoids periodic errors caused by multiple jumps of the truncation edge caused by noise, but also avoids periodic errors caused by Gray code binarization errors.

[0048] 2. During phase unwrapping, the phase truncation edge is not at the edge of the phase order, and the Gray code edge error will not affect the phase order determination;

[0049] 3. This invention adopts a traditional Gray code + phase-shift stripe encoding scheme, which does not require the projection of additional stripes. It avoids periodic errors without affecting the normal phase and has good robustness in high-noise environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of Gray code phase unwrapping error analysis in Example 1 of the present invention, where (a) shows a three-step phase shift with a period of 4 and Gray code encoding, (b) shows a cross-sectional diagram of the wrapped phase and phase order, and (c) shows the result of traditional phase unwrapping.

[0051] Figure 2 A flow chart of an interleaved phase unwrapping method for avoiding periodic errors;

[0052] Figure 3 Schematic diagram of a cross section of a wrapped phase constructed in Example 1 of the present invention, wherein (a) shows a cross section of the wrapped phase, and (b) shows an enlarged cross section of the wrapped phase;

[0053] Figure 4 Schematic diagram of the phase unwrapping region division in Example 1 of the present invention, wherein (a) shows a phase truncation edge diagram, (b) shows a phase unwrapping region diagram, and (c) shows an error-free phase cross-section diagram obtained by region division.

[0054] Figure 5 Schematic diagram of the forward phase unwrapping result in Example 1 of the present invention;

[0055] Figure 6 This is the Gray code + phase-shift fringe pattern captured in Example 2 of the present invention;

[0056] Figure 7 Figures 1 and 2 are experimental diagrams of an interleaved phase unwrapping method for avoiding periodic errors in Example 2 of the present invention, where (a) shows the forward phase unwrapping diagram, (b) shows the reverse phase unwrapping diagram, (c) shows the fused absolute phase diagram, and (d) shows the restored absolute phase diagram.

[0057] Figure 8 The depth distribution map of the object reconstructed in Example 2 of the present invention; DETAILED DESCRIPTION

[0058] Example 1:

[0059] The present invention will be further described below with reference to the accompanying drawings and embodiments. However, this should not be construed as limiting the scope of the present invention to the following embodiments, as all technologies implemented based on the present invention fall within the scope of the present invention.

[0060] In order to better illustrate the method of the present invention, this embodiment is described using a three-step phase shift with a period number of 4 and a Gray code encoding scheme, such as Figure 1 As shown in (a). Due to the defocus of the projector and the discrete sampling effect of the camera, the phase order originally aligned with the phase cutoff is shifted to the left or right by multiple pixels, as shown in Figure 1 As shown in (b), a periodic phase error with an amplitude of ±2π appears in the absolute phase, as shown in Figure 1 As shown in (c), the present invention is proposed to avoid the occurrence of periodic errors.

[0061] like Figure 2 FIG. 1 is a flow chart of the present invention, which shows a staggered phase unwrapping method for avoiding periodic errors, comprising the following steps:

[0062] Step 1: Obtain a deformed fringe pattern of a three-step phase-shifted grating modulated by the object. Use the phase-shift formula to calculate the wrapping phase, and then construct additional wrapping phases from the current wrapping phase. The fringe pattern is projected onto the object. Because the object is irregular, the fringe pattern will be deformed, which is technically called "modulation."

[0063] A projector projects a three-step phase-shifted fringe pattern onto the scene to be tested (a scene represents the existence of something or a certain structure, in which there must be something, and these things are objects). The industrial camera captures the pattern, which can be expressed as:

[0064]

[0065] Where i = 1, 2, 3, (u, v) represents the pixel coordinates, which are retrieved from zero to the maximum value, r(u, v) represents the reflectivity of the object, e1(u, v) represents the ambient light on the object surface, e2(u, v) represents the ambient light directly entering the camera from the environment, α(u, v) is the average grayscale of the image, β(u, v) is the grayscale modulation of the image, superscript c represents the camera, superscript p represents the projector, and φ1(u, v) is the wrapping phase to be calculated. The calculation formula for the initial wrapping phase is:

[0066]

[0067] Substituting the wrapping phase φ1(x) into formula (3), three additional wrapping phases are constructed through the staggered operation, so that the initial phase difference between any two adjacent wrapping phases in these four wrapping phases is π / 2. The cross section and local magnification details of the wrapping phase are shown as follows: Figure 3 (a) and 3(b).

[0068]

[0069] Among them, wrap[φ n (u,v)]=φ n (u,v)+2π,φ n (u, v)<π, φ2(u, v), φ3(u, v) and φ4(u, v) are the wrapping phases obtained by misalignment calculation.

[0070] Step 2: Obtain the traditional binary Gray code deformed fringe pattern modulated by the object, and after image processing, establish a lookup table to determine the phase order;

[0071] Use a DLP projector to project multiple traditional binary Gray code fringe patterns onto the test scene, where the minimum grating pitch of the Gray code is twice the period of the phase-shifted fringe, and an industrial camera synchronously captures the deformed fringe pattern. and Use formula (4) to perform threshold segmentation on the image.

[0072]

[0073] Among them, the threshold Use formula (5-6) to perform opening operation and rounding on the binarized image to remove the error points on the image. The opening operation formula is:

[0074]

[0075] The rounding function is: GC i (u,v)=round[GC i (u,v)] (6)

[0076] Where K is the image operation kernel, p and q are the length and width of the kernel respectively. In this embodiment, the values ​​are p=3 and q=3. is the opening operator, the kernel size and shape are adjusted by the size of the error on the Gray code stripe, and round[·] is the rounding operator. Then, the binary image is converted into decimal mode using formula (7).

[0077]

[0078] Where V(u,v) represents the decimal mode. Then, the Gray code-phase order lookup table is combined with formula (8) to obtain the phase order information.

[0079] Gray code-phase order lookup table

[0080]

[0081] k(u,v)=ε[V(u,v)] (8)

[0082] Here, ε[·] represents the mapping from decimal numbers to phase orders, and k(u,v) represents the phase order information. Two images are required for calculation, and since 2^2 = 4, one image can only encode two cycles. The fusion process uses Equation 6.

[0083] Step 3: Detect the phase truncation point, use the phase jump point to divide the forward phase unwrapping area, and combine the phase order with the unwrapping area to perform forward and reverse phase unwrapping;

[0084] First, the truncated edges of the wrapped phases φ2(x) and φ4(x) are detected using formula (9).

[0085]

[0086] Among them, E φ2 (u,v) and E φ4 (u,v) represent the truncated edges of φ2(u,v) and φ4(u,v) respectively. Figure 4 (a) is shown. Next, the truncated edges are used to divide the phase unwrapping area. Since the two types of truncated edges are in the form of alternating clusters, two identifiers are defined using formula (10) to assist in the division of the area.

[0087]

[0088] If a phase truncation point is detected, flag1 and flag2 are assigned and locked. When the next phase truncation point is detected, flag1 and flag2 are reassigned and locked, and two expansion intervals are divided using formula (11-12).

[0089]

[0090]

[0091] Among them, A1(u,v) and A2(u,v) are two sets of alternating phase unwrapping intervals. The phase unwrapping intervals are as follows: Figure 4 As shown in (b), Figure 4(b) shows the phase value segments in the unwrapped interval. Next, the phase unwrapping region is combined with the wrapped phase using formula (13) to perform segmented phase unwrapping.

[0092]

[0093] Among them, k mode1 It represents the mode of the level information in the current A1(u,v) area, Φ1(u,v) is the positive absolute phase result, and the phase unwrapping process is as follows: Figure 5 However, in the actual phase unwrapping process, the first phase truncation edge detected may not accurately cover the edge of the object to be measured, which causes the phase value on the left side of the first truncation edge to have an integer multiple error of 2π, which requires the absolute phase to be repaired. Therefore, the retrieval direction of the pixel coordinate is changed from the maximum value to the zero point, and flag1 and flag2 are initialized using formula 10, and E is set to φ2 Perform reverse search and use formula (14-15) to divide two new expansion intervals

[0094]

[0095]

[0096] Among them, A3(u,v) and A4(u,v) are two sets of new alternating phase unwrapping intervals, and the superscript ← indicates reverse retrieval. Next, use formula (16) to combine the phase unwrapping region with the wrapped phase to perform segmented phase unwrapping.

[0097]

[0098] Among them, k mode2 It represents the mode of the level information in the current A3(u,v) region, and Φ2(u,v) is the inverse absolute phase result.

[0099] Step 4: Unwrap the forward and reverse phases and combine them with the phase order information to obtain the final absolute phase.

[0100] Using the reverse phase unwrapping and the back-to-front phase retrieval method, the reverse absolute phase Φ2(u,v) is obtained, and the forward and reverse phase unwrapping results are fused using formula (17).

[0101]

[0102] Among them, Φ merge (u, v) represents the absolute phase after fusion. To further remove the small amount of stripe-like errors in the phase, median filtering is further used using formula (18-19) for error detection.

[0103] Φmedfilt =medfilt[Φ merge ] a×b (18)

[0104] (u,v) errors ∈{|Φ merge (u,v)-Φ medfilt (u,v)|≥Th} (19);

[0105] Among them, medfilt[·] a×b represents the median filter operator, a and b represent the length and width of the filter kernel respectively. In this embodiment, the values ​​are a=3 and b=3, Φ medfilt Represents Φ after median filtering merge Th represents the threshold value, which is π / 2 in this embodiment, (u, v) errors The coordinate value of the error phase is represented by . Then, the error phase is replaced by the correct phase using formula (20) to obtain the error-free absolute phase.

[0106]

[0107] Where Φ(u,v) is the final absolute phase. The absolute phase can then be used to achieve 3D reconstruction.

[0108] Example 2:

[0109] The embodiment of the present invention demonstrates the phase recovery results for a complex scene with two objects. The measurement scene contains a fan blade and a sphere. Using a three-step phase shift, each fringe has 32 cycles. The number of Gray codes required for projection is calculated to be 5. The captured fringe pattern is as follows: Figure 6 As shown. First, three phase-shifted stripes By changing the sequence order, the phase shift truncation phases φ1(u,v), φ2(u,v), φ3(u,v) and φ4(u,v) can be obtained using formula (3). Then, the five Gray codes are used to calculate the phase order k(u,v) using formula (4-8). Then, formula (9) is used in combination with the four wrapped phases to accurately locate the truncation edges of φ2(u,v) and φ4(u,v). Formulas (10-12) are used in combination with the truncation edges to divide the phase expansion areas A1(u,v) and A2(u,v). Then, the expanded areas are combined with the wrapped phases using formula (13) to calculate the positive absolute phase Φ1(u,v). The positive absolute phase is as follows: Figure 7 As shown in (a), the inverse phase expansion is then performed using formulas (10, 15-16) to obtain the absolute phase Φ2(u, v). The inverse absolute phase is as follows: Figure 7 As shown in (b), the forward and reverse phase unwrapping results are combined and phase fusion is performed using formula (17) to obtain Φ merge (u,v), such as Figure 7 As shown in (c), the phase is further repaired using formula (18-20) to obtain the final error-free absolute phase Φ(u,v). The absolute phase is as follows Figure 7 As shown in (d), the depth distribution map of the object is reconstructed by combining the structured light system parameters. Figure 8 shown.

Claims

1. A staggered phase unwrapping method for avoiding periodic errors, characterized in that: The following steps are involved: Step 1: Obtain a deformed fringe pattern of an N-step phase-shifted grating modulated by an object in the scene to be measured, calculate the wrapping phase using the phase shift formula, and then construct additional wrapping phases from the current wrapping phase; Step 2: Obtain the traditional binary Gray code deformed fringe pattern modulated by the object, and after image processing, establish a lookup table to determine the phase order; Step 3: Detect the phase truncation point, use the phase jump point to divide the forward phase unwrapping area, and combine the phase order with the unwrapping area to perform forward and reverse phase unwrapping; Step 3.1, using phase value constraints between multiple phases to detect the truncation edge of the wrapped phase; Step 3.2: Use the truncated edge information combined with the marker to divide the phase unwrapping area; Step 3.3, use the phase unwrapping region combined with the wrapped phase to perform segmented phase unwrapping; Step 4: Unwrap the forward and reverse phases and combine them with the phase order information to obtain the final absolute phase.

2. The interleaved phase unwrapping method for avoiding periodic errors according to claim 1, wherein: The wrapped phase in step 1 is obtained as follows: Step 1.1: A projector projects an N-step phase-shifted fringe pattern onto the scene to be measured, and an industrial camera simultaneously captures the deformed fringe pattern to obtain the initial wrapping phase. Step 1.2: Use the initial wrapping phase and construct three additional wrapping phases through stagger calculation, so that the initial phase difference between any two adjacent wrapping phases is ; Among them, the deformation phase shift fringe pattern obtained in step 1.1 is , the calculation formula of the initial wrapping phase is: Where, the superscript c represents the pattern captured by the camera, N is the number of phase-shifted patterns, (u, v) represents the pixel coordinates, and the pixel coordinates are retrieved from zero to the maximum value. is the initial wrapping phase; The formula for calculating the misalignment in step 1.2 is: ; in, , , and is the parcel phase calculated for the misalignment.

3. The interleaved phase unwrapping method for avoiding periodic errors according to claim 2, wherein: The phase order in step 2 is obtained as follows: Step 2.1: Use a DLP projector to project multiple traditional binary Gray code fringe patterns onto the test scene, where the minimum grating pitch of the Gray code is twice the period of the phase-shifted fringe. An industrial camera is used to simultaneously capture the deformed fringe patterns. Step 2.2, perform binary segmentation, opening operation and rounding operation on the image; Step 2.3: Establish a Gray code-phase order lookup table to convert multiple binary images into decimal images and phase order information.

4. The interleaved phase unwrapping method for avoiding periodic errors according to claim 3, wherein: The forward and reverse phase unwrapping results in step 3 are obtained as follows: The formula for piecewise forward phase unwrapping is: Among them, Indicates the current The mode of the sub-level information in the region, is the positive absolute phase result; the retrieval direction of the pixel coordinate is from zero point to the maximum value; The formula for piecewise reverse phase unwrapping is: Among them, Indicates the current The mode of the sub-level information in the region, It is the inverse absolute phase result; the retrieval direction of the pixel coordinate is from the maximum value to the zero point.

5. The interleaved phase unwrapping method for avoiding periodic errors according to claim 4, wherein: The final absolute phase in step 4 is: ； in, represents the absolute phase after fusion; ； represents the coordinate value of the error phase, in represents the threshold value, Indicates the use of median filtering ; 。

Images