Fast phase unwrapping method for non-continuous path based on modulation degree

Abstract

The application discloses a non-continuous path fast phase unwrapping method based on modulation degree, which comprises the following steps: calculating the wrapped phase value and the sequence corresponding to the Gray code of the collected image with phase shift fringes and Gray code fringes respectively, and calculating the modulation degree value of each pixel point; the calculated modulation degree is put into an array container in descending order, the row and column positions of each value in the array corresponding to the original image are saved, and the modulation degree threshold value is obtained; the pixels on the original image are sequentially phase unwrapped according to the size order of the modulation degree, and the unreliable phase is removed according to the modulation degree. The modulation degree is introduced into the phase unwrapping process in the three-dimensional reconstruction field. Since the modulation degree can reflect the phase reliability of the pixel position, the wrapped phase is unwrapped according to the modulation degree, on the one hand, the reliable phase can be solved preferentially, and the unreliable phase can be removed, thereby improving the reliability and accuracy of phase unwrapping; on the other hand, compared with the row-by-row and column-by-column phase unwrapping method, the speed of phase unwrapping can be accelerated, and the algorithm efficiency can be improved.

Description

Technical Field

[0001] This invention belongs to the field of structured light 3D reconstruction technology, and is specifically applied to the process of expanding the wrapped phase obtained by the phase shift method to solve for the absolute phase. Background Technology

[0002] With the continuous development of 3D reconstruction technology, its application in various fields is becoming increasingly widespread and in-depth. Research on efficient and accurate 3D reconstruction methods not only has significant academic importance but also demonstrates immense value in practical applications such as industry, medicine, and cultural relic preservation. In structured light 3D reconstruction technology, sinusoidal phase-shifting fringe projection is a widely used encoding scheme. It involves projecting a series of phase-shifting fringe patterns onto the object being measured, and then capturing the fringe pattern information with a camera to obtain 3D information.

[0003] In the aforementioned process, determining the absolute phase is a crucial step, as its efficiency and accuracy directly impact the efficiency of the entire 3D reconstruction process and the accuracy of subsequent point cloud generation. Currently, the commonly used method is to perform phase unwrapping row by row and column by column based on pixel positions. This method has certain bottlenecks in unwrapping efficiency. Furthermore, unreliable phases can interfere with the final generated point cloud, leading to a decrease in the accuracy of the reconstruction. Therefore, improving the efficiency and accuracy of phase unwrapping is an important research direction in structured light 3D reconstruction technology. Summary of the Invention

[0004] The purpose of this invention is to overcome the problems of slow efficiency and unreliable phases caused by the traditional method of phase unfolding row by row and column by column based on pixel. It proposes a fast phase unfolding method based on discontinuous path of modulation. According to the characteristic that modulation can reflect the reliability of pixels, reliable phases are unfolded first and unreliable phases are eliminated. This method can improve the efficiency of phase unfolding and the accuracy of generating absolute phase.

[0005] The objective of this invention is achieved through the following technical solution: a fast phase expansion method based on discontinuous paths of modulation, comprising the following steps:

[0006] A fast phase expansion method based on discontinuous paths of modulation intensity is characterized by the following steps:

[0007] S1. For the acquired image with phase shift stripes and Gray code stripes, calculate the sequence corresponding to the wrapping phase and Gray code respectively, and calculate the modulation value at each pixel.

[0008] For structured light 3D reconstruction technology, time coding is usually used, which involves projecting a pattern with striped coding onto the surface of the target object in chronological order while the camera captures the image. However, the method used in this invention is a coding method that combines phase shifting and Gray code.

[0009] The phase-shifting method is a method for solving the wrapping phase by projecting multiple sinusoidal fringes of different phases onto the target object. In a computer, a fringe pattern with phase shift can be calculated according to equation (1).

[0010] (1);

[0011] Where x and y represent the row and column coordinates of the pixel, respectively, A is the background light intensity, B is the modulation intensity, and N is the number of phase shift steps; Let be the phase value to be solved; This represents the grayscale distribution of the nth phase-shifted image.

[0012] Assuming an N-step phase-shifting method is used, since there are three unknowns in equation (1), the value of N must be at least 3. Combining the gray-level distribution equations formed by the N-step phase shifting, the wrapping phase value and modulation value of each pixel can be derived:

[0013] (2);

[0014] (3);

[0015] The modulation index B effectively reflects the reliability of the phase. Generally, the more reliable the phase, the larger the modulation index; the less reliable the phase, the smaller the modulation index.

[0016] The package phase obtained above is in The phase is continuous and unambiguous, but it contains multiple wrapper phase periods within the entire period, causing phase non-uniformity within the image scene. Therefore, Gray code and phase shifting methods need to be used in combination. The number of wrapper phase periods is marked by the sequence solved by Gray code to obtain absolute phase information. In order to eliminate the jump error that occurs at the edge of the period, complementary Gray code can be used for optimization. The following is the specific process of solving the Gray code and complementary Gray code sequences:

[0017] (4);

[0018] In equation (4), m represents the number of bits in the Gray code, and k1 and k2 represent the sequences corresponding to the Gray code and complementary Gray code, respectively. j This represents the Gray code value of the j-th bit. i represents the mapping transformation from the Gray code value to the binary value.

[0019] S2. The calculated modulation levels are placed into an array container in descending order, and the row and column positions of each value in the array corresponding to the original image are saved. The modulation threshold is then calculated.

[0020] The modulation values ​​calculated above are sorted, that is, the pixels with high reliability are placed at the front of the phase expansion, and the phases with low reliability or even unreliability are placed at the back of the phase expansion, so that reliable phases are expanded first and unreliable phases can be uniformly eliminated.

[0021] Furthermore, when sorting the modulation values, the row and column positions of the original pixels are saved at the same time. Therefore, in the phase unfolding process, unlike the traditional row and column unfolding, it is a discontinuous phase unfolding for the wrapped phase image. Obviously, after the above processing, the phase of the target object can be unfolded first, and the unfolding efficiency is improved.

[0022] Furthermore, for the calculation of the modulation threshold, the Otsu algorithm and the characteristics of modulation can be combined to adopt appropriate weight values ​​to calculate the modulation threshold.

[0023] S3. Perform phase unrolling on the pixels in the original image in order of modulation magnitude, and remove unreliable phases according to the modulation magnitude.

[0024] Step S3 includes the following sub-steps:

[0025] S3-1. Traverse the array container that stores the modulation index value and its corresponding row and column positions of the original image. Since the array container has been sorted from largest to smallest, it can expand the phase of pixels with large modulation index first, expand the phase of pixels with small modulation index later or even remove them. Moreover, the phase expansion is performed according to the modulation index, and the phase expansion path is no longer row by row and column by column, but non-continuous.

[0026] S3-2. For each modulation value in the array container, compare it with the preset modulation threshold. If it is less than the modulation threshold, set the absolute phase value of the corresponding pixel to 0, that is, remove the phase.

[0027] S3-3. If the modulation value of the pixel is greater than the modulation threshold, then the phase corresponding to the pixel is expanded, that is, the absolute phase is solved by combining the phase shift method and Gray code, as shown in the following formula:

[0028] (5);

[0029] S3-4. After traversing the modulation values ​​in the array container, output the absolute phase diagram obtained from the solution, which is the result of the phase expansion.

[0030] The beneficial effects of this invention are as follows: Based on the characteristic that the magnitude of the modulation index reflects the reliability of the phase, this invention sorts the modulation in descending order and decodes the corresponding pixels according to the order of the modulation indices. On the one hand, compared with the traditional method of decoding pixels row by row and column by column, this invention optimizes the efficiency of phase unfolding and allows reliable phases to be decoded first. On the other hand, by comparing the modulation index value of each pixel with the modulation index threshold, the phases of pixels with excessively small modulation are eliminated, that is, unreliable phase values ​​are eliminated, thereby improving the accuracy of subsequent point cloud generation. Attached Figure Description

[0031] Figure 1 This is a flowchart of the fast phase expansion method based on discontinuous paths of modulation according to the present invention;

[0032] Figure 2 This is a block diagram illustrating the phase expansion based on modulation magnitude in this invention. Detailed Implementation

[0033] Regarding modulation, the modulation of the object being measured is generally high. When capturing image data, the object is typically placed in the center of the image to obtain better image information. Therefore, this invention assumes a 3x3 cell containing modulation values ​​of different sizes. To better illustrate the process, the modulation threshold is preset to 2. The following will be discussed in conjunction with the appendix... Figure 1 and Figure 2 The technical solution of the present invention will be further explained.

[0034] like Figure 1 As shown, the fast phase expansion method for discontinuous paths based on modulation density of the present invention includes the following steps:

[0035] S1. For the acquired image with phase shift stripes and Gray code stripes, calculate the sequence corresponding to the wrapping phase and Gray code respectively, and calculate the modulation value at each pixel.

[0036] For structured light 3D reconstruction technology, time coding is usually used, which involves projecting a pattern with striped coding onto the surface of the target object in chronological order while the camera captures the image. However, the method used in this invention is a coding method that combines phase shifting and Gray code.

[0037] The phase-shifting method is a method for solving the wrapping phase by projecting multiple sinusoidal fringes of different phases onto a target object. In a computer, a fringe pattern with phase shift can be calculated according to equation (1).

[0038] (1);

[0039] Where x and y represent the row and column coordinates of the pixel, respectively, A is the background light intensity, B is the modulation intensity, and N is the number of phase shift steps; Let be the phase value to be solved; This represents the grayscale distribution of the nth phase-shifted image.

[0040] Assuming an N-step phase-shifting method is used, since there are three unknowns in equation (1), the value of N must be at least 3. Combining the gray-level distribution equations formed by the N-step phase shifting, the wrapping phase value and modulation value of each pixel can be derived:

[0041] (2);

[0042] (3);

[0043] The modulation index B effectively reflects the reliability of the phase. Generally, the more reliable the phase, the larger the modulation index; the less reliable the phase, the smaller the modulation index.

[0044] The package phase obtained above is in The phase is continuous and unambiguous, but it contains multiple wrapper phase periods within the entire period, causing phase non-uniformity within the image scene. Therefore, Gray code and phase shifting methods need to be used in combination. The number of wrapper phase periods is marked by the sequence solved by Gray code to obtain absolute phase information. In order to eliminate the jump error that occurs at the edge of the period, complementary Gray code can be used for optimization. The following is the specific process of solving the Gray code and complementary Gray code sequences:

[0045] (4);

[0046] In equation (4), m represents the number of bits in the Gray code, and k1 and k2 represent the sequences corresponding to the Gray code and complementary Gray code, respectively. j This represents the Gray code value of the j-th bit. i represents the mapping transformation from the Gray code value to the binary value.

[0047] S2. The calculated modulation levels are placed into an array container in descending order, and the row and column positions of each value in the array corresponding to the original image are saved. The modulation threshold is then calculated.

[0048] The modulation values ​​calculated above are sorted, that is, the pixels with high reliability are placed at the front of the phase expansion, and the phases with low reliability or even unreliability are placed at the back of the phase expansion, so that reliable phases are expanded first and unreliable phases can be uniformly eliminated.

[0049] Furthermore, when sorting the modulation values, the row and column positions of the original pixels are saved at the same time. Therefore, in the phase unfolding process, unlike the traditional row and column unfolding, it is a discontinuous phase unfolding for the wrapped phase image. Obviously, after the above processing, the phase of the target object can be unfolded first, and the unfolding efficiency is improved.

[0050] Furthermore, for the calculation of the modulation threshold, the Otsu algorithm and the characteristics of modulation can be combined to adopt appropriate weight values ​​to calculate the modulation threshold.

[0051] S3. Perform phase unrolling on the pixels of the original image sequentially according to the modulation index, and remove unreliable phases based on the modulation index. For example... Figure 2 As shown:

[0052] Step S3 includes the following sub-steps:

[0053] S3-1. Traverse the array container that stores the modulation index value and its corresponding row and column positions of the original image. Since the array container has been sorted from largest to smallest, it can expand the phase of pixels with large modulation index first, expand the phase of pixels with small modulation index later or even remove them. Moreover, the phase expansion is performed according to the modulation index, and the phase expansion path is no longer row by row and column by column, but non-continuous.

[0054] S3-2. For each modulation value in the array container, compare it with the preset modulation threshold. If it is less than the modulation threshold, set the absolute phase value of the corresponding pixel to 0, that is, remove the phase.

[0055] S3-3. If the modulation value of the pixel is greater than the modulation threshold, then the phase corresponding to the pixel is expanded, that is, the absolute phase is solved by combining the phase shift method and Gray code, as shown in the following formula:

[0056] (5);

[0057] S3-4. After iterating through the modulation values ​​in the array container, output the obtained absolute phase map, which is the result of the phase expansion; Figure 2 In the diagram, each square stores a modulation value. The unfolding order of each pixel can be easily seen by traversing the modulation array. When the traversed modulation value is less than a preset modulation threshold, the absolute phase value corresponding to that pixel is set to 0. Figure 2 Different colored blocks were used to represent this.

[0058] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A fast phase expansion method based on discontinuous paths of modulation, characterized in that, Includes the following steps: S1. For the acquired image with phase shift stripes and Gray code stripes, calculate the sequence corresponding to the wrapping phase and Gray code respectively, and calculate the modulation value at each pixel. The method adopted is an encoding method that combines phase shifting and Gray code; The phase-shifting method is a method of solving the wrapping phase by projecting multiple sinusoidal fringes of different phases onto the target object; in a computer, a fringe pattern with phase shift is calculated according to equation (1): （1）； Where x and y represent the row and column coordinates of the pixel, respectively, A is the background light intensity, B is the modulation intensity, and N is the number of phase shift steps; Let be the phase value to be solved; This represents the grayscale distribution of the nth phase-shifted image; Assuming an N-step phase-shifting method is used, since there are three unknowns in equation (1), the value of N must be at least 3. Combining the gray-level distribution equations formed by the N-step phase shifting, the wrapping phase value and modulation value of each pixel are derived: （2）； （3）； Among them, the modulation degree B can effectively reflect the reliability of the phase; the more reliable the phase, the larger the modulation degree; the less reliable the phase, the smaller the modulation degree. The obtained package phase is in The inner phase is continuous and unambiguous, but it contains multiple wrapper phase periods within the entire period, causing phase non-uniformity in the image scene. Therefore, Gray code and phase shifting method are used in combination. The number of wrapper phase periods is marked by the sequence solved by Gray code to obtain absolute phase information. In order to eliminate the jump error that occurs at the edge of the period, complementary Gray code is used for optimization. The following is the specific process of solving the Gray code and complementary Gray code sequences: （4）； In equation (4), m represents the number of bits in the Gray code, k1 and k2 represent the sequences corresponding to the Gray code and complementary Gray code, respectively, and GC j represents the code value of the j-th Gray code bit; i represents the mapping transformation from the Gray code value to the binary value; S2. The calculated modulation levels are placed into an array container in descending order, and the row and column positions of each value in the array corresponding to the original image are saved. The modulation threshold is then calculated. The calculated modulation is sorted, that is, the pixel positions with high reliability are placed at the front of the phase expansion, and the phases with low reliability or even unreliable are placed at the back of the phase expansion, so that reliable phases are expanded first and unreliable phases can be uniformly eliminated. Furthermore, when sorting the modulation values, the row and column positions of the original pixels are saved at the same time. Therefore, in the phase unwrapping process, unlike the traditional row and column unwrapping, it is a discontinuous phase unwrapping for the wrapped phase image. Obviously, after processing, the phase of the target object can be unwrapped first, and the unwrapping efficiency is improved. Furthermore, for the calculation of the modulation threshold, an appropriate weight value is adopted to calculate the modulation threshold by combining the Otsu algorithm and the characteristics of modulation. S3. Perform phase unrolling on the pixels in the original image in order of modulation magnitude, and remove unreliable phases according to the modulation magnitude.

2. The fast phase expansion method based on discontinuous paths according to claim 1, characterized in that, Step S3 includes the following sub-steps: S3-1. Traverse the array container that stores the modulation index value and its corresponding row and column positions of the original image. Since the array container has been sorted from largest to smallest, it can expand the phase of pixels with large modulation index first, expand the phase of pixels with small modulation index later or even remove them. Moreover, the phase expansion is performed according to the modulation index, and the phase expansion path is no longer row by row and column by column, but non-continuous. S3-2. For each modulation value in the array container, compare it with the preset modulation threshold. If it is less than the modulation threshold, set the absolute phase value of the corresponding pixel to 0, that is, remove the phase. S3-3. If the modulation value of the pixel is greater than the modulation threshold, then the phase corresponding to the pixel is expanded, that is, the absolute phase is solved by combining the phase shift method and Gray code, as shown in the following formula: （5）； S3-4. After traversing the modulation values ​​in the array container, output the absolute phase diagram obtained from the solution, which is the result of the phase expansion.

Images