Color Gray code assisted phase unwrapping method and applied three-dimensional imaging method and system

Abstract

The present invention discloses a color Gray code encoding assisted phase unwrapping method and a three-dimensional imaging method and system using the method, belonging to the field of three-dimensional imaging technology. The method comprises the following steps: constructing two sets of colored sinusoidal stripe patterns of different frequencies, capturing the two sets of stripe patterns after projecting the scene to be measured; converting the stripe pattern from RGB format to HSV format, extracting the hue channel and performing Gray code decoding; extracting the hue channel from the RGB format stripes for intensity normalization, and then filling it into a single-channel image, calculating the wrapped phase using a phase shift formula, and then calculating the heterodyne phase of the stripe patterns of different frequencies by a multi-frequency heterodyne method, and performing a first unwrapping of the wrapped phase; using the Gray code decoding order to assist in unwrapping the first unwrapped phase map, thereby obtaining the final absolute phase. The present invention embeds Gray code encoding into different channels of the two sets of patterns, significantly reducing the number of patterns required and effectively improving measurement efficiency.

Description

Technical Field

[0001] The invention discloses a color Gray code auxiliary phase unwrapping method and an applied three-dimensional imaging method and system, belonging to three-dimensional shape measurement technology. Background Art

[0002] With advances in modern computing methods, optical technology, and graphics computing, research in computer vision has rapidly grown and has achieved widespread application over the past few decades. To meet the needs of production and daily life, computer vision technology has gradually expanded from two-dimensional images to three-dimensional space. Enabling machines to obtain high-quality spatial stereoscopic information has become a key research topic in computer vision. Due to its non-contact, high-precision, and robust nature, structured light measurement has become one of the most practical technologies in 3D reconstruction.

[0003] Optical 3D measurement technology, with its high precision, fast response, and contactless nature, is widely used in fields such as industrial modeling, virtual reality, and microscopy. Fringe projection profilometry is particularly well-known. This method uses a projector to project a grating pattern carrying phase information, and a camera to synchronously capture the modulated image of the object surface. Phase shifting is used to decode the phase information in the image. However, because the contour information of the object surface is hidden in the phase, and the phase can only be obtained through inverse trigonometric functions, its effective range is limited, resulting in phase ambiguity. To eliminate this ambiguity, phase unwrapping technology must be used to recover the continuous absolute phase.

[0004] In order to stably unfold the wrapped phase into the absolute phase, it is necessary to project additional coded structured light to assist the wrapped phase unfolding. Phase order information is obtained through the additional coded structured light, and this information is used to assist the wrapped phase in obtaining the absolute phase. Common methods include projecting three sets of sinusoidal coded stripes of different frequencies for multi-frequency heterodyne phase unfolding, such as the stripe structured light 3D reconstruction method based on virtual binocular disclosed in Chinese patent application No. CN202111413425.1, or projecting a set of sinusoidal stripes and a set of Gray code coded stripes, and using Gray code to assist the sinusoidal stripe unfolding, such as the 3D measurement method of low-reflectivity workpiece based on structured light disclosed in Chinese patent application No. CN202410785197.8. However, whether projecting three sets of sinusoidal stripes or combining sinusoidal stripes and Gray code coded stripes, the number of stripes required is large, which reduces the measurement efficiency of the system. Therefore, calculating the absolute phase while reducing the number of projected patterns is one of the key issues to improve the performance of the phase measurement system. Summary of the Invention

[0005] The technical problem solved by the present invention is: to address the problem of low efficiency of projected coded structured light assisted phase unwrapping in existing phase unwrapping technology, and to provide a color Gray code assisted phase unwrapping method and an applied three-dimensional imaging method and system.

[0006] The present invention is implemented by the following technical solutions:

[0007] The present invention first discloses a color Gray code assisted phase unwrapping method, comprising the following steps:

[0008] Step 1: Construct two sets of colored sinusoidal fringe patterns with different frequencies to perform fringe projection on the outline of the scene to be measured;

[0009] Step 2: Capture two sets of colored sinusoidal fringe patterns after projecting the scene to be tested, and convert the projected images from RGB format images to HSV format images;

[0010] Step 3: Extract the red and blue areas from the HSV image based on the hue value, extract the channels corresponding to the red and blue areas from the RGB projection image, perform intensity normalization on the extracted red and blue area channels, merge and fill them into a single-channel image after intensity normalization, calculate the wrapped phases of the single-channel images of the two sets of colored sinusoidal fringe patterns with different frequencies using the phase shift formula, and perform the first unfolding of the wrapped phases of the two sets of colored sinusoidal fringe patterns with different frequencies;

[0011] Step 4: Extract the hue channel from the colored sinusoidal stripe pattern of the HSV format image in step 2 and perform Gray code decoding to obtain the Gray code decoding order.

[0012] Step 5: Use the Gray code decoding order obtained in step 4 to assist in the second unfolding of the phase diagram obtained in step 3, and obtain the final absolute phase through error compensation.

[0013] In the color Gray code assisted phase unwrapping method of the present invention, further, the process of constructing two sets of color sinusoidal stripe patterns of different frequencies in step 1 is as follows:

[0014] Define an N-step phase-shifted fringe pattern:

[0015] ,

[0016] in, represents the grayscale value of the phase-shifted fringe pattern at the pixel coordinate (u, v), where n is a positive integer representing the number of phase shift steps, a(u, v) and b(u, v) represent the background light intensity and modulation amplitude of the fringe pattern at the pixel coordinate (u, v), respectively, and f0 is the fringe frequency. a(u, v) and b(u, v) are encoded as follows: a(u, v) = 0.6, b(u, v) = 0.4;

[0017] There are two groups of phase-shifted fringe patterns, each group contains N images, which are N-step phase-shifted images, totaling 2N images, corresponding to the encoded 2N-bit Gray code image.

[0018] In the color Gray code assisted phase unwrapping method of the present invention, the two sets of phase-shifted stripe patterns are further Gray-coded using the three RGB channels, and the red and blue color channels are selected as coding colors to construct a color sinusoidal stripe pattern.

[0019] In the color Gray code assisted phase unwrapping method of the present invention, further, step 2 includes the following sub-steps:

[0020] Step 2.1: The projector projects two sets of colored sinusoidal fringe patterns of different frequencies onto the scene to be tested. The industrial camera simultaneously captures the two sets of projected colored sinusoidal fringe patterns. The RGB mode of the captured projected fringe pattern is expressed as:

[0021] ,

[0022] Where c represents the pattern captured by the camera, N represents the number of phase-shifted patterns, which is equal to the number of phase-shifted steps of the fringe pattern, and (u, v) represents the pixel coordinates. Represents the intensity of the RGB channels of the stripe pattern at the pixel coordinates (u, v);

[0023] Step 2.2: Convert the two projected colored sinusoidal stripe patterns from the RGB channel mode to the HSV channel mode. The conversion formula for the hue angle H is:

[0024] 、

[0025] ,

[0026] Where H(u,v) represents the hue angle of the pixel coordinate (u,v), and θ(u,v) represents the hue angle of the pixel coordinate (u,v) when the phase is (0,2π).

[0027] In the color Gray code assisted phase unwrapping method of the present invention, further, step 3 includes the following sub-steps:

[0028] Step 3.1: Extract the red and blue areas from the HSV projection image based on the hue value. Extract the corresponding channels from the RGB projection image based on the extracted red and blue areas. Normalize the intensity of the extracted regional channels and merge them into a single-channel image. The normalization formula is as follows:

[0029] ,

[0030] in, is the fringe intensity information of the fringe pattern after normalization of the red and blue areas at the pixel coordinates (u, v), l is the lower limit of intensity, and h is the upper limit of intensity;

[0031] Step 3.2: Use the following phase shift formula to calculate the wrapped phase of the red and blue regions in two sets of colored sinusoidal fringe patterns with different frequencies:

[0032] For the red area:

[0033] ,

[0034] For the blue area:

[0035] ,

[0036] where φ r (u,v) and φ b (u, v) refers to the phase of the stripe pattern wrapping of the pixel coordinate (u, v) in the red area and the blue area;

[0037] Step 3.3: Perform the first expansion of the wrapped phases of the two sets of fringe patterns with different frequencies by using the multi-frequency heterodyne method. The multi-frequency heterodyne calculation formula is as follows:

[0038] 、

[0039] ,

[0040] Among them, λ b is the frequency corresponding to the heterodyne phase Δφ, λ1 and λ2 are the phase functions corresponding to two sets of fringe patterns with different frequencies, Δφ is the heterodyne phase, that is, the difference between the two sets of fringe patterns with different frequencies, φ1 is the wrapping phase of the first set of fringe patterns, and φ2 is the wrapping phase of the second set of fringe patterns;

[0041] The wrapped phase of one group of fringe patterns is selected and unfolded once to obtain the first unfolded phase image.

[0042] In the color Gray code assisted phase unwrapping method of the present invention, further, the Gray code decoding in step 4 includes the following sub-steps:

[0043] Step 4.1: Extract the red area from the HSV stripe pattern using the hue channel. Use the following formula to assign the red area a value of 1 and the rest of the area a value of 0.

[0044] ,

[0045] Among them GC i (u,v) represents the Gray code image obtained by the pixel coordinates (u,v) in the hue channel of the HSV format image, and H(u,v) is the hue angle of the pixel coordinates (u,v) in the HSV format image;

[0046] Step 4.2: Convert the extracted red area Gray code into decimal code. The conversion formula is as follows:

[0047] 、

[0048] ,

[0049] Among them, B i is the i-th bit of the binary code, GC i is the i-th bit of the Gray code encoding diagram obtained in step 4.1, ⊕ represents the exclusive OR operation, k is the decimal number of the Gray code order, and m is the number of bits in the binary code.

[0050] In the color Gray code assisted phase unwrapping method of the present invention, further, step 5 includes the following process:

[0051] Step 5.1. Use the Gray code decoding order obtained in step 4 to assist in unfolding the first phase unwrapped image in step 3 and calculate the absolute phase image using the following formula:

[0052] ,

[0053] Among them, Φ erros (u, v) represents the absolute phase of the pixel coordinate (u, v) with periodic error, φ(u, v) represents the first phase unwrapped image of the pixel coordinate (u, v) obtained in step 3, and k(u, v) represents the decimal number of the Gray code order of the pixel coordinate (u, v) obtained by Gray code decoding in step 4;

[0054] Step 5.2: Perform median filtering on the absolute phase with periodic error to obtain a phase correction template;

[0055] Step 5.3: Use the correction template to assist in wrapping the phase and perform the second phase unwrapping to compensate for the error.

[0056] ,

[0057] Among them, Φ template represents the absolute phase after median filtering, Round(x) represents the integer closest to x, Φ(u,v) represents the error-free absolute phase of the pixel coordinates (u,v), and φ(u,v) is the first phase unwrapped image of the pixel coordinates (u,v) obtained in step 3.

[0058] The present invention further provides a three-dimensional imaging method, which uses a fringe projection profilometry method for three-dimensional imaging, and during the imaging process uses the above-mentioned color Gray code encoding assisted phase unwrapping method of the present invention to perform wrapped phase unwrapping of the projected fringe pattern.

[0059] Based on the above three-dimensional imaging method, the present invention further provides a three-dimensional imaging system, comprising:

[0060] The visible light projection module generates two sets of colored sinusoidal fringe patterns with different frequencies and projects fringe patterns onto the outline of the scene to be measured;

[0061] An image acquisition module captures two sets of colored sinusoidal fringe patterns projected onto the scene to be tested;

[0062] one or more processors;

[0063] a memory storing one or more computer programs;

[0064] The processor receives image data from the image acquisition module and calls the computer program in the memory to implement the three-dimensional imaging method of the present invention.

[0065] The present invention has the following beneficial effects:

[0066] The beneficial effects of the present invention are:

[0067] (1) The present invention proposes a color Gray code assisted phase unwrapping method, which embeds phase shift stripes into different channels of a color image in a Gray code encoding manner. The phase shift information in the color stripes is first extracted, the wrapped phase is calculated, and the first phase unwrapping is performed using a multi-frequency heterodyne method. The hue information of the color stripes is then used to decode the gray code. Finally, the second phase unwrapping is performed using the gray code to obtain the absolute phase.

[0068] (2) The present invention uses two sets of phase-shifted stripes to complete phase unwrapping, which not only avoids the problem of large errors in low-frequency stripes, but also avoids the problem of large periodic errors caused by Gray code encoding.

[0069] (3) When calculating the wrapped phase, the present invention takes into account the fact that the red and blue areas in the color image have high visual contrast and the hue values ​​of the two colors are greatly different, making them easy to extract and distinguish, thereby reducing crosstalk between stripes. The present invention specifically normalizes the red and blue coding areas in the color stripes and combines them into a single-channel image, ensuring that stripes of different colors are at the same intensity level.

[0070] (4) This method uses a color Gray code + phase-shift stripe encoding scheme, eliminating the need to project additional stripes. Because the Gray code is embedded in different channels of the two sets of patterns, the number of required patterns is significantly reduced. This method effectively improves measurement efficiency, avoids periodic errors, and does not affect the normal phase. It also exhibits good robustness in high ambient noise.

[0071] In summary, the color Gray code assisted phase unwrapping method and the applied three-dimensional imaging method disclosed in the present invention reduce the number of auxiliary projection fringes required for wrapping phase unwrapping, streamline the three-dimensional imaging calculation process, and improve the measurement efficiency of the three-dimensional imaging system.

[0072] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 This is a flow chart of the color Gray code assisted phase unwrapping method of the present invention.

[0074] Figure 2 This is a schematic diagram of color Gray code assisted phase unwrapping in Example 1.

[0075] Figures 3a-3f These are multiple groups of color deformed stripes captured in Example 1.

[0076] Figure 4 This is the expanded diagram of the Gray code in Example 1.

[0077] Figure 5a 、 5b This is a phase main value diagram of the two frequency stripe patterns in Example 1.

[0078] Figure 6 This is the dual-frequency heterodyne diagram in Example 1.

[0079] Figure 7 This is the phase unwrapping diagram in Example 1. DETAILED DESCRIPTION

[0080] Example

[0081] like Figure 1 and Figure 2As shown, the color Gray code assisted phase unwrapping method of the present invention specifically includes the following steps:

[0082] Step 1: Construct two sets of N-step colored sinusoidal fringe patterns with different frequencies, where N is a positive integer representing the number of phase shift steps. It can be selected based on the actual measurement accuracy requirements and system performance, and is usually in the range of 3-10.

[0083] Specifically, the construction process of two sets of N-step colored sinusoidal stripe patterns is as follows:

[0084] First, define the N-step phase-shifted fringe pattern:

[0085] .

[0086] in, represents the grayscale value of the fringe pattern, a(u,v) and b(u,v) represent the background light intensity and modulation amplitude, respectively, and f0 is the fringe frequency. To facilitate separation of different color regions by hue channel, a(u,v) and b(u,v) are encoded as follows: a(u,v)=0.6, b(u,v)=0.4.

[0087] Secondly, there are two sets of phase-shifted fringe patterns, totaling 2N images, corresponding to the 2N-bit Gray code image. The two sets of phase-shifted fringe patterns are Gray-coded using the three RGB channels. The color-H value mapping range is as follows:

[0088] Color-H value mapping range table.

[0089] .

[0090] The two sets of phase-shifted stripe patterns are encoded by Gray code through the three RGB channels, and the red and blue color channels are selected as the coding colors to construct a colored sinusoidal stripe pattern.

[0091] Step 2: Capture two sets of N-step colored sinusoidal fringe patterns after projecting the scene to be tested, and convert the projected images from RGB format images to HSV format images.

[0092] The specific conversion of the projection image from RGB format to HSV format includes the following sub-steps:

[0093] Step 2.1: The projector projects two sets of N-step colored sinusoidal fringe patterns with different frequencies onto the scene to be tested, and the industrial camera synchronously captures the two sets of N-step colored sinusoidal fringe patterns after projection. The obtained colored deformed fringe pattern is , the projected fringe pattern captured is expressed in RGB mode as follows:

[0094] .

[0095] Wherein, the superscript c represents the pattern captured by the camera, N represents the number of phase-shifted patterns, which is equal to the number of phase-shifted steps of the fringe pattern, (u, v) represents the pixel coordinates, and the pixel coordinates are retrieved from the zero point to the maximum value. Represents the intensity of the RGB channels of the stripe pattern at the pixel coordinates (u, v);

[0096] Step 2.2: Convert the two projected N-step colored sinusoidal stripe patterns from the RGB channel mode to the HSV channel mode. The conversion formula for the hue angle H is:

[0097] 、

[0098] .

[0099] Where H(u,v) represents the hue angle of each pixel coordinate (u,v), and θ(u,v) represents the hue angle of each pixel coordinate (u,v) when the phase is (0,2π).

[0100] Step 3. Extract the red area and blue area from the HSV format image according to the hue value, extract the channels corresponding to the red area and blue area from the RGB format projection image, and perform intensity normalization on the extracted red area channel and blue area channel parts. Since the red area and the blue area correspond to different positions in the image, they are merged and filled into a single-channel image after intensity normalization. A set of images has N images. Since the two sets of projected images have different frequencies corresponding to each set of images, the phase shift formula is used to calculate the wrapped phase of the single-channel image of the two sets of colored sinusoidal stripe patterns with different frequencies, and the wrapped phase of the two sets of colored sinusoidal stripe patterns with different frequencies is expanded for the first time.

[0101] Specifically, step 3 includes the following sub-steps:

[0102] Step 3.1: Extract the red and blue areas from the HSV projection image based on the hue value. Extract the corresponding channels from the RGB projection image based on the extracted red and blue areas. Normalize the intensity of the extracted regional channels and merge them into a single-channel image. The normalization formula is as follows:

[0103] .

[0104] in, is the stripe intensity information of the pixel coordinate (u, v) after normalization in the red and blue areas, l is the lower limit of intensity, l=0, h is the upper limit of intensity, h=255.

[0105] Step 3.2: Use the following phase shift formula to calculate the wrapped phase of the two groups of different frequency fringe patterns in the red area and the blue area respectively.

[0106] For the red area:

[0107] ,

[0108] For the blue area:

[0109] .

[0110] Among them, φ r (u,v) and φ b (u,v) refers to the phase of the stripe pattern wrapping of the pixel coordinate (u,v) in the red area and the blue area.

[0111] Step 3.3: Perform the first expansion of the wrapped phases of the two sets of fringe patterns with different frequencies by using the multi-frequency heterodyne method. The multi-frequency heterodyne calculation formula is as follows:

[0112] 、

[0113] .

[0114] Among them, λ b is the frequency corresponding to the heterodyne phase Δφ, λ1 and λ2 are the phase functions corresponding to two sets of fringe patterns with different frequencies, Δφ is the heterodyne phase, that is, the difference between the two sets of fringe patterns with different frequencies, φ1 is the wrapping phase of the first set of fringe patterns, and φ2 is the wrapping phase of the second set of fringe patterns;

[0115] Since the projected images are two sets of fringe patterns with different frequencies, each set has N images, the above formula is used to calculate the wrapped phase of the red and blue areas of the two sets of fringe patterns with different frequencies to obtain φ r1 、φ r2 、φ b1 、φ b2 , where φ r1 Wrap the phase of the red area of ​​the first set of images, φ r2 Wrap the phase of the red area of ​​the second set of images, φ b1 is the blue area wrapped phase of the first set of images, φ b2 Wrap the phase for the blue areas of the second set of images.

[0116] The processed single-channel image used is only expanded for the first set of frequencies. Due to the nature of multi-frequency heterodyning, only the first set of images needs to be expanded, and the absolute phase map can be obtained later. The following formula is used to expand the wrapped phase φ1 solved for the first set of images:

[0117] .

[0118] Wherein, Round(x) represents taking the integer closest to x, φ is the phase after the first expansion of φ1, and O1 is the phase order.

[0119] Step 4: Extract the hue channel from the colored sinusoidal stripe pattern of the HSV format image in step 2, and perform Gray code decoding to obtain the Gray code decoding order.

[0120] Gray code decoding includes the following sub-steps:

[0121] Step 4.1: Extract the red area from the HSV stripe pattern using the hue channel. Use the following formula to assign the red area a value of 1 and the rest of the area a value of 0.

[0122] .

[0123] Among them GC i (u,v) represents the Gray code encoding of the pixel coordinates (u,v) obtained from the hue channel of the HSV format image, and H(u,v) is the hue angle of the pixel coordinates (u,v) in the HSV format image.

[0124] Step 4.2: Convert the extracted red area Gray code into decimal code. The conversion formula is as follows:

[0125] 、

[0126] .

[0127] Among them, B i is the i-th bit of the binary code, GC i is the i-th bit of the Gray code encoding diagram obtained in step 4.1, ⊕ represents the exclusive OR operation, k is the decimal number of the Gray code order, and m is the number of bits of the Gray code binary code.

[0128] Step 5: Use the Gray code decoding order obtained in step 4 to assist in the second expansion of the phase diagram expanded for the first time in step 3, and obtain the final absolute phase through error compensation.

[0129] Specifically, step 5 includes the following sub-steps:

[0130] Step 5.1: Use the Gray code decoding order obtained in step 4 to assist in unwrapping the heterodyne phase obtained in step 3. The phase unwrapping formula is as follows:

[0131] .

[0132] Among them, Φ erros(u, v) represents the absolute phase with the periodic error of the pixel coordinates (u, v), φ(u, v) represents the first phase unwrapped image of the pixel coordinates (u, v) obtained in step 3, and k(u, v) represents the decimal number of the Gray code order of the pixel coordinates (u, v) obtained by Gray code decoding in step 4.

[0133] Step 5.2: Perform median filtering on the absolute phase with periodic error to obtain a phase correction template. The median filtering formula is as follows:

[0134] .

[0135] Among them, medfilt[] a×b represents the median filter operator, a and b represent the length and width of the filter kernel respectively, Φ template Represents Φ after median filtering erros .

[0136] Step 5.3: Use the correction template to assist in wrapping the phase and perform the second phase unwrapping to compensate for the error.

[0137] .

[0138] Among them, Φ template represents the absolute phase after median filtering, Round(x) represents the integer closest to x, Φ(u,v) represents the error-free absolute phase, and φ(u,v) is the first phase unwrapped diagram obtained in step 3.

[0139] The above-mentioned color Gray code encoding assisted phase unwrapping method of the present invention is used for a three-dimensional imaging method, especially for three-dimensional imaging using a fringe projection profilometry method. During the imaging process, a color Gray code encoding assisted phase unwrapping method is used to perform wrapped phase unwrapping of the projected fringe pattern. By embedding the Gray code encoding into different channels of two groups of patterns, the number of required projection patterns is significantly reduced, effectively improving the measurement efficiency of three-dimensional imaging.

[0140] The present invention also discloses a three-dimensional imaging system, including a visible light projection module, an image acquisition module, and an image processing computer. The visible light projection module generates two sets of N-step colored sinusoidal fringe patterns and performs fringe projection on the outline of the scene to be measured. The device uses a DLP6500 digital visible light projector as the device for projecting coded structured light. The image acquisition module captures the two sets of N-step colored sinusoidal fringe patterns after the scene to be measured is projected and transmits them to the image processing computer. The device uses a Hikvision MV-CS050-10UM industrial camera with a Hikvision FA1202D lens. The image processing computer includes one or more processors and a memory storing one or more computer programs. The processor receives image data from the image acquisition module and calls the computer program in the memory to implement the three-dimensional imaging method described above in this embodiment.

[0141] The color Gray code assisted phase unwrapping method of the present invention is further described in detail below with reference to the specific embodiments in the accompanying drawings.

[0142] In order to better illustrate the method of the present invention, this embodiment uses two sets of frequencies as This paper presents a three-step phase shift and color Gray code scheme, and illustrates the measurement of complex scenes such as portraits and statues. Traditional phase unwrapping algorithms using three-frequency, three-step phase shift encoding require nine images for global encoding. Using phase shift plus Gray code also requires at least three phase shift stripes and six Gray code stripes. This invention aims to reduce projected patterns and achieve high-precision phase unwrapping.

[0143] The specific steps are as follows:

[0144] Construct two sets of colored sinusoidal stripe patterns.

[0145] In order to separate different color regions by hue channel, we encode the sinusoidal stripes as:

[0146] .

[0147] In order to minimize the crosstalk between stripes of different colors, two color channels with large hue differences in the HIS color space are selected for encoding, namely, two sets of color Gray code-sine fringe images constructed with red and blue.

[0148] Two sets of colored sinusoidal fringe patterns are projected onto the portrait statue scene and captured, and the RGB format images are converted into HSV format images.

[0149] The projector projects two sets of colored sinusoidal stripe patterns with different frequencies into the scene to be tested, and the industrial camera synchronously captures the colored deformed stripes as shown in the following figure. Figures 3a-3fAs shown, the captured color pattern can be expressed in the RGB mode as follows:

[0150] .

[0151] in, Represents the intensity of the RGB channels of the color stripe pattern at the pixel coordinates (u, v). The color stripes are then decoded using an HSV-based color recognition algorithm. First, the color stripes are converted from RGB channel mode to HSV channel mode. The hue angle H is obtained using the following formula.

[0152] 、

[0153] .

[0154] Where H(u,v) represents the hue angle of each pixel, and cos represents the cosine function.

[0155] According to the hue value, the red hue value area and the blue hue value area are extracted from the projection image in HSV format respectively. The red channel and blue channel of the corresponding area are extracted from the projection image in RGB format using the extracted red hue area and blue hue area. After intensity normalization, they are filled into the single-channel image. The phase shift formula is used to calculate the wrapped phases of the two groups of different frequencies. Then, the heterodyne phases of the two groups of frequencies are calculated by the multi-frequency heterodyne method, and the wrapped phases of the two groups of different frequency stripe patterns are unfolded for the first time.

[0156] First, extract the red and blue areas, namely redmask(u,v) and bluemask(u,v), according to the hue value. The formula is as follows:

[0157] 、

[0158] .

[0159] After that, the channel I corresponding to redmask(u,v) and bluemask(u,v) is extracted from the RGB format stripes ri (u,v),I bi (u,v), the formula is as follows:

[0160] .

[0161] Afterwards, the intensity normalization is performed on them respectively, and the formula is as follows:

[0162] .

[0163] is the stripe intensity information of the pixel coordinate (u, v) after normalization in the red and blue areas, l is the lower limit of intensity, l=0, h is the upper limit of intensity, h=255.

[0164] Then, the two normalized images are merged into a single-channel image I i In (u,v), the formula is as follows:

[0165] .

[0166] Furthermore, the phase shift method is used to calculate the fringe pattern wrapping phase φ of the two frequency groups in the red area and the blue area. r (u,v),φ b (u, v), since each set of images contains N images, the wrapped phase is calculated for the stripe patterns extracted from the red area and the blue area respectively. The specific formula is as follows:

[0167] For the red area:

[0168] .

[0169] For the blue area:

[0170] .

[0171] where φ r (u,v) and φ b (u, v) refers to the wrapped phase of the pixel coordinates (u, v) of the stripe patterns in the red and blue regions of the two sets of images, and N is the value selected in the above N-step phase shift.

[0172] Finally, the first phase unwrapping of the fringes at one of the frequencies is performed according to the multi-frequency heterodyne method. The formula is as follows:

[0173] 、

[0174] 、

[0175] .

[0176] Among them, λ b is the frequency corresponding to the heterodyne phase Δφ, λ1 and λ2 are the phase functions corresponding to two sets of fringe patterns with different frequencies, Δφ is the heterodyne phase, that is, the difference between the two sets of fringe patterns with different frequencies, φ1 is the wrapped phase of the first set of fringe patterns, φ2 is the wrapped phase of the second set of fringe patterns, Round(x) means taking the integer closest to x, φ is the phase after the first expansion of φ1, and O1 is the phase order. The main phase values ​​of the two frequency fringe patterns are as follows: Figure 5a and 5bAs shown, the phase expansion diagram of the first phase expansion is as follows Figure 4 shown.

[0177] Extracts the hue channel from the stripe pattern in HSV format and decodes it into Gray code.

[0178] First, use the red mask to convert the Gray code into binary code, extract the red area from the HSV format stripe pattern through the hue channel, and assign the red area to 1 and the rest to 0 using the following formula;

[0179] .

[0180] Among them GC i (u, v) represents the Gray code image obtained by the hue channel of the HSV format image, and H(u, v) is the hue angle in the HSV format image.

[0181] Then convert the extracted red area Gray code into decimal code. The conversion formula is as follows:

[0182] 、

[0183] .

[0184] Among them, B i is the i-th bit of the binary code, GC i is the i-th bit of the obtained Gray code encoding diagram, ⊕ represents the exclusive OR operation, k is the decimal number of the Gray code order, that is, the order information for subsequent phase unwrapping, and m is the number of bits of the Gray code binary code.

[0185] The obtained Gray code decoding order is used to assist the second unfolding of the phase diagram after the first unfolding, and the final absolute phase is obtained through error compensation.

[0186] The following formula is used to use the obtained Gray code decoding order to assist in unfolding the first phase unwrapped image and calculate the absolute phase image:

[0187] .

[0188] Among them, Φ erros (u, v) represents the absolute phase of the pixel coordinate (u, v) with periodic error, φ(u, v) represents the first phase unwrapped image of the pixel coordinate (u, v) obtained, and k(u, v) represents the Gray code decoding order of the pixel coordinate (u, v) obtained by the Gray code decoding process above.

[0189] The absolute phase with periodic error is processed by median filtering to obtain a phase correction template.

[0190] The error is compensated by using the following formula to perform the second phase unwrapping using the correction template to assist in wrapping the phase:

[0191] .

[0192] Among them, Φ template represents the absolute phase after median filtering, Round(x) represents the integer closest to x, Φ(u,v) represents the error-free absolute phase of the pixel coordinates (u,v), and φ(u,v) is the first phase unwrapped image of the pixel coordinates (u,v).

[0193] The final error-free absolute phase diagram is as follows: Figure 7 shown.

[0194] In this document, the directions or positional relationships indicated by terms such as "up", "down", "front", "back", "left", "right", "top", "bottom", "inside", "outside", "vertical", and "horizontal" are based on the directions or positional relationships shown in the accompanying drawings and are only for the clarity of the technical solution and the convenience of description, and therefore should not be understood as limiting the present invention.

[0195] As used herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion of elements other than the listed elements and may also include additional elements not specifically listed.

[0196] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A color Gray code assisted phase unwrapping method, characterized by: The steps include: Step 1: Construct two sets of colored sinusoidal fringe patterns with different frequencies to perform fringe projection on the outline of the scene to be measured; Step 2: Capture two sets of colored sinusoidal fringe patterns after projecting the scene to be tested, and convert the projected images from RGB format images to HSV format images; Step 3: Extract the red and blue areas from the HSV image based on the hue value, extract the channels corresponding to the red and blue areas from the RGB projection image, perform intensity normalization on the extracted red and blue area channels, merge and fill them into a single-channel image after intensity normalization, calculate the wrapped phases of the single-channel images of the two sets of colored sinusoidal fringe patterns with different frequencies using the phase shift formula, and perform the first unfolding of the wrapped phases of the two sets of colored sinusoidal fringe patterns with different frequencies; Step 4: Extract the hue channel from the colored sinusoidal stripe pattern of the HSV format image in step 2 and perform Gray code decoding to obtain the Gray code decoding order. Step 5: Use the Gray code decoding order obtained in step 4 to assist in the second unfolding of the phase diagram obtained in step 3, and obtain the final absolute phase through error compensation.

2. The color Gray code assisted phase unwrapping method according to claim 1, characterized in that: The process of constructing two sets of colored sinusoidal stripe patterns of different frequencies in step 1 is as follows: Define an N-step phase-shifted fringe pattern: ， in, represents the grayscale value of the phase-shifted fringe pattern at the pixel coordinate (u, v), where n is a positive integer representing the number of phase shift steps, a(u, v) and b(u, v) represent the background light intensity and modulation amplitude of the fringe pattern at the pixel coordinate (u, v), respectively, and f0 is the fringe frequency. a(u, v) and b(u, v) are encoded as follows: a(u, v) = 0.6, b(u, v) = 0.4; There are two groups of phase-shifted fringe patterns, each group contains N images, which are N-step phase-shifted images, totaling 2N images, corresponding to the encoded 2N-bit Gray code image.

3. The color Gray code assisted phase unwrapping method according to claim 2, characterized in that: The two sets of phase-shifted stripe patterns are encoded by Gray code through the three RGB channels, and the red and blue color channels are selected as the coding colors to construct a colored sinusoidal stripe pattern.

4. The color Gray code assisted phase unwrapping method according to claim 1, characterized in that: The step 2 includes the following sub-steps: Step 2.1: The projector projects two sets of colored sinusoidal fringe patterns of different frequencies onto the scene to be tested. The industrial camera simultaneously captures the two sets of projected colored sinusoidal fringe patterns. The RGB mode of the captured projected fringe pattern is expressed as: ， Where c represents the pattern captured by the camera, N represents the number of phase-shifted patterns, which is equal to the number of phase-shifted steps of the fringe pattern, and (u, v) represents the pixel coordinates. Represents the intensity of the RGB channels of the stripe pattern at the pixel coordinates (u, v); Step 2.2: Convert the two projected colored sinusoidal stripe patterns from the RGB channel mode to the HSV channel mode. The conversion formula for the hue angle H is: 、 ， Where H(u,v) represents the hue angle of the pixel coordinate (u,v), and θ(u,v) represents the hue angle of the pixel coordinate (u,v) when the phase is (0,2π).

5. The color Gray code assisted phase unwrapping method according to claim 4, characterized in that: The step 3 includes the following sub-steps: Step 3.1: Extract the red and blue areas from the HSV projection image based on the hue value. Extract the corresponding channels from the RGB projection image based on the extracted red and blue areas. Normalize the intensity of the extracted regional channels and merge them into a single-channel image. The normalization formula is as follows: ， in, is the fringe intensity information of the fringe pattern after normalization of the red and blue areas at the pixel coordinates (u, v), l is the lower limit of intensity, and h is the upper limit of intensity; Step 3.2: Use the following phase shift formula to calculate the wrapped phase of the red and blue regions in two sets of colored sinusoidal fringe patterns with different frequencies: For the red area: ， For the blue area: ， where φ r (u,v) and φ b (u, v) refers to the phase of the stripe pattern wrapping of the pixel coordinate (u, v) in the red area and the blue area; Step 3.3: Perform the first expansion of the wrapped phases of the two sets of fringe patterns with different frequencies by using the multi-frequency heterodyne method. The multi-frequency heterodyne calculation formula is as follows: 、 ， Among them, λ b is the frequency corresponding to the heterodyne phase Δφ, λ1 and λ2 are the phase functions corresponding to two sets of fringe patterns with different frequencies, Δφ is the heterodyne phase, that is, the difference between the two sets of fringe patterns with different frequencies, φ1 is the wrapping phase of the first set of fringe patterns, and φ2 is the wrapping phase of the second set of fringe patterns; The wrapped phase of one set of fringe patterns is selected and unfolded once to obtain a first unfolded phase image.

6. The color Gray code assisted phase unwrapping method according to claim 1, characterized in that: Gray code decoding in step 4 includes the following sub-steps: Step 4.1: Extract the red area from the HSV stripe pattern using the hue channel. Use the following formula to assign the red area a value of 1 and the rest of the area a value of 0. ， Among them GC i (u,v) represents the Gray code image obtained by the pixel coordinates (u,v) in the hue channel of the HSV format image, and H(u,v) is the hue angle of the pixel coordinates (u,v) in the HSV format image; Step 4.2: Convert the extracted red area Gray code into decimal code. The conversion formula is as follows: 、 ， Among them, B i is the i-th bit of the binary code, GC i is the i-th bit of the Gray code encoding diagram obtained in step 4.1, ⊕ represents the exclusive OR operation, k is the decimal number of the Gray code order, and m is the number of bits in the binary code.

7. The color Gray code assisted phase unwrapping method according to claim 1, characterized in that: The step 5 includes the following process: Step 5.

1. Use the Gray code decoding order obtained in step 4 to assist in unfolding the first phase unwrapped image in step 3 and calculate the absolute phase image using the following formula: ， Among them, Φ erros (u, v) represents the absolute phase of the pixel coordinate (u, v) with periodic error, φ(u, v) represents the first phase unwrapped image of the pixel coordinate (u, v) obtained in step 3, and k(u, v) represents the decimal number of the Gray code order of the pixel coordinate (u, v) obtained by Gray code decoding in step 4; Step 5.2: Perform median filtering on the absolute phase with periodic error to obtain a phase correction template; Step 5.3: Use the correction template to assist in wrapping the phase and perform the second phase unwrapping to compensate for the error. ， Among them, Φ template represents the absolute phase after median filtering, Round(x) represents the integer closest to x, Φ(u,v) represents the error-free absolute phase of the pixel coordinates (u,v), and φ(u,v) is the first phase unwrapped image of the pixel coordinates (u,v) obtained in step 3.

8. A three-dimensional imaging method, characterized in that: Three-dimensional imaging is performed using a fringe projection profilometry method, and during the imaging process, the color Gray code assisted phase unwrapping method according to any one of claims 1 to 7 is used to perform wrapped phase unwrapping of the projected fringe pattern.

9. A three-dimensional imaging system, characterized in that: include: The visible light projection module generates two sets of colored sinusoidal fringe patterns with different frequencies and projects fringe patterns onto the outline of the scene to be measured; An image acquisition module captures two sets of colored sinusoidal fringe patterns projected onto the scene to be tested; one or more processors; a memory storing one or more computer programs; The processor receives image data from the image acquisition module and calls the computer program in the memory to implement the three-dimensional imaging method according to claim 8.

Images