Structured light three-dimensional measurement method based on continuous phase constraint

Abstract

The application discloses a kind of structure light three-dimensional measurement method based on continuous phase constraint, including constructing encoding stripe, project the encoding stripe and obtain corresponding image, project periodic stripe pattern with preset phase offset and obtain multiple images, and the phase information of each pixel is calculated by the multiple images;Determine the stripe period interval to which the pixel belongs based on the phase information, and based on the continuity of phase on space, the initial stripe order is corrected consistently, and the three-dimensional appearance of the measured object is recovered according to the corrected stripe order and phase information.The present application encodes by time domain, space domain and color domain, the present method significantly reduces the number of patterns that need to be projected, while ensuring measurement accuracy through color adaptation and phase-assisted decoding, and has stronger robustness for color surfaces.

Description

Technical Field

[0001] This invention relates to the field of optical three-dimensional measurement technology, and in particular to a structured light three-dimensional measurement method based on continuous phase constraint, which is based on the fusion of coded stripes and phase information. Background Technology

[0002] Structured light 3D measurement is a typical active non-contact 3D measurement technology. It projects a pre-designed optically encoded pattern onto the surface of the object being measured, and an imaging device acquires the image information modulated by the object. This image information is then combined with geometric relationships or phase calculations to reconstruct the 3D shape of the object's surface. Compared to contact measurement methods, this technology offers advantages such as non-contact operation, high precision, and wide applicability, and has been widely applied in fields such as industrial inspection, reverse engineering, biomedicine, and cultural relic preservation.

[0003] Chinese patent CN201181204Y discloses a structured light 3D measurement device based on Gray-coded fringes. This device establishes a correspondence between pixels and spatial points of an object by projecting coded fringes and acquiring images, thereby achieving 3D reconstruction. In further improvements, existing technologies typically combine Gray coding with phase-shifting fringes. Specifically, coded fringes determine the fringe order, while phase-shifting fringes acquire continuous phase information, thus achieving high-precision measurement. This combined approach has become the mainstream technology in the field of structured light measurement.

[0004] However, such methods typically require the projection of multiple coded patterns and multiple phase-shifting patterns, resulting in high system complexity and limited measurement efficiency.

[0005] US Patent 11808564B2 discloses a measurement method based on multi-frequency phase-shift fringes. This method obtains the corresponding folded phase by projecting multiple sets of phase-shift fringe patterns at different frequencies, and then recovers the absolute phase using a multi-frequency phase unwrapping algorithm. The typical process involves obtaining the absolute phase after acquiring the folded phase, followed by 3D reconstruction. In practical applications, Gray encoding is easily affected by factors such as changes in the reflectivity of the object's surface, ambient light interference, and blurred fringe boundaries during decoding, leading to encoding errors, especially noticeable in the fringe boundary region. These encoding errors can further cause phase unwrapping errors or amplify the overall error.

[0006] Furthermore, existing technologies typically reduce errors by increasing the number of coded patterns or introducing additional unfolding algorithms, which leads to reduced measurement efficiency. Therefore, how to improve accuracy without significantly increasing the number of projected patterns has become a pressing technical problem in this field. Summary of the Invention

[0007] To address the aforementioned problems in existing technologies, the present invention aims to provide a structured light three-dimensional measurement method based on continuous phase constraints, and establishes a coding result correction mechanism based on phase continuity constraints to achieve the fusion and unification of discrete coding information and continuous phase information.

[0008] To achieve the above-mentioned technical objectives, the technical solution of the present invention is as follows: A structured light three-dimensional measurement method based on continuous phase constraint includes: Construct coded stripes so that the projected pattern forms a unique stripe hierarchy in the time and / or spatial dimensions; The coded stripes are projected and the corresponding images are acquired to determine the initial stripe order of each pixel; Projecting a periodic stripe pattern with a preset phase offset and acquiring multiple images, the phase information of each pixel is calculated from the multiple images; Based on the phase information, the stripe period interval to which the pixel belongs is determined, and the initial stripe level is corrected for consistency based on the spatial continuity of the phase, so as to correct the misjudgment at the stripe boundary and obtain the position of continuous stripes. The three-dimensional shape of the measured object is recovered based on the corrected fringe order and phase information.

[0009] In one specific implementation, the encoding results of the boundary region are corrected based on the encoding results of adjacent pixels within the same period.

[0010] In one specific implementation, the encoding result is divided into connected components, and the fringe period of each connected component is determined by combining phase information. Connected components that are inconsistent with the phase region are encoded and corrected.

[0011] In one specific implementation, the phase information is calculated from at least three periodic stripe images with different phase offsets.

[0012] In one specific implementation, the color channel of the projected stripes is adaptively selected based on the color characteristics of the surface of the object being measured, in order to improve the stripe contrast.

[0013] In one specific implementation, the response intensity of each pixel in different color channels is obtained based on the acquired reference image, and the channel with the highest response intensity is selected as the projection color.

[0014] The advantages and beneficial effects of this invention are as follows: This invention proposes a spatiotemporally encoded structured light measurement method for color surfaces, combining temporal binary encoding with spatial de Bruin sequences: Multiple simple fringes are generated using binary encoding in the temporal domain, and adjacent fringe windows are uniquely identified using the spatial de Bruin sequences, thus achieving pixel-level unique mapping while reducing the number of projection patterns. During projection, phase-shifted fringes and spatiotemporally encoded patterns are projected time-divisionally. In the decoding stage, the phase information obtained from the phase shift is used to correct misalignments of the binary-coded fringes at the fringe boundaries, and this information is then used for 3D reconstruction. Through joint encoding in the temporal, spatial, and color domains, this method significantly reduces the number of patterns that need to be projected, while ensuring measurement accuracy through color adaptation and phase-assisted decoding, exhibiting stronger robustness for color surfaces. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the stripe hierarchy in the encoded sequence. Figure 2 The three generated coded patterns.

[0016] Figure 3 The diagram shows the coordinate mapping between the projector and the camera, where (a) is a schematic diagram of the coordinate system and (b) is a dot matrix diagram.

[0017] Figure 4 For phase shift coding stripes, Figure 5 Encode stripes for the sequence.

[0018] Figure 6 The color distribution.

[0019] Figure 7 For adaptive spatiotemporal coded stripes.

[0020] Figure 8 This refers to the connected components before correction.

[0021] Figure 9 This is the corrected connected component.

[0022] Figure 10 To correct the previous boundary distribution.

[0023] Figure 11 This is for the corrected boundary distribution.

[0024] Figure 12 The plaster cast image to be tested.

[0025] Figure 13 Seven adaptive color stripe images were captured for the camera.

[0026] Figure 14 This is the result of the 3D reconstruction.

[0027] Figure 15 This is a magnified view of a portion of the corresponding 3D reconstruction result.

[0028] For those skilled in the art, other related figures can be obtained from the above figures without any creative effort. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0030] The structured light three-dimensional measurement method based on continuous phase constraint of the present invention includes: First, coded stripes are constructed to create unique stripe levels in the projected pattern across the time and / or spatial dimensions. Constructing coded stripes involves encoding the stripes in the time and / or spatial dimensions, ensuring that different stripe periods correspond to unique identifiers, thereby uniquely determining the stripe level. Encoding methods can include time-domain binary encoding, spatial-domain sequence encoding, etc. Preferably, multiple coded patterns are projected in the time domain to form a code element combination, which is then arranged in the spatial domain to achieve unique identification of the stripe level.

[0031] Specifically, as one embodiment, a spatiotemporal coding method based on de Bruin sequences can be used. To ensure the discriminability of the encoding and the robustness of the decoding, this embodiment imposes constraints on the code element arrangement rules during the sequence generation stage to avoid adjacent code elements taking the same value. Due to the light attenuation during actual measurement, ambiguity in code value judgment can easily occur during decoding. Therefore, to improve the flexibility of sequence construction while reducing code value judgment ambiguity, this invention combines binary coding, which has the characteristics of simple grayscale levels and clear boundaries, in the encoding process. De Bruin sequence code elements are generated through binary coding in the time domain. For a two-grayscale image, after binarization, two code values, 0 and 1, can be generated. Projected in time sequence... A two-dimensional grayscale image can be formed Different combinations of code values ​​correspond to different code elements. The correspondence between code elements and binary code values ​​is shown in the following formula: (1) In the formula: For code elements; No. Zhang's image binarization result; The number of projected binary patterns.

[0032] After obtaining the required code elements, the encoded information is extended from the time domain to the spatial domain by introducing a de Brouin sequence. The uniqueness of the de Brouin sequence is used to determine the stripe order, thereby reducing the number of encoded patterns required in the time domain. Since this embodiment introduces a de Brouin sequence in the spatial domain for stripe encoding, in isolated object measurement scenarios, if the subsequence length... If the value is too large, it can easily introduce ambiguity in subsequence determination within local regions, thus interfering with stripe level recognition. Therefore, while ensuring the sequence length... Capable of covering the entire stripe period Under the premise, The value of should be as small as possible. For example, when selecting =144 =3、 =6. Number of projected binary patterns When =3, select code element =0, 1, 2, 3, 4, 5, a de Bruin sequence of length 150 can be constructed. Extracting the first 144 symbols of this sequence yields the de Bruin sequence corresponding to the number of fringe periods: 012013014… In the spatial domain, the symbols are multiplexed according to the sequence, resulting in the coded sequence as follows… Figure 1 and 2 As shown.

[0033] Then, the coded stripes are projected and the corresponding image is obtained. The image is then decoded to determine the initial stripe level of each pixel. The decoding can be achieved through binarization, symbol recognition and sequence matching to achieve coarse localization of each pixel, i.e., to obtain the stripe number.

[0034] Projecting periodic stripe patterns (typically sinusoidal stripes) with different phase shifts, and by acquiring multiple images, such as 3-4 images, calculating the phase information of each pixel based on the intensity changes of each pixel, achieves a much higher accuracy than encoding methods, even reaching sub-pixel level. Furthermore, this phase information is converted into... The periodically continuous folded phase information can be used to characterize the relative position of pixels within the stripe period.

[0035] Next, consistency correction is performed on the initial stripe levels based on the phase information. This includes dividing the stripe periodic region according to the phase, determining whether the encoding result is consistent with the phase continuity, correcting inconsistent regions, especially boundary regions, and obtaining continuous stripe positions. This eliminates stripe boundary misjudgments, improves stripe level accuracy, reduces the encoding method's sensitivity to noise, and improves decoding accuracy without increasing the number of encoded frames. Furthermore, spatial continuity constraints are established through phase information to perform error detection and adaptive correction on the encoding results. In the process of using phase information to perform consistency correction on the encoded stripe levels, since phase information is a periodic continuous quantity that can characterize the relative position of pixels within the stripe period, the stripe period interval to which the pixel belongs is first determined based on the phase value, thereby establishing the correlation between phase information and the encoded stripe levels. Based on this, the consistency judgment of the encoding results is performed using the continuous spatial variation characteristics of the phase. When the encoding result is inconsistent with the phase continuity, it is determined as an encoding error, and the stripe levels in the corresponding regions are corrected, thereby achieving error correction and optimization of the encoding results.

[0036] In actual projection imaging, interference from factors such as camera and projector defocusing and noise makes it difficult for the boundary position obtained after binarization to remain consistent with the folded phase boundary. This invention corrects the binary boundary using a folded phase-assisted method. Based on the folded phase, the region where a pixel is located within one fringe period is divided into a central region and two types of boundary regions: (2) In the formula: The region where the current pixel is located; For the central area; This is the left boundary area; This is the right boundary region. Because boundary misalignment only occurs at the boundary (i.e.,...) , (Within) and usually no more than 1 / 4 of a cycle, the error value at the boundary can be corrected based on the central area code value within the same cycle, thereby correcting the boundary misalignment phenomenon.

[0037] Connectivity detection is performed on the binary symbol image based on the code value. An 8-neighborhood criterion is used to construct connected components for pixels with the same code value, and these are further divided into several sub-connected components by combining the folded phase region. In the absence of boundary misalignment, only one type of connected component should exist within the same boundary region, and the code value of this connected component should be consistent with the adjacent phase center region. Therefore, boundary misalignment can be determined by judging whether the segmented sub-connected component is adjacent to a phase center region with the same code value: if the sub-connected component is adjacent to a center region with the same code value, its code value is considered correct; otherwise, it is considered to have a boundary misalignment. For sub-connected components judged to be misaligned, the code value within the connected component is corrected to the correct code value within its corresponding phase region. This method effectively corrects connected components near the folded phase boundary, such as... Figure 8 As shown in the enlarged view, the right boundary region The presence of multiple connected regions with the same code value, and the fact that the dashed area is not adjacent to the central region with the same code value, indicates a boundary misalignment, requiring correction of the code values ​​within this connected region. Correction is performed based on the correct code value within the same phase region, as shown below. Figure 9 As shown. A profile curve is plotted by selecting one row of binarized and folded phase boundary distribution data. The boundary distribution before and after correction is shown below. Figure 10 , 11 As shown in the figure, the correction eliminated the boundary misalignment, providing a precise data foundation for the subsequent decoding process.

[0038] Finally, the absolute phase is recovered based on the corrected fringe order and phase information, and the three-dimensional coordinates are calculated through the system calibration relationship, thus recovering the three-dimensional shape of the measured object.

[0039] Specifically, in sequence decoding, the symbols are first calculated pixel by pixel, and the 8-neighborhood criterion is used to construct connected components for pixels with the same symbol. Let the set of all connected components be . ={ , , ,… In the sequence decoding process, the forward search direction is defined as from left to right. For each connected component, starting from the current connected component, three adjacent and consecutive connected components are selected sequentially along the forward search direction. The short sequence formed , short sequence In the reference sequence In the matching process, find its location. This position corresponds to the current connected component. stripe levels .

[0040] For connected components located at the ends Since the two connected components cannot form a complete sequence in the forward search, only the following can be determined. The position within the reference sequence. Therefore, this invention employs a bidirectional search strategy to decode the remaining connected components. (Using connected components...) For example, the reference sequence Reverse the order to obtain the reverse sequence And construct the corresponding terminal connected components along the reverse search direction. The short sequence formed r ',exist Find its location in Thus, the connected components are obtained. The corresponding stripe order. Combined with the total length of the resulting sequence. The reverse decoding is shown in the following formula: (3) In the formula: The desired stripe level. In both forward and reverse sequence matching, this invention employs a hash lookup method for the reference sequence. and A hash index table is established to achieve sequence lookup with constant time complexity, which is O(1).

[0041] By decoding the proposed spatiotemporal encoded fringes through the above process, the fringe order corresponding to the sinusoidal fringes can be obtained. The phase is then expanded by solving the folded phase obtained using phase-shifted fringes. Finally, the three-dimensional data of the object surface is calculated using the mapping relationship between the phase and the three-dimensional coordinates to complete the three-dimensional reconstruction.

[0042] The structured light 3D measurement method based on continuous phase constraint of the present invention achieves unique identification of fringe order through spatiotemporal encoded fringes, realizing global positioning capability of 3D measurement; it obtains phase information using phase-shifted fringes; and it uses phase information to perform consistency correction on the encoding results, effectively correcting the misjudgment problem at the fringe boundary; it improves the accuracy of fringe order determination without increasing the number of encoded patterns, thus balancing measurement accuracy and efficiency; and it enhances the robustness of 3D reconstruction in the presence of boundary ambiguity and noise.

[0043] Furthermore, to address the issue of decreased measurement accuracy caused by differences in surface color reflectivity, this invention also includes adaptively selecting the color channel of the projected stripes based on the color characteristics of the object being measured, thereby improving stripe contrast. Specifically, the response intensity of each pixel in different color channels is obtained based on the acquired reference image, and the channel with the highest response intensity is selected as the projected color.

[0044] Adaptive color channel selection: During initial illumination testing of the object under test, the red, green, or blue channel is adaptively selected as the encoding channel based on the reflectivity of each pixel. This reduces measurement errors caused by differences in stripe contrast due to the selective absorption of colors by the object's surface.

[0045] As one specific embodiment, the present invention determines the optimal color channel by pre-analyzing the color feature distribution of the surface of the object being tested, thereby achieving an adaptive balance of surface reflection.

[0046] To obtain the color feature distribution of the surface of the object under test, this invention projects a pure white image onto the surface of the object and simultaneously acquires the corresponding white light response image. Since the pure white image projection has the same intensity distribution in each color channel, the acquired white light image can reflect the reflection characteristics of the surface under test to different wavelengths of light to a large extent. By analyzing the intensity response of each color channel in the white light response image, the reflectivity differences of different surface regions to the red, green, and blue channels can be obtained, providing a basis for the selection of adaptive projection color channels.

[0047] In captured images, there are often some low-brightness and background areas. The light intensity in these areas is mainly due to camera noise or environmental factors, resulting in unstable light intensity information that is difficult to reflect true color characteristics. Therefore, these areas are unsuitable for projection color discrimination. This invention defines these areas as invalid regions and generates corresponding image masks. Invalid areas are removed before adaptive projection color selection. The formula for generating the image mask is: (4) In the formula: The maximum intensity of the three channels in the figure; This represents the average intensity of the three channels of the current pixel. This represents the threshold coefficient for the dark area. After processing the original image, an effective region containing only the color information of the object's surface is obtained. Each pixel within the effective region is analyzed pixel-by-pixel, and its three-channel response values ​​are read. Since a higher channel response value corresponds to a stronger reflection signal at the same pixel location, resulting in higher contrast, it is beneficial for the stable acquisition of stripe information. Therefore, this invention selects the channel response value as the criterion, choosing the channel with the strongest response as the projection color channel of that pixel, as shown in the following formula: (5) In the formula: This is the result of adaptive projection color selection. By judging the color response of pixels in the effective area according to the above formula, adaptive selection of the projected color channel of the measured surface can be achieved.

[0048] Since the projected pattern and the image captured by the camera reside in two different pixel coordinate systems, a mapping relationship between the camera and projector pixel coordinates needs to be established to ensure that the adaptive projection color pattern can be accurately projected onto each color region of the object being measured. This is achieved by projecting a known dot matrix pattern of markers located in the projector's pixel coordinate system onto the object and capturing the image. Interpolation can then be performed using the coordinate correspondence between the markers in the horizontal and vertical directions to establish a mapping relationship between any point in the projector's projection pixel plane and the corresponding point in the camera's pixel plane, thus realizing coordinate transformation between the two. Figure 3 As shown.

[0049] The center of the marker points in the dot matrix image is extracted, and a coordinate mapping model between the two coordinate systems is constructed based on the coordinate correspondence of the marker point centers. Several marker points satisfying the non-collinearity condition are selected within the neighborhood of the measured point. Two-dimensional linear interpolation is performed on the pixel coordinates of the measured point in the horizontal and vertical directions to obtain the coordinate mapping of the measured pixel in the projector pixel coordinate system, as shown in the following formula: (6) In the formula: , , These are the pixel coordinates of the top left, top right, and bottom left corners of the measured point in the camera coordinate system, respectively. , , These are the pixel coordinates of the top left, top right, and bottom left corners of the measured point in the projector coordinate system. Based on the above mapping model, by mapping the coordinates of all pixels within the effective area, the complete pixel color distribution of the measured area in the projector pixel coordinate system can be obtained.

[0050] An adaptive color spatiotemporal coded pattern can be obtained by modulating the generated spatiotemporal coded stripe pattern based on the adaptive color distribution after coordinate transformation, such as... Figure 4-7 As shown. Phase shift fringes. - With spatiotemporal coding stripes - Adaptive spatiotemporal coded stripes are generated after modulation according to adaptive color distribution. - Furthermore, since the fringe projection measurement method has certain requirements on the scattering characteristics of the measured surface, and blue light has a shorter wavelength and can provide better scattering characteristics compared to other monochromatic lights, this invention defaults to encoding the stripes in the blue channel in areas where coordinate mapping has not been performed.

[0051] Because color difference is unavoidable in color stripe projection measurement systems, it will cause a decrease in measurement accuracy. Therefore, color difference correction is first performed after image acquisition, and color difference compensation is achieved by establishing a pixel deviation model. Then, the intensity information is converted into code values ​​through grayscale binarization processing, the binary boundary misalignment phenomenon is corrected and the code elements are determined, and finally, subsequence extraction is performed to determine the stripe level.

[0052] Since the intensity information of the image captured by the camera is distributed across three color channels, the information from each channel of the captured image needs to be integrated before decoding. The calculation method is shown in the following formula: (7) In the formula: For intensity-integrated pixels Strength value at location; It is a three-channel index; After intensity integration, the pixel grayscale values ​​of the sequence-coded image exhibit obvious brightness distribution characteristics, which can be approximately divided into two categories: "highlight" and "lowlight". Therefore, image binarization can be completed through threshold decision. Regarding the selection of the decision threshold, the average intensity of the four phase-shifting fringe patterns yields the background light intensity, which falls within the binary grayscale range and is suitable as a threshold. Therefore, this invention selects the phase-shifting fringe background light intensity as the decision threshold, and the binarization process can be expressed as: (8) In the formula: For the first Pixels in a spatiotemporally encoded image The binarization result at the location; For the first Pixels in a spatiotemporally encoded image Strength at the location; =1, 2, 3.

[0053] To verify the effectiveness of the method proposed in this invention, an experimental system was built consisting of a Light Crafter4500 digital projector with a resolution of 1280 pixels × 800 pixels, an SVCam-ECO655 camera with a resolution of 2448 pixels × 2050 pixels, and a moving stage with an accuracy of 1 μm.

[0054] To demonstrate the effectiveness of the proposed method when dealing with complex colored objects, a three-dimensional reconstruction of a colored Hello Kitty plaster statue was performed, such as... Figure 12-15 As shown. The plaster cast to be tested is as follows. Figure 12 As shown, a pure white image and a dot matrix image are projected onto the object under test to generate an adaptive color spatiotemporal coded image, which is then projected. The camera acquires seven adaptive color stripe images, as shown below. Figure 13 As shown. Figure 14 For the 3D reconstruction results, Figure 15 The image shown is a magnified view of a portion of the 3D reconstruction result. As can be seen from the magnified view, the method of this invention can achieve 3D reconstruction of a colored Hello Kitty plaster statue, while preserving local text details. This indicates that the method of this invention has good reconstruction capabilities for object details even with fewer projected patterns, maintains high measurement accuracy, and exhibits strong robustness in complex color measurement environments.

[0055] The present invention has been described above by way of example. It should be noted that any simple modifications, alterations or other equivalent substitutions that can be made by those skilled in the art without creative effort without departing from the core of the present invention fall within the protection scope of the present invention.

Claims

1. A structured light three-dimensional measurement method based on continuous phase constraint, characterized in that, include: Construct coded stripes so that the projected pattern forms a unique stripe hierarchy in the time and / or spatial dimensions; The coded stripes are projected and the corresponding images are acquired to determine the initial stripe order of each pixel; Projecting a periodic stripe pattern with a preset phase offset and acquiring multiple images, the phase information of each pixel is calculated from the multiple images; Based on the phase information, the stripe period interval to which the pixel belongs is determined, and the initial stripe level is corrected for consistency based on the spatial continuity of the phase, so as to correct the misjudgment at the stripe boundary and obtain the position of continuous stripes. The three-dimensional shape of the measured object is recovered based on the corrected fringe order and phase information.

2. The structured light three-dimensional measurement method based on continuous phase constraint according to claim 1, characterized in that: The encoding results of the boundary region are corrected based on the encoding results of adjacent pixels within the same period.

3. The structured light three-dimensional measurement method based on continuous phase constraint according to claim 2, characterized in that: The encoded results are divided into connected components, and the fringe period of each connected component is determined by combining phase information. Connected components that are inconsistent with the phase region are encoded and corrected.

4. The structured light three-dimensional measurement method based on continuous phase constraint according to claim 1, characterized in that: The phase information is calculated from at least three periodic stripe images with different phase offsets.

5. The structured light three-dimensional measurement method based on continuous phase constraint according to claim 1, characterized in that: Based on the color characteristics of the surface of the object being measured, the color channel of the projected stripes is adaptively selected to improve stripe contrast.

6. The structured light three-dimensional measurement method based on continuous phase constraint according to claim 5, characterized in that: The response intensity of each pixel in different color channels is obtained based on the acquired reference image, and the channel with the highest response intensity is selected as the projection color.

Images