Differential Gray coding structured light-based three-dimensional vision reconstruction method

Abstract

The invention discloses a three-dimensional vision reconstruction method based on differential Gray coding structured light, which comprises the following steps of: projecting N phase shift fringe pictures by using a projector, and acquiring a fringe pattern projected to a measured object from a camera to solve a folding phase; the projector projects log2T + 1 Gray code pictures and the same number of differential Gray code pictures, every two Gray code pictures form a group, and T is the fringe period number of the phase shift picture; calculating a mean value of the Gray code graph and the differential Gray code graph in each group, carrying out binarization segmentation on the Gray code, and solving a stripe order; and unfolding a folded phase by using a stripe order, and realizing three-dimensional reconstruction of an object by combining calibration parameters of a camera projector. Aiming at strong scattering environments such as turbid underwater, the problem of picture segmentation in different environments is solved, and by additionally adding a Gray code picture, order edge part code words which are easy to cause errors are prevented from being used when a folded phase is unfolded, so that the method has higher adaptive capacity and ensures the precision of final three-dimensional reconstruction.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and three-dimensional reconstruction, and in particular to a three-dimensional visual reconstruction method in a strong scattering environment. Background Art

[0002] Three-dimensional visual reconstruction technology has a wide range of applications in industrial manufacturing, archaeology, product quality monitoring, underwater exploration and other fields. The optical visual reconstruction technology currently on the market can be divided into two categories according to the imaging method: active measurement and passive measurement. Passive measurement technology does not require an additional light source, and directly obtains images through one or more cameras to restore the three-dimensional information of the object. Commonly used technologies include binocular vision, which calculates the parallax of the left and right cameras for reconstruction, but because the feature point extraction and matching are easily affected by the repetitive texture of the object surface and the extremely dark external environment, the reconstruction effect is average. The more commonly used active optical visual reconstruction method is three-dimensional reconstruction based on structured light. A set of pre-designed stripe patterns are projected onto the object to be measured by a projector, and the camera is used to obtain the image after being modulated by the object, which can achieve high-precision and fast measurement.

[0003] Currently, commonly used structured light 3D reconstruction techniques include phase shifting, Gray code, and DeBruijin sequence. While the combination of phase shifting and Gray code offers strong anti-interference capabilities and high accuracy, Gray code binarization using traditional structured light is difficult in highly scattering environments, such as turbid water. Furthermore, after Gray code binarization, there is a mismatch between the decoded order edges and the folded phase edges, resulting in phase unwrapping errors and ultimately affecting 3D reconstruction quality. Summary of the Invention

[0004] In highly scattering environments, traditional Gray code structured light systems struggle to accurately binarize the Gray code and address the mismatch between decoded order edges and folded phase edges. This paper provides a 3D visual reconstruction method based on differential Gray code structured light. This method calculates a binarization threshold using differential Gray code, thereby addressing the image segmentation problem in different environments. Furthermore, by adding an additional Gray code image, this method avoids the use of codewords along the order edges, which are prone to errors, during folded phase unwrapping. This method offers enhanced adaptability and ensures the accuracy of the final 3D reconstruction.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] A three-dimensional vision reconstruction method based on differential Gray-coded structured light includes the following steps:

[0007] Step 1) The projector projects N phase-shifted fringe images, and the folded phase is extracted from the fringe images collected and projected onto the object under test.

[0008] Step 2) The projector projects log2T+1 Gray code images, where T is the number of fringe periods of the phase-shifted image.

[0009] In step 3, the projector projects log2T+1 differential Gray code images. The differential Gray code has the opposite code value to the Gray code, and pairs of them form a group. The mean of the Gray code image and the differential Gray code image in each group is calculated, which serves as the threshold for subsequent Gray code binarization. Values ​​greater than the threshold are considered 1, and values ​​less than the threshold are considered 0. All Gray code images are binarized.

[0010] Step 4) Decode the log2T Gray code diagrams to obtain the fringe order K1, and decode the log2T+1 Gray code diagrams to obtain the fringe order K2. Use the fringe orders K1 and K2 to unfold the folded phase.

[0011] In step 5), according to the principle of triangulation, the unfolded folded phase is combined with the calibration parameters of the camera and projector to realize the three-dimensional reconstruction of the object.

[0012] Advantages or beneficial effects of the present invention:

[0013] (1) The present invention uses differential Gray code to effectively solve the problem of Gray code binarization segmentation in a strong scattering environment.

[0014] (2) The present invention uses log2T+1 Gray codes for folded phase unfolding, effectively avoiding the use of code words that are prone to error-order edge portions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flowchart of three-dimensional visual reconstruction based on differential Gray-coded structured light in an embodiment;

[0016] Figure 2 The images are projected by the projector, with a portion of each image captured;

[0017] Figure 3 This is a schematic diagram of Gray code decoding;

[0018] Figure 4a-4b is a three-dimensional reconstruction of the object under test in turbid water in the embodiment; wherein, Figure 4a is the front view of the object being measured; Figure 4b A side view of the object being measured. DETAILED DESCRIPTION

[0019] The present invention will be described in detail below with reference to the accompanying drawings and embodiments, examples of which are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0020] Example:

[0021] The following describes, with reference to the accompanying drawings, the 3D reconstruction of an underwater cup using the differential Gray-coded structured light 3D visual reconstruction method proposed in the present invention. First, a projector projects a set of phase-shifted fringes, a set of Gray code patterns, and a set of differential Gray code patterns. A camera captures an image modulated by the surface of the object being measured. The phase-shifted fringes are used to extract the folded phase. The differential Gray code pattern assists in binarizing the Gray code pattern and determining the fringe order. Finally, the folded phase is expanded into an absolute phase using the fringe order, and the surface shape of the object is reconstructed using the system's calibration parameters.

[0022] like Figure 1 As shown, the three-dimensional vision reconstruction method based on differential Gray coding structured light includes the following steps:

[0023] Step 1) The projector projects a set of phase-shifted fringes. The generation formula of the phase-shifted fringes is as follows:

[0024]

[0025] Among them I k Represents the pixel value of different fringe images, A is the background light intensity, B is the modulation intensity, is the initial phase value, f is the phase shift frequency, and x is the pixel coordinate in the encoding direction. After the camera captures the image projected onto the object being measured, the folded phase is solved from the N phase-shifted fringe patterns. The formula is as follows:

[0026]

[0027] In the formula Represents the folded phase. Due to the characteristics of the inverse tangent function, the folded phase And not continuous.

[0028] Step 2) Project log2T+1 Gray code images, each image represents a Gray code value, and each Gray code image is divided into regions with the same number of periods as the phase shift stripes.

[0029] Step 3), project log2T+1 differential Gray code images, such as Figure 2 The code values ​​of the differential Gray code shown are opposite to those of the original Gray code, and two of them form a group. The mean of the Gray code image and the differential Gray code image in each group is calculated as the threshold for subsequent Gray code binarization. Values ​​greater than the threshold are judged as 1, and values ​​less than the threshold are judged as 0. All Gray code images are binarized. The binarization formula is as follows:

[0030]

[0031] Where G(x,y) represents the binarized image, I(x,y) is the Gray code image, and I′(x,y) is the differential Gray code image. In environments with strong scattering, such as turbid water, the scattering of projected light by the medium blurs the black and white boundaries of the Gray code, increasing the difficulty of image binarization. Differential Gray code ensures binarization accuracy to a certain extent.

[0032] Step 4) Decode the log2T Gray code images to obtain the fringe order K1. Each Gray code image represents a binary number, such as Figure 3 As shown, the Gray code of log2T bits is constructed according to the projection order, and the decoding is completed by mapping the Gray code to a decimal number. The specific formula is as follows:

[0033]

[0034] V1(x,y) is the binary value corresponding to the Gray code, i(·) is used to find the known relationship between the decimal code word and the fringe order. K1 is the decimal value corresponding to the Gray code, which is the fringe order.

[0035] Then, solve the log2T+1 Gray code diagram to get K2. The solution formula for K2 is as follows:

[0036]

[0037] Where V2(x,y) is the binary value corresponding to the Gray code. K is the decimal value corresponding to the log2T+1 Gray code value, and INT(·) indicates rounding down. Based on the fringe orders K1 and K2, the folded phase is expanded into a continuous absolute phase, so that each phase value corresponds to a pixel coordinate. The specific expansion formula is as follows:

[0038]

[0039] in the formula is the folding phase obtained by the phase-shifted fringe, K1 and K2 are the fringe orders obtained by the Gray code, and Φ(x,y) is the absolute phase.

[0040] When the folding phase value is in the middle area of ​​the current cycle, When the folded phase value is or When , the folded phase uses the fringe order K2 to solve the phase. Figure 3 As shown, the fringe order calculated using the log2T+1 Gray code is staggered by half a cycle compared to the fringe order calculated using the log2T Gray code, so that the edge of each folded phase cycle falls in the middle area of ​​the fringe order K2, effectively avoiding the use of codewords in the order K1 part that is prone to edge errors.

[0041] Step 5) The unwrapped phase is combined with the calibration information of the camera projector to reconstruct the object. The projector pixel coordinates obtained from the absolute phase are:

[0042]

[0043] Where W is the field of view of the projector in the encoding direction, T is the number of cycles in one field of view of the projector, and Φ(x,y) is the absolute phase. Calibrate the camera and projector to obtain the homography matrix A of the camera and projector respectively. C and A P as follows:

[0044]

[0045] The monocular inverse camera method is used for 3D reconstruction. The projector is regarded as a special camera. The imaging formulas of the projector and camera are as follows:

[0046]

[0047] where u c 、v c is the horizontal and vertical pixel coordinates of the camera, u p 、v p is the horizontal and vertical pixel coordinates of the projector. c 、s p are the scale factors of the camera and projector respectively, and x, y, z are the three-dimensional coordinates of the object. Combining the two, the three-dimensional coordinates can be calculated:

[0048]

[0049] After the reconstruction is completed, the original point cloud data is obtained, and the point cloud obtained by subsequent filtering is as follows Figure 4a This is the front view of the water cup. Figure 4b This is a side view of the water cup.

[0050] After the above steps, the object can be measured in three dimensions based on a set of pictures.

[0051] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention, and that the scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A three-dimensional visual reconstruction method based on differential Gray coded structured light, comprising the following steps: Step 1), the projector projects N phase-shifted fringe images, and the folded phase is extracted from the fringe image projected onto the object under test by the camera. Step 2), the projector projects log2T+1 Gray code images, where T is the number of fringe periods of the phase-shifted image. Step 3), the projector projects log2T+1 differential Gray code images. The code value of the differential Gray code is opposite to that of the Gray code, and two of them form a group. The mean of the Gray code image and the differential Gray code image in each group is calculated as the threshold for subsequent Gray code binarization. If it is greater than the threshold, it is judged as 1, and if it is less than the threshold, it is judged as 0. All Gray code images are binarized. Step 4), solve the log2T Gray code diagrams to get the fringe order K1 value, and solve the log2T+1 Gray code diagrams to get the K2 value. Unfold the folded phase according to the range of the solved folded phase value. Step 5), based on the triangulation principle, the unfolded folded phase is combined with the calibration parameters of the camera projector to achieve three-dimensional reconstruction of the object.

2. The method for 3D visual reconstruction based on differential Gray coded structured light according to claim 1, characterized in that: In step 3), the mean of each group of Gray code images and the differential Gray code image is calculated, and this mean is used as the threshold for the subsequent binarization. If it is greater than the threshold, it is judged as 1, and if it is less than the threshold, it is judged as 0, and all Gray code images are binarized. The differential Gray code is used to binarize the image, and the specific formula is as follows: Where G(x,y) represents the binarized image, I(x,y) is the Gray code image, and I′(x,y) is the Gray code inverse image. In environments with scattering effects such as turbid water, the scattering effect of the medium on the projected light will cause the color of the binary boundary to be blurred, resulting in image decoding errors. The differential Gray code guarantees the accuracy of the binarization to a certain extent.

3. The three-dimensional visual reconstruction method based on differential Gray coded structured light according to claim 1, characterized in that: In step 4), the log2T Gray code images are solved to obtain the fringe order K1 value, and the log2T+1 Gray code images are solved to obtain the K2 value. The folded phase is unfolded according to the range of the solved folded phase value. The specific unfolding formula is as follows: When the folding phase value is in the middle area of ​​the current cycle, When the folded phase value is or When the folded phase is solved by using fringe order K2, the fringe order calculated by log2T+1 Gray code is staggered by half a cycle compared to the fringe order calculated by log2T Gray code, so that the edge of each fringe cycle falls in the middle area of ​​fringe order K2, effectively avoiding the use of codewords in the order K1 part that are prone to edge errors.

Images