A structured light three-dimensional imaging method based on multi-frequency phase shift and related device

Abstract

The present invention discloses a structured light three-dimensional imaging method and related device based on multi-frequency phase shift, wherein the method comprises: obtaining multiple pieces of light stripe image data to be processed collected under multi-frequency phase, performing Gray code encoding processing on each pixel point in each piece of the multiple pieces of light stripe image data to be processed, and forming a set of phase-shifted light stripe image data; performing image preprocessing on the set of phase-shifted light stripe image data based on a preset algorithm, and obtaining a set of preprocessed phase-shifted light stripe image data; performing decoding processing on the preprocessed set of phase-shifted light stripe image data based on a preset decoding algorithm, and forming absolute phase image data; performing structured light three-dimensional geometric reconstruction processing based on the absolute phase image data, and generating a three-dimensional imaging map corresponding to the structured light. In the embodiment of the present invention, a high-precision phase value can be quickly obtained, and rapid three-dimensional reconstruction can be achieved at the same time.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and is a structured light three-dimensional imaging method based on multi-frequency phase shift and a related device. Background Art

[0002] The three-dimensional reconstruction technology of structured light coding method is to mark the coordinates of each pixel in the image by encoding. The reconstruction process is as follows: first, the encoded structured light pattern is projected onto the target object, and the image containing the structured light pattern is acquired by the camera, and then decoded by a computer decoding scheme. Finally, combined with the calibrated system parameters, the three-dimensional information of the object is obtained by the principle of optical triangulation, thereby reconstructing the three-dimensional surface of the object; the use of structured light coding method can shorten the time of three-dimensional reconstruction and improve the reconstruction efficiency.

[0003] In structured light 3D reconstruction technology, the structured light encoding and decoding process directly affects the speed of 3D reconstruction. To achieve fast 3D reconstruction of a virtual 3D model in reality, high-speed structured light encoding and decoding are required. The accuracy of the fringe phase extracted after structured light decoding directly affects the 3D reconstruction results of the object being measured. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides a structured light three-dimensional imaging method and related devices based on multi-frequency phase shift, which can quickly obtain high-precision phase values ​​and achieve rapid three-dimensional reconstruction.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a structured light three-dimensional imaging method based on multi-frequency phase shift, the method comprising:

[0006] Acquire a plurality of light fringe image data to be processed collected under multi-frequency phases, and perform Gray code encoding processing on each pixel point in each of the plurality of light fringe image data to be processed to form a set of phase-shifted light fringe image data;

[0007] Performing image preprocessing on the set of phase-shifted light fringe image data based on a preset algorithm to obtain a set of preprocessed phase-shifted light fringe image data;

[0008] Decoding a set of preprocessed phase-shifted light fringe image data based on a preset decoding algorithm to form absolute phase image data;

[0009] A three-dimensional geometric reconstruction process of the structured light is performed based on the absolute phase image data to generate a three-dimensional imaging image corresponding to the structured light.

[0010] Optionally, performing Gray code encoding processing on each pixel point in each of the plurality of light fringe image data to be processed to form a set of phase-shifted light fringe image data includes:

[0011] A preset number of different Gray code codes are obtained, and Gray code coding is performed on each pixel point in each of the plurality of light streak image data to be processed based on the different Gray code codes to form a set of phase-shifted light streak image data, wherein the set of phase-shifted light streak image data is a one-dimensional coded image generated by a coded image formed by performing Gray code coding on each pixel point in the plurality of light streak image data to be processed, and a one-bit code is expanded to a column of codes, and the Gray code is increased from top to bottom, and each level is formed by phase-shifted light streak image data composed of a coded projection pattern.

[0012] Optionally, performing image preprocessing on the set of phase-shifted light fringe image data based on a preset algorithm to obtain a set of preprocessed phase-shifted light fringe image data includes:

[0013] Obtaining a maximum grayscale value and a minimum grayscale value among all pixel grayscale values ​​in a set of phase-shifted light fringe image data, and normalizing the current grayscale value of each phase-shifted light fringe image data in the set of phase-shifted light fringe image data according to the maximum grayscale value and the minimum grayscale value to obtain a set of normalized processed images;

[0014] A binarization threshold is obtained based on a normalization result of pixel gray values ​​in a set of normalized processed images;

[0015] A group of normalized processed images are binarized based on the binarization threshold to obtain a group of pre-processed phase-shifted light fringe image data.

[0016] Optionally, obtaining a binarization threshold based on a normalization result of pixel grayscale values ​​in a set of normalized processed images includes:

[0017] Obtaining a normalized value corresponding to each pixel grayscale value of the pixel grayscale value normalization result;

[0018] When the normalized value is greater than or equal to 0.5, the binarization threshold is set to 1; when the normalized value is less than 0.5, the binarization threshold is set to 0.

[0019] Optionally, the decoding of a set of pre-processed phase-shifted light fringe image data based on a preset decoding algorithm to form absolute phase image data includes:

[0020] Based on the three-step phase shift formula, a relative phase calculation process is performed on each pixel value in a set of phase-shifted light fringe image data after preprocessing to obtain the relative phase corresponding to each pixel value in each phase-shifted light fringe image data;

[0021] Calculate the Gray code encoding value corresponding to each pixel value in a set of preprocessed phase-shifted light fringe image data based on the Gray code decoding formula;

[0022] Based on the relative phase corresponding to each pixel value and the Gray code encoding value, three-step phase shift calculation is performed using high-frequency fringes and low-frequency fringes to obtain absolute phase image data.

[0023] Optionally, performing structured light three-dimensional geometric reconstruction processing based on the absolute phase image data to generate a three-dimensional imaging image corresponding to the structured light includes:

[0024] Performing point cloud generation processing based on the absolute phase image data to obtain point cloud data;

[0025] The point cloud data is processed in sequence by noise reduction, smoothing and hole filling to obtain processed point cloud data;

[0026] The processed point cloud data is subjected to three-dimensional geometric reconstruction based on the surface reconstruction algorithm to obtain a three-dimensional imaging image corresponding to the structured light.

[0027] Optionally, performing point cloud generation processing based on the absolute phase image data to obtain point cloud data includes:

[0028] Obtaining a first coordinate system of the camera corresponding to the paired phase image data and a second coordinate system of the projection, and converting the first coordinate system and the second coordinate system into a world coordinate system;

[0029] Calculating the three-dimensional coordinate data of the absolute phase image data in the world coordinate system through the projection model of the camera and the light reflection projection of the projector;

[0030] Calculating and processing the distance and parallax of the feature points of the absolute phase image data based on the camera parameters of the camera and the light reflection projection of the projector to obtain parallax data and distance data of the feature points of the absolute phase image data;

[0031] The disparity data and distance data of the feature points are connected in the three-dimensional coordinate data to generate point cloud data.

[0032] In addition, an embodiment of the present invention further provides a structured light three-dimensional imaging device based on multi-frequency phase shift, the device comprising:

[0033] An encoding module is used to obtain a plurality of light fringe image data to be processed collected under multi-frequency phases, and perform Gray code encoding processing on each pixel point in each of the plurality of light fringe image data to be processed to form a set of phase-shifted light fringe image data;

[0034] Preprocessing module: used for performing image preprocessing on the set of phase-shifted light fringe image data based on a preset algorithm to obtain a set of preprocessed phase-shifted light fringe image data;

[0035] Decoding module: used for decoding a set of pre-processed phase-shifted light fringe image data based on a preset decoding algorithm to form absolute phase image data;

[0036] The three-dimensional reconstruction module is used to perform three-dimensional geometric reconstruction processing of the structured light based on the absolute phase image data to generate a three-dimensional imaging image corresponding to the structured light.

[0037] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the structured light three-dimensional imaging method as described in any one of the above is implemented.

[0038] In addition, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0039] one or more processors;

[0040] Memory;

[0041] One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the one or more processors, and the one or more applications are configured to: execute the structured light three-dimensional imaging method according to any one of the above.

[0042] In an embodiment of the present invention, by performing Gray code encoding on the light streak image data to be processed, and then performing corresponding preprocessing and finally performing decoding processing, absolute phase image data can be obtained, and finally structured light three-dimensional geometric reconstruction processing is performed; high-precision phase values ​​can be quickly obtained, and fast three-dimensional reconstruction can be achieved at the same time; high-precision phase values ​​can be quickly and accurately solved through Gray code encoding and decoding; then high-precision three-dimensional coordinates can be calculated through accurate phase values, thereby achieving fast three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 is a flow chart of a structured light three-dimensional imaging method based on multi-frequency phase shift in an embodiment of the present invention;

[0045] Figure 2 is a schematic diagram of the structure of a structured light three-dimensional imaging device based on multi-frequency phase shift in an embodiment of the present invention;

[0046] Figure 3 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] For example, see Figure 1 , Figure 1 It is a flow chart of a structured light three-dimensional imaging method based on multi-frequency phase shift in an embodiment of the present invention.

[0049] like Figure 1 As shown, a structured light three-dimensional imaging method based on multi-frequency phase shift, the method comprising:

[0050] S11: obtaining a plurality of light fringe image data to be processed collected under multi-frequency phases, and performing Gray code encoding processing on each pixel point in each of the plurality of light fringe image data to be processed to form a set of phase-shifted light fringe image data;

[0051] In a specific implementation process of the present invention, Gray code encoding is performed on each pixel point in each of the plurality of light fringe image data to be processed to form a set of phase-shifted light fringe image data, including:

[0052] A preset number of different Gray code codes are obtained, and Gray code coding is performed on each pixel point in each of the plurality of light streak image data to be processed based on the different Gray code codes to form a group of phase-shifted light streak image data; the group of phase-shifted light streak image data is a one-dimensional coded image generated by extending a bit of coding to a column of coding on a coded image formed by Gray code coding each pixel point in the plurality of light streak image data to be processed, and the Gray code is increased from top to bottom, and each level is formed by phase-shifted light streak image data composed of a coded projection pattern; each of the light streak image data to be processed is encoded with a certain bit of Gray code; and in the Gray code coding process, the following Gray code is generated from the existing previous Gray code; assuming that a k-bit Gray code has been generated, then the k+1-bit Gray code is generated in the following manner: inserting a bit of 0 in front of the k-bit Gray code in sequence to generate a group of codes; inserting a bit of 1 in front of the k-bit Gray code in reverse order to generate another group of codes; the two groups of codes together are the k+1-bit Gray code.

[0053] Specifically, Gray code is a special binary encoding method. The benefits of using it in structured light 3D reconstruction include: reducing errors: Gray code can represent digital changes through minimal light pattern changes. There is only one bit difference between adjacent codes, which can reduce errors caused by factors such as illumination changes and noise; improving recognition accuracy: Gray code encoding can improve the recognition accuracy of feature points. There is only one bit difference between adjacent codes, which can reduce the ambiguity when matching feature points. Compared with traditional binary encoding, Gray code can be more easily recognized and decoded; increasing encoding capacity: Gray code encoding can represent more encoding values ​​with the same number of encoding bits. There is only one bit difference between adjacent codes, which can increase the encoding capacity and improve the depth range of the reconstructed object; reducing crosstalk: Due to the characteristics of Gray code encoding, crosstalk (i.e., light pattern aliasing between adjacent codes) has a small impact. This can reduce confusion and crossover problems in reconstruction and improve the accuracy and reliability of reconstruction; fast decoding: Gray code encoding can achieve fast light pattern decoding through fast decoding technology. The decoding process of Gray code is relatively simple and can be implemented through fast algorithms such as bit operations, thereby speeding up the reconstruction. In summary, Gray code has the advantages of reducing errors, improving recognition accuracy, increasing coding capacity, reducing crosstalk and fast decoding in structured light 3D reconstruction, so it is widely used in structured light 3D reconstruction.

[0054] By collecting images at different frequencies using a camera or other device, multiple light streak image data to be processed collected at multiple frequency phases can be obtained. At this time, it is necessary to perform Gray code encoding processing on each pixel point in each of the multiple light streak image data to be processed. Before the encoding processing, it is necessary to first obtain a preset number of different Gray code codes. In the technical solution of the present application, 11 Gray code images can be used to generate 2 11 Different Gray code encodings are used, so that the cycle of the generated Gray code encoding is relatively small, and Gray code decoding errors can be effectively corrected later; then, Gray code encoding is performed on each pixel point in each of the multiple light streak image data to be processed by using different Gray code encodings to form a group of phase-shifted light streak image data; that is, the encoding of one bit is expanded to a column of encoding on the one-dimensional encoding image generated by performing Gray code encoding on each pixel point in the multiple light streak image data to be processed, and the Gray code is increased from top to bottom, and each level is a phase-shifted light streak image data composed of a coded projection pattern, thereby forming a group of phase-shifted light streak image data.

[0055] After each pixel is encoded with Gray code, each picture represents a certain bit of Gray code. Taking the first column of the picture as an example, its Gray code is 00001. The stripes of Gray code in the first column of the first 4 pictures are all black, representing 0, while the Gray code in the first column of the last picture is white, representing 1. The encoding algorithm of Gray code is: generate the subsequent Gray code from the existing previous Gray code; assuming that k-bit Gray code has been generated, then the generation method of k+1-bit Gray code is: insert a 0 in front of the k-bit Gray code in sequence to generate a group of codes; insert a 1 in front of the k-bit Gray code in reverse order to generate another group of codes; the two groups of codes together are k+1-bit Gray code.

[0056] S12: performing image preprocessing on the set of phase-shifted light fringe image data based on a preset algorithm to obtain a set of preprocessed phase-shifted light fringe image data;

[0057] In a specific implementation of the present invention, the image preprocessing of the set of phase-shifted light streak image data based on a preset algorithm to obtain a set of preprocessed phase-shifted light streak image data includes: obtaining a maximum grayscale value and a minimum grayscale value among all pixel grayscale values ​​in a set of phase-shifted light streak image data, and normalizing the current grayscale value of each phase-shifted light streak image data in the set of phase-shifted light streak image data according to the maximum grayscale value and the minimum grayscale value to obtain a set of normalized processed images; obtaining a binarization threshold value based on the normalization result of the pixel grayscale values ​​in the set of normalized processed images; and binarizing the set of normalized processed images based on the binarization threshold value to obtain a set of preprocessed phase-shifted light streak image data.

[0058] Furthermore, the method of obtaining a binarization threshold based on a normalization result of pixel grayscale values ​​in a set of normalized processed images includes: obtaining a normalized value corresponding to each pixel grayscale value of the pixel grayscale value normalization result; when the normalized value is greater than or equal to 0.5, setting the binarization threshold to 1; and when the normalized value is less than 0.5, setting the binarization threshold to 0.

[0059] Specifically, since the grayscale values ​​obtained by the same light stripes on different objects are different, a fixed threshold cannot solve this problem, and different thresholds need to be calculated for different pixels; therefore, it is necessary to normalize the maximum grayscale value and the minimum grayscale value of each pixel in a set of phase-shifted light stripe image data, and the current grayscale value of each phase-shifted light stripe image data in a set of phase-shifted light stripe image data according to the maximum grayscale value and the minimum grayscale value, to obtain a set of normalized images; the specific normalization formula is as follows:

[0060]

[0061] Among them I n is the normalized result of the grayscale value of each pixel; I is the grayscale value of the current pixel; I min is the minimum gray value; I max is the maximum grayscale value; and then a set of normalized images are binarized to obtain a set of preprocessed phase-shifted light fringe image data.

[0062] During binarization, it is necessary to obtain the corresponding binarization threshold value, which is obtained based on the normalization results of the pixel grayscale values ​​in a set of normalized processed images, specifically, the normalized value corresponding to each pixel grayscale value for which the pixel grayscale normalization results need to be obtained; when the normalized value is greater than or equal to 0.5, the binarization threshold value is set to 1; when the normalized value is less than 0.5, the binarization threshold value is set to 0.

[0063] S13: decoding the preprocessed set of phase-shifted light fringe image data based on a preset decoding algorithm to form absolute phase image data;

[0064] In a specific implementation process of the present invention, the decoding process of a set of phase-shifted light fringe image data after preprocessing is performed based on a preset decoding algorithm to form absolute phase image data, including: performing relative phase calculation processing on each pixel value in a set of phase-shifted light fringe image data after preprocessing based on a three-step phase shift formula to obtain the relative phase corresponding to each pixel value in each phase-shifted light fringe image data; calculating the Gray code encoding value corresponding to each pixel value in a set of phase-shifted light fringe image data after preprocessing based on the Gray code decoding formula; performing three-step phase shift calculation using high-frequency fringes and low-frequency fringes based on the relative phase corresponding to each pixel value and the Gray code encoding value to obtain absolute phase image data.

[0065] Specifically, a decoding operation is performed on a set of preprocessed phase-shifted light fringe image data to obtain the absolute phase value of each pixel in the set; in the technical solution, a relative phase calculation process is performed through a relative phase formula of a three-step phase shift to obtain the relative phase corresponding to each pixel value in each phase-shifted light fringe image data; the formula is as follows:

[0066]

[0067] Among them, A0, A1, and A2 respectively represent the pixel values ​​of the image of the phase fringes corresponding to the light fringes image data with different phase offsets.

[0068] Then, the Gray code encoding value corresponding to each pixel value in a set of preprocessed phase-shifted light fringe image data is calculated by the Gray code decoding formula; the Gray code decoding formula is as follows:

[0069]

[0070] k(x,y)=i(V(x,y));

[0071] By first obtaining the relative phase corresponding to each pixel, and then the Gray code value of each pixel, the absolute phase value of each pixel is obtained through the above formula; after phase expansion, the absolute phase corresponding to each pixel of the camera image is obtained, which is used for the spatial coordinate reconstruction of the subsequent three-dimensional model.

[0072] In order to meet the accuracy and speed requirements of industrial 3D cameras, Gray code multi-frequency phase shift technology is used to achieve high-speed and high-precision encoding and decoding; high-frequency stripes and low-frequency stripes are used to perform three-step phase shifts respectively to obtain absolute phase image data; combined with Gray code, the accuracy can be greatly improved, while the number of stripe patterns can be reduced and the calculation speed can be increased.

[0073] The calculation process of using high-frequency fringes and low-frequency fringes to perform three-step phase shifts is as follows:

[0074] Φ h (x,y)=(λ l / λ h )Φ l (x,y);

[0075] Φ h (x,y)=φ h (x,y)+2πk h (x,y);

[0076] Φ l (x,y)=φ l (x,y)+2πk l (x,y);

[0077] where Φ h (x,y) is the high-frequency fringe; Φ l (x,y) is the low-frequency fringe; λ l is the wavelength of the low-frequency fringes; h is the wavelength of the high-frequency fringes.

[0078] High-frequency stripes and low-frequency stripes are used to make three-step phase shifts respectively. Combining with Gray code can greatly improve the accuracy, while reducing the number of stripe patterns and increasing the calculation speed. After decoding, the absolute phase map is obtained. However, due to the strong mirror reflection on the surface of some highly reflective objects, the solved phase may have outliers, which need to be removed.

[0079] S14: performing structured light three-dimensional geometric reconstruction processing based on the absolute phase image data to generate a three-dimensional imaging image corresponding to the structured light.

[0080] In the specific implementation process of the present invention, the three-dimensional geometric reconstruction processing of structured light is performed based on the absolute phase image data to generate a three-dimensional imaging diagram corresponding to the structured light, including: performing point cloud generation processing based on the absolute phase image data to obtain point cloud data; performing denoising, smoothing and hole filling processing on the point cloud data in turn to obtain processed point cloud data; performing three-dimensional geometric reconstruction processing on the processed point cloud data based on a surface reconstruction algorithm to obtain a three-dimensional imaging diagram corresponding to the structured light.

[0081] Furthermore, the point cloud generation processing based on the absolute phase image data to obtain the point cloud data includes: obtaining a first coordinate system of the camera corresponding to the paired phase image data and a second coordinate system of the projection, and converting the first coordinate system and the second coordinate system into a world coordinate system; calculating the three-dimensional coordinate data of the absolute phase image data in the world coordinate system through the projection model of the camera and the light reflection projection of the projector; calculating the distance and view of the feature points of the absolute phase image data based on the camera parameters of the camera and the light reflection projection of the projector to obtain the disparity data and distance data of the feature points of the absolute phase image data; and connecting the disparity data and distance data of the feature points in the three-dimensional coordinate data to generate point cloud data.

[0082] Specifically, point cloud generation processing is first performed through absolute phase image data to obtain point cloud data; then, noise reduction, smoothing and hole filling processing are performed on the point cloud data in turn to obtain processed point cloud data; finally, three-dimensional geometric construction algorithms such as surface reconstruction algorithms are used to perform three-dimensional geometric reconstruction processing on the processed point cloud data to obtain a three-dimensional imaging image corresponding to the structured light.

[0083] In the process of generating point cloud, it is first necessary to obtain the first coordinate system of the camera corresponding to the paired phase image data and the second coordinate system of the projection, and then convert the first coordinate system and the second coordinate system into the world coordinate system; then calculate the three-dimensional coordinate data of the absolute phase image data in the world coordinate system through the projection model of the camera and the light reflection projection of the projector; and calculate and process the distance and view of the feature points of the absolute phase image data according to the camera parameters of the camera and the light reflection projection of the projector, and then obtain the disparity data and distance data of the feature points of the absolute phase image data; connect the disparity data and distance data of the feature points in the three-dimensional coordinate data to generate point cloud data.

[0084] In an embodiment of the present invention, by performing Gray code encoding on the light streak image data to be processed, and then performing corresponding preprocessing and finally performing decoding processing, absolute phase image data can be obtained, and finally structured light three-dimensional geometric reconstruction processing is performed; high-precision phase values ​​can be quickly obtained, and fast three-dimensional reconstruction can be achieved at the same time; high-precision phase values ​​can be quickly and accurately solved through Gray code encoding and decoding; then high-precision three-dimensional coordinates can be calculated through accurate phase values, thereby achieving fast three-dimensional reconstruction.

[0085] For example 2, please refer to Figure 2 , Figure 2 It is a schematic diagram of the structural composition of a structured light three-dimensional imaging device based on multi-frequency phase shift in an embodiment of the present invention.

[0086] like Figure 2 As shown, a structured light three-dimensional imaging device based on multi-frequency phase shift, the device comprising:

[0087] The encoding module 21 is used to obtain a plurality of light fringe image data to be processed collected under multi-frequency phases, and perform Gray code encoding processing on each pixel point in each of the plurality of light fringe image data to be processed to form a set of phase-shifted light fringe image data;

[0088] In a specific implementation of the present invention, the Gray code encoding process is performed on each pixel point in each of the plurality of light streak image data to be processed to form a set of phase-shifted light streak image data, including: obtaining a preset number of different Gray code codes, and performing Gray code encoding process on each pixel point in each of the plurality of light streak image data to be processed based on the different Gray code codes to form a set of phase-shifted light streak image data; the set of phase-shifted light streak image data is a set of codes formed by performing Gray code encoding on each pixel point in the plurality of light streak image data to be processed. The one-dimensional coded image generated by the code image is expanded from one bit of code to one column of code, which is an increasing series of Gray codes from top to bottom, and each stage is formed by phase-shifted light stripe image data composed of a coded projection pattern; each light stripe image data to be processed is encoded with a certain bit of Gray code; and in the Gray code encoding process, the subsequent Gray code is generated by the existing previous Gray code; assuming that a k-bit Gray code has been generated, then the generation method of the k+1-bit Gray code is: insert a 0 in front of the k-bit Gray code in sequence to generate a group of codes; insert a 1 in front of the k-bit Gray code in reverse order to generate another group of codes; the two groups of codes together are the k+1-bit Gray code.

[0089] Specifically, Gray code is a special binary coding method. The benefits of using it in structured light 3D reconstruction include: reducing errors: Gray code can represent digital changes through minimal light pattern changes. There is only one bit difference between adjacent codes, which can reduce errors caused by factors such as illumination changes and noise; improving recognition accuracy: Gray code encoding can improve the recognition accuracy of feature points. There is only one bit difference between adjacent codes, which can reduce the ambiguity when matching feature points. Compared with traditional binary coding, Gray code can be more easily recognized and decoded; increasing coding capacity: Gray code encoding can represent more coding values ​​with the same number of coding bits. There is only one bit difference between adjacent codes, which can increase the coding capacity and improve the depth range of reconstructed objects; reducing crosstalk: Due to the characteristics of Gray code encoding, crosstalk (i.e., light pattern aliasing between adjacent codes) has a small impact. This can reduce confusion and crossover problems in reconstruction and improve the accuracy and reliability of reconstruction; fast decoding: Gray code encoding can achieve fast light pattern decoding through fast decoding technology. The decoding process of Gray code is relatively simple and can be implemented through fast algorithms such as bit operations, thereby speeding up the reconstruction. In summary, Gray code has the advantages of reducing errors, improving recognition accuracy, increasing coding capacity, reducing crosstalk and fast decoding in structured light 3D reconstruction, so it is widely used in structured light 3D reconstruction.

[0090] By collecting images at different frequencies using a camera or other device, multiple light streak image data to be processed collected at multiple frequency phases can be obtained. At this time, it is necessary to perform Gray code encoding processing on each pixel point in each of the multiple light streak image data to be processed. Before the encoding processing, it is necessary to first obtain a preset number of different Gray code codes. In the technical solution of the present application, 11 Gray code images can be used to generate 2 11 Different Gray code encodings are used, so that the cycle of the generated Gray code encoding is relatively small, and Gray code decoding errors can be effectively corrected later; then, Gray code encoding is performed on each pixel point in each of the multiple light streak image data to be processed by using different Gray code encodings to form a group of phase-shifted light streak image data; that is, the encoding of one bit is expanded to a column of encoding on the one-dimensional encoding image generated by performing Gray code encoding on each pixel point in the multiple light streak image data to be processed, and the Gray code is increased from top to bottom, and each level is a phase-shifted light streak image data composed of a coded projection pattern, thereby forming a group of phase-shifted light streak image data.

[0091] After each pixel is encoded with Gray code, each picture represents a certain bit of Gray code. Taking the first column of the picture as an example, its Gray code is 00001. The stripes of Gray code in the first column of the first 4 pictures are all black, representing 0, while the Gray code in the first column of the last picture is white, representing 1. The encoding algorithm of Gray code is: generate the subsequent Gray code from the existing previous Gray code; assuming that k-bit Gray code has been generated, then the generation method of k+1-bit Gray code is: insert a 0 in front of the k-bit Gray code in sequence to generate a group of codes; insert a 1 in front of the k-bit Gray code in reverse order to generate another group of codes; the two groups of codes together are k+1-bit Gray code.

[0092] Preprocessing module 22: used for performing image preprocessing on the set of phase-shifted light fringe image data based on a preset algorithm to obtain a set of preprocessed phase-shifted light fringe image data;

[0093] In a specific implementation of the present invention, the image preprocessing of the set of phase-shifted light streak image data based on a preset algorithm to obtain a set of preprocessed phase-shifted light streak image data includes: obtaining a maximum grayscale value and a minimum grayscale value among all pixel grayscale values ​​in a set of phase-shifted light streak image data, and normalizing the current grayscale value of each phase-shifted light streak image data in the set of phase-shifted light streak image data according to the maximum grayscale value and the minimum grayscale value to obtain a set of normalized processed images; obtaining a binarization threshold value based on the normalization result of the pixel grayscale values ​​in the set of normalized processed images; and binarizing the set of normalized processed images based on the binarization threshold value to obtain a set of preprocessed phase-shifted light streak image data.

[0094] Furthermore, the method of obtaining a binarization threshold based on a normalization result of pixel grayscale values ​​in a set of normalized processed images includes: obtaining a normalized value corresponding to each pixel grayscale value of the pixel grayscale value normalization result; when the normalized value is greater than or equal to 0.5, setting the binarization threshold to 1; and when the normalized value is less than 0.5, setting the binarization threshold to 0.

[0095] Specifically, since the grayscale values ​​obtained by the same light stripes on different objects are different, a fixed threshold cannot solve this problem, and different thresholds need to be calculated for different pixels; therefore, it is necessary to normalize the maximum grayscale value and the minimum grayscale value of each pixel in a set of phase-shifted light stripe image data, and the current grayscale value of each phase-shifted light stripe image data in a set of phase-shifted light stripe image data according to the maximum grayscale value and the minimum grayscale value, to obtain a set of normalized images; the specific normalization formula is as follows:

[0096]

[0097] Among them I n is the normalized result of the grayscale value of each pixel; I is the grayscale value of the current pixel; I min is the minimum gray value; I max is the maximum grayscale value; and then a set of normalized images are binarized to obtain a set of preprocessed phase-shifted light fringe image data.

[0098] During binarization, it is necessary to obtain the corresponding binarization threshold value, which is obtained based on the normalization results of the pixel grayscale values ​​in a set of normalized processed images, specifically, the normalized value corresponding to each pixel grayscale value for which the pixel grayscale normalization results need to be obtained; when the normalized value is greater than or equal to 0.5, the binarization threshold value is set to 1; when the normalized value is less than 0.5, the binarization threshold value is set to 0.

[0099] Decoding module 23: used for decoding a set of pre-processed phase-shifted light fringe image data based on a preset decoding algorithm to form absolute phase image data;

[0100] In a specific implementation process of the present invention, the decoding process of a set of phase-shifted light fringe image data after preprocessing is performed based on a preset decoding algorithm to form absolute phase image data, including: performing relative phase calculation processing on each pixel value in a set of phase-shifted light fringe image data after preprocessing based on a three-step phase shift formula to obtain the relative phase corresponding to each pixel value in each phase-shifted light fringe image data; calculating the Gray code encoding value corresponding to each pixel value in a set of phase-shifted light fringe image data after preprocessing based on the Gray code decoding formula; performing three-step phase shift calculation using high-frequency fringes and low-frequency fringes based on the relative phase corresponding to each pixel value and the Gray code encoding value to obtain absolute phase image data.

[0101] Specifically, a decoding operation is performed on a set of preprocessed phase-shifted light fringe image data to obtain the absolute phase value of each pixel in the set; in the technical solution, a relative phase calculation process is performed through a relative phase formula of a three-step phase shift to obtain the relative phase corresponding to each pixel value in each phase-shifted light fringe image data; the formula is as follows:

[0102]

[0103] Among them, A0, A1, and A2 respectively represent the pixel values ​​of the image of the phase fringes corresponding to the light fringes image data with different phase offsets.

[0104] Then, the Gray code encoding value corresponding to each pixel value in a set of preprocessed phase-shifted light fringe image data is calculated by the Gray code decoding formula; the Gray code decoding formula is as follows:

[0105]

[0106] k(x,y)=i(V(x,y));

[0107] By first obtaining the relative phase corresponding to each pixel, and then the Gray code value of each pixel, the absolute phase value of each pixel is obtained through the above formula; after phase expansion, the absolute phase corresponding to each pixel of the camera image is obtained, which is used for the spatial coordinate reconstruction of the subsequent three-dimensional model.

[0108] In order to meet the accuracy and speed requirements of industrial 3D cameras, Gray code multi-frequency phase shift technology is used to achieve high-speed and high-precision encoding and decoding; high-frequency stripes and low-frequency stripes are used to perform three-step phase shifts respectively to obtain absolute phase image data; combined with Gray code, the accuracy can be greatly improved, while the number of stripe patterns can be reduced and the calculation speed can be increased.

[0109] The calculation process of using high-frequency fringes and low-frequency fringes to perform three-step phase shifts is as follows:

[0110] Φ h (x,y)=(λ l / λ h )Φ l (x,y);

[0111] Φ h (x,y)=φ h (x,y)+2πk h (x,y);

[0112] Φ l (x,y)=φ l (x,y)+2πk l (x,y);

[0113] where Φ h (x,y) is the high-frequency fringe; Φ l (x,y) is the low-frequency fringe; λ l is the wavelength of the low-frequency fringes; h is the wavelength of the high-frequency fringes.

[0114] High-frequency stripes and low-frequency stripes are used to make three-step phase shifts respectively. Combining with Gray code can greatly improve the accuracy, while reducing the number of stripe patterns and increasing the calculation speed. After decoding, the absolute phase map is obtained. However, due to the strong mirror reflection on the surface of some highly reflective objects, the solved phase may have outliers, which need to be removed.

[0115] The three-dimensional reconstruction module 24 is used to perform three-dimensional geometric reconstruction processing of the structured light based on the absolute phase image data to generate a three-dimensional imaging image corresponding to the structured light.

[0116] In the specific implementation process of the present invention, the three-dimensional geometric reconstruction processing of structured light is performed based on the absolute phase image data to generate a three-dimensional imaging diagram corresponding to the structured light, including: performing point cloud generation processing based on the absolute phase image data to obtain point cloud data; performing denoising, smoothing and hole filling processing on the point cloud data in turn to obtain processed point cloud data; performing three-dimensional geometric reconstruction processing on the processed point cloud data based on a surface reconstruction algorithm to obtain a three-dimensional imaging diagram corresponding to the structured light.

[0117] Furthermore, the point cloud generation processing based on the absolute phase image data to obtain the point cloud data includes: obtaining a first coordinate system of the camera corresponding to the paired phase image data and a second coordinate system of the projection, and converting the first coordinate system and the second coordinate system into a world coordinate system; calculating the three-dimensional coordinate data of the absolute phase image data in the world coordinate system through the projection model of the camera and the light reflection projection of the projector; calculating the distance and view of the feature points of the absolute phase image data based on the camera parameters of the camera and the light reflection projection of the projector to obtain the disparity data and distance data of the feature points of the absolute phase image data; and connecting the disparity data and distance data of the feature points in the three-dimensional coordinate data to generate point cloud data.

[0118] Specifically, point cloud generation processing is first performed through absolute phase image data to obtain point cloud data; then, noise reduction, smoothing and hole filling processing are performed on the point cloud data in turn to obtain processed point cloud data; finally, three-dimensional geometric construction algorithms such as surface reconstruction algorithms are used to perform three-dimensional geometric reconstruction processing on the processed point cloud data to obtain a three-dimensional imaging image corresponding to the structured light.

[0119] In the process of generating point cloud, it is first necessary to obtain the first coordinate system of the camera corresponding to the paired phase image data and the second coordinate system of the projection, and then convert the first coordinate system and the second coordinate system into the world coordinate system; then calculate the three-dimensional coordinate data of the absolute phase image data in the world coordinate system through the projection model of the camera and the light reflection projection of the projector; and calculate and process the distance and view of the feature points of the absolute phase image data according to the camera parameters of the camera and the light reflection projection of the projector, and then obtain the disparity data and distance data of the feature points of the absolute phase image data; connect the disparity data and distance data of the feature points in the three-dimensional coordinate data to generate point cloud data.

[0120] In an embodiment of the present invention, by performing Gray code encoding on the light streak image data to be processed, and then performing corresponding preprocessing and finally performing decoding processing, absolute phase image data can be obtained, and finally structured light three-dimensional geometric reconstruction processing is performed; high-precision phase values ​​can be quickly obtained, and fast three-dimensional reconstruction can be achieved at the same time; high-precision phase values ​​can be quickly and accurately solved through Gray code encoding and decoding; then high-precision three-dimensional coordinates can be calculated through accurate phase values, thereby achieving fast three-dimensional reconstruction.

[0121] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the structured light three-dimensional imaging method of any one of the above embodiments is implemented. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (EraSable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the storage device includes any medium that can store or transmit information in a readable form by a device (for example, a computer, a mobile phone), which can be a read-only memory, a disk or an optical disk, etc.

[0122] An embodiment of the present invention further provides a computer application program, which runs on a computer and is used to execute the structured light three-dimensional imaging method of any one of the above embodiments.

[0123] also, Figure 3 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention.

[0124] The embodiment of the present invention further provides an electronic device, such as Figure 3 The electronic device includes a processor 302, a memory 303, an input unit 304, a display unit 305 and other devices. Those skilled in the art will understand that Figure 3The electronic device structure device shown does not constitute a limitation on all devices, and may include more or fewer components than shown, or combine certain components. The memory 303 can be used to store the application 301 and various functional modules, and the processor 302 runs the application 301 stored in the memory 303, thereby executing various functional applications and data processing of the device. The memory can be an internal memory or an external memory, or include both internal and external memories. The internal memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, or a random access memory. The external memory may include a hard disk, a floppy disk, a ZIP disk, a U disk, a magnetic tape, etc. The memory disclosed in the present invention includes but is not limited to these types of memories. The memory disclosed in the present invention is only used as an example and not as a limitation.

[0125] The input unit 304 is used to receive the input of the signal and the keyword input by the user. The input unit 304 may include a touch panel and other input devices. The touch panel can collect the touch operation of the user on or near it, and drive the corresponding connection device according to a pre-set program; other input devices may include but are not limited to one or more of a physical keyboard, a function key (such as a play control key, a switch key, etc.), a trackball, a mouse, a joystick, etc. The display unit 305 can be used to display the information input by the user or the information provided to the user and various menus of the terminal device. The display unit 305 can be in the form of a liquid crystal display, an organic light emitting diode, etc. The processor 302 is the control center of the terminal device, which uses various interfaces and lines to connect various parts of the entire device, and executes various functions and processes data by running or executing software programs and / or modules stored in the memory 303, and calling data stored in the memory.

[0126] As an embodiment, the electronic device includes: one or more processors 302, a memory 303, and one or more applications 301, wherein the one or more applications 301 are stored in the memory 303 and are configured to be executed by the one or more processors 302, and the one or more applications 301 are configured to execute the structured light three-dimensional imaging method in any one of the above-mentioned embodiments.

[0127] In an embodiment of the present invention, by performing Gray code encoding on the light streak image data to be processed, and then performing corresponding preprocessing and finally performing decoding processing, absolute phase image data can be obtained, and finally structured light three-dimensional geometric reconstruction processing is performed; high-precision phase values ​​can be quickly obtained, and fast three-dimensional reconstruction can be achieved at the same time; high-precision phase values ​​can be quickly and accurately solved through Gray code encoding and decoding; then high-precision three-dimensional coordinates can be calculated through accurate phase values, thereby achieving fast three-dimensional reconstruction.

[0128] In addition, the above is a detailed introduction to a structured light three-dimensional imaging method based on multi-frequency phase shift and related devices provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A structured light three-dimensional imaging method based on multi-frequency phase shift, characterized in that: The method comprises: Acquire a plurality of light fringe image data to be processed collected under multi-frequency phases, and perform Gray code encoding processing on each pixel point in each of the plurality of light fringe image data to be processed to form a set of phase-shifted light fringe image data; Performing image preprocessing on the set of phase-shifted light fringe image data based on a preset algorithm to obtain a set of preprocessed phase-shifted light fringe image data; Decoding a set of preprocessed phase-shifted light fringe image data based on a preset decoding algorithm to form absolute phase image data; Performing structured light three-dimensional geometric reconstruction processing based on the absolute phase image data to generate a three-dimensional imaging image corresponding to the structured light; The performing structured light three-dimensional geometric reconstruction processing based on the absolute phase image data to generate a three-dimensional imaging image corresponding to the structured light includes: Performing point cloud generation processing based on the absolute phase image data to obtain point cloud data; The point cloud data is processed in sequence by noise reduction, smoothing and hole filling to obtain processed point cloud data; Perform three-dimensional geometric reconstruction on the processed point cloud data based on the surface reconstruction algorithm to obtain a three-dimensional imaging image corresponding to the structured light; The step of performing point cloud generation processing based on the absolute phase image data to obtain point cloud data includes: Obtaining a first coordinate system of a camera and a second coordinate system of a projector corresponding to the absolute phase image data, and converting the first coordinate system and the second coordinate system into a world coordinate system; Calculating the three-dimensional coordinate data of the absolute phase image data in the world coordinate system through the projection model of the camera and the light reflection projection of the projector; Calculating and processing the distance and parallax of the feature points of the absolute phase image data based on the camera parameters of the camera and the light reflection projection of the projector to obtain the parallax data and distance data of the feature points of the absolute phase image data; The disparity data and distance data of the feature points are connected in the three-dimensional coordinate data to generate point cloud data.

2. The structured light three-dimensional imaging method according to claim 1, characterized in that: The step of performing Gray code encoding processing on each pixel point in each of the plurality of light fringe image data to be processed to form a set of phase-shifted light fringe image data comprises: A preset number of different Gray code codes are obtained, and Gray code coding is performed on each pixel point in each of the plurality of light streak image data to be processed based on the different Gray code codes to form a set of phase-shifted light streak image data, wherein the set of phase-shifted light streak image data is a one-dimensional coded image generated by a coded image formed by performing Gray code coding on each pixel point in the plurality of light streak image data to be processed, and a one-bit code is expanded to a column of codes, and the Gray code is increased from top to bottom, and each level is formed by phase-shifted light streak image data composed of a coded projection pattern.

3. The structured light three-dimensional imaging method according to claim 1, characterized in that: The performing image preprocessing on the set of phase-shifted light fringe image data based on a preset algorithm to obtain a set of preprocessed phase-shifted light fringe image data includes: Obtaining a maximum grayscale value and a minimum grayscale value among all pixel grayscale values ​​in a set of phase-shifted light fringe image data, and normalizing the current grayscale value of each phase-shifted light fringe image data in the set of phase-shifted light fringe image data according to the maximum grayscale value and the minimum grayscale value to obtain a set of normalized processed images; A binarization threshold is obtained based on a normalization result of pixel gray values ​​in a set of normalized processed images; A group of normalized processed images are binarized based on the binarization threshold to obtain a group of pre-processed phase-shifted light fringe image data.

4. The structured light three-dimensional imaging method according to claim 3, characterized in that: The step of obtaining a binarization threshold based on a normalization result of pixel grayscale values ​​in a set of normalized processed images comprises: Obtaining a normalized value corresponding to each pixel grayscale value of the pixel grayscale value normalization result; When the normalized value is greater than or equal to 0.5, the binarization threshold is set to 1; when the normalized value is less than 0.5, the binarization threshold is set to 0.

5. The structured light three-dimensional imaging method according to claim 1, characterized in that: The method of decoding a set of preprocessed phase-shifted light fringe image data based on a preset decoding algorithm to form absolute phase image data includes: Based on the three-step phase shift formula, a relative phase calculation process is performed on each pixel value in a set of phase-shifted light fringe image data after preprocessing to obtain the relative phase corresponding to each pixel value in each phase-shifted light fringe image data; Calculate the Gray code encoding value corresponding to each pixel value in a set of preprocessed phase-shifted light fringe image data based on the Gray code decoding formula; Based on the relative phase corresponding to each pixel value and the Gray code encoding value, three-step phase shift calculation is performed using high-frequency fringes and low-frequency fringes to obtain absolute phase image data.

6. A structured light three-dimensional imaging device based on multi-frequency phase shift, characterized in that: The device comprises: An encoding module is used to obtain a plurality of light fringe image data to be processed collected under multi-frequency phases, and perform Gray code encoding processing on each pixel point in each of the plurality of light fringe image data to be processed to form a set of phase-shifted light fringe image data; Preprocessing module: used for performing image preprocessing on the set of phase-shifted light fringe image data based on a preset algorithm to obtain a set of preprocessed phase-shifted light fringe image data; Decoding module: used for decoding a set of pre-processed phase-shifted light fringe image data based on a preset decoding algorithm to form absolute phase image data; A three-dimensional reconstruction module is used to perform three-dimensional geometric reconstruction of structured light based on the absolute phase image data to generate a three-dimensional imaging image corresponding to the structured light; The performing structured light three-dimensional geometric reconstruction processing based on the absolute phase image data to generate a three-dimensional imaging image corresponding to the structured light includes: Performing point cloud generation processing based on the absolute phase image data to obtain point cloud data; The point cloud data is processed in sequence by noise reduction, smoothing and hole filling to obtain processed point cloud data; Perform three-dimensional geometric reconstruction on the processed point cloud data based on the surface reconstruction algorithm to obtain a three-dimensional imaging image corresponding to the structured light; The step of performing point cloud generation processing based on the absolute phase image data to obtain point cloud data includes: Obtaining a first coordinate system of a camera and a second coordinate system of a projector corresponding to the absolute phase image data, and converting the first coordinate system and the second coordinate system into a world coordinate system; Calculating the three-dimensional coordinate data of the absolute phase image data in the world coordinate system through the projection model of the camera and the light reflection projection of the projector; Calculating and processing the distance and parallax of the feature points of the absolute phase image data based on the camera parameters of the camera and the light reflection projection of the projector to obtain the parallax data and distance data of the feature points of the absolute phase image data; The disparity data and distance data of the feature points are connected in the three-dimensional coordinate data to generate point cloud data.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the structured light three-dimensional imaging method as described in any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to: execute the structured light three-dimensional imaging method according to any one of claims 1 to 5.

Images