Three-dimensional reconstruction method and 3D structured light machine

Abstract

The application discloses a kind of three-dimensional reconstruction method and 3D structure light, it is related to three-dimensional reconstruction technical field, it is difficult to obtain accurate circle point edge in circle point calibration process, resulting in the technical problem that three-dimensional reconstruction effect is not ideal.The method includes S1, using the circle point calibration method based on ENet model to carry out camera calibration, obtains camera internal and external parameters;S2, using 16 step phase shift method to carry out projector calibration, obtains the internal and external parameters of projector;Wherein, using the grating compensation algorithm based on binary stripe to carry out grating compensation in projection process;S3, according to the internal and external parameters of the camera, projector, the conversion relationship of image pixel coordinate system and world coordinate system is established, according to the conversion relationship obtains the three-dimensional coordinate information of target object, completes three-dimensional reconstruction.The application extracts accurate circle point edge by establishing ENet model, greatly improves the accuracy of three-dimensional reconstruction.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, and in particular to a three-dimensional reconstruction method and 3D structured light. Background Technology

[0002] 3D reconstruction refers to establishing a mathematical model of a 3D object that is suitable for computer representation and processing. It is the foundation for processing, manipulating, and analyzing the properties of the object in a computer environment. It is also a key technology for creating virtual reality that expresses the objective world in a computer. It is a common scientific problem and core technology in fields such as computer-aided geometric design, computer graphics, computer animation, computer vision, medical image processing, scientific computing, virtual reality, and digital media creation. It includes steps such as image acquisition, calibration, feature extraction, stereo matching, and reconstruction.

[0003] Structured light is a system consisting of a projector and a camera. Specific light information is projected onto the surface of an object and its background by the projector, and then captured by the camera. Based on the changes in the light signal caused by the object, information such as the object's position and depth is calculated, thus reconstructing the entire three-dimensional space. Common structured light systems include 3D scanners used to create point clouds of object geometric surfaces.

[0004] The role of a camera is to transform the real three-dimensional world into a two-dimensional planar world. We can think of a camera as a function: the input is a three-dimensional scene, and the output is a two-dimensional image. Camera calibration involves calculating camera parameters by inputting a calibration plate with a calibration pattern, using a simple mathematical model to express the complex imaging process. Solving this mathematical model means solving for the camera's parameters, including its intrinsic, extrinsic, and distortion parameters. With this mathematical model, we can correct distortion in images captured by the camera, perform 3D reconstruction using images from multiple cameras, measure object distances, and perform other computer vision applications. The dot-matrix calibration method is commonly used for camera calibration.

[0005] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art:

[0006] During the dot calibration process, it is difficult to obtain accurate dot edges, resulting in unsatisfactory 3D reconstruction results. Summary of the Invention

[0007] The purpose of this invention is to provide a 3D reconstruction method and 3D structured light to solve the technical problem in the prior art where it is difficult to obtain accurate dot edges during dot calibration, resulting in unsatisfactory 3D reconstruction effects. The various technical effects of the preferred solutions among the many technical solutions provided by this invention are detailed below.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] The present invention provides a three-dimensional reconstruction method, comprising:

[0010] S1. Camera calibration is performed using the dot-matrix calibration method based on the ENet model to obtain the camera's intrinsic and extrinsic parameters;

[0011] S2. The projector is calibrated using a 16-step phase-shifting method to obtain the projector's internal and external parameters; among them, a grating compensation algorithm based on binarized fringes is used for grating compensation during the projection process.

[0012] S3. Based on the internal and external parameters of the camera and projector, establish the transformation relationship between the image pixel coordinate system and the world coordinate system, obtain the three-dimensional coordinate information of the target object based on the transformation relationship, and complete the three-dimensional reconstruction.

[0013] Preferably, step S1 includes:

[0014] S11. Lightweight edge detection based on the ENet model is used to obtain the edge points of the calibration circle on the calibration board;

[0015] S12. Obtain the center coordinates of the calibration circle point based on the edge point;

[0016] S13. Obtain the internal and external parameters of the camera based on the coordinates of the center of the circle.

[0017] Preferably, step S11 includes:

[0018] S111. Take a batch of calibration plate images in different poses, manually mark the edges of the calibration points on the calibration plate and convert them into binary images Y;

[0019] S112. The binary image Y is trained using the ENet model to obtain the feature map S and the edge prediction map E;

[0020] S113. Calculate the loss function of the binary image Y and the feature map S, and calculate the loss function of the binary image Y and the edge prediction map E, and assign them different weights.

[0021] S114. Based on the loss function value, the stochastic echelon optimization algorithm is used for iterative optimization. When the loss function value is lower than the threshold δ, training is stopped, and the trained ENet model is obtained.

[0022] S115. Input the calibration image to be detected into the trained ENet model to obtain the edge points of the calibration dots.

[0023] Preferably, step S2 includes:

[0024] S21. Project a set of gratings onto the dot calibration plate and capture the grating images with a camera.

[0025] S22. The principal phase of the grating image is solved by the 16-step phase shift method, and grating compensation is performed by the grating compensation algorithm based on binarized fringes.

[0026] S23. Project another set of auxiliary gratings to obtain the auxiliary grating image. Perform phase expansion on the principal value phase using the auxiliary grating image to obtain the absolute phase.

[0027] S24. Obtain the internal and external parameters of the projector based on the absolute phase.

[0028] Preferably, the grating compensation in step S22 includes:

[0029] S221. Add a completely black image during the projection process. dark Simultaneously, the mean image I of the raster image is obtained. mean ;

[0030] S222, Based on the completely black image I dark Mean Image I mean Synthetic all-white image I white ;

[0031] S223. Obtain the completely black image I. dark All-white image I white The mean value is used to obtain the threshold image;

[0032] S224. Perform raster compensation based on the threshold image.

[0033] Preferably, the synthesized all-white image I white The formula is:

[0034]

[0035] Among them, I frige This refers to a sinusoidal fringe image from the set of raster images.

[0036] Preferably, the grating consists of 16 transverse phase-shift codes and 16 longitudinal phase-shift codes; the auxiliary grating consists of 7 transverse Gray codes and 6 longitudinal Gray codes.

[0037] A 3D structured light engine, characterized in that it uses the three-dimensional reconstruction method described above for three-dimensional reconstruction.

[0038] Implementing one of the above-described technical solutions of the present invention has the following advantages or beneficial effects:

[0039] By establishing the ENet model, precise edge pixels of the dots were obtained, enabling accurate camera intrinsic and extrinsic parameters. Using a structured light system for 3D reconstruction, since the projector always receives binary images rather than grayscale images, the exposure time when the camera acquires data for each image channel can be set much shorter than the channel duration, allowing the measurement speed to reach the projector's maximum refresh rate, thus significantly improving measurement speed. Calibration was performed using a phase-shifting method supplemented by Gray code, employing 16 horizontal phase-shifted images + 7 horizontal Gray code images + 16 vertical phase-shifted images + 6 vertical Gray code images + 1 completely black image. This not only overcame the influence of ambient light but also reduced calibration errors, improving the quality of the 3D reconstruction. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0041] Figure 1 This is a flowchart of a three-dimensional reconstruction method according to an embodiment of the present invention;

[0042] Figure 2 This is a flowchart of step S1 of a three-dimensional reconstruction method according to an embodiment of the present invention;

[0043] Figure 3 This is a flowchart of step S11 of a three-dimensional reconstruction method according to an embodiment of the present invention;

[0044] Figure 4 This is a flowchart of step S2 of a three-dimensional reconstruction method according to an embodiment of the present invention;

[0045] Figure 5 This is a flowchart of step S22 of a three-dimensional reconstruction method according to an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, various exemplary embodiments described below will be referenced to the accompanying drawings, which form part of the exemplary embodiments, illustrating various exemplary embodiments that may be used to implement the present invention. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. It should be understood that they are merely examples of processes, methods, and apparatuses consistent with some aspects of the present invention disclosed as detailed in the appended claims, and other embodiments may be used, or structural and functional modifications may be made to the embodiments listed herein without departing from the scope and spirit of the present invention.

[0047] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," etc., indicate the orientation or positional relationship based on the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the referred element must have a specific orientation, or be constructed and operated in a specific orientation. The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. The term "multiple" means two or more. The terms "connected" and "linked" should be interpreted broadly, for example, they can be fixed connections, detachable connections, integral connections, mechanical connections, electrical connections, communication connections, direct connections, indirect connections through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more of the related listed items. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0048] To illustrate the technical solution described in this invention, specific embodiments are described below, showing only the parts related to the embodiments of this invention.

[0049] Example 1:

[0050] like Figure 1 As shown, the present invention provides a three-dimensional reconstruction method, including:

[0051] S1. Camera calibration is performed using the dot-matrix calibration method based on the ENet model to obtain the camera's intrinsic and extrinsic parameters;

[0052] S2. The projector is calibrated using a 16-step phase-shifting method to obtain the projector's internal and external parameters; among them, a grating compensation algorithm based on binarized fringes is used for grating compensation during the projection process.

[0053] S3. Based on the internal and external parameters of the camera and projector, establish the transformation relationship between the image pixel coordinate system and the world coordinate system, obtain the three-dimensional coordinate information of the target object according to the transformation relationship, and complete the three-dimensional reconstruction.

[0054] This embodiment uses a structured light system for 3D reconstruction. First, the camera's intrinsic and extrinsic parameters are obtained through camera calibration. During this process, a dot-matrix calibration method based on the ENet model is used for camera calibration, which greatly improves the accuracy. Then, a phase-shifting method combined with Gray code is used for projector calibration, which not only overcomes the influence of ambient light but also reduces calibration errors and improves the quality of 3D reconstruction.

[0055] As an alternative implementation, step S1 includes:

[0056] S11. Lightweight edge detection based on the ENet model is used to obtain the edge points of the calibration circle on the calibration board; this step includes:

[0057] S111. Take a batch of calibration plate images in different poses, manually mark the edges of the calibration points on the calibration plate and convert them into binary images Y;

[0058] S112. The binary image Y is trained using the ENet model to obtain the feature map S and the edge prediction map E.

[0059] S113. Calculate the loss function of binary image Y and feature map S, and calculate the loss function of binary image Y and edge prediction map E, and assign them different weights.

[0060] S114. Based on the loss function value, the stochastic echelon optimization algorithm is used for iterative optimization. When the loss function value is lower than the threshold δ, training is stopped, and the trained ENet model is obtained.

[0061] S115. Input the calibration image to be detected into the trained ENet model to obtain the edge points of the calibration circles.

[0062] S12. Obtain the center coordinates of the calibration circle based on the edge points;

[0063] S13. Obtain the intrinsic and extrinsic parameters of the camera based on the coordinates of the center of the circle.

[0064] As an optional implementation, step S2 includes:

[0065] S21. Project a set of gratings onto the dot calibration plate, and capture the grating images with a camera; obtain the projected positive phase shift code: Among them, I n (x,y) represents the grayscale value of the nth image, A(x,y) represents the background value of the stripe intensity, and B(x,y) represents the modulation intensity. Let (x, y) be the principal phase distribution to be determined, and (x, y) be the pixel coordinates in the projector.

[0066] S22. The principal phase of the grating image is solved using a 16-step phase-shifting method. The principal phase distribution is as follows: A grating compensation algorithm based on binarized fringes is used for grating compensation; this step of grating compensation includes:

[0067] S221. Add a completely black image during the projection process. dark Simultaneously, the mean image I of the raster image is obtained. mean ;

[0068] S222, Based on the completely black image I dark Mean Image I mean Synthetic all-white image I white ;

[0069] S223, Obtain the completely black image I dark All-white image I white The mean value is used to obtain the threshold image;

[0070] S224. Perform raster compensation based on the threshold image. This raster compensation is for the auxiliary raster image, which can reduce the influence of ambient light on projector calibration and obtain more accurate internal and external parameters of the projector.

[0071] S23. Project another set of auxiliary gratings to obtain the auxiliary grating image. Perform phase expansion on the principal phase using the auxiliary grating image to obtain the absolute phase. The absolute phase is: θ(x,y)=φ(x,y)+2k(x,y)π;

[0072] S24. Obtain the internal and external parameters of the projector based on the absolute phase.

[0073] Among them, the synthesized all-white image I white The formula is:

[0074]

[0075] Among them, I frige This is a sinusoidal fringe image from a set of 16 raster images; therefore, the mean image...

[0076] The grating consists of 16 transverse phase-shift codes and 16 longitudinal phase-shift codes; the auxiliary grating consists of 7 transverse Gray codes and 6 longitudinal Gray codes.

[0077] This embodiment obtains accurate dot edge pixels by establishing an ENet model, thereby acquiring precise camera intrinsic and extrinsic parameters. Using a structured light system for 3D reconstruction, since the projector always receives binary images rather than grayscale images, the exposure time when the camera acquires data for each image channel can be set much shorter than the channel's duration, allowing the measurement speed to reach the projector's maximum refresh rate, resulting in a significant improvement in measurement speed. Calibration is performed using a phase-shifting method supplemented by Gray code, employing 16 horizontal phase-shifted codes + 7 horizontal Gray code images + 16 vertical phase-shifted codes + 6 vertical Gray code images + 1 completely black image. This not only overcomes the influence of ambient light but also reduces calibration errors and improves the quality of 3D reconstruction.

[0078] Example 2:

[0079] The difference between Embodiment 2 and Embodiment 1 lies in the following: a 3D structured light engine, characterized by employing the aforementioned 3D reconstruction method for 3D reconstruction. This 3D structured light engine can obtain precise intrinsic and extrinsic parameters of the camera and projector during the 3D reconstruction process, thereby achieving high-precision 3D reconstruction.

[0080] The above description is merely a preferred embodiment of the present invention. Those skilled in the art will understand that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A three-dimensional reconstruction method, characterized in that, include: S1. Camera calibration is performed using the dot-matrix calibration method based on the ENet model to obtain the camera's intrinsic and extrinsic parameters; S2. The projector is calibrated using a 16-step phase-shifting method to obtain the projector's internal and external parameters; among them, a grating compensation algorithm based on binarized fringes is used for grating compensation during the projection process. S3. Based on the internal and external parameters of the camera and projector, establish the transformation relationship between the image pixel coordinate system and the world coordinate system, obtain the three-dimensional coordinate information of the target object based on the transformation relationship, and complete the three-dimensional reconstruction. Step S1 includes: S11. Lightweight edge detection based on the ENet model is used to obtain the edge points of the calibration circle on the calibration board; S12. Obtain the center coordinates of the calibration point based on the edge; S13. Obtain the internal and external parameters of the camera based on the coordinates of the center of the circle; Step S11 includes: S111. Take a batch of calibration plate images in different poses, manually mark the edges of the calibration points on the calibration plate and convert them into binary images Y; S112. The binary image Y is trained using the ENet model to obtain the feature map S and the edge prediction map E; S113. Calculate the loss function of the binary image Y and the feature map S, and calculate the loss function of the binary image Y and the edge prediction map E, and assign them different weights. S114. Based on the loss function value, the stochastic echelon optimization algorithm is used for iterative optimization. When the loss function value is lower than the threshold δ, training is stopped, and the trained ENet model is obtained. S115. Input the calibration image to be detected into the trained ENet model to obtain the edge points of the calibration dots.

2. The three-dimensional reconstruction method according to claim 1, characterized in that, Step S2 includes: S21. Project a set of gratings onto the dot calibration plate and capture the grating images with a camera. S22. The principal phase of the grating image is solved by the 16-step phase shift method, and grating compensation is performed by the grating compensation algorithm based on binarized fringes. S23. Project another set of auxiliary gratings to obtain an auxiliary grating image. Perform phase expansion on the principal value phase using the auxiliary grating image to obtain the absolute phase. S24. Obtain the internal and external parameters of the projector based on the absolute phase.

3. The three-dimensional reconstruction method according to claim 2, characterized in that, Step S22 includes grating compensation, which includes: S221. Add a completely black image during the projection process. dark Simultaneously, the mean image I of the raster image is obtained. mean ; S222, Based on the completely black image I dark Mean Image I mean Synthetic all-white image I white ; S223. Obtain the completely black image I. dark White Image I white The mean value is used to obtain the threshold image; S224. Perform raster compensation based on the threshold image.

4. The three-dimensional reconstruction method according to claim 3, characterized in that, Synthesize the all-white image I white The formula is: in, This refers to a sinusoidal fringe image from the set of raster images.

5. A three-dimensional reconstruction method according to claim 4, characterized in that, The grating consists of 16 transverse phase-shift codes and 16 longitudinal phase-shift codes; the auxiliary grating consists of 7 transverse Gray codes and 6 longitudinal Gray codes.

6. A 3D structured light engine, characterized in that, Three-dimensional reconstruction is performed using a three-dimensional reconstruction method as described in claims 1-5.

Images