Three-dimensional model construction method, device, terminal and computer-readable storage medium

Abstract

The present invention provides a three-dimensional model construction method, device, terminal and computer-readable storage medium. The three-dimensional model construction method includes: obtaining a grating image corresponding to a target object; inputting the grating image into a pre-trained deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image; determining the corresponding number of steps according to the wrapped phase information and the depth information; and constructing a three-dimensional model of the target object according to the wrapped phase information and the number of steps. The present application detects the grating image through a deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image, thereby realizing three-dimensional reconstruction of the target object based on a single grating image, and improving the accuracy of the three-dimensional model of the target object; by determining the corresponding number of steps through the wrapped phase information and the depth information, the speed and accuracy of reconstructing the three-dimensional model of the target object can be further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of structured light three-dimensional imaging, and in particular to a three-dimensional model construction method, device, terminal and computer-readable storage medium. Background Art

[0002] With the continuous development of video technology, a large part of the research in the field of computer vision is about how to convert the acquired two-dimensional images into three-dimensional information. The three-dimensional reconstruction method based on structured light has the advantages of high precision, non-contact and low cost, and has been widely used in various fields. In the process of three-dimensional reconstruction using traditional structured light methods, the speed of three-dimensional reconstruction is greatly reduced due to the need to shoot multiple stripe grating images. Summary of the invention

[0003] The main technical problem solved by the present invention is to provide a three-dimensional model construction method, device, terminal and computer-readable storage medium to solve the problem of low accuracy in the construction of three-dimensional models of target objects in the prior art.

[0004] To solve the above technical problems, the first technical solution adopted by the present invention is: to provide a three-dimensional model construction method, the three-dimensional model construction method comprising: obtaining a grating image corresponding to the target object; inputting the grating image into a pre-trained deep learning network to obtain the package phase information and depth information corresponding to the grating image; determining the corresponding number of steps according to the package phase information and the depth information; and constructing a three-dimensional model of the target object according to the package phase information and the number of steps.

[0005] The method of obtaining the grating image corresponding to the target object also includes: constructing a structured light measurement system; and calibrating the structured light measurement system to obtain calibration data of the structured light detection system.

[0006] The method further includes: constructing an initial deep learning network; obtaining a first data set and a second data set; wherein the first data set includes a plurality of first grating images and real wrapped phase information corresponding to the first grating images; and the second data set includes a plurality of second grating images and real depth information corresponding to each second grating image;

[0007] The first data set and the second data set are input into an initial deep learning network to train the initial deep learning network to obtain a deep learning network.

[0008] Among them, obtaining the first data set includes: obtaining a first sample set, the first sample set including multiple first grating images; performing phase extraction on the multiple first grating images respectively to obtain real package phase information corresponding to each first grating image; and constructing the first data set based on each first grating image included in the first sample set and the corresponding real package phase information.

[0009] Wherein, obtaining the second data set includes: obtaining a second sample set, the second sample set including a plurality of second grating images; performing phase extraction on each second grating image to obtain real package phase information corresponding to each second grating image; determining real depth information corresponding to the second grating image according to the real package phase information; and constructing the second data set based on each second grating image contained in the second sample set and the real depth information corresponding to each second grating image.

[0010] Among them, the deep learning network includes a first deep learning network model; the grating image is input into a pre-trained deep learning network to obtain the package phase information and depth information corresponding to the grating image, including: inputting the grating image into a pre-trained first deep learning network model to obtain the package phase information of the grating image.

[0011] Among them, the step of training to obtain the first deep learning network model includes: obtaining a first sample set, the first sample set includes multiple first grating images, and the first grating images are annotated with real wrapped phase information; predicting the first grating image through a first neural network to obtain predicted wrapped phase information corresponding to the first grating image; constructing a first loss function based on the predicted wrapped phase information and the real wrapped phase information corresponding to the same first grating image; iteratively training the first neural network according to the first loss function to obtain the first deep learning network model.

[0012] Among them, the deep learning network includes a second deep learning network model; the grating image is input into a pre-trained deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image, including: inputting the grating image into a pre-trained second deep learning network model to obtain the depth information of the grating image.

[0013] Among them, the step of training to obtain a second deep learning network model includes: obtaining a second sample set, the second sample set includes multiple second grating images, and the second grating images are annotated with corresponding real depth information; predicting the second grating image through a second neural network to obtain predicted depth information of the second grating image; constructing a second loss function based on the predicted depth information and real depth information corresponding to the same second grating image; iteratively training the second neural network according to the second loss function to obtain a second deep learning network model.

[0014] Among them, the depth information is a depth image; according to the wrapped phase information and the depth information, the corresponding step number is determined, including: according to the depth image of the grating image, the corresponding three-dimensional point cloud is determined; according to the wrapped phase information and the three-dimensional point cloud, the corresponding step number is determined.

[0015] Wherein, determining the corresponding three-dimensional point cloud according to the depth image of the grating image includes: converting the pixel coordinates in the depth image into the corresponding three-dimensional point cloud by formula 1;

[0016]

[0017] In formula 1: (x w ,y w , z w ) is the coordinate of the three-dimensional world point; (u0, v0) is the pixel coordinate of the intersection of the image plane and the camera optical axis, that is, the coordinate of the center of the computer image; f x With f y is the focal length of the camera lens, in mm.

[0018] Among them, the corresponding number of steps is determined according to the wrapped phase information and the three-dimensional point cloud, including: determining the initial absolute phase of the grating image according to the three-dimensional point cloud of the grating image and the calibration data of the structured light measurement system; determining the corresponding number of steps based on the initial absolute phase of the grating image and the wrapped phase information corresponding to the grating image.

[0019] Among them, constructing a three-dimensional model of the target object according to the wrapped phase information and the number of steps includes: determining the corresponding absolute phase according to the number of steps and the wrapped phase information corresponding to the grating image; and constructing the three-dimensional model of the target object based on the absolute phase of the grating image.

[0020] Wherein, determining the corresponding absolute phase according to the step number and wrapped phase information corresponding to the grating image includes: determining the absolute phase of the grating image based on Formula 2;

[0021]

[0022] In formula 2: Ф pre is the absolute phase; is the package phase information; N pre is the number of steps.

[0023] In order to solve the above technical problems, the second technical solution adopted by the present invention is: to provide a three-dimensional model construction device, the three-dimensional model construction device includes: an acquisition module, used to acquire a grating image corresponding to the target object; a detection module, used to input the grating image into a pre-trained deep learning network to obtain the package phase information and depth information corresponding to the grating image; an analysis module, used to determine the corresponding number of steps according to the package phase information and depth information; a construction module, used to construct a three-dimensional model of the target object according to the package phase information and the number of steps.

[0024] To solve the above technical problems, the third technical solution adopted by the present invention is: to provide a terminal, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor is used to execute program data to implement the steps in the above three-dimensional model construction method.

[0025] In order to solve the above technical problems, the fourth technical solution adopted by the present invention is: providing a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above three-dimensional model construction method are implemented.

[0026] The beneficial effects of the present invention are as follows: Different from the prior art, a three-dimensional model construction method, device, terminal and computer-readable storage medium are provided, and the three-dimensional model construction method includes: obtaining a grating image corresponding to the target object; inputting the grating image into a pre-trained deep learning network to obtain the package phase information and depth information corresponding to the grating image; determining the corresponding number of steps according to the package phase information and the depth information; and constructing a three-dimensional model of the target object according to the package phase information and the number of steps. The present application detects the grating image through a deep learning network to obtain the package phase information and depth information corresponding to the grating image, thereby realizing three-dimensional reconstruction of the target object based on a single grating image, and improving the accuracy of the three-dimensional model of the target object; determining the corresponding number of steps according to the package phase information and the depth information; and then constructing the three-dimensional model of the target object according to the package phase information and the number of steps, which can further improve the speed and accuracy of reconstructing the three-dimensional model of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 It is a schematic diagram of the process of the three-dimensional model construction method provided by the present invention;

[0029] Figure 2 It is a flowchart of a specific embodiment of the three-dimensional model construction method provided by the present invention;

[0030] Figure 3 yes Figure 2 The grating image and the corresponding wrapped phase image in S203 of the provided three-dimensional model construction method;

[0031] Figure 4 yes Figure 2A schematic diagram of a process flow of a specific embodiment of S204 in the provided three-dimensional model building method;

[0032] Figure 5 yes Figure 2 The grating image and the corresponding depth map in S205 of the provided three-dimensional model construction method;

[0033] Figure 6 yes Figure 2 A schematic diagram of a process flow of a specific embodiment of S206 in the provided three-dimensional model building method;

[0034] Figure 7 yes Figure 2 A schematic diagram of an application embodiment of the provided three-dimensional model construction method;

[0035] Figure 8 yes Figure 2 A depth map corresponding to the raster image in S209 in the provided three-dimensional model construction method;

[0036] Fig. 9 yes Figure 2 The number of steps corresponding to the grating image in S211 in the provided three-dimensional model construction method;

[0037] Fig.10 yes Figure 2 The three-dimensional model of the target object in S213 of the provided three-dimensional model building method;

[0038] Fig.11 is a schematic block diagram of a three-dimensional model building device provided by the present invention;

[0039] Fig.12 is a schematic block diagram of an implementation manner of a terminal provided by the present invention;

[0040] Fig.13 It is a schematic block diagram of an embodiment of a computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0041] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0042] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0043] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0044] In order to enable those skilled in the art to better understand the technical solution of the present invention, a three-dimensional model construction method provided by the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0045] See also Figure 1 , Figure 1 Schematic diagram of the process of the three-dimensional model construction method provided by the present invention. In this embodiment, a three-dimensional model construction method is provided, and the three-dimensional model construction method includes the following steps.

[0046] S11: Obtain a raster image corresponding to the target object.

[0047] In one embodiment, a structured light measurement system may be pre-built; the structured light measurement system may be calibrated to obtain calibration data of the structured light detection system.

[0048] Specifically, the target object is placed on a reference plane, and any one of the standard N-step phase-shift gratings is projected onto the target object by a projection device, and a deformed fringe image after surface modulation of the target object is acquired by an image acquisition device, and the obtained deformed fringe image is used as a grating image of the target object. The target object may be a human face, a human body, or various objects, etc., and no limitation is imposed here.

[0049] S12: Input the grating image into a pre-trained deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image.

[0050] Specifically, the deep learning network includes a first deep learning network model and a second deep learning network model. The grating image is input into the pre-trained first deep learning network model to obtain the wrapped phase information of the grating image. The grating image is input into the pre-trained second deep learning network model to obtain the depth information of the grating image.

[0051] S13: Determine the corresponding number of steps according to the wrapped phase information and depth information.

[0052] Specifically, the depth information is a depth image. The corresponding three-dimensional point cloud is determined according to the depth image of the grating image; and the corresponding step number is determined according to the wrapped phase information and the three-dimensional point cloud. The step number represents the number of grating fringe periods of each pixel constituting the target object.

[0053] In a specific embodiment, the pixel coordinates in the depth image are converted into the corresponding three-dimensional point cloud using Formula 1.

[0054]

[0055] Where: (x w ,y w , z w ) is the coordinate of the three-dimensional world point; (u0, v0) is the pixel coordinate of the intersection of the image plane and the camera optical axis, that is, the coordinate of the center of the computer image; f x With f y is the focal length of the camera lens, in mm.

[0056] In a specific embodiment, the initial absolute phase of the grating image is determined based on the three-dimensional point cloud of the grating image and the calibration data of the structured light measurement system; the corresponding step level is determined based on the initial absolute phase of the grating image and the wrapped phase information corresponding to the grating image.

[0057] S14: constructing a three-dimensional model of the target object according to the wrapped phase information and the number of steps.

[0058] Specifically, the corresponding absolute phase is determined according to the step number and wrapped phase information corresponding to the grating image; and the three-dimensional model of the target object is constructed based on the absolute phase of the grating image.

[0059] In a specific embodiment, the absolute phase Φ of the grating image is determined based on Formula 2 pre .

[0060]

[0061] Where: Ф pre is the absolute phase; is the package phase information; N pre is the number of steps.

[0062] Specifically, the absolute phase of each pixel is determined according to the number of grating fringe periods of each pixel and the predicted wrapped phase information, so that each pixel has a unique absolute phase, thereby improving the accuracy of the constructed three-dimensional model corresponding to the target object.

[0063] The three-dimensional model construction method provided in this embodiment includes obtaining a grating image corresponding to the target object; inputting the grating image into a pre-trained deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image; determining the corresponding number of steps according to the wrapped phase information and the depth information; and constructing a three-dimensional model of the target object according to the wrapped phase information and the number of steps. The present application detects the grating image through a deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image, thereby realizing three-dimensional reconstruction of the target object based on a single grating image, and improving the accuracy of the three-dimensional model of the target object; determining the corresponding number of steps according to the wrapped phase information and the depth information; and then constructing the three-dimensional model of the target object according to the wrapped phase information and the number of steps, which can further improve the speed and accuracy of constructing the three-dimensional model of the target object.

[0064] See also Figure 2 , Figure 2 1 is a flow chart of a specific embodiment of a three-dimensional model construction method provided by the present invention. In this embodiment, a three-dimensional model construction method is provided, and the three-dimensional model construction method includes the following steps.

[0065] S201: Build a structured light measurement system.

[0066] Specifically, a projection device and an image acquisition device are prepared, and a structured light measurement system is constructed by the projection device, the image acquisition device and the reference plane. The projection device may be a projector to project an N-step phase-shifted grating onto a target object to be constructed into a three-dimensional model. The image acquisition device may be a camera to capture an image of deformation stripes on the surface of the target object. In this embodiment, there is one projector and one camera respectively.

[0067] S202: Calibrate the structured light measurement system to obtain calibration data of the structured light detection system.

[0068] Specifically, the intrinsic and extrinsic parameters of the projection device in the structured light measurement system are calibrated, and the intrinsic and extrinsic parameters of the camera are calibrated. In a specific embodiment, the parallel matrix and rotation matrix of the camera are calibrated, the components of the focal length of the camera in the X-axis and Y-axis directions are calibrated, and the distortion parameters of the camera are calibrated. The positional relationship of the camera and the projection device relative to the reference plane is calibrated, and a coordinate system can be established with the camera or the projector as the coordinate origin.

[0069] S203: Acquire a first data set.

[0070] See also Figure 3 , Figure 3 yes Figure 2 The grating image and the corresponding wrapped phase map in S203 in the provided three-dimensional model construction method.

[0071] Specifically, a first sample set is obtained, the first sample set including a plurality of first grating images, such as Figure 3 The plurality of first grating images may be a standard N-step phase-shifted grating projected by a projection device onto a preset target placed on a reference plane, and a camera may be used to collect N deformed grating images modulated by the surface of the preset target, and the deformed grating images may be used as the first grating images.

[0072] Phase extraction is performed on multiple first grating images respectively to obtain the real wrapped phase image corresponding to each first grating image, such as Figure 3 (a') and (b'). Specifically, a corresponding phase deconvolution algorithm is performed on the first grating image to obtain the real wrapped phase information corresponding to the first grating image. The phase deconvolution algorithm can be a phase shift method, a Fourier transform method phase deconvolution and a convolution method.

[0073] A first data set is constructed based on each first grating image included in the first sample set and the corresponding real wrapped phase information, wherein in the first data set, each first grating image corresponds to a real wrapped phase information.

[0074] S204: Train to obtain a first deep learning network model.

[0075] See also Figure 4 , Figure 4 yes Figure 2 A flowchart of a specific embodiment of S204 in the three-dimensional model building method is provided.

[0076] Specifically, the specific steps of training the first neural network to obtain the first deep learning network model are as follows.

[0077] S2041: Obtain a first sample set.

[0078] Specifically, the first sample set includes a plurality of first grating images, each of which is annotated with corresponding real wrapped phase information, wherein the first grating image is annotated with a real wrapped phase map.

[0079] S2042: Predicting the first grating image through the first neural network to obtain predicted wrapped phase information corresponding to the first grating image.

[0080] Specifically, the first grating image is input into the first neural network, and the first neural network learns the first grating image and predicts the predicted wrapped phase information of the first grating image, wherein the predicted wrapped phase information is a predicted wrapped phase map.

[0081] S2043: Construct a first loss function based on the predicted wrapped phase information and the real wrapped phase information corresponding to the same first grating image.

[0082] Specifically, a first loss function is constructed by the error between the predicted wrapped phase image and the true wrapped phase image corresponding to each first grating image, wherein the first loss function may be a cross-entropy loss.

[0083] S2044: Iteratively train the first neural network according to the first loss function to obtain a first deep learning network model.

[0084] Specifically, the first neural network is iteratively trained based on the error value between the predicted wrapped phase information and the actual wrapped phase information corresponding to the same first grating image to obtain the first deep learning network model.

[0085] In an optional embodiment, the result of the first neural network is back-propagated, and the weights of the first neural network are modified according to the loss value fed back by the first loss function. In other words, the weights in the first neural network are modified according to the loss value fed back by the first loss function to implement the training of the first neural network.

[0086] The first grating image is input into the first neural network, and the first neural network predicts the first grating image. When the error value between the predicted wrapped phase information and the real wrapped phase information corresponding to the same first grating image is less than a preset threshold value, which can be set by itself, such as 1%, 5%, etc., the training of the first neural network is stopped and the first deep learning network model is obtained.

[0087] By training the first neural network to obtain the first deep learning network model, the detection accuracy of the network can be increased, and the detection accuracy of the wrapped phase image corresponding to the grating image can be improved.

[0088] S205: Acquire a second data set.

[0089] See also Figure 5 , Figure 5 yes Figure 2 The grating image and the corresponding depth map in S205 in the provided three-dimensional model construction method.

[0090] Specifically, a second sample set is obtained, the second sample set includes a plurality of second grating images, such as Figure 5 (a) and (b); performing phase extraction on each second grating image to obtain the real wrapped phase information corresponding to each second grating image. In another optional embodiment, the second grating image may be the first grating image.

[0091] According to the real wrapped phase information, the real depth information corresponding to the second grating image is determined. In a specific embodiment, a three-dimensional point cloud of the second grating image is determined based on the real wrapped phase information by a three-dimensional reconstruction algorithm, and the three-dimensional point cloud is converted into a two-dimensional depth map according to the calibration data of the structured light detection system, such as Figure 5 (a') and (b'), that is, the real depth information of the second grating image is obtained. A second data set is constructed based on each second grating image included in the second sample set and the corresponding real depth information. In the second data set, each second grating image corresponds to a real depth information.

[0092] S206: Train to obtain a second deep learning network model.

[0093] See also Figure 6 , Figure 6 yes Figure 2 A flowchart of a specific embodiment of S206 in the three-dimensional model building method is provided.

[0094] Specifically, the specific steps of training the second neural network to obtain the second deep learning network model are as follows.

[0095] S2061: Acquire a second sample set.

[0096] Specifically, the second sample set includes a plurality of second grating images, and the second grating images are annotated with corresponding real depth information. The second grating images are annotated with corresponding real depth maps.

[0097] S2062: Predicting the second grating image through a second neural network to obtain predicted depth information of the second grating image.

[0098] Specifically, the second grating image is input into the second neural network, and the second neural network learns the second grating image and predicts the predicted depth information of the second grating image, wherein the predicted depth information is a predicted depth map.

[0099] S2063: Construct a second loss function based on the predicted depth information and the real depth information corresponding to the same second grating image.

[0100] Specifically, the second loss function is constructed by the error between the predicted depth map and the real depth map respectively corresponding to each second grating image. The second loss function may be a cross-entropy loss.

[0101] S2064: Iteratively train the second neural network according to the second loss function to obtain a second deep learning network model.

[0102] Specifically, the second neural network is iteratively trained using the error value between the predicted depth information and the actual depth information corresponding to the same second grating image to obtain a second deep learning network model.

[0103] In an optional embodiment, the result of the second neural network is back-propagated, and the weight of the second neural network is modified according to the loss value fed back by the second loss function. In other words, the weight in the second neural network is modified according to the loss value fed back by the second loss function to implement the training of the second neural network.'

[0104] The second grating image is input into the second neural network, and the second neural network predicts the second grating image. When the error value between the predicted depth information and the real depth information corresponding to the same second grating image is less than a preset threshold value, which can be set by itself, such as 1%, 5%, etc., the training of the second neural network is stopped and a second deep learning network model is obtained.

[0105] By training the second neural network to obtain a second deep learning network model, the detection accuracy of the network can be increased, and the detection accuracy of the depth map corresponding to the grating image can be improved.

[0106] See also Figure 7 , Figure 7 yes Figure 2 A schematic diagram of an application embodiment of the provided three-dimensional model construction method.

[0107] S207: Acquire a raster image corresponding to the target object.

[0108] Specifically, the target object is placed on a reference plane, any one of the standard N-step phase-shift gratings is projected onto the target object by a projection device, and a deformed fringe image after surface modulation of the target object is acquired by an image acquisition device, and the obtained deformed fringe image is used as the grating image of the target object.

[0109] S208: Input the grating image into a pre-trained first deep learning network model to obtain the wrapped phase information of the grating image.

[0110] Specifically, the acquired grating image is input into the first deep learning network model, and the wrapped phase map corresponding to the grating image is detected.

[0111] S209: Input the grating image into a pre-trained second deep learning network model to obtain depth information of the grating image.

[0112] See also Figure 8 , Figure 8 yes Figure 2The depth map corresponding to the grating image in S209 of the provided three-dimensional model construction method. Specifically, the acquired grating image is input into the second deep learning network model, and the depth map corresponding to the grating image is detected and obtained.

[0113] S210: Determine a corresponding three-dimensional point cloud according to the depth image of the grating image.

[0114] Specifically, the coordinates of each pixel in the depth map of the grating image are converted into a three-dimensional point cloud corresponding to the grating image by using the calibration data of the structured light detection system.

[0115] In a specific embodiment, the pixel coordinates in the depth map are converted into a three-dimensional point cloud corresponding to the raster image using Formula 1.

[0116]

[0117] In formula 1: x w ,y w , z w is the coordinate of the three-dimensional world point; (u0, v0) is the pixel coordinate of the intersection of the image plane and the camera optical axis, that is, the coordinate of the center of the computer image; f x With f y is the focal length of the camera lens, in mm.

[0118] Since the three-dimensional point cloud directly predicted based on the depth map of the raster image is insensitive to the number of stripes, the three-dimensional point cloud used to construct the three-dimensional model of the target object has a large error, so the three-dimensional point cloud needs to be corrected.

[0119] S211: Determine the corresponding number of steps according to the wrapped phase information and the three-dimensional point cloud.

[0120] Specifically, the three-dimensional point cloud of the grating image can be determined by the wrapped phase map predicted by the grating image and the initial step order predicted by the grating image, so the fringe order of the grating image can be inversely deduced from the three-dimensional point cloud directly predicted by the depth map and the predicted wrapped phase map. In this embodiment, the fringe order of the grating image can be obtained by the three-dimensional point cloud directly predicted by the depth map and the predicted wrapped phase map without using the minimum phase method.

[0121] In this embodiment, the fringe order is a linear order. In order to reduce the error and make the obtained three-dimensional point cloud more sensitive to the fringe order, the obtained fringe order is converted into a step order. That is, when the fringe order of a pixel in the grating image is 1.5, the fringe order of the pixel is adjusted to 1, thereby making the constructed three-dimensional model of the target object more accurate and the contour of the three-dimensional model of the target object clearer.

[0122] See also Fig. 9 , Fig. 9 yes Figure 2 The number of steps corresponding to the grating image in S211 of the provided three-dimensional model construction method. In a specific embodiment, the initial absolute phase of the grating image is determined based on the three-dimensional point cloud of the grating image and the calibration data of the structured light measurement system; and the corresponding number of steps is determined based on the initial absolute phase of the grating image and the wrapped phase information corresponding to the grating image.

[0123] S212: Determine the corresponding absolute phase according to the step number and wrapped phase information corresponding to the grating image.

[0124] Specifically, the absolute phase of the grating image is determined according to the step number corresponding to the grating image and the corresponding wrapped phase map.

[0125] In a specific embodiment, according to the number of steps corresponding to the grating image and the corresponding wrapped phase map, the absolute phase Φ of the grating image is determined based on Formula 2. pre .

[0126]

[0127] In formula 2: Ф pre is the absolute phase; is the package phase information; N pre is the number of steps.

[0128] S213: Constructing a three-dimensional model of the target object based on the absolute phase of the grating image.

[0129] See also Fig.10 , Fig.10 yes Figure 2 The three-dimensional model of the target object in S213 in the provided three-dimensional model construction method.

[0130] Specifically, a corrected three-dimensional point cloud of the grating image is determined according to the absolute phase of the grating image, and then a three-dimensional model of the target object corresponding to the grating image is constructed according to the corrected three-dimensional point cloud.

[0131] The three-dimensional model construction method provided in this embodiment includes obtaining a grating image corresponding to the target object; inputting the grating image into a pre-trained deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image; determining the corresponding number of steps according to the wrapped phase information and the depth information; and constructing a three-dimensional model of the target object according to the wrapped phase information and the number of steps. The present application detects the grating image through a deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image, thereby realizing three-dimensional reconstruction of the target object based on a single grating image, and improving the accuracy of the three-dimensional model of the target object; determining the corresponding number of steps according to the wrapped phase information and the depth information; and then constructing the three-dimensional model of the target object according to the wrapped phase information and the number of steps, which can further improve the speed and accuracy of constructing the three-dimensional model of the target object.

[0132] See also Fig.11 , Fig.11 3D model building device 60 is provided in this embodiment, and the 3D model building device 60 includes an acquisition module 61 , a detection module 62 , an analysis module 63 and a building module 64 .

[0133] The acquisition module 61 is used to acquire the grating image corresponding to the target object; the detection module 62 is used to input the grating image into a pre-trained deep learning network to obtain the package phase information and depth information corresponding to the grating image; the analysis module 63 is used to determine the corresponding number of steps based on the package phase information and depth information; the construction module 64 is used to construct a three-dimensional model of the target object based on the package phase information and the number of steps.

[0134] In this embodiment, a grating image is detected through a deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image, so that three-dimensional reconstruction of the target object can be achieved based on a single grating image, and the accuracy of the three-dimensional model of the target object is improved; the corresponding number of steps is determined through the wrapped phase information and depth information; and the three-dimensional model of the target object is constructed according to the wrapped phase information and the number of steps, which can further improve the speed and accuracy of constructing the three-dimensional model of the target object.

[0135] See also Fig.12 , Fig.12 It is a schematic block diagram of an embodiment of a terminal provided by the present invention. The terminal 70 in this embodiment includes: a processor 71, a memory 72, and a computer program stored in the memory 72 and executable on the processor 71. When the computer program is executed by the processor 71, the above-mentioned three-dimensional model construction method is implemented. To avoid repetition, it is not described one by one here.

[0136] See also Fig.13 , Fig.13It is a schematic block diagram of an embodiment of a computer-readable storage medium provided by the present invention. In the embodiment of the present application, a computer-readable storage medium 90 is also provided, and the computer-readable storage medium 90 stores a computer program 901, and the computer program 901 includes program instructions. The processor executes the program instructions to implement the three-dimensional model construction method provided in the embodiment of the present application.

[0137] The computer-readable storage medium 90 may be an internal storage unit of the computer device of the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium 90 may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.

[0138] The above are only implementation modes of the present invention, and are not intended to limit the patent protection scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A three-dimensional model construction method, characterized in that: The method comprises: Get the raster image corresponding to the target object; Inputting the grating image into a pre-trained deep learning network to obtain wrapped phase information and depth information corresponding to the grating image; the deep learning network includes a first deep learning network model and a second deep learning network model; Determining a corresponding number of steps according to the wrapped phase information and the depth information; Constructing a three-dimensional model of the target object according to the wrapped phase information and the number of steps; The step of inputting the grating image into a pre-trained deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image comprises: Inputting the grating image into the first pre-trained deep learning network model to obtain the wrapped phase information of the grating image; The grating image is input into the pre-trained second deep learning network model to obtain depth information of the grating image.

2. The three-dimensional model construction method according to claim 1, characterized in that: The step of obtaining a grating image corresponding to the target object further includes: Build a structured light measurement system; The structured light measurement system is calibrated to obtain calibration data of the structured light measurement system.

3. The three-dimensional model construction method according to claim 1, characterized in that: The method further comprises: Build the initial deep learning network; Acquire a first data set and a second data set; wherein the first data set includes a plurality of first grating images and real wrapped phase information corresponding to the first grating images; and the second data set includes a plurality of second grating images and real depth information corresponding to each of the second grating images; The first data set and the second data set are input into the initial deep learning network to train the initial deep learning network to obtain the deep learning network.

4. The three-dimensional model construction method according to claim 3, characterized in that: The obtaining of the first data set includes: Acquire a first sample set, wherein the first sample set includes a plurality of first grating images; Performing phase extraction on the plurality of first grating images respectively to obtain real wrapped phase information corresponding to each of the first grating images; The first data set is constructed based on each of the first grating images included in the first sample set and the respectively corresponding real wrapped phase information.

5. The three-dimensional model construction method according to claim 3, characterized in that: The obtaining of the second data set includes: Acquire a second sample set, wherein the second sample set includes a plurality of second grating images; Performing phase extraction on each of the second grating images to obtain real wrapped phase information corresponding to each of the second grating images; Determining true depth information corresponding to the second grating image according to the true wrapped phase information; The second data set is constructed based on each of the second grating images included in the second sample set and the respectively corresponding real depth information.

6. The three-dimensional model construction method according to claim 1, characterized in that: The step of training to obtain the first deep learning network model includes: Acquire a first sample set, the first sample set comprising a plurality of first grating images, wherein the first grating images are marked with real wrapped phase information; Predicting the first grating image by using a first neural network to obtain predicted wrapped phase information corresponding to the first grating image; Constructing a first loss function based on the predicted wrapped phase information and the real wrapped phase information corresponding to the same first grating image; The first neural network is iteratively trained according to the first loss function to obtain the first deep learning network model.

7. The three-dimensional model construction method according to claim 1, characterized in that: The step of training to obtain the second deep learning network model includes: Acquire a second sample set, where the second sample set includes a plurality of second grating images, and the second grating images are annotated with corresponding real depth information; Predicting the second grating image by a second neural network to obtain predicted depth information of the second grating image; constructing a second loss function based on the predicted depth information and the real depth information corresponding to the same second grating image; The second neural network is iteratively trained according to the second loss function to obtain the second deep learning network model.

8. The three-dimensional model construction method according to claim 2, characterized in that: The depth information is a depth image; The step of determining the corresponding number of steps according to the wrapped phase information and the depth information includes: Determine a corresponding three-dimensional point cloud according to the depth image of the grating image; The corresponding number of steps is determined according to the wrapped phase information and the three-dimensional point cloud.

9. The three-dimensional model construction method according to claim 8, characterized in that: The step of determining a corresponding three-dimensional point cloud according to the depth image of the grating image comprises: Convert the pixel coordinates in the depth image into the corresponding three-dimensional point cloud by formula 1; In formula 1: (x w ,y w , z w ) is the coordinate of the three-dimensional world point; (u0, v0) is the pixel coordinate of the intersection of the image plane and the camera optical axis, that is, the coordinate of the center of the computer image; f x With f y is the focal length of the camera lens, in mm.

10. The three-dimensional model construction method according to claim 8, characterized in that: Determining the corresponding number of steps according to the wrapped phase information and the three-dimensional point cloud includes: Determining an initial absolute phase of the grating image according to the three-dimensional point cloud of the grating image and calibration data of the structured light measurement system; Based on the initial absolute phase of the grating image and the wrapped phase information corresponding to the grating image, the corresponding step level is determined.

11. The three-dimensional model construction method according to claim 8, characterized in that: The step of constructing a three-dimensional model of the target object according to the wrapped phase information and the number of steps includes: Determining a corresponding absolute phase according to the number of steps corresponding to the grating image and the wrapped phase information; A three-dimensional model of the target object is constructed based on the absolute phase of the grating image.

12. The three-dimensional model construction method according to claim 11, characterized in that: Determining the corresponding absolute phase according to the number of steps corresponding to the grating image and the wrapped phase information includes: Determining the absolute phase of the grating image based on Formula 2; In formula 2: Ф pre is the absolute phase; is the package phase information; N pre is the number of steps.

13. A three-dimensional model construction device, characterized in that: The three-dimensional model building device comprises: An acquisition module, used for acquiring a raster image corresponding to a target object; A detection module, used for inputting the grating image into a pre-trained deep learning network to obtain the wrapped phase information and depth information corresponding to the grating image; the deep learning network includes a first deep learning network model and a second deep learning network model; and also used for inputting the grating image into the pre-trained first deep learning network model to obtain the wrapped phase information of the grating image; and inputting the grating image into the pre-trained second deep learning network model to obtain the depth information of the grating image; An analysis module, used for determining a corresponding step number according to the package phase information and the depth information; A construction module is used to construct a three-dimensional model of the target object according to the package phase information and the number of steps.

14. A terminal, characterized in that: The terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor is used to execute program data to implement the steps in the three-dimensional model construction method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the three-dimensional model construction method according to any one of claims 1 to 12 are implemented.

Images