Three-dimensional reconstruction method, apparatus and device

Abstract

Embodiments of the present application provide a three-dimensional reconstruction method, device and equipment, the method comprising: obtaining a binocular stereo view corresponding to a target object; the binocular stereo view corresponding to the target object is determined based on a monocular RGB image and a depth image of the target object; and constructing a three-dimensional model of the target object according to the binocular stereo view corresponding to the target object. The method of the embodiments of the present application reduces the requirement and cost of hardware equipment for three-dimensional reconstruction, improves the applicable scene of three-dimensional reconstruction, and thus the three-dimensional reconstruction based on the binocular stereo view can overcome the depth ambiguity problem existing in the implicit function three-dimensional modeling method of the monocular view, and improve the accuracy of three-dimensional reconstruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional vision, and in particular to a three-dimensional reconstruction method, device and equipment. BACKGROUND

[0002] Three-dimensional reconstruction technology has gradually developed into a research hotspot in the field of three-dimensional vision due to its wide range of application scenarios, and the basic idea thereof is to obtain a three-dimensional model corresponding to two-dimensional image data input into a computer.

[0003] In the related art, a single-view image is usually taken as input to predict a three-dimensional model, but there is a depth ambiguity problem in the single-view-based implicit function three-dimensional modeling method, and therefore how to accurately reconstruct a three-dimensional model is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0004] To solve the problems in the prior art, the embodiments of the present application provide a three-dimensional reconstruction method, device and equipment.

[0005] Specifically, the embodiments of the present application provide the following technical solutions:

[0006] In a first aspect, the embodiments of the present application provide a three-dimensional reconstruction method, comprising:

[0007] obtaining a binocular stereo view corresponding to a target object, wherein the binocular stereo view corresponding to the target object is determined based on a monocular RGB image and a depth image of the target object;

[0008] constructing a three-dimensional model of the target object based on the binocular stereo view corresponding to the target object.

[0009] Further, the obtaining of the binocular stereo view corresponding to the target object comprises:

[0010] obtaining a monocular RGB image and a depth image of the target object;

[0011] generating a single-view mesh model of the target object according to the monocular RGB image and the depth image of the target object;

[0012] generating the binocular stereo view corresponding to the target object according to the single-view mesh model of the target object.

[0013] Further, the generating of the single-view mesh model of the target object according to the RGB image and the depth image of the target object comprises:

[0014] generating three-dimensional single-view point cloud data of the target object according to the RGB image and the depth image of the target object;

[0015] Based on the Poisson surface reconstruction method and the three-dimensional single-view point cloud data of the target object, a single-view mesh model of the target object is generated.

[0016] Further, generating a binocular stereo view corresponding to the target object based on the single-view mesh model of the target object includes:

[0017] Based on the single-view mesh model of the target object and the preset parallax parameters, a binocular stereo view corresponding to the target object is generated.

[0018] Further, the step of constructing a three-dimensional model of the target object based on the binocular stereo view corresponding to the target object includes:

[0019] The stereoscopic view corresponding to the target object is input into the target model to obtain a three-dimensional model of the target object; the target model is used to output a three-dimensional model based on the stereoscopic view.

[0020] Furthermore, before obtaining the stereoscopic view corresponding to the target object, the process further includes:

[0021] Obtain the RGB image of the target object;

[0022] The RGB image of the target object is input into the depth image generation model to obtain the depth image of the target object; the depth image generation model is used to output the depth image based on the RGB image; the depth image generation model is trained based on RGB image samples and the depth image samples corresponding to the RGB image samples.

[0023] Secondly, embodiments of the present invention also provide a three-dimensional reconstruction apparatus, comprising:

[0024] The acquisition module is used to acquire a stereoscopic view corresponding to the target object; the stereoscopic view corresponding to the target object is determined based on the monocular RGB image and depth image of the target object;

[0025] The reconstruction module is used to construct a three-dimensional model of the target object based on the stereoscopic view corresponding to the target object.

[0026] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the three-dimensional reconstruction method as described in the first aspect.

[0027] Fourthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the three-dimensional reconstruction method as described in the first aspect.

[0028] In a fifth aspect, an embodiment of the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the three-dimensional reconstruction method according to the first aspect.

[0029] The three-dimensional reconstruction method, device and equipment provided by the embodiment of the present application can obtain the binocular stereo view corresponding to the target object according to the monocular RGB image and the depth image of the target object after the monocular RGB-D camera captures the monocular RGB image and the depth image of the target object, so that the binocular stereo view corresponding to the target object can be determined according to the monocular camera in the case where the binocular camera cannot be obtained, the requirement and cost of the hardware equipment for three-dimensional reconstruction are reduced, the applicable scene of the three-dimensional reconstruction is improved, and then the three-dimensional reconstruction based on the binocular stereo view can overcome the depth ambiguity problem existing in the single-view implicit function three-dimensional modeling method, and the accuracy of the three-dimensional reconstruction is improved. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0031] Figure 1 is a flowchart of the three-dimensional reconstruction method, device and equipment method provided by the embodiment of the present application;

[0032] Figure 2 is another flowchart of the three-dimensional reconstruction method provided by the embodiment of the present application;

[0033] Figure 3 is a structural schematic diagram of the three-dimensional reconstruction method, device and equipment device provided by the embodiment of the present application;

[0034] Figure 4 is a structural schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort are within the protection scope of the present application.

[0036] The method of this invention can be applied to 3D reconstruction scenarios, reducing the demand and cost of 3D reconstruction hardware equipment, expanding the applicable scenarios of 3D reconstruction, and thus overcoming the depth ambiguity problem existing in the implicit function 3D modeling method of single view by performing 3D reconstruction based on binocular stereo view, thereby improving the accuracy of 3D reconstruction.

[0037] In related technologies, single-view images are usually used as input to predict 3D models. However, the depth ambiguity problem exists in the implicit function 3D modeling method based on single view. Therefore, how to accurately reconstruct the 3D model is a technical problem that needs to be solved by those skilled in the art.

[0038] The 3D reconstruction method of this invention utilizes a monocular RGB-D camera to capture monocular RGB images and depth images of a target object. Based on these images, a binocular stereo view of the target object is obtained. This allows for the determination of the corresponding binocular stereo view even when a binocular camera is unavailable, reducing the hardware requirements and costs for 3D reconstruction and expanding its applicability. Furthermore, 3D reconstruction based on a binocular stereo view overcomes the depth ambiguity problem inherent in implicit function 3D modeling methods using a single view, thereby improving the accuracy of 3D reconstruction.

[0039] The following is combined with Figures 1-4 The technical solution of the present invention will be described in detail with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0040] Figure 1 This is a schematic flowchart of an embodiment of the three-dimensional reconstruction method provided by this invention. Figure 1 As shown, the method provided in this embodiment includes:

[0041] Step 101: Obtain the stereoscopic view corresponding to the target object; the stereoscopic view corresponding to the target object is determined based on the monocular RGB image and depth image of the target object;

[0042] Specifically, existing technologies typically use single-view images as input to predict 3D models. However, the implicit function 3D modeling method based on single-view images suffers from depth blurring. Furthermore, due to the higher cost of binocular cameras compared to monocular cameras, and because of the photographer's position and viewing angle, the pixels in the two images directly captured by a binocular camera are not parallel to the points of the subject; that is, there is an angle between the pixels in the two images and the points of the subject. This results in poor accuracy in 3D reconstruction based on binocular images directly captured by a binocular camera.

[0043] To address the aforementioned issues, this embodiment of the invention first acquires a binocular stereo view corresponding to the target object, which is determined based on a monocular RGB-D camera. Optionally, a monocular RGB image and a depth image of the target object can be captured using a monocular RGB-D camera. The binocular stereo view corresponding to the target object is then acquired based on these images. This allows for the determination of the binocular stereo view even when a binocular camera is unavailable, reducing the hardware requirements and costs for 3D reconstruction, expanding the applicability of 3D reconstruction, and overcoming the depth ambiguity problem inherent in implicit function 3D modeling methods based on monocular views.

[0044] Step 102: Construct a 3D model of the target object based on the stereoscopic view corresponding to the target object.

[0045] Specifically, after determining the stereo view of the target object based on the monocular camera, a three-dimensional model of the target object can be constructed based on the stereo view of the target object, thereby overcoming the depth ambiguity problem in the implicit function three-dimensional modeling method of single view and improving the accuracy of three-dimensional reconstruction.

[0046] The method described in the above embodiments uses a monocular RGB-D camera to capture a monocular RGB image and a depth image of the target object. Based on these images, a binocular stereo view of the target object is obtained. This allows for the determination of the binocular stereo view of the target object even when a binocular camera is unavailable. This reduces the hardware requirements and costs for 3D reconstruction, expands the applicability of 3D reconstruction, and overcomes the depth ambiguity problem in the implicit function 3D modeling method based on a single view, thereby improving the accuracy of 3D reconstruction.

[0047] In one embodiment, obtaining a binocular stereo view corresponding to a target object includes:

[0048] Acquire the monocular RGB image and depth image of the target object;

[0049] Generate a single-view mesh model of the target object based on its monocular RGB image and depth image;

[0050] Generate a stereoscopic view of the target object based on its single-view mesh model.

[0051] Specifically, in order to solve the depth ambiguity problem in the implicit function 3D modeling method based on a single view, this embodiment of the invention constructs a 3D model of the target object based on the binocular stereo view corresponding to the target object, thereby solving the depth ambiguity problem in the implicit function 3D modeling method based on a single view and improving the accuracy of 3D reconstruction.

[0052] Optionally, in the process of acquiring the stereoscopic view of the target object in this embodiment of the invention, the monocular RGB image and depth image of the target object are first acquired. Since the distance between each point in the image and the camera can be accurately determined through the depth image of the target object, the spatial coordinates of each point of the target object can be calculated by combining the (x,y) coordinates corresponding to the RGB image of the target object. Then, the spatial coordinates of each point of the target object can be orthogonally projected to obtain the three-dimensional single-view point cloud data of the target object. Optionally, after orthogonal projection, each point in the three-dimensional single-view point cloud data of the target object is parallel to the shooting point, thereby solving the problem that there is an angle between the pixel points in the two images directly captured by the stereo camera and each point of the photographed object, which leads to poor accuracy of three-dimensional reconstruction based on the stereo images directly captured by the stereo camera, thus improving the accuracy of three-dimensional reconstruction. In other words, the points in the three-dimensional single-view point cloud data of the target object obtained by orthogonally projecting the RGB image and depth image of the two-dimensional image of the target object acquired by the monocular camera are parallel to the shooting point. This makes the stereo image of the binocular camera in this application more effective and accurate in the three-dimensional reconstruction process compared with the two images directly captured by the binocular camera.

[0053] Furthermore, by connecting the three-dimensional single-view point cloud data of the target object into a surface using the Poisson surface reconstruction method, the discrete three-dimensional single-view point cloud data of the target object can be converted into continuous three-dimensional data information of the target object, thus obtaining the single-view mesh model of the target object.

[0054] Optionally, after obtaining the single-view mesh model of the target object, that is, after obtaining the continuous three-dimensional data information of the target object, a binocular stereo view corresponding to the target object can be generated based on the single-view mesh model of the target object and the preset disparity parameters. Optionally, the disparity parameters indicate the disparity interval and distance between each pixel in the three-dimensional image and the left-eye image and the right-eye image. Therefore, based on the known single-view mesh model of the target object, the mesh model corresponding to the other eye can be accurately determined based on the preset disparity parameters, and thus the binocular stereo view corresponding to the target object can be accurately obtained.

[0055] In one embodiment, generating a single-view mesh model of the target object based on its RGB image and depth image includes:

[0056] Generate 3D single-view point cloud data of the target object based on its RGB and depth images;

[0057] Based on the Poisson surface reconstruction method and the 3D single-view point cloud data of the target object, a single-view mesh model of the target object is generated.

[0058] Specifically, the distance between each point in the two-dimensional image and the camera can be accurately determined using the depth image. By combining this with the corresponding (x, y) coordinates in the RGB image of the target object, the spatial coordinates of each point on the target object can be calculated. Then, orthogonally projecting these spatial coordinates yields the three-dimensional single-view point cloud data of the target object. Optionally, after orthogonal projection, each point in the three-dimensional single-view point cloud data of the target object is parallel to the shooting point. This solves the problem that pixels in the two images directly captured by a stereo camera have angles with the points of the photographed object, resulting in poor accuracy in 3D reconstruction based on stereo images directly captured by a stereo camera. In other words, the points in the three-dimensional single-view point cloud data of the target object obtained by orthogonally projecting the RGB and depth images from the two-dimensional image of the target object acquired by a monocular camera are parallel to the shooting point, making the stereo images in this embodiment more effective and accurate in 3D reconstruction compared to the two images directly captured by a stereo camera.

[0059] Optionally, after generating the three-dimensional single-view point cloud data of the target object, since the point cloud data is discrete, the three-dimensional single-view point cloud data can be transformed by the Poisson surface reconstruction method to obtain continuous three-dimensional data information of the target object, thereby obtaining the single-view mesh model of the target object.

[0060] In one embodiment, generating a binocular stereo view corresponding to the target object based on a single-view mesh model of the target object includes:

[0061] Generate a stereoscopic view of the target object based on the single-view mesh model of the target object and the preset parallax parameters.

[0062] Specifically, the disparity parameter indicates the disparity range and distance between each pixel in the three-dimensional image and the left-eye image and the right-eye image. Therefore, based on the known single-view mesh model of the target object, the mesh model corresponding to the other eye can be accurately determined based on the preset disparity parameter, and thus the binocular stereo view corresponding to the target object can be accurately obtained.

[0063] In one embodiment, constructing a three-dimensional model of the target object based on the stereoscopic view corresponding to the target object includes:

[0064] Input the stereo view corresponding to the target object into the target model to obtain the three-dimensional model of the target object; the target model is used to output the three-dimensional model based on the stereo view; the target model is trained based on the three-dimensional model corresponding to the stereo view sample, the stereo image sample and the preset disparity parameters.

[0065] Specifically, after determining the corresponding binocular stereo view of the target object based on its monocular RGB image and depth image, the binocular stereo view can be input into the target model to obtain a 3D reconstruction model of the target object. Optionally, to ensure that the target model can accurately output the 3D reconstruction model of the target object, in this embodiment of the invention, the initial target model is trained using binocular stereo view samples to predict the 3D reconstruction model corresponding to the binocular stereo view samples. Then, supervised training is performed based on the predicted 3D reconstruction model corresponding to the binocular stereo view samples and the 3D model corresponding to the binocular stereo image samples (i.e., label information). This allows the trained target model to accurately output the 3D model of the target object based on the input binocular stereo view of the target object.

[0066] It should be noted that the depth ambiguity problem exists in the existing implicit function 3D modeling method based on a single view. In the embodiments of this application, the training samples of the target model are stereo image samples, which are obtained by orthogonally projecting the RGB image and depth image of the target object in the 2D image obtained by a monocular camera. That is, compared with the existing technology, the training samples of the model not only overcome the depth ambiguity problem in the implicit function 3D modeling method based on a single view, but also overcome the problem that there is an angle between the pixels in the two images directly captured by the stereo camera and the various points of the photographed object in the process of directly capturing stereo images by the stereo camera. This leads to the poor accuracy of 3D reconstruction based on the stereo images directly captured by the stereo camera. As a result, the target model trained by the stereo image samples obtained by orthogonally projecting the RGB image and depth image of the target object in the 2D image obtained by the monocular camera can output the 3D model of the target object more accurately.

[0067] On the other hand, during the training process of the target model in this application, the influence of disparity parameters between various stereo images in the binocular stereo image samples on the accuracy of the predicted 3D model is also considered. In the embodiments of this application, the preset parameters in the target model training process are consistent with the disparity parameters in the process of generating the binocular stereo view corresponding to the target object based on the single-view mesh model of the target object and the preset disparity parameters. This allows the trained target model to accurately output the 3D model under the preset disparity parameters, improving the accuracy of the 3D model prediction of the target object. That is, during the training process of the target model, not only is the depth blur problem of the binocular stereo image samples compared to the single view considered, but also the angle between the pixels in the two images directly captured by the binocular camera and the various points of the photographed object. This leads to the problem that the accuracy of 3D reconstruction based on the binocular images directly captured by the binocular camera is also poor. Furthermore, the influence of the disparity parameters corresponding to the binocular stereo images on the model training effect is also considered, so that the target model trained based on the binocular stereo view samples, the 3D model corresponding to the binocular stereo image samples, and the preset disparity parameters can output the 3D model more accurately, improving the accuracy of 3D reconstruction.

[0068] The method described in the above embodiments, during the training of the target model, not only considers the depth blur problem of binocular stereo image samples compared to single-view images, but also considers that there are angles between the pixels in the two images directly captured by the binocular camera and the various points of the photographed object, resulting in poor accuracy of 3D reconstruction based on binocular images directly captured by the binocular camera. Furthermore, it also considers the impact of the disparity parameters corresponding to the binocular stereo images on the model training effect, so that the target model trained based on binocular stereo view samples, the 3D model corresponding to the binocular stereo image samples, and the preset disparity parameters can output a more accurate 3D model, thereby improving the accuracy of 3D reconstruction.

[0069] In one embodiment, before obtaining the stereoscopic view corresponding to the target object, the method further includes:

[0070] Obtain the RGB image of the target object;

[0071] The RGB image of the target object is input into the depth image generation model to obtain the depth image of the target object; the depth image generation model is used to output the depth image based on the RGB image; the depth image generation model is trained based on RGB image samples and the corresponding depth image samples of the RGB image samples.

[0072] Specifically, in this embodiment of the invention, a 3D model of the target object is reconstructed based on the stereoscopic view corresponding to the target object. The stereoscopic view is determined using a monocular RGB-D camera. This allows for the capture of monocular RGB and depth images of the target object using a monocular RGB-D camera, and the resulting stereoscopic view. This enables the determination of the stereoscopic view even when a monocular camera is unavailable, reducing the hardware requirements and costs for 3D reconstruction and expanding its applicability. Furthermore, 3D reconstruction based on the stereoscopic view overcomes the depth ambiguity problem inherent in the implicit function 3D modeling method using a monocular view. The stereoscopic image obtained by orthogonally projecting the RGB and depth images of the target object's 2D image acquired by the monocular camera ensures that all points in the resulting 3D single-view point cloud data are parallel to the shooting point. This makes the stereoscopic image in this application more effective and accurate in 3D reconstruction compared to two images directly captured by a stereo camera.

[0073] Optionally, in cases where even a monocular RGB-D camera cannot acquire depth images, i.e., when only RGB images can be acquired based on ordinary cameras and depth images cannot be obtained, the embodiments of the present invention can also acquire depth images based on ordinary RGB images, and then acquire binocular stereo views based on RGB images and depth images, further reducing the hardware requirements and costs of 3D reconstruction and improving the applicability of 3D reconstruction.

[0074] Optionally, when only RGB images can be acquired, the RGB image of the target object is input into the depth image generation model to obtain the depth image of the target object; wherein, the depth image generation model is used to output the depth image based on the RGB image; optionally, the depth image generation model is trained based on RGB image samples and the corresponding depth image samples, that is, by inputting RGB image samples into the depth image generation model, the predicted depth image is obtained, and then supervised training is performed based on the predicted depth image and the depth image samples (label information), so that the trained depth image generation model can accurately output the depth image corresponding to the RGB image, and thus the binocular stereo image can be accurately acquired based on the RGB image and the depth image.

[0075] The method described above simplifies the acquisition of depth images by using ordinary RGB images when even a monocular RGB-D camera cannot obtain them, i.e., only RGB images can be acquired but depth images cannot be obtained. This method is more versatile and can obtain binocular stereo views based on RGB images and depth images, further reducing the hardware requirements and costs for 3D reconstruction and expanding the applicable scenarios for 3D reconstruction.

[0076] For example, the flow of the three-dimensional reconstruction method in this application is as follows: Figure 2 As shown, the details are as follows:

[0077] First, generate single-view point cloud based on RGB-D data: First, use an RGB-D camera to acquire the front view of the target. Each set of data consists of an RGB image and a Depth image. Use the parameters obtained from the RGB-D camera calibration to construct a camera projection model and convert each set of RGB-D data into corresponding three-dimensional point cloud data under a single view.

[0078] Then, single-view mesh reconstruction is performed: a single-view mesh model is generated from the single-view point cloud using the Poisson surface reconstruction method;

[0079] Next, the binocular stereo view data is generated: based on the single-view mesh model of the target object and the preset disparity parameters, the binocular view is rendered from the single-view mesh.

[0080] Finally, complete model reconstruction: The generated binocular view is used as input data and loaded into the target model to reconstruct the complete 3D model.

[0081] The three-dimensional reconstruction device provided by the present invention is described below. The three-dimensional reconstruction device described below and the three-dimensional reconstruction method described above can be referred to in correspondence.

[0082] Figure 3 This is a schematic diagram of the structure of the three-dimensional reconstruction device provided by the present invention. The three-dimensional reconstruction device provided in this embodiment includes:

[0083] The acquisition module 710 is used to acquire the stereo view corresponding to the target object; the stereo view corresponding to the target object is determined based on the monocular RGB image and depth image of the target object;

[0084] Reconstruction module 720 is used to construct a 3D model of the target object based on the stereoscopic view corresponding to the target object.

[0085] Optionally, the acquisition module 710 is specifically used to: acquire a monocular RGB image and a depth image of the target object;

[0086] Generate a single-view mesh model of the target object based on its monocular RGB image and depth image;

[0087] Generate a stereoscopic view of the target object based on its single-view mesh model.

[0088] Optionally, the acquisition module 710 is specifically used to: generate three-dimensional single-view point cloud data of the target object based on the RGB image and depth image of the target object;

[0089] Based on the Poisson surface reconstruction method and the 3D single-view point cloud data of the target object, a single-view mesh model of the target object is generated.

[0090] Optionally, the acquisition module 710 is specifically used to: generate a binocular stereo view corresponding to the target object based on the single-view mesh model of the target object and preset disparity parameters.

[0091] Optionally, the reconstruction module 720 is specifically used to: input the binocular stereo view corresponding to the target object into the target model to obtain a three-dimensional model of the target object; the target model is used to output a three-dimensional model based on the binocular stereo view; the target model is trained based on the three-dimensional model corresponding to the binocular stereo view sample, the binocular stereo image sample and the preset disparity parameters.

[0092] Optionally, the acquisition module 710 is further configured to: acquire an RGB image of the target object;

[0093] The RGB image of the target object is input into the depth image generation model to obtain the depth image of the target object; the depth image generation model is used to output the depth image based on the RGB image; the depth image generation model is trained based on RGB image samples and the corresponding depth image samples of the RGB image samples.

[0094] The apparatus of this invention is used to execute the method in any of the foregoing method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0095] Figure 4A schematic diagram of the physical structure of an electronic device is provided. This electronic device may include a processor 810, a communication interface 820, a memory 830, and a communication bus 840. The processor 810, communication interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a 3D reconstruction method. This method includes: acquiring a binocular stereo view corresponding to a target object; determining the binocular stereo view corresponding to the target object based on a monocular RGB image and a depth image of the target object; and constructing a 3D model of the target object based on the binocular stereo view corresponding to the target object.

[0096] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer is able to execute the three-dimensional reconstruction method provided by the above methods, the method including: obtaining a binocular stereo view corresponding to a target object; the binocular stereo view corresponding to the target object is determined based on a monocular RGB image and a depth image of the target object; and constructing a three-dimensional model of the target object based on the binocular stereo view corresponding to the target object.

[0098] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the three-dimensional reconstruction methods provided above, the method comprising: acquiring a binocular stereo view corresponding to a target object; the binocular stereo view corresponding to the target object being determined based on a monocular RGB image and a depth image of the target object; and constructing a three-dimensional model of the target object based on the binocular stereo view corresponding to the target object.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0100] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional reconstruction method, characterized in that, include: Obtain the stereoscopic view corresponding to the target object; The binocular stereo view corresponding to the target object is determined based on the monocular RGB image and depth image of the target object; Based on the binocular stereo view corresponding to the target object, construct a three-dimensional model of the target object; The process of obtaining the stereoscopic view corresponding to the target object includes: Acquire the monocular RGB image and depth image of the target object; Based on the monocular RGB image and depth image of the target object, generate a single-view mesh model of the target object; Generate a binocular stereo view of the target object based on the single-view mesh model of the target object; The step of generating a single-view mesh model of the target object based on the RGB image and depth image of the target object includes: Based on the RGB image and depth image of the target object, generate three-dimensional single-view point cloud data of the target object; Based on the Poisson surface reconstruction method and the 3D single-view point cloud data of the target object, a single-view mesh model of the target object is generated. The step of generating a binocular stereo view corresponding to the target object based on the single-view mesh model of the target object includes: Based on the single-view mesh model of the target object and the preset parallax parameters, a binocular stereo view corresponding to the target object is generated.

2. The three-dimensional reconstruction method according to claim 1, characterized in that, The step of constructing a 3D model of the target object based on the binocular stereo view corresponding to the target object includes: The stereoscopic view corresponding to the target object is input into the target model to obtain a three-dimensional model of the target object; the target model is used to output a three-dimensional model based on the stereoscopic view; the target model is trained based on stereoscopic view samples, the three-dimensional model corresponding to the stereoscopic view samples, and the preset disparity parameters.

3. The three-dimensional reconstruction method according to claim 2, characterized in that, Before obtaining the stereoscopic view corresponding to the target object, the process also includes: Obtain the RGB image of the target object; The RGB image of the target object is input into the depth image generation model to obtain the depth image of the target object; the depth image generation model is used to output the depth image based on the RGB image; the depth image generation model is trained based on RGB image samples and the depth image samples corresponding to the RGB image samples.

4. A three-dimensional reconstruction device, characterized in that, include: The acquisition module is used to acquire a stereoscopic view corresponding to the target object; the stereoscopic view corresponding to the target object is determined based on the monocular RGB image and depth image of the target object; The reconstruction module is used to construct a three-dimensional model of the target object based on the stereoscopic view corresponding to the target object; The process of obtaining the stereoscopic view corresponding to the target object includes: Acquire the monocular RGB image and depth image of the target object; Based on the monocular RGB image and depth image of the target object, generate a single-view mesh model of the target object; Generate a binocular stereo view of the target object based on the single-view mesh model of the target object; The step of generating a single-view mesh model of the target object based on the RGB image and depth image of the target object includes: Based on the RGB image and depth image of the target object, generate three-dimensional single-view point cloud data of the target object; Based on the Poisson surface reconstruction method and the 3D single-view point cloud data of the target object, a single-view mesh model of the target object is generated. The step of generating a binocular stereo view corresponding to the target object based on the single-view mesh model of the target object includes: Based on the single-view mesh model of the target object and the preset parallax parameters, a binocular stereo view corresponding to the target object is generated.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the three-dimensional reconstruction method as described in any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the three-dimensional reconstruction method as described in any one of claims 1 to 3.

7. A computer program product having executable instructions stored thereon, characterized in that, When executed by the processor, this instruction causes the processor to implement the three-dimensional reconstruction method as described in any one of claims 1 to 3.

Images