A single image 3D object reconstruction method based on deep learning

Abstract

The present invention discloses a single image 3D object reconstruction method based on deep learning, comprising: 1. Inputting a single-color image containing multiple objects; 2. Utilizing a pre-trained RPN to output each region of a domain candidate of a specified category; 3. 1. Send each area in parallel to the shape-attitude prediction sub-network; 4. Predict the shape parameters and attitude parameters of the original object in three-dimensional space; 5. The shape sampling sub-network decodes the point cloud model corresponding to the shape space according to the shape parameters; 6. Carry out rigid transformation of the attitude parameters of the generated point cloud model; 7. Output the structure and attitude three-dimensional reconstruction results of the objects contained in the area. The present invention selects the point cloud model as the three-dimensional data representation that interacts with the network structure, so that the network has a better understanding of the 3D data, and at the same time uses "sampling points" instead of "sampling quantization unit size" for precision control, which can be better The control complexity, and guarantee the invariance of the rigid motion of the object.

Description

technical field [0001] The invention relates to the fields of computer vision, computer graphics, and machine learning, in particular to a method for reconstructing three-dimensional structures and attitudes based on single image information. Background technique [0002] Under current graphics technology, according to a given viewing angle of a given object, a computer can generate 2D renderings of the 3D object under different simulated lighting environments. And the corresponding reverse process—recovering the structure of the original 3D object based on the existing 2D rendering image is also in many scenarios, such as autonomous driving, and virtual reality technology (VR) also has broad demand and far-reaching research significance. [0003] Up to now, image-based 3D reconstruction methods have achieved a lot of research results. In summary, they can be divided into three categories: 1) Scene reconstruction based on a collection of RGB images. Ideally, 3D structures ...

AI Technical Summary

Problems solved by technology

However, there are still many problems in these works in terms of the robustness of reconstruction and the quality of shape restoration; in addition, in terms of hardware cost, the depth camera is higher than ordinary equipment

Therefore, the complexity of the algorithm and the immaturity of the hardware limit the use of the above two methods in daily life activities; in contrast, the second method is for ordinary cameras to generate a single image of the scene, if combined with the current data Driven learning methods will win in many aspects such as real-time, low cost, and convenience

However, this method also has difficulties to be solved in many aspects: 1) projection is a non-reversible transformation, and this problem itself is ill-posed; 2) many three-dimensional object representation methods have their own advantages and disadvantages, and the appropriate representation method must be determined according to the problem, and Appropriately improve the existing learning model to adapt to new problems; 3) Under new problems, a new loss function must be designed, which can reasonably evaluate the difference between the generated 3D model and the target; 4) In complex scenes, improve The robustness of the model needs to be significantly improved before the method can be put into practical use

Images