A angle-image multi-stage neural network based 3D reconstruction method

Abstract

The invention discloses a single-image three-dimensional reconstruction method based on a multi-stage neural network. The three-dimensional shapes in the existing three-dimensional shape set are rendered from multiple angles to obtain a training image set, and the training point cloud is obtained at the surface acquisition points thereof. A point cloud generation network is constructed, an image encoder is constructed by using depth residual network to extract image information, and a dual-branch primary decoder is constructed by using deconvolution network and full-connection network to generate initial point cloud. A point cloud refinement network is constructed, a point cloud encoder is constructed using a posture transformation network, a multilayer perceptron and a maximum pool function, an image encoder is constructed using a depth residual network, and an image is constructed using a full connection layer. A point cloud coupler and an advanced decoder generate a fine point cloud; Training the point cloud generation network and pre-training and fine-tuning the point cloud refinement network; The input image is reconstructed by using the trained model to obtain the 3D point cloud, and the surface mesh is reconstructed to generate the 3D shape represented by polygonal mesh.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and in particular relates to a single-image three-dimensional reconstruction method based on a multi-stage neural network. Background technique [0002] 3D reconstruction oriented to a single image is to restore the 3D shape of the object contained in the image from a single image using specific techniques. However, this task is a pathological problem, because the information that a single image can provide is extremely limited, so it needs strong prior information to complete. [0003] In fact, the academic community has proposed many related technologies and methods to solve the problem of 3D reconstruction for a single image. Among them, the reconstruction method based on visual cues is to perform three-dimensional reconstruction on a single image according to certain knowledge or theory that human beings have summarized in computer vision. Such as literature 1: Bichsel, Martin, and ...

AI Technical Summary

Problems solved by technology

Compared with previous works, the 3D shape reconstructed by this method has improved a lot in terms of effect, but there is still a phenomenon that the reconstructed 3D shape is very different from the shape of the object depicted in the original image.

The reason for this phenomenon is that these methods only use a single neural network (that is, a pair of encoder and decoder structure, referred to as the codec structure) for 3D reconstruction, and the learning ability of a single neural network is very limited, it is difficult to Fully mining the visual cues in the image, resulting in the learned 3D shape prior is not enough to make the shape of the reconstruction result highly consistent with the original image

Images