Multi-view stereoscopic vision three-dimensional scene reconstruction method based on deep learning

Abstract

The invention relates to a multi-view stereoscopic vision three-dimensional scene reconstruction method based on deep learning, and aims to solve the problem that the existing reconstruction method based on deep learning generates a 3D cost body by extracting the feature of the last layer of an image, and does not well utilize shallow features, so that information of different scales is lost. Moreover, according to those methods, during depth map refinement, only the effect of the reference image on depth refinement is considered, and the contribution of the depth of the adjacent image to depth map prediction is ignored. In order to solve the problem, a multi-scale feature extraction and fusion network and a depth map refinement network based on inter-frame correlation are provided to improve the prediction precision and integrity of a scene. Compared with an existing method based on deep learning, the method of the invention has the advantages that the context features of the input image can be learned better, the shielded and missing area of the target scene can be reconstructed, the three-dimensional information of the scene can be restored more completely, and high-precision three-dimensional scene reconstruction is achieved.

Description

technical field [0001] The invention belongs to the field of computer vision and three-dimensional reconstruction, and studies a new three-dimensional reconstruction method. Background technique [0002] High-precision 3D scene reconstruction is crucial for many applications, such as urban 3D maps, monument reconstruction, autonomous driving, and augmented reality, etc. The 3D reconstruction method based on multi-view stereo vision is also one of the core research issues of computer vision. Traditional multi-view stereo matching reconstruction methods use subjectively designed similarity measures and engineered regularizations (such as normalized cross-correlation and semi-global matching) to compute dense correspondences and recover 3D points. Although these methods show good reconstruction results in the ideal Lambertian (Lambert) situation, they also have some common limitations. For example, the presence of low-texture, high-gloss and specular regions of the scene make...

AI Technical Summary

Problems solved by technology

To this end, the key technical issues that need to be solved include: using deep neural networks to extract and fuse high-performance multi-scale features, avoiding the accumulation of multi-link errors in manual design; Images, generally with similar depths, use the depth information of adjacent images to refine the predicted initial depth map

Images