3D reconstruction method for weighing stereo matching and visual appearance

Abstract

The invention discloses a 3D reconstruction method for weighing stereo matching and visual appearance, the specific steps of which are as follows: (1) making a sample database, including a stereo matching depth map, a visual appearance depth map, a multiview RGB chart and a real depth map; (2) constructing a depth convolution neural network; (3) using database to train neural network for the weight distribution value of the stereo matching and the visual appearance, the structure of the neural network is adjusted according to this value, until the neural network model is obtained with a better effect; (4) inputting the stereo matching depth map, the visual appearance depth map and the RGB chart, and a new depth map is acquired through the neural network model; (5) reconstructing a 3D model with the new depth map. The 3D reconstruction method combines the two practices of the stereo matching and the visual appearance, through the distribution weight value of the depth neural network, and can not only solve the problem that the stereo matching cannot rebuild a highlight and have no texture region, but also avoid the defect that the visual appearance cannot reconstruct a concave. The high quality reconstruction of the complex objects is realized.

Description

technical field [0001] The invention relates to the technical fields of computer vision, three-dimensional reconstruction, artificial intelligence, etc., and in particular to a three-dimensional fusion method that weighs stereo matching and visual appearance. Background technique [0002] 3D reconstruction technology is a popular research direction in the field of computer vision, which can be divided into active and passive reconstruction methods. Active reconstruction methods use detectors to actively transmit and receive signals to obtain depth information of real objects, such as lasers, infrared scanners, using structured light or time-of-flight principles. Passive reconstruction methods recover 3D models from multiple 2D images. [0003] For passive reconstruction methods, stereo information can be obtained in a variety of ways, and the most important two types of information come from stereo matching and visual appearance: [0004] Stereo matching refers to extracti...

AI Technical Summary

Problems solved by technology

Representative algorithms include: Matusik W et al. proposed an image-based method IBVH (Image-Based Visual Hulls) (International Conference on Computer Graphics and Interactive Techniques, 2000: 369-374.), which uses epipolar geometry The method of calculation and incremental calculation calculates and renders the model, which overcomes the problems of high computational complexity and limited image resolution of traditional algorithms; FrancoJ and Boyer E. They use a small number of geometric operations to calculate the single-channel polyhedron visual shape (IEEETransactions on Pattern Analysis and Machine Intelligence, 2009, 31 (3): 414-427), recovering a surface with a good topology; Gregor Miller and Adrian Hilton of the University of Surrey in the UK proposed the visual shape crossover theorem (International Conference on Pattern Recognition, 2006, 10.1109 / ICPR.2006.515), using the cross ratio to select the intersection between pictures

Images