Context-aware multi-view three-dimensional reconstruction system and method based on deep learning

Abstract

The invention discloses a context-aware multi-view three-dimensional reconstruction system and method based on deep learning. The system comprises an encoder, a decoder, a context fusion module, a refiner and a network loss function. The encoder generates n feature maps according to the n input images; the decoder reconstructs n initial three-dimensional shapes by taking each feature map as input;the context fusion module takes the initial three-dimensional shape as an input and adaptively selects a reconstruction part with relatively high quality of each initial three-dimensional shape for fusion to obtain a fused three-dimensional shape; and the refiner takes the fused three-dimensional shape as input to further correct a reconstructed error part, so that a final three-dimensional shapeis reconstructed. The invention provides a non-contact, simple and convenient technology for quickly reconstructing the three-dimensional shape of an object from views shot from multiple angles, andthe method has higher system robustness and reconstruction precision.

Description

Technical field [0001] The present invention belongs to the technical field of three-dimensional shape based on multi-view reconstruction objects, and involves a context-sensitive manager based on deep learning. Figure three Dimensional redding system and its methods. [0002] technical background [0003] Traditional three-dimensional reconstruction method, such as: multi-view reconstruction, Structure Rommotion: SFM), synchronous positioning and construction and mapping: SLAM, etc., cross-view image feature matches, multi-view geometric constraints Solved by technology. However, when the interval between multiple views is large, feature matching is very difficult due to a variation of appearance or self-obstruction. To overcome these limits, many deep learning-based methods have been developed to rebuild the three-dimensional shape of the object, including 3D-R2N2 [Choy et al, 3D-R2N2: a unifiedapproach for single and multi-view 3D object reconstruction.eccv 2016.], LSM [Kar et...

AI Technical Summary

Problems solved by technology

However, RNN-based methods suffer from three limitations: first, when given the same set of images with different orders, the 3D shape of the object reconstructed by RNN may be inconsistent; second, due to the long-term memory loss of RNN ), cannot fully utilize the input image to optimize the reconstruction result; finally, the RNN-based method is time-consuming because the input image is processed sequentially and cannot be processed in parallel

Images