Piecewise planar reconstruction of three-dimensional scenes

Abstract

Methods, systems, and computer-readable media for reconstruction a three-dimensional scene from a collection of two-dimensional images are provided. A computerized reconstruction system executes computer vision algorithms on the collection of two-dimensional images to identify candidate planes that are used to model visual characteristics of the environment depicted in the two-dimensional images. The computer vision algorithms may minimize an energy function that represents the relationships and similarities among features of the two-dimensional images to assign pixels of the two dimensional images to planes in the three dimensional scene. The three-dimensional scene is navigable and depicts viewpoint transitions between multiple two-dimensional images.

Description

Background technique [0001] Traditionally, a computer's reconstruction engine can be configured to provide automatic feature matching or feature extraction that restores the sparse structure of the scene and camera calibration from an unordered collection of 2D images. Traditional rendering engines can use camera calibration, sparse scene structures, and 2D images to triangulate the location of each pixel in 3D space. Reproduces pixels in a triangulated location to form a 3D scene. [0002] However, the quality and detail of the generated 3D scenes often suffer from various deficiencies. For example, conventional rendering engines reproduce untextured or non-Lambertian surfaces captured in 2D images as holes. Holes are covered by interpolating the depths of neighboring pixels. But the interpolation of conventional rendering engines can wrongly reproduce flat surfaces with straight lines as bumpy surfaces in 3D scenes. Traditional rendering engines can also erroneously intr...

AI Technical Summary

Problems solved by technology

Furthermore, reconstructions of 3D scenes from sparse sets of 2D images are not realistically navigable because the assumptions of traditional

Images