Generation and rendering of extended view geometry in video perspective (VST) augmented reality (AR) systems

Abstract

A method includes obtaining, using a plurality of image sensors of an augmented reality (AR) device, a plurality of see-through image frames of an environment surrounding the AR device. The method further includes generating a depth map based on the perspective image frames and generating a three-dimensional (3D) representation of the environment based on the depth map. The method further includes projecting the 3D representation onto the curved surface, mapping points of the projected 3D representation to a plurality of virtual view images, and presenting the virtual view images on one or more displays of the AR device. The step of generating the depth map may include generating an initial depth map using the trained machine learning model, and modifying the initial depth map to provide both spatial and temporal coherency in order to generate a refined depth map. The curved surface may include a portion of a cylindrical, spherical, or conical surface.

Images