Augmented reality display

Abstract

An AR system that leverages a pre-generated 3D model of the world to improve rendering of 3D graphics content for AR views of a scene, for example an AR view of the world in front of a moving vehicle.By leveraging the pre-generated 3D model, the AR system may use a variety of techniques to enhance the rendering capabilities of the system. The AR system may obtain pre-generated 3D data (e.g., 3D tiles) from a remote source (e.g., cloud-based storage), and may use this pre-generated 3D data (e.g., a combination of 3D mesh, textures, and other geometry information) to augment local data (e.g., apoint cloud of data collected by vehicle sensors) to determine much more information about a scene, including information about occluded or distant regions of the scene, than is available from the local data.

Description

Background technique

[0001] Remote sensing technology provides different systems with information about the external environment of that system. Various technical applications may rely on remote sensing systems and remote sensing devices to operate. Furthermore, as more and more systems attempt to utilize greater amounts of data to perform different tasks in dynamic environments; remote sensing provides environmental data that can facilitate decision-making. For example, a control system directing the operation of a machine may utilize remote sensing devices to detect objects within the workspace. As another example, augmented reality (AR) systems may utilize remote sensing devices to provide depth information about objects in the environment. In some cases, laser-based sensing technologies, such as optical ranging and detection (lidar), can provide high-resolution environmental data, such as depth maps, which can represent the proximity of different objects to lidar. [00...

Examples

Embodiment Construction

[0030]This disclosure describes methods and systems that may be used, for example, in augmented reality (AR) displays in vehicles. This disclosure describes embodiments of an AR system that utilizes pre-generated stereoscopic reconstructions of the world to assist in anchoring and improve rendering of AR scenes. By utilizing a stereoscopic reconstruction of the world, implementations of AR systems may use a variety of techniques to enhance the system's rendering capabilities. In an embodiment, the AR system may acquire pre-generated 3D data from a stereoscopic reconstruction of the world generated from real-world imagery acquired over time from a multitude of sources, and use this pre-generated 3D data (e.g., AR rendering can benefit from a combination of 3D meshes, textures, and other geometric information) to determine more information about a scene than can be obtained from local sources (e.g., point cloud data collected by vehicle sensors) . Furthermore, pre-generated 3D...

Background technique

[0001] Remote sensing technology provides different systems with information about the external environment of that system. Various technical applications may rely on remote sensing systems and remote sensing devices to operate. Furthermore, as more and more systems attempt to utilize greater amounts of data to perform different tasks in dynamic environments; remote sensing provides environmental data that can facilitate decision-making. For example, a control system directing the operation of a machine may utilize remote sensing devices to detect objects within the workspace. As another example, augmented reality (AR) systems may utilize remote sensing devices to provide depth information about objects in the environment. In some cases, laser-based sensing technologies, such as optical ranging and detection (lidar), can provide high-resolution environmental data, such as depth maps, which can represent the proximity of different objects to lidar. [00...

Examples

Embodiment Construction

[0030]This disclosure describes methods and systems that may be used, for example, in augmented reality (AR) displays in vehicles. This disclosure describes embodiments of an AR system that utilizes pre-generated stereoscopic reconstructions of the world to assist in anchoring and improve rendering of AR scenes. By utilizing a stereoscopic reconstruction of the world, implementations of AR systems may use a variety of techniques to enhance the system's rendering capabilities. In an embodiment, the AR system may acquire pre-generated 3D data from a stereoscopic reconstruction of the world generated from real-world imagery acquired over time from a multitude of sources, and use this pre-generated 3D data (e.g., AR rendering can benefit from a combination of 3D meshes, textures, and other geometric information) to determine more information about a scene than can be obtained from local sources (e.g., point cloud data collected by vehicle sensors) . Furthermore, pre-generated 3D...

Images