Depth mapping with a head mounted display using stereo cameras and structured light

Abstract

A an augmented reality (AR) headset includes a depth camera assembly that combines stereo imaging with structured light (SL) to generate depth information for an area of interest. The depth camera assembly includes at least two image capture devices and a SL illuminator and determines an imaging mode based on a signal to noise ratio or spatial variance of images captured by one or more of the cameras. Different imaging modes correspond to different operation of one or more image capture devices and the SL illuminator. The depth camera assembly includes different ranges of signal to noise ratios that each correspond to an imaging mode, and the depth camera assembly configures the image capture devices and the SL illuminator based on an imaging mode associated with a range of signal to noiseratios including the signal to noise ratio of a captured image.

Description

Background technique [0001] The present disclosure generally relates to augmented reality systems or virtual reality systems, and more particularly, to head-mounted devices of virtual or augmented reality systems for obtaining depth information of a local area. [0002] Virtual reality (VR) systems may include augmented reality (AR) systems that may affect the capture of an environment surrounding a user in three-dimensional space (3D). However, conventional depth camera imaging architectures are relatively large, heavy, and consume large amounts of power. For example, time-of-flight (direct detection of pulses and encoding waveform patterns), structured light (SL), and stereo vision are commonly used camera architectures for obtaining three-dimensional information of a scene. Within the SL architecture, an asymmetric camera design with a single camera with a known baseline distance and a single SL source is a commonly used framework for capturing three-dimensional scene info...

AI Technical Summary

Problems solved by technology

However, measurements of SL designs are affected by having an in-band background light level in the local area being imaged

For example, in cases where the brightness of the ambient light in a localized area spans multiple levels of magnitude (equal to or brighter than the apparent intensity of the SL source), the SL scheme degrades (since the signal of interest eventually disappears in the background light source in the photon noise limit of

Time-of-Flight (ToF) architectures experience similar performance degradation at increasing ambient brightness

However, stereo vision cannot capture 3D scene information without at least minimal background luminance via controlled or uncontrolled environmental sources

Images