SPAD Array for Temporally Aligned Depth and Intensity Sensing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional image sensors in MR HMDs face challenges in generating parallax-corrected pass-through images due to temporal or spatial offsets between depth and intensity information, leading to artifacts like motion blur and jitter, especially under low light conditions.

Innovation Solution

The use of SPAD arrays in HMDs for intensity image sensing, which allows for interleaved intensity image capture and time-of-flight capture operations at the sub-frame level, enabling temporally aligned intensity and depth information and reducing motion blur.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional image sensors are used for intensity image sensing in MR HMDs, then the device can capture images, but temporal or spatial offsets between depth and intensity information occur, leading to motion blur and jitter artifacts

Engineering Contradiction:

Improvetemporal alignment between depth and intensity informationVSAvoidimage quality (motion blur and jitter artifacts)

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent combines depth sensing and intensity image sensing functions into a single SPAD array sensor. Each pixel in the SPAD array can operate in different modes (photon counting for depth, analog integration for intensity), allowing simultaneous capture of both depth and intensity information with perfect temporal alignment, thereby eliminating motion blur and jitter artifacts

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The SPAD array sensor is designed to perform multiple functions: it can capture depth information through time-of-flight measurements, capture intensity information through analog integration mode, and capture photon arrival timing information. This multi-functionality in a single sensor eliminates the need for separate depth and intensity cameras, ensuring spatial and temporal alignment

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Illumination intensity

If conventional CMOS or CCD image sensors are used, then intensity images can be captured, but the signal-to-noise ratio deteriorates under low light conditions due to read noise

Engineering Contradiction:

Improvelow light imaging capabilityVSAvoidsignal-to-noise ratio

Core Design Contradiction:

Illumination intensityVSMeasurement precision

Solution Approach 1:

The patent changes the operating parameters of the SPAD pixels by implementing dual modes: photon counting mode for high sensitivity in low light conditions where read noise is eliminated, and analog integration mode for higher dynamic range applications. This parameter change allows the sensor to maintain excellent signal-to-noise ratio across different lighting conditions

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent replaces the conventional CMOS/CCD sensor mechanism with a SPAD-based mechanism. Instead of using charge accumulation and readout circuits that introduce read noise, the SPAD uses single-photon detection with avalanche multiplication, fundamentally changing the detection mechanism to achieve superior low light performance

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If separate cameras are used for depth and intensity information, then both types of data can be captured, but spatial offset occurs between the captured information

Engineering Contradiction:

Improvedepth and intensity information captureVSAvoidspatial alignment between depth and intensity information

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent merges depth sensing and intensity sensing into a single co-located SPAD array sensor, eliminating the spatial offset that occurs when using separate cameras. Each pixel in the array provides both depth and intensity information from the exact same spatial location, ensuring perfect spatial alignment

Inventive Principle:

Principle #5Merging (Combining)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

SPAD arrays improve the signal-to-noise ratio by reducing read noise and provide temporally aligned intensity and depth information, resulting in clearer parallax-corrected pass-through images, especially in low light environments.

Implementation Method 1

Each pixel of the sensor array is configured to detect a photon and generate an electrical signal in response

Methodology Applied

Scientific EffectPhotoelectric Effect: Photoelectric Effect

Implementation Method 2

obtain depth information associated with the environment based on time of flight measurements

Methodology Applied

Scientific EffectTime of Flight: Time of Flight

Data Source

PatentUS12309501B2SPAD array for intensity image sensing on head-mounted displays

Publication Date: 2025.05.20 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

An HMD includes a single photon avalanche diode (SPAD) array comprising a plurality of SPAD pixels. The HMD also includes a display positioned to display images for viewing by an eye of a user. The HMD also includes one or more processors and one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the HMD to perform various acts associated with using the SPAD array to capture an image frame of an environment for display to the user.