Depth Based Dynamic Vision Sensor for AR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented and mixed reality systems face challenges in acquiring low-latency and low-power image information about physical objects, leading to unrealistic user experiences due to high power consumption and data processing requirements, especially when objects move relative to the user's field of view.
Innovation Solution
The implementation of an image sensor with angle-of-arrival to-intensity converters and differential readout circuitry that outputs signals only when light intensity changes significantly, combined with a processor that adapts threshold values based on event frequency, allowing for passive depth measurement and reduced data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image sensors continuously capture and process all image data, then complete image information is obtained, but power consumption and data processing requirements increase significantly
Solution Approach 1:
The patent extracts only the essential image information needed for augmented reality by using event-driven pixels that detect and report only significant changes in the optical flow field, rather than continuously capturing all image data. This extraction approach maintains measurement precision for motion detection while dramatically reducing power consumption and data processing requirements.
Solution Approach 2:
The patent implements partial action by having pixels selectively respond only to certain types of optical flow changes (e.g., motion events above a threshold) rather than continuously capturing all light intensity variations. This partial monitoring approach maintains sufficient image information accuracy for AR applications while reducing overall power consumption and data volume.
2Measurement precision
If high-resolution image sensors are used to track moving objects, then object tracking accuracy is improved, but data processing requirements and latency increase
Solution Approach 1:
The patent extracts only the critical motion information from the optical flow field by using event-driven pixels that detect and report significant changes rather than processing complete high-resolution image frames. This extraction maintains object tracking accuracy by capturing essential motion events while dramatically reducing data processing requirements and latency.
Solution Approach 2:
The patent applies preliminary anti-action by using asynchronous event-driven pixels that immediately detect and report optical flow changes as they occur, rather than waiting for periodic frame captures. This approach prevents motion tracking latency by having pixels proactively signal important events in real-time, maintaining tracking accuracy without the delay of continuous high-resolution frame processing.
3Measurement precision
If angle-of-arrival to-intensity converters are added to pixel cells, then depth measurement capability is improved, but device complexity increases
Solution Approach 1:
The patent merges the angle-of-arrival detection functionality with the existing photodetector structure by integrating angle-of-arrival to-intensity converters directly into the pixel cell. This merging approach enables depth measurement capability while minimizing additional device complexity by combining multiple functions into a unified pixel structure rather than adding separate components.
Solution Approach 2:
The patent implements multi-functionality by designing pixel cells that simultaneously perform photodetection, angle-of-arrival measurement, and depth estimation through the integrated angle-of-arrival to-intensity converters. This universal pixel design maintains relatively simple device structure while enabling multiple measurement capabilities including depth perception and optical flow field analysis.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables low-latency and low-power image sensing, reducing data processing and power consumption while maintaining realistic virtual object positioning relative to physical objects, even in dynamic environments.
Implementation Method 1
each of the plurality of pixel cells including a photodetector configured to generate an electric signal based on an intensity of light incident upon the photodetector
Implementation Method 2
the first and second diffraction gratings convert the angle of arrival of incident light into a position within the image plane based on the Talbot effect
Implementation Method 3
the first and second diffraction gratings convert the angle of arrival of incident light into a position within the image plane based on the Talbot effect
Data Source
AI summary
An image sensor suitable for use in an augmented reality system to provide low latency image analysis with low power consumption. The image sensor may have multiple pixel cells, each of the pixel cells comprising a photodetector to generate an electric signal based on an intensity of light incident upon the photodetector. The pixel cells may include multiple subsets of pixel cells, each subset of pixel cells including at least one angle-of-arrival to-intensity converter to modulate incident light reaching one or more of the pixel cells in the subset based on an angle of arrival of the incident light. Each pixel cell within the plurality of pixel cells may include differential readout circuitry configured to output a readout signal only when an amplitude of a current electric signal from the photodetector is different from an amplitude of a previous electric signal from the photodetector.