3D Imaging Using Event-Based Sensors and Pattern Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing three-dimensional image sensors face limitations in achieving high spatial and temporal resolutions while being computationally efficient and compatible with dynamic scenes, particularly in low-power applications such as augmented reality and robotics.
Innovation Solution
The system projects patterns of electromagnetic pulses and uses event-based image sensors with state machines to track patterns across pixels, allowing for triangulation of depths even with dynamic scenes, and increases temporal resolution by capturing events based on illumination changes, enabling higher light sampling rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If time-of-flight cameras use laser projectors to measure depth directly, then depth measurement capability is improved, but spatial resolution deteriorates to 100,000 pixels or lower and power consumption increases
Solution Approach 1:
The patent segments the depth measurement process into multiple temporal phases within a single frame period. Different regions of the scene are illuminated with different temporal patterns of light pulses, allowing the sensor to capture depth information for multiple depth ranges sequentially. This temporal segmentation enables high spatial resolution (exceeding 300,000 pixels) while maintaining accurate depth measurement capability.
2Manufacturing precision
If structured light cameras project patterns to achieve high spatial resolution up to 300,000 pixels, then spatial resolution is improved, but temporal resolution deteriorates to around 30 fps due to computational expense
Solution Approach 1:
The patent replaces the computationally intensive point-by-point triangulation process with a global optimization approach that processes all pixels simultaneously. By formulating depth estimation as a joint optimization problem across the entire image, the system achieves high spatial resolution (exceeding 300,000 pixels) while dramatically improving temporal resolution to exceed 30 fps, making it suitable for dynamic scene capture.
3Measurement precision
If stereo cameras match points between two views to estimate three-dimensional position, then depth estimation capability is improved, but computational cost increases leading to low temporal resolution
Solution Approach 1:
The patent introduces an intermediary optimization framework that bridges the two-view stereo correspondence problem. Instead of directly matching points between views, the system uses a joint optimization approach with regularization terms that act as intermediaries to guide the matching process. This reduces computational complexity while maintaining accurate depth estimation capability.
4Manufacturing precision
If active stereo systems use pattern projection with two cameras to achieve high spatial resolution, then spatial resolution is improved, but the system reverts to passive stereo in dynamic situations, reducing temporal resolution
Solution Approach 1:
The patent implements a dynamic system that automatically adapts between active and passive stereo modes based on scene characteristics. The optimization framework can handle both static pattern matching and dynamic scene variations within the same computational pipeline. This allows the system to maintain high spatial resolution (exceeding 300,000 pixels) while remaining versatile and adaptive to dynamic situations without reverting to lower-resolution passive stereo.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach provides high-quality three-dimensional image reconstruction with low latency and improved accuracy, suitable for dynamic scenes and low-power applications, surpassing the limitations of existing technologies in spatial and temporal resolution.
Implementation Method 1
a projector configured to project a plurality of electromagnetic pulses onto a scene; an image sensor comprising a plurality of pixels and configured to detect reflections in the scene caused by the projected plurality of electromagnetic pulses
Implementation Method 2
Each pixel may comprise a first photosensitive element, a detector that is electrically connected to the first photosensitive element and configured to generate a trigger signal when an analog signal proportional to brightness of light impinging on the first photosensitive element matches a condition
Data Source
AI summary
The present disclosure generally relates to systems and methods for three-dimensional image sensing. More specifically, and without limitation, this disclosure relates to systems and methods for detecting three-dimensional images, and using asynchronous image sensors for detecting the same. In one implementation, at least one processor determines a plurality of patterns associated with a plurality of electromagnetic pulses emitted by a projector onto a scene; receives, from an image sensor, one or more first signals based on reflections caused by the plurality of electromagnetic pulses; detects one or more first events corresponding to one or more first pixels of the image sensor based on the received signals; based on the one or more first events, initializes one or more first events more state machines; receives, from the image sensor, one or more second signals corresponding to the reflections; detects one or more second events corresponding to one or more second pixels of the image sensor based on the received signals; determines candidates for connecting the one or more state machines more second events to the one or more first events; and determines three-dimensional points for the one or more first pixels and the one or more second pixels based on the candidates and the one or more state machines.