Event-Based Depth Sensing With Sparse Illumination Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional depth estimation techniques face challenges with high memory requirements, latency, and limited pixel resolution due to the need for storing intensity values from all pixels for all illuminations, and event-based sensors introduce inaccuracies from temporal dynamics in illumination patterns.
Innovation Solution
A sensor device using event-based vision sensors to detect changes in intensity, projecting sparse illumination patterns, and controlling the pattern sequence to ensure robustness against temporal dynamics, allowing precise mapping of illumination to pixels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional systems store all intensity values from all pixels for all illuminations, then measurement precision is improved, but memory space requirements increase and latency increases
Solution Approach 1:
The patent extracts only the necessary information (intensity changes at specific pixels during specific illumination patterns) from the complete image data, discarding redundant information. Instead of storing all pixel values for all illumination patterns, the system stores only the intensity changes detected by event-based sensors at pixels where illumination patterns caused detectable changes, significantly reducing memory requirements while maintaining measurement precision for depth mapping.
Solution Approach 2:
The patent segments the image data processing by using event-based sensors that only report changes at specific pixels and times, rather than processing complete image frames. This segmentation of data collection to only necessary pixels and time points reduces the quantity of stored data while preserving the essential information needed for accurate depth measurement through triangulation.
2Measurement precision
If conventional systems store all intensity values for all pixels, then measurement precision is improved, but processing time increases and readout speed becomes limiting
Solution Approach 1:
The patent extracts only the necessary intensity change information from the complete image data using event-based sensors. Instead of reading out and processing all pixel values for all illumination patterns, the system only reads and processes intensity changes detected by events, dramatically reducing processing time and enabling real-time depth mapping applications.
Solution Approach 2:
The patent segments the data processing task by using event-triggered readout mechanism that only processes data at pixels and times when changes occur, rather than continuously processing complete image frames. This segmentation enables the system to maintain high processing speed while achieving accurate depth measurement through triangulation of illumination patterns.
3Quantity of substance
If event-based sensors are used, then memory space and processing time are reduced, but measurement precision deteriorates due to temporal dynamics in readout
Solution Approach 1:
The patent applies preliminary action by using controlled illumination patterns that are switched on and off in a predetermined sequence before the event-based sensor reads out the intensity changes. This timing control ensures that the illumination changes occur before the sensor sampling, allowing the system to distinguish between different illumination patterns and accurately determine which pattern caused each detected event, thereby maintaining measurement precision despite the event-based readout method.
Solution Approach 2:
The patent implements feedback by using the detected intensity changes from event-based sensors to determine the spatial mapping of illumination patterns to pixels. The system uses the event data as feedback to reconstruct the depth map, and the illumination pattern timing information provides feedback to correctly associate events with their corresponding illumination patterns, resolving the ambiguity introduced by temporal dynamics in the readout process.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces memory usage and processing time, enhances resolution, and improves the accuracy of depth maps by precisely mapping illumination patterns to pixels.
Implementation Method 1
a projector unit (1010) configured to project in a temporally consecutive manner a plurality of different illumination patterns in a projection solid angle PS to the object O
Implementation Method 2
a receiver unit (1020) which comprises a plurality of pixels (1025), the receiver unit (1020) being configured to detect on each pixel (1025) intensities of light reflected from the object O
Implementation Method 3
detect on each pixel intensities of light reflected from the object while it is illuminated with the illumination patterns
Data Source
AI summary
A sensor device for generating a depth map of an object includes a projector unit, a receiver unit, and a control unit. The projector unit projects multiple illumination patterns in a predefined projection solid angle, determining for each solid angle whether to project light. The receiver unit, equipped with multiple pixels, detects reflected light intensities during illumination and generates events when intensity changes exceed a threshold. The control unit adjusts illumination patterns, collects event data, and maps the reflected patterns to corresponding pixels. This enables the generation of a depth map by associating solid angles with pixel locations based on event data.