Active Stereo Depth Sensing with Offset Exposure for Motion Blur
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Depth cameras operating at low frame rates face challenges in correlating high frame-to-frame movements and motion blur, making it difficult to accurately reconstruct depth maps.
Innovation Solution
Utilizing offset exposure times and alternating illumination patterns with stereo depth cameras, combined with a processor that matches spatio-temporal image patches, computes a disparity map, and refines subpixels to generate a depth map, minimizing noise and computation cost.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If depth cameras operate at low frame rates, then power consumption is reduced, but depth reconstruction accuracy deteriorates due to high frame-to-frame movements and motion blur
Solution Approach 1:
The system uses periodic illumination patterns (alternating between first and second patterns) synchronized with the camera frame rate to encode depth information in each frame. This allows depth reconstruction even at low frame rates by using the temporal structure of the periodic illumination to disambiguate motion between frames.
Solution Approach 2:
The system changes illumination parameters (pattern type, phase, frequency) across multiple frames to encode depth information. By varying these parameters periodically and correlating them with captured frames, the system can reconstruct depth accurately even when camera frame rate is low, without increasing power consumption.
2Device complexity
If standard correlation methods are used for depth reconstruction, then computational simplicity is maintained, but accuracy deteriorates in the presence of motion blur and high frame-to-frame movements
Solution Approach 1:
The system performs preliminary correlation of captured frames with reference frames using known illumination patterns before final depth reconstruction. This preliminary correlation step uses the structured illumination information to pre-align and disambiguate motion, making the subsequent depth reconstruction more accurate without requiring complex optimization algorithms.
Solution Approach 2:
The structured illumination patterns serve as an intermediary that mediates between the camera captures and the depth reconstruction process. By correlating captured frames with known illumination patterns, the system creates intermediate correlation maps that guide the depth reconstruction, improving accuracy without directly increasing computational complexity of the core algorithm.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances depth map estimation by reducing noise and computation cost, allowing for high-quality depth reconstruction even with high frame-to-frame movements, suitable for applications like augmented reality and simultaneous localization and mapping.
Implementation Method 1
VCSELs emit divergent laser light at relatively low power consumption
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An electronic device [100] estimates a depth map of an environment based on stereo depth images [410, 415] captured by depth cameras [114, 115] having exposure times that are offset from each other in conjunction with illuminators [118, 1119] pulsing illumination patterns [305, 310] into the environment. A processor [220] of the electronic device matches small sections [430, 432] of the depth images from the cameras to each other and to corresponding patches of immediately preceding depth images (e.g., a spatio-temporal image patch "cube"). The processor computes a matching cost for each spatio-temporal image patch cube by converting each spatio-temporal image patch into binary codes and defining a cost function between two stereo image patches as the difference between the binary codes. The processor minimizes the matching cost to generate a disparity map, and optimizes the disparity map by rejecting outliers using a decision tree with learned pixel offsets and refining subpixels to generate a depth map of the environment.