Temporal Time-of-Flight Depth Map Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing time-of-flight cameras fail to account for temporal data, leading to inaccurate depth maps when the camera or scene is in motion, as they discard previous frames and use a single measurement pattern for all frames.
Innovation Solution
A depth detection apparatus that utilizes temporal time-of-flight data, allowing each frame to contribute to multiple depth maps, and dynamically adjusts the measurement patterns for different frames, incorporating both temporal and static time-of-flight models for improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If previous frames are discarded and a single measurement pattern is used for all frames, then the device complexity is reduced, but the measurement precision deteriorates in dynamic scenarios
Solution Approach 1:
The patent segments the processing of depth frames by dividing them into static and dynamic components. Static frames are processed using a single measurement pattern while dynamic frames use multiple measurement patterns. This segmentation allows the system to maintain low complexity for stable scenes while improving precision for moving objects, resolving the contradiction between device complexity and measurement precision.
Solution Approach 2:
The patent introduces dynamic adaptation by automatically switching between single and multiple measurement patterns based on scene motion detection. The system dynamically adjusts its processing mode - using single pattern for static scenes (low complexity) and multiple patterns for dynamic scenes (high precision), thereby resolving the fixed trade-off between complexity and precision.
2Productivity
If each frame contributes to only one depth map, then the productivity is improved, but the measurement precision deteriorates due to insufficient data utilization
Solution Approach 1:
The patent makes each depth frame multi-functional by allowing it to contribute to multiple depth maps simultaneously. A single captured frame can be used in the computation of multiple depth maps depending on the measurement pattern and scene dynamics, thereby improving measurement precision without proportionally increasing processing load, as the same data serves multiple purposes.
Solution Approach 2:
The patent changes the parameter of frame utilization from fixed (one frame to one depth map) to variable (one frame to multiple depth maps). By dynamically adjusting how many depth maps each frame contributes to based on motion detection and measurement patterns, the system optimizes both productivity and precision adaptively.
3Measurement precision
If temporal data from multiple frames is utilized, then the measurement precision is improved, but the device complexity increases due to additional processing requirements
Solution Approach 1:
The patent segments temporal data processing into essential and optional components. The essential component processes only the necessary frames and measurement patterns for current depth map generation, while optional historical frames are selectively utilized. This segmentation reduces processing complexity compared to using all temporal data, while still improving precision through selective multi-frame utilization.
Solution Approach 2:
The patent extracts only the relevant temporal information needed for depth map computation, discarding redundant data. By extracting and processing only the essential temporal components (selected frames and measurement patterns) rather than all available temporal data, the system improves measurement precision while keeping processing complexity manageable.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach results in more accurate depth maps compared to traditional static time-of-flight systems, with empirical testing showing reduced absolute depth errors and improved robustness in dynamic scenarios.
Implementation Method 1
A light source at the TOF camera illuminates the scene and the light is reflected by objects in the scene. The camera receives the reflected light that, dependent on the distance of an object to the camera, experiences a delay. Given the fact that the speed of light is known, a depth map may be computed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A depth detection apparatus is described which has a memory and a computation logic. The memory stores frames of raw time-of-flight sensor data received from a time-of-flight sensor, the frames having been captured by a time-of-flight camera in the presence of motion such that different ones of the frames were captured using different locations of the camera and/or with different locations of an object in a scene depicted in the frames. The computation logic has functionality to compute a plurality of depth maps from the stream of frames, whereby each frame of raw time-of-flight sensor data contributes to more than one depth map.