Mixed-mode depth imaging using ToF and structured light
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth imaging technologies face challenges in achieving accurate and efficient depth estimation, with structured light systems providing high accuracy but limited resolution and high computational complexity, while Time-of-Flight (ToF) systems offer high-resolution depth maps with degraded accuracy due to noise and light scattering.
Innovation Solution
A mixed-mode depth imaging system that combines structured light and ToF techniques, using a structured light source to project a pattern with uniquely identifiable features and a ToF sensor to determine distance measurements, reducing the search space for structured light decoding by utilizing ToF distance measurements to expedite and improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If structured light techniques are used for depth imaging, then measurement precision is improved, but device complexity and computational complexity increase
Solution Approach 1:
The patent segments the depth imaging process into two distinct stages: a coarse depth estimation stage using ToF technology, and a fine depth refinement stage using structured light techniques. This segmentation allows each method to operate in its optimal performance range, with ToF handling the computationally intensive initial estimation and structured light providing precise local refinement, thereby reducing overall computational complexity while maintaining high measurement precision.
Solution Approach 2:
The patent performs preliminary depth estimation using ToF measurements before applying structured light decoding. The ToF-based coarse depth map is generated first, which then serves as prior information to guide and constrain the subsequent structured light processing. This preliminary action reduces the search space for structured light feature matching, significantly lowering computational complexity while preserving the high precision benefits of structured light.
2Measurement precision
If structured light techniques are used for depth imaging, then measurement precision is improved, but productivity decreases
Solution Approach 1:
The patent divides the depth imaging workflow into two processing streams with different resolution requirements: a low-resolution but fast ToF stream for initial depth estimation, and a high-resolution but computationally intensive structured light stream for detail refinement. By processing only critical regions with structured light and using ToF for the remainder, the system achieves high overall precision while dramatically improving processing throughput and productivity.
Solution Approach 2:
The patent applies structured light decoding selectively to only those regions where high precision is critical or where ToF measurements are ambiguous, rather than processing the entire scene with structured light. This partial application of the more complex method maintains measurement precision in key areas while significantly reducing total processing time and improving productivity.
3Productivity
If ToF techniques are used for depth imaging, then productivity is improved, but measurement precision deteriorates
Solution Approach 1:
The patent merges ToF and structured light techniques into a hybrid depth imaging system where both methods operate simultaneously on the same scene. The ToF component provides fast, low-resolution depth information, while the structured light component provides slow, high-resolution depth information. By merging these complementary data streams and fusing them through multi-scale processing, the system achieves both high productivity (from ToF) and high measurement precision (from structured light).
Solution Approach 2:
The patent uses the ToF depth estimates as an intermediary to guide and constrain the structured light decoding process. The coarse ToF depth map serves as a mediator that provides initial geometric constraints, which then improve the accuracy and convergence of the subsequent structured light refinement. This intermediary approach allows the slower structured light method to achieve higher precision more efficiently by starting from a good initial guess provided by the fast ToF measurements.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system generates accurate and efficient depth maps with reduced computational complexity, achieving high-resolution depth maps comparable to structured light systems while minimizing noise and ambiguity, thus improving the overall performance of depth imaging.
Implementation Method 1
obtaining a frame including a reflected pattern of light generated based on a pattern of light emitted by a structured light source
Implementation Method 2
determining, using a ToF sensor, a first distance measurement associated with a pixel of the frame
Data Source
AI summary
Systems and techniques are described for generating depth map(s). For Depth Imaging System instance, a process can include obtaining a frame including a reflected pattern of light generated based on a pattern of light that is based on a primitive including a set of flight (ToF) sensor, a first distance measurement associated with a pixel of the frame, and determining a search space within the primitive based on the first distance measurement. The process can include determining, based on searching the search space, a feature of the primitive corresponding to a region around the pixel of the frame. The process can include determining a second distance measurement associated with the pixel of the frame based on determining the feature of the primitive. The process can include generating a depth map based at least in part on the second distance measurement.