Hybrid ToF Stereo Sensor Depth Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth cameras face limitations such as limited Field of View (FoV), inaccuracies with pattern-less surfaces, specular reflections, repetitive patterns, and interference with other light sources, leading to inaccurate depth estimation.
Innovation Solution
An apparatus comprising multiple sensors, including Time of Flight (ToF) sensors and pairs of stereo sensors arranged perpendicularly, determines disparity and confidence values to estimate depth accurately, even in challenging conditions like repetitive patterns and specular reflections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ToF cameras are used for depth estimation, then depth measurement capability is provided, but Field of View is limited and interference with other light sources occurs
Solution Approach 1:
The patent combines ToF sensors with stereo sensor pairs to create a hybrid depth estimation system. The ToF sensors provide direct depth measurement capability while the stereo sensors expand the Field of View and provide alternative depth estimation through disparity calculation. This merging allows the system to overcome the limited FoV of ToF cameras while maintaining accurate depth measurement capabilities.
2Adaptability or versatility
If stereo vision-based depth cameras are used, then Field of View is wide, but depth estimation is inaccurate with pattern-less surfaces, specular reflections, and repetitive patterns
Solution Approach 1:
The patent uses ToF sensors as an intermediary to provide accurate depth information for regions where stereo vision fails. The ToF depth data serves as a reference or mediator to correct and refine the stereo disparity-based depth estimates, particularly for pattern-less surfaces, specular reflections, and repetitive patterns where stereo correlation is unreliable.
Solution Approach 2:
The system implements a feedback mechanism where ToF depth measurements are used to evaluate and refine stereo disparity calculations. The confidence values generated from multiple stereo pairs are compared with ToF depth data, and the system can adjust or correct stereo-based depth estimates based on the more reliable ToF measurements, creating a feedback loop that improves overall depth accuracy.
3Reliability
If multiple pairs of stereo sensors are used, then depth estimation robustness is improved, but device complexity increases
Solution Approach 1:
The patent segments the depth estimation task across multiple stereo sensor pairs arranged in different orientations (horizontal and vertical). Each stereo pair handles specific spatial regions or viewing angles, and their results are combined through confidence-based selection. This segmentation allows the system to achieve robust depth estimation across diverse conditions while managing complexity through modular sensor organization.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The apparatus achieves accurate and robust depth estimation across various conditions by utilizing multiple sensors with different baselines and confidence-based selection, enhancing the accuracy and reliability of depth computation.
Implementation Method 1
Time of Flight (ToF) cameras which estimates the depth based on the time for light reflected from objects to return to sensor
Data Source
AI summary
The present disclosure relates to apparatus (100) for estimating depth of scene, comprising plurality of sensors (101) including Time of Flight (ToF) sensors and stereo sensors pairs, memory (103) and one or more processors (104). At least one pair of stereo sensors is placed perpendicular to other pairs of stereo sensors. One or more processors (104) are configured to determine disparity for plurality of sensors (101) based on comparison between one or more features of each image with other images received from plurality of pairs of stereo sensors, and evaluation of each image based on one or more pre-defined cost function parameters; determine disparity based on depth information received from ToF sensors; determine confidence value for each of plurality of sensors (101), based on disparity, to generate plurality of confidence values and estimate depth of scene based on disparity associated with a sensor selected based on plurality of confidence values.