ToF Stereo Camera Depth Reconstruction via Pixel-Wise Phase Unwrapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Time-of-Flight (ToF) cameras face challenges in accurately measuring the travel time of light due to the speed of light being exceptionally fast, limiting their ability to determine distance accurately, especially for objects beyond a certain maximum acquirable distance, resulting in incorrect distance calculations.
Innovation Solution
A method combining a ToF camera with a stereo camera for reliable wide-range depth acquisition, involving error estimation, phase unwrapping, and stereo matching to generate a reconstructed depth image, which calculates a maximum distance multiple per pixel and corrects depth images using disparity values from both cameras, thereby overcoming the limitations of ToF cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If phase unwrapping is conducted using a fixed maximum distance value, then the processing is simple, but the depth accuracy deteriorates in regions with large depth errors
Solution Approach 1:
The patent applies dynamics by transitioning from a fixed maximum distance value to a dynamic, pixel-wise maximum distance value. The system calculates the maximum distance value individually for each pixel based on its depth error, allowing the unwrapping process to adapt to local conditions. This resolves the contradiction by making the processing complexity acceptable (through automated per-pixel calculation) while significantly improving depth accuracy in regions with large depth errors.
Solution Approach 2:
The patent implements local quality by applying different maximum distance values to different pixels based on their individual depth errors. Instead of using a uniform value across the entire image, the system tailors the maximum distance value to each pixel's specific error characteristics, thereby improving depth accuracy locally without uniformly increasing processing complexity.
2Length of stationary object
If the maximum acquirable distance of ToF camera is increased, then the depth-of-field is extended, but the depth measurement precision deteriorates due to the speed of light being exceptionally fast
Solution Approach 1:
The patent introduces stereo camera disparity information as an intermediary to correct depth measurement errors in the ToF camera. By using the disparity values from the stereo camera as a reference, the system can identify and correct inaccurate depth measurements in the ToF image, thereby maintaining depth measurement precision while extending the depth-of-field through phase unwrapping with pixel-wise maximum distance values.
Solution Approach 2:
The system implements feedback by using depth error estimation to adjust the maximum distance value for each pixel. The depth error is calculated based on the difference between ToF depth values and stereo disparity-derived depth values, and this error information feeds back into the phase unwrapping process to determine appropriate maximum distance values, thereby maintaining precision across extended depth ranges.
3Measurement precision
If stereo matching is performed to correct depth errors, then the depth accuracy is improved, but the processing time increases
Solution Approach 1:
The patent applies partial action by performing stereo matching selectively rather than uniformly across the entire image. The system focuses computational resources on pixels where depth errors are likely to occur (those with large residuals between ToF and stereo depth estimates), thereby improving depth accuracy where needed while minimizing unnecessary processing time in regions where the ToF measurements are already accurate.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables accurate wide-range depth acquisition and correction of depth images, enhancing the depth-of-field beyond the maximum acquirable distance of ToF cameras and providing improved stereo disparity images for 3D object modeling.
Implementation Method 1
The ToF camera allows distance calculation by measuring a travel time taken by light to return to the sensor after bouncing off an object
Implementation Method 2
ttime-of-flight is calculated in an indirect manner of measuring the intensity of light based on at least two different phases after frequency-modulated light is emitted
Implementation Method 3
a left color image and a right color image taken with a stereo camera
Data Source
AI summary
Provided is a synthesis system of a time-of-flight (ToF) camera and a stereo camera for reliable wide range depth acquisition and a method therefor. The synthesis system may estimate an error per pixel of a depth image, may calculate a value of a maximum distance multiple per pixel of the depth image using the error per pixel of the depth image, a left color image, and a right color image, and may generate a reconstructed depth image by conducting phase unwrapping on the depth image using the value of the maximum distance multiple per pixel of the depth image.