Active Stereo Matching with Plane Hypotheses for Real-Time Depth Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional active stereo matching techniques are computationally expensive and fail to meet the needs of real-time processing and high accuracy applications.
Innovation Solution
A real-time active stereo system that uses a pair of stereo cameras to capture stereo image data, computes descriptors for pixel neighborhoods, evaluates plane hypotheses using these descriptors, and applies edge-aware filtering to generate high-resolution and accurate depth maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional stereo matching algorithms are used, then depth information can be obtained, but the processing time is too long for real-time applications
Solution Approach 1:
The patent segments the stereo matching process into multiple stages: pre-processing, coarse matching, and refinement. The coarse matching stage uses a simplified algorithm to quickly establish initial correspondences, while the refinement stage processes only the uncertain regions. This segmentation enables real-time processing while maintaining depth information accuracy.
Solution Approach 2:
The patent applies partial action by focusing computational resources only on uncertain or difficult-to-match regions rather than processing the entire image uniformly. The refinement stage selectively processes only the regions identified as needing attention during coarse matching, significantly reducing overall processing time while maintaining measurement precision.
2Productivity
If computational resources are increased to improve matching speed, then processing time decreases, but energy consumption increases
Solution Approach 1:
The patent implements dynamic computational resource allocation by adjusting the level of processing based on image characteristics and matching difficulty. The system dynamically switches between coarse and refinement modes, and selectively applies refinement only to uncertain regions, optimizing the balance between processing speed and energy consumption.
Solution Approach 2:
The patent changes computational parameters adaptively based on image content and matching progress. The system adjusts parameters such as matching threshold, region size, and algorithm complexity based on the current state, enabling efficient energy usage while maintaining high matching speed.
3Loss of time
If the stereo matching algorithm is simplified to reduce processing time, then real-time performance is achieved, but matching accuracy deteriorates
Solution Approach 1:
The patent segments the matching process into coarse and refinement stages, where the coarse stage provides quick initial results and the refinement stage improves accuracy only where necessary. This segmentation allows the system to achieve real-time performance while maintaining high accuracy through selective refinement.
Solution Approach 2:
The patent incorporates feedback mechanisms where the refinement stage uses information from the coarse matching results to guide its processing. The system identifies uncertain regions based on coarse matching outcomes and focuses refinement efforts on those areas, ensuring accuracy improvement without unnecessary computational overhead.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
According to an aspect, a real-time active stereo system includes a capture system configured to capture stereo image data, the stereo image data including reference images and secondary images, and a depth sensing computing system configured to generate a depth map, the depth sensing computing system configured to compute descriptors based on the reference images and the secondary images compute a stability penalty based on pixel change information and disparity change information. evaluate a plurality of plane hypotheses for a group of pixels using the descriptors, including compute matching cost between the descriptors associated with each plane hypothesis, update the matching cost with the stability penalty, and select a plane hypothesis from the plurality of plane hypotheses for the group of pixels based on the updated matching cost.