Hybrid Depth Regularization With Confidence-Guided Stereo Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth estimation methods in computer stereo vision are prone to noise and errors due to factors like textureless regions, non-Lambertian surfaces, occlusions, and spatially varying lens characteristics, which affect the accuracy of depth maps, especially in real-time applications such as augmented reality.
Innovation Solution
A hybrid depth regularization process that categorizes image regions based on characteristics and applies region-specific techniques, including smoothing filters and edge-preserving filters, to improve depth map quality by combining multiple depth estimates and confidence maps, while handling occlusions and textureless areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional depth estimation methods are used, then processing speed is maintained, but depth map accuracy deteriorates due to noise and errors from textureless regions, non-Lambertian surfaces, occlusions, and lens characteristics
Solution Approach 1:
The patent segments the depth estimation process into multiple independent modules: a first depth estimation process that handles textureless regions and occluded areas, and a second depth estimation process that processes textured regions. This segmentation allows each process to be optimized for its specific challenges, improving overall accuracy without requiring a single complex unified system.
Solution Approach 2:
The patent applies different depth estimation techniques to different regions of the image based on their characteristics. Textureless regions and occluded areas use one estimation approach, while textured regions use another approach. This local differentiation enables each region to receive the most appropriate processing method, enhancing measurement precision while maintaining manageable system complexity.
2Measurement precision
If multiple depth estimation processes are combined to improve accuracy, then depth map quality improves, but computational overhead increases
Solution Approach 1:
Instead of applying complex depth estimation to entire images, the patent applies partial processing only where needed. The first depth estimation process is applied to textureless regions and occluded areas, while the second process handles textured regions. This partial action approach reduces overall computational overhead while maintaining high accuracy in challenging regions.
Solution Approach 2:
The system performs preliminary classification of image regions to determine which depth estimation process should be applied to each region. By pre-identifying textureless regions, occluded areas, and textured regions, the system can efficiently route data to the appropriate processing pipeline, avoiding unnecessary computational operations and reducing overall energy consumption.
Data Source
AI summary
Systems and methods for hybrid depth regularization in accordance with various embodiments of the invention are disclosed. In one embodiment of the invention, a depth sensing system comprises a plurality of cameras; a processor; and a memory containing an image processing application. The image processing application may direct the processor to obtain image data for a plurality of images from multiple viewpoints, the image data comprising a reference image and at least one alternate view image; generate a raw depth map using a first depth estimation process, and a confidence map; and generate a regularized depth map. The regularized depth map may be generated by computing a secondary depth map using a second different depth estimation process; and computing a composite depth map by selecting depth estimates from the raw depth map and the secondary depth map based on the confidence map.