Stereo Depth Mapping with SNR-Based Fallback for Parallax Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing MR systems face challenges in generating accurate depth maps due to parallax errors and poor signal-to-noise ratios (SNR) in stereo camera images, particularly in low light or low ambient conditions, leading to low-quality depth maps that affect the user's perception of the real-world environment.
Innovation Solution
The system computes a smoothness penalty against the smoothness term of a cost function based on the SNR of texture images to improve depth map generation, adjusting the stereo matching algorithm to generate usable depth maps even in low-quality image conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full parallax correction using stereo depth matching is applied, then depth map accuracy is improved, but system reliability deteriorates in low SNR conditions
Solution Approach 1:
The system dynamically adjusts the depth mapping approach based on real-time SNR conditions. When SNR is high, full stereo depth matching is used for maximum accuracy. When SNR is low, the system transitions to planar reprojection to maintain reliability, thus making the system adaptive to changing environmental conditions rather than static
Solution Approach 2:
The system changes the operational parameters of the depth mapping algorithm based on SNR measurements. By monitoring signal quality and adjusting the processing approach (from full parallax correction to planar reprojection), the system optimizes performance across varying environmental conditions while maintaining both accuracy and reliability
2Loss of information
If stereo depth matching is performed on low quality images, then depth information is obtained, but depth map quality deteriorates due to poor SNR
Solution Approach 1:
The system introduces SNR measurement as an intermediary assessment step between image capture and depth mapping. This intermediary evaluation allows the system to select the appropriate processing pathway (full stereo matching or planar reprojection) based on image quality, thus preventing degradation of depth map quality while preserving depth information availability
3Adaptability or versatility
If gradual fallback from full parallax correction to planar reprojection is implemented, then system adaptability is improved, but device complexity increases
Solution Approach 1:
The depth mapping system is segmented into distinct processing pathways: full stereo depth matching for high SNR conditions and planar reprojection for low SNR conditions. This segmentation allows the system to adapt to different environmental conditions while keeping each individual pathway relatively simple, managing overall complexity through modular design
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Improved techniques for generating depth maps are disclosed. A stereo pair of images of an environment is accessed. This stereo pair of images includes first and second texture images. A signal to noise ratio (SNR) is identified within one or both of those images. Based on the SNR, which may be based on the texture image quality or the quality of the stereo match, there is a process of selectively computing and imposing a smoothness penalty against a smoothness term of a cost function used by a stereo depth matching algorithm. A depth map is generated by using the stereo depth matching algorithm to perform stereo depth matching on the stereo pair of images. The stereo depth matching algorithm performs the stereo depth matching using the smoothness penalty.