Low Compute Depth Map Generation via Downsampling and Edge-Preserving Filters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional mixed-reality systems face challenges in generating depth maps due to high computational costs and latency issues, particularly when addressing parallax problems, which can result in imprecise depth borders and temporal inconsistencies, degrading the user experience.
Innovation Solution
The system employs a method for low compute depth map generation by downsampling stereo pairs of images, performing stereo matching, and using edge-preserving filters to generate upsampled depth maps, reducing computational burden and noise while improving precision and smoothness of depth-dependent operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional stereo matching is performed on full-resolution images to generate depth maps, then measurement precision of depth borders is improved, but computational cost and processing time increase significantly
Solution Approach 1:
The patent segments the depth map generation process into multiple resolution levels. It first generates a coarse depth map at low resolution, then progressively refines it at higher resolutions. This multi-scale approach allows the system to capture global depth structures efficiently while adding local details only where necessary, resolving the contradiction between precision and speed.
Solution Approach 2:
The patent performs preliminary depth estimation at low resolution before refining at full resolution. By pre-computing the coarse depth structure first, the system establishes a foundation that guides subsequent high-resolution processing, reducing the overall computational burden while maintaining final precision.
2Measurement precision
If stereo matching is performed on high-resolution stereo pairs to reduce noise, then measurement precision is improved, but latency increases
Solution Approach 1:
The patent divides the processing into temporal stages: first processing at low resolution to establish baseline depth information quickly, then progressively refining at higher resolutions. This segmentation reduces latency by delivering usable depth information earlier while continuing to improve precision in the background.
Solution Approach 2:
The patent implements progressive refinement where the system can deliver partial results at intermediate resolutions before completing full high-resolution processing. This allows the system to provide acceptable depth maps faster while continuing to refine them, reducing perceived latency.
3Reliability
If conventional depth map generation methods are used, then depth information is obtained, but temporal inconsistencies and artifacts appear in the output
Solution Approach 1:
The patent performs preliminary processing at low resolution to establish a stable baseline depth structure that is less prone to noise and artifacts. This preliminary coarse depth map serves as a reference that constrains subsequent high-resolution processing, preventing temporal inconsistencies from propagating to the final output.
Solution Approach 2:
The patent adds the resolution dimension to the processing pipeline, treating depth map generation as a multi-resolution problem rather than a single-resolution problem. By solving the problem across multiple resolution dimensions and combining results, the system achieves temporal consistency that single-resolution methods cannot provide.
Data Source
AI summary
Systems and methods are provided performing for low compute depth map generation by implementing acts of obtaining a stereo pair of images of a scene, downsampling the stereo pair of images, generating a depth map by stereo matching the downsampled stereo pair of images, and generating an upsampled depth map based on the depth map using an edge-preserving filter for obtaining at least some data of at least one image of the stereo pair of images.