FOV-Adjusted Depth Seed Fusion for Stable XR Depth Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth estimation techniques in extended reality (XR) systems often produce unstable, low-confidence, and excessively sparse depth maps, particularly on texture-less surfaces and specular objects, leading to inaccurate XR experiences.
Innovation Solution
Fusing sparse depth seeds obtained from multiple depth estimation techniques to generate high-quality depth maps, using a method that involves obtaining depth data from different field of views and generating a fused depth seed associated with a target field of view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple depth estimation techniques are used, then depth map quality improves, but system complexity increases
Solution Approach 1:
The patent combines multiple depth estimation techniques (ToF, stereo, and single-image depth estimation) into a unified depth map generation system. Different depth estimation modules process the same input image simultaneously, and their results are fused through a confidence-weighted fusion mechanism to produce a high-quality depth map, directly resolving the contradiction by merging multiple approaches.
Solution Approach 2:
The system implements a universal depth estimation framework that can handle multiple depth estimation techniques through a single integrated architecture. The confidence map generation and fusion mechanism serves as a multi-functional component that works across different depth estimation methods, reducing overall system complexity while maintaining high depth map quality.
2Area of stationary object
If depth estimation is performed on texture-less surfaces, then coverage area increases, but measurement precision decreases
Solution Approach 1:
The patent merges results from multiple depth estimation techniques with different strengths. ToF provides accurate measurements on texture-less surfaces, while stereo and single-image methods excel on textured surfaces. By combining these complementary approaches and fusing their results based on confidence weights, the system achieves both wide coverage and high precision across all surface types.
Solution Approach 2:
The system dynamically adjusts the confidence weights of different depth estimation techniques based on local image characteristics. For texture-less surfaces, the system increases the weight of ToF-based depth estimation, while for textured surfaces, it favors stereo and single-image methods. This parameter adaptation allows the system to maintain high measurement precision across diverse surface types.
3Area of stationary object
If depth estimation is performed on specular objects, then measurement coverage improves, but reliability decreases
Solution Approach 1:
The patent combines multiple depth estimation techniques that have different performance characteristics on specular surfaces. By fusing the results with confidence-based weighting, the system compensates for the weaknesses of individual methods on specular objects, maintaining both coverage and reliability.
Solution Approach 2:
The system generates confidence maps that provide feedback on the reliability of depth estimates from each technique. For specular objects, the confidence feedback mechanism identifies regions where certain methods fail and adjusts the fusion accordingly, allowing the system to maintain reliable depth estimation even on challenging specular surfaces.
Data Source
AI summary
An example method for estimating depth includes obtaining first depth data from a first depth data source, wherein the first depth data is associated with a first field of view (FOV), obtaining second depth data from a second depth data source, wherein the second depth data is associated with a second FOV, the second FOV being different from the first FOV, generating FOV adjusted depth data based on the second depth data associated with the second FOV, generating a fused depth seed based on the FOV adjusted depth data and at least one of the first depth data or an additional FOV adjusted depth data, and determining a depth map based on the fused depth seed. The FOV adjusted depth data is associated with a target FOV, the target FOV being different from the second FOV. The fused depth seed is associated with the target FOV.