Binocular Depth Super-Resolution for Consistent XR Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR systems face issues with mismatched depth spaces between the left and right eyes when using binocular input, leading to depth loss and inaccurate regions due to factors like parallax and depth holes, especially with objects such as mirror metal and low-reflectivity surfaces.
Innovation Solution
A method that processes binocular images by generating high-resolution depth maps using cross-view correlations between left and right-eye images, ensuring consistency through a binocular depth super-resolution module that enhances depth quality and consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If monocular input is used for depth processing, then processing simplicity is maintained, but binocular depth space mismatch occurs leading to depth loss and inaccurate regions
Solution Approach 1:
The patent merges monocular depth estimation results from both left and right eyes with binocular disparity information. The depth processing module combines these multiple depth sources to generate a unified, accurate depth map that resolves the depth space mismatch problem while maintaining processing efficiency through integrated computation.
2Loss of information
If binocular input is used for depth processing, then depth information completeness is improved, but depth space mismatch between left and right eyes occurs
Solution Approach 1:
The patent implements a feedback mechanism where the depth processing module continuously refines depth estimates by comparing left and right eye depth maps. The system uses the correlation between binocular images to identify and correct depth space mismatches, feeding back adjusted depth values until consistency is achieved across both eyes.
Solution Approach 2:
The patent changes the parameter representation of depth information by generating high-resolution depth maps from lower-resolution inputs. The depth processing module transforms depth parameters through super-resolution techniques and binocular fusion, converting inconsistent low-resolution depth data into consistent high-resolution depth maps that resolve space mismatch issues.
3Measurement precision
If high-resolution depth maps are generated for both eyes, then depth quality is improved, but processing complexity and computational load increase
Solution Approach 1:
The patent applies preliminary action by first generating lower-resolution depth maps from binocular images, then using these as inputs for subsequent super-resolution processing. This staged approach allows the system to perform initial depth estimation efficiently before enhancing resolution, reducing overall computational complexity compared to generating high-resolution maps directly.
Solution Approach 2:
The patent uses copying by generating depth maps for one eye and then creating corresponding depth maps for the other eye through correlation-based transformation. Instead of independently processing both eyes to full resolution, the system copies and adapts depth information across binocular views, reducing redundant computation while maintaining high resolution in the final output.
Data Source
AI summary
A extended reality (XR) processing method includes acquiring a binocular image including a left-eye image including at least one of a first image and a first depth map, and a right-eye image including at least one of a second image and a second depth map, generating a third depth map based on the left-eye image and a correlation between the left-eye image and the right-eye image, and generating a fourth depth map based on the right-eye image and the correlation between the left-eye image and the right-eye image, and performing extended reality (XR) processing on the binocular image based on the third depth map and the fourth depth map, wherein a resolution of the third depth map is greater than a resolution of the first depth map, and a resolution of the fourth depth map is greater than a resolution of the second depth map.