VST AR Parallax Correction Using Verified Depth Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optical see-through (OST) AR systems face challenges such as limited fields of view, indoor-only usage, and complex optical pipelines, while video see-through (VST) AR systems struggle with parallax correction due to differing camera and eye viewpoints, leading to suboptimal user experiences.
Innovation Solution
A method and system for parallax correction in VST AR that involves generating feature maps, disparity maps, and verified depth maps using high-resolution see-through cameras and depth sensors to align virtual views with user eye viewpoints, employing algorithms for depth clarification and noise reduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If video see-through (VST) AR systems are used to overcome OST limitations, then field of view and outdoor usability are improved, but parallax correction accuracy deteriorates due to differing camera and eye viewpoints
Solution Approach 1:
The system performs preliminary depth estimation using a depth sensor before image processing, creating an initial depth map that guides subsequent feature matching and parallax correction operations. This preliminary action enables the system to prepare depth information in advance, improving the accuracy of viewpoint transformation despite the fundamental camera-eye position mismatch in VST AR systems
Solution Approach 2:
The system implements an iterative feedback loop where depth maps are refined through multiple passes: initial depth estimation, feature matching, disparity calculation, and depth map refinement. Each iteration uses the results of previous steps to improve accuracy, with the refined depth maps feeding back into the parallax correction process to progressively reduce errors from viewpoint differences
2Manufacturing precision
If complex image processing algorithms are applied to improve depth map accuracy, then virtual object placement precision is improved, but processing time increases
Solution Approach 1:
The system extracts image features and generates initial depth maps before performing computationally intensive disparity calculations and depth refinement. By preparing feature maps and initial depth estimates in advance, the system reduces the real-time processing burden during critical display updates while maintaining high precision in virtual object placement
Solution Approach 2:
The system applies full-depth refinement algorithms selectively rather than uniformly across all pixels. It focuses computational resources on regions with significant depth variations or uncertainty, using simplified processing for areas with clear depth information, thereby achieving high placement precision where needed while reducing overall processing time
Data Source
AI summary
A method includes obtaining a stereo image pair including a first image and a second image. The method also includes generating a first feature map of the first image and a second feature map of the second image, the first and second feature maps including extracted positions associated with pixels in the images. The method further includes generating a disparity map between the first and second images based on a dense depth map. The method also includes generating a verified depth map based on a pixelwise comparison of predicted positions and the extracted positions associated with at least some of the pixels in at least one of the images, the predicted positions determined based on the disparity map. In addition, the method includes generating a first virtual view and a second virtual view to present on a display panel of an VST AR device based on the verified depth map.