Virtual Surface Anchoring for Stable AR Passthrough Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing passthrough technologies in artificial reality systems suffer from instability and resource-intensive rendering due to continuous depth calculations and inaccurate depth estimations, leading to inconsistent and aesthetically poor user experiences, especially when interacting with physical objects.
Innovation Solution
The system defines a fixed virtual surface in the virtual environment that corresponds to a frequently interacted-with physical surface, allowing passthrough images to be displayed without continuous depth calculations, using a one-time setup to anchor the virtual surface's pose, and rendering images from the user's perspective.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If continuous depth calculations are performed for passthrough images, then depth estimation accuracy may be improved, but system resource consumption increases and stability decreases
Solution Approach 1:
The system performs depth calculations only once during the initial setup phase to establish the virtual surface pose, rather than continuously recalculating. This preliminary action stores the depth information for reuse, significantly reducing subsequent computational resource consumption while maintaining stable passthrough image display.
Solution Approach 2:
The patent creates a virtual copy of the physical surface with fixed pose parameters. This virtual surface copy is used for rendering passthrough images without requiring continuous depth sensing and calculation, thereby reducing system resource usage while maintaining visual consistency.
2Reliability
If continuous depth calculations are performed for passthrough images, then depth information may be updated, but image stability deteriorates due to inconsistent rendering
Solution Approach 1:
The virtual surface pose is determined in advance during setup and remains fixed during operation. This preliminary determination eliminates continuous depth calculations that cause rendering inconsistencies, thereby improving image stability without sacrificing rendering efficiency.
3Use of energy by moving object
If a fixed virtual surface is defined instead of continuous depth calculation, then resource efficiency improves, but adaptability to dynamic environments decreases
Solution Approach 1:
The system performs environment adaptation once during the setup phase by capturing images and determining the virtual surface pose. This preliminary adaptation is sufficient for stable passthrough rendering, eliminating the need for continuous environmental scanning and calculation, thus achieving both resource efficiency and adequate adaptability.
Data Source
AI summary
A method includes detecting an object of interest in a real environment and depth information of the object; determining one or more anchor locations in a three-dimensional space that correspond to a position of the object in the three-dimensional space; and generating a virtual surface anchored in the three-dimensional space. The method may further determine a pose of a camera when an image is captured and determine a region in the image that corresponds to the virtual surface. The method may further determine a first viewpoint of a first eye of the user; render a first output image based on (1) the first viewpoint relative to the virtual surface and (2) the image region corresponding to the virtual surface; and display the first output image on a first display of the computing device, the first display being configured to be viewed by the first eye of the user.