AR-Guided Depth Estimation Using Rendering-Driven ROIs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) devices face computational and power constraints in estimating depth due to limited resources, leading to inefficient use of computational resources and power consumption when performing depth estimation for the entire image frame.
Innovation Solution
A system that utilizes feedback from a graphics rendering engine to predict the area of interest in the camera's field of view and estimates depth only for this limited region, leveraging a six-degrees of freedom tracking system to save computational resources and power by determining depth for a specific area rather than the entire image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth estimation is performed for the entire image frame, then depth information completeness is improved, but computational resource consumption and power usage increase
Solution Approach 1:
The patent divides the image frame into multiple regions of interest (ROIs) based on virtual object positions and user gaze directions. Depth estimation is performed only within these segmented ROIs rather than the entire frame, reducing computational load and power consumption while maintaining necessary depth information for augmented reality rendering.
Solution Approach 2:
The patent applies different processing quality levels to different regions of the image. High-quality depth estimation is applied only to regions containing virtual objects or within the user's field of view, while other regions receive reduced or no depth processing. This local quality differentiation optimizes the balance between depth information completeness and power consumption.
2Measurement precision
If depth estimation is performed for the entire image frame, then depth information completeness is improved, but computational resource consumption increases
Solution Approach 1:
The patent segments the image frame into multiple regions of interest (ROIs) based on virtual object positions and user gaze directions. Depth estimation is performed only within these segmented ROIs rather than the entire frame, reducing computational load and power consumption while maintaining necessary depth information for augmented reality rendering.
Solution Approach 2:
The patent performs depth estimation only for necessary portions of the image frame (partial action) rather than the entire frame. By identifying regions containing virtual objects and user gaze directions, the system applies depth estimation selectively to these partial regions, reducing computational resource consumption while sufficient depth information is obtained for AR rendering.
3Productivity
If feedback from graphics rendering engine is used to predict area of interest, then resource usage efficiency is improved, but system complexity increases
Solution Approach 1:
The patent merges the graphics rendering engine with the depth estimation system, allowing the rendering engine's feedback about virtual object positions and user gaze to directly inform depth estimation regions. This integration enables efficient resource usage by combining information from both systems to identify areas requiring depth processing.
Solution Approach 2:
The patent implements a feedback loop where the graphics rendering engine provides information about virtual object positions and user gaze directions to the depth estimation module. This feedback mechanism enables dynamic adjustment of depth estimation regions based on current rendering needs, optimizing resource usage efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for AR-guided depth estimation is described. The method includes identifying a virtual object rendered in a first frame that is generated based on a first pose of an augmented reality (AR) device, determining a second pose of the AR device, the second pose following the first pose, identifying an augmentation area in the second frame based on the virtual object rendered in the first frame, and the second pose, determining depth information for the augmentation area in the second frame, and rendering the virtual object in the second frame based on the depth information.