XR Experience State Hierarchy for Resource-Aware Augment Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) and virtual reality (VR) systems lack an efficient framework for managing multiple experiences and their corresponding augments, leading to resource wastage and suboptimal user interaction, as they often consume equal resources without prioritization.
Innovation Solution
Implementing a hierarchical state management system for AR and VR experiences and augments, defining states such as active, foregrounded, backgrounded, and closed, along with focused, displayed, minimized, and unloaded states, to optimize resource allocation and user interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple AR experiences are managed without a hierarchical state system, then user experience flexibility is improved, but resource consumption increases and battery life decreases
Solution Approach 1:
The system segments AR experiences and augments into distinct hierarchical levels with defined states. Experiences are divided into active, foregrounded, and backgrounded states, while augments are divided into focused, displayed, minimized, and unloaded states. This segmentation allows the system to manage multiple experiences efficiently by allocating resources based on state priority, reducing overall battery consumption while maintaining user flexibility.
Solution Approach 2:
The system implements dynamic state transitions for both experiences and augments based on user interaction and system conditions. Experiences can transition between active, foregrounded, and backgrounded states, while augments transition between focused, displayed, minimized, and unloaded states. This dynamic management allows the system to adapt resource allocation in real-time, optimizing battery usage without compromising user experience flexibility.
2Manufacturing precision
If all augments are rendered with high fidelity, then visual quality is improved, but processing power consumption increases
Solution Approach 1:
The system applies different rendering qualities to different augments based on their state and importance. Focused augments receive high-fidelity rendering with full visual quality, while displayed augments receive moderate quality, and minimized or unloaded augments receive low or no rendering. This local quality differentiation maintains visual quality for important elements while significantly reducing overall processing power consumption.
Solution Approach 2:
The system renders only the necessary portion of each augment based on its state. Focused augments receive complete rendering attention, while displayed augments receive partial rendering with reduced fidelity, and minimized or unloaded augments receive minimal or no rendering. This partial action approach ensures that processing power is allocated efficiently, focusing computational resources on augments that require high visual quality while reducing effort for less critical elements.
3Productivity
If a hierarchical state management system is implemented, then resource efficiency is improved, but system complexity increases
Solution Approach 1:
The hierarchical state management system is segmented into two distinct levels: experience states (active, foregrounded, backgrounded) and augment states (focused, displayed, minimized, unloaded). This segmentation simplifies the overall system complexity by creating clear, manageable state categories with well-defined transition rules, making the complex resource management task more tractable while maintaining high resource efficiency.
4Adaptability or versatility
If multiple experiences run simultaneously without state prioritization, then user interaction options increase, but resource allocation efficiency decreases
Solution Approach 1:
The system dynamically allocates resources to multiple experiences based on their current state and user interaction patterns. Active experiences receive maximum resources, foregrounded experiences receive moderate resources, and backgrounded experiences receive minimal resources. This dynamic prioritization allows multiple experiences to run simultaneously with flexible user interaction options while maintaining efficient resource allocation, as resources are continuously adjusted based on current system conditions.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Aspects of the present disclosure are directed to providing an augmented world environment model for an artificial reality (XR) environment. Some implementations of the present technology define how experiences will run on an XR system as a hierarchy of entities, including experiences, augments those experiences create, and an exclusive mode in which a particular experience takes control of the entire view. Some implementations can establish a framework for multiple experiences to interface with each other and to run in an XR ecosystem. This can improve upon traditional mobile and desktop operating systems by providing different states optimized for XR, which can include improved utilization of resources on an XR device, such as an XR head-mounted display. Experiences and augments can enter a particular state based on criteria, and can be, based on the state, processed differently by an operating system or provided with particular capabilities.