Extended Reality Scene Graph Interactivity Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current extended reality (XR) systems lack the ability to describe how users can interact with scene objects at runtime and do not provide a mechanism for updating these interactions during the rendering of XR applications, limiting immersive XR experiences.
Innovation Solution
Incorporating behavior metadata into XR scene descriptions, which includes triggers, actions, and behaviors that allow for user-specific interactivity, enabling time-evolving and updateable interactions through a scene description update mechanism, and utilizing a node tree structure with interactivity extensions like MPEG_scene_interactivity to manage these interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If scene description frameworks ensure timed media and virtual content availability, then content reliability is improved, but interactivity and user engagement remain limited
Solution Approach 1:
The patent applies dynamics by making the scene description framework adaptable and updateable at runtime. The system transitions from static content availability to dynamic interactivity where behaviors can be modified, added, or removed during execution. This is achieved through the interactivity extension that allows runtime updates to behavior metadata while maintaining scene graph integrity.
2Loss of information
If XR systems provide comprehensive scene description, then scene representation completeness is improved, but runtime interactivity update capability is lost
Solution Approach 1:
The patent segments the scene description into distinct components: core scene graph structure and interactivity extension. This segmentation allows the interactivity portion to be updated independently at runtime without affecting the fundamental scene description integrity. The interactivity extension acts as a separate, modifiable layer that enhances rather than replaces the base scene description framework.
3Adaptability or versatility
If behavior metadata is incorporated into scene descriptions, then user interactivity is improved, but scene description complexity increases
Solution Approach 1:
The patent implements nesting by embedding behavior metadata within the existing scene graph node structure. Rather than creating a completely separate interactivity framework, the behavior information is nested within node properties and relationships. This allows interactivity to be integrated into the existing scene description hierarchy, reducing the perceived complexity while maintaining full functionality.
Data Source
AI summary
An extended reality scene description is provided comprising relationships between real and virtual objects and interactive triggering and processing of the extended reality scene; An extended reality system can read the scene description and run a corresponding extended reality application. The scene description comprises a scene graph structuring descriptions of the real and virtual objects which may be linked to media content items. It also comprises of behavior metadata items describing how a user can interact with the scene objects at runtime. Method and devices are disclosed to manage the behaviors comprising triggers and actions and to manage the on-going behaviors when a second scene description is received.