JSON Patch Scene Updates for Timed and Event-Triggered XR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current scene graphs, such as glTF, do not support dynamic scenes containing timed media, as they lack principles for presentation time and event-based updates, making it difficult to update scenes dynamically in extended reality (XR) multimedia.
Innovation Solution
The method involves storing JSON patch update documents in a timed metadata track using ISO/IEC ISOBMFF format, enabling time-triggered and event-triggered updates by referencing a presentation timeline or user interaction, with additional metadata defining conditions and restrictions for scene updates, and allowing random access to scenes at specified timestamps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If scene graphs use static description formats like glTF, then the scene structure is simple and well-defined, but the scene cannot support dynamic updates with timed media
Solution Approach 1:
The scene description is segmented into a base scene graph and separate update documents. The base scene graph contains static information, while updates are applied through separate JSON patch documents that modify specific portions of the scene at defined times or events, enabling dynamic updates without restructuring the entire scene description system
Solution Approach 2:
Update documents are prepared in advance with predefined presentation times and event triggers. The timing information and update operations are specified beforehand, allowing the scene to transition smoothly from static to dynamic state without requiring real-time decision-making or complex runtime generation of update instructions
2Loss of time
If scene updates are applied without defined timing principles, then the update process is flexible, but it becomes unclear when and how to update the scene
Solution Approach 1:
A timing intermediary layer is introduced between the update documents and the scene graph. This intermediary uses defined timing principles (presentation times, event triggers, and synchronization mechanisms) to determine when updates should be applied, providing clear temporal guidance without complicating the update operation itself
Solution Approach 2:
The scene update system incorporates time-related parameters (presentation time, event time, duration) into the update documents. By changing the parameter set to include temporal information, the system gains precise control over when updates occur while maintaining the same basic update mechanism
3Adaptability or versatility
If scene descriptions lack event trigger mechanisms, then the structure remains simple, but user interaction and immersive experiences cannot be supported
Solution Approach 1:
The scene description system transitions from a static structure to a dynamic one by incorporating event trigger mechanisms. Events are defined with specific conditions and actions, allowing the scene to respond to user interactions and external stimuli while maintaining a structured approach through predefined event handlers and response patterns
Data Source
AI summary
The disclosure relates to a method and an apparatus for dynamic scene updates (scenes including timed media) requiring extensions to scene description patch documents supporting timed media. A method for timed and event triggered updates in a scene description for extended reality (XR) multimedia is provided. The method includes parsing a scene description and at least one scene update track sample in a timed scene description update track or at least one scene update item to update the scene description and rendering the updated scene description on a display.