AR/VR Interaction Recording Format for Low-Bandwidth Replay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented and virtual reality technologies lack industry standards for input mechanisms, leading to challenges in recording and replaying user actions for training and interactive content, particularly due to storage space, processor computation, and transmission bandwidth constraints.
Innovation Solution
A portable file format is developed to capture user inputs, such as gestures, voice commands, and textual messages, allowing for efficient storage, computation, and distribution of interactive content across various AR/VR devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If traditional video/image recording methods are used to capture user actions in AR/VR environments, then comprehensive recording of user interactions is achieved, but storage space requirements increase significantly
Solution Approach 1:
The patent extracts only the essential interactive elements (user inputs, command sequences, object interactions) from the complete AR/VR experience, storing merely the data necessary to replay interactions rather than capturing entire video streams. This selective extraction reduces storage requirements while preserving the ability to reconstruct user actions.
Solution Approach 2:
The recording system segments the AR/VR experience into discrete interactive events (gestures, voice commands, object selections) rather than continuous video streams. Each interaction is captured as an individual data element that can be stored and replayed independently, significantly reducing the total data volume required.
2Measurement precision
If high-fidelity recording of AR/VR interactions is implemented, then accurate replay of user actions is achieved, but processor computation requirements increase
Solution Approach 1:
Instead of storing and processing complete sensory data from AR/VR environments, the system creates simplified copies consisting of structured data elements (input types, command sequences, object identifiers). These lightweight data copies require minimal computational resources to store and replay while maintaining accurate representation of user interactions.
3Loss of information
If detailed user action data is transmitted for cloud processing, then comprehensive analysis is achieved, but transmission bandwidth requirements increase
Solution Approach 1:
The system extracts only the essential interaction data elements needed for analysis and replay, transmitting merely the structured interaction records rather than complete sensory streams. This extraction approach enables comprehensive interaction analysis while minimizing bandwidth consumption during data transmission.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method includes receiving (404) data defining user actions associated with an AR/VR space. The method also includes translating (410) the user actions into associated commands and identifying (412) associations of the commands with visual objects in the AR/VR space. The method further includes aggregating (414) the commands, the associations of the commands with the visual objects, and an AR/VR environment setup into at least one file (120) and transmitting or storing (416) the file(s). Another method includes receiving (504) at least one file (120) containing commands, associations of the commands with visual objects in an AR/VR space, and an AR/VR environment setup. The other method also includes translating (508) the commands into associated user actions and recreating or causing a user device (204, 206, 208) to recreate (512-514) (i) the AR/VR space containing the visual objects based on the AR/VR environment setup and (ii) the user actions in the AR/VR space based on the associations of the commands with the visual objects.