Avatar Action Synchronization Using Spatiotemporal Scene Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote communication systems using avatars face challenges due to latency discrepancies between the timing of avatar actions and the actual actions of remote individuals, leading to ineffective communication.
Innovation Solution
An information processing device generates scene graphs based on sensor measurements to predict and synchronize avatar actions, using spatiotemporal scene graphs to align actions across different spaces, thereby reducing latency and ensuring synchronized communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If real-time action data transmission is used to reproduce avatar actions, then communication responsiveness is improved, but latency discrepancies cause timing mismatches between expected and actual avatar actions
Solution Approach 1:
The system pre-generates multiple candidate action sequences offline and stores them in a database. When real-time action recognition occurs, the system selects and retrieves the most appropriate pre-generated candidate action that matches the recognized action pattern, rather than generating actions in real-time. This preliminary preparation eliminates timing mismatches while maintaining responsiveness.
Solution Approach 2:
The system changes the parameter of action generation from real-time synthesis to retrieval of pre-generated candidates with varying temporal parameters. By storing multiple candidate actions with different timing characteristics and selecting based on match quality, the system achieves both responsiveness and synchronization reliability.
2Measurement precision
If motion capture systems are used to obtain high-fidelity avatar reproduction, then action accuracy is improved, but system complexity and processing requirements increase
Solution Approach 1:
Instead of using complex real-time motion capture and reproduction systems, the patent creates simplified representations by generating candidate action sequences that capture the essential patterns of human actions. These candidate actions serve as copies or approximations that are sufficient for effective communication without requiring full-fidelity motion capture infrastructure.
Solution Approach 2:
The system performs complex action pattern analysis and candidate generation in advance offline, storing results for rapid retrieval. This shifts computational complexity from real-time operation to preliminary preparation, reducing the complexity of the actual communication system while maintaining high action recognition accuracy through pre-processed data.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present technology relates to an information processing device, an information processing method, and a recording medium that enable persons located remotely from each other to smoothly communicate via their respective avatars. An information processing device according to one aspect of the present technology is configured to: generate, on the basis of a sensor measurement result indicating a plurality of objects, a first scene graph including nodes that represent the plurality of objects at different times and are connected by a change edge representing a temporal change in state, the plurality of objects including a first person watching a second avatar that reproduces an action corresponding to an action of a second person and the second person watching a first avatar that reproduces an action corresponding to an action of the first person; and select an action to be reproduced by the second avatar. The present technology can be applied to a system in which persons located remotely from each other wear AR display devices and communicate via their respective avatars.