Distributed Virtual Representation Generation via Mesh Data Exchange
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current extended reality (XR) systems face challenges in efficiently generating high-quality virtual representations of users in virtual environments, such as metaverses, due to computational complexity and resource demands, which limit their ability to provide low-latency and high-quality XR experiences, especially in mobile and wearable devices.
Innovation Solution
A distributed system for generating virtual content that involves receiving input information from user devices, processing it to create virtual representations, and transmitting frames depicting these representations to other devices, allowing for efficient communication and reduced data transmission rates through mesh information and animation parameters exchange.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a single device generates virtual representations of users, then quality of virtual representation is improved, but computational complexity and resource demands increase
Solution Approach 1:
The system segments the virtual representation generation task across multiple devices. Each device captures local information (images, videos, sensor data) and processes it locally to generate virtual representations, while a central server coordinates the overall process. This segmentation reduces the computational burden on any single device while maintaining high representation quality.
Solution Approach 2:
A central server acts as an intermediary between multiple user devices. The server receives input information from various devices, coordinates the generation of virtual representations, and distributes the results back to the devices. This intermediary approach allows devices to focus on data collection while the server handles complex processing tasks.
2Manufacturing precision
If high-quality virtual representations are generated, then XR experience quality is improved, but processing time and latency increase
Solution Approach 1:
The system performs preliminary actions by capturing and processing visual and sensor data in advance before the actual virtual representation generation is needed. Devices continuously collect images, videos, and sensor information, which are pre-processed and stored for later use in generating virtual representations, reducing real-time processing delays.
Solution Approach 2:
By segmenting the processing task across multiple devices with local processing capabilities, the system can generate virtual representations in parallel rather than sequentially. Each device processes its own captured data independently, significantly reducing the overall time required to generate high-quality representations.
3Speed
If virtual content is generated locally on each device, then data transmission speed is improved, but device resources are exhausted
Solution Approach 1:
Each device processes only the local information it captures (images, videos, sensor data from its own sensors) to generate virtual representations relevant to its user. This local quality approach ensures that devices transmit only necessary data rather than processing and transmitting all data locally, optimizing the balance between transmission speed and resource consumption.
Solution Approach 2:
The central server acts as an intermediary that receives data from multiple devices, processes it to generate virtual representations, and distributes the results back to the devices. This eliminates the need for each device to independently process all data, significantly reducing the computational resources required at each device while maintaining efficient data transmission.
Data Source
AI summary
Systems and techniques are described for establishing one or more virtual sessions between users. For instance, a first device can transmit, to a second device, a call establishment request for a virtual representation call for a virtual session and can receive, from the second device, a call acceptance indicating acceptance of the call establishment request. The first device can transmit, to the second device, first mesh information for a first virtual representation of a first user of the first device and first mesh animation parameters for the first virtual representation. The first device can receive, from the second device, second mesh information for a second virtual representation of a second user of the second device and second mesh animation parameters for the second virtual representation. The first device can generate, based on the second mesh information and the second mesh animation parameters, the second virtual representation of the second user.