XR Telepresence Audio Video Management Spatial Culling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Extended reality (XR) telepresence systems face challenges in interface design, security, and scalability, particularly in 3D environments, due to bandwidth limitations and latency issues, which affect the ability to support multiple users with immersive and interactive video conferencing experiences.
Innovation Solution
A telepresence management system that dynamically adjusts audio and video qualities based on spatial relationships between users and implements frustum video culling, along with dynamic server allocation to optimize resource utilization and reduce latency, allowing for scalable and immersive XR telepresence sessions with tens or thousands of users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all video streams are transmitted to all users in an XR telepresence session, then complete visual information is provided to each user, but bandwidth consumption increases significantly
Solution Approach 1:
The patent implements frustum culling to transmit video streams selectively based on each user's field of view. Instead of providing complete visual information to all users, the system determines which video streams are relevant to each user's current view direction and transmits only those streams. This localizes the quality of information transmission to where it is actually needed, reducing overall bandwidth consumption while maintaining visual information completeness for each user's perspective.
2Manufacturing precision
If high-resolution video streams are transmitted to all users, then video quality is improved, but network bandwidth requirements increase
Solution Approach 1:
The system transmits high-resolution video streams selectively based on spatial relationships and user field of view. Video quality is maintained at high resolution only for streams that are currently visible to users, while other streams are transmitted at lower resolution or not at all. This localized quality approach ensures video quality improvement where needed without proportionally increasing network bandwidth requirements across the entire system.
3Productivity
If the system supports thousands of concurrent users, then scalability is improved, but system complexity increases
Solution Approach 1:
The patent segments the user base into spatial groups or clusters based on their positions and interactions in the virtual environment. Instead of managing all thousands of users as a single homogeneous system, the server divides users into manageable segments that can be processed independently. This segmentation allows the system to scale to thousands of users by handling each segment separately, reducing overall system complexity while maintaining high user capacity.
Solution Approach 2:
The system dynamically adjusts video stream transmission based on real-time spatial relationships between users. As users move and change their field of view, the system dynamically determines which video streams to transmit and to which users. This dynamic approach allows the system to efficiently manage thousands of concurrent users by adapting resource allocation to current spatial configurations, reducing complexity compared to static resource allocation methods.
4Loss of energy
If video streams are transmitted based on user field of view, then bandwidth usage is reduced, but processing complexity increases
Solution Approach 1:
The system performs preliminary calculations of user field of view and spatial relationships before transmitting video streams. By pre-determining which video streams are relevant to each user based on their current position and orientation, the system avoids the need for complex real-time processing during stream transmission. This preliminary action reduces bandwidth usage while keeping processing complexity manageable by performing calculations in advance rather than during active data transmission.
Data Source
AI summary
Some embodiments of the present inventive concept provide for improved telepresence and other virtual sessions using localized projection of audible noises and/or dynamic adjustment of audio and/or video qualities based on spatial relationships between users. An XR telepresence platform can allow for immersive multi-user video conferencing from within a web browser or other medium. The platform can support spatial audio and/or user video. The platform can scale to hundreds or thousands of users concurrently in a single or multiple virtual environments. Disclosed herein are quality-of-service techniques for dynamically selecting or modifying audio and/or video traffic.