Virtual Space Traversal With Prefetched Adjacent View Bitstreams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing immersive XR technologies face significant latency and bandwidth challenges due to the need for a client device to receive whole bitstreams from multiple camera views when transitioning between virtual spaces, leading to increased computational complexity and latency in viewport switching.
Innovation Solution
A system that generates high-quality bitstreams for the current space and lower-quality bitstreams for adjacent spaces in advance, using multi-pass encoding and inter-space pruning to reduce latency and bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a client device receives whole bitstreams from multiple camera views when transitioning between virtual spaces, then the viewport switching is complete, but the latency and computational complexity increase significantly
Solution Approach 1:
The system performs preliminary encoding of multiple camera views using multi-pass encoding before the user actually transitions between spaces. Lower-quality bitstreams for adjacent spaces are prepared in advance, so when a transition occurs, the client device already has pre-encoded data available, eliminating the need to receive and process complete bitstreams in real-time.
Solution Approach 2:
The patent applies different quality levels to different spatial locations. The current active space is encoded with high quality, while adjacent spaces are encoded with lower quality. This allows the system to maintain visual fidelity where the user is currently looking while reducing bandwidth and processing requirements for spaces the user might transition to, thereby reducing latency without significantly impacting perceived quality.
2Reliability
If multiple camera views are compressed using independent MIV encoders for each virtual space, then each space has complete encoding, but the bandwidth requirements and computational complexity increase
Solution Approach 1:
The system encodes camera views at different quality levels based on spatial importance. The current space receives full-quality encoding while adjacent spaces receive lower-quality encoding. This differential encoding approach reduces the total bandwidth requirement while maintaining encoding completeness for the active space, allowing the system to serve multiple spaces without transmitting full-bitrate data for all of them simultaneously.
Solution Approach 2:
Instead of providing full-quality encoding for all camera views in all spaces, the system provides partial encoding (lower quality) for adjacent spaces and full encoding only for the current active space. This partial action approach reduces bandwidth consumption while still providing sufficient data for seamless transitions, as the lower-quality adjacent space bitstreams are adequate for preview and transition purposes.
3Adaptability or versatility
If the client device receives complete bitstreams for adjacent spaces, then space transitions can be performed, but the computational complexity and processing time increase
Solution Approach 1:
The server performs the computationally intensive encoding work in advance using multi-pass encoding, preparing bitstreams for multiple spaces before the user needs them. This shifts the computational burden from the client device to the server, allowing the client to receive pre-processed data with minimal local processing required, thereby reducing device complexity while maintaining transition capability.
Solution Approach 2:
The system reduces computational complexity by encoding adjacent spaces at lower quality levels. Lower-quality encoding requires less computational resources both on the server side during preparation and on the client side during reception and processing. This differential quality approach maintains the ability to perform space transitions while significantly reducing the computational burden on the client device.
Data Source
AI summary
Systems and methods are provided for traversing virtual spaces. The system receives first and second image data of respective views of an environment simultaneously captured by a first and a second plurality of cameras associated with a first and a second space of the environment, respectively. The system detects, based on at least one of the first or the second image data, that a subject is located within the first space of the environment. In response to detecting that the subject is located within the first space of the environment: the system generates, for transmission at a first bitrate, a first bitstream based on at least a portion of the first image data; and the system generates, for transmission at a second bitrate lower than the first bitrate, a second bitstream based on at least a portion of the second image data.