Latency-Adaptive Viewport Prediction for VR/AR Content Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing viewport prediction methods in virtual reality and augmented reality content streaming suffer from accuracy loss due to a fixed prediction interval that does not adapt to dynamic changes in network status, streaming content complexity, and device capabilities, leading to increased motion-to-photon latency and user discomfort.
Innovation Solution
A dynamic viewport prediction system that selects the most appropriate model from a pool based on real-time feedback of compensation latency, dividing latency into roundtrip network and process components, and using weighted averages to estimate compensation latency for accurate viewport prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a fixed prediction interval is used for viewport prediction, then the system complexity is reduced, but the prediction accuracy deteriorates due to inability to adapt to dynamic network and device conditions
Solution Approach 1:
The patent implements dynamic viewport prediction by selecting different prediction models based on current latency conditions. The system divides prediction models into multiple groups with different time intervals and dynamically selects the appropriate group based on real-time compensation latency measurements, making the prediction interval adaptive rather than fixed.
Solution Approach 2:
The system changes the prediction interval parameter dynamically by selecting from multiple prediction model groups with different time intervals (e.g., 10ms, 20ms, 30ms). This parameter change allows the system to adapt to varying network and device conditions while maintaining prediction accuracy.
2Ease of manufacture
If a fixed prediction interval is used, then the system is easier to implement, but motion-to-photon latency increases due to prediction accuracy loss
Solution Approach 1:
The system dynamically adjusts the prediction interval by selecting from multiple pre-defined model groups based on real-time latency measurements. This dynamic adaptation reduces motion-to-photon latency by ensuring the prediction interval matches current system conditions, while still maintaining implementation feasibility through pre-defined model groups.
Solution Approach 2:
The patent prepares multiple prediction model groups with different time intervals in advance, categorized by compensation latency ranges. This preliminary preparation allows the system to quickly select an appropriate model without complex real-time calculations, balancing implementation ease with latency reduction.
3Measurement precision
If dynamic model selection based on latency feedback is implemented, then viewport prediction accuracy is improved, but device complexity increases
Solution Approach 1:
The patent segments the prediction models into multiple groups with different time intervals, each optimized for specific latency conditions. This segmentation allows the system to manage complexity by organizing models into discrete, manageable categories rather than implementing a single complex adaptive model.
Solution Approach 2:
The system implements feedback by measuring compensation latency and using this information to select the appropriate prediction model group. This feedback mechanism improves prediction accuracy by adapting to real-time conditions while keeping the selection logic simple through predefined latency thresholds.
4Adaptability or versatility
If multiple prediction models with different time intervals are maintained, then adaptability to dynamic conditions is improved, but loss of information increases due to bandwidth requirements
Solution Approach 1:
The system dynamically selects from multiple prediction model groups based on current latency conditions, maintaining adaptability to changing network and device environments. By activating only the necessary model group for current conditions, the system avoids the bandwidth overhead of maintaining and transmitting all possible models simultaneously.
Solution Approach 2:
Different prediction models are optimized for specific latency conditions (local quality), with each model group tailored to particular compensation latency ranges. This allows the system to use the most appropriate model for current conditions without wasting bandwidth on models that would not be applicable.
Data Source
AI summary
This disclosure describes systems, methods, and devices related to latency-adaptive viewport prediction for use in viewport-dependent content streaming. A method may include identifying, by a virtual reality (VR) or augmented reality (AR) device, first viewport data used by and received from a display device; generating a first estimated compensation latency based on at least one of a first network latency and by the processing circuitry, from among multiple candidate viewport prediction models each using a different respective time interval, a first viewport prediction model based on a comparison of the first estimated compensation latency to a first time interval used by the first viewport prediction model; generating, using the first viewport prediction model and the first viewport data, a first viewport prediction; and selecting, based on the first viewport prediction, a first content tile for rendering by the display device.