Trajectory-Based Viewport Prediction for 360-Degree Video Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for viewport-based adaptive streaming of 360-degree videos are limited to short-term time horizons, leading to quality degradation and susceptibility to bandwidth fluctuations, as they fail to accurately predict user viewports for long-term time horizons due to reliance on physical movements and second-order statistics without considering past viewing patterns.
Innovation Solution
A trajectory-based approach that clusters angle trajectories from previous users, determines trend trajectories and score thresholds, and predicts future viewports by matching new user angles to these clusters, enabling accurate long-term viewport prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional short-term viewport prediction systems are used, then the system complexity remains low, but the viewport prediction accuracy degrades for long-term time horizons
Solution Approach 1:
The system performs preliminary actions by collecting and storing angle trajectories from multiple users during their viewing sessions. These trajectories are pre-processed and stored in a database before being used for prediction, allowing the system to leverage historical viewing patterns without adding complexity during real-time prediction operations
Solution Approach 2:
The system creates copies of angle trajectories from multiple users and uses these copied patterns to predict future viewports. Instead of modeling complex physical movements, the system replicates and analyzes actual viewing behavior patterns from trajectory data, simplifying the prediction mechanism while improving accuracy
2Reliability
If video buffers with long-term time horizons (10-15 seconds) are used, then the system can handle bandwidth fluctuations better, but quality degradation occurs due to poor short-term prediction algorithms
Solution Approach 1:
The system implements feedback by continuously monitoring actual user angle trajectories during viewing and comparing them against predicted trajectories. This feedback loop allows the system to refine its prediction models over time, ensuring that long-term video buffer usage maintains high quality by accurately anticipating viewport changes throughout the entire buffer period
Solution Approach 2:
The system changes parameters by transitioning from short-term prediction horizons to long-term prediction horizons (10-15 seconds), matching the video buffer delay. This parameter change allows the prediction algorithm to optimize for the actual time scale at which video content is delivered and buffered, improving both reliability and quality consistency
3Loss of energy
If viewport-based adaptive streaming with short-term prediction is implemented, then bandwidth requirements are reduced, but the system becomes susceptible to bandwidth fluctuations causing video freezes
Solution Approach 1:
The system performs preliminary actions by predicting viewports far in advance (10-15 seconds) and pre-loading the corresponding video tiles into the buffer. This advance preparation ensures that video content is ready for playback even when bandwidth fluctuates, preventing video freezes while maintaining efficient bandwidth usage through selective high-quality encoding of predicted regions
Data Source
AI summary
In implementations of trajectory-based viewport prediction for 360-degree videos, a video system obtains trajectories of angles of users who have previously viewed a 360-degree video. The angles are used to determine viewports of the 360-degree video, and may include trajectories for a yaw angle, a pitch angle, and a roll angle of a user recorded as the user views the 360-degree video. The video system clusters the trajectories of angles into trajectory clusters, and for each trajectory cluster determines a trend trajectory. When a new user views the 360-degree video, the video system compares trajectories of angles of the new user to the trend trajectories, and selects trend trajectories for a yaw angle, a pitch angle, and a roll angle for the user. Using the selected trend trajectories, the video system predicts viewports of the 360-degree video for the user for future times.