Viewport Prediction for Omnidirectional Video Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Streaming omnidirectional video, such as VR 360 video, requires significant bandwidth to maintain quality and responsiveness to user head movements, leading to challenges in balancing bandwidth usage and user experience, especially when the user moves their head.
Innovation Solution
The method involves predicting a future head gaze of the user based on current head gaze, shoulder position, and eye movements, allowing for the efficient request and selection of tiles related to the viewport from a server, thereby optimizing bandwidth usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full 360 video is streamed in high resolution, then user experience is maintained, but bandwidth consumption increases significantly
Solution Approach 1:
The video stream is divided into multiple tiles, where only the tiles within the user's viewport are streamed in high resolution. The video content is spatially segmented into foreground (viewport) and background regions, allowing selective quality delivery based on user attention and head movement predictions.
Solution Approach 2:
The system performs preliminary actions by predicting future head gaze positions before the user actually moves their head. This allows the system to pre-fetch and prepare high-resolution tiles for the anticipated viewport, ensuring seamless transitions without black holes or quality degradation when the user moves.
2Loss of energy
If only viewport tiles are streamed, then bandwidth is reduced, but user experience deteriorates when user moves head
Solution Approach 1:
The system dynamically adjusts the viewport and tile selection based on real-time head movement predictions. As the user's head position changes, the system continuously updates which tiles require high resolution, ensuring that the streamed content always matches the current and predicted future viewport, thus maintaining quality while optimizing bandwidth.
Solution Approach 2:
The system uses feedback from head tracking sensors and eye movement data to continuously refine tile selection. By monitoring actual head movements and comparing them with predictions, the system adapts its streaming strategy to maintain optimal quality where needed while minimizing unnecessary high-resolution tile transmission.
3Reliability
If more tiles are streamed than fit in viewport, then user experience is preserved during head movement, but bandwidth requirement increases
Solution Approach 1:
Different quality levels are applied to different spatial regions. High-resolution tiles are streamed only for the predicted viewport area, while peripheral and background regions use lower resolution. This local quality differentiation maintains user experience in the critical viewport while significantly reducing overall bandwidth requirements.
Solution Approach 2:
The system changes the resolution parameter dynamically based on spatial location and predicted importance. Tiles within the predicted viewport maintain high resolution parameters, while tiles outside use reduced resolution parameters. This parameter adaptation allows the system to balance quality and bandwidth efficiently.
Data Source
AI summary
A method performed by an electronic device for requesting tiles relating to a viewport of an ongoing omnidirectional video stream is provided. The ongoing omnidirectional video stream is provided by a server to be displayed to a user of the electronic device. The electronic device predicts for an impending time period, a future head gaze of the user in relation to a current head gaze of the user, based on: A current head gaze relative to a position of shoulders of the user, a limitation of the head gaze of the user bounded by the shoulders position of the user, and a current eye gaze and eye movements of the user. The electronic device then sends a request to the server. The request requests tiles relating to the viewport for the impending time period, selected based on the predicted future head gaze of the user.