360-Degree Video Streaming Tile Prediction and Quality Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 360-degree video streaming techniques either consume excessive network resources by transmitting the entire video sphere or degrade user experience due to imperfect prediction of head movements, leading to suboptimal data transmission and quality issues.
Innovation Solution
A method that iteratively estimates and refines the display window prediction for a head-mounted display, allowing for accurate tile requests with varying quality levels and priority, using multiple iterations before the display time to ensure timely and efficient tile reception, leveraging the HTTP/2 protocol for efficient stream management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all content of the video sphere is transmitted to the user's terminal, then the complete video sphere is available for display, but the network bandwidth consumption increases significantly
Solution Approach 1:
The patent extracts only the necessary portion of the video sphere (tiles within the display window) and transmits them to the user's terminal, rather than transmitting the entire video sphere. This is achieved by identifying the display window based on predicted head orientation and selectively requesting only the tiles that cover this window, thereby significantly reducing network bandwidth consumption while maintaining video completeness within the visible area.
Solution Approach 2:
The patent applies different quality levels to different tiles based on their position relative to the predicted display window. Tiles within the display window are requested in high quality, while tiles outside the display window are requested in low quality or not requested at all. This local differentiation of quality optimizes network bandwidth usage by allocating higher bandwidth to visually important areas and lower or zero bandwidth to less important areas.
2Loss of energy
If only predicted display window tiles are transmitted in high quality, then network bandwidth is reduced, but user experience degrades when prediction is inaccurate
Solution Approach 1:
The patent performs preliminary actions by predicting the user's head orientation and pre-requesting the tiles that will be needed in the display window before the actual display time. This allows the system to prepare the necessary video content in advance, ensuring that high-quality tiles are available when needed while minimizing last-minute requests that would require lower quality or cause delays.
Solution Approach 2:
The patent dynamically adjusts the quality level and request status of tiles based on their position relative to the predicted display window. Tiles within the display window are dynamically assigned high quality, while tiles outside are dynamically assigned low quality. This dynamic adjustment allows the system to adapt to changing user head orientations and maintain optimal video quality within the actual display window while managing network bandwidth effectively.
3Reliability
If low quality tiles are requested for areas outside the predicted display window, then video continuity is maintained during unexpected head movements, but network bandwidth is wasted on unnecessary data transmission
Solution Approach 1:
The patent applies partial action by requesting only the necessary tiles for the predicted display window in high quality, and optionally requesting only a minimal set of neighboring tiles in low quality as a buffer for unexpected head movements. This avoids the excessive action of requesting all tiles in the entire video sphere, while still providing sufficient coverage for typical head movement patterns. The system requests just enough additional tiles to maintain video continuity without wasting excessive bandwidth.
Data Source
AI summary
A method for obtaining video chunks of a video sphere for display on a head-mounted display connected to a video server is described, the video chunks being spatially split into a plurality of tiles that can be encoded on at least two different quality levels, including a high quality level and a low quality level, and a display window includes a video sphere portion that is to be displayed at a particular display time. Prior to the display time, the display window is predicted, tiles covering the estimated display window are estimated, and assigned a high quality level, and tiles adjoining those tiles are estimated and assigned a low quality level. A request is sent to a video server to obtain an encoded tile at the associated quality level. The tiles are received from the video server, the display window is determined, and the tiles corresponding to the determined display window are decoded and displayed.