Low-Latency 360 VR Streaming with Predictive ROI Tile Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 360 VR streaming systems face high switching and streaming latencies due to the need for continuous high-quality tile stream requests and bandwidth requirements, leading to reduced immersion and increased motion sickness in users.
Innovation Solution
A low-latency 360 VR streaming system that utilizes tiled CMAF chunks and a simulcast rendering algorithm to minimize latency by predicting ROI changes, caching adjacent tiles, and overlaying high-quality streams on low-quality streams, allowing seamless video playback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If continuous high-quality tile stream requests are made to maintain video quality, then video quality is improved, but switching latency and streaming latency increase
Solution Approach 1:
The system performs preliminary actions by predicting the user's future ROI based on current pose information and pre-requesting those tiles in advance. This allows the tiles to be cached before they are actually needed, eliminating switching latency when the user looks in a new direction. The prediction mechanism proactively prepares content before the user actually needs it, resolving the contradiction between maintaining quality and reducing latency.
Solution Approach 2:
The video stream is segmented into multiple tiles, and the system selectively requests only the tiles within or near the predicted ROI rather than continuous high-quality streams of the entire scene. This segmentation allows bandwidth to be focused on relevant areas, reducing overall streaming latency while maintaining quality where the user actually looks. The tile-based approach enables independent processing and caching of small units rather than large continuous streams.
2Manufacturing precision
If high-quality tile streams are requested for the entire field of view, then video quality is maintained, but network bandwidth requirements increase
Solution Approach 1:
The system applies local quality by providing high-quality video streams only within the predicted ROI while using lower quality or skipping streams outside this region. The tile quality parameter is adjusted locally based on whether each tile falls within the predicted area of interest. This resolves the contradiction by concentrating bandwidth on the small portion of the scene the user will actually look at, rather than uniformly distributing high quality across the entire 360-degree field of view.
Solution Approach 2:
The system performs partial action by requesting high-quality streams for only the predicted ROI tiles rather than for the entire scene. The tile request logic selectively activates high-quality streaming for a subset of tiles based on predicted user attention, accepting that some areas outside the ROI will have lower quality or no stream at all. This partial approach significantly reduces total bandwidth consumption while maintaining quality where it matters most to the user experience.
3Loss of time
If ROI prediction and tile caching mechanisms are implemented, then switching latency is reduced, but system complexity increases
Solution Approach 1:
The system implements self-service by using the user's own pose information to automatically predict their next viewing direction and pre-request the corresponding tiles without external intervention. The prediction algorithm leverages the already-collected pose data to autonomously determine which tiles to cache next, eliminating the need for complex external control systems or manual ROI specification. This self-directed approach reduces switching latency while keeping the system architecture relatively simple.
Data Source
AI summary
A low-latency 360 virtual reality (VR) streaming method for estimating a region of interest (ROI) of a user is proposed. The method may include receiving device pose information from a user device, extracting an ROI of a user, based on the device pose information, and generating ROI tile information corresponding to the ROT. The method may also include generating ROI estimation tile information corresponding to a position at which the ROI is to be changed, and requesting, from a streaming server, a tile corresponding to the ROI tile information and the ROI estimation tile information to receive a high-quality tile stream and a low-quality tile stream of full video that includes a low-quality full video tile stream. The method may further include decoding and rendering the high-quality tile stream and the low-quality full video tile stream and providing the rendered video to the user device.