Personalized 360 Video Delivery via Spatial Tile Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Delivering high-quality video streams with a format larger than a user's viewport is challenging due to high bandwidth requirements and latency issues, particularly in 360-degree video delivery, where sending full video content wastes processing power and bandwidth, and updating the viewport with low latency is crucial to avoid motion sickness.
Innovation Solution
A method that personalizes video streams by determining user preferences to prioritize and encode spatially more relevant parts at higher resolution, using an edge server to add priority information to metadata, and pre-fetching high-quality relevant parts to minimize delay and optimize delivery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the complete video content is sent to the client, then the client can decode and render the current viewport, but processing power and transport bandwidth are wasted
Solution Approach 1:
The video content is segmented into multiple tiles, where only the tiles corresponding to the user's viewport are transmitted at high quality, while other tiles are transmitted at lower quality or omitted entirely. This segmentation allows the system to deliver only the necessary portions of the video content, reducing bandwidth consumption while maintaining visual quality in the viewport region.
Solution Approach 2:
Different quality levels are applied to different spatial regions of the video content. The viewport region receives high-quality encoding and transmission, while peripheral regions receive lower-quality encoding. This local quality differentiation ensures that computational and bandwidth resources are concentrated on the visually important areas while reducing waste in less critical regions.
2Loss of energy
If only the viewport part of the video is sent to the user, then bandwidth is saved, but latency increases when the user changes the viewport
Solution Approach 1:
The system pre-fetches and pre-encodes video tiles that are likely to become the next viewport based on user viewing patterns and head movement prediction. By preparing these tiles in advance and storing them in buffer memory, the system can immediately switch to displaying pre-rendered high-quality tiles when the user changes their viewport, thereby minimizing latency without requiring real-time encoding and transmission.
3Reliability
If the full video is delivered at low quality and the viewport part at high quality, then video quality in viewport is improved, but bandwidth requirements increase
Solution Approach 1:
The video stream is divided into multiple tile segments, with different quality levels assigned to different segments. The viewport-containing tiles are encoded and transmitted at high quality, while non-viewport tiles are encoded at lower quality or compressed more aggressively. This selective quality segmentation reduces overall bandwidth consumption while maintaining high visual quality in the critical viewport region.
Data Source
AI summary
A method for personalizing a video stream for a user, wherein the video stream comprises a sequence of video frames having a format such that a user with a viewport is not able to view the whole content present in the video frames but only a part of the content present in the video frames, the method comprising: —determining user preference information which indicates which spatial part of content present in the video frames is of higher relevance than other parts of the content present in the video frames, —selecting a spatially more relevant part in the video frames having a higher viewing relevance compared to a remaining part in the video frames based on the determined user preferences, —indicating the spatially more relevant part in the video frames to a video providing entity from where the video stream is delivered to the user.