360-Degree Video Tile Shuffling for VR Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video encoding and decoding techniques for 360-degree video in virtual reality (VR) struggle with efficiently managing and rendering high-quality video segments to users, leading to resource inefficiencies and potential tiling effects due to quick head movements.
Innovation Solution
The proposed solution involves a decoder device and method that maintain a fixed number of high-quality tiles in the compressed domain, replacing tiles that fall out of the viewport with high-quality tiles that enter the viewport, and utilizing a frame-packing approach with a low-resolution background to minimize Intra coded samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional video encoding techniques are used for 360-degree video, then video quality can be maintained, but resource efficiency deteriorates and tiling effects occur due to quick head movements
Solution Approach 1:
The video content is divided into multiple tiles or segments that can be independently encoded and decoded. This segmentation allows the system to prioritize encoding resources for tiles currently in the viewport while using lower quality for tiles outside the viewport, thus improving resource efficiency without significantly impacting perceived video quality.
Solution Approach 2:
Different quality levels are applied to different regions of the 360-degree video based on their importance. Tiles within the current viewport are encoded at high quality, while tiles outside the viewport are encoded at lower quality. This local quality differentiation optimizes resource usage by focusing computational resources on visually critical areas.
2Reliability
If high-quality video segments are maintained for all tiles, then video quality is preserved, but resource usage increases significantly
Solution Approach 1:
The system dynamically adjusts the quality of video tiles based on real-time viewport information. As the user moves their head and the viewport changes, the encoding quality of different tiles is adjusted accordingly. Tiles entering the viewport receive higher quality encoding, while tiles leaving the viewport are encoded at lower quality, creating a dynamic resource allocation system that adapts to user behavior.
Solution Approach 2:
The encoding parameters of video tiles are changed based on their spatial location relative to the viewport. Tiles within the viewport are encoded with higher bitrate and quality parameters, while tiles outside the viewport use lower parameters. This parameter differentiation allows the system to maintain acceptable overall quality while significantly reducing total resource consumption.
3Adaptability or versatility
If Intra coded samples are used frequently to handle head movements, then video adaptability to head movements improves, but the amount of Intra coded samples increases leading to inefficiency
Solution Approach 1:
The video is segmented into tiles with independent encoding structures. This segmentation allows the system to use Inter prediction (temporal prediction) within tiles while maintaining Intra prediction boundaries at tile edges. By carefully managing tile boundaries and prediction modes, the system reduces the total number of Intra coded samples needed while maintaining adaptability to head movements through the tile-based structure.
Data Source
AI summary
A decoder receives a set of N compressed segments of a first picture, and a set of M compressed segments for a second picture that follows the first picture in a decoding order. Based on the set of N compressed segments, the decoder constructs a decodable picture for the first picture and decodes the decodable picture. Based on the set of M compressed segments, the decoder constructs a decodable second picture in the compressed domain, in which each of the segments of the first picture are placed at the same spatial position as that of a corresponding segment in the second picture and decodes the decodable second picture. Additionally, at least one Intra coded segment is placed at one of the M spatial positions in the compressed domain in which there is no segment of the N segments.