Gaze Vector Video Tile Encoding for Bandwidth Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 360-degree video delivery systems face challenges in efficiently managing bandwidth, leading to unsatisfactory viewing experiences due to the high data requirements of ultra-high resolution spherical videos, even with advancements in video compression technologies.
Innovation Solution
The implementation of optimized encoding schemes that utilize gaze vector information to determine tile weights and select bandwidth-optimized tiles, allowing for adaptive bitrate allocation and stitching of higher quality data into the video sequence based on user viewport changes, leveraging techniques like tile encoding and phased or block-intra encoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If ultra-high resolution spherical video is delivered to provide immersive 360-degree viewing experience, then viewing quality is improved, but bandwidth consumption increases enormously
Solution Approach 1:
The video content is divided into multiple tiles that can be independently encoded and transmitted. This segmentation allows the system to selectively deliver only the tiles within the user's field of view at high quality, while reducing or omitting tiles outside the viewport, thereby significantly reducing bandwidth consumption while maintaining viewing quality in the visible region.
Solution Approach 2:
Different quality levels are applied to different regions of the video based on their importance. Tiles within the user's field of view are encoded at high quality to ensure immersive viewing experience, while tiles outside the viewport are encoded at lower quality or omitted entirely, optimizing the trade-off between viewing quality and bandwidth consumption.
2Loss of energy
If video compression is applied to reduce bandwidth requirements, then bandwidth consumption is reduced, but viewing quality deteriorates
Solution Approach 1:
The system dynamically adjusts the quality and resolution of video tiles based on real-time user gaze information and viewport position. Tiles within the current field of view are transmitted at high quality with minimal compression, while tiles outside the viewport receive lower quality encoding. This dynamic adaptation ensures optimal viewing quality in the visible region while minimizing overall bandwidth consumption.
Solution Approach 2:
Different compression parameters and quality settings are applied to different video tiles based on their spatial location relative to the user's viewport. By changing encoding parameters dynamically according to viewport position and gaze direction, the system achieves efficient bandwidth utilization without compromising viewing quality in the critical visible regions.
3Loss of energy
If gaze vector information is used to select and stitch tiles adaptively, then bandwidth efficiency is improved, but system complexity increases
Solution Approach 1:
The video content is pre-segmented into tiles and pre-encoded at multiple quality levels before transmission. This preliminary preparation allows the receiving system to quickly select and stitch appropriate tiles based on current viewport information without requiring complex real-time encoding operations, thereby reducing system complexity while maintaining bandwidth efficiency.
Data Source
AI summary
A system and method for providing immersive video based on gaze vector information. A gaze vector is obtained from a client device operating to display an immersive video asset to a user, wherein each video frame comprises an array of tiles projected on a 3-dimensional (3D) display environment viewed by the user in which the user is immersed, the gaze vector defining a gaze direction in the 3D display environment where the user is viewing (i.e., a viewport) at any particular time. Angular separations between the viewport and remaining portions of the video frame are used in selecting different video qualities of the immersive video asset such that the viewport comprises a higher video quality than the remaining portions when displayed by the client device.