Independent Multi-Resolution Video Coding for Panoramic Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video decoding techniques for panoramic displays often result in a drastic quality drop when a user's field of vision changes, leading to delayed and lower quality video playback, especially in high-resolution scenarios, due to limitations in inter-layer prediction and tile-based coding.
Innovation Solution
The implementation of an independent multi-resolution coding scheme that allows for a gradual quality drop-off from regions away from the user's current viewpoint, combined with techniques like offsetting I-frames in stereo video data to reduce latency, enables efficient decoding and playback of high-resolution video on lower-capable devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional video decoding techniques with inter-layer prediction and tile-based coding are used, then device complexity is reduced, but video quality drops drastically when user's field of vision changes
Solution Approach 1:
The video canvas is divided into multiple independent regions, each encoded at different resolutions. This segmentation allows the decoder to selectively decode only the regions within the user's current field of view at high resolution, while other regions are decoded at lower resolutions or skipped entirely, thus maintaining video quality without requiring full high-resolution decoding capability
Solution Approach 2:
Different regions of the video canvas are encoded with different quality levels based on their importance. The central region corresponding to the user's field of view is encoded at high resolution, while peripheral regions are encoded at lower resolutions. This local quality differentiation maintains overall video quality perception while reducing decoding complexity and bandwidth requirements
2Manufacturing precision
If high-resolution video data is decoded for the entire canvas, then video quality is maintained, but bitrate and storage requirements increase significantly
Solution Approach 1:
Instead of decoding the entire video canvas at high resolution, the system decodes only the necessary portions (regions within or near the user's field of view) at high resolution. This partial action approach maintains video quality where needed while significantly reducing the total bitrate and storage requirements by avoiding redundant high-resolution decoding of peripheral regions
3Device complexity
If conventional tile-based coding with inter-layer prediction is used, then device complexity is reduced, but latency increases when field of vision changes
Solution Approach 1:
Multiple regions are pre-encoded at different resolutions before playback. When the user's field of view changes, the decoder can immediately switch to decoding the pre-encoded high-resolution region corresponding to the new field of view without needing to perform complex inter-layer prediction or wait for re-encoding, thus reducing playback latency while maintaining manageable decoding complexity
Data Source
AI summary
In one example, a device for retrieving video data includes a display interface coupled to a display, a memory configured to store video data, and one or more processors configured to determine a plurality of regions of video data to be displayed via the display, retrieve video data having a first quality for a first subset of the plurality of regions at which a visual focus of a user is directed, retrieve video data having a second quality for a second subset of the plurality of regions that neighbor the first subset of the plurality of regions, wherein the second quality is lower than the first quality, and retrieve video data having a third quality for a third subset of the plurality of regions that is outside the first subset and the second subset, wherein the third quality is lower than the second quality.