Switchable Inter-Coded Pictures for 360-Degree Video Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In viewport-adaptive streaming for 360-degree video, the current technologies face challenges in maintaining high-quality playback when the user pauses and changes the viewing orientation, leading to degraded user experience due to lack of efficient signaling mechanisms for retrieving and decoding the 360-degree sphere frame, and exceeding decoding capabilities.
Innovation Solution
The method involves encoding video media content into two representations: a first representation with switchable inter-coded pictures and a second representation with decoder reset pictures, where the frame rate and resolution of both are equal or lesser, allowing for efficient switching and decoding, especially during playback pauses and viewport changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a complete 360-degree picture in high resolution is transmitted during pause state, then the user experience is improved, but the decoding capability is exceeded
Solution Approach 1:
The 360-degree video is segmented into multiple representations with different resolutions and frame rates. During pause state, the decoder can select and decode only the high-resolution representation corresponding to the current viewport, rather than decoding the entire 360-degree sphere at full resolution. This segmentation allows the system to manage decoding complexity by processing only necessary portions of the video data.
Solution Approach 2:
Different regions of the 360-degree video are assigned different quality levels based on their importance. The primary viewport region is transmitted at high resolution, while peripheral regions are transmitted at lower resolution. During pause state, the decoder focuses computational resources on reconstructing the high-quality viewport region, thereby improving local picture quality without exceeding overall decoding capability.
2Loss of energy
If switchable inter-coded pictures are used in the first representation, then the bitrate is reduced, but the decoding complexity increases
Solution Approach 1:
The encoder pre-processes the video data by creating switchable inter-coded pictures with embedded reference information and decoding dependencies. This preliminary action allows the decoder to efficiently navigate between different representations by following pre-established reference chains, reducing the computational burden during actual decoding operations despite the increased structural complexity.
Solution Approach 2:
The switchable inter-coded pictures act as intermediaries between different video representations. These pictures contain reference information that mediates the decoding process, allowing the decoder to reconstruct high-quality images by combining data from multiple representations without directly processing all raw data, thereby managing decoding complexity while maintaining bitrate efficiency.
3Manufacturing precision
If viewport-adaptive streaming is implemented, then the primary viewport quality is improved, but the user experience degrades when user changes viewing orientation during pause
Solution Approach 1:
The video streaming system dynamically adapts between different representations based on the current playback state. During active playback, the system transmits only the primary viewport at high quality. During pause state, the system dynamically switches to transmitting multiple representations including decoder reset pictures that enable the user to view any orientation at high quality, thereby providing dynamic adaptability to different user behaviors while maintaining primary viewport quality during playback.
Solution Approach 2:
The encoding structure is designed with multi-functionality to serve different playback scenarios. The same bitstream contains both switchable inter-coded pictures for efficient playback and decoder reset pictures for pause-state exploration. This universal structure allows the system to function effectively in both active playback and pause states, improving adaptability without sacrificing primary viewport quality.
Data Source
AI summary
A method comprising: encoding video media content into at least a first representation and a second representation, wherein the first representation comprises a switchable inter-coded picture, wherein the switchable picture has a first characteristic that no pictures preceding the switchable picture in decoding order are used as reference pictures for any non-switchable pictures following the switchable picture in decoding order; and the second representation comprises one or more decoder reset pictures aligned with the switchable pictures of the first representation, wherein the frame rate of the second representation is smaller than or equal to the frame rate of the first representation and the resolutions of the first and the second representation are equal.