Adaptive Resolution Coding for Drift-Free Video Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing adaptive resolution coding (ARC) schemes incur drift and quality deterioration when a client switches resolution without encoder awareness, particularly in encoder-agnostic scenarios like video on demand, necessitating delays or increased bandwidth for IDR frames.
Innovation Solution
Utilizing spatial subsampling schemes to create low-resolution versions of video based on high-resolution frames, allowing resolution changes at any point during streaming by constructing reference pictures from decoded low-resolution base and auxiliary pictures, eliminating the need for IDR frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a client switches resolution without encoder awareness in encoder-agnostic scenarios, then resolution flexibility is improved, but drift and quality deterioration occur
Solution Approach 1:
The encoder pre-generates multiple resolution versions of reference pictures and includes them in the bitstream in advance. When the client needs to switch resolution, the appropriate reference picture is already available, eliminating drift caused by real-time upsampling/downsampling. This preliminary preparation allows immediate resolution switching without quality loss.
Solution Approach 2:
The patent introduces an intermediary mechanism where the encoder provides pre-processed reference pictures at multiple resolutions. This intermediary layer between the original video and the client's resolution needs allows seamless switching without direct real-time transformation, preventing drift accumulation.
2Reliability
If IDR frames are used to prevent drift during resolution switching, then video quality is maintained, but bandwidth requirements increase and switching delays occur
Solution Approach 1:
Instead of using full IDR frames that reset all prediction structures, the patent applies partial action by providing only the necessary reference pictures at the target resolution. This selective approach maintains quality where needed while avoiding the excessive bandwidth consumption of complete IDR frame insertion.
Solution Approach 2:
The patent changes the resolution parameter of reference pictures dynamically based on client needs. By modifying the resolution parameter of pre-encoded reference pictures rather than inserting new IDR frames, the system achieves resolution switching with minimal bandwidth overhead and without the delays associated with waiting for next IDR frames.
3Productivity
If inter-layer prediction is used for resolution switching in SHVC, then encoding efficiency is improved, but drift occurs when encoder is unaware of switching points
Solution Approach 1:
The encoder performs preliminary encoding of reference pictures at multiple resolutions and includes them in the bitstream before the client performs any resolution switching. This preliminary action ensures that when inter-layer prediction is used, the reference pictures are already at the correct resolution, preventing drift while maintaining encoding efficiency.
Data Source
AI summary
Decoding coded pictures of a video includes receiving, by a decoder apparatus, a bitstream comprising an encoded representation of first pictures of a first resolution version of the video and decoding the coded first pictures into first pictures, the first pictures being associated with corresponding second pictures of a second resolution version of the video, the first pictures being formed by sampling the corresponding second pictures using a sampling lattice. Signalling information received in the bitstream informs the decoder apparatus about a resolution change. The method includes identifying at least one of the first pictures as a first base picture and identifying one or more auxiliary pictures, the first base picture being associated with a corresponding second picture, and, each of the one or more auxiliary pictures being formed based on sampling the second picture using a different sampling lattice of the plurality of sampling lattices. The method further includes receiving an encoded representation of the one or more auxiliary pictures in the bitstream, decoding the encoded representation of the one or more auxiliary pictures into one or more auxiliary pictures and constructing a reference picture of the second resolution based on the first base picture and the one or more auxiliary pictures. The method further includes receiving an encoded representation of second pictures in the bitstream and decoding at least part of the encoded representation of the second pictures into one or more second pictures using the constructed reference picture.