Video Coding for Open GOP Resolution Switching With Limited Drift
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards like VVC face challenges in efficiently managing drift artifacts during open GOP resolution switching in HTTP streaming, particularly when concatenating segments with different spatial resolutions or signal-to-noise ratios, leading to non-constant frame rates and visual artifacts.
Innovation Solution
Implement a video decoder and encoder that signal an indication to exclude certain coding tools, such as cross-component linear-model based prediction, for RASL pictures, ensuring smooth resolution switching by preventing excessive drift during open GOP transitions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If open GOP coding structures with inter-prediction from earlier segments are used, then coding efficiency is improved, but drift artifacts and visual quality degradation occur during resolution switching
Solution Approach 1:
The patent segments the video stream into separate representations with different resolutions and qualities, allowing independent encoding of each segment while maintaining temporal dependencies through reference picture resampling. This enables open GOP coding efficiency while controlling drift through managed segmentation boundaries.
Solution Approach 2:
The patent changes the resolution parameter dynamically during playback by switching between different representations (e.g., 720p and 1080p). The reference picture resampling mechanism adjusts reference pictures to match the current representation resolution, enabling smooth parameter changes without excessive drift artifacts.
2Adaptability or versatility
If reference pictures from different resolutions are used during resolution switching, then adaptability to different representations is improved, but frame rate non-constancy and playback issues occur
Solution Approach 1:
The patent performs preliminary action by pre-encoding multiple representations (different resolutions and qualities) before playback. The reference picture resampling is prepared in advance, allowing smooth transitions during playback without causing frame rate non-constancy or playback issues.
Solution Approach 2:
The reference picture resampling mechanism acts as an intermediary between reference pictures of different resolutions and the current picture. This intermediary process scales reference pictures to match the current representation resolution, enabling adaptability while maintaining frame rate consistency through proper temporal spacing.
3Productivity
If coding tools like cross-component linear-model based prediction are applied to RASL pictures, then coding efficiency is improved, but drift artifacts increase during open GOP transitions
Solution Approach 1:
The patent extracts or removes certain coding tools (such as cross-component linear-model based prediction) from specific contexts (RASL pictures during open GOP transitions). This selective exclusion prevents drift artifacts in critical transition regions while maintaining coding efficiency in other areas where these tools are beneficial.
Solution Approach 2:
The patent applies different coding quality and tool availability locally to different picture types and contexts. Advanced coding tools are applied where they benefit coding efficiency (e.g., regular inter-predicted pictures) but excluded where they cause problems (RASL pictures during resolution switches), achieving local optimization of quality versus artifact trade-off.
Data Source
AI summary
Video Coding Concept Allowing for Limitation of Drift A video decoder for decoding a video from a data stream is configured to decode an indication from the data stream which is valid for a sequence of pictures of the video, and indicates that RASL pictures within the sequence of pictures are coded in a manner excluding a predetermined set of one or more coding tools.