Adaptive Resolution Scaling in Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards, such as HEVC, lack the ability to adaptively change resolution without introducing an IDR or IRAP picture, which can lead to increased complexity, latency, and poor user experience in scenarios like video conferencing and streaming, where dynamic resolution adjustments are necessary.
Innovation Solution
The proposed solution involves methods for video processing that allow for adaptive resolution change (ARC) and reference picture resampling, enabling seamless conversion between different spatial resolutions by signaling offset values in N-pel precision and applying consistent processing rules to samples within windows of varying dimensions, and using interpolation filters based on resolution differences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If adaptive resolution change is implemented without IDR or IRAP pictures, then coding efficiency and user experience are improved, but complexity management and reference picture consistency become problematic
Solution Approach 1:
The patent implements dynamic resolution adjustment by allowing the window size (W×H and W'×H') to vary between different video pictures while maintaining inter-picture prediction. The conformance window parameters can be changed without requiring IDR pictures, enabling adaptive resolution scaling to match display requirements while preserving coding efficiency through incremental updates to reference picture structures.
Solution Approach 2:
The patent changes the parameters of reference pictures by applying different conformance window scaling factors (W/W' and H/H') to transform reference pictures between different resolutions. This allows the system to adapt resolution by modifying picture parameters rather than requiring complete picture resets, thereby improving coding efficiency while managing complexity through parameter transformation.
2Loss of time
If adaptive resolution change is implemented without IDR or IRAP pictures, then latency is reduced, but reference picture resampling complexity increases
Solution Approach 1:
The patent performs preliminary resampling of reference pictures by creating scaled versions with different conformance window parameters before they are needed for prediction. By pre-computing and storing reference pictures at multiple resolutions, the system avoids real-time resampling operations during decoding, thereby reducing latency while managing resampling complexity through advance preparation.
Solution Approach 2:
The patent creates copied and scaled versions of reference pictures with different conformance window dimensions (W×H vs W'×H'). Instead of performing complex real-time resampling operations, the system generates multiple copies of reference pictures at different resolutions and selects the appropriate copy based on current picture dimensions, reducing both latency and computational complexity.
3Adaptability or versatility
If different resolution pictures are used for current and reference pictures, then adaptability is improved, but processing consistency becomes challenging
Solution Approach 1:
The patent applies local quality processing by treating samples within the conformance window differently from samples outside it. The processing rule is applied specifically to samples within the W×H and W'×H' windows, allowing different processing for different regions. This enables the system to handle pictures of different resolutions while maintaining processing consistency within each picture's defined conformance window.
Solution Approach 2:
The patent implements a universal processing rule that can be applied to pictures of any resolution. The same processing operation is performed on samples within the conformance window regardless of the picture's overall dimensions, making the processing consistent and adaptable across different resolutions. The processing rule is designed to work universally with any W×H and W'×H' window configuration.
Data Source
AI summary
An example method of video processing includes determining, for a conversion between a current picture of a video and a bitstream of the video, a scaling ratio based on a first window in the current picture and a second window in a reference picture associated with the current picture; resampling the reference picture by using the determined scaling ratio; and performing the conversion based on the result of the resampling, wherein a dimension of the first window is W×H and a dimension of the second window is W′×H′, where W, H, W′ and H′ are integers, and wherein W, H, W′ and H′ satisfy a constraint.