Adaptive Resolution Conversion Signaling for Video Resampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards, such as HEVC, lack the ability to adaptively change resolution without introducing an IDR or IRAP picture, which can lead to inefficiencies in bandwidth usage, latency, and user experience issues, especially in applications like video conferencing and streaming where network conditions and viewer orientation changes occur.
Innovation Solution
The implementation of an adaptive resolution conversion (ARC) process that allows for dynamic resolution changes by resampling reference pictures and signaling this information in the bitstream, enabling efficient prediction structures and seamless representation switching between different spatial resolutions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If adaptive resolution conversion is implemented without IDR/IRAP pictures, then bandwidth efficiency is improved and latency is reduced, but video coding standards compatibility is worsened
Solution Approach 1:
The video stream is segmented into different resolution representations (e.g., first representation at higher resolution, second representation at lower resolution). The encoder can selectively switch between these segments based on network conditions, allowing adaptive resolution changes without requiring full IDR/IRAP picture resets, thus improving bandwidth efficiency while maintaining resolution adaptability.
Solution Approach 2:
The system dynamically adjusts the resolution of video representations during encoding and transmission. By implementing dynamic resolution conversion mechanisms that operate within the video coding framework, the system can adapt to changing network conditions and viewer requirements without breaking buffer conditions or requiring standard-compliant resolution changes.
2Loss of time
If resolution changes are made without IDR/IRAP pictures, then latency is reduced, but prediction structure efficiency is worsened
Solution Approach 1:
Multiple video representations at different resolutions are pre-encoded and prepared in advance. When network conditions change or viewer orientation changes occur, the system can immediately switch to the appropriate pre-prepared representation without waiting for IDR/IRAP pictures, thus reducing latency while maintaining efficient prediction structures through the pre-encoded nature of the representations.
3Adaptability or versatility
If multiple video representations are transmitted, then adaptability to network conditions is improved, but bandwidth consumption is worsened
Solution Approach 1:
Different regions or segments of the video content are encoded at different qualities and resolutions based on local requirements. By applying local quality adjustment rather than uniform resolution changes across the entire video stream, the system can adapt to network conditions and viewer needs more efficiently, reducing overall bandwidth demand while maintaining necessary adaptability.
Data Source
AI summary
Devices, systems and methods for digital video coding, which includes reference picture resampling, are described. An example method for video processing includes performing a conversion between a video comprising one or more video segments comprising one or more video units and a bitstream representation of the video, wherein the bitstream representation conforms to a format rule and comprises information related to an adaptive resolution conversion (ARC) process, wherein the format rule specifies the applicability of the ARC process to a video segment, wherein an indication that the one or more video units of the video segment are coded with different resolutions is included in the bitstream representation in a syntax structure that is different from a header syntax structure, a decoder parameter set, a video parameter set, a picture parameter set, a sequence parameter set, and an adaptation parameter set.