Resolution-Adaptive Video Coding via Motion Vector Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video coding formats, such as H.264/AVC and H.265/HEVC, face challenges in adaptively changing frame resolution without incurring significant bandwidth costs, as down-sampling requires generating new video sequences and existing solutions like switch_frames in AV1 and new motion prediction tools in VVC/H.266 do not fully realize bandwidth savings due to restrictions in motion vector coding.
Innovation Solution
The implementation of motion prediction coding formats like Temporal Motion Vector Predictor (TMVP), Sub-block Temporary Motion Vector Predictor (SbTMVP), and Bi-directional Optical Flow (BIO) enables adaptive resolution changes by up-sampling and down-sampling reconstructed frames, allowing motion vectors to reference previous frames, thus maintaining bandwidth savings and enabling efficient resolution adaptation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional video coding formats (H.264/AVC, H.265/HEVC) are used to change frame resolution, then a new video sequence must be generated starting with an intra-coded frame, but this incurs significantly larger bandwidth costs to transmit than inter-coded frames
Solution Approach 1:
The patent enables dynamic resolution changing within a video sequence by allowing inter-coded frames to be transmitted at different resolutions than previous frames. The switch_frame type provides this dynamic capability, allowing the system to adapt resolution based on network conditions without requiring a complete sequence restart with intra-coded frames.
Solution Approach 2:
The patent introduces an intermediary mechanism (switch_frame with resolution change capability) that bridges the gap between fixed-resolution conventional coding and adaptive resolution requirements. This intermediary allows motion vector references to work across resolution boundaries through appropriate scaling, enabling bandwidth savings while maintaining resolution adaptability.
2Adaptability or versatility
If switch_frame is used in AV1 codec to support resolution changing, then different resolution transmission is enabled, but motion vector coding cannot reference motion vectors of previous frames which sustains greater bandwidth consumption
Solution Approach 1:
The patent enhances the switch_frame type to serve multiple functions: it can change resolution while also maintaining motion vector referencing capability from previous frames. This multi-functionality resolves the limitation where switch_frames could only change resolution but not reference motion vectors, thereby achieving both resolution adaptability and bandwidth savings.
Solution Approach 2:
The patent applies parameter changes by scaling motion vectors according to resolution ratios when referencing previous frames with different resolutions. This allows the motion vector parameter to be adapted to different resolution contexts, enabling cross-resolution motion compensation and reducing bandwidth consumption while maintaining resolution changing capability.
3Loss of energy
If down-sampling is applied to transmit low resolution video over network with low bandwidth, then bandwidth savings are achieved, but the bandwidth costs of adaptively down-sampling offset the bandwidth gains
Solution Approach 1:
The patent performs down-sampling and up-sampling operations in advance during encoding and decoding processes, respectively. By preparing reference frames at appropriate resolutions beforehand and scaling them as needed, the system avoids repeated complex down-sampling operations during transmission, thereby achieving bandwidth savings while managing complexity efficiently.
Data Source
AI summary
Systems and methods are provided for implementing methods for resolution-adaptive video coding in a motion prediction coding format by obtaining a current frame of a bitstream, obtaining one or more reference pictures from a reference frame buffer, up-sampling or down-sampling the obtained one or more reference pictures that have resolutions different from a resolution of the current frame, and generating a reconstructed frame from the current frame based on the one or more reference pictures and a selected motion candidate of the current frame, thereby achieving substantial reduction of network transport costs in video coding and delivery without requiring the transport of additional data that would offset or compromise these savings.