Affine Motion Compensation for VVC Memory Bandwidth Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The current VVC standard's affine motion model for video coding results in excessive memory bandwidth consumption due to fixed sub-block sizes, which is inadequate for processing high-resolution videos like 8K at 60 fps, exceeding the memory bandwidth budget.
Innovation Solution
Implementing a method to dynamically adjust the reference block bounding box size and sub-block motion vectors based on predefined thresholds, switching to a fallback mode when the bounding box exceeds the threshold, and using adaptive sub-block sizes to constrain memory bandwidth consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed sub-block size of 4×4 is used for affine mode, then the motion compensation can be implemented with current memory bandwidth, but the memory bandwidth consumption becomes excessive for high-resolution videos like 8K at 60 fps
Solution Approach 1:
The patent applies dynamics by making the sub-block size adaptive rather than fixed. The encoder dynamically determines the sub-block size based on the actual motion characteristics and reference block bounding box dimensions. This allows the system to adjust the number of sub-blocks and corresponding memory bandwidth requirements according to the specific video content and resolution, enabling efficient processing of high-resolution videos without excessive memory bandwidth consumption.
Solution Approach 2:
The patent changes the parameter of sub-block size from a fixed value to a variable that depends on the reference block bounding box dimensions. By calculating the sub-block size as a function of the actual motion compensation reference block dimensions, the system optimizes the balance between motion compensation accuracy and memory bandwidth consumption for different video resolutions and motion scenarios.
2Loss of energy
If the reference block bounding box size is reduced to control memory bandwidth, then memory bandwidth consumption decreases, but the motion compensation accuracy may be compromised
Solution Approach 1:
The system dynamically adjusts the reference block bounding box size based on the actual motion vectors and sub-block configuration. Rather than using a fixed reduced size, the bounding box adapts to the specific motion compensation requirements of each block, ensuring sufficient accuracy while controlling memory bandwidth consumption through the relationship between sub-block size and bounding box dimensions.
Solution Approach 2:
The patent changes the bounding box size parameter to be dependent on the sub-block size and motion characteristics. By establishing a proportional relationship where the bounding box size scales with the sub-block configuration, the system maintains motion compensation accuracy while controlling memory bandwidth consumption through parameter adaptation rather than fixed reduction.
3Loss of energy
If a larger sub-block size is used, then memory bandwidth consumption decreases, but the motion compensation precision for complex motions deteriorates
Solution Approach 1:
The patent applies segmentation by dividing the prediction block into multiple sub-blocks, where each sub-block can have its own motion vectors and compensation parameters. This segmentation allows the system to handle complex motions more accurately by applying different motion models to different regions, while controlling overall memory bandwidth consumption through the relationship between the number of sub-blocks and the reference block bounding box size.
Solution Approach 2:
The patent changes the sub-block size parameter to be adaptive rather than fixed or uniformly large. By calculating sub-block size based on the reference block bounding box dimensions and motion characteristics, the system optimizes the balance between motion compensation precision and memory bandwidth consumption, allowing larger sub-blocks where appropriate while maintaining precision for complex motions through the segmentation approach.
Data Source
AI summary
Video encoding or decoding includes performing affine motion compensation in an affine mode in which a prediction unit (“PU”) of the digital video coded in the affine mode uses inter prediction and a reference block bounding box size and determining whether the reference block bounding size exceeds a predefined threshold. In response to a determination that the reference block bounding size exceeds the predefined threshold, the affine motion compensation is performed using a first motion compensation operation. In response to a determination that the reference block bounding size does not exceed the predefined threshold, the affine motion compensation is performed using a second motion compensation operation that is different from the first motion compensation operation.