Inter Prediction Bandwidth Reduction via Optical Flow Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards face challenges in reducing inter prediction bandwidth, particularly for small blocks, due to high memory access bandwidth and computational complexity, especially when using fractional motion vector precision and interpolation filters.
Innovation Solution
The method involves rounding motion vectors to integer precision, generating an unrefined prediction by copying integer samples, calculating spatial gradients, and refining the prediction using the rounding error vector and optical flow equation, thereby reducing memory access bandwidth and computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If fractional motion vector precision and interpolation filters are used for inter prediction, then prediction accuracy is improved, but memory access bandwidth and computational complexity increase significantly
Solution Approach 1:
The patent segments the prediction process into two distinct stages: an unrefined prediction stage using rounded motion vectors and a refinement stage using optical flow compensation. This segmentation allows the system to use simpler operations for the bulk computation while applying more complex refinement only where needed, thereby reducing overall computational complexity while maintaining prediction accuracy.
Solution Approach 2:
The patent introduces an intermediary rounding error vector that captures the difference between the rounded and original motion vectors. This intermediary representation enables the system to compensate for quantization errors without directly using the full precision motion vectors, thus reducing memory bandwidth requirements while maintaining prediction accuracy through the optical flow refinement equation.
2Measurement precision
If fractional motion vector precision and interpolation filters are used for inter prediction, then prediction accuracy is improved, but memory access bandwidth increases
Solution Approach 1:
The patent segments memory access operations into two categories: accessing reference picture data for unrefined prediction using rounded motion vectors (lower bandwidth), and accessing gradient data for refinement (selective access). This segmentation reduces peak memory bandwidth requirements compared to always using full precision motion vectors with interpolation filters.
Solution Approach 2:
The patent uses a disposable rounding error vector that is computed from the difference between rounded and original motion vectors. This error vector serves as a compact intermediary representation that enables accuracy recovery without requiring access to the full precision motion vectors, effectively replacing expensive high-precision memory accesses with cheaper computations.
3Device complexity
If rounding motion vectors to integer precision is performed, then memory access bandwidth and computational complexity are reduced, but prediction accuracy deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where the rounding error vector (difference between original and rounded motion vectors) is computed and then used to refine the unrefined prediction. This feedback loop compensates for the accuracy loss from rounding, allowing the system to use integer precision motion vectors while recovering prediction accuracy through the optical flow refinement equation.
Solution Approach 2:
The patent introduces the rounding error vector as an intermediary that bridges the gap between rounded motion vectors and the original high-precision motion information. This intermediary enables the system to work with simplified integer precision data structures while still achieving accurate predictions through the refinement stage, thus resolving the accuracy loss from rounding.
4Quantity of substance
If rounding motion vectors to integer precision is performed, then memory access bandwidth is reduced, but prediction accuracy deteriorates
Solution Approach 1:
The patent uses feedback by computing the rounding error vector and applying it through the optical flow refinement equation to correct the unrefined prediction. This feedback mechanism allows the system to use reduced precision motion vectors for memory-efficient access while recovering prediction accuracy through the refinement stage, thus resolving the contradiction between bandwidth reduction and accuracy maintenance.
Solution Approach 2:
The rounding error vector serves as an intermediary that enables the system to work with rounded motion vectors while recovering the information lost through rounding. This intermediary representation allows memory-efficient processing with integer precision vectors while maintaining prediction accuracy through the optical flow compensation, effectively bridging the gap between bandwidth reduction and accuracy preservation.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces memory access bandwidth and computational complexity while maintaining coding efficiency, particularly for small blocks, by avoiding interpolation operations and using shorter tap filters.
Implementation Method 1
refining the prediction using the rounding error vector and optical flow equation
Data Source
AI summary
Systems and methods are described for video coding. In some embodiments, inter prediction of a sample in a current block is performed by rounding an initial motion vector and determining a rounding error vector caused by the rounding. An unrefined prediction of the sample is generated using the rounded motion vector. Unrefined predictions are similarly generated for other samples in the current block. Based on the unrefined predictions, a spatial gradient is determined for each sample position in the block. A refined prediction is generated for each sample position by adding, to the unrefined prediction, a scalar product between the spatial gradient and the rounding error vector at the sample position. Example methods can reduce the number of reference pixels used to predict a current block and thus may reduce memory access bandwidth.