Bi-directional Optical Flow Motion Vector Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding techniques, such as those in HEVC, face challenges in accurately predicting video blocks for bi-predicted inter-coded blocks, leading to inefficiencies in compression and quality due to limitations in motion compensation methods.
Innovation Solution
The implementation of bi-directional optical flow (BIO) techniques during motion compensation, which modifies predictive sample values based on optical flow trajectories to determine more accurate predictive blocks, allowing for per-pixel modifications to motion vectors and improved reference block construction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional motion compensation methods are used for bi-predicted inter-coded blocks, then device complexity is reduced, but prediction accuracy deteriorates
Solution Approach 1:
The current block is divided into multiple sub-blocks, and motion vector reconstruction is performed separately for each sub-block using BIO. This segmentation allows the system to achieve higher prediction accuracy through per-sub-block optimization while managing computational complexity by processing smaller units independently rather than applying complex methods to the entire block.
Solution Approach 2:
Different motion characteristics are applied to different regions of the block through sub-block-based BIO. Each sub-block can have its own motion vector reconstruction parameters, allowing the system to adapt to local motion variations and achieve higher prediction accuracy in regions with complex motion patterns while maintaining lower complexity in uniform regions.
2Measurement precision
If BIO is applied to all blocks, then prediction accuracy is improved, but computational complexity increases
Solution Approach 1:
BIO is applied selectively rather than universally - the technique can be enabled only for blocks where it provides significant benefit, or applied to a subset of sub-blocks within a block. This partial application maintains prediction accuracy where needed while reducing overall computational complexity and improving processing throughput.
Solution Approach 2:
By dividing blocks into sub-blocks, the system can apply BIO to only certain sub-blocks rather than processing entire blocks uniformly. This selective sub-block processing allows the encoder/decoder to achieve accuracy improvements in critical regions while skipping less critical regions, thereby improving overall computational efficiency.
3Manufacturing precision
If per-pixel motion vector modifications are implemented, then predictive block quality is improved, but memory bandwidth requirements increase
Solution Approach 1:
Motion vector reconstruction is performed on sub-blocks rather than entire blocks, which reduces the amount of reference pixel data that needs to be loaded from memory at once. This segmentation of the processing region decreases peak memory bandwidth requirements while still achieving high predictive block quality through localized per-pixel modifications within each sub-block.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
For a bi-directional inter predicted block, a video decoder is configured, using a first MV, to locate a first predictive block in a first reference picture; using a second MV, locate a second predictive block in a second reference picture; for a first sub-block of the first predictive block, determine a first amount of bi-directional optical flow (BIO) motion; determine a first final predictive sub-block for the block of video data based on the first amount of BIO motion; for a second sub-block of the first predictive block, determine a second amount of BIO motion; determine a second final predictive sub-block for the block of video data based on the second amount of BIO motion; and based on the first final predictive sub-block and the second final predictive sub-block, determine a final predictive block for the block of video data.