Optical Flow Prediction Refinement for Affine Video Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies, such as VVC, face inefficiencies in motion compensation due to limitations in block-based motion estimation, particularly for affine motion models, which can be improved by refining prediction methods like PROF and BDOF to enhance coding efficiency.
Innovation Solution
Implementing prediction refinement with optical flow (PROF) and bi-directional optical flow (BDOF) to refine motion estimation by using gradient values and motion refinements, and applying weighting and offset parameters for improved prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If block-based motion estimation is used in video coding, then device complexity is reduced and ease of operation is improved, but motion compensation precision deteriorates
Solution Approach 1:
The patent divides the current video block into multiple sub-blocks and performs motion estimation independently for each sub-block. This segmentation allows the system to maintain lower complexity at each sub-block level while achieving higher overall precision through the combination of multiple refined motion vectors, resolving the contradiction between device complexity and motion compensation precision.
Solution Approach 2:
The patent applies different motion compensation techniques to different regions of the video block based on local characteristics. By performing gradient calculations and optical flow refinement at the sub-block level rather than uniformly across the entire block, the system achieves higher local precision where needed while maintaining overall computational efficiency, thus resolving the contradiction between precision and complexity.
2Measurement precision
If prediction refinement with optical flow is applied, then motion compensation precision is improved, but device complexity increases
Solution Approach 1:
The patent segments the optical flow calculation process into discrete steps performed on sub-blocks: gradient calculation, motion refinement, and prediction combination. This segmentation reduces the computational burden at each step compared to applying full optical flow to the entire block, thereby achieving higher precision while controlling device complexity.
Solution Approach 2:
The patent applies optical flow refinement selectively rather than uniformly to all video blocks. By using control flags and conditional logic to determine when PROF/BDOF should be applied, the system achieves high prediction precision for blocks that benefit most from refinement while avoiding unnecessary complexity for blocks where simple motion compensation suffices.
3Measurement precision
If affine motion models are used, then motion representation accuracy is improved, but device complexity and computational load increase
Solution Approach 1:
The patent implements affine motion compensation by dividing the block into sub-blocks, each with its own motion vector derived from control point motion vectors (CPMVs). This segmentation allows the system to capture complex affine motion patterns with higher accuracy while reducing the computational complexity of calculating and storing full affine transformation matrices for the entire block.
Solution Approach 2:
The patent transforms the affine motion model parameters into control point motion vectors that can be more efficiently encoded and processed. By changing the parameter representation from full affine matrices to discrete CPMVs at block corners, the system achieves equivalent motion representation accuracy with reduced device complexity and computational load.
Data Source
AI summary
Methods, apparatuses, and non-transitory computer-readable storage mediums are provided for PROF. A decoder obtains first reference picture and second reference picture associated with a video block that is coded by affine mode within a video signal. The decoder obtains first and second horizontal and vertical gradient values based on first prediction samples I(0)(i,j) and second prediction samples I(1)(i,j) associated with the first reference picture and second reference picture. The decoder obtains first and second horizontal and vertical motion refinements based on CPMVs associated with the first reference picture and second reference picture. The decoder obtains first prediction refinement ΔI(0)(i,j) and second prediction refinement ΔI(1)(i,j) based on the horizontal and vertical gradient values and the first and second horizontal and vertical motion refinements. The decoder obtains final prediction samples of the video block.