Optical Flow Prediction Refinement for Unified Bi-Prediction Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing video coding technologies, such as VVC, face inefficiencies in motion compensation due to limitations in block-based motion estimation, particularly with bi-predicted coding blocks, and the non-unified designs of bi-directional optical flow (BDOF) and prediction refinement with optical flow (PROF) for affine mode, which hinder hardware implementation and coding efficiency.
Innovation Solution
Harmonize the designs of BDOF and PROF by aligning bit-depth representations and gradient calculations, allowing shared pipeline processing and unified gradient derivation methods to enhance motion refinement precision and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate pipeline processing is used for BDOF and PROF, then flexibility in processing is maintained, but hardware implementation complexity increases and processing efficiency decreases
Solution Approach 1:
The patent merges the separate BDOF and PROF pipeline processing into a unified processing structure. The harmonized design allows both BDOF and PROF to share common processing stages including gradient calculation, motion refinement, and prediction sample generation, thereby reducing hardware complexity while improving processing efficiency through resource sharing and streamlined operations.
Solution Approach 2:
The unified pipeline design implements multi-functional processing units that can handle both BDOF and PROF operations. The gradient calculation unit, motion refinement unit, and prediction generation unit are designed to serve both prediction modes, eliminating redundant hardware components and enabling a single processing architecture to perform multiple functions efficiently.
2Measurement precision
If non-unified bit-depth representations are used for BDOF and PROF, then each mode can be optimized independently, but motion refinement precision decreases and processing efficiency is reduced
Solution Approach 1:
The patent standardizes the bit-depth representation parameters for both BDOF and PROF to use consistent precision levels (e.g., 10-bit or 12-bit internal precision). This parameter harmonization ensures that gradient calculations, motion refinements, and prediction samples are computed with uniform precision across both prediction modes, improving overall motion refinement accuracy while enabling more efficient processing through consistent data path widths.
3Productivity
If different gradient calculation methods are used for BDOF and PROF, then each mode can use its optimal method, but device complexity increases and processing speed decreases
Solution Approach 1:
The patent implements a universal gradient calculation unit that serves both BDOF and PROF modes. The unit calculates horizontal and vertical gradients from prediction samples using a standardized algorithm, providing consistent gradient values for both prediction modes. This eliminates the need for separate gradient calculation circuits, reducing hardware complexity and enabling parallel processing that improves overall processing speed.
Data Source
AI summary
Methods, apparatuses, and non-transitory computer-readable storage mediums are provided for decoding a video signal. A method may include obtain a first reference picture and a second reference picture associated with a video block, wherein the first reference picture is before a current picture and the second reference picture is after the current picture in display order; obtain first prediction samples of the video block from the first reference picture; obtain second prediction samples of the video block from the second reference picture; obtain padded prediction samples, and obtain horizontal and vertical gradient values of the first prediction samples and the second prediction samples based on the padded prediction samples; obtain motion refinements for samples in the video block based on the horizontal and vertical gradient values; and obtain bi-prediction samples of the video block based on the motion refinements.