Motion Refinement Using Adaptive Block Segmentation for Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current motion compensation techniques, such as Bi-directional Optical Flow (BIO), Pattern-based MV Derivation (PMVD), and Decoder-side MV Refinement (DMVR), face challenges in reducing system memory bandwidth and computational complexity, particularly in bi-direction predicted blocks, which affects coding efficiency and performance.
Innovation Solution
The proposed solution involves reducing the required bandwidth by using adaptive block sizes for BIO and applying filter coefficient shifting or padding in PMVD and DMVR processes, limiting the use of reference data to only the valid reference blocks and their immediate surroundings, and employing simplified gradient filters to generate motion refinement without additional 6-tap filters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If BIO uses a 5x5 window to derive sample-level motion refinement, then motion refinement precision is improved, but memory bandwidth requirement increases
Solution Approach 1:
The patent divides the current block into multiple smaller BIO blocks (e.g., 4x4 blocks within an 8x8 block). Each BIO block independently uses a 5x5 window for motion refinement, but the segmentation allows reuse of reference data across adjacent BIO blocks, reducing overall memory bandwidth requirements compared to applying BIO to the entire block at once.
Solution Approach 2:
The patent applies BIO motion refinement selectively to specific BIO blocks within the current block based on criteria such as block size, motion complexity, or prediction mode. This partial application of BIO reduces the total computational load and memory bandwidth consumption while maintaining refinement precision where it is most beneficial.
2Measurement precision
If BIO uses 6-Tap gradient filters and 6-Tap interpolation filters, then motion refinement accuracy is improved, but computational complexity increases
Solution Approach 1:
By segmenting the current block into multiple BIO blocks, the patent reduces the number of 6-Tap gradient filters and 6-Tap interpolation filters that need to be applied. Each BIO block requires its own filters, but segmentation into smaller blocks reduces the total number of filter applications compared to applying filters to every pixel in a larger block, thereby reducing computational complexity while maintaining accuracy within each BIO block.
Solution Approach 2:
The patent applies the computationally intensive 6-Tap gradient and interpolation filters only to specific BIO blocks that require high precision motion refinement, rather than uniformly applying them to all blocks. This selective application based on local characteristics (such as motion complexity or block position) maintains accuracy where needed while reducing overall computational complexity.
3Reliability
If additional reference data outside the valid reference block is used, then motion refinement performance is improved, but system memory bandwidth increases
Solution Approach 1:
The patent segments the reference data access into multiple smaller BIO blocks, each accessing only its necessary portion of the reference picture. This segmentation allows the system to reuse reference data across adjacent BIO blocks rather than loading entire reference blocks into memory, reducing overall memory bandwidth consumption while maintaining the ability to access necessary reference data for accurate motion refinement.
Data Source
Figure 1~2
Figure 3~4
Figure 5A~5B
AI summary
Method and apparatus of using Pattern-based MV derivation (PMVD), Bi-directional optical flow (BIO) or Decoder-side MV refinement (DMVR) to refine motion for a bi-direction predicted block are disclosed. According to one method of the present invention, a first and a second motion-compensated reference blocks associated with the current block in a first and a second reference pictures from reference picture list 0 and list 1 are determined respectively. The PMVD process, BIO process or DMVR process is applied to generate motion refinement for the current block using reference data corresponding to the first motion-compensated reference block and the second motion-compensated reference block without other reference data outside the first motion-compensated reference block and the second motion-compensated reference block so as to reduce required system memory bandwidth. Also an adaptive block size method for block-based BIO is disclosed.