Motion Vector Refinement with Template Matching for Low-Latency Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video decoding technologies face challenges in achieving high throughput and low latency due to decoder-side motion vector refinement processes that create bottlenecks and memory latency issues, particularly with the use of Template Matching (DTM) in Versatile Video Coding (VVC) standards.
Innovation Solution
The proposed solution decouples the DTM stage from the motion data reconstruction process, allowing for advanced reference block fetching and enables pipelining by using decoded motion data for initial vector fetching, and employs template matching with reconstructed samples only from inter-coded neighboring blocks, while disabling DTM for small block sizes to maintain throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If decoder-side motion vector refinement with template matching is implemented, then motion estimation accuracy is improved, but memory latency and processing bottlenecks increase
Solution Approach 1:
The patent performs preliminary actions by fetching reference blocks before they are strictly needed in the decoding pipeline. The decoder fetches reference blocks from L0 and L1 reference pictures in advance, stores them in buffer memory, and makes them available for subsequent motion compensation operations. This preliminary fetching eliminates memory latency bottlenecks by ensuring data is ready before required.
Solution Approach 2:
The patent segments the motion vector refinement process into independent parallel operations. Different coding units (CUs) are processed independently with their own motion vector differences (MVDs) and motion vector predictors (MVPs). This segmentation allows parallel processing of multiple CUs simultaneously, increasing throughput while maintaining accuracy through individual template matching for each CU.
2Measurement precision
If template matching is performed for all blocks, then motion vector refinement accuracy is improved, but processing complexity and throughput requirements increase
Solution Approach 1:
The patent applies local quality by performing template matching selectively based on block characteristics. The process checks whether a current block is inter-coded and whether neighboring blocks are available before applying DTM. This selective application reduces processing complexity for blocks that don't benefit from refinement while maintaining high accuracy for suitable blocks.
Solution Approach 2:
The patent changes parameters dynamically based on block size and type. For small blocks (e.g., 4x4, 8x8), the patent disables DTM to reduce complexity. For larger blocks, template matching is applied with appropriate search ranges and cost functions. The patent also adjusts the refinement process based on whether L0 or L1 reference pictures are used, optimizing the balance between accuracy and complexity for different scenarios.
Data Source
AI summary
A video decoder is provided that includes memory and a processor coupled to the memory. The processor may be configured to convert a bitstream into inter-prediction parameters and reconstruct motion data based on the inter-prediction parameters. The processor may further be configured to refine the motion data based on finding a match between a current template of a current picture and a reference template of a reference picture and perform a motion compensation operation with the refined motion data and a reference block to generate an inter-prediction block. The processor may be configured to add the inter-prediction block to an inter-residual block to produce a reconstructed block. The motion data may be reconstructed without refined motion data associated with a previous prediction unit.