Template-Matching Motion Vector Refinement for Low-Latency Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The conventional template matching (TM) motion vector refinement process in video coding systems experiences significant latency due to the need to access fully reconstructed neighboring blocks, which hinders efficient processing.
Innovation Solution
The proposed method reduces latency by using predefined regions, parent nodes, or adaptive L-shapes to generate templates for motion vector refinement, allowing the use of partially reconstructed or predicted samples, and employing multiple template options based on CU partitioning and processing order, thereby minimizing the reliance on fully reconstructed neighboring blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional template matching motion vector refinement is used, then motion vector accuracy is improved, but processing latency increases due to waiting for fully reconstructed neighboring blocks
Solution Approach 1:
The patent performs motion vector refinement using template matching before the neighboring blocks are fully reconstructed. It uses prediction samples and DC values that are available earlier in the processing pipeline, allowing the refinement to proceed in advance without waiting for the complete reconstruction of neighboring blocks, thus reducing processing latency while maintaining accuracy
Solution Approach 2:
The patent introduces intermediate prediction samples and DC values as mediators to bridge the gap between incomplete reconstructed data and the requirements for accurate template matching. These intermediaries allow the motion vector refinement to proceed with available data rather than waiting for fully reconstructed neighboring blocks
2Measurement precision
If fully reconstructed neighboring blocks are used for template matching, then matching accuracy is improved, but processing speed deteriorates due to reconstruction delays
Solution Approach 1:
The patent performs template matching and motion vector refinement using prediction samples and DC values before the neighboring blocks are fully reconstructed. This preliminary action allows the process to proceed with available data, improving processing speed without significantly compromising matching accuracy
Solution Approach 2:
The patent changes the data parameters used for template matching from requiring fully reconstructed blocks to using prediction samples and DC values. This parameter change enables the process to operate with lower-quality intermediate data, thereby improving processing speed while maintaining acceptable accuracy through subsequent refinement steps
Data Source
AI summary
A method and apparatus for video coding system that utilizes low-latency template-matching motion-vector refinement are disclosed. The method receives input data comprising a plurality of blocks of a video unit in a current picture and determines a position of a current template for a current block of the plurality of blocks based on at least one parameter of the current block. The method then determines candidate reference templates associated with the current block at a set of candidate locations in a reference picture, wherein each candidate reference template corresponds to the current template at one corresponding candidate location, determining a location of a target reference template among the candidate reference templates that achieves a best match between the current template and the candidate reference templates; and refining an initial motion vector (MV) according to the location of the target reference template.