Template Matching Motion Prediction for Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video coding techniques using motion vectors for motion prediction are inefficient, particularly in inter-coding, as they require encoding and decoding of motion vectors, which can lead to increased complexity and residual errors.
Innovation Solution
The implementation of template matching motion prediction, where video blocks are encoded and decoded based on predictive reference data from neighboring pixels, eliminating the need for motion vectors by deriving them from a template matching process, and incorporating improved hypothesis selection, weighting, and luma residual techniques to enhance accuracy and compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional motion vector-based motion prediction is used, then motion compensation can be achieved, but encoding complexity and residual errors increase
Solution Approach 1:
The patent extracts and eliminates motion vectors from the encoding process entirely. Instead of using conventional motion vectors to predict block positions, the invention uses template matching on neighboring pixels to directly generate prediction values, removing the motion vector encoding and decoding steps that contribute to complexity while maintaining prediction accuracy
Solution Approach 2:
The patent substitutes the mechanical motion vector system with a template matching system. Rather than calculating and encoding motion vectors that point to reference blocks, the system creates templates from neighboring pixels and matches them against reference frames to directly obtain prediction values, replacing the vector-based mechanical approach with a pattern-matching approach
2Productivity
If motion vectors are encoded and decoded, then motion prediction can be performed, but residual errors increase
Solution Approach 1:
The patent creates copies of neighboring pixel patterns (templates) and uses these copies to match against reference frames. By copying the local pixel structure rather than using motion vectors, the system achieves more accurate prediction of actual pixel values, reducing residual errors while maintaining compression efficiency
Solution Approach 2:
The patent changes the fundamental parameter used for prediction from motion vector coordinates to template matching scores. By transforming the prediction mechanism from vector-based displacement to pattern-based matching, the system achieves both better precision and maintained productivity
3Measurement precision
If template matching is used instead of motion vectors, then residual values are reduced, but computational complexity increases
Solution Approach 1:
The patent segments the prediction process into localized template operations rather than global motion vector calculations. By dividing the frame into blocks and creating small templates from neighboring pixels for each block, the system achieves high matching accuracy through many small, independent operations that can be efficiently parallelized, reducing overall computational power requirements
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
In one aspect of this disclosure, template matching motion prediction is applied to B-frames. In another aspect of this disclosure, template matching motion prediction as applied to video block coding may include generating a template offset, generating a weighted sum of absolute differences, selecting a number of hypotheses used to encode video blocks based on the cost associated with the number of hypotheses and signaling, with a new syntax, to a decoder, the number of hypotheses used in encoding, rejecting hypotheses if the difference in value between a hypothesis and a reference hypothesis is greater than a threshold value, and/or generating the content of a sub-block that does not have reconstructed data available by combining motion-compensated prediction and luma residuals.