Video Encoder Merge Candidate Ranking for Top-K Motion Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern video codecs face challenges in reducing computational complexity while maintaining video quality due to the need to evaluate a large number of possible merge candidates for motion vector encoding, making it impractical to perform motion compensation and rate-distortion optimization on all candidates.
Innovation Solution
Implementing a reference frame priority list to reduce the number of possible reference frame combinations and using a top-K sorting algorithm to efficiently select the top merge candidates for motion vector encoding, thereby reducing the number of candidates evaluated while maintaining quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If motion compensation and rate-distortion optimization are performed on all merge candidates, then video quality is improved, but computational complexity increases significantly
Solution Approach 1:
The patent applies partial action by performing motion compensation and rate-distortion optimization only on a subset of top-K merge candidates (e.g., top 3 candidates) rather than all candidates. This selective approach maintains acceptable video quality while significantly reducing computational complexity, as the highest-weighted candidates are most likely to yield optimal results.
Solution Approach 2:
The patent implements preliminary action by calculating weights for all merge candidates in advance based on reference frame priorities and spatial/temporal relationships, then sorting them before performing expensive motion compensation and rate-distortion optimization. This pre-screening step identifies the most promising candidates upfront, avoiding unnecessary computations on low-priority candidates.
2Measurement precision
If all merge candidates are evaluated, then encoding accuracy is improved, but encoding time increases
Solution Approach 1:
The patent evaluates only the top-K merge candidates (partial action) rather than all candidates, achieving acceptable encoding accuracy by focusing computational resources on the most promising candidates. This approach reduces encoding time significantly while maintaining quality through selective evaluation based on pre-calculated weights.
Solution Approach 2:
The patent performs preliminary weight calculation and sorting of merge candidates before the main encoding evaluation. This advance preparation identifies high-priority candidates based on reference frame importance and spatial/temporal relationships, enabling faster encoding by pre-filtering out low-value candidates that would otherwise consume encoding time.
3Productivity
If the number of reference frame combinations is reduced, then computational workload is decreased, but video quality may deteriorate
Solution Approach 1:
The patent performs preliminary prioritization of reference frames based on their importance weights calculated from temporal and spatial relationships. By pre-identifying the most significant reference frames and their combinations, the system can reduce the number of combinations evaluated while maintaining video quality, as the highest-priority combinations are processed first.
Solution Approach 2:
The patent applies partial action by evaluating only the top reference frame combinations (those with highest weights) rather than all possible combinations. This selective evaluation maintains computational efficiency while preserving video quality, as the most important reference frame combinations contribute most to prediction accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A block of a video frame can be encoded using inter-prediction, and the motion vector of the block can be encoded based on a motion vector reference of a merge candidate. Some video codecs allow a large range of temporal and spatial neighbors to be considered as potential merge candidates. It is not practical to perform motion compensation and rate-distortion optimization for all possible merge candidates. To address this concern, a hardware-efficient process can be implemented to rank and select merge candidates. A reference frame priority list is applied to select a subset of potential reference frame combinations. An efficient top-K sorting algorithm is applied to identify merge candidates for each reference frame combination and keep top merge candidates with highest weights. Motion compensation and rate-distortion optimization are performed on the top merge candidates only.