Motion Vector Selection in Hardware Video Encoders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hardware video encoders face performance issues due to pipeline dependencies, where the actual predicted motion vector may not be available in time for encoding, leading to suboptimal motion vector selection and reduced real-time performance.
Innovation Solution
Selecting a plurality of candidate predicted motion vectors to seed motion searches, calculating initial cost scores, filtering for low-cost motion vectors, and updating scores with the actual predicted motion vector when available, to select the optimal motion vector for encoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If motion searches are performed using candidate predicted motion vectors before the actual predicted motion vector is available, then real-time processing performance is maintained, but the accuracy of motion vector selection deteriorates
Solution Approach 1:
The system performs motion searches in advance using candidate predicted motion vectors (spatial, temporal, and combined predictions) before the actual predicted motion vector from the previous macroblock is available. This preliminary action maintains pipeline flow and real-time processing performance while accepting temporarily suboptimal results that will be corrected later when the actual predicted motion vector becomes available for re-scoring.
2Measurement precision
If the pipeline waits for the actual predicted motion vector to be available before performing motion searches, then motion vector selection accuracy is improved, but processing speed and real-time performance deteriorate
Solution Approach 1:
Motion searches are performed in advance using available candidate predicted motion vectors rather than waiting for the actual predicted motion vector. This preliminary action eliminates pipeline stalls and maintains processing speed, with the understanding that results will be refined later when accurate prediction information becomes available.
Solution Approach 2:
When the actual predicted motion vector from the previous macroblock becomes available, the system re-scores the previously obtained motion vectors using this new information. This feedback mechanism corrects the initial suboptimal selections and improves motion vector accuracy without impacting real-time processing performance.
3Reliability
If multiple candidate predicted motion vectors are used for motion searches, then motion vector selection robustness is improved, but computational complexity increases
Solution Approach 1:
The system segments the motion vector prediction process into multiple independent candidate sources (spatial prediction from neighboring macroblocks, temporal prediction from corresponding macroblocks in reference frames, and combined predictions). Each candidate is evaluated separately through motion searches, allowing robust selection without requiring a single complex prediction mechanism.
Solution Approach 2:
The system performs motion searches for multiple candidate predicted motion vectors (more than the single optimal vector that would be used if available) to ensure robustness. This excessive action of searching multiple candidates provides redundancy and reliability, with the understanding that not all candidates will be equally useful, but the extra searches guarantee finding good matches even when predictions are imperfect.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Various embodiments relating to selecting a motion vector in a hardware encoder are disclosed. In one example, a plurality of candidate predicted motion vectors are selected, and a plurality of motion searches are performed in an image region surrounding each candidate predicted motion vector to produce a plurality of resulting motion vectors, wherein each resulting motion vector has an initial cost score determined using a corresponding candidate predicted motion vector. After an actual predicted motion vector becomes available, the initial cost score of each resulting motion vector is re-scored using the actual predicted motion vector to produce an updated cost score, and video data is encoded using a motion vector selected from the plurality of resulting motion vectors based on the updated cost score of that motion vector.