Co-Located Reference Frame Refinement Using Fine Motion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video encoding techniques fail to accurately capture individual pixel movements within a block, leading to inefficiencies in data compression and increased computational complexity when generating co-located reference frames for inter-prediction.
Innovation Solution
The method involves reconstructing reference frames and determining a coarse motion field, refining it with fine motion vectors for sub-blocks using optical flow estimation, and updating motion vectors to generate a co-located reference frame for improved inter-prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If coarse motion field is used for all blocks, then computational complexity is reduced, but prediction accuracy deteriorates due to inability to capture individual pixel movements
Solution Approach 1:
The current frame is divided into multiple blocks, and each block is further divided into sub-blocks. Different motion refinement strategies are applied to different blocks based on their characteristics, allowing the system to balance computational complexity and prediction accuracy by processing only certain blocks with fine motion vectors.
Solution Approach 2:
Fine motion vectors are applied selectively to specific blocks rather than uniformly across the entire frame. Blocks with higher motion variability or greater prediction error receive fine motion refinement, while other blocks use only coarse motion fields, optimizing the trade-off between accuracy and complexity locally.
2Measurement precision
If fine motion vectors are applied to all blocks, then prediction accuracy is improved, but computational complexity increases
Solution Approach 1:
The frame is segmented into multiple blocks with different refinement requirements. By processing only certain blocks with fine motion vectors rather than all blocks, the system achieves improved prediction accuracy where needed while avoiding unnecessary computational overhead in regions where coarse motion fields suffice.
Solution Approach 2:
Fine motion refinement is applied partially to only those blocks that benefit most from it, rather than applying it excessively to all blocks. This selective approach achieves sufficient prediction accuracy improvement without the full computational cost of universal fine motion application.
3Productivity
If motion field refinement is performed, then data compression efficiency is improved, but processing time increases
Solution Approach 1:
The video frame processing is divided into multiple blocks that can be processed independently and in parallel. This segmentation allows the system to achieve improved compression efficiency through selective motion refinement while reducing overall processing time by distributing the computational workload across multiple blocks simultaneously.
4Speed
If coarse motion field is used, then processing speed is maintained, but motion capture precision deteriorates for individual pixels
Solution Approach 1:
By dividing the frame into blocks and selectively applying fine motion vectors to certain blocks, the system maintains high processing speed through efficient block-level processing while improving motion capture precision for pixels in blocks that require it, achieving a balance between speed and precision.
Data Source
AI summary
Techniques for generating and using a co-located reference frame are described, A first reference frame and a second reference frame for a current frame to be encoded or decoded are reconstructed and used to determine a coarse motion field for the current frame. Fine motion of one or more blocks of the current frame are estimated (e.g., using optical flow estimation). A motion vector of a block of the current frame is updated using the fine motion to result in an updated motion field. A co-located reference frame is determined using the updated motion field, and a prediction process for the current frame is performed using the co-located reference frame. Various techniques may be used to determine whether to adjust the coarse motion field using the fine motion to reduce computing requirements.