Co-located Reference Frame Interpolation via Optical Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional motion compensated prediction in video compression fails to capture true motion activities, especially non-translational motion, due to direct block-based prediction methods that do not fully utilize bi-directional reference frames.
Innovation Solution
The method involves interpolating a co-located reference frame using optical flow estimation to generate an optical flow reference frame, which tracks complicated motion activities by calculating per-pixel motion fields from available forward and backward reference frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If direct block-based prediction methods are used, then device complexity is reduced, but measurement precision of motion activities deteriorates
Solution Approach 1:
The patent segments the reference frames into multiple regions (first reference frame region and second reference frame region) and applies different prediction methods to each region. Block-based prediction is used in the first region while optical flow-based prediction is used in the second region, allowing the system to balance complexity and accuracy by applying computationally intensive methods only where needed.
Solution Approach 2:
The patent applies different prediction qualities to different regions of the current frame based on motion characteristics. Regions with complex non-translational motion (second region) receive high-precision optical flow-based prediction, while regions with simple motion (first region) use standard block-based prediction, optimizing overall performance without uniformly increasing complexity.
2Measurement precision
If optical flow estimation is performed using multiple reference frames, then measurement precision of motion information is improved, but device complexity increases
Solution Approach 1:
The patent divides the current frame into multiple regions and performs optical flow estimation only for specific regions (second reference frame region) where complex motion is detected, rather than applying it uniformly across the entire frame. This segmentation reduces overall computational complexity while maintaining precision where needed.
Solution Approach 2:
The patent performs partial optical flow estimation only for portions of the frame that require it, rather than applying the computationally intensive optical flow algorithm to the entire frame. This partial action approach maintains motion information accuracy for complex regions while reducing overall processing complexity.
3Measurement precision
If per-pixel motion fields are calculated from bi-directional reference frames, then measurement precision of motion activities is improved, but loss of time for processing increases
Solution Approach 1:
The patent performs preliminary identification of regions requiring complex motion tracking and prepares appropriate prediction methods in advance. By pre-segmenting the frame and identifying areas with non-translational motion, the system avoids unnecessary optical flow calculations in simple regions, reducing overall processing time while maintaining accuracy where needed.
Solution Approach 2:
The patent applies high-precision per-pixel motion field calculation only to specific regions where complex motion is detected, rather than processing the entire frame with the same level of detail. This local application of intensive processing maintains motion tracking accuracy for complex areas while reducing total processing time.
Data Source
AI summary
An optical flow reference frame is generated that can be used for inter prediction of blocks of a current frame in a video sequence. A first (e.g., forward) reference frame and a second (e.g., backward reference frame) are used in an optical flow estimation that produces a respective motion field for pixels of the current frame. The motion fields are used to warp the reference frames to the current frame. The warped reference frames are blended to forms the optical flow reference frame.