Relative Motion Field Estimation for Video Super-Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for multi-frame integration in video processing, particularly for high-definition video output, face significant computational burdens due to the need for recursive estimation of inter-frame motion fields, which slows down the process and limits the frame rate of video output.
Innovation Solution
The proposed solution involves using relative motion field estimation between neighboring frames instead of absolute motion estimation with respect to a reference frame, allowing for the reuse of previously calculated motion fields by shifting the time of integration window and calculating new relative motion fields only for frames not in the previous window, thereby reducing the computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If absolute motion field estimation with respect to a reference frame is used for multi-frame integration, then motion accuracy is maintained, but computational complexity increases significantly and processing speed decreases
Solution Approach 1:
The patent segments the motion field estimation process into multiple passes. In the first pass, a coarse motion field is estimated using a large search window. In subsequent passes, finer motion fields are estimated using smaller search windows and the results from previous passes as initial values. This segmentation reduces computational complexity while maintaining accuracy through iterative refinement.
Solution Approach 2:
The patent performs preliminary motion field estimation in the first pass with relaxed accuracy requirements (larger search windows). The results from this preliminary estimation are then used as initial values or constraints in subsequent passes, eliminating the need to re-estimate entire motion fields and significantly reducing computational load in later iterations.
2Speed
If recursive multi-frame integration is performed at high frame rates (e.g., 30 fps), then real-time video output is achieved, but the motion field estimation stage becomes a severe bottleneck
Solution Approach 1:
The patent dynamically adjusts the search window size and estimation precision based on the integration stage and available computational resources. Early stages use larger windows for coarse estimation, while later stages use smaller windows for refinement. This dynamic adaptation allows the system to maintain high frame rates by reducing computational burden in later recursive iterations.
Solution Approach 2:
The patent maintains continuous useful action by using motion field estimates from previous integration stages as initial values for subsequent stages. This continuity eliminates redundant computations and allows the recursive process to proceed efficiently at high frame rates, as each iteration builds upon previous results rather than starting from scratch.
3Productivity
If the time of integration window is shifted to process subsequent frames, then progressive video output is achieved, but previously calculated motion fields must be discarded and recalculated
Solution Approach 1:
The patent performs preliminary motion field estimation that is valid across multiple integration windows. By estimating motion fields with sufficient temporal extent in advance, the system avoids discarding and recalculating motion fields when shifting the integration window, thereby reducing recomputation time and maintaining high productivity.
Solution Approach 2:
The patent makes motion field estimates serve multiple functions and multiple integration windows simultaneously. A single motion field estimation can be reused across several consecutive frames or integration windows, allowing the system to maintain high video output rates without the penalty of recalculating motion fields for each new window.
Data Source
AI summary
The invention relates to a method for estimation of interframe motion fields operating on a stream of video frames, and more particularly for accelerating video output in multiframe super-resolution thus improving the efficiency of the multiframe integration. Relative motion field estimation is used between neighboring or close images instead of with respect to a reference image, for at least some of the frames within an integration window (TOI). The integration window is slid along the time axis each time by one (or two or a few) frames so that the current integration window preferably covers the majority of the frames in the previous integration window. Using relative motion estimation and then (tracking and) summing up the related motion fields enables, in each recursion, the absolute motion fields in a new integration window to be obtained without re-computing all the motion fields of earlier frames in the new integration window.