Super-Resolution Algorithm Using Projective Motion Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current super-resolution algorithms face challenges in accurately registering and interpolating low-resolution images with complex motion and aliasing, leading to difficulties in reconstructing high-resolution images, especially when local object motion and scene changes are present, which affects the fidelity of the enhanced images.
Innovation Solution
The method involves a two-phase process that includes relative-motion estimation using the eight-parameter projective model and regular-to-irregular spatio-temporal interpolation, along with restoration techniques like deblurring, to enhance the resolution of low-resolution digital images or videos, addressing the complexities of motion and aliasing by initializing motion estimation and using bicubic interpolation for accurate pixel intensity reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If non-uniform interpolation is used to handle complex motion between LR images, then subpixel motion accuracy can be achieved, but the computational complexity and difficulty of implementation increase significantly
Solution Approach 1:
The patent segments the complex motion estimation problem into two distinct phases: (1) an initial coarse estimation phase using standard block-matching algorithms to obtain pixel-level motion vectors, and (2) a refined precision phase using the eight-parameter projective model to achieve subpixel accuracy. This segmentation allows the system to first establish a rough motion framework and then refine it, avoiding the need to implement complex non-uniform interpolation directly while still achieving high precision motion compensation.
Solution Approach 2:
The patent performs preliminary motion estimation using conventional block-matching algorithms before applying the eight-parameter projective model. This preliminary action provides initial motion vectors that serve as the foundation for subsequent refinement, allowing the system to prepare the data structure and motion hypotheses in advance, thereby simplifying the final precision estimation step and reducing overall computational complexity.
2Manufacturing precision
If multiple LR images with complex motion are registered to reconstruct HR images, then super-resolution can be achieved, but the presence of local object motion and scene changes degrades reconstruction fidelity
Solution Approach 1:
The patent employs the eight-parameter projective model which dynamically adapts to complex motion patterns including local object motion and scene changes. Unlike rigid global motion models, this dynamic model can accommodate varying motion characteristics across different regions of the image, allowing accurate registration even when objects move independently or when the scene structure changes between frames, thereby maintaining high reconstruction fidelity.
Solution Approach 2:
The patent changes the parameter space from standard block-matching (which assumes simple translational motion) to the eight-parameter projective model that incorporates perspective transformations, scaling, and rotation. This parameter expansion allows the system to model complex real-world motion scenarios more accurately, including local object motion and scene changes, thereby improving reconstruction fidelity while achieving super-resolution.
3Ease of manufacture
If conventional block-matching algorithms are used for motion estimation, then computational simplicity is maintained, but accuracy is insufficient for super-resolution reconstruction
Solution Approach 1:
The patent uses conventional block-matching algorithms as a preliminary step to obtain initial motion vectors at pixel level accuracy. This preliminary estimation provides a solid foundation that is computationally simple to obtain, and then these initial vectors are refined using the eight-parameter projective model to achieve subpixel accuracy, thereby combining the advantages of both simple and complex approaches.
Solution Approach 2:
The patent segments the motion estimation process into two distinct stages: a first stage using computationally simple block-matching for coarse estimation, and a second stage using the more complex eight-parameter projective model for precision refinement. This segmentation allows the system to maintain computational efficiency for the bulk of the work while applying sophisticated algorithms only where high precision is critical, thus balancing simplicity and accuracy.
Data Source
AI summary
In particular embodiments, a process relating to the creation of a high-resolution video from a low-resolution video. In a particular embodiment, the process receives as input a sequence of low-resolution video frames. The process first determines a matching score for consecutive frames in the sequence, where the matching score is based on a preliminary global transformation between consecutive frames. From the matching scores, the process determines a set of matching windows and relative-motion estimates and then uses the set and estimates to calculate more thorough global transformations and any residual relative-motion which can be explained using independent object motion and/or optical flow. The process uses the latter global transformation and any independent object motion and/or optical flow to create motion trajectories and to generate high-resolution frames by interpolating low-resolution frames at trajectory-defined points, using regular-to-irregular spatio-temporal interpolation. The process then restores the frames, using deblurring, and outputs them.