Temporal Stability in Single Frame Super Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current super resolution and frame rate conversion techniques often result in temporal instabilities and noise in upscaling images, failing to distinguish between noise and details, leading to artifacts and inconsistencies in the temporal domain due to insufficient information from single frames or external libraries.
Innovation Solution
Combining single frame super resolution with multi-frame super resolution and frame rate conversion, using motion vector refinement and frequency channel temporal filtering to generate high-resolution images with improved temporal stability, by fusing information from multiple frames and employing adaptive filtering techniques to reduce noise and enhance detail preservation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If single frame super resolution is used to generate higher resolution frames, then computation cost is reduced, but temporal stability deteriorates and noise increases
Solution Approach 1:
The patent combines single-frame super-resolution processing with multi-frame temporal filtering, merging the computational efficiency of single-frame methods with the temporal stability of multi-frame approaches to achieve both low computation cost and high temporal stability
Solution Approach 2:
The patent introduces motion compensation and temporal filtering as intermediary processes that process the output of single-frame super-resolution, acting as mediators to remove noise and enhance temporal stability without requiring multiple input frames
2Adaptability or versatility
If upscaling techniques are used to convert low resolution to high resolution, then device resolution compatibility is improved, but temporal stability deteriorates and artifacts increase
Solution Approach 1:
The patent employs temporal filtering that uses feedback from previous and future frames to refine the current frame's high-resolution output, reducing temporal artifacts and improving stability while maintaining adaptability to different display resolutions
Solution Approach 2:
The patent uses motion-compensated temporal filtering that dynamically adjusts the filtering strength based on motion detection, applying stronger filtering to stable regions and weaker filtering to moving regions to maintain temporal stability without compromising motion fidelity
3Speed
If information from a single image is used for super resolution, then processing speed is improved, but noise discrimination capability deteriorates
Solution Approach 1:
The patent performs motion estimation and temporal filtering in preliminary processing stages before final super-resolution output, preparing noise discrimination capabilities in advance to enable single-frame processing to achieve better noise discrimination without sacrificing processing speed
Data Source
AI summary
A method includes receiving, at a processor, at least one frame of input image data, producing motion vector fields between the frames of input image data, and applying temporal stability to the at least one frame of the input image data to produce noise reduced image data, wherein applying temporal stability comprises separating pixel data into frequency bands.