Video Super-Resolution With Gradient Blending for Flicker Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video super-resolution models face challenges such as temporal coherence issues, visible flickers, hallucinations, and computational inefficiencies when applied to video frames, particularly on resource-constrained devices like smartphones, leading to suboptimal image quality and increased computational requirements.
Innovation Solution
A video super-resolution technology that records at a lower resolution for better light gathering and uses temporal super-resolution techniques, combined with gradient blending and generative adversarial networks, to upscale video frames while minimizing artifacts and maintaining temporal consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If generative adversarial network (GAN) models are used for super-resolution, then image quality and detail generation are improved, but brightness and color shift from input frames occurs
Solution Approach 1:
The patent introduces a brightness and color correction module as an intermediary component that processes the output of the GAN model. This module uses adaptive color transformation and brightness adjustment to restore the original brightness and color characteristics, thereby mediating between the high image quality generated by GAN and the brightness/color fidelity requirement.
Solution Approach 2:
The patent dynamically adjusts brightness and color parameters based on the input frame characteristics. By monitoring the input frame's brightness and color distribution, the system applies corresponding transformation parameters to the GAN output, ensuring that brightness and color shifts are corrected while maintaining the enhanced image quality.
2Manufacturing precision
If super-resolution models generate more details to sharpen frames, then image sharpness is improved, but artifacts and hallucinations increase
Solution Approach 1:
The patent applies different processing strategies to different regions of the image based on their characteristics. High-frequency regions with strong edges receive the GAN-generated details to enhance sharpness, while low-frequency regions maintain the original content to avoid hallucinations. This spatially adaptive approach ensures local quality optimization without global degradation.
Solution Approach 2:
The system incorporates feedback mechanisms that monitor the generated output for artifacts and hallucinations. By comparing the super-resolved output with the input frame characteristics and using attention maps to identify problematic regions, the system adjusts the detail generation intensity dynamically, reducing artifacts while maintaining sharpness in valid regions.
3Manufacturing precision
If video super-resolution is applied to enhance resolution, then image quality is improved, but temporal coherence issues and visible flickers occur
Solution Approach 1:
The patent implements dynamic blending weights that adapt to the temporal characteristics of video frames. By analyzing motion vectors and frame differences, the system dynamically adjusts the blending ratio between different super-resolution methods for each frame, ensuring temporal coherence while maintaining image quality enhancement.
Solution Approach 2:
The patent merges multiple super-resolution methods (GAN-based and traditional GOMP) in a unified framework. By combining the strengths of both approaches and using attention mechanisms to weight their contributions, the system achieves both image quality enhancement and temporal coherence, reducing flickers that would occur with single-method approaches.
4Loss of time
If tiling is used for parallel computation, then inference latency is reduced, but proper tiling strategy and intersection handling become challenging
Solution Approach 1:
The patent segments the video frame into multiple tiles that can be processed in parallel. By dividing the large input frame into smaller manageable tiles, the system achieves parallel computation and reduced inference latency. The segmentation is designed to be computationally efficient, avoiding complex intersection handling by processing tiles independently and combining results through simple blending operations.
Data Source
AI summary
An example method includes receiving, by a computing device, a plurality of video frames captured at a first resolution. The method also includes applying a trained machine learning model to upscale the plurality of video frames to a second resolution, wherein the second resolution is higher than the first resolution. The method additionally includes applying a gradient blending process to the upscaled plurality of video frames. The method also includes providing the gradient blended and upscaled plurality of video frames.