Motion Compensation of High-Frequency Image Detail to Reduce Flicker
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Upscaling low-resolution images to high-resolution images using neural networks often results in inconsistent added texture across frames, leading to visible flicker and a deteriorated user experience.
Innovation Solution
A system that includes an image generator using a neural network to enhance images and a motion analyzer to perform motion compensation on high-frequency components, adjusting them based on motion vectors to generate an adjusted enhanced image, thereby reducing flicker by selectively applying temporal filtering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network upscaling is applied to low-resolution images, then image resolution and texture quality are improved, but temporal consistency deteriorates causing visible flicker
Solution Approach 1:
The image is divided into frequency components (low-frequency and high-frequency parts). Motion compensation is selectively applied only to the high-frequency component that causes flicker, while the low-frequency component remains unchanged. This segmentation allows targeted correction of temporal inconsistency without affecting overall image quality.
Solution Approach 2:
Different processing approaches are applied to different parts of the image based on their frequency characteristics. The high-frequency component (which contains texture details and is prone to flicker) receives motion compensation processing, while the low-frequency component (which contains overall structure) is preserved as-is. This local quality approach ensures that correction is applied where needed without unnecessary processing elsewhere.
2Stability of the object's composition
If temporal filtering is applied to reduce flicker, then temporal consistency is improved, but image texture quality deteriorates
Solution Approach 1:
The image processing is segmented by frequency components. Motion compensation is applied exclusively to the high-frequency component that exhibits flicker, while the low-frequency component is left unprocessed. This selective segmentation prevents the degradation of overall image quality that would result from applying temporal filtering to the entire image.
Solution Approach 2:
Different quality preservation strategies are applied to different frequency domains. The high-frequency component undergoes motion-based correction to maintain temporal consistency, while the low-frequency component maintains its original texture quality. This local quality approach ensures that texture degradation is avoided in regions where it is not necessary.
Data Source
Figure 1
Figure 2A~2B
Figure 2C
AI summary
A device includes a memory and one or more processors. The memory is configured to store instructions. The one or more processors are configured to execute the instructions to apply a neural network to a first image to generate an enhanced image. The one or more processors are also configured to execute the instructions to adjust at least a portion of a high-frequency component of the enhanced image based on a motion compensation operation to generate an adjusted high-frequency image component. The one or more processors are further configured to execute the instructions to combine a low-frequency component of the enhanced image and the adjusted high-frequency image component to generate an adjusted enhanced image.