CNN Image Processing for Flicker-Free Video Super-Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Flickering occurs in high-resolution images generated from low-resolution images due to inconsistent patterns in frames, and existing methods for motion estimation and compensation are complex and ineffective.
Innovation Solution
An image processing apparatus using a convolutional neural network to generate high-resolution images by extracting feature information from low-resolution images and removing flickering by utilizing previous frames' high-resolution images, thereby reinforcing frame correlation and skipping complicated motion estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If interframe motion estimation and compensation is used to remove flickering, then flickering removal effect is improved, but processing complexity increases and correct motion estimation becomes difficult
Solution Approach 1:
The patent extracts only the necessary temporal correlation information from previous frames through simple subtraction operations, rather than performing complete motion estimation and compensation. This extracts the essential flickering removal capability while avoiding the complexity of full motion analysis
Solution Approach 2:
Instead of estimating motion to compensate for flickering, the patent inverts the approach by directly subtracting previous frame information from current frame to eliminate flickering patterns, achieving flicker removal without traditional motion estimation
2Productivity
If high-resolution image of current frame is not used in next frame generation, then processing speed is improved, but correlation between frames is weakened
Solution Approach 1:
The patent performs preliminary subtraction of previous frame information from the current frame to remove flickering before generating the final high-resolution image. This preliminary action ensures temporal correlation is established early, improving both frame correlation and subsequent processing efficiency
Solution Approach 2:
The patent maintains continuous temporal correlation by using the generated high-resolution image of the current frame (after flicker removal) as input for the next frame's flicker removal process, creating a continuous chain of useful information flow across frames
3Reliability
If complete high-resolution images are stored for flicker removal, then flickering removal accuracy is improved, but memory usage increases
Solution Approach 1:
The patent extracts only the essential feature information from low-resolution images rather than storing complete high-resolution images. This extraction provides sufficient data for accurate flicker removal while dramatically reducing memory requirements
Solution Approach 2:
The patent uses lightweight feature representations instead of heavy full-resolution images, creating a more efficient, resource-light solution that achieves the same flicker removal functionality with minimal memory overhead
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided is an image processing apparatus for generating a high-resolution image. The image processing apparatus includes a memory storing one or more instructions and a processor configured to execute the one or more instructions stored in the memory, wherein the processor is configured to extract feature information regarding a low-resolution image of a current frame by using a first convolutional neural network, generate, based on the feature information, a first high-resolution image of the current frame, remove flickering of the first high-resolution image by using a high-resolution image of a previous frame, and remove flickering of a high-resolution image of a next frame by using at least one of a flickering-removed second high-resolution image of the current frame, or the feature information.