Super-Resolution Video Processing via RNN-CNN Temporal Sequencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current super-resolution processing methods for moving images using convolutional neural networks (CNNs) often result in flickering and discontinuity artifacts due to independent processing of frames, which affects the continuity and quality of the upsampled image.
Innovation Solution
A method involving the sequential input of frames into a recurrent neural network (RNN) and CNN for super-resolution processing, followed by deconvolution to upscale the resolution, which helps maintain visual-temporal dependency and reduce artifacts by reflecting information from previous frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If super-resolution processing is performed independently on each frame using CNN, then processing speed is maintained, but flickering and discontinuity artifacts occur
Solution Approach 1:
The patent combines multiple consecutive frames as input to the CNN, merging temporal information across frames. This allows the network to process frames in a sequence-dependent manner, reducing flickering and discontinuity artifacts while maintaining processing efficiency through batch operations.
Solution Approach 2:
The patent introduces temporal dynamics by processing frames in sequences rather than independently. The CNN receives multiple frames at different time points, enabling the system to adapt processing based on temporal variations and maintain visual continuity across the video sequence.
2Reliability
If multiple frames are processed sequentially to maintain temporal dependency, then visual continuity is improved, but processing time increases
Solution Approach 1:
The patent processes a limited number of consecutive frames (e.g., 3-5 frames) rather than all frames in the video sequence. This partial processing approach captures sufficient temporal dependency to reduce flickering while avoiding the excessive processing time that would result from analyzing the entire video sequence.
Solution Approach 2:
The video sequence is divided into segments of consecutive frames, where each segment is processed independently by the CNN. This segmentation allows parallel processing of multiple segments, reducing overall processing time while maintaining temporal continuity within each segment.
3Reliability
If frame sequences are used as input, then temporal dependency is captured, but computational complexity increases
Solution Approach 1:
The patent applies different processing strategies to different parts of the frame sequence. Early frames in the sequence receive full CNN processing, while later frames may use simplified processing or reference previous frame outputs, reducing overall computational complexity while maintaining temporal dependency.
Data Source
AI summary
A super-resolution processing method of a moving image is provided. The super-resolution processing method of a moving image includes sequentially inputting a plurality of input frames included in the video to any one of a recurrent neural network (RNN) for super-resolution processing and a convolutional neural network (CNN) for super-resolution processing, sequentially inputting a frame sequentially output from the any one of the RNN and the CNN to an additional one of the RNN and the CNN, and upscaling a resolution of the output frame by carrying out deconvolution with respect to a frame sequentially output from the additional one of the RNN and the CNN.