Video Frame Resolution Enhancement via Adaptive Neural Network Complexity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video frame resolution enhancement methods require excessive computations due to the use of a single neural network for all frames, leading to inefficient processing and unnecessary power waste, as they do not adapt to varying frame content or user interest.
Innovation Solution
A method that selects and applies neural networks of different complexities based on the scene change rate and type of video frames, using both single image super resolution and multiple images super resolution models to optimize processing efficiency and quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the same type of CNN is applied to all frames for video super resolution, then manufacturing precision (image quality) is improved, but productivity (processing speed) deteriorates due to excessive computations
Solution Approach 1:
The patent applies different complexity levels of CNN models dynamically based on frame characteristics. Frames are classified into different types (e.g., key frames, non-key frames) and appropriate CNN complexity levels are selected for each type, making the processing system adaptive rather than static.
Solution Approach 2:
Different parts of the video (different frames) are processed with different levels of complexity according to their specific needs. Key frames receive higher complexity processing while non-key frames use lower complexity models, optimizing the overall balance between quality and speed.
2Manufacturing precision
If high complexity neural networks are applied to all frames regardless of user interest, then manufacturing precision (image quality) is improved, but use of energy (computing power) increases unnecessarily
Solution Approach 1:
The patent identifies regions or frames of user interest and applies high complexity processing only to those specific areas. Other frames or regions receive lower complexity processing, thereby reducing overall energy consumption while maintaining quality where needed.
Solution Approach 2:
Instead of applying full complexity processing to all frames, the patent uses partial processing (lower complexity models) for frames that do not require high quality enhancement, reserving excessive action (high complexity models) only for critical frames.
3Ease of operation
If the same frame resolution enhancement method is used for all video frames, then ease of operation (processing simplicity) is improved, but manufacturing precision (image quality) deteriorates because user interest is not considered
Solution Approach 1:
The system dynamically adjusts processing methods based on frame analysis and user interest detection. The processing pipeline automatically adapts its complexity and approach for different frames, maintaining simplicity in operation while achieving variable quality outcomes.
Data Source
AI summary
A method for enhancing video frame resolution according to one embodiment of the present disclosure may include loading video data including a plurality of frames having low resolution; selecting, from the group of artificial neural networks for image processing, artificial neural networks for image processing having different complexity to apply to two different frames of a video; and generating a high resolution frame by processing each frame of the video according to the selected artificial neural networks for image processing. A neural network for image processing according to one embodiment of the present disclosure may be a deep neural network generated via machine learning, and an input and output of the video may take place in an Internet of Things environment using a 5G network.