Video Frame Super-Resolution Using Quality-Scored Reference Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video super-resolution methods result in varying frame quality due to differing compression levels, affecting the super-resolution effect in actual streaming media services.
Innovation Solution
An image frame super-resolution method that utilizes super-resolution reference information, including quantization parameters and image quality scores, to select and process multiple frames through a trained video super-resolution network, enhancing the super-resolution processing efficiency and quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multiple consecutive frames are used for super-resolution processing, then the super-resolution effect is improved, but the processing resource consumption and computational load increase
Solution Approach 1:
The transmit end performs preliminary actions by pre-processing video frames before transmission. Specifically, it performs mode selection to determine reference frames, calculates quantization parameters for each frame, and pre-evaluates image quality scores. This preparation work is done in advance so that the terminal receives organized super-resolution reference information without needing to perform heavy computational tasks, thus improving super-resolution effect while reducing terminal processing resources
Solution Approach 2:
The patent introduces an intermediary mechanism by transmitting structured super-resolution reference information from the transmit end to the terminal. This intermediary data structure includes pre-selected reference frames, quantization parameters, and image quality scores, which mediate between the complex multi-frame processing at the transmit end and the simpler processing at the terminal, achieving both high resolution and low computational load
2Productivity
If frame compression is applied in streaming media service, then transmission efficiency is improved, but the quality difference between adjacent frames increases and affects super-resolution effect
Solution Approach 1:
The patent implements feedback by having the transmit end evaluate and calculate image quality scores for each video frame based on its quantization parameter and compression level. This quality information is fed back to the terminal along with the compressed video data, allowing the terminal to make informed decisions about super-resolution processing by selecting reference frames with appropriate quality levels, thus maintaining good super-resolution effect despite compression
Solution Approach 2:
The patent utilizes parameter changes by dynamically adjusting the selection of reference frames and super-resolution processing based on quantization parameters and image quality scores. Different frames with different compression levels (parameter variations) are selectively used as references depending on their quality scores, allowing the system to adapt to varying compression conditions and maintain optimal super-resolution effect across different transmission efficiency requirements
Data Source
AI summary
An image frame super-resolution implementation method and apparatus are disclosed. The method includes: obtaining super-resolution reference information including a quantization parameter and an image quality score set, and the image quality score set includes image quality scores of a plurality of image frames; selecting M image frames from the plurality of image frames based on the image quality score set, where Mis greater than or equal to 1; obtaining a video super-resolution network corresponding to the quantization parameter, where the video super-resolution network has a super-resolution function; and inputting the M image frames and a first image frame into the video super-resolution network, where the video super-resolution network is configured to perform super-resolution processing via the super-resolution function, on the first image frame based on the M image frames to obtain a second image frame, and the resolution of the second image frame is higher than the resolution of the first image frame.