Super Resolution Upsampling for Video Coding Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding techniques do not optimally utilize available information such as prediction, partition, and intra prediction mode, and perform up-sampling either at the frame-level or block-level, which is not always optimal due to varied video content, leading to suboptimal video coding processes.
Innovation Solution
Applying a super resolution process to a video unit at a level of an SR unit, where the SR unit includes more than one pixel, and performing conversions between video and bitstream, with the SR unit potentially changing levels within a sequence of frames or pictures based on content, using neural networks for up-sampling, and combining frame-level and block-level up-sampling for improved video coding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If up-sampling is performed at frame-level or block-level using conventional techniques, then the video coding process is simplified, but the video coding efficiency and quality are suboptimal
Solution Approach 1:
The patent applies dynamic selection of up-sampling levels (frame-level or block-level) based on video content characteristics. The system adaptively determines whether to perform super-resolution at the frame level or block level by analyzing content features, allowing the processing granularity to change dynamically according to the specific video sequence requirements, thereby optimizing coding efficiency without excessive complexity
Solution Approach 2:
The patent applies different up-sampling strategies to different regions or blocks of video content. By performing block-level super-resolution only where needed (in regions requiring higher quality) while using frame-level or other methods in other regions, the system achieves local optimization of video quality and coding efficiency without uniformly increasing complexity across the entire video stream
2Adaptability or versatility
If fixed up-sampling level (frame-level or block-level) is used, then the processing is simpler, but it cannot adapt to varied video content
Solution Approach 1:
The system dynamically switches between frame-level and block-level SR units based on video content analysis. The patent implements content-aware determination of SR unit levels, where the processing granularity adapts in real-time according to the specific characteristics of each video sequence or region, achieving high adaptability without requiring complex manual configuration
Solution Approach 2:
The patent creates a universal up-sampling framework that can handle both frame-level and block-level operations within a single system. The super-resolution process is designed to be multi-functional, capable of operating at different hierarchical levels (frame or block) depending on content requirements, thereby achieving versatility without proportionally increasing system complexity
3Loss of information
If conventional video coding techniques are used, then the processing is faster, but additional information (prediction, partition, intra prediction mode) is not optimally utilized
Solution Approach 1:
The patent performs super-resolution up-sampling before certain video coding operations to enhance the quality of subsequent processing steps. By preliminarily reconstructing high-resolution video content using available prediction and partition information, the system enables better utilization of these features in later encoding stages, reducing information loss without significantly impacting overall coding speed
Data Source
AI summary
A method of processing video data. The method includes applying a super resolution (SR) process to a video unit at a level of an SR unit, where the SR unit includes more than one pixel of the video unit, and performing a conversion between a video comprising the video unit and a bitstream of the video based on the SR process as applied. A corresponding video coding apparatus and non-transitory computer-readable recording medium are also disclosed.