Region-Adaptive Super Resolution for Video Coding Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies apply uniform up-sampling methods to all frames, which can be inefficient and suboptimal for different video content characteristics, leading to potential compression inefficiencies.
Innovation Solution
Applying different super resolution (SR) processes to different sub-regions of a video unit, including both neural network (NN)-based and non-NN-based methods, allowing for tailored up-sampling based on specific characteristics such as color components, slice types, and quantization parameters, with indications signaled in the bitstream or derived dynamically.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If uniform up-sampling methods are applied to all frames, then device complexity is reduced and ease of operation is improved, but video coding efficiency deteriorates and compression performance is suboptimal
Solution Approach 1:
The video unit is divided into multiple sub-regions, and different super resolution processes are applied to different sub-regions based on their characteristics. This segmentation allows the system to optimize coding efficiency for each region while managing complexity through selective processing.
Solution Approach 2:
Different quality levels and processing methods are applied to different sub-regions of the video unit based on local characteristics such as color components, slice types, and quantization parameters. This ensures that regions requiring higher quality receive more sophisticated processing while simpler regions use efficient but lighter processing.
2Manufacturing precision
If different super resolution processes are applied to different sub-regions, then video coding efficiency is improved and compression performance is enhanced, but device complexity increases and processing becomes more complex
Solution Approach 1:
The system dynamically selects and applies different super resolution processes to different sub-regions based on local characteristics such as color components, slice types, and quantization parameters. This dynamic adaptation allows the system to achieve high quality where needed while managing complexity through context-aware processing decisions.
Solution Approach 2:
The patent changes processing parameters including the type of super resolution process, color component handling, slice type considerations, and quantization parameters across different sub-regions. These parameter changes enable tailored up-sampling that optimizes quality for each region's specific characteristics.
3Manufacturing precision
If neural network-based SR processes are applied, then up-sampling quality is improved, but computational energy consumption and processing time increase
Solution Approach 1:
Neural network-based super resolution processes are selectively applied to specific sub-regions where they provide the most benefit, rather than uniformly across the entire video unit. This local application reduces overall computational energy consumption while maintaining high quality where it matters most.
Solution Approach 2:
The system applies neural network-based processing partially, only to sub-regions that benefit most from it, rather than excessively applying it to all regions. This partial action approach balances quality improvement with computational energy consumption.
Data Source
AI summary
A method of processing video data. The method includes applying different super resolution (SR) processes to different sub-regions of a video unit, and performing a conversion between a video including the different regions of the video unit and a bitstream of the video based on the different SR processes as applied. A corresponding video coding apparatus and non-transitory computer-readable recording medium are also disclosed.