Super Resolution Neural Network Backbone for Multi-Scale In-Loop Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding techniques require extensive memory and processing power for neural network-based in-loop filtering, especially when multiple filters are needed for different resolutions, leading to inefficiencies and unsatisfactory resolution modification options.
Innovation Solution
A super resolution neural network (SRNN) architecture with a common SR backbone and multiple output layers, utilizing scale factor information to generate optimized intermediate filtered data for different target resolutions, reducing memory and processing requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple neural network filters are implemented for different resolutions, then resolution modification capability is improved, but memory requirements and processing complexity increase
Solution Approach 1:
The patent applies a single universal neural network filter architecture that can handle multiple resolution modifications by accepting scale factor information as input. This multi-functional approach eliminates the need for separate dedicated filters for different resolutions, thereby reducing memory requirements and processing complexity while maintaining versatile resolution modification capability.
Solution Approach 2:
The patent changes the parameter input to the neural network filter by incorporating scale factor information alongside standard filter inputs. This parameter change enables the same filter architecture to adapt to different resolution requirements dynamically, resolving the contradiction between versatility and complexity by using parameter variation rather than architectural duplication.
2Adaptability or versatility
If multiple neural network filters are implemented for different resolutions, then resolution modification capability is improved, but training time increases
Solution Approach 1:
The patent trains a single universal neural network filter that can perform multiple resolution modifications, eliminating the need to train and maintain multiple separate filters. This universal approach significantly reduces total training time while preserving the ability to handle various resolution requirements through the same trained model.
Solution Approach 2:
The patent merges multiple resolution-specific filter training tasks into a single unified training process. By combining the training of multiple dedicated filters into one training session for a universal filter, the patent reduces cumulative training time while achieving the same versatile resolution modification capability.
3Manufacturing precision
If neural network-based in-loop filtering is applied, then visual quality is improved, but processing power requirements increase
Solution Approach 1:
The patent modifies the input parameters to the neural network filter by adding scale factor information, which enables the filter to achieve high visual quality across different resolutions without requiring proportional increases in processing power. The scale factor parameter allows the same computational resources to be efficiently utilized for various resolution scenarios.
Data Source
AI summary
A method of processing video data includes receiving, with a headblock of a super resolution (SR) backbone, scale factor information for modifying a resolution of a current block of the video data; filtering, based on the scale factor information, the current block with the SR backbone to generate intermediate filtered data, the SR backbone implementing a neural network (NN)-filter that utilizes the scale factor information; and generating a super resolution current block based on the intermediate filtered data, the super resolution current block having a resolution different than the resolution of the current block.