Video Codec In-Loop Filtering With Reduced NN Input Complexity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high complexity of neural network-based (NN) models for in-loop filtering in video codecs poses a major challenge for practical hardware implementations, and the structure of existing NN models is not optimal, leading to inefficiencies in video compression.
Innovation Solution
A method involving machine learning (ML) models that generate encoded or decoded video data by utilizing reconstructed and predicted sample values, along with quantization parameters and block boundary strength information, while omitting certain input data such as partition information, to reduce complexity without compromising performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network-based models are used for in-loop filtering in video codecs, then filtering quality is improved, but device complexity increases significantly
Solution Approach 1:
The patent extracts and removes partition information from the neural network input data. By taking out this specific type of information that contributes to complexity without providing proportional filtering quality improvement, the model complexity is reduced while maintaining essential filtering performance.
Solution Approach 2:
The patent changes the input parameters of the neural network by selectively including or excluding certain types of data (partition information). This parameter modification approach allows the model to achieve comparable filtering quality with reduced computational complexity and smaller model size.
2Measurement precision
If complete input information including partition data is provided to NN models, then filtering accuracy is improved, but processing time and computational load increase
Solution Approach 1:
The patent removes partition information from the input data stream, extracting only the essential filtering-related parameters. This reduction in input data volume decreases the computational load and processing time required by the neural network while retaining sufficient information for accurate filtering.
Solution Approach 2:
The patent applies partial action by providing only a subset of available information (excluding partition data) to the neural network. This partial input approach is sufficient to achieve the required filtering accuracy without the need to process all possible input parameters, thereby reducing computational overhead.
Data Source
AI summary
There is provided a method of selecting from a picture a patch for training a machine learning, ML, model used for encoding or decoding video data. The method comprises randomly selecting one or more coordinates of the patch, and converting said one or more coordinates of the patch into converted one or more coordinates of the patch. The method further comprises training the ML model based on the converted one or more coordinates of the patch. Each of said one or more converted coordinates of the patch is an integer multiple of 2p, where p is an integer.