ML Video In-Loop Filtering With Reduced Model Complexity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high complexity of neural network-based in-loop filters in video codecs, such as those described in JVET-X0066 and JVET-Y0143, poses challenges for practical hardware implementations, necessitating a reduction in complexity while maintaining or improving performance.
Innovation Solution
The proposed methods involve generating encoded or decoded video data using machine learning models that utilize reconstructed and predicted sample values, block boundary strength, and quantization parameters, without requiring partition information, and employ attention residual blocks to reduce complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network-based in-loop filters are used in video codecs, then compression efficiency and quality are improved, but device complexity increases
Solution Approach 1:
The patent extracts and removes partition information from the neural network input, keeping only the most essential features (reconstructed samples, predicted samples, boundary strength, quantization parameters). This extraction principle reduces the input data dimensionality and model complexity while preserving the critical information needed for effective filtering and compression.
Solution Approach 2:
The patent applies different filtering strategies to different regions of the video data by using boundary strength information to identify block boundaries. The neural network applies stronger filtering locally at boundaries where artifacts are more prominent, while using lighter filtering in smooth regions, thereby improving overall quality without uniformly increasing complexity across the entire video stream.
2Measurement precision
If partition information is included in neural network input, then filtering accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent explicitly removes partition information from the neural network input data, extracting only the essential features needed for filtering. This decision demonstrates the extraction principle by eliminating redundant information (partition details) that would increase processing complexity without providing sufficient additional benefit to filtering accuracy.
Solution Approach 2:
Instead of including partition information to improve filtering accuracy, the patent inverts the approach by excluding it and relying on alternative features like boundary strength and quantization parameters. This inversion shows that accurate filtering can be achieved without explicitly processing partition data, thereby reducing complexity while maintaining effectiveness.
Data Source
AI summary
There is provided a method of training a machine learning, ML, model used for generating encoded video data or decoded video data. The method comprises obtaining original video data. The method further comprises converting the original video data into ML input video data. The method further comprises providing the ML input video data into the ML model, thereby generating first ML output video data, and training the ML model based on a difference between the original video data and the ML input video data and a difference between the original video data and the first ML output video data.


