Adaptive In-Loop Filter Retraining for Variable Video Noise
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video compression techniques face challenges in maintaining coding efficiency and image quality due to varying image qualities and characteristics, as in-loop filters trained under similar image conditions perform poorly when image qualities differ.
Innovation Solution
Retraining a deep learning-based in-loop filter on the decoder side using video samples selected during decoding to adapt to varying image qualities and characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a pre-trained in-loop filter is used for video decoding, then coding efficiency is improved, but the filter performance deteriorates when image quality differs from training conditions
Solution Approach 1:
The in-loop filter transitions from a static pre-trained model to a dynamic retrainable model. The filter parameters are updated during video decoding based on actual image characteristics, allowing the filter to adapt to varying image quality and content while maintaining high coding efficiency. This dynamic adjustment resolves the contradiction between using a fixed efficient filter and needing to adapt to different conditions.
Solution Approach 2:
The filter parameters are changed during operation through retraining on actual video data. Instead of using fixed pre-trained parameters, the system updates the filter parameters based on the specific video content and quality characteristics, enabling the filter to maintain optimal performance across diverse image conditions while preserving coding efficiency.
2Reliability
If general supervised learning is applied to noise removal model, then compression noise removal performance is improved, but the model performance decreases when training and verification image qualities differ
Solution Approach 1:
The system implements feedback by using the actual decoded video data to retrain the noise removal model. The model receives feedback from the real video content and adjusts its parameters accordingly, enabling it to adapt to different image qualities and maintain high noise removal performance across various conditions.
Solution Approach 2:
The noise removal model performs self-service by retraining itself using the actual video data during decoding. Instead of relying on pre-trained models that may not match the specific video content, the model automatically adapts to the actual image characteristics, improving both noise removal performance and adaptability.
3Adaptability or versatility
If in-loop filter is retrained on decoder side using video samples, then adaptability to varying image qualities is improved, but processing time increases
Solution Approach 1:
The system applies partial retraining by selecting only the most relevant video samples and parameters for retraining, rather than processing the entire video stream. This partial action approach enables the filter to adapt to varying image qualities while minimizing the processing time required for retraining.
Solution Approach 2:
The system performs preliminary actions by pre-selecting and pre-processing video samples before retraining the in-loop filter. This preliminary preparation reduces the computational burden during the actual retraining process, enabling adaptability to varying image qualities with acceptable processing time.
Data Source
AI summary
A video coding method and device using an in-loop filter adaptive to various types of noise and characteristics. The video decoding device generates an output block by inputting a reconstruction block for a current block to a deep learning-based in-loop filter. The video decoding device selects a block for retaining the in-loop filter from the reconstruction block and retrains the in-loop filter using the selected reconstruction block and a target block.