Unified Neural Network In-Loop Filter for Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies face challenges in efficiently reducing distortion during compression, particularly due to the lack of effective in-loop filtering methods that can adapt to varying quality levels and coding characteristics.
Innovation Solution
The development of a unified neural network (NN) filter model that uses a quality-level indicator (QI) as input to generate filtered samples, allowing for different NN filter models to process samples with unique characteristics and constructing a candidate list based on coding statistics to optimize filtering for each video unit.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple separate NN filter models are used for different quality levels, then filtering accuracy for specific quality levels is improved, but device complexity increases
Solution Approach 1:
The patent combines multiple quality-level-specific filter models into a single unified NN filter model by integrating quality-level indicators as additional input channels. This allows the model to adapt to different quality levels without requiring separate models, thereby reducing device complexity while maintaining filtering accuracy.
Solution Approach 2:
The unified NN filter model is designed to handle multiple quality levels through a single multi-functional architecture. By incorporating quality-level indicators as input, the model can dynamically adjust its filtering behavior to suit different quality requirements, eliminating the need for multiple specialized models.
2Device complexity
If a unified NN filter model is used for all quality levels, then device complexity is reduced, but adaptability to varying quality levels deteriorates
Solution Approach 1:
The unified NN filter model incorporates dynamic adaptability by treating quality-level indicators as variable inputs. The model can dynamically adjust its filtering operations based on the input quality-level indicator, allowing it to adapt to varying quality levels without requiring separate static models for each level.
Solution Approach 2:
The patent adds a new dimension to the input space by incorporating quality-level indicators as additional input channels. This dimensional expansion allows the unified model to distinguish between different quality levels and adapt its filtering behavior accordingly, maintaining versatility while using a single model.
3Measurement precision
If quality-level indicators are processed through fully-connected layers to generate scaling factors, then feature recalibration accuracy is improved, but computational complexity increases
Solution Approach 1:
Instead of processing all feature maps through complex recalibration, the patent applies scaling factors selectively to channel dimensions. This partial action approach achieves effective feature recalibration by focusing computational resources on the most critical aspects (channel-wise scaling) while avoiding excessive computation on spatial dimensions.
Data Source
AI summary
A method implemented by a video coding apparatus includes applying a neural network (NN) filter to an unfiltered sample of a video unit to generate a filtered sample, wherein the NN filter is based on an NN filter model generated using a quality-level indicator (QI) input. The method also includes converting between a video media file and a bitstream based on the filtered sample that was generated.