Neural Network Loop Filtering With Residual Offset for Video Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital video compression technologies struggle to efficiently reduce bandwidth and traffic pressure in video transmission, despite advancements in standards like H.264/AVC, H.265/HEVC, and H.266/VVC, necessitating improved methods for enhancing video encoding and decoding processes.
Innovation Solution
Implementing neural network-based loop filtering (NNLF) with a skip connection branch and residual offset adjustment at both encoding and decoding sides, utilizing a neural network to optimize filtering modes based on rate distortion costs, and encoding residual offset usage flags to enhance encoding performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network based loop filtering is applied to improve video quality, then manufacturing precision is improved, but device complexity increases
Solution Approach 1:
The filter is divided into multiple independent components: a neural network module for residual prediction, a skip connection branch for preserving original information, and a residual offset module for fine-tuning. Each component performs a specific function, allowing the complex filtering task to be broken down into manageable segments that can be processed independently.
Solution Approach 2:
A residual offset module is introduced as an intermediary between the neural network output and the final filtered result. This mediator adjusts the residual values predicted by the neural network to better match the actual residuals, improving accuracy without requiring the neural network itself to be more complex.
2Manufacturing precision
If residual offset is performed to improve filtering accuracy, then manufacturing precision is improved, but use of energy increases
Solution Approach 1:
The residual offset module dynamically adjusts residual values based on local picture characteristics and rate-distortion considerations. By changing the residual parameters adaptively rather than using fixed values, the system achieves higher accuracy while maintaining reasonable computational energy consumption through selective application.
Solution Approach 2:
Residual offset is applied selectively rather than uniformly across all picture regions. The system performs partial action by applying offset only where it provides significant benefit, determined by rate-distortion optimization, avoiding excessive computational energy expenditure in regions where simple filtering suffices.
3Manufacturing precision
If mode selection based on rate distortion cost is implemented to improve video quality, then manufacturing precision is improved, but productivity decreases
Solution Approach 1:
The system changes key parameters including the neural network architecture, skip connection configuration, and residual offset values to optimize for different picture types and content characteristics. This allows the encoder to adapt to varying requirements without always performing exhaustive mode selections, improving both quality and speed.
Solution Approach 2:
The neural network is pre-trained on large datasets to learn optimal filtering patterns. During actual encoding, the pre-trained network provides high-quality results with minimal additional computation, as the heavy optimization work was performed in advance during training rather than during real-time encoding.
Data Source
AI summary
A video decoding method, includes: performing following processes when neural network based loop filtering (NNLF) is performed on a reconstructed picture: responsive to that NNLF enables residual offset, performing NNLF on the reconstructed picture; where performing NNLF on the reconstructed picture includes: decoding a residual offset usage flag roflag of the reconstructed picture, where the roflag is used to indicate whether residual offset needs to be performed when NNLF is performed on the reconstructed picture through a filter for NNLF; and performing NNLF on the reconstructed picture using a first mode in response to determining, according to the roflag, that residual offset does not need to be performed, or performing NNLF on the reconstructed picture using a second mode in response to determining, according to the roflag, that residual offset needs to be performed.