Neural Network Video Filter Adaptive Offset Mask
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video encoding and decoding methods face challenges in efficiently reducing coding artifacts and improving compression efficiency due to limitations in existing filters, such as SAO and ALF, which require complex neural network training and significant signaling overhead.
Innovation Solution
The proposed solution involves using a neural network-based filter that adjusts reconstructed samples with adaptive offsets, generating a weight mask to determine whether a sample is corrected and the strength of the correction, with only a single control parameter encoded in the bitstream, thereby improving filtering efficiency and reducing signaling overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If existing filters (SAO, ALF) are used to reduce coding artifacts, then filtering capability is provided, but signaling overhead and training complexity increase
Solution Approach 1:
The patent extracts the complex filtering operations from traditional SAO and ALF filters and replaces them with a simplified neural network-based filter. The neural network is trained offline to learn optimal filtering operations, and only the trained model parameters are deployed in the encoder/decoder, removing the need for complex runtime filtering logic and reducing signaling overhead.
Solution Approach 2:
The patent changes the approach from using multiple complex filter parameters that require signaling to using a pre-trained neural network model with fixed parameters. The neural network takes reconstructed samples as input and directly outputs filtered samples, eliminating the need for complex parameter signaling and reducing filtering complexity while maintaining artifact reduction capability.
2Manufacturing precision
If complex neural network training is performed to improve filtering, then filtering accuracy improves, but training complexity and computational overhead increase
Solution Approach 1:
The patent performs the complex neural network training operation in advance (offline) using ground truth data and evaluation metrics. The trained model parameters are then fixed and deployed in the encoder/decoder system. This preliminary training action separates the complex training process from the runtime encoding/decoding process, achieving high filtering accuracy without adding training complexity to the operational system.
3Measurement precision
If sample-wise adjustment with multiple parameters is used, then filtering precision improves, but signaling overhead increases
Solution Approach 1:
The patent uses the neural network to learn optimal filtering patterns from training data, effectively copying the best filtering strategies into the model parameters. During runtime, the pre-trained network applies these learned patterns directly to reconstructed samples without requiring additional parameter signaling, achieving sample-wise filtering precision with minimal overhead.
Data Source
AI summary
In one implementation, to perform in-loop filtering of a version of reconstructed samples of a block, only a single offset parameter is signaled in the bitstream. Based on the version of reconstructed samples, a pixel-wise weight mask is generated using a neural network. Because the neural network parameters are known at both the encoder and decoder, these parameters need not to be signaled in the bitstream. The single offset parameter scaled by the weighted mask is used to adjust the samples in the block. Thus, even though only a single offset parameter is used, the samples are adjusted by pixel-wise offsets. The neural network may also take other parameters, such as quantization parameters and picture types as input. Further, there can be multiple neural networks that generate different weight masks, where different offsets are signaled and one or more of the neural networks are to be selected for filtering.