Video Decoder Loop Filter Bypass for Drift-Free Neural Post-Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network-based loop filters, while offering improved compression efficiency, are not feasible for all devices due to high computational requirements, leading to potential drift issues when disabled in less capable decoders.
Innovation Solution
Utilize post-reconstruction samples that have not been loop-filtered to apply neural network-based post-filters, ensuring compatibility across devices without causing prediction drift.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural network-based loop filters are applied in the decoding loop, then compression efficiency is improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent segments the filtering process by separating loop filtering from post-filtering. Loop filters (deblocking, SAO, ALF) are applied within the decoding loop for compression efficiency, while neural network-based post-filtering is applied outside the loop. This segmentation allows capable devices to apply both for maximum compression efficiency, while less capable devices can skip the post-filtering stage without affecting decoder compatibility.
Solution Approach 2:
The patent introduces an intermediary post-filtering stage that processes the output of the loop-filtered decoded picture buffer. This intermediary layer acts as a bridge between the constrained decoding loop and the final output, allowing neural network-based filtering to enhance quality without disrupting the internal decoding loop operations that maintain compatibility across all devices.
2Device complexity
If neural network-based loop filters are disabled in less capable decoders, then device complexity is reduced, but prediction drift issues occur
Solution Approach 1:
By segmenting the filtering into loop filtering (mandatory for all decoders) and post-filtering (optional for capable devices), the patent ensures that all decoders maintain consistent prediction behavior through the loop filter stage. Less capable decoders can skip the computationally intensive post-filtering while still producing compatible predictions, eliminating drift issues.
Solution Approach 2:
The loop filtering is performed as a preliminary action within the decoding loop that all decoders must execute. This preliminary filtering establishes a consistent baseline that prevents prediction drift, while the optional post-filtering is applied afterward as a quality enhancement that does not affect prediction consistency across different decoder capabilities.
3Device complexity
If post-filtering is applied to loop-filtered samples, then device complexity is reduced, but picture fidelity decreases
Solution Approach 1:
Instead of applying post-filtering to the already loop-filtered samples (which would limit quality improvement), the patent inverts the approach by making loop-filtered samples available as a separate output path. This allows post-filtering to be applied to the original post-reconstruction samples before loop filtering, or to combine both filtered and unfiltered paths, thereby maximizing picture fidelity while maintaining implementation flexibility.
Data Source
AI summary
A method performed by an apparatus comprising a decoder, where the method includes deriving a first set of post-reconstruction samples from a bitstream. The method also includes outputting (e.g., sending out or making available for retrieval) the first set of post-reconstruction samples (e.g., outputting a copy of the first set of post-reconstruction samples to a post-filter). The method also includes filtering the first set post-reconstruction samples using one or more loop filters to produce last loop filter output samples, wherein the last loop filter output samples are not identical to the first set of post-reconstruction samples. The method also include storing the last loop filter output samples in a decoded picture buffer.