Combined Deblocking and ML Filtering for Video Artifact Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine Learning (ML)-based filtering in video encoding and decoding fails to sufficiently remove blocking artifacts, leading to residual artifacts in video frames, and existing deblocking processes are inadequate when ML-based filtering is not used for certain Coding Tree Units (CTUs) or frames.
Innovation Solution
A method combining deblocking filtering with ML-based filtering and adaptive loop filtering to enhance video frames, including a deblocking filter and a machine-learning-based filter to reduce blocking artifacts and improve subjective quality while reducing bitrate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If ML-based filtering is used alone, then filtering operation is simplified, but blocking artifacts are not sufficiently removed
Solution Approach 1:
The patent combines deblocking filtering and ML-based filtering into a unified filtering stage. The deblocking filter processes block boundaries first, then ML-based filtering is applied to the output to remove residual artifacts, achieving both simplification and high quality artifact removal.
2Manufacturing precision
If deblocking filtering is applied strongly, then blocking artifacts are removed effectively, but natural structures may be removed
Solution Approach 1:
The deblocking filter is applied selectively at block boundaries where discontinuities occur, rather than uniformly across the entire image. This localized approach targets blocking artifacts specifically while preserving natural structures in other regions.
Solution Approach 2:
The patent uses a two-stage filtering approach where deblocking filtering is applied first, then ML-based filtering processes the output to remove residual artifacts. This feedback mechanism allows the second filter to address remaining issues without reprocessing already-corrected areas.
3Manufacturing precision
If separate deblocking and ML-based filtering stages are used, then artifact removal quality is improved, but processing complexity increases
Solution Approach 1:
The patent merges deblocking filtering and ML-based filtering into a single unified filtering stage in the decoding process. This integration maintains the benefits of both filters while reducing overall system complexity compared to separate processing stages.
4Productivity
If ML-based filtering replaces deblocking filtering, then processing speed is improved, but blocking artifacts remain
Solution Approach 1:
The deblocking filter is applied as a preliminary processing step before ML-based filtering. This preliminary action removes the majority of blocking artifacts efficiently, allowing the subsequent ML-based filter to focus on residual artifacts and operate more effectively.
Data Source
AI summary
There is provided a method. The method comprises obtaining an input video frame data associated with an input video frame. The method comprises performing a deblocking filtering operation on one or more samples included in the input video frame, thereby generating one or more deblocked samples. The method further comprises performing a machine-learning (ML)-based filtering operation and/or adaptive loop filtering operation on one or more samples included in the input video frame, thereby generating one or more filtered samples. The method comprises, using said one or more deblocked samples and/or said one or more filtered samples, producing encoded or decoded video frame data including an encoded or decoded video frame.