Deep Learning Loop Filtering with Quantization-Aware Image Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding and decoding technologies face challenges in improving the filtering effect of loop filtering to enhance coding and decoding efficiency.
Innovation Solution
A deep learning-based filtering method that utilizes fine-grained quantization parameters, such as frame-level, slice-level, and block-level quantization parameters, to generate an input for a deep learning filter through pixel-wise multiplications and convolutional operations, enhancing the filtering effect.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional loop filtering is used on reconstructed images, then the filtering process can be performed, but the filtering effect is insufficient and coding efficiency is limited
Solution Approach 1:
The patent replaces traditional mechanical filtering algorithms (deblocking filter, SAO filter) with a deep learning-based neural network filter. The neural network learns optimal filtering operations from training data, substituting hand-crafted filtering rules with data-driven intelligent filtering that adapts to different image characteristics, thereby improving filtering effectiveness while maintaining coding efficiency.
Solution Approach 2:
The patent incorporates quantization parameters (QP) as additional input features to the deep learning filter. By changing the input parameters to include QP values that reflect the compression level and image characteristics, the filter can adapt its filtering strength and behavior dynamically, improving filtering precision without requiring complex multi-parameter control systems.
2Manufacturing precision
If deep learning filter is applied to improve filtering effect, then recognition accuracy improves, but computational complexity and processing time increase
Solution Approach 1:
The patent segments the filtering process by applying the deep learning filter at specific stages in the video coding pipeline (e.g., after reconstruction, before output). The filtering is applied selectively to reconstructed blocks rather than entire frames, and can be configured to process only certain regions or quality levels, reducing overall computational complexity while maintaining high recognition accuracy where needed.
Solution Approach 2:
The patent implements partial filtering by applying the deep learning filter to only certain blocks or regions of the image based on quality requirements, rather than processing every pixel uniformly. This allows the system to achieve sufficient filtering effect in critical areas while reducing computational burden in less important regions, balancing recognition accuracy with computational complexity.
Data Source
AI summary
Some aspects of the disclosure provide a method of deep learning-based filtering. The method includes obtaining a reconstructed image corresponding to a coded image and at least a quantization parameter. At least a portion of the reconstructed image is reconstructed based on the quantization parameter. The method also includes generating an input to a deep learning filter according to a point multiplication operation of the reconstructed image and the at least the quantization parameter. The point multiplication operation includes pixel-wise multiplications of the at least the portion of the reconstructed image with the quantization parameter. Further, the method includes generating, by the deep learning filter and in response to the input, a filtered image corresponding to the reconstructed image. Apparatus and non-transitory computer-readable storage medium counterpart embodiments are also contemplated.