CNN In-Loop Filter for Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional in-loop filters in video coding and decoding systems, such as De-Blocking Filter (DBF), Sample Adaptive Offset (SAO), and Adaptive Loop Filter (ALF), require manual design and configuration, leading to suboptimal filtering effects and increased bit rates due to the need for signaling filter parameters, which limits the subjective and objective quality of reconstructed pictures.
Innovation Solution
Implementing a Convolutional Neural Network (CNN) filter that fuses multiple color components and auxiliary information like block partitioning and Quantization Parameter (QP) information for in-loop filtering, reducing calculation complexity and bit rates while improving picture quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional in-loop filters (DBF, SAO, ALF) are used with manual design and configuration, then the filtering process is simple to implement, but the filtering effect is suboptimal and bit rate increases due to parameter signaling
Solution Approach 1:
The patent replaces manual filter design and configuration (mechanical/systematic approach) with a Convolutional Neural Network (CNN) based automatic filtering system. The CNN model learns optimal filtering parameters automatically from training data, eliminating the need for manual parameter signaling in the bitstream while achieving superior filtering effects compared to conventional DBF, SAO, and ALF filters.
Solution Approach 2:
The CNN filter performs self-configuration by automatically learning optimal filtering parameters from training data during the model training phase. Once trained, the filter can operate autonomously without requiring parameter signaling in the bitstream, as the filter adapts to different picture characteristics through its learned weights and biases, thereby reducing bit rate while maintaining high filtering quality.
2Manufacturing precision
If CNN filter is used to perform filter processing on reconstructed picture, then picture quality is improved, but not all related information is fully utilized
Solution Approach 1:
The patent merges multiple color components (Y, U, V channels) and auxiliary information (block partitioning information, QP information) into a unified input for the CNN filter. This fusion allows the filter to comprehensively utilize all available picture information, enabling more accurate distortion elimination and achieving better picture quality compared to filters that process color components separately or utilize limited information.
Solution Approach 2:
The CNN filter is designed to process multiple types of input data simultaneously - color components, block partitioning information, and QP information - making it a universal filtering solution that can adapt to various picture characteristics and coding conditions. This multi-functional capability allows the filter to fully utilize all available information for optimal filtering performance across different scenarios.
3Manufacturing precision
If multiple color components are processed separately through complete network forward calculation, then filtering can be performed on each component, but calculation complexity increases
Solution Approach 1:
The patent merges the processing of multiple color components into a single unified CNN forward pass. By fusing Y, U, and V channels along with auxiliary information into a combined input tensor, the filter performs all color component processing simultaneously in one calculation, rather than executing separate forward calculations for each component. This approach maintains complete filtering coverage while significantly reducing computational complexity.
Data Source
AI summary
Provided are a method and apparatus for in-loop filtering and a computer storage medium. The method includes that: a picture to be filtered is acquired, the picture to be filtered being generated in a video coding process of an original picture in a video to be coded, the video to be coded including at least one original picture and the at least one original picture including the original picture; fused information of the picture to be filtered is determined, the fused information being obtained by fusing at least two colour components of the picture to be filtered and corresponding auxiliary information; and in-loop filtering is performed on the picture to be filtered based on the fused information to obtain at least one colour component of the picture subjected to the in-loop filtering.