CNN-Based In-Loop Filter for Video Residual Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards face challenges in achieving high compression efficiency, particularly with the development of advanced standards like VVC, where existing in-loop filters may not adequately address residual errors and artifacts, impacting the quality of compressed video.
Innovation Solution
Implementing a convolutional neural network (CNN) based in-loop filter that utilizes motion estimation to determine residuals by comparing target coding blocks with reference blocks, enhancing the video processing through improved prediction and residual determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional in-loop filters are used in video coding, then device complexity is low, but coding efficiency and video quality are insufficient
Solution Approach 1:
The patent replaces traditional mechanical filtering operations with a neural network-based system. The CNN filter uses learned patterns and adaptive processing to achieve superior coding efficiency while managing complexity through software-based neural network inference rather than complex hardware filtering circuits.
Solution Approach 2:
The patent dynamically adjusts filter parameters based on local video characteristics. The neural network adapts its filtering behavior by analyzing local variance, gradient directions, and other features, changing filter strength and type according to the specific content region, thereby improving coding efficiency without uniformly increasing complexity across all blocks.
2Loss of substance
If compression ratio is increased, then storage memory and transmission bandwidth are reduced, but residual errors and artifacts increase
Solution Approach 1:
The patent converts the harmful residual errors and artifacts introduced by high compression into beneficial information for the neural network filter. The CNN is trained to recognize and correct these specific types of artifacts, transforming the degradation caused by compression into an opportunity for intelligent restoration, thereby maintaining video quality at higher compression ratios.
Solution Approach 2:
The neural network filter acts as an intermediary between the compressed video data and the final reconstructed output. It processes the degraded signal from high-compression encoding, applying learned corrections to remove artifacts before presenting the restored video, thus mediating between compression efficiency and quality preservation.
3Measurement precision
If motion estimation is applied to determine reference blocks, then prediction accuracy improves, but processing time increases
Solution Approach 1:
The patent performs motion estimation and identifies reference blocks in advance during the encoding phase. The neural network filter is pre-trained with motion-compensated reference information, allowing it to quickly process the actual filtering without repeating the computationally intensive motion search during real-time decoding, thus improving prediction accuracy while minimizing additional processing time.
Data Source
AI summary
The present disclosure provides methods for convolutional-neural-network (CNN) based filter for video coding. An exemplary method includes: applying motion estimation to a target coding block, to determine a reference block of the target coding block; inputting, to a convolutional neural network (CNN) filter, image data associated with the target coding block and the reference block; and executing the CNN filter to determine a residual associated with the target coding block based on the input image data.