WCDAB In-Loop CNN Filter for Video Compression Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video encoding methods, such as HEVC and VVC, suffer from irreversible information loss and compression artifacts like blocking, blurring, and banding, particularly at high compression ratios, and current in-loop filters are limited in performance due to being handcrafted for stationary signals and lacking effective multi-input processing.
Innovation Solution
An in-loop CNN filter using a multi-stage training strategy with a WCDAB backbone and depth-wise separable convolutions, incorporating prediction frames, partition frames, and QP maps to enhance video quality, with a lightweight design that achieves improved performance and reduced computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If handcrafted in-loop filters are used for stationary signals, then filtering can be performed, but performance is limited and cannot handle dynamic video content effectively
Solution Approach 1:
The patent replaces handcrafted signal processing filters with a deep learning-based CNN in-loop filter. The neural network model, trained on video data, automatically learns optimal filtering operations that adapt to both stationary and dynamic content, eliminating the limitations of traditional handcrafted filters designed only for stationary signals.
Solution Approach 2:
The patent employs quantization parameter (QP) maps as input to the CNN filter, allowing the filter to adapt its behavior based on the compression level. The network processes multiple input frames (current, previous, and future) with different QP values, enabling it to adjust filtering strength and characteristics dynamically according to the video content and compression settings.
2Adaptability or versatility
If traditional in-loop filters process only single input frame, then processing is simple, but multi-input processing capability is lacking
Solution Approach 1:
The CNN in-loop filter is designed to process multiple input frames simultaneously (current frame, previous frame, and future frame) with different quantization parameters. This multi-input capability allows the filter to exploit temporal relationships and predict optimal filtering operations, enhancing performance for both stationary and dynamic video content within a unified architecture.
Solution Approach 2:
The patent uses future frames as input to the CNN filter before they are actually needed for final reconstruction. This preliminary action allows the network to perform predictive filtering operations in advance, leveraging information from future frames to improve the quality of current frame reconstruction, thereby reducing artifacts before they occur in the decoding process.
3Manufacturing precision
If deep learning models are used for in-loop filtering, then video quality enhancement is achieved, but computational complexity increases
Solution Approach 1:
The patent divides the video processing into separate stages: encoding with quantization, CNN-based in-loop filtering, and reconstruction. The CNN filter processes multiple input frames with different QP values through dedicated convolutional layers, separating the filtering operation from the encoding process. This segmentation allows for optimized computational resources allocation and improved video quality enhancement.
Solution Approach 2:
The CNN in-loop filter processes multiple input frames (current, previous, and future) with different quantization parameters, performing more filtering operations than traditional single-frame filters. This excessive action of processing multiple frames simultaneously enables the network to capture temporal dependencies and improve reconstruction quality, achieving better video quality despite increased computational demands.
Data Source
AI summary
A method of generating an enhanced frame at a video encoder is provided. The method may include receiving, by a weakly connected dense attention block (WCDAB) backbone of a convolutional neural network (CNN) of an in-loop filter, a first set of feature extractions as an input. The first set of feature extractions may be associated with a reconstructed frame. The method may include applying, by the WCDAB backbone of the CNN of the in-loop filter, a plurality of depth-wise separable convolutions to the first set of feature extractions to generate a set of global features.