Wavelet-Domain CNN In-Loop Filtering for Video Coding Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current CNN-based in-loop filtering methods for video coding face challenges such as being stuck in local optima during blind end-to-end training, neglecting image signal priors, and coarse model switching, which hampers coding efficiency.
Innovation Solution
Implement wavelet-domain CNNLF with QP-dependent and QP-independent training methods, region-based CNNLF switching, and CNNLF model index merging to enhance coding efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If blind end-to-end training is used for CNNLF, then training automation is improved, but training effectiveness deteriorates due to getting stuck in local optima
Solution Approach 1:
The patent applies preliminary action by performing wavelet transform on the video block before feeding it to the CNNLF. This preprocessing step transforms the spatial domain data into frequency domain coefficients, providing a better initial representation that helps the CNN avoid local optima during training. The wavelet coefficients capture multi-scale features that guide the optimization process more effectively than raw pixel data.
Solution Approach 2:
The patent changes the parameter space by operating in the wavelet domain rather than the spatial domain. By transforming the input data through wavelet decomposition, the optimization landscape is modified to have more favorable properties for gradient-based training, reducing the likelihood of getting stuck in poor local optima while maintaining automation.
2Device complexity
If traditional CNNLF is applied, then computational simplicity is maintained, but coding efficiency deteriorates due to coarse model switching
Solution Approach 1:
The patent segments the video block into wavelet subbands through wavelet transform, processing different frequency components separately. This segmentation allows the CNNLF to focus on specific frequency regions, improving coding efficiency by targeting artifacts in different bands while maintaining manageable computational complexity through the structured decomposition.
Solution Approach 2:
The patent transitions from spatial domain processing to frequency domain processing by applying wavelet transform. This dimensionality change allows the CNNLF to operate on wavelet coefficients rather than pixels, enabling more efficient artifact removal in the frequency domain while preserving important visual information, thus improving coding efficiency without excessive complexity increase.
3Productivity
If wavelet-domain CNNLF is implemented, then coding efficiency is improved, but computational complexity increases
Solution Approach 1:
The wavelet transform is performed as a preliminary action before CNN filtering, organizing the data into frequency subbands that are more amenable to efficient processing. This preprocessing step, while adding computational overhead, enables the subsequent CNN operations to be more effective and targeted, achieving better coding efficiency that justifies the additional complexity.
Data Source
AI summary
Implementations of the disclosure provide video processing apparatuses and methods. The method receives, by a video processor, a video block of a video for in-loop filtering. The method then performs a wavelet-domain CNN filtering on video data of at least a part of the video block, by performing, by the video processor, a wavelet transform on the video data to obtain data in a wavelet domain comprising a plurality of wavelet subbands; filtering, by the video processor, the data in the wavelet domain by applying respective CNN models on the plurality of wavelet subbands, where the CNN models for the plurality of wavelet subbands are trained in the wavelet domain; and performing, by the video processor, an inverse wavelet transform on the filtered data to obtain reconstructed video data.