Neural Video Filtering with Hybrid Offline-Online CNN Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding and decoding systems face challenges with filtering performance degradation due to fixed weight coefficients in offline-trained neural network filters and high complexity in online-trained filters, limiting their applicability and adaptability to various video contents.
Innovation Solution
A filtering method using a combination of offline and online trained convolutional neural network filters, where offline filters are deployed at both encoding and decoding ends with fixed weights, and online filters have adaptable weights transmitted in a bitstream, ensuring wide applicability and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If filters are trained offline with fixed weight coefficients, then coding efficiency is improved and implementation is simplified, but filtering performance degrades for certain video contents due to lack of adaptability
Solution Approach 1:
The filter is divided into two distinct portions: an offline-trained portion with fixed weight coefficients and an online-trained portion with adjustable weight coefficients. This segmentation allows each portion to serve different purposes - the offline portion provides stable baseline performance while the online portion adapts to specific video content characteristics
Solution Approach 2:
The online-trained portion introduces dynamic adaptability to the filtering system. Its weight coefficients can be adjusted during operation based on video content characteristics, transforming the static filter into a dynamic system that responds to different video scenarios
2Adaptability or versatility
If filters are trained online with updated weight coefficients, then filtering performance is improved for specific video contents, but computational complexity and bitstream transmission requirements increase
Solution Approach 1:
By separating the filter into offline and online portions, the computational burden is distributed. The offline portion handles general filtering tasks with pre-computed coefficients, while the online portion only adjusts coefficients for specific video content, reducing overall computational complexity compared to fully online training
Solution Approach 2:
Instead of retraining all filter coefficients online, only the online-trained portion's coefficients are updated. This partial action approach achieves adaptability for specific video contents without the excessive computational cost of complete retraining
3Manufacturing precision
If online training is performed frequently to maintain filtering performance, then video quality is improved, but encoding time and processing overhead increase
Solution Approach 1:
The online training is performed periodically or selectively based on video content changes rather than continuously. This periodic action maintains video quality through adaptive filtering while avoiding the continuous processing overhead of frequent retraining
Solution Approach 2:
Only the necessary online-trained portion is updated when adaptability is needed, rather than performing complete filter retraining. This partial action maintains video quality while reducing encoding time and processing overhead
Data Source
AI summary
Provided in implementations of the present application are a filtering method and apparatus, and a computer storage medium; the method comprises: acquiring pixel information to be filtered; determining edge information; and inputting the sample information to be filtered and the edge information into a neural network-based filter so as to output filtered pixels, the filter being obtained by an online filtering part combined with an offline filtering part.