CNN Loop Filtering for Video Coding Artifact Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards face challenges in efficiently addressing artifacts and distortions in digital video due to the limitations of traditional loop filters, particularly in high-efficiency video coding (HEVC) and versatile video coding (VVC), which impact the integrity and quality of encoded digital media.
Innovation Solution
The implementation of convolutional neural network (CNN) filters for video processing, which are applied to enhance the loop filtering process by improving the deblocking, sample adaptive offset, and adaptive loop filter techniques, utilizing machine learning to optimize filtering operations based on visual media data characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional loop filters are used in video coding standards (HEVC, VVC), then device complexity is kept manageable, but manufacturing precision (filtering quality) deteriorates due to inability to effectively address artifacts and distortions
Solution Approach 1:
The patent replaces traditional mechanical/mathematical filtering systems with a neural network-based system. The neural network learns optimal filtering operations from training data and applies them to video blocks, substituting the fixed mathematical transformations of traditional filters with adaptive, data-driven filtering that achieves superior artifact removal and distortion correction while maintaining computational feasibility through optimized network architecture.
2Manufacturing precision
If neural network filters are implemented for video processing, then manufacturing precision (filtering quality) improves by significantly reducing artifacts and distortions, but device complexity increases due to the introduction of machine learning models
Solution Approach 1:
The patent changes the fundamental parameters of the filtering system by transitioning from fixed filter coefficients to learned neural network weights and activation functions. The neural network parameters are trained offline to optimize filtering performance for various video content types, and during encoding, the pre-trained network processes video blocks with adaptive filtering that responds to local characteristics, achieving high filtering quality without requiring complex real-time parameter optimization.
Solution Approach 2:
The patent segments the video processing into distinct stages: traditional filtering operations (deblocking, sample adaptive offset, adaptive loop filter) are separated and enhanced by inserting neural network processing at specific points in the filtering pipeline. This segmentation allows the neural network to focus on specific artifact types while maintaining compatibility with existing filtering stages, thereby improving overall filtering quality without completely redesigning the entire system.
3Manufacturing precision
If neural network filters are applied to all video blocks, then manufacturing precision (artifact reduction) improves, but productivity (encoding speed) deteriorates due to increased computational load
Solution Approach 1:
The patent applies neural network filtering selectively rather than uniformly to all video blocks. The system evaluates characteristics of each video block (such as artifact presence, block type, and content complexity) and applies neural network filtering only to blocks that benefit from it, while using traditional filters for blocks where they remain sufficient. This partial application approach maintains high artifact reduction where needed while preserving encoding speed by avoiding unnecessary neural network processing.
Solution Approach 2:
The patent introduces dynamic adaptivity into the filtering process by making the application of neural network filters conditional and variable. The filtering strength, network selection, and processing intensity are dynamically adjusted based on real-time analysis of video content characteristics, block type, and local artifact severity. This dynamic approach allows the system to optimize the balance between artifact reduction and encoding speed for each specific video block rather than using a static one-size-fits-all approach.
Data Source
AI summary
Methods, systems, apparatus for media processing are described. One example method of digital media processing includes determining, for a conversion between visual media data and a bitstream of the visual media data, how to apply one or more convolutional neural network filters to at least some samples of a video unit of the visual media data according to a rule; and performing the conversion based on the determining.