CNN In-Loop Filtering for Video Coding Artifacts and Block Boundaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards face challenges in efficiently addressing artifacts and distortions in digital video due to the limitations of current in-loop filtering techniques, particularly in handling chroma subsampling and block boundaries, which affect video quality and compression efficiency.
Innovation Solution
The implementation of a convolutional neural network (CNN) filter for video processing, which applies adaptive filtering based on sample classification and geometric transformations to enhance the loop filter in video coding, addressing issues in deblocking, sample adaptive offset, and adaptive loop filtering to improve video quality and compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional in-loop filtering techniques are used, then device complexity is reduced, but video quality deteriorates due to artifacts and distortions
Solution Approach 1:
The patent replaces traditional mechanical/mathematical filtering operations (deblocking, SAO, ALF) with a neural network-based system. The neural network learns optimal filtering operations through training, substituting hand-crafted filtering algorithms with data-driven models that adapt to different video content characteristics, thereby improving video quality while managing complexity through learned patterns.
Solution Approach 2:
The patent changes the parameters of the filtering system by introducing learnable weights and biases in the neural network. Instead of fixed filtering parameters, the system dynamically adjusts parameters based on input video characteristics, allowing adaptive optimization of filtering strength and type for different regions and content types, thus resolving the contradiction between quality and complexity.
2Manufacturing precision
If adaptive filtering is applied to all video regions, then video quality improves, but processing time increases
Solution Approach 1:
The patent applies different filtering strategies to different regions of the video based on local characteristics. The neural network processes video blocks independently and can apply varying degrees of filtering or different filter types depending on the specific block's content, motion characteristics, and artifact severity. This localized approach improves quality where needed while reducing processing time in regions requiring minimal filtering.
Solution Approach 2:
The patent implements selective adaptive filtering where not all video regions receive the same level of processing. The system identifies regions with significant artifacts and applies intensive filtering only to those areas, while using lighter or no filtering in regions that already have acceptable quality. This partial action approach balances video quality improvement with processing time constraints.
3Productivity
If convolutional neural network filters are implemented, then compression efficiency improves, but device complexity increases
Solution Approach 1:
The patent segments the neural network into multiple smaller convolutional layers rather than using a single large network. Each layer performs a specific filtering function and processes a portion of the feature extraction task. This segmentation reduces the memory footprint and computational complexity of each individual layer, making implementation more manageable while maintaining overall compression efficiency through the cumulative effect of multiple specialized layers.
Data Source
AI summary
Methods, systems, apparatus for media processing are described. One example method of digital media processing includes determining, for a conversion between visual media data and a bitstream of the visual media data, how to apply one or more convolutional neural network filters to at least some samples of a video unit of the visual media data according to a rule; and performing the conversion based on the determining.