Multi-Density Neural Network In-Loop Filtering to Reduce Blocking Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards face challenges in effectively reducing blocking artifacts during video compression, which affect the quality of compressed video streams.
Innovation Solution
A neural network with a multi-density block structure is employed, comprising a first branch for maintaining full resolution and a second branch for down-sampling and up-sampling to capture spatially-precise representations and richer spatial correlations, enhancing the ability to reduce blocking artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional video coding standards are used for compression, then coding efficiency is improved, but blocking artifacts remain in the compressed video
Solution Approach 1:
The patent replaces traditional mechanical filtering methods (deblocking filter, SAO filter) with a neural network-based in-loop filter. The neural network learns optimal filtering operations from training data and applies them to reduce blocking artifacts while preserving coding efficiency, achieving superior artifact reduction compared to conventional filtering approaches.
Solution Approach 2:
The patent changes the operational parameters of the filtering system by introducing learnable filters within the neural network. These filters adapt their parameters (weights, biases) during training to optimize the balance between blocking artifact reduction and maintaining coding efficiency, dynamically adjusting filtering strength based on local image characteristics.
2Object-affected harmful factors
If in-loop filtering is applied to reduce blocking artifacts, then video quality is improved, but coding efficiency decreases
Solution Approach 1:
The patent segments the filtering process into multiple specialized filters (deblocking filter, SAO filter, and neural network-based in-loop filter) that operate at different stages of the coding loop. Each filter targets specific types of artifacts with optimized processing, reducing the overall computational burden and maintaining coding efficiency while effectively removing blocking artifacts.
Solution Approach 2:
The patent uses trained filter parameters and models that can be copied and applied across different video sequences and coding scenarios. The neural network weights and filter configurations learned during training are reused during decoding, enabling efficient artifact reduction without requiring repeated training computations, thus preserving coding efficiency.
3Object-affected harmful factors
If complex filtering processes are used to remove blocking artifacts, then video quality is improved, but processing complexity increases
Solution Approach 1:
The patent performs preliminary training of the neural network filters offline using representative training data. This preliminary action pre-computes optimal filter parameters and configurations that can be directly applied during video decoding without real-time complex computations. The heavy computational work is done beforehand, simplifying the runtime filtering process and reducing device complexity during actual video processing.
Data Source
AI summary
The present disclosure provides methods for performing training and executing of a multi-density neural network in video processing. An exemplary method comprises: receiving a video stream comprising a plurality of pictures; processing the plurality of pictures using a first branch of a first block in the neural network, wherein the neural network is configured to reduce blocking artifacts in video compression of the video stream and the first branch comprises one or more residual blocks; and processing the plurality of pictures using a second branch of the first block in the neural network, wherein the second branch comprises a down-sampling processing, an up-sampling processing, and one or more residual blocks.