Neural Network Filter Depth Switching for Video Coding Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in efficiently managing bandwidth demand due to the increasing number of connected devices receiving and displaying video, necessitating improved video coding efficiency.
Innovation Solution
Implementing neural network (NN) filter models with configurable depths to enhance video coding by replacing or augmenting modules in video codecs, utilizing external information for attention mechanisms and applying these filters to video units based on specific criteria.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single fixed-depth NN filter model is used for all video units, then the device complexity is reduced, but the video coding efficiency and adaptability deteriorate
Solution Approach 1:
The patent implements dynamic depth configuration for NN filter models by selecting different model depths (first depth or second depth) based on video unit characteristics such as temporal layer, slice type, and picture type. This allows the system to adaptively adjust filtering complexity to match the actual needs of different video content regions, improving coding efficiency while managing device complexity through conditional selection rather than uniform high complexity.
Solution Approach 2:
The patent applies different NN filter model depths to different video units based on their specific characteristics. Video units in certain temporal layers, slice types, or picture types receive deeper models (more residual blocks) while others receive shallower models. This localized differentiation ensures that computational resources are concentrated where most needed, improving overall video coding efficiency without uniformly increasing device complexity across all video units.
2Productivity
If multiple NN filter models with different depths are applied to different video units, then video coding efficiency is improved, but the device complexity increases
Solution Approach 1:
The system dynamically selects between first and second NN filter models with different depths based on video unit characteristics including temporal layer index, slice type (intra/inter), and picture type (I/P/B). This dynamic selection mechanism improves video coding efficiency by applying appropriate filtering strength to each video unit while managing device complexity through algorithmic selection rather than parallel implementation of all models.
Solution Approach 2:
The patent segments the video stream into different categories (temporal layers, slice types, picture types) and applies different NN filter model depths to each segment. This segmentation allows the system to improve overall video coding efficiency by treating different video regions differently, while managing device complexity by processing each segment with its designated model rather than requiring all models to be simultaneously active.
3Manufacturing precision
If deeper NN filter models with more residual blocks are used, then video quality is improved, but the processing time and computational load increase
Solution Approach 1:
The patent changes the depth parameter of NN filter models (number of residual blocks) based on video unit characteristics. Deeper models with more residual blocks are applied to video units requiring higher quality (certain temporal layers, slice types, picture types), while shallower models are applied elsewhere. This parameter adaptation improves video quality where needed while reducing processing time and computational load in less demanding regions.
Solution Approach 2:
The system dynamically adjusts the number of residual blocks in NN filter models according to video unit characteristics. Video units in specific temporal layers, slice types, or picture types receive deeper models with more residual blocks for higher quality, while other video units receive shallower models with fewer residual blocks to reduce processing time. This dynamic adjustment balances video quality improvement against processing time constraints.
Data Source
AI summary
A method implemented by a video coding apparatus includes applying a neural network (NN) filter to an unfiltered sample of a video unit to generate a filtered sample, where the NN filter is based on a first NN filter model having a first depth, or a second NN filter model having a second depth, where the depth comprises a number of residual blocks of the respective NN filter model, and where the second depth is different than the first depth. The method also includes performing a conversion between a video media file and a bitstream based on the filtered sample.