Neural Network Filter Model Selection for Video Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies face challenges in efficiently reducing distortion during compression, particularly as samples with different characteristics are processed by a single CNN model, and there is a lack of adaptability in CNN filter models for varying video units and temporal layers.
Innovation Solution
The development of multiple neural network (NN) filter models trained for specific video units and temporal layers, with mechanisms to select and signal the appropriate NN filter index for each unit, allowing for different NN filters to be used based on characteristics and quality parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a single CNN filter model is used for all video units, then device complexity is reduced, but manufacturing precision deteriorates because samples with different characteristics cannot be processed optimally
Solution Approach 1:
The patent divides the video content into different video units (e.g., CTUs, CUs, or other partitioning structures) and applies different NN filter models to different segments. Each video unit is evaluated based on its characteristics (such as complexity, motion, or texture) and assigned an appropriate filter model from a plurality of candidates, enabling optimized processing for each segment while managing overall system complexity.
Solution Approach 2:
The patent implements local quality by selecting different NN filter models tailored to specific video unit characteristics. Instead of applying a uniform filter across the entire video, the system adapts the filtering approach locally based on the content properties of each video unit, thereby improving overall reconstruction quality without uniformly increasing complexity across all regions.
2Manufacturing precision
If multiple NN filter models are selected for different video units, then manufacturing precision improves, but device complexity increases due to model selection and signaling overhead
Solution Approach 1:
The patent performs preliminary classification of video units based on their characteristics before applying the filtering process. By pre-evaluating video unit properties and determining the appropriate filter model in advance, the system avoids complex real-time decisions during filtering and reduces the computational burden of model selection.
Solution Approach 2:
The patent changes the parameter of filter model selection based on video unit characteristics. Different parameters (such as complexity metrics, motion characteristics, or texture features) are used to determine which NN filter model should be applied to each video unit, enabling adaptive filtering without requiring a fixed complex selection mechanism.
3Adaptability or versatility
If different NN filters are applied to different video units, then adaptability improves, but loss of information increases due to signaling overhead for filter indices
Solution Approach 1:
The patent establishes a universal set of NN filter models that can serve multiple video unit types and characteristics. By designing filter models with multi-functionality that can handle various video content types, the system reduces the number of specialized models needed and consequently reduces the signaling overhead required to indicate which model should be used.
Solution Approach 2:
The patent uses copying mechanisms where filter model indices are inherited or copied from neighboring video units or parent units when appropriate. Instead of signaling the filter index for every video unit independently, the system copies the index from adjacent units with similar characteristics, significantly reducing the bitrate overhead while maintaining adaptability.
Data Source
AI summary
A method implemented by a video coding apparatus. The method includes selecting a neural network (NN) filter model from a plurality of NN filter model candidates for each video unit. The NN filter model selected for a first video unit is different than the NN filter model selected for a second video unit. The method also includes converting between a video media file and a bitstream based on the one or more NN filter models selected for the video unit. A corresponding video coding apparatus and non-transitory computer readable medium are also disclosed.