Multi-Quality Loop Filter With Micro-Structured Parameter Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network-based quality enhancement methods for video compression struggle to accommodate multiple Quantization Parameter (QP) values, leading to high storage and computation costs due to the need for individual model instances for each QP value.
Innovation Solution
A Multi-Task Neural Network (MTNN) with micro-structured parameter sharing is used, where one model instance accommodates multiple QP values, utilizing a binary Micro-Structured Mask (MSM) for each QP value to guide inference computation, and a progressive multi-stage training framework is employed to learn and prune model parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If individual neural network model instances are used for each Quantization Parameter (QP) value, then the video compression quality can be optimized for each specific QP value, but the storage and computation costs increase significantly
Solution Approach 1:
The patent implements a universal neural network model that can handle multiple QP values through parameter sharing. The shared layers contain weight parameters that are common across different QP values, while task-specific layers are selectively activated based on the input QP value. This allows a single model instance to perform the function of multiple individual models, reducing storage requirements while maintaining optimized compression quality for each QP value.
Solution Approach 2:
The neural network is segmented into two distinct parts: shared layers that contain common weight parameters applicable to all QP values, and task-specific layers that are dedicated to specific QP values. This segmentation allows the system to reuse computations from shared layers across different QP values while still providing specialized processing when needed, thereby reducing overall computational costs and storage requirements.
2Quantity of substance
If a single neural network model instance is used for multiple QP values, then storage and computation costs are reduced, but the ability to optimize compression quality for each specific QP value is compromised
Solution Approach 1:
The patent implements a universal neural network model that can handle multiple QP values through parameter sharing. The shared layers contain weight parameters that are common across different QP values, while task-specific layers are selectively activated based on the input QP value. This allows a single model instance to perform the function of multiple individual models, reducing storage requirements while maintaining optimized compression quality for each QP value.
Solution Approach 2:
Different parts of the neural network have different levels of specialization. The shared layers provide general-purpose processing that works across all QP values, while the task-specific layers provide localized optimization for specific QP values. This local quality approach ensures that the model maintains high compression quality for each specific QP value while still benefiting from the efficiency of a unified architecture.
3Manufacturing precision
If neural networks are used to reduce compression artifacts, then visual quality is enhanced, but the computational complexity and processing time increase
Solution Approach 1:
The neural network is segmented into shared layers and task-specific layers. The shared layers perform general denoising and artifact reduction that applies to all QP values, while task-specific layers provide additional specialized processing only when needed. This segmentation reduces overall computational complexity by avoiding redundant computations across different QP values while maintaining high visual quality enhancement.
Solution Approach 2:
The patent merges multiple individual neural network models into a single unified model by combining shared layers with task-specific layers. This merging eliminates the need to run separate models for different QP values, reducing computational overhead and processing time while maintaining the ability to enhance visual quality across all QP values through the shared processing infrastructure.
Data Source
AI summary
Video processing with a multi-quality loop filter using a multi-task neural network is performed by at least one processor and includes generating a first set of masked weight parameters, based on an input and a plurality of quantization parameter values with a corresponding first set of masks and first plurality of weight parameters, for a first set of shared neural network layers, selecting a second set of task specific neural network layers for the plurality of quantization parameter values with a second plurality of weight parameters, based on the plurality of quantization parameter values, computing an inference output, based on the first set of masked weight parameters and the second plurality of weight parameters, and outputting the computed inference output as an enhanced result.