Neural Network Loop Filter for Video Quality Adaptation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing video compression methods face challenges in accommodating different quality settings, leading to inefficient compression quality and increased distortion with higher compression ratios, as they require multiple neural network model instances for varying quality parameters, which is not flexible and effective for arbitrary smooth settings.

Innovation Solution

A neural network-based loop filtering method using meta-learning that computes adaptive weight parameters based on current decoded video and quality factors, enabling a single model instance to enhance videos with arbitrary smooth quality settings, including both seen and unseen settings during application.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If multiple neural network model instances are used for different quality settings, then video quality enhancement can be achieved for specific quality factors, but device complexity and computational overhead increase significantly

Engineering Contradiction:

Improvevideo quality enhancementVSAvoidnumber of neural network model instances

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent implements a single neural network model that can process multiple quality factor settings through a quality factor embedding module. This universal model replaces the need for multiple separate models, allowing the same network to adapt to different compression qualities by receiving QF values as input and adjusting its filtering behavior accordingly, thus reducing device complexity while maintaining video quality enhancement capability

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent introduces quality factor values as adjustable parameters that influence the neural network's filtering behavior. By changing the QF parameter input to the model, the system can adapt its enhancement strength and characteristics without requiring separate trained models for each quality level, enabling flexible quality control through parameter adjustment rather than model multiplication

Inventive Principle:

Principle #35Parameter changes

2Device complexity

If a single neural network model is used for all quality settings, then device complexity is reduced, but the ability to accommodate different quality factors and achieve optimal enhancement for each setting is lost

Engineering Contradiction:

Improvenumber of neural network model instancesVSAvoidaccommodation of different quality factors

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent transforms the static single-model approach into a dynamic system where the neural network's behavior adapts to different quality factors in real-time. The quality factor embedding module dynamically adjusts the model's filtering characteristics based on the input QF value, allowing the same physical model to exhibit different functional behaviors suited to various compression qualities, thus achieving both simplicity and adaptability

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent introduces a quality factor embedding module as an intermediary component between the input video and the neural network. This mediator translates the quality factor value into appropriate filtering parameters that guide the neural network's enhancement process, enabling the single model to effectively accommodate different quality settings without requiring separate specialized models for each scenario

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If traditional block-based hybrid prediction methods are used, then compression efficiency is maintained, but video quality suffers from artifacts that degrade quality of experience

Engineering Contradiction:

Improvecompression efficiencyVSAvoidvideo quality

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The patent replaces traditional mechanical block-based filtering mechanisms with a neural network-based loop filter. This substitution introduces a learned, data-driven approach that can identify and remove compression artifacts more effectively than fixed block-based methods, significantly improving video quality while preserving compression efficiency achieved through the underlying video coding standard

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11750847B2Quality-adaptive neural network-based loop filter with smooth quality control by meta-learning

Publication Date: 2023.09.05 TENCENT AMERICA LLC

AI summary

A method and apparatus of for video enhancement based on neural network based loop filtering using meta learning may include receiving reconstructed video data; receiving one or more quality factors associated with the reconstructed video data; determining a neural network based loop filter comprising neural network based loop filter parameters and a plurality of layers, wherein the neural network based loop filter parameters include shared parameters and adaptive parameters; and generating enhanced video data with artefact reduction, based on the one or more quality factors and the reconstructed video data, using a neural network based loop filter, wherein the neural network based loop filter comprises neural network based loop filter parameters that include shared parameters and adaptive parameters.