ML Video In-Loop Filtering With Reduced Model Complexity

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

The high complexity of neural network-based in-loop filters in video codecs, such as those described in JVET-X0066 and JVET-Y0143, poses challenges for practical hardware implementations, necessitating a reduction in complexity while maintaining or improving performance.

Innovation Solution

The proposed methods involve generating encoded or decoded video data using machine learning models that utilize reconstructed and predicted sample values, block boundary strength, and quantization parameters, without requiring partition information, and employ attention residual blocks to reduce complexity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If neural network-based in-loop filters are used in video codecs, then compression efficiency and quality are improved, but device complexity increases

Engineering Contradiction:
Improvevideo qualityVSAvoidmodel complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent extracts and removes partition information from the neural network input, keeping only the most essential features (reconstructed samples, predicted samples, boundary strength, quantization parameters). This extraction principle reduces the input data dimensionality and model complexity while preserving the critical information needed for effective filtering and compression.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies different filtering strategies to different regions of the video data by using boundary strength information to identify block boundaries. The neural network applies stronger filtering locally at boundaries where artifacts are more prominent, while using lighter filtering in smooth regions, thereby improving overall quality without uniformly increasing complexity across the entire video stream.

Inventive Principle:
Principle #3Local quality

2Measurement precision

If partition information is included in neural network input, then filtering accuracy is improved, but processing complexity increases

Engineering Contradiction:
Improvefiltering accuracyVSAvoidprocessing complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent explicitly removes partition information from the neural network input data, extracting only the essential features needed for filtering. This decision demonstrates the extraction principle by eliminating redundant information (partition details) that would increase processing complexity without providing sufficient additional benefit to filtering accuracy.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of including partition information to improve filtering accuracy, the patent inverts the approach by excluding it and relying on alternative features like boundary strength and quantization parameters. This inversion shows that accurate filtering can be achieved without explicitly processing partition data, thereby reducing complexity while maintaining effectiveness.

Inventive Principle:
Principle #13The other way round (Inversion)

Data Source

PatentUS20250390738A1Generating encoded video data and decoded video data
Publication Date: 2025.12.25 TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)
  • US20250390738A1 patent drawing
  • US20250390738A1 patent drawing
  • US20250390738A1 patent drawing

AI summary

There is provided a method of training a machine learning, ML, model used for generating encoded video data or decoded video data. The method comprises obtaining original video data. The method further comprises converting the original video data into ML input video data. The method further comprises providing the ML input video data into the ML model, thereby generating first ML output video data, and training the ML model based on a difference between the original video data and the ML input video data and a difference between the original video data and the first ML output video data.