Neural Network Video Filter Partitioning

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current video coding technologies face inefficiencies in bandwidth usage and compression quality due to limitations in predictive modeling and filtering techniques, particularly in handling diverse video content and quality levels.

Innovation Solution

The implementation of neural network-based filter models that utilize external information such as partitioning data and coding parameters as attention mechanisms to enhance intra and inter prediction, transform kernels, and loop filtering processes, thereby improving video coding efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional predictive modeling and filtering techniques are used in video coding, then device complexity is reduced and ease of operation is maintained, but compression efficiency and bandwidth usage are insufficient

Engineering Contradiction:
Improvecompression efficiencyVSAvoidmodel complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent replaces traditional mechanical filtering techniques (deblocking filter, sample adaptive offset filter, adaptive loop filter) with a neural network-based filtering system. The neural network model learns optimal filtering operations from training data, substituting hand-crafted filtering algorithms with data-driven intelligent filtering that adapts to different video content characteristics, thereby improving compression efficiency while managing complexity through learned patterns.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the parameters and structure of filtering models by introducing neural network architectures with multiple layers, filters, and activation functions. The system dynamically adjusts filtering parameters based on input video characteristics, transforming fixed-parameter traditional filters into adaptive neural network filters that optimize performance for different content types and quality requirements.

Inventive Principle:
Principle #35Parameter changes

2Loss of information

If neural network-based filter models are implemented to improve compression efficiency, then bitrate is reduced and quality is improved, but device complexity and computational requirements increase

Engineering Contradiction:
Improvecompression qualityVSAvoidcomputational complexity
Core Design Contradiction:
Loss of informationVSDevice complexity

Solution Approach 1:

The patent segments the video coding process into distinct neural network filtering stages, including deblocking filtering, sample adaptive offset filtering, and adaptive loop filtering, each handled by specialized neural network components. This segmentation allows the system to apply appropriate complexity to each stage independently, optimizing overall performance while managing computational load through distributed processing architecture.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements partial neural network filtering by selectively applying neural network-based filtering only to specific video regions or blocks that benefit most from it, rather than uniformly applying complex neural networks to all video content. This approach achieves improved compression quality where needed while reducing overall computational complexity for simpler video regions.

Inventive Principle:
Principle #16Partial or excessive action

3Adaptability or versatility

If traditional filtering techniques are used, then computational complexity is low and processing speed is maintained, but adaptability to diverse video content and quality levels is limited

Engineering Contradiction:
Improvecontent adaptabilityVSAvoidprocessing efficiency
Core Design Contradiction:
Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent introduces dynamic adaptability by designing neural network filtering systems that automatically adjust their behavior based on input video characteristics. The neural networks learn to adapt filtering strength, kernel sizes, and processing parameters dynamically according to content type, motion complexity, and desired quality levels, enabling the system to optimize performance for diverse video content while maintaining efficient processing through learned patterns.

Inventive Principle:
Principle #15Dynamics

Data Source

PatentUS20230051066A1Partitioning Information In Neural Network-Based Video Coding
Publication Date: 2023.02.16 LEMON INC(GB)
  • US20230051066A1 patent drawing
  • US20230051066A1 patent drawing
  • US20230051066A1 patent drawing

AI summary

A method implemented by a video coding apparatus. The method includes applying a neural network (NN) filter to an unfiltered sample of a video unit to generate a filtered sample, where the NN filter includes an NN filter model generated based on partitioning information of the video unit; and performing a conversion between a video media file and a bitstream based on the filtered sample.