Neural Network Post Filter for Video Coding via SEI Indicators

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current video coding technologies face challenges in efficiently managing and applying neural network (NN) filter models for video units, leading to suboptimal video quality and increased bandwidth usage.

Innovation Solution

The proposed solution involves including indicators in the supplemental enhancement information (SEI) message of bitstreams to specify neural network (NN) filter model candidates or selections for video units or samples within those units, allowing for dynamic enablement or disablement of NN filter models based on video unit parameters.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If neural network filter models are applied to all video units, then video quality is improved, but bandwidth usage increases

Engineering Contradiction:
Improvevideo qualityVSAvoidbandwidth usage
Core Design Contradiction:
Manufacturing precisionVSQuantity of substance

Solution Approach 1:

The patent applies different filtering approaches to different video units based on their specific characteristics. Neural network filters are selectively applied only to video units that benefit from them, while other units use traditional filtering methods. This localized application improves video quality where needed without unnecessarily increasing bandwidth usage for all units.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent dynamically determines whether to apply neural network filters to each video unit based on real-time analysis of video characteristics, motion complexity, and other parameters. This dynamic decision-making process allows the system to adaptively optimize the balance between video quality improvement and bandwidth consumption, applying NN filters only when they provide meaningful benefits.

Inventive Principle:
Principle #15Dynamics

2Manufacturing precision

If neural network filter models are applied to all video units, then video quality is improved, but computational complexity increases

Engineering Contradiction:
Improvevideo qualityVSAvoidcomputational complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent applies neural network filters selectively to specific video units rather than uniformly across all units. By identifying which video units have characteristics that benefit from NN filtering (such as complex textures or difficult-to-compress regions), the system applies computational resources only where needed, reducing overall computational complexity while maintaining video quality improvements.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent applies neural network filtering partially, only to the extent necessary for achieving acceptable video quality. Rather than applying NN filters to all video units regardless of need, the system performs partial application based on video content analysis, thereby reducing unnecessary computational complexity while still achieving quality improvements where they matter most.

Inventive Principle:
Principle #16Partial or excessive action

3Adaptability or versatility

If indicators for NN filter model selection are included in SEI messages, then adaptability is improved, but bitstream overhead increases

Engineering Contradiction:
Improvefilter model selection flexibilityVSAvoidbitstream overhead
Core Design Contradiction:
Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent segments the filter model indication information into different levels of detail. Rather than including complete filter model specifications for every video unit, the system uses a hierarchical approach where general filter model selections are indicated at higher levels (e.g., sequence or picture level) and specific adjustments are made only where necessary. This segmentation reduces bitstream overhead while maintaining adaptability for selective video units.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies filter model indication partially, providing detailed model selection information only for video units that require neural network filtering. For units that use standard filtering or default NN models, the system omits or simplifies the indication data. This partial information provision maintains adaptability for units that need it while significantly reducing overall bitstream overhead.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS20250030881A1Neural Network-Based Post Filter For Video Coding
Publication Date: 2025.01.23 BYTEDANCE INC
  • US20250030881A1 patent drawing
  • US20250030881A1 patent drawing
  • US20250030881A1 patent drawing

AI summary

A method of processing video data. The method includes determining that a supplemental enhancement information (SEI) message of a bitstream includes indicators specifying one or more neural network (NN) filter model candidates or selections for a video unit or samples within the video unit, and converting between a video media file comprising the video unit and the bitstream based on the indicators. A corresponding video coding apparatus and non-transitory computer readable medium are also disclosed.