Neural Network Post Filter for Video Coding via SEI Indicators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies face challenges in efficiently managing and applying neural network (NN) filter models for video units, leading to suboptimal video quality and increased bandwidth usage.
Innovation Solution
The proposed solution involves including indicators in the supplemental enhancement information (SEI) message of bitstreams to specify neural network (NN) filter model candidates or selections for video units or samples within those units, allowing for dynamic enablement or disablement of NN filter models based on video unit parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network filter models are applied to all video units, then video quality is improved, but bandwidth usage increases
Solution Approach 1:
The patent applies different filtering approaches to different video units based on their specific characteristics. Neural network filters are selectively applied only to video units that benefit from them, while other units use traditional filtering methods. This localized application improves video quality where needed without unnecessarily increasing bandwidth usage for all units.
Solution Approach 2:
The patent dynamically determines whether to apply neural network filters to each video unit based on real-time analysis of video characteristics, motion complexity, and other parameters. This dynamic decision-making process allows the system to adaptively optimize the balance between video quality improvement and bandwidth consumption, applying NN filters only when they provide meaningful benefits.
2Manufacturing precision
If neural network filter models are applied to all video units, then video quality is improved, but computational complexity increases
Solution Approach 1:
The patent applies neural network filters selectively to specific video units rather than uniformly across all units. By identifying which video units have characteristics that benefit from NN filtering (such as complex textures or difficult-to-compress regions), the system applies computational resources only where needed, reducing overall computational complexity while maintaining video quality improvements.
Solution Approach 2:
The patent applies neural network filtering partially, only to the extent necessary for achieving acceptable video quality. Rather than applying NN filters to all video units regardless of need, the system performs partial application based on video content analysis, thereby reducing unnecessary computational complexity while still achieving quality improvements where they matter most.
3Adaptability or versatility
If indicators for NN filter model selection are included in SEI messages, then adaptability is improved, but bitstream overhead increases
Solution Approach 1:
The patent segments the filter model indication information into different levels of detail. Rather than including complete filter model specifications for every video unit, the system uses a hierarchical approach where general filter model selections are indicated at higher levels (e.g., sequence or picture level) and specific adjustments are made only where necessary. This segmentation reduces bitstream overhead while maintaining adaptability for selective video units.
Solution Approach 2:
The patent applies filter model indication partially, providing detailed model selection information only for video units that require neural network filtering. For units that use standard filtering or default NN models, the system omits or simplifies the indication data. This partial information provision maintains adaptability for units that need it while significantly reducing overall bitstream overhead.
Data Source
AI summary
A method of processing video data. The method includes determining that a supplemental enhancement information (SEI) message of a bitstream includes indicators specifying one or more neural network (NN) filter model candidates or selections for a video unit or samples within the video unit, and converting between a video media file comprising the video unit and the bitstream based on the indicators. A corresponding video coding apparatus and non-transitory computer readable medium are also disclosed.


