Neural-Network Post-Filter SEI Signaling for Adaptive Video Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards, such as VVC, lack efficient mechanisms for signaling neural-network post-processing filters (NNPFs) and their activation, leading to suboptimal video quality enhancement in digital video streaming and storage.
Innovation Solution
Incorporation of neural-network post-filter characteristics (NNPFC) and activation (NNPFA) supplemental enhancement information (SEI) messages within suffix SEI network abstraction layer (NAL) units to facilitate conversion between visual media data and bitstreams, enabling effective signaling and application of NNPFs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If existing video coding standards (VVC) are used without NNPF signaling mechanisms, then the standard remains simple and compatible, but video quality enhancement capability is insufficient
Solution Approach 1:
The patent introduces SEI messages as an intermediary mechanism to carry NNPF characteristics and activation information. These messages act as a mediator between the video coding standard and neural network post-processing filters, enabling quality enhancement without fundamentally altering the core coding framework. The SEI messages transport filter parameters, activation flags, and synchronization information, resolving the contradiction by providing a standardized interface that adds capability while maintaining compatibility.
Solution Approach 2:
The patent segments the NNPF signaling into distinct components: NNPFC SEI messages for filter characteristics and NNPFA SEI messages for activation control. This segmentation allows the system to selectively apply complexity only where needed - filter definition is separated from activation control, enabling flexible implementation where decoders can choose to support only certain aspects. This resolves the contradiction by making the complexity optional and modular rather than mandatory and monolithic.
2Manufacturing precision
If NNPFs are applied dynamically to enhance video quality, then visual quality and resolution improve, but processing complexity and computational requirements increase
Solution Approach 1:
The patent implements dynamic NNPF application through NNPFA SEI messages that can activate or deactivate filters on a per-picture or per-region basis. The activation mechanism allows the system to adaptively apply neural network processing only when beneficial, rather than uniformly across all video data. This dynamic control resolves the contradiction by enabling quality enhancement while providing flexibility to reduce processing complexity when full enhancement is not required.
Solution Approach 2:
The patent enables local application of NNPFs through region-specific activation and selective filter application. Different regions of the video stream can receive different levels of processing, with NNPFA messages targeting specific picture regions or entire frames as needed. This local quality approach resolves the contradiction by concentrating computational resources on areas that benefit most from enhancement while reducing processing elsewhere.
3Adaptability or versatility
If NNPFC and NNPFA SEI messages are included in suffix SEI NAL units, then filter signaling capability is enhanced, but bitstream complexity and processing overhead increase
Solution Approach 1:
The patent utilizes the existing SEI message framework for multiple purposes: error recovery, timing information, and now NNPF signaling. By extending the already-established SEI mechanism to carry NNPFC and NNPFA messages, the system achieves multi-functionality without adding separate dedicated signaling structures. This resolves the contradiction by leveraging existing infrastructure, thereby enhancing signaling capability while minimizing additional processing overhead compared to creating entirely new message types.
Data Source
AI summary
A mechanism for processing video data is disclosed. The mechanism includes determining a neural-network post-filter characteristics (NNPFC) supplemental enhancement information (SEI) message and a neural-network post-filter activation (NNPFA) SEI message are allowed to be included in suffix SEI network abstraction layer (NAL) units. A conversion is performed between a visual media data and a bitstream based on the NNPFC SEI message and the NNPFA SEI message.