Neural-Network Post-Filter SEI Signaling for Lower-Bandwidth Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards, such as VVC, lack efficient mechanisms for signaling and applying neural-network post-filters, leading to suboptimal video processing and increased bandwidth usage.
Innovation Solution
The implementation of neural-network post-filter characteristics (NNPFC) supplemental enhancement information (SEI) messages to manage and apply neural-network post-filters across video sequences, ensuring efficient processing and reduced bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural-network post-filters are applied to video sequences, then video processing efficiency and quality are improved, but bandwidth consumption increases
Solution Approach 1:
The patent applies neural-network post-filters to decoded pictures before they are used for further processing or display. By performing the filtering operation at the decoding stage rather than at the encoding stage, the system improves processing efficiency without requiring additional bandwidth for transmitting filter parameters through the entire video stream.
Solution Approach 2:
The patent introduces SEI messages as intermediaries to carry neural-network post-filter characteristics between the decoder and the filtering operation. This intermediary mechanism allows the filter parameters to be efficiently transmitted and applied without disrupting the main video bitstream, thus improving processing efficiency without significantly increasing bandwidth consumption.
2Manufacturing precision
If neural-network post-filter characteristics are signaled using SEI messages, then video quality is improved, but device complexity increases
Solution Approach 1:
The patent makes the SEI message structure universal by defining it with optional fields that can be configured based on specific needs. The message can accommodate different types of neural-network post-filter characteristics (denoising, sharpening, color transformation) using the same basic framework, thereby improving video quality across multiple applications without proportionally increasing device complexity.
Solution Approach 2:
The patent implements dynamic configuration of the SEI message fields, allowing the decoder to adaptively parse and apply only the necessary filter characteristics based on the actual video content and processing requirements. This dynamic approach enables high video quality while reducing the effective complexity by avoiding unnecessary processing of unused filter parameters.
3Manufacturing precision
If neural-network post-filters are applied to all decoded pictures, then video quality is improved, but processing time increases
Solution Approach 1:
The patent applies neural-network post-filters selectively based on local characteristics of the video content and the specific filter type. By determining whether a filter is needed for each decoded picture or picture region, the system improves video quality where necessary while avoiding unnecessary processing time consumption in cases where the filter would have minimal impact or where computational resources should be allocated elsewhere.
Data Source
AI summary
A mechanism for processing video data is disclosed. The mechanism includes determining a neural-network post-filter characteristics (NNPFC) supplemental enhancement information (SEI) message. The NNPFC SEI message pertains to a current decoded picture and all subsequent decoded pictures of a current layer, in output order, until an end of a current coded layer video sequence (CLVS) or up to but excluding a decoded picture that follows the current decoded picture in output order within the current CLVS and is associated with a subsequent NNPFC SEI message, in decoding order, having a NNPFC base flag (nnpfc_base_flag) equal to 0 and a particular NNPFC identifier (nnpfc_id) value within the current CLVS, whichever is earlier.