NNPF SEI Message Ordering for Video Coding Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards like VVC and VSEI lack efficient methods for processing neural-network post-filter (NNPF) SEI messages, leading to suboptimal video quality enhancement and increased bandwidth usage.
Innovation Solution
Implementing a method to determine a supplemental enhancement information (SEI) processing order based on differentiated types of SEI messages, particularly for neural-network post-filter (NNPF) messages with unique identifier values, and performing conversions between visual media data and bitstreams to optimize NNPF application.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural-network post-filter (NNPF) SEI messages are processed without differentiated processing order, then the processing is simpler, but video quality enhancement is suboptimal
Solution Approach 1:
The patent segments SEI messages into different types based on their function and importance. NNPF SEI messages are identified by a specific message_type value (e.g., 27) and are separated from other SEI messages for dedicated processing. This segmentation allows the decoder to apply neural network post-filtering to specific video frames while maintaining standard processing for other messages, thereby improving video quality without requiring complete reprocessing of all SEI messages.
Solution Approach 2:
The patent implements preliminary identification and classification of NNPF SEI messages during the bitstream parsing phase. The decoder detects the message_type field in incoming SEI messages and pre-sorts them into a dedicated processing queue before actual neural network filtering is applied. This preliminary action ensures that NNPF messages are processed in the correct temporal order and at the appropriate stages of video decoding, optimizing quality enhancement while maintaining processing efficiency.
2Productivity
If standard SEI message processing is used for NNPF messages, then bandwidth usage is lower, but video coding efficiency is reduced
Solution Approach 1:
The patent introduces specific parameter changes in the SEI message structure to enable efficient NNPF processing. A message_type parameter is added to distinguish NNPF SEI messages from other SEI messages. Additionally, the patent utilizes existing SEI message fields (such as payload_size and payload) to carry neural network filter parameters and activation information. These parameter changes allow the system to convey rich filtering information without significantly increasing bandwidth consumption, as the same SEI message transport mechanism is used.
Solution Approach 2:
The patent makes the SEI message structure universal by designing it to handle multiple types of supplemental information through a unified framework. The same SEI message format and transmission mechanism are used for both traditional SEI messages and NNPF-specific messages. This multi-functionality allows the system to maintain backward compatibility with existing video coding standards while efficiently supporting advanced neural network post-filtering capabilities, thereby improving coding efficiency without proportionally increasing bandwidth usage.
Data Source
AI summary
A mechanism for processing video data is disclosed. The mechanism includes determining a supplemental enhancement information (SEI) processing order of SEI messages based on differentiated types of SEI messages, wherein neural-network post-filter (NNPF) SEI messages are differentiated by having different NNPF identifier values. A conversion is performed between a visual media data and a bitstream based on the SEI messages.