Fragmented SEI Payload Handling for Large Video Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing video coding standards, such as H.266/VVC, impose a limit of 255 bytes for Supplementary Enhancement Information (SEI) payloads, which is insufficient for data structures like EXIF or thumbnail images, necessitating a mechanism to extend this limit without altering the syntax significantly.
Innovation Solution
The proposed solution involves fragmenting large SEI messages into smaller fragments, each within the 255-byte limit, and using a single NAL unit type to indicate the relationship between these fragments, allowing for payloads exceeding 255 bytes while maintaining compatibility with existing syntax structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the SEI payload size is limited to 255 bytes as per existing video coding standards, then the syntax and decoding process remain simple and compatible, but the capability to include larger data structures like EXIF data or thumbnail images is restricted
Solution Approach 1:
The patent applies segmentation by dividing a large SEI payload (exceeding 255 bytes) into multiple smaller payload segments, each fitting within the 255-byte limit. The encoder splits the original payload into segments and transmits them as separate SEI messages or as a single SEI message with multiple payload segments. The decoder reconstructs the original payload by concatenating the segments in the correct order, thereby enabling support for larger data structures while maintaining compatibility with existing syntax constraints.
2Quantity of substance
If the payload size is extended beyond 255 bytes, then larger supplementary information can be included, but the existing syntax and maximum payload size constraints must be modified
Solution Approach 1:
The patent uses segmentation to bypass the need for syntax modification by dividing the large payload into smaller segments that each fit within the existing 255-byte constraint. This approach allows the system to effectively support larger payloads without changing the underlying syntax definitions or maximum payload size limits in the video coding standard.
Solution Approach 2:
The patent implements nesting by embedding multiple payload segments within a single SEI message structure or nesting SEI messages within NAL units. This hierarchical nesting allows the system to convey more than 255 bytes of supplementary information while maintaining the appearance of compliant message structures, thereby avoiding syntax modifications while effectively extending the payload capacity.
3Quantity of substance
If fragmentation techniques are used to split large payloads into 255-byte segments, then larger data can be included in the bitstream, but additional control mechanisms and headers are required
Solution Approach 1:
The patent applies segmentation by dividing a large SEI payload (exceeding 255 bytes) into multiple smaller payload segments, each fitting within the 255-byte limit. The encoder splits the original payload into segments and transmits them as separate SEI messages or as a single SEI message with multiple payload segments. The decoder reconstructs the original payload by concatenating the segments in the correct order, thereby enabling support for larger data structures while maintaining compatibility with existing syntax constraints.
Solution Approach 2:
The patent implements feedback mechanisms where the encoder includes segment identification information (such as segment indices or length indicators) in each fragmented payload segment. This feedback allows the decoder to correctly identify and reconstruct the original payload by understanding the structure and ordering of segments, thereby managing the complexity of fragmentation while enabling extended payload capacity.
Data Source
AI summary
A method and apparatus comprising computer code configured to cause a processor or processors to obtain video data comprising at least one coded picture; identify, by a decoder, at least one first supplementary information enhancement (SEI) message, the at least one first SEI message indicating variables specifying a type payloadType and a size payloadSize of a payload of the at least one SEI message and specified in bytes; and decode, by the decoder, the video data based on the first SEI message.