Image Metadata Carriage in Video Bitstreams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards, such as H.264, AVC, H.265, HEVC, and VVC, do not efficiently carry or reference popular image metadata formats like EXIF, JFIF, and XMP within the coded video stream, which limits the integration of AI-based applications that do not require neural network models to be carried or referenced from the video stream.
Innovation Solution
The method involves setting an image format metadata type identifier and signaling it in the bitstream, along with encoding the current image based on this identifier. Additionally, an image format metadata universal resource identifier flag is set to indicate whether the image information payload is obtained from the payload of the SEI message or a universal resource identifier.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If SEI messages are used to carry neural network models in video streams, then AI-based video processing is enabled, but the device complexity and bitrate requirements increase
Solution Approach 1:
The patent extracts the neural network model carriage function from the general SEI message framework and creates a dedicated syntax structure specifically for image format metadata. This specialization reduces the complexity of implementing AI-based video processing by providing a targeted solution rather than requiring general-purpose SEI message handling for all metadata types.
Solution Approach 2:
The patent introduces specific parameters (image_format_metadata_type_identifier, image_format_metadata_uri_flag) that change the state of the SEI message structure to accommodate image format metadata. These parameter changes enable the system to switch between different metadata carrying modes (embedded vs. external reference) thereby managing complexity through configurable parameters.
2Loss of information
If image format metadata is embedded in the video bitstream, then accessibility to image information is improved, but the bitrate requirements increase
Solution Approach 1:
The patent creates a dynamic system where the image_format_metadata_uri_flag can switch between two states: embedding metadata directly in the bitstream (when flag=0) or referencing external metadata (when flag=1). This dynamic approach allows the system to adapt bitrate consumption based on whether image format information is readily available externally, thereby managing the trade-off between information accessibility and bitrate requirements.
3Adaptability or versatility
If popular image metadata formats (EXIF, JFIF, XMP) are integrated into video streams, then compatibility with image processing applications is improved, but the device complexity increases
Solution Approach 1:
The patent creates a universal syntax structure that can accommodate multiple popular image metadata formats (EXIF, JFIF, XMP) through a single standardized interface. The image_format_metadata_type_identifier parameter allows the system to recognize and handle different metadata formats uniformly, thereby improving compatibility with image processing applications while reducing the complexity of implementing separate handling mechanisms for each format.
Data Source
AI summary
A method and apparatus comprising computer code for video processing, the method including setting an image format metadata type identifier, wherein the image format metadata type identifier indicates a type of metadata included in a supplemental enhancement information (SEI) message associated with a current image; signaling the image format metadata type identifier in a bitstream; and encoding the current image in the bitstream based on the image format metadata type identifier. The method may also include signaling an image format metadata universal resource identifier flag, wherein the image format metadata universal resource identifier flag indicates whether image information payload associated with the current image is obtained from a payload of the SEI message.