HEIF Overlay Image Transport in RTP Video Conferencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video streaming protocols, such as Real-time Transport Protocol (RTP), do not specify the carriage of High Efficiency Image File (HEIF) format images, which are crucial for overlay images in immersive teleconferencing, leading to challenges in sharing images between participants during video conferencing sessions.
Innovation Solution
Extensions to RTP payload format are developed to carry HEIF image collections and sequences, along with signaling in Session Description Protocol (SDP), enabling the transport and presentation of HEIF images during video conferencing sessions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing video streaming protocols (RTP) are used for transmitting overlay images, then the transmission can be performed with standard protocols, but the HEIF format images cannot be properly carried or specified in the protocol
Solution Approach 1:
The patent segments the RTP payload structure into distinct components: a header portion containing format identification and control information, and a payload portion containing the actual HEIF image data. This segmentation allows the protocol to handle HEIF format specifically while maintaining compatibility with existing RTP infrastructure.
Solution Approach 2:
The patent introduces an intermediary payload format structure that mediates between the HEIF image format and the RTP transport protocol. This intermediary structure includes specific fields for identifying HEIF format, storing image data, and providing control information, thereby enabling HEIF images to be transmitted through RTP without requiring changes to the core RTP protocol.
2Productivity
If HEIF format is used for overlay images, then efficient image compression and quality are achieved, but the protocol lacks specifications for carrying this format
Solution Approach 1:
The patent designs a universal payload format structure that can handle multiple image formats (not just HEIF) through a common framework. The header contains format identification fields that allow the same payload structure to accommodate different image encodings, thereby achieving multi-functionality without requiring separate protocol specifications for each format.
Solution Approach 2:
The patent utilizes parameter changes in the payload header to indicate different image formats and properties. By modifying header parameters such as format identification bytes and control flags, the protocol can adapt to transmit HEIF images efficiently while maintaining a consistent overall structure, thus managing complexity through parameterization rather than structural changes.
3Ease of operation
If overlay images are transmitted in real-time during video conferencing, then immersive teleconferencing experience is enhanced, but challenges arise in sharing images between participants
Solution Approach 1:
The patent incorporates preliminary action by including all necessary format identification, control, and metadata information in the RTP payload header before the actual image data transmission. This allows receiving systems to prepare for HEIF format processing in advance, ensuring reliable real-time transmission by eliminating format detection delays during playback.
Data Source
AI summary
An example device for retrieving media data, the device comprising: a memory configured to store scene data and image data; and one or more processors implemented in circuitry and configured to: receive data for a bitstream indicating that the bitstream includes one or more overlay images, the overlay images comprising either a single static image to be presented with scene data that remains constant while being presented frame-by-frame with frames of the scene data or an image sequence that repeats with a regular periodicity while being presented frame-by-frame with the frames of the scene data; receive the one or more overlay images from the bitstream; receive the scene data including one or more scene images; and present the scene images and the overlay images.