RTP Payload Extension for HEIF Image Overlay Transport
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video streaming protocols, such as Real-time Transport Protocol (RTP), lack specifications for carrying High Efficiency Image File (HEIF) format image collections and sequences, which are essential for immersive teleconferencing and telepresence applications, where static or repeating image overlays are used to enhance dynamic video content.
Innovation Solution
The proposed solution involves using the HEIF format to store image overlays and extending RTP payload formats for HEVC to enable the transmission of HEIF image collections and sequences, along with signaling through Session Description Protocol (SDP), allowing for the carriage of these images during video conferencing sessions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If RTP payload formats for HEVC are extended to carry HEIF image collections and sequences, then the adaptability and versatility of the transport protocol is improved, but the device complexity increases due to the need to process and handle multiple image formats and sequences
Solution Approach 1:
The patent segments the image data into distinct collections and sequences within the HEIF format, allowing the RTP protocol to handle different image types (static overlays, repeating sequences, progressive sequences) as separate, manageable units. Each image collection or sequence is packaged with specific metadata indicating its type and presentation parameters, enabling the receiver to process them independently according to their specific requirements.
2Manufacturing precision
If HEIF format is used to store and transmit image overlays, then the manufacturing precision and image quality are improved, but the ease of operation decreases due to the lack of existing RTP specifications for HEIF format
Solution Approach 1:
The patent extends the existing RTP payload format for HEVC to serve multiple functions: it can carry both traditional video streams and HEIF image collections/sequences using the same transport mechanism. By defining a unified payload structure that accommodates different image types (static, repeating, progressive) within a single specification framework, the patent makes HEIF format carriage as easy as existing HEVC video transmission while maintaining high image quality.
3Adaptability or versatility
If static and repeating image sequences are transmitted over dynamic video content, then the immersive experience is improved through overlay superimposition, but the loss of time increases due to the additional processing required to synchronize and present multiple content layers
Solution Approach 1:
The patent includes presentation timing information and synchronization metadata within the HEIF payload structure itself, allowing the receiving device to pre-configure the presentation schedule for image collections and sequences before actual playback. This preliminary setup enables seamless synchronization between static/repeating images and dynamic video content without real-time processing delays, as the presentation timeline is predetermined and embedded in the transmitted data.
Data Source
AI summary
An example device for retrieving media data, the device comprising: a memory configured to store scene data and image data; and one or more processors implemented in circuitry and configured to: receive data for a bitstream indicating that the bitstream includes one or more overlay images, the overlay images comprising either a single static image to be presented with scene data that remains constant while being presented frame-by-frame with frames of the scene data or an image sequence that repeats with a regular periodicity while being presented frame-by-frame with the frames of the scene data; receive the one or more overlay images from the bitstream; receive the scene data including one or more scene images; and present the scene images and the overlay images.