Mixed-Media Transport Protocol for Real-Time XR Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video communication technologies struggle to synchronize the presentation of multiple media streams, such as audio and virtual avatar animations, in real-time communication sessions, leading to suboptimal user experiences.
Innovation Solution
A transport protocol and payload format are developed to support various media types, including audio, video, and extended reality data, ensuring synchronization through header data that indicates media types and attributes, enabling accurate animation representation in real-time communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple media streams are transmitted separately using existing video communication technologies, then each media stream can be transmitted independently, but the presentation of multiple media streams cannot be synchronized
Solution Approach 1:
The patent combines multiple media stream transmissions into a single unified protocol framework. The RTP payload format integrates audio, video, and XR media types with common header structures and synchronization mechanisms, allowing all media streams to be transmitted and synchronized through one protocol rather than requiring separate protocols for each media type.
Solution Approach 2:
The patent creates a universal transport protocol that handles multiple media types (audio, video, XR) through a single standardized format. The RTP payload format defines media type indicators and attribute fields that can accommodate different media formats, making the protocol multi-functional and eliminating the need for separate specialized protocols for each media type.
2Reliability
If a unified transport protocol is implemented to synchronize multiple media streams, then presentation synchronization is achieved, but the protocol structure becomes more complex
Solution Approach 1:
The patent segments the protocol structure into distinct functional components: common RTP headers, media type indicator fields, attribute fields for each media type, and payload data sections. This segmentation allows the complex synchronization functionality to be organized into manageable, modular elements that can be processed independently while maintaining overall coordination.
Solution Approach 2:
The patent uses parameter fields within the protocol structure to dynamically indicate media types and attributes. By changing the values of these parameters (media type indicators, attribute flags), the protocol can adapt to handle different media formats without requiring fundamentally different protocol structures, thus managing complexity through parameterization rather than structural multiplication.
3Ease of operation
If existing video communication technologies are used, then implementation is simpler, but user experience deteriorates due to lack of synchronization in real-time communication
Solution Approach 1:
The patent introduces synchronization metadata as an intermediary element within the RTP payload format. This metadata carries timing and synchronization information that mediates between the transmitted media streams and the playback system, enabling coordinated presentation of audio, video, and XR content without requiring complex changes to the underlying transmission infrastructure.
Data Source
AI summary
An example device for processing media data includes a memory configured to store media data; and a processing system comprising one or more processors implemented in circuitry, the processing system being configured to: determine one or more user interactions with a virtual scene, each of the one or more user interactions corresponding to a particular media type of one or more media types; construct a packet including data for a media communication session, the packet including header data indicating the one or more media types for the one or more user interactions and one or more attributes for each of the one or more media types; and send the packet to a device involved in the media communication session.