Video Audio Synchronization Over Heterogeneous Transports

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current synchronization techniques fail to provide sufficient quality of experience for video-calls over heterogeneous communications transports, as video and audio data transmitted over different networks often result in lip-sync inaccuracies due to differing characteristics and priorities.

Innovation Solution

Client devices and intervening servers dynamically adjust priority and Quality of Service (QoS) settings of different transports to minimize delay between video and audio data arrival, using synchronization indicia and hint data to coordinate synchronization, and employ a video-call synchronization application that adds and filters these indicia to ensure alignment.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If video data and audio data are transmitted over different communications transports, then the data capacity requirements are met, but synchronization accuracy deteriorates

Engineering Contradiction:

Improvedata capacityVSAvoidsynchronization accuracy

Core Design Contradiction:

Quantity of substanceVSManufacturing precision

Solution Approach 1:

The transmitting device adds synchronization indicia to the video and audio data packets before transmission. These indicia contain timing information that enables the receiving device to synchronize the data streams despite different transport characteristics

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Synchronization indicia act as an intermediary mechanism between the video and audio data streams. The indicia are embedded in both streams and serve as reference points that the receiving device uses to align the two different data streams in time

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If different communications transports are used for video and audio data, then network adaptability is improved, but quality of experience deteriorates

Engineering Contradiction:

Improvenetwork adaptabilityVSAvoidquality of experience

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system dynamically adjusts synchronization parameters based on the characteristics of the different transports being used. The receiving device uses the synchronization indicia to adaptively compensate for timing variations introduced by heterogeneous network conditions

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The synchronization indicia contain timing parameters that are adjusted according to the specific transport characteristics. This allows the system to maintain quality of experience across different network types by changing the synchronization parameters to match the transport conditions

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS9191413B2Synchronizing video and audio over heterogeneous transports

Publication Date: 2015.11.17 T MOBILE US INC

AI summary

Techniques to synchronize video data and audio data of a video-call over heterogeneous communications transports are disclosed. A transmitting client device adds synchronization indicia for a first communications transport to the video data and synchronization indicia for a second communications transport to the audio data. A receiving client device receives the video and audio data in a buffer, synchronizes the video and audio data, and removes the synchronization indicia for both the video and audio data with a filter. Disclosed techniques to optimize the synchronization include selectively dropping video frames in order to reduce delay, to reduce memory footprint, or to meet quality of service levels as set by a user.