Mobile Audio Spatial Steering and Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conference audio management systems face delays and inability to adjust individual participant audio levels due to network mixing, which introduces synchronization issues and limits control over audio streams.
Innovation Solution
A mobile device manages audio streams by receiving and synchronizing individual audio streams from participants using buffers and delay controllers, allowing for independent adjustment of gain and spatial processing through head-related transfer functions (HRTFs), thereby bypassing network mixing and reducing delay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a network mixer generates a mixed audio stream from multiple participants, then audio processing is simplified at individual devices, but audio processing delays increase and individual participant audio adjustment becomes impossible
Solution Approach 1:
The patent segments the mixed audio stream into individual participant audio streams at the receiving device. Instead of receiving one pre-mixed stream from the network mixer, the device receives separate streams from each participant and performs local mixing. This segmentation eliminates the need for network-side decoding, mixing, and re-encoding operations, thereby reducing audio processing delays while maintaining simplified processing architecture.
2Device complexity
If a network mixer provides a single mixed stream to each participant, then network infrastructure is simplified, but individual audio stream synchronization becomes problematic
Solution Approach 1:
The patent delivers separate audio streams from each participant directly to the receiving device without network-side mixing. This segmentation allows the receiving device to independently manage and synchronize each stream using local buffers and timing mechanisms, thereby maintaining reliable synchronization even though network infrastructure remains relatively simple.
Solution Approach 2:
The patent introduces buffers as intermediary elements between the network mixer and the audio processing stage. These buffers temporarily store incoming audio streams and enable timing adjustments, allowing the system to compensate for network jitter and synchronization issues without requiring complex network infrastructure.
3Adaptability or versatility
If individual audio streams are received separately from each participant, then individual audio adjustment capability is enabled, but synchronization between streams becomes more difficult
Solution Approach 1:
The patent uses buffers as intermediary elements to manage individual audio streams. Each participant's audio stream is placed in a separate buffer that can be independently controlled. The buffers enable timing adjustments and synchronization without requiring complex real-time coordination mechanisms, thus allowing individual audio adjustment while managing synchronization complexity.
Solution Approach 2:
The patent adjusts timing parameters of individual audio streams by modifying buffer delivery times. By changing the release timing of audio packets from individual buffers, the system synchronizes multiple streams without requiring complex real-time coordination, enabling individual audio adjustment while keeping synchronization mechanisms manageable.
Data Source
AI summary
A method for managing audio during a conference includes steering, at a spatial steering processor of a mobile device, first decoded audio to be projected from a speaker at a first angle. The method also includes steering, at the spatial steering processor, second decoded audio to be projected from the speaker at a second angle. The first decoded audio corresponds to a decoded version of a first audio stream from a first device associated with a first participant of the conference, and the second decoded audio corresponds to a decoded version of a second audio stream from a second device associated with a second participant of the conference. The first decoded audio is synchronized with the second decoded audio.