Distributed Audio Mixing via Global Stream Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale multimedia conferences face inefficiencies in audio stream mixing and distribution across multiple servers, leading to increased delay and disruption in conference sessions.
Innovation Solution
A distributed media server system where each server determines local top streams based on importance, creates a global list, and forwards only those streams to other servers for local mixing and output, reducing the need for a global mixer and minimizing delay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a global mixer is used to mix all audio streams in large-scale multimedia conferences, then audio distribution can be achieved, but delay increases and network traffic is excessive
Solution Approach 1:
The patent divides the centralized global mixing function into multiple distributed mixing operations across different servers. Each server performs local mixing of audio streams independently, eliminating the need for a single global mixer that causes delay. This segmentation of the mixing function allows parallel processing and reduces overall delay while maintaining audio distribution reliability.
Solution Approach 2:
The patent extracts only the necessary audio streams (top N streams based on importance) from the complete set of available streams for mixing and distribution. By taking out only the most important streams rather than mixing all streams, the system reduces network traffic and processing delay while still providing adequate audio coverage for conference participants.
2Reliability
If all audio streams are forwarded across the network for mixing, then complete audio coverage is achieved, but network traffic increases significantly
Solution Approach 1:
The patent extracts and identifies only the top N most important audio streams based on various criteria (speaker activity, user importance, etc.) and forwards only these selected streams across the network. This extraction approach maintains adequate audio coverage by ensuring the most relevant streams are distributed, while dramatically reducing network traffic compared to forwarding all available streams.
Solution Approach 2:
The patent applies different quality levels to different audio streams by prioritizing certain streams over others. The top N streams receive full distribution and mixing resources, while less important streams are either not distributed or received at lower quality. This local quality differentiation maintains essential audio coverage while optimizing network resource usage.
3Reliability
If multiple servers each mix all received streams, then audio completeness is improved, but processing complexity and delay increase
Solution Approach 1:
The patent segments the audio mixing task so that each server is responsible for mixing only the top N streams rather than all received streams. This segmentation reduces the complexity of mixing operations at each server while maintaining audio completeness through the distributed architecture. Each server performs a simplified mixing function on a subset of streams, reducing overall processing complexity.
Solution Approach 2:
The patent applies partial action by having each server mix only the most important N streams rather than performing complete mixing of all available streams. This partial mixing approach at each server reduces processing complexity and delay, while the distributed nature of multiple servers performing similar partial mixing ensures adequate audio completeness is achieved across the system.
Data Source
AI summary
A scalable media conferencing system minimizes delay while maintaining a consistent session across all of the distributed media servers handling the conference session. Each of the media servers sends a list of the local top streams and their importance. A global list of top streams is created and disseminated to all of the media servers. Each of the media servers then forwards any local stream that is on the list of global top streams to all of the other media servers. Once a media server receives all of the streams on the list of global top streams, it locally mixes the streams together and outputs the mixed stream to the endpoints that are locally connected to that media server.