Dynamic Speaker Selection in Multi-Party Video Conferencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing videoconferencing technologies face scalability issues due to the need for separate connections between each pair of users in multi-party conferencing, leading to resource-intensive requirements that do not scale well, and the use of relays can result in overloaded systems and poor call quality.
Innovation Solution
An overlay network platform utilizing WebRTC protocols allows for dynamic speaker selection and live stream delivery, where streams are prioritized based on relevance, with clients determining the quality of streams they receive based on their local resources, and multiplexing is handled in the cloud to reduce bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If full mesh connectivity is established between all users for multi-party video conferencing, then each user can receive high-quality streams from all participants, but the resources required grow proportionally to the square of the number of users, which does not scale
Solution Approach 1:
The patent segments the full mesh connectivity model into individual peer-to-peer connections maintained by a server. Instead of requiring O(n²) direct connections between all users, each user establishes only one connection to the server, reducing the connection complexity from quadratic to linear scaling. The server acts as a stream multiplexer that distributes relevant streams to each user without requiring direct connections between all user pairs.
Solution Approach 2:
The patent introduces a server as an intermediary that mediates between all users. The server receives streams from participants, determines relevance based on speaker detection, and selectively distributes streams to users. This intermediary eliminates the need for direct peer-to-peer connections between all users, enabling scalable multi-party conferencing while maintaining stream quality through intelligent selection and delivery.
2Reliability
If relays are used to establish connections between peers behind NATs, then connectivity is ensured, but the relay infrastructure becomes overloaded, significantly impacting call quality
Solution Approach 1:
The patent extracts the relay function from the traditional peer-to-peer model and consolidates it into a centralized server that performs stream multiplexing and distribution. Instead of relying on overloaded relays to forward all streams between all users, the server selectively receives and distributes only relevant streams based on speaker detection, reducing relay load and maintaining call quality.
Solution Approach 2:
The patent changes the operational parameters of the relay system by implementing intelligent stream selection based on speaker detection and user relevance. Instead of relaying all streams to all users (high load, low quality), the system dynamically adjusts which streams are relayed to which users based on current speaking activity and user preferences, reducing overall relay load while maintaining or improving call quality for relevant streams.
3Ease of operation
If separate connections are established for each pair of users, then direct peer-to-peer communication is achieved, but the complexity and resource requirements do not scale for large numbers of participants
Solution Approach 1:
The patent merges multiple peer-to-peer connections into a single connection to a server for each user. Instead of requiring O(n²) direct connections between all user pairs, each user establishes one connection to the server that handles all communication with other participants. This combining approach maintains the simplicity of direct connection capability while reducing connection management complexity from quadratic to linear scaling.
Data Source
AI summary
An overlay network platform facilitates a multi-party videoconference. End users participate using client-based browser or mobile application rendering software, such as a web browser, and using a protocol such as WebRTC. Dynamic speaker selection among the participants is facilitated by continuously determining the “relevance” of a participant's stream with respect to other participants in the conference. Thus, for example, a particular participant stream might be deemed to have a higher relevance if it is the stream corresponding to a person that is currently speaking, that has spoken recently, or that is about to speak. Streams with higher relevance are then selectively delivered at higher quality relative to less relevant streams. Preferably, individual client machines determine which streams are requested and at which quality based in part on their local resource capabilities.