Media Switching Conference Server Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-party videoconferencing systems using a centralized transcoding MCU introduce latency due to the push model, limiting flexibility in user experience and user interface, as the MCU determines video stream quality and resolution for clients.
Innovation Solution
A method and apparatus that utilize a media switching conference server to negotiate video formats and connection information between client devices, allowing clients to subscribe to and transmit media streams based on their capabilities, optimizing resolution and bitrate to match client capabilities and requests, and enabling virtual streams to hide complexity from clients.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized transcoding MCU is used to determine video stream quality and resolution for clients, then the system can manage multi-party videoconferencing, but latency increases and flexibility in user experience is limited
Solution Approach 1:
Instead of the server pushing video streams to clients (centralized control), the invention inverts the model by allowing clients to pull streams they need based on their own capabilities and preferences. Clients subscribe to specific video streams from publishers, giving them control over quality and resolution while reducing server processing latency.
Solution Approach 2:
Clients independently determine their own video stream requirements and subscribe to appropriate streams without server intervention. Each client autonomously selects which video publishers to subscribe to and what quality levels to request, eliminating the need for centralized transcoding and layout composition while reducing latency.
2Extent of automation
If a centralized MCU implements audio mixing and video layout composition, then conference control is centralized, but client flexibility to override is reduced
Solution Approach 1:
The system segments control authority between server and clients. The server handles connection management and stream routing, while clients independently control their own video layout composition and stream selection. This segmentation allows clients to override server defaults and customize their experience based on local preferences and capabilities.
Solution Approach 2:
The system dynamically adapts to client capabilities and preferences. Clients can change their subscribed streams, quality requirements, and layout preferences at any time during the conference. The server dynamically routes streams based on real-time client subscriptions rather than enforcing fixed centralized layouts.
3Adaptability or versatility
If the MCU decodes and re-encodes video streams for layout composition, then video layout can be customized for each client, but processing delays are introduced
Solution Approach 1:
The invention extracts the video decoding and encoding operations from the centralized server and moves them to the client devices. Clients decode incoming streams locally and compose their own video layouts without server intervention, eliminating the server's decode-reencode cycle and its associated latency while maintaining layout customization capability.
Solution Approach 2:
The client device acts as an intermediary between the server and the user's display. Instead of the server composing video layouts and pushing them to clients, clients receive raw or minimally processed streams and independently compose their own layouts, with the client's video processing pipeline serving as the intermediary that eliminates server-side processing delays.
4Adaptability or versatility
If a non-transcoding MCU passes on all partial frames to all clients, then client flexibility is improved, but network bandwidth consumption increases
Solution Approach 1:
The system applies local quality adaptation where each client receives only the specific video streams and quality levels appropriate for its local capabilities and preferences. Clients subscribe to streams with resolutions and bitrates matched to their device capabilities, eliminating the need to send all possible quality versions to all clients while maintaining flexibility.
Solution Approach 2:
The system changes stream parameters (resolution, bitrate, format) based on client subscriptions rather than sending all possible variations. Clients indicate their desired stream parameters through subscription requests, and the server adjusts the transmitted stream parameters to match client requirements, optimizing bandwidth usage while maintaining adaptability.
Data Source
AI summary
A method including: establishing connections, at a server, to at least two client devices using a call control protocol, the call control protocol negotiating video formats and connection information for sending and receiving media streams; receiving information from a first client at the server, the information comprising meta-data describing different media streams the first client is configured to transmit; transmitting the information received from the first client to the at least one other client; receiving a subscribe message from the at least one other client at the server, subscribing to at least one available media stream from the first client; in response to receiving at least one subscribe message from the at least one other client, transmitting, by the server, a message instructing the first client to start transmitting media streams subscribed to by the at least one other client.