Programmable Renderer Service for Composed Real-Time Media

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing real-time media communication systems, particularly those using Selective Forwarding Units (SFUs), face increased processing costs and network usage due to the need to send all media feeds to each receiving participant, lacking flexibility in composing and mixing media streams, and limited capabilities in incorporating supplementary information and visual effects in recordings.

Innovation Solution

Implementing a Programmable Renderer Service Application (PRSA) that generates a composed media feed by programmatically mixing sender feeds and supplementary information, reducing network load by sending a single composed stream to each participant, and allowing for customizable visual effects and supplementary information integration.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If all individual media streams are sent to each receiving participant in real-time media communication, then each endpoint can arrange media streams flexibly, but processing costs and network usage increase

Engineering Contradiction:

Improveflexibility in arranging media streamsVSAvoidprocessing costs and network usage

Core Design Contradiction:

Adaptability or versatilityVSLoss of energy

Solution Approach 1:

The patent merges multiple individual media streams into a single composed media stream that contains all participant videos arranged in a grid layout. This composed stream is then sent to all receiving participants, eliminating the need to send multiple separate streams to each endpoint while maintaining the ability to display all participants' videos.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The server implements a universal media composition service that can generate a single composed media stream suitable for all receiving participants. This multi-functional approach allows the system to serve multiple endpoints with the same composed stream, reducing network bandwidth consumption and processing overhead while still providing each participant with access to all other participants' videos.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Productivity

If a Selective Forwarding Unit is used instead of a Multipoint Control Unit, then processing burden on the server is reduced, but flexibility in composing and mixing media streams is limited

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidflexibility in composing media streams

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent segments the media processing into two distinct parts: (1) the SFU performs efficient packet forwarding and basic stream selection without complex composition operations, and (2) a dedicated media composition service handles the flexible arrangement and mixing of media streams. This segmentation allows the SFU to maintain high processing efficiency while the composition service provides the needed flexibility.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a media composition service as an intermediary component between the SFU and the receiving participants. This intermediary receives individual media streams from the SFU, composes them into arranged layouts (such as grid views), and then distributes the composed streams. This mediator layer adds flexibility in media composition without burdening the SFU with complex processing tasks.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Ease of operation

If individual media streams are preserved independently for each endpoint, then endpoints can process streams individually, but network bandwidth consumption increases

Engineering Contradiction:

Improveindividual stream processing capabilityVSAvoidnetwork bandwidth consumption

Core Design Contradiction:

Ease of operationVSLoss of energy

Solution Approach 1:

The patent combines multiple individual media streams into a single composed stream that contains all participant videos arranged in a grid layout. This composed stream is then sent to all receiving participants, eliminating the need to send multiple separate streams to each endpoint while maintaining the ability to display all participants' videos.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS20250330504A1Systems and methods for providing media communication programmable services

Publication Date: 2025.10.23 VONAGE BUSINESS INC

AI summary

Systems and methods for providing real-time media communication services to make use of a software application resident on a server that receives the media feeds of multiple sending participants, and generates a single composed media feed that includes media feeds of the sending participants and that sends the composed media feed to other computing services for manifold purposes like recording, re-broadcasting and/or re-transmission to remote computing devices of multiple real-time media communication participants. The composed media feed can include supplementary information in addition to the media feeds of live participants. This supplementary information is provided by means of API configurable programmatic code that is then executed and used as the software application resident on the server.