Mixed Media Stream Processing with Separation Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional audio and video conferencing systems face resource constraints and limitations when handling large numbers of users, particularly in generating and transmitting customized media streams, which can lead to bandwidth issues and restricted user control over individual streams.
Innovation Solution
A method and system for processing mixed media streams by receiving and mixing media data from multiple sources into a single stream, compiling metadata for stream separation, and transmitting this metadata to enable remote devices to extract and differently process individual streams, allowing for spatial localization and independent volume adjustment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If individual audio/video streams are relayed to all users in a conference, then user control over individual streams is improved, but bandwidth consumption increases considerably
Solution Approach 1:
The system segments the audio stream into multiple independent channels, each corresponding to a specific user or sound source. Metadata is generated to identify and separate these channels, allowing users to selectively control individual streams while receiving them through a single mixed audio transmission, thus reducing bandwidth while maintaining control capabilities
Solution Approach 2:
Metadata acts as an intermediary between the mixed audio stream and the user interface. This metadata contains identification information for each sound source and enables the system to separate and control individual streams without requiring separate audio transmissions for each user
2Quantity of substance
If a single customized audio stream containing mixed streams of all users is transmitted, then bandwidth consumption is reduced, but user control over individual streams is limited
Solution Approach 1:
The audio stream is segmented into identifiable channels through metadata tagging, allowing each user's contribution to be marked and subsequently controlled independently. This segmentation occurs in the data domain rather than requiring separate physical transmissions
Solution Approach 2:
Metadata serves as an intermediary layer that carries control information for individual streams within the mixed audio transmission. This metadata enables users to selectively enable or disable specific participants' audio without requiring separate audio channels
3Ease of operation
If customized audio/video streams are generated for each user in a conference, then user-specific stream control is improved, but resource consumption increases as the number of users increases
Solution Approach 1:
Multiple user-specific audio streams are merged into a single mixed audio stream for transmission. The metadata maintains individual stream identification, allowing the system to provide customized stream control without generating multiple separate customized audio streams, thus reducing computational resources while maintaining user-specific control capabilities
Data Source
AI summary
The disclosed computer-implemented method may include (1) receiving, at a first device, a first stream that includes first media data from a first media object, (2) receiving, at the first device, a second stream that includes second media data from a second media object, (3) mixing, at the first device, the first media data and the second media data into a third stream, (4) compiling, while mixing the third stream, a metadata stream that includes information enabling separation of the first media data and the second media data from the third stream, (5) transmitting, from the first device to a second device, the third stream, and (6) transmitting, from the first device to the second device, the metadata stream to enable the second device to separate the first media data and the second media data from the third stream. Various other methods, systems, and computer-readable media are also disclosed.