Mixed Voice Chat Audio Masking for Blocked Avatars
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In virtual environments, muting individual audio streams is inefficient when audio streams are mixed together, as precise timing and quality matching is required for subtraction, which undermines the efficiency of mixing all streams into a single stream.
Innovation Solution
Generate additional audio at the client device that matches the location and orientation of the muted avatar, using synthetic speech to drown out the muted player's voice, thereby effectively muting the abusive participant without requiring precise timing and quality matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If audio streams are mixed together and sent as a single mixed stream, then network bandwidth efficiency is improved (scales as N streams), but the ability to mute individual participants is lost
Solution Approach 1:
The patent segments the audio processing into two parts: (1) mixing all audio streams into a single combined stream for efficient transmission, and (2) providing side information data that identifies which audio segments correspond to which participants. This segmentation allows the system to maintain both the efficiency of mixed streaming and the capability to control individual streams through the side information.
Solution Approach 2:
The patent introduces side information data as an intermediary element that bridges the gap between mixed audio streams and individual participant identification. This side information acts as a mediator that enables clients to selectively mute individual participants by providing metadata about which audio segments belong to which participants, without requiring separate audio streams for each participant.
2Adaptability or versatility
If individual audio streams are maintained separately to enable muting, then individual stream control is possible, but network bandwidth efficiency deteriorates (scales as N2 streams)
Solution Approach 1:
The patent merges all individual audio streams into a single mixed audio stream for transmission, combining multiple streams into one to improve network efficiency. Simultaneously, it combines side information data with the mixed stream that provides the metadata needed to identify and control individual participant audio segments, thus achieving both efficiency and control.
Solution Approach 2:
The mixed audio stream serves multiple functions: it efficiently transmits all participant audio over the network while the accompanying side information enables individual participant control. This multi-functionality allows a single mixed stream to replace what would otherwise require multiple separate streams, achieving both bandwidth efficiency and individual stream control capability.
3Productivity
If audio streams are mixed together, then processing efficiency is improved, but the precision required for stream subtraction increases (timing and quality must match precisely)
Solution Approach 1:
The side information data acts as an intermediary that provides precise timing and quality metadata without requiring actual audio stream subtraction. This side information enables the system to identify and mute specific participants by referencing the metadata rather than performing complex real-time audio subtraction operations, thus maintaining processing efficiency while avoiding the precision requirements of stream subtraction.
4Adaptability or versatility
If stream subtraction is performed to mute a participant, then individual muting is achieved, but computational complexity increases and processing efficiency decreases
Solution Approach 1:
The patent extracts the essential information needed for individual participant control (timing, quality, and identification metadata) from the mixed audio stream and places it in side information data. This extraction allows the system to achieve individual muting capability by referencing the extracted metadata rather than performing computationally intensive real-time audio stream subtraction operations.
Solution Approach 2:
Instead of performing complex real-time audio stream subtraction, the patent uses side information data that copies or references the essential metadata about each participant's audio contribution. This copying approach allows individual muting to be achieved by referencing pre-analyzed metadata rather than performing real-time computational subtraction, significantly reducing processing complexity.
Data Source
AI summary
A metaverse application receives encoded audio that includes a first audio stream associated with a first avatar in a virtual environment and a first voice-activity detection (VAD) signal for the first audio stream, and a second audio stream associated with a second avatar in the 3D virtual environment and a second VAD signal for the second audio stream. The metaverse application determines that the first avatar is blocked by a user associated with the user avatar. The metaverse application determines that the first VAD signal indicates that the first audio stream includes speech. The metaverse application generates additional audio. The metaverse application mixes the additional audio with the encoded audio. The metaverse application provides the mixed audio to a speaker for output.