Spatial Audio Alignment for Multi-Device Video Calls
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video call systems struggle with audio leakage and misalignment between audio directions and image positions when multiple participants use different devices in the same acoustic location, leading to errors in spatial audio rendering.
Innovation Solution
The system identifies participant devices in the same acoustic location, selects a primary device, separates audio signals into parts, associates these parts with corresponding images, and adjusts audio directions to align with image positions in the composite image, using techniques like blind source separation and spatial audio rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple participant devices in the same acoustic location are used to provide audio signals, then the spatial audio rendering can be enhanced, but audio leakage between devices causes misalignment between audio directions and image positions
Solution Approach 1:
The audio signal from the primary device is segmented into multiple parts, with each part associated with a specific participant device in the acoustic location. This segmentation allows precise control of audio direction for each device, counteracting the audio leakage effect by directing audio signals to match their corresponding image positions in the composite image.
Solution Approach 2:
The system adjusts the direction parameter of audio signals dynamically. By modifying the audio direction parameters to account for audio leakage between devices, the system ensures that audio from each participant device is rendered in the correct spatial position corresponding to its image in the composite image.
2Adaptability or versatility
If audio signals from multiple devices are combined, then spatial audio coverage is improved, but misalignment between audio directions and image positions occurs
Solution Approach 1:
The system uses feedback mechanisms to associate audio signal parts with corresponding images from participant devices. By continuously monitoring and adjusting the association between audio parts and image positions, the system maintains accurate audio-image alignment even when combining signals from multiple devices with different spatial configurations.
Solution Approach 2:
The system introduces an additional dimension of control by separating audio signals into multiple parts along the time-frequency domain. This allows independent adjustment of audio direction for each part, enabling precise alignment with corresponding images while maintaining comprehensive spatial coverage from multiple devices.
Data Source
Figure 1~2
Figure 3
Figure 4A
AI summary
Examples of the disclosure relate to spatial audio for video calls where two or more participants use different devices to participate in the video call but are in the same acoustic location. An apparatus identifies two or more participant devices of a video call in an acoustic location where the participant devices provide images for a composite image for one or more recipients in the video call. At least one of the identified participant devices provides audio signals for the video call such that the audio signals correspond to the composite image. The apparatus also determines positions of images from the identified participant devices in the composite image and associates the audio signals from the two or more participant devices with images from the identified participant devices in the composite image. The apparatus also adjusts the directions of audio signals from the identified participant devices to increase the angle from centre to counteract audio leakage between the identified participant devices and renders audio from the identified participant devices to the adjusted direction.