Spatial Audio Filtering for AR Conferencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current communication systems face challenges in effectively filtering and balancing real-world and virtual sounds during audio/video conferencing, particularly in distinguishing and managing near-field and far-field sounds within augmented reality environments, leading to suboptimal audio experiences.
Innovation Solution
A conferencing system employing a spatially-diverse microphone array and augmented reality display that captures and processes real-world and virtual audio sources, using audio-focus and background-audio layers to selectively amplify or dampen sounds based on their positioning within the physical space, allowing users to control audio treatment policies through user-defined regions and selections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If noise cancellation is used to filter far-field noise and remove echo, then near-field sounds are emphasized, but the ability to capture and process virtual audio sources in augmented reality environments is compromised
Solution Approach 1:
The audio processing system is segmented into multiple independent layers: an audio-focus layer for near-field sounds and a background-audio layer for far-field and virtual sounds. This segmentation allows each layer to be processed independently with different filtering and amplification strategies, resolving the contradiction between emphasizing near-field sounds and processing virtual audio sources.
Solution Approach 2:
The patent introduces spatial audio positioning as an intermediary mechanism that bridges real-world and virtual audio sources. By determining the spatial position of audio sources and mapping virtual sources to physical locations, the system can apply appropriate filtering and amplification based on position, enabling both near-field emphasis and virtual source processing.
2Measurement precision
If spatially-diverse microphone arrays are used to capture real-world and virtual audio sources, then audio source positioning is improved, but device complexity increases
Solution Approach 1:
The spatially-diverse microphone array is designed to serve multiple functions simultaneously: capturing real-world sounds, capturing virtual audio sources, and providing spatial positioning information. This multi-functionality reduces the need for separate systems and justifies the increased complexity by delivering comprehensive audio processing capabilities.
Solution Approach 2:
The system transitions from traditional two-dimensional audio processing to three-dimensional spatial audio processing by incorporating vertical and horizontal positioning data. This dimensional expansion enables more precise audio source localization and creates new possibilities for audio treatment based on spatial coordinates.
3Measurement precision
If audio-focus and background-audio layers are used to selectively amplify or dampen sounds, then audio clarity is enhanced, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary spatial positioning and classification of audio sources into audio-focus and background-audio layers before applying filtering and amplification. By pre-organizing audio sources based on their spatial characteristics and importance, the system reduces the computational complexity of subsequent processing steps and enables faster real-time audio enhancement.
Data Source
AI summary
A conferencing system includes a display device that displays video received from a remote communication device of a communication partner. An audio stream is transmitted to the remote communication device. The audio stream includes real-world sounds produced by one or more real-world audio sources captured by a microphone array and virtual sounds produced by one or more virtual audio sources. A relative volume of sounds in the audio stream is selectively adjusted based, at least in part, on real-world positioning of corresponding audio sources, including real-world and/or virtualized audio sources.