Multi-Beamformer Audio Source Enhancement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for enhancing sound sources from noisy recordings, such as beamforming and audio source separation, face challenges in real-time applications with limited processing power and are not effective in scenarios with high reverberation or unknown numbers of sources, particularly in mobile devices with few microphones.
Innovation Solution
A method utilizing multiple beamformers pointing to different spatial directions, combined with source localization and post-processing techniques, to enhance target sound sources by attenuating non-dominant signals and reducing noise, suitable for real-time applications with light computation load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If beamforming and audio source separation methods are used to enhance sound sources, then signal quality of target sound sources is improved, but processing power requirements increase and real-time performance deteriorates
Solution Approach 1:
The patent segments the audio signal processing into multiple beamformers pointing to different spatial directions, processing each beamformer output independently through source localization and post-processing. This segmentation allows parallel computation and reduces the computational burden on any single processing unit while maintaining overall signal quality enhancement.
Solution Approach 2:
The patent applies partial action by using a limited number of beamformers pointing to specific dominant source directions rather than exhaustive spatial coverage. This selective approach processes only the most relevant spatial components, reducing computational complexity while maintaining effective signal enhancement for the target sound sources.
2Measurement precision
If multiple beamformers pointing to different spatial directions are used, then noise attenuation and source isolation are improved, but device complexity increases
Solution Approach 1:
The patent implements multi-functionality by designing a unified processing pipeline that handles source localization, beamforming, and post-processing within a single system architecture. The same computational framework processes multiple beamformer outputs consistently, reducing overall system complexity compared to separate specialized systems.
Solution Approach 2:
The patent transitions from single-direction beamforming to multi-directional spatial processing by introducing angular dimensionality. Multiple beamformers operate in different spatial directions simultaneously, adding a directional dimension to the processing that improves noise attenuation without requiring proportional increases in computational complexity.
3Ease of operation
If existing audio enhancement methods are applied in mobile devices with few microphones, then portability is maintained, but effectiveness in high reverberation or unknown number of sources deteriorates
Solution Approach 1:
The patent adapts the enhancement method to mobile device constraints by adjusting parameters such as the number of beamformers, spatial direction sampling density, and processing depth. These parameter modifications enable effective operation with few microphones while maintaining portability, and the system can adapt parameters based on detected number of sources and reverberation conditions.
Solution Approach 2:
The patent implements dynamic adaptation by adjusting the number and directions of beamformers based on real-time source detection and reverberation estimation. When the number of sources or reverberation levels change, the system dynamically reconfigures the beamforming parameters to maintain effectiveness without requiring a fixed complex setup.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A recording is usually a mixture of signals from several sound sources. The directions of the dominant sources in the recording may be known or determined using a source localization algorithm. To isolate or focus on a target source, multiple beamformers may be used. In one embodiment, each beamformer points to a direction of a dominant source and the outputs from the beamformers are processed to focus on the target source. Depending on whether the beamformer pointing to the target source has an output that is larger than the outputs of other beamformers, a reference signal or a scaled output of the beamformer pointing to the target source can be used to determine the signal corresponding to the target source. The scaling factor may depend on a ratio of the output of the beamformer pointing to the target source and the maximum value of the outputs of the other beamformers.