Sound Source Classification with Continuous Beamforming for Speech Isolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices struggle to isolate desired speech from multiple sound sources, including users and noise sources, during communication sessions, as they cannot accurately associate sound sources with speech.
Innovation Solution
Implementing a continuously active beamformer and classifier system that processes audio data to determine unique characteristics of sound sources, allowing for sound source classification and beamforming to isolate desired speech.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sound source localization is performed to separate audio data based on sound source, then spatial separation of sound sources is achieved, but the device cannot identify which sound source is associated with desired speech
Solution Approach 1:
The patent introduces an intermediary system consisting of a continuously active beamformer and a classifier. The beamformer acts as an intermediary that processes audio data to extract directional information, while the classifier serves as another intermediary that analyzes characteristics of sound sources to identify which ones contain desired speech. This intermediary system bridges the gap between sound source separation and speech identification.
Solution Approach 2:
The system implements feedback through the classifier that continuously analyzes sound source characteristics and provides identification information back to the beamforming process. The classifier receives source directions from the beamformer and uses this feedback to identify speech-containing sources, which then informs the beamformer's output selection. This closed-loop feedback mechanism enables accurate speech source identification while maintaining separation accuracy.
2Reliability
If a continuously active beamformer and classifier system is implemented to identify speech sources, then speech isolation capability is improved, but device complexity increases
Solution Approach 1:
The beamformer is designed to be continuously active and multi-functional, serving both as a sound source separator and as a feature extractor for the classifier. This universal component performs multiple functions simultaneously, reducing the need for separate dedicated components and thereby managing system complexity while maintaining reliable speech isolation capability.
Solution Approach 2:
The beamformer performs preliminary action by continuously processing audio data and extracting directional information before the classification stage. This preliminary processing prepares the data in advance, making the subsequent speech identification more efficient and reducing the computational burden on the classifier, thus managing overall system complexity.
3Measurement precision
If multiple sound sources are processed to determine unique characteristics, then speech classification accuracy is improved, but processing time increases
Solution Approach 1:
The classifier focuses on analyzing only the necessary characteristics of sound sources that are most indicative of speech content, rather than processing all possible features. This partial action approach concentrates computational resources on the most relevant features, improving speech classification accuracy while minimizing processing time by avoiding unnecessary computations.
Solution Approach 2:
The beamformer performs preliminary processing to extract and organize directional information and sound source characteristics before they reach the classifier. This preliminary organization of data reduces the processing time required by the classifier, as the input data is already structured and ready for classification, thereby maintaining high accuracy without excessive processing time.
Data Source
AI summary
A system configured to perform source classification using a continuously active beamformer and a classifier to improve beamforming. A device processes audio data representing sounds from multiple sound sources to determine a source direction corresponding to each detected sound source. For each audio frame, the classifier receives a source direction for each unknown sound source and uses the source direction to determine first spectral characteristics that are unique to the individual sound source. By comparing the first spectral characteristics to spectral characteristics associated with labeled sound sources, the device identifies a match and associates the source directions with a corresponding labeled sound source. For each labeled sound source, the classifier determines attributes of a corresponding signal, and the device uses these attributes to select a single sound source. Using a desired look direction associated with the selected sound source, the beamformer generates audio data representing desired speech.