Microphone Selection via TDOA Clustering for Speaker Diarization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current automatic speech recognition (ASR) systems face challenges in multi-microphone scenarios due to noise, reverberation, and unknown microphone positions, limiting their ability to accurately segment speakers in single-microphone environments.

Innovation Solution

The method involves determining time delays of arrival (TDOAs) between audio signals from multiple microphones, clustering them to distinguish between audio sources and interference, and selecting the best microphone pair based on confidence measures to enhance speaker segmentation and diarization accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If TDOA is used for microphone selection in multi-microphone scenarios, then spatial information can be utilized to improve speaker diarization, but noise, reverberation, and head movements create excessive noise that degrades TDOA estimates

Engineering Contradiction:

ImproveTDOA estimation accuracyVSAvoidnoise, reverberation, head movements

Core Design Contradiction:

Measurement precisionVSObject-affected harmful factors

Solution Approach 1:

The system changes the parameter being measured from raw TDOA values to confidence scores derived from TDOA clustering. By transforming the measurement parameter and using statistical clustering to aggregate multiple TDOA estimates, the system reduces the impact of transient noise and reverberation on the final microphone selection decision.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system computes TDOA for multiple microphone pairs (creating copies of the measurement process) and clusters these estimates together. By taking multiple copies of TDOA measurements from different microphone combinations and aggregating them through clustering, the system obtains a more robust estimate that is less sensitive to individual noisy measurements.

Inventive Principle:

Principle #26Copying

2Area of stationary object

If microphones located at long distance from audio source are used, then spatial coverage is improved, but TDOA estimates become unreliable

Engineering Contradiction:

Improvespatial coverageVSAvoidTDOA estimate reliability

Core Design Contradiction:

Area of stationary objectVSReliability

Solution Approach 1:

The system merges TDOA estimates from multiple microphone pairs through clustering. By combining information from multiple microphone combinations including those at long distances, the system achieves both broad spatial coverage and reliable estimates through the aggregation effect of clustering, which filters out unreliable individual measurements.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system uses confidence scores as feedback to evaluate the quality of TDOA estimates from different microphone pairs. This feedback mechanism allows the system to identify and weight reliable estimates more heavily, compensating for the reduced reliability of distant microphones while maintaining their contribution to spatial coverage.

Inventive Principle:

Principle #23Feedback

3Device complexity

If single-microphone scenario is used, then device complexity is reduced, but speaker segmentation accuracy deteriorates due to inability to utilize spatial information

Engineering Contradiction:

Improvemicrophone system complexityVSAvoidspeaker segmentation accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The system segments the audio signal processing into distinct stages: TDOA computation for each microphone pair, clustering of TDOA estimates, confidence score calculation, and final microphone selection. This segmentation allows the system to apply complex spatial processing only where needed while maintaining simplicity in single-microphone scenarios, thus balancing complexity and accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system designs a universal framework that can operate with either single or multiple microphones. The TDOA-based microphone selection methodology is universally applicable across different microphone configurations, allowing the system to automatically adapt to available hardware and provide enhanced speaker segmentation accuracy when multiple microphones are present, while degrading gracefully to single-microphone operation when needed.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS10847171B2Method for microphone selection and multi-talker segmentation with ambient automated speech recognition (ASR)

Publication Date: 2020.11.24 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Disclosed methods and systems are directed to determining a best microphone pair and segmenting sound signals. The methods and systems may include receiving a collection of sound signals comprising speech from one or more audio sources (e.g., meeting participants) and/or background noise. The methods and systems may include calculating a TDOA and determining, based on the TDOA and via robust statistics, the best pair of microphones. The methods and systems may also include segmenting sound signals from multiple sources.