Neural Network Diarization for Multi-Speaker Audio Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speech recognition (ASR) techniques face challenges in accurately identifying individual voice activity in audio inputs with multiple speakers, especially when speech overlaps, leading to degraded performance and increased processing time.
Innovation Solution
A neural network-based audio analysis platform, referred to as a diarization neural network (DNN), is trained using reference audio data and diarization data to determine individual voice activity in multi-speaker audio inputs, enabling more accurate separation of speech and recognition of speaker identity, while reducing processing time for real-time analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing ASR techniques are used to identify individual voice activity in multi-speaker audio, then the system can process audio inputs, but the accuracy of identifying individual speakers degrades and processing time increases
Solution Approach 1:
The audio input is divided into multiple frames or segments that are processed independently by the neural network. Each frame is analyzed for voice activity detection and speaker identification, allowing parallel processing that reduces overall processing time while maintaining accuracy through consistent segmentation across all audio portions.
Solution Approach 2:
The neural network is pre-trained using reference audio data and reference diarization data before actual processing. This preliminary training phase enables the network to learn speaker characteristics and voice activity patterns in advance, so that during real-time processing, the network can quickly and accurately identify individual speakers without requiring complex runtime computations.
2Reliability
If existing ASR techniques are used for multi-speaker audio analysis, then the system can operate, but the performance degrades when speech overlaps occur
Solution Approach 1:
The neural network acts as an intermediary between the raw audio input and the final diarization output. It processes the overlapping speech signals through learned representations, transforming complex mixed audio into separated speaker attributions. The network handles the complexity of overlapping speech internally through its architecture and training, presenting a simplified and reliable output interface.
Solution Approach 2:
The system changes key parameters including the number of speakers, audio frame duration, and sampling rates between training and processing phases. By training on diverse reference data with varying parameters and applying appropriate transformations during processing, the system adapts to different multi-speaker scenarios and overlapping speech conditions without requiring complex scenario-specific processing logic.
Data Source
AI summary
An audio analysis platform may receive a portion of an audio input, wherein the audio input corresponds to audio associated with a plurality of speakers. The audio analysis platform may process, using a neural network, the portion of the audio input to determine voice activity of the plurality of speakers during the portion of the audio input, wherein the neural network is trained using reference audio data and reference diarization data corresponding to the reference audio data. The audio analysis platform may determine, based on the neural network being used to process the portion of the audio input, a diarization output associated with the portion of the audio input, wherein the diarization output indicates individual voice activity of the plurality of speakers. The audio analysis platform may provide the diarization output to indicate the individual voice activity of the plurality of speakers during the portion of the audio input.