Multi-Output RNN for Far-Field Speech Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automatic speech recognition systems face challenges in distinguishing between multiple audio signals with similar power levels, particularly in far-field microphone scenarios where speech from multiple speakers or background noise interferes with the target speech, leading to decreased recognition accuracy.
Innovation Solution
A multiple-output layer recursive neural network (RNN) is employed to process raw audio signals, generating signal-specific audio signals that represent individual speakers' utterances by using multiple output layers, each tracing a speaker or noise source, with shared layers and dynamic gate control to adjust the number of output layers based on speaker activity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Length of stationary object
If a far-field microphone is used to capture speech, then the system can understand speech from a distance, but the energy of competing speakers becomes very close to the energy of the target speaker, decreasing speech recognition accuracy
Solution Approach 1:
The patent segments the mixed audio signal into multiple speaker-specific signals using a multi-output RNN that learns to separate and trace individual speakers' speech patterns, voice characteristics, and temporal dynamics from the overlapping far-field recording
Solution Approach 2:
The patent introduces an intermediate processing stage with a multi-output RNN that acts as a mediator between the raw far-field audio and the ASR system, separating and enhancing target speaker speech while suppressing competing speakers and background noise before recognition
2Quantity of substance
If multiple speakers speak simultaneously with similar power levels, then the system captures all speakers' utterances, but the target to masking ratio decreases, making it difficult to distinguish the target speaker
Solution Approach 1:
The patent applies local quality by assigning different output layers to different speakers, with each output layer specialized to trace a specific speaker's characteristics, allowing the system to maintain high TMR for the target speaker while capturing multiple speakers simultaneously
Solution Approach 2:
The patent transitions from a single-output to a multi-output RNN architecture, adding the dimension of multiple independent output streams that can separately track and enhance different speakers' signals, effectively increasing the target to masking ratio for each speaker
3Device complexity
If a single-output RNN is used for speech processing, then the system structure is simple, but it cannot separate and trace individual speakers' speech from multi-speaker audio
Solution Approach 1:
The patent segments the single RNN into multiple output layers, where each output layer independently processes and traces a specific speaker's speech, maintaining a unified RNN structure while achieving multi-speaker separation through distributed output representations
Solution Approach 2:
The patent creates a universal multi-output RNN architecture that can handle multiple speakers simultaneously, where the shared hidden layers learn common speech patterns while the specialized output layers extract speaker-specific features, making the system adaptable to varying numbers of speakers
Data Source
AI summary
The technology described herein uses a multiple-output layer RNN to process an acoustic signal comprising speech from multiple speakers to trace an individual speaker's speech. The multiple-output layer RNN has multiple output layers, each of which is meant to trace one speaker (or noise) and represent the mask for that speaker (or noise). The output layer for each speaker (or noise) can have the same dimensions and can be normalized for each output unit across all output layers. The rest of the layers in the multiple-output layer RNN are shared across all the output layers. The result from the previous frame is used as input to the output layer or to one of the hidden layers of the RNN to calculate results for the current frame. This pass back of results allows the model to carry information from previous frames to future frames to trace the same speaker.