GRU-Based Covariance Estimation for Stable MVDR Speech Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning based speech enhancement and separation methods face challenges with high residual noise levels, especially at low signal-to-noise ratios and in overlapped speech scenarios, due to instability in matrix inversion and principal component analysis when jointly trained with neural networks, affecting the performance of automated speech recognition systems.
Innovation Solution
Employing gated recurrent unit-based networks to estimate covariance matrices and generate predicted target waveforms using a minimum variance distortionless response function, which stabilizes the matrix inversion and principal component analysis, allowing for frame-wise weight adaptation in the mask-based MVDR beamforming framework.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning based speech enhancement and separation methods are used, then speech recognition performance is improved, but residual noise levels increase especially at low signal-to-noise ratios
Solution Approach 1:
The patent segments the speech enhancement task into multiple components: covariance matrix estimation using Gated Recurrent Units, MVDR beamforming weight calculation, and spectral masking. This segmentation allows each component to be optimized independently, with the GRU-based covariance estimation specifically addressing the noise problem while maintaining recognition performance.
Solution Approach 2:
The patent introduces an intermediary Gated Recurrent Unit-based network that mediates between the raw audio input and the MVDR beamforming process. This intermediary component provides stable covariance matrix estimates that reduce residual noise while maintaining the benefits of deep learning-based speech enhancement.
2Adaptability or versatility
If matrix inversion and principal component analysis are jointly trained with neural networks, then speech separation capability is enhanced, but training stability deteriorates
Solution Approach 1:
The patent extracts the matrix inversion and principal component analysis operations from the joint training process with neural networks. By using GRU-based networks to estimate covariance matrices separately, the unstable matrix operations are taken out of the end-to-end training process, improving stability while maintaining speech separation capability.
Solution Approach 2:
The patent performs preliminary covariance matrix estimation using GRU-based networks before applying MVDR beamforming. This preliminary action stabilizes the matrix inversion process by providing pre-processed, stable covariance estimates, allowing the subsequent speech separation to proceed without training instability.
3Object-generated harmful factors
If mask-based MVDR beamforming is used, then speech distortion is reduced, but automated speech recognition performance is affected at low signal-to-noise ratios
Solution Approach 1:
The patent changes the parameters used in MVDR beamforming by using GRU-based networks to estimate time-varying covariance matrices instead of using fixed or简单地 estimated parameters. This allows the beamformer to adapt to changing acoustic conditions while maintaining low speech distortion, improving recognition performance at low signal-to-noise ratios.
Data Source
AI summary
A method, computer program, and computer system is provided for automated speech recognition. Audio data corresponding to one or more speakers is received. Covariance matrices of target speech and noise associated with the received audio data are estimated based on a gated recurrent unit-based network. A predicted target waveform corresponding to a target speaker from among the one or more speakers is generated by a minimum variance distortionless response function based on the estimated covariance matrices.