Multichannel Neural Network Spatial Spectral Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Speech recognition performance degrades in environments with reverberation and additive noise, especially when microphones are far from speakers, and existing systems struggle to effectively enhance the speech signal and reduce noise.
Innovation Solution
A neural network-based speech recognition system incorporating a spatial filtering convolutional layer and a spectral filtering convolutional layer processes multichannel audio signals from multiple microphones to generate spatial and spectral filtered outputs, which are then processed to predict sub-word units, using multi-task learning to suppress noise and de-reverberate features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple microphones are used to capture speech signals, then speech recognition accuracy can be improved, but the system becomes more susceptible to reverberation and additive noise
Solution Approach 1:
The patent segments the speech recognition system into multiple independent convolutional layers, each specializing in different filtering functions (spatial filtering layer for directional signal enhancement, spectral filtering layer for frequency-domain noise reduction). This segmentation allows each layer to independently process specific aspects of the multichannel audio signal, thereby improving speech recognition accuracy while systematically addressing reverberation and noise through dedicated processing pathways.
2Device complexity
If a single neural network processes audio signals, then the system structure is simple, but it cannot effectively separate spatial and spectral filtering functions
Solution Approach 1:
The patent extends the neural network architecture from a single processing dimension to multiple hierarchical dimensions by introducing separate spatial filtering and spectral filtering convolutional layers. The spatial filtering layer operates on the channel dimension to extract spatial features and suppress reverberation, while the spectral filtering layer operates on the frequency dimension to reduce additive noise. This multi-dimensional approach enhances signal processing effectiveness while maintaining a structured and manageable network architecture.
3Adaptability or versatility
If microphones are placed far from speakers, then the system has flexible deployment options, but speech recognition performance degrades due to increased reverberation and noise
Solution Approach 1:
The patent introduces specialized convolutional filtering layers as intermediary processing stages between the microphone input and the speech recognition model. These intermediary layers act as mediators that actively process the multichannel audio signals to suppress reverberation and additive noise, thereby maintaining speech recognition performance even when microphones are positioned at greater distances from speakers, thus preserving deployment flexibility.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using neural networks. One of the methods includes receiving, by a neural network in a speech recognition system, first data representing a first raw audio signal and second data representing a second raw audio signal, the first raw audio signal and the second raw audio signal for the same period of time, generating, by a spatial filtering convolutional layer in the neural network, a spatial filtered output the first data and the second data, generating, by a spectral filtering convolutional layer in the neural network, a spectral filtered output using the spatial filtered output, and processing, by one or more additional layers in the neural network, the spectral filtered output to predict sub-word units encoded in both the first raw audio signal and the second raw audio signal.