Multi-Channel Acoustic Model Frequency Domain Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multichannel acoustic models for speech recognition systems face high computational complexity, which hinders their efficiency and scalability, despite providing improved accuracy over single-channel models.
Innovation Solution
The implementation of optimizations such as reducing the number of look directions, modifying convolution operation parameters, and processing audio data in the frequency domain instead of the time domain, which reduces computational complexity while maintaining accuracy by replacing time-domain convolutions with element-wise multiplications in the frequency domain.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-channel acoustic models are used to process audio data from multiple microphones, then speech recognition accuracy is improved, but computational complexity increases significantly
Solution Approach 1:
The patent replaces time-domain convolution operations with frequency-domain element-wise multiplications. This substitution transforms computationally intensive mechanical operations (convolutions requiring multiple multiplications and additions) into simpler algebraic operations, dramatically reducing computational complexity while preserving the acoustic modeling functionality.
Solution Approach 2:
The patent modifies convolution operation parameters by increasing the stride value and reducing the number of look directions. These parameter changes decrease the quantity of mathematical operations required, directly reducing computational complexity while maintaining adequate speech recognition performance.
2Measurement precision
If time-domain convolution operations are used for spatial filtering, then processing accuracy is maintained, but computational requirements increase
Solution Approach 1:
The patent substitutes time-domain convolution with frequency-domain element-wise multiplication for spatial filtering operations. This replacement maintains the accuracy of spatial filtering by preserving the underlying signal processing mathematics while dramatically reducing the computational power required to execute the operations.
3Measurement precision
If the number of look directions is increased, then directional speech recognition accuracy improves, but computational complexity increases
Solution Approach 1:
The patent reduces the number of look directions parameter in the multi-channel model. This parameter change decreases the quantity of computations required for spatial filtering across different directions, improving processing efficiency while maintaining adequate directional speech recognition accuracy through optimized frequency-domain processing.
Data Source
AI summary
This specification describes computer-implemented methods and systems. One method includes receiving, by a neural network of a speech recognition system, first data representing a first raw audio signal and second data representing a second raw audio signal. The first raw audio signal and the second raw audio signal describe audio occurring at a same period of time. The method further includes generating, by a spatial filtering layer of the neural network, a spatial filtered output using the first data and the second data, and generating, by a spectral filtering layer of the neural network, a spectral filtered output using the spatial filtered output. Generating the spectral filtered output comprises processing frequency-domain data representing the spatial filtered output. The method still further includes processing, by one or more additional layers of the neural network, the spectral filtered output to predict sub-word units encoded in both the first raw audio signal and the second raw audio signal.