Audio-to-Spikes Pipeline for Low-Power Speech Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Edge devices with limited computing capability struggle to perform efficient and real-time speech recognition due to their power constraints, despite the benefits of neuromorphic processors for speech recognition.
Innovation Solution
An audio to spikes conversion pipeline that converts speech into spikes for input to SNNs, optimized through hyperparameter sweeps, suitable for low-power neuromorphic processors, enabling efficient and real-time speech recognition on edge devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by stationary object
If traditional speech recognition techniques are used on edge devices, then speech recognition can be performed, but power consumption is high and computing capability is exceeded
Solution Approach 1:
The patent replaces traditional CPU/GPU-based speech recognition systems with neuromorphic processors that use spiking neural networks. This substitution of computational architecture enables speech recognition to run on edge devices with dramatically reduced power consumption while maintaining recognition capability, as neuromorphic processors are specifically designed to handle the computational requirements of SNNs efficiently
Solution Approach 2:
The patent transforms the computational parameters by converting audio data into spike trains that match the temporal and intensity characteristics of neural spikes. This parameter transformation allows the speech recognition system to operate in the neuromorphic domain, enabling efficient processing on low-power devices without sacrificing recognition accuracy
2Productivity
If audio data is converted to spikes using traditional methods, then speech recognition can be performed, but computational overhead and latency are high
Solution Approach 1:
The patent performs preliminary conversion of audio data into spike format using an optimized audio-to-spikes pipeline before processing by the SNN. This pre-processing step includes converting audio to Mel-frequency cepstral coefficients and then to spikes, which reduces the computational burden during real-time recognition and enables faster processing on edge devices
Solution Approach 2:
The patent segments the audio processing pipeline into distinct stages: audio preprocessing, Mel-frequency transformation, spike conversion, and SNN processing. This segmentation allows each stage to be optimized independently, with the spike conversion stage specifically designed to minimize computational overhead and latency for real-time operation
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for converting audio to spikes for input to a spiking neural network configured to recognize speech based on the spikes are described. In some aspects, a method includes obtaining audio data and generating frequency domain audio signals that represent the audio data by converting the audio data into a frequency domain. The frequency domain audio signals are mapped into a set of Mel-frequency bands to obtain Mel-scale frequency audio signals. A log transformation is performed on the Mel-scale frequency audio signals to obtain log-Mel signals. Spike input is generated for input to a spiking neural network (SNN) model by converting the log-Mel signals to the series of spikes. The spike input is provided as an input to the SNN model.