Convolutional Neural Network Keyword Spotting with Frequency Striding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing keyword spotting systems on mobile devices face challenges in efficiency and accuracy due to high computational requirements and parameter counts, particularly when using deep neural networks (DNNs) for speech recognition tasks.
Innovation Solution
Implementing a convolutional neural network (CNN) architecture that employs frequency and time striding to reduce the number of multiplication operations and parameters, allowing for efficient keyword detection with improved performance by using a single CNN layer for pooling in both dimensions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep neural networks (DNNs) are used for keyword spotting, then the system can be adjusted by changing the number of parameters, but the computational requirements and parameter counts become excessively high for mobile devices
Solution Approach 1:
The patent changes the architectural parameters of the neural network from a standard DNN to a CNN structure with specific filter sizes, strides, and pooling operations. This parameter change reduces the total number of parameters while maintaining adaptability through configurable filter dimensions and stride values that can be adjusted for different keyword spotting requirements.
Solution Approach 2:
The patent segments the frequency dimension by using a frequency stride greater than one, which divides the frequency space into discrete intervals. This segmentation reduces the number of multiplication operations required while preserving the essential frequency information needed for keyword detection, thereby reducing computational complexity without sacrificing adaptability.
2Device complexity
If a CNN architecture with frequency stride greater than one is used, then the number of multiplication operations is reduced, but the computational efficiency and accuracy must be maintained
Solution Approach 1:
The patent compensates for the reduced resolution in frequency caused by striding by enhancing the temporal dimension through pooling operations and by utilizing multiple filters across the frequency bands. This dimensional redistribution maintains the overall information content and detection accuracy while reducing the total number of multiplication operations in the frequency domain.
Solution Approach 2:
The patent applies different processing strategies to different regions of the frequency spectrum by using localized filters that operate on specific frequency bands. This allows the system to focus computational resources on the most relevant frequency regions for keyword detection, maintaining accuracy while reducing overall computational load through selective processing.
3Device complexity
If the CNN pools in time to remain within computational constraints, then the number of parameters is reduced, but the temporal resolution may be affected
Solution Approach 1:
The patent applies pooling operations selectively in the time dimension rather than uniformly across all time steps. By pooling only in regions where temporal precision is less critical and maintaining higher resolution in regions where timing is important for keyword detection, the system reduces the total number of parameters while preserving essential temporal information.
4Device complexity
If a single CNN layer with pooling in both time and frequency is used, then the model size is reduced for mobile devices, but the performance must still achieve ten percent relative improvement
Solution Approach 1:
The patent merges the functions of multiple separate processing layers into a single CNN layer that simultaneously performs convolution with strided filters and pooling operations in both time and frequency dimensions. This consolidation reduces the overall model size and computational overhead while maintaining the functional capabilities needed to achieve performance improvement through efficient feature extraction.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for keyword spotting. One of the methods includes training, by a keyword detection system, a convolutional neural network for keyword detection by providing a two-dimensional set of input values to the convolutional neural network, the input values including a first dimension in time and a second dimension in frequency, and performing convolutional multiplication on the two-dimensional set of input values for a filter using a frequency stride greater than one to generate a feature map.