Broadcasted Residual Learning for Keyword Spotting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning architectures for keyword spotting (KWS) face challenges in achieving high accuracy while maintaining efficient model designs, particularly in edge devices with limited resources, where low latency is also crucial.
Innovation Solution
The proposed solution involves broadcasted residual learning, which processes input tensors with a frequency dimension and a temporal dimension. This method includes a first convolution operation to generate a multidimensional intermediate feature map, followed by a frequency dimension reduction to convert it into a one-dimensional feature map. Subsequently, a second convolution operation is applied to generate a temporal feature map, which is then expanded back to the frequency dimension and augmented with the original feature map via a residual connection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional convolutional neural networks are used for keyword spotting, then model accuracy can be maintained, but computational complexity and resource requirements increase
Solution Approach 1:
The patent segments the convolutional neural network into multiple stages: initial convolution layer for feature extraction, followed by residual blocks with dimensionality reduction and restoration. This segmentation allows complex computations to be distributed and optimized at different stages, reducing overall computational complexity while maintaining accuracy
Solution Approach 2:
The patent dynamically changes parameters including dimensionality of feature maps (reducing and restoring), learning rates, and filter sizes across different layers. These parameter changes enable the model to adapt computation to the specific characteristics of different input data, reducing unnecessary computational complexity while preserving accuracy
2Measurement precision
If larger models are used to improve accuracy, then classification performance improves, but latency increases
Solution Approach 1:
The patent applies preliminary dimensionality reduction to intermediate feature maps before subsequent processing. By reducing the dimensionality of feature maps early in the network, the computational burden of subsequent layers is significantly reduced, decreasing latency while maintaining the essential features needed for accurate classification
Solution Approach 2:
The patent introduces dimensionality reduction and restoration operations that temporarily change the dimensional representation of feature maps. This allows the network to process information more efficiently in reduced dimensions and then restore the original dimensionality, achieving faster processing without sacrificing accuracy
3Measurement precision
If complex convolution operations are applied to maintain accuracy, then model performance is preserved, but resource requirements increase
Solution Approach 1:
The patent extracts and processes only the most essential features by applying dimensionality reduction operations that retain critical information while discarding redundant data. This extraction approach reduces resource requirements by focusing computational efforts on the most informative features rather than processing all data equally
Solution Approach 2:
The patent uses residual connections that copy and augment intermediate feature maps through the network. This allows the model to preserve important features across multiple layers without requiring increasingly complex operations, reducing resource requirements while maintaining accuracy through feature reuse
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Certain aspects of the present disclosure provide techniques for efficient broadcasted residual machine learning. An input tensor comprising a frequency dimension and a temporal dimension is received, and the input tensor is processed with a first convolution operation to generate a multidimensional intermediate feature map comprising the frequency dimension and the temporal dimension. The multidimensional intermediate feature map is converted to a one-dimensional intermediate feature map in the temporal dimension using a frequency dimension reduction operation, and the one-dimensional intermediate feature map is processed using a second convolution operation to generate a temporal feature map. The temporal feature map is expanded to the frequency dimension using a broadcasting operation to generate a multidimensional output feature map, and the multidimensional output feature map is augmented with the multidimensional intermediate feature map via a first residual connection.