Tiny GRU-Attention Speech Enhancement for On-Device Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern deep learning-based speech enhancement models require significant computational resources and complexity, making them impractical for deployment on resource-constrained devices like embedded systems or IoT devices, despite achieving satisfactory performance.
Innovation Solution
A tiny DNN architecture utilizing gated recurrent units (GRUs), multi-head-self-attention (MHSA) module, and normalization layers is employed to enhance speech signals, with a mask decoder predicting a mask to output an estimated clean magnitude, optimized for low complexity and efficient deployment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If modern deep learning-based speech enhancement models are used, then speech enhancement performance is improved, but computational resource requirements and model complexity increase significantly
Solution Approach 1:
The model is segmented into functional modules: encoder with GRUs for feature extraction, attention module for relationship extraction, and mask decoder for prediction. This modular segmentation allows each component to be optimized independently for computational efficiency while maintaining overall performance.
Solution Approach 2:
The patent applies parameter changes by using gated recurrent units with specific gate mechanisms (reset gate, update gate) that modify the flow of information and computational operations. The GRU architecture changes the parameterization of recurrent connections to reduce computational complexity compared to standard RNNs or LSTMs.
2Device complexity
If tiny DNN models are developed to reduce computational footprint, then deployment on resource-constrained devices becomes feasible, but model performance becomes insufficient
Solution Approach 1:
The patent uses knowledge distillation techniques where a smaller student model learns from a larger teacher model. The attention module captures higher-order relationships by modeling the interaction patterns between features, effectively copying the essential computational logic of larger models into a compact architecture.
Solution Approach 2:
The attention module introduces a new dimensional aspect by computing higher-order relationships between features through multi-head self-attention mechanisms. This adds a relationship dimension to the feature space, allowing the tiny model to capture complex dependencies without increasing the number of parameters proportionally.
3Device complexity
If architecture optimization is applied to reduce layer depth and filter sizes, then computational overhead is reduced, but model accuracy and performance deteriorate
Solution Approach 1:
The patent replaces traditional mechanical convolutional operations with gated recurrent unit mechanisms that use gate functions (sigmoid activations) to control information flow. This substitution allows the model to achieve similar or better accuracy with fewer computational operations, as the gate mechanisms provide built-in feature selection and transformation capabilities.
Solution Approach 2:
The architecture uses composite material principles by combining different types of layers (GRU encoder layers, attention module, FC layers, normalization layers) to create a hybrid system where each component contributes specific computational efficiency and performance characteristics, achieving overall optimization.
Data Source
AI summary
A tiny DNN architecture and a method thereof are disclosed for speech enhancement. The tiny DNN architecture may include an encoder comprising a plurality of GRUs for receiving a noisy input magnitude and extracting features from the noisy input magnitude; an attention module for extracting a higher order relationship between the extracted features from the noisy input magnitude; and a mask decoder for predicting a mask, based on the higher order relationship and the extracted features, to output an estimated clean magnitude.