Lightweight Neural Network for On-Device Spoken Language Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current spoken language recognition models are too large-scale to run efficiently on user devices and struggle with identifying non-supported languages, often resulting in subpar performance when encountering languages similar to the target set.
Innovation Solution
A lightweight neural network architecture with convolutional layers and a multi-label classification approach is employed, allowing for zero, one, or multiple languages to be identified, using techniques like Per-Channel Energy Normalization and augmented training data to enhance robustness and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large-scale neural networks are used for spoken language recognition, then recognition accuracy is improved, but device complexity and computational resources increase making it impossible to run on user devices
Solution Approach 1:
The patent extracts and removes unnecessary components from traditional large-scale speech recognition models. By eliminating the text decoder, language model, and other complex modules that are not essential for language identification, the system achieves competitive accuracy with a significantly reduced model size that can run on mobile devices.
Solution Approach 2:
The patent employs lightweight neural network architectures that use fewer parameters and less computational power. By designing simpler model structures with reduced complexity, the system maintains effective recognition performance while being suitable for deployment on resource-constrained user devices with limited processing power and memory.
2Device complexity
If traditional multi-class classification is used, then the model is simpler, but it cannot identify multiple languages simultaneously and struggles with non-supported languages
Solution Approach 1:
The patent implements a multi-label classification output layer that enables the model to perform multiple functions simultaneously. Each output neuron corresponds to a specific language and can independently activate, allowing the system to identify zero, one, or multiple languages in a single audio input, thereby handling mixed-language scenarios and non-supported languages effectively.
Solution Approach 2:
The patent employs dynamic thresholding mechanisms that adapt to different input scenarios. By adjusting classification thresholds based on activation values, the system can flexibly determine when to classify languages and when to reject uncertain predictions, improving adaptability to various language combinations and non-supported languages.
Data Source
AI summary
Some aspects of the technology described herein employ a neural network with an efficient and lightweight architecture to perform spoken language recognition. Given an audio signal comprising speech, features are generated from the audio signal, for instance, by converting the audio signal to a normalized spectrogram. The features are input to the neural network, which has one or more convolutional layers and an output activation layer. Each neuron of the output activation layer corresponds to a language from a set of language and generates an activation value. Based on the activations values, an indication of zero or more languages from the set of languages is provided for the audio signal.