Keyword Spotting Model Training via Soft Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing keyword spotting models face challenges in accurately processing aggressively augmented training data due to the degradation of input audio features, leading to incorrect hard labels and reduced accuracy, especially in noisy or diverse conditions.
Innovation Solution
The use of a student-teacher learning approach with a teacher model generating soft labels that reflect the degradation of input audio data, allowing for the training of keyword spotting models with aggressively augmented unlabeled data, which increases data diversity and reduces the need for labeled data, thereby preventing overfitting and improving robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If aggressive data augmentation is applied to increase data diversity, then data diversity and model robustness are improved, but input audio feature degradation occurs leading to incorrect hard labels
Solution Approach 1:
The patent changes the parameter of label representation from discrete hard labels to continuous soft labels (probability distributions). This allows the model to handle degraded audio features from aggressive augmentation by expressing uncertainty through probability distributions rather than definitive class assignments, resolving the contradiction between data diversity and label accuracy
Solution Approach 2:
The patent introduces soft labels as an intermediary between the augmented data and the student model. These soft labels act as a bridge that preserves the relationship between degraded audio features and original labels through probability distributions, enabling the model to learn from aggressively augmented data without losing label information
2Productivity
If hard labels are used for supervised training, then training efficiency is improved, but model accuracy decreases in noisy or diverse conditions
Solution Approach 1:
The patent transforms the label parameter from discrete (hard labels) to continuous (soft labels with probability distributions). This change allows the model to capture uncertainty and nuanced information, improving reliability in noisy conditions while maintaining training efficiency through the structured probability format that can be directly used in loss functions
3Reliability
If more labeled data is collected to improve model accuracy, then model accuracy is improved, but computational costs and labeling requirements increase
Solution Approach 1:
The patent implements self-service through the student-teacher framework where the teacher model generates soft labels for augmented versions of the training data. This allows the system to create its own training signals from existing labeled data through augmentation, eliminating the need to collect and label additional data while improving model accuracy
Solution Approach 2:
The patent creates copies of the original labeled data through aggressive augmentation to generate diverse training samples. The teacher model processes these copied and augmented samples to generate soft labels, allowing the student model to learn from multiple variations of the same underlying data without requiring new labeled examples
Data Source
AI summary
Teacher-student learning can be used to train a keyword spotting (KWS) model using augmented training instance(s). Various implementations include aggressively augmenting (e.g., using spectral augmentation) base audio data to generate augmented audio data, where one or more portions of the base instance of audio data can be masked in the augmented instance of audio data (e.g., one or more time frames can be masked, one or more frequencies can be masked, etc.). Many implementations include processing augmented audio data using a KWS teacher model to generate a soft label, and processing the augmented audio data using a KWS student model to generate predicted output. One or more portions of the KWS student model can be updated based on a comparison of the soft label and the generated predicted output.