AI Speech Recognition Personalization Through Loss-Ratio Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition models face issues of overfitting and excessive adaptation when personalized using synthetic speech data, leading to inferior performance in specific domains and personalization, and potential personal information leakage.
Innovation Solution
A method utilizing a pre-trained speech encoder and prediction network with fixed parameters, combined with a loss ratio-driven data filtering technique, to adapt the model to individual domains while preventing overfitting and maintaining performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If speech recognition models are trained with synthetic speech data for personalization, then model adaptability to individual domains is improved, but overfitting and excessive adaptation occur leading to inferior performance
Solution Approach 1:
The patent implements a feedback mechanism where the model continuously monitors loss rates during training and adjusts its behavior accordingly. When the loss rate exceeds a threshold, the system filters out synthetic data samples and uses only real data for training, preventing overfitting while maintaining adaptability to individual domains
Solution Approach 2:
The patent dynamically changes training parameters based on loss rate thresholds. The system switches between training with synthetic data (when loss rate is low) and training with real data only (when loss rate exceeds threshold), thereby controlling the degree of adaptation to avoid overfitting while preserving domain-specific performance
2Adaptability or versatility
If speech recognition models are trained with synthetic speech data, then personalization capability is improved, but computational costs increase and personal information leakage risk increases
Solution Approach 1:
The patent applies partial action by selectively using synthetic data only when beneficial (when loss rate is below threshold) and switching to real data when necessary. This partial application of synthetic data training reduces overall computational costs compared to always using synthetic data, while still achieving effective personalization when needed
3Adaptability or versatility
If speech recognition models are trained with synthetic speech data, then personalization capability is improved, but personal information leakage risk increases
Solution Approach 1:
The feedback mechanism monitors training quality through loss rates and automatically switches from synthetic to real data when quality deteriorates. This prevents the model from learning spurious patterns from synthetic data that could lead to personal information leakage, while still enabling personalization through controlled use of real data
Solution Approach 2:
The patent converts the potential harm of synthetic data (which could cause overfitting and information leakage) into a benefit by using it selectively. The system leverages synthetic data for initial training and low-risk scenarios, then switches to real data when precision is required, thereby maximizing personalization benefits while minimizing risks
Data Source
AI summary
A method performed by an electronic device using artificial intelligence according to an embodiment of the present disclosure, the method includes receiving a speech signal; and generating a text corresponding to the speech signal by using the speech signal as input in a pre-trained first artificial intelligence algorithm model, wherein the first artificial intelligence algorithm model including a first model parameter outputs a first predicted text by using a synthetic speech and a reference text as input, extracts a first loss based on the first predicted text and the reference text, and performs a first pre-training based on the first loss, wherein the first artificial intelligence algorithm model that performed the first pre-training includes a second model parameter, wherein the first artificial intelligence algorithm model that performed the first pre-training outputs a second predicted text by using the synthetic speech and the reference text as input, and extracts a second loss based on the second predicted text and the reference text, wherein the electronic device determines a loss rate which is a ratio of the first loss and the second loss, determines an adaptation parameter based on the second model parameter and the second loss if the loss rate is below a threshold value, and determines a third model parameter based on the adaptation parameter and the second model parameter, wherein the first artificial intelligence algorithm model is repeatedly pre-trained such that the first artificial intelligence algorithm model that performed a second pre-training is configured to include the third model parameter.