ASR Model Training With Differential Privacy Across Multiple Domains
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing domain adaptation techniques for automatic speech recognition (ASR) models fail to leverage diverse data from multiple sources due to privacy constraints, resulting in performance gaps when training is performed in isolation without combining private data.
Innovation Solution
A method that uses differentially private stochastic gradient descent (DP-SGD) to collaboratively train ASR models on private data from multiple sources, ensuring privacy preservation by applying differentially private gradients to update the model without exposing sensitive information, and adapting the model to specific domains using modular domain adaptation techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data from multiple sources is combined into a single comprehensive dataset for training, then model performance is improved, but privacy constraints are violated
Solution Approach 1:
The training process is segmented into multiple independent training runs, each using data from a single source. The model is trained separately on each source's data without combining datasets, thus maintaining privacy constraints while still benefiting from multi-source training through sequential exposure to different data domains.
Solution Approach 2:
A privacy-preserving mechanism acts as an intermediary between the training process and the data sources. This intermediary enables the model to learn from multiple sources without direct access to or combination of the actual private datasets, resolving the conflict between performance improvement and privacy protection.
2Object-affected harmful factors
If domain adaptation is performed in isolation using only one domain's data, then privacy is preserved, but model performance deteriorates
Solution Approach 1:
The training process continues across multiple sequential training runs, each exposing the model to different data sources. This continuous training action allows the model to accumulate knowledge from diverse domains over time, maintaining performance while preserving privacy through isolated training sessions.
Solution Approach 2:
The model undergoes preliminary training on public data before domain adaptation, and undergoes multiple sequential adaptations to different private domains. This preliminary and iterative preparation enables the model to build a strong foundation that can be progressively adapted to various domains without requiring simultaneous access to all private datasets.
3Object-affected harmful factors
If separate adapted models are generated for each domain without combining private data, then privacy is maintained, but the performance gap compared to multi-source training increases
Solution Approach 1:
The model merges knowledge from multiple sources through sequential training runs. Instead of creating separate models for each domain, a single model is iteratively trained on data from different sources in sequence, allowing it to integrate diverse domain knowledge while maintaining privacy through isolated training sessions.
Solution Approach 2:
The training parameters and data domains are changed across multiple sequential training runs. The model adapts its parameters progressively as it encounters different data sources, enabling it to learn diverse characteristics without requiring simultaneous access to all private datasets, thus closing the performance gap while maintaining privacy.
Data Source
AI summary
A method includes obtaining a plurality of sets of private training utterances. Each corresponding set of private training utterances is obtained from a different source and associated with a speech domain that is different than the speech domains associated with the other sets of private training utterances. The method also includes training a speech recognition model by obtaining a current version of the speech recognition model, selecting a batch of private training utterances from one of the plurality of sets of private training utterances, determining a differentially private gradient for updating the current version of the speech recognition model based on the selected batch of private training utterances, and updating the current version of the speech recognition model using the differentially private gradient. The method also includes adapting the trained speech recognition model to learn how to recognize speech in a target speech domain.