Neural Network Acoustic Model Adaptation with Partial Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for adapting neural network acoustic models to a speaker require large amounts of speech data, making them inapplicable or unusable when only a small amount of enrollment data is available, such as less than a minute of speech data.
Innovation Solution
The method involves augmenting a trained neural network acoustic model with a partial layer of nodes and estimating the weights for a linear transformation using enrollment data, allowing adaptation with less data, and enabling online adaptation without enrollment data by iteratively updating speaker information values during speech recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional techniques are used to adapt neural network acoustic models to a speaker, then speech recognition performance is improved, but large amounts of speech data are required
Solution Approach 1:
The patent segments the adaptation process into two distinct phases: (1) offline adaptation using a small amount of enrollment data to initialize speaker-specific parameters, and (2) online adaptation using iterative refinement during actual speech recognition tasks. This segmentation allows the system to achieve good performance with minimal initial data while continuously improving with additional usage.
Solution Approach 2:
The patent performs preliminary adaptation using a small set of enrollment data (10-20 seconds of speech) to initialize speaker-specific parameters before actual speech recognition begins. This preliminary action creates a baseline adapted model that can then be further refined online, eliminating the need for large amounts of pre-training data.
2Ease of manufacture
If speaker-independent acoustic models are used, then training data requirements are reduced, but speech recognition performance for individual speakers deteriorates
Solution Approach 1:
The patent creates a universal base acoustic model trained on diverse speaker data that can serve multiple speakers, then adds speaker-specific adaptation layers that can be customized for individual users. This multi-functionality allows the same base model to work universally while accommodating individual speaker characteristics through optional adaptation.
Solution Approach 2:
The patent implements dynamic adaptation where the acoustic model transitions from a static speaker-independent state to a dynamic speaker-adapted state through iterative online learning. The model continuously adjusts speaker-specific parameters during use, allowing performance to improve over time as more speaker data becomes available.
3Measurement precision
If large amounts of enrollment data are collected for adaptation, then adaptation accuracy is improved, but system complexity and data management requirements increase
Solution Approach 1:
The patent applies partial action by using only a minimal subset of data (10-20 seconds of enrollment speech) sufficient for initial adaptation, rather than collecting and processing large amounts of data. This partial approach achieves adequate adaptation accuracy while significantly reducing data management complexity.
Data Source
AI summary
Techniques for adapting a trained neural network acoustic model, comprising using at least one computer hardware processor to perform: generating initial speaker information values for a speaker; generating first speech content values from first speech data corresponding to a first utterance spoken by the speaker; processing the first speech content values and the initial speaker information values using the trained neural network acoustic model; recognizing, using automatic speech recognition, the first utterance based, at least in part on results of the processing; generating updated speaker information values using the first speech data and at least one of the initial speaker information values and/or information used to generate the initial speaker information values; and recognizing, based at least in part on the updated speaker information values, a second utterance spoken by the speaker.