Out-of-Vocabulary Phrase Prediction in ASR Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern automatic speech recognition (ASR) systems face challenges in accurately recognizing out-of-vocabulary named entities and open titles due to their fixed nature and the need for extensive training data, leading to limited support and increased complexity, especially in commercial applications where variations are not robustly handled.
Innovation Solution
The implementation of an electronic device with an ASR model and an entity prediction model, coupled with a knowledge base, that performs out-of-vocabulary phrase prediction and supports multiple domains within a single system, allowing for on-device AI inferences and continuous expansion of database coverage without requiring frequent retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If modern ASR systems use end-to-end neural architectures, then ease of training and maintainability improve, but the ability to accurately recognize out-of-vocabulary named entities deteriorates
Solution Approach 1:
The system segments the speech recognition task into two parts: a general ASR model for standard language and a specialized entity recognition model for out-of-vocabulary entities. This segmentation allows each model to be optimized for its specific function, with the entity model focusing exclusively on learning entity patterns from curated datasets while the ASR model handles general speech-to-text conversion.
Solution Approach 2:
The patent introduces an intermediary entity recognition model that sits between the ASR model and the final output. This intermediary component receives ASR hypotheses and refines them by identifying and correcting out-of-vocabulary entity recognition errors, effectively mediating between the general-purpose ASR system and the requirement for accurate entity recognition.
2Reliability
If ASR systems are trained with complete utterance samples, then model robustness improves, but the ability to recognize rare entities deteriorates due to insufficient training data
Solution Approach 1:
The system performs preliminary action by pre-curating comprehensive datasets of out-of-vocabulary entities across multiple domains before training the entity recognition model. This advance preparation ensures that the model has access to extensive entity examples without requiring proportional increases in complete utterance training data, solving the data scarcity problem for rare entities.
Solution Approach 2:
The patent applies local quality by creating domain-specific entity datasets with varying concentrations of entity examples tailored to each domain's characteristics. Rather than uniformly distributing training data, the system concentrates high-quality entity examples in relevant domains, allowing the model to achieve high recognition accuracy for domain-specific entities without needing exhaustive coverage of all possible utterances.
3Quantity of substance
If synthetic data generation is used to augment training data, then data coverage improves, but modeling accuracy deteriorates due to distribution mismatches
Solution Approach 1:
The patent employs synthetic utterance generation as a disposable data augmentation technique specifically for entity pattern learning. Rather than relying on synthetic data to replace real data, the system uses it as a supplementary resource that can be generated on-demand to cover edge cases and rare entity patterns, accepting that synthetic data has lower fidelity but providing sufficient coverage for entities where real examples are scarce.
4Measurement precision
If multiple domain-specific ASR models are used to support different entities, then recognition accuracy improves, but system complexity increases
Solution Approach 1:
The patent implements universality by designing a single entity recognition model that handles multiple domains and entity types through domain identification and routing mechanisms. Rather than maintaining separate ASR models for each domain, the system uses one multi-functional entity model that adapts to different domains by identifying the relevant domain context and applying appropriate entity recognition rules, significantly reducing system complexity while maintaining high accuracy across domains.
Data Source
AI summary
An electronic device includes an audio sensor, a memory, and at least one processor coupled to the audio sensor and the memory. The at least one processor is configured to receive, via the audio sensor an audio input. The at least one processor is further configured to perform, using an automatic speech recognition (ASR) model and an entity prediction model, out-of-vocabulary prediction of an entity. The at least one processor is further configured to receive an ASR hypothesis including the predicted entity. The at least one processor is further configured to output text including the predicted entity.