Knowledge Graph Infusion for Domain-Specific Speech Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automatic speech recognition (ASR) systems struggle with accurately transcribing domain-specific terms due to the long-tail distribution of these terms in training data, requiring manual domain-specific tuning and custom vocabularies, which is costly and inefficient.
Innovation Solution
Integrate domain-specific knowledge graphs into ASR processes to enhance accuracy without manual vocabulary input, using a two-pass approach with a first pass for initial transcription and a second pass for knowledge graph infusion to refine the transcript.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual domain-specific tuning with custom vocabularies is used, then transcription accuracy for domain-specific terms is improved, but device complexity and data storage requirements increase
Solution Approach 1:
The patent introduces an intermediary component - the knowledge graph - that mediates between the ASR system and domain-specific terminology. The knowledge graph stores structured domain knowledge and relationships, allowing the system to access domain-specific terms contextually without requiring manual vocabulary lists or complex custom ASR models for each domain.
Solution Approach 2:
The knowledge graph serves multiple functions: it acts as a vocabulary source, provides contextual information for disambiguation, enables domain adaptation without retraining, and supports various query types. This single universal structure replaces multiple domain-specific configurations that would otherwise be needed.
2Measurement precision
If manual domain-specific tuning with custom vocabularies is used, then transcription accuracy for domain-specific terms is improved, but data storage and computing resources increase
Solution Approach 1:
The knowledge graph serves as an intermediary that stores domain knowledge in a compact, structured format. Instead of storing extensive custom vocabularies and domain-specific training data for each application, the system stores a centralized knowledge graph that can be queried efficiently during transcription, reducing overall data storage requirements.
Solution Approach 2:
The knowledge graph is pre-populated with domain-specific entities and relationships before the ASR process begins. This preliminary organization of domain knowledge allows the system to quickly retrieve relevant terms and context during transcription without requiring large amounts of runtime data or repeated processing of domain-specific corpora.
3Device complexity
If a single base ASR model is used for all domains, then device complexity is reduced, but transcription accuracy for domain-specific terms deteriorates
Solution Approach 1:
The knowledge graph acts as an intermediary layer between the universal ASR model and domain-specific requirements. The ASR model remains simple and general-purpose, while the knowledge graph provides domain-specific context and term suggestions during the decoding process, enabling accurate domain-specific transcription without complicating the core model.
Solution Approach 2:
The system enables the ASR model to access domain-specific information on-demand through the knowledge graph during the decoding process. The model itself remains unchanged and domain-agnostic, but it can query the knowledge graph for domain-specific terms and contextual information when needed, making the system adaptable to different domains without retraining or model changes.
Data Source
AI summary
Systems and methods are described for converting speech to text. In one aspect, a method for converting speech to text includes generating an initial transcript of an audio or video file. At least one named entity may be extracted from the initial transcript using entity recognition. A subset of nodes of a knowledge graph that include the at least one named entity may be selected, where the subset of nodes of the knowledge graph correspond to a set of named entities. The method may further include encoding the set of named entities to generate a set of entity embeddings. The speech in the audio or video file may then be decoded using the set of entity embeddings to produce a final transcript of the audio file.