Bias-Token Speech Recognition for Infrequent Word Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional end-to-end (E2E) speech recognition models struggle with accurately recognizing infrequent words, phrases, and sentences without the need for extensive retraining, particularly personal names and coined terms.
Innovation Solution
An E2E-ASR model is enhanced with a bias encoder and decoder that register pre-defined words, phrases, and sentences as dynamic tokens, allowing for improved recognition by incorporating a bias list and adjusting token scores through an embedding layer and output layer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional E2E models are used without dictionaries, then the model structure is simple and training is straightforward, but the recognition accuracy for infrequent words such as personal names deteriorates
Solution Approach 1:
The patent introduces a dictionary as an intermediary component between the audio input and the E2E model. The dictionary stores pre-registered infrequent words (personal names, terms) and provides them to the model during recognition, improving accuracy without requiring the entire model to be retrained. This mediator enables the system to handle out-of-vocabulary words effectively.
Solution Approach 2:
The patent applies preliminary action by pre-registering infrequent words and phrases in a dictionary before the actual speech recognition task. This pre-processing step allows the model to access these specialized terms during recognition without requiring extensive retraining, thus improving recognition accuracy for personal names and coined terms while maintaining model simplicity.
2Measurement precision
If the entire model is retrained to recognize infrequent words, then recognition accuracy for personal names improves, but the time and computational resources required increase significantly
Solution Approach 1:
The patent performs preliminary action by pre-registering infrequent words in a dictionary structure before recognition. This allows the system to incorporate personal names and coined terms without retraining the entire E2E model, significantly reducing the time and computational resources required compared to full model retraining.
Solution Approach 2:
The patent extracts the handling of infrequent words from the main E2E model by creating a separate dictionary component. This extraction allows the core model to remain unchanged and avoid retraining, while still improving recognition accuracy for personal names and specialized terms through the dedicated dictionary lookup mechanism.
3Adaptability or versatility
If conventional E2E models are used, then the model structure is simple, but the ability to easily register personal names and terms deteriorates
Solution Approach 1:
The patent introduces a dictionary as an intermediary that simplifies the registration process. Users can easily register personal names and terms in the dictionary without modifying the complex E2E model structure. This mediator layer provides ease of registration while maintaining the simplicity of the core recognition model.
Solution Approach 2:
The patent segments the speech recognition system into two independent parts: the E2E model for general recognition and a separate dictionary for infrequent words. This segmentation allows independent management of each component, enabling easy registration of personal names in the dictionary without affecting or requiring changes to the main model structure.
Data Source
AI summary
A speech recognition device includes an acquisition unit configured to acquire audio data of an utterance and a speech recognition unit configured to generate text from the audio data using an automatic speech recognition model. The automatic speech recognition model includes an audio encoder configured to convert the audio data into a feature, a bias encoder configured to convert a registered bias token into a feature, and a bias decoder expanded to correspond to a bias token and configured to estimate the next token on the basis of a feature output by the audio encoder, a feature output by the bias encoder, and a previously estimated token sequence.