Hybrid Language Model for Speech Recognition Probability Assignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition systems face inefficiencies in assigning probabilities to language sequences, particularly in handling high-frequency queries and queries outside defined sets, leading to reduced accuracy and increased storage requirements.
Innovation Solution
A hybrid language model is developed, comprising a first component that assigns probabilities based on the relative frequency of high-frequency queries and a second component that uses n-gram modeling for queries not in the defined set, with adjustment data normalizing the probabilities to ensure accurate scoring across all sequences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single language model uses n-gram modeling for all language sequences, then it can handle any language sequence, but it fails to accurately reflect the frequency of high-frequency queries and increases computational complexity
Solution Approach 1:
The language model is divided into two separate components: a first language model that stores complete language sequences with their frequencies, and a second language model that uses n-gram modeling. This segmentation allows each component to specialize - the first model handles high-frequency queries with exact frequency data, while the second model handles other sequences through conditional probabilities, thereby improving overall accuracy without requiring a single overly complex model
2Measurement precision
If a hybrid language model with two components is used, then probability assignment accuracy improves, but storage requirements and device complexity increase
Solution Approach 1:
The first language model component stores complete language sequences and their frequencies locally for high-frequency queries, while the second component uses n-gram modeling for other sequences. This local quality approach ensures that storage resources are concentrated on the most frequently occurring sequences where accurate probability assignment is most critical, rather than uniformly storing all possible sequences
3Measurement precision
If complete language sequences are stored with their frequencies, then probability assignment for high-frequency queries becomes more accurate, but storage requirements increase
Solution Approach 1:
The first language model component stores complete language sequences and their frequencies, but this is done selectively for high-frequency queries rather than attempting to store all possible language sequences. This partial action approach focuses storage resources on the subset of sequences that occur most frequently, providing accurate probability assignment where it matters most while avoiding the prohibitive storage costs of comprehensive coverage
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for language modeling of complete language sequences. Training data indicating language sequences is accessed, and counts for a number of times each language sequence occurs in the training data are determined. A proper subset of the language sequences is selected, and a first component of a language model is trained. The first component includes first probability data for assigning scores to the selected language sequences. A second component of the language model is trained based on the training data, where the second component includes second probability data for assigning scores to language sequences that are not included in the selected language sequences. Adjustment data that normalizes the second probability data with respect to the first probability data is generated, and the first component, the second component, and the adjustment data are stored.