Multi-Language Speech Recognition via Word-Level Reliability Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition models perform poorly when handling mixed-language utterances, as they are trained for specific languages and struggle with loanwords or foreign words pronounced in different languages, leading to low recognition accuracy.
Innovation Solution
An artificial intelligence apparatus and method that combines word recognition results using an acoustic model of a main language and at least one sub-language, calculates word recognition reliability, and converts speech data into text based on the selected language for each word, improving recognition accuracy by selecting the language with the highest reliability for each word.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a speech recognition model is trained for a specific language, then the recognition accuracy for that language is improved, but the recognition accuracy for mixed-language utterances deteriorates
Solution Approach 1:
The patent segments the speech recognition process by dividing the utterance into individual words and evaluating each word separately against multiple language models. This allows the system to select the most appropriate language model for each word based on recognition reliability scores, rather than committing to a single language model for the entire utterance.
Solution Approach 2:
The patent changes the parameter of language model selection dynamically based on recognition reliability. Instead of using a fixed language model, the system calculates reliability scores for each word using different language models and selects the model that produces the highest reliability score for each word, thereby adapting to mixed-language inputs.
2Measurement precision
If multiple speech recognition models for different languages are used, then the recognition accuracy for mixed-language utterances is improved, but the system complexity increases
Solution Approach 1:
The patent applies partial action by calculating recognition reliability scores only for words that are likely to be loanwords or foreign words, rather than processing every word through all language models. This selective approach reduces computational overhead while maintaining accuracy for mixed-language utterances.
Solution Approach 2:
The patent introduces an intermediary component that calculates recognition reliability scores and determines which language model to use for each word. This intermediary layer manages the complexity of multiple language models by providing a systematic method for selecting the appropriate model based on reliability metrics.
3Measurement precision
If speech recognition models are trained with loanwords and foreign words, then the recognition performance for borrowed words is improved, but the performance deteriorates when these words are pronounced in their original language
Solution Approach 1:
The patent makes the language model selection dynamic rather than static. Instead of training a single model to handle both borrowed words and original language pronunciations, the system dynamically selects between multiple language models based on the actual pronunciation characteristics of each word, allowing it to adapt to different pronunciation styles.
Solution Approach 2:
The patent changes the language model parameter based on recognition reliability scores calculated for each word. This allows the system to switch between language models depending on whether the word is pronounced as a loanword or in its original language, optimizing recognition performance for both cases.
Data Source
AI summary
Disclosed herein an artificial intelligence apparatus for recognizing speech with multiple languages including a microphone, and a processor configured to obtain, via the microphone, speech data including speech of a user with multiple languages, calculate a word recognition reliability of each word in the obtained speech data using an acoustic model of a main language, calculate a word recognition reliability of each word in the obtained speech data using an acoustic model of at least one sub language, select a language having a highest word recognition reliability for each word, convert the speech data into text in consideration of a word recognition result corresponding to the selected language for each word, and generate a speech recognition result corresponding to the speech data using the converted text.