Speech Recognition via Syllable Conversion for Mixed Language Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition systems face low accuracy in mixed language scenarios, particularly when English words are interspersed with Chinese, due to the shortness of English speech data, leading to inadequate language identification and recognition.
Innovation Solution
A method and system for speech recognition that employs feature extraction, an acoustic model with multiple layers and a softmax function, and a language model to convert speech features into syllable data, which are then used to generate text data without initial language identification, utilizing a pronouncing dictionary and language model to improve recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If language identification is performed first before speech recognition, then the recognition process follows a standard pipeline, but the accuracy decreases when English words are short and interspersed in Chinese speech
Solution Approach 1:
The patent extracts and removes the language identification step from the traditional speech recognition pipeline. Instead of identifying language first and then recognizing speech, the system directly performs speech recognition on mixed-language audio by extracting speech features and processing them through acoustic and language models without determining language boundaries beforehand.
Solution Approach 2:
The patent creates a universal speech recognition system that handles multiple languages (Chinese and English) simultaneously within a single recognition framework. The acoustic model and language model work together to process mixed-language speech without requiring separate processing paths for different languages, making the system multi-functional for handling various language combinations.
2Reliability
If traditional speech recognition pipeline is used with language identification, then the system structure is simple, but the recognition accuracy drops for short English words in mixed language
Solution Approach 1:
The patent merges the language identification function into the speech recognition process itself. The acoustic model extracts features directly from the audio signal, and the language model evaluates text data without requiring separate language identification. This combining of functions improves accuracy while maintaining reasonable system complexity through integrated processing.
3Reliability
If English speech data is short, then the language identification becomes inadequate, but the speech recognition accuracy further decreases
Solution Approach 1:
The patent introduces speech features as an intermediary representation between the raw audio signal and the final text output. The acoustic model extracts comprehensive speech features that capture linguistic information without requiring language identification. These features serve as a mediator that preserves information from short English words while enabling accurate recognition through the language model.
Data Source
AI summary
Disclosed are a method, device and readable storage medium for speech recognition. The method includes: determining speech features of the speech data by feature extraction on the speech data; determining syllable data corresponding to each of the speech features based on a plurality of feature extraction layers and a softmax function layer included in an acoustic model, where the acoustic model is configured to convert the speech feature into the syllable data; determining text data corresponding to the speech data based on a language model, a pronouncing dictionary and the syllable data, where the pronouncing dictionary is configured to convert the syllable data into the text data, and the language model is configured to evaluate the text data; and outputting the text data.