End-to-End Pronunciation Scoring Without Forced Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pronunciation assessment technologies for non-native speakers require forced alignment and text information, limiting their effectiveness and applicability.
Innovation Solution
A foreign language pronunciation assessment apparatus using a transformer structure-based end-to-end speech recognizer that trains on native speaker data, tunes with non-native speaker data, and calculates a prediction score through a scoring module, eliminating the need for forced alignment and text information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If forced alignment and text information are used for pronunciation assessment, then measurement precision can be improved, but device complexity and ease of operation deteriorate
Solution Approach 1:
The patent extracts and removes the forced alignment step and text information requirement from the pronunciation assessment system. By using an end-to-end speech recognition model that directly processes speech signals without requiring intermediate text alignment, the system eliminates complex preprocessing steps while maintaining assessment accuracy
Solution Approach 2:
The end-to-end speech recognition model serves multiple functions simultaneously: it performs speech recognition, pronunciation assessment, and scoring in a single unified system. This multi-functional approach replaces the need for separate forced alignment tools and text processing components, reducing overall system complexity
2Measurement precision
If forced alignment is performed for pronunciation assessment, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The speech recognition model is pre-trained on large datasets of native speaker speech before deployment. This preliminary training phase allows the model to learn pronunciation patterns and assessment criteria in advance, enabling it to perform accurate assessments without requiring time-consuming forced alignment during actual use
Solution Approach 2:
The end-to-end model processes speech signals continuously in a single pass, eliminating the need for separate forced alignment steps that interrupt the assessment workflow. The model maintains continuous processing from raw speech input to final score output, reducing total assessment time
3Measurement precision
If text information is required for training ASR system, then measurement precision is improved, but ease of operation deteriorates
Solution Approach 1:
The system uses native speaker speech data that is automatically collected and processed without requiring manual text transcription or annotation. The model learns to associate speech patterns with pronunciation quality directly from audio data, making the training process more automated and easier to operate
Solution Approach 2:
The patent replaces the mechanical process of manual text creation and alignment with a learning-based approach. The end-to-end speech recognition model automatically learns to map speech signals to pronunciation assessments through training on labeled speech data, eliminating the need for manual text preparation
Data Source
AI summary
A technology for assessing foreign language pronunciation of a non-native learner is provided. The foreign language pronunciation assessment technology includes training an end-to-end speech recognizer with native speaker data, tuning the trained end-to-end speech recognizer with non-native speaker data, training a scoring module on the basis of the finely tuned end-to-end speech recognizer, and calculating, by the trained scoring module, a prediction score for non-native speaker speech input when the non-native speaker speech is input to the tuned end-to-end speech recognizer.