Automatic Speech Analysis Using Phonological Feature Deviation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speech recognition systems for language learning are limited in detecting mispronunciations, particularly when phonemes from the learner's native language are not present in the target language, and require extensive training data to accommodate a global phoneme set, leading to complexity and inefficiency.
Innovation Solution
A method and system that utilize phonological features to provide more accurate feedback, using Automatic Speech Recognition (ASR) to determine phonological features in speech signals, with a neural network and Hidden Markov Model to assess deviations and generate feedback, allowing for the identification of phoneme insertions and deletions, and employing a Finite State Transducer to handle phoneme insertions at word boundaries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a global phoneme set is used to accommodate phonemes from all languages, then the system can detect mispronunciations of phonemes not in the L2 set, but the system becomes overly complex and requires extensive training data
Solution Approach 1:
The patent segments the phoneme detection task by using language-specific phoneme sets for each target language rather than a single global phoneme set. This allows the system to be adapted to different languages without requiring a comprehensive global phoneme inventory, reducing overall system complexity while maintaining the ability to detect mispronunciations of non-L2 phonemes through the articulatory attribute framework
Solution Approach 2:
The patent changes the parameter space from phoneme identity to articulatory attributes (tongue position, lip configuration, vocal cord state, etc.). This transformation allows the system to handle phonemes from any language using a unified set of articulatory parameters, avoiding the need for extensive training data on global phoneme sets while maintaining versatility across languages
2Measurement precision
If phoneme-level analysis is used for mispronunciation detection, then the system can provide detailed feedback, but the processing power required increases significantly
Solution Approach 1:
The patent extracts only the essential articulatory attributes needed for pronunciation assessment from the full phoneme analysis, rather than processing all phoneme-level details. By focusing on key articulatory parameters (tongue position, lip configuration, etc.), the system maintains high measurement precision for pronunciation feedback while reducing the computational burden of processing entire phoneme sequences
Solution Approach 2:
The patent applies partial action by selectively analyzing only the articulatory attributes that are most relevant for detecting mispronunciations, rather than performing complete phoneme-level analysis. This partial analysis approach provides sufficient precision for language learning purposes while significantly reducing processing power requirements
3Productivity
If Extended Recognition Networks with force alignment are used, then the system can achieve simultaneous detection and feedback, but the mispronunciations detectable are limited by the decoding network
Solution Approach 1:
The patent creates a universal articulatory attribute framework that can handle phonemes from any language, making the system multi-functional across different language contexts. This universal framework maintains the productivity of simultaneous detection and feedback while expanding adaptability to detect mispronunciations of phonemes outside the L2 set by representing all phonemes through common articulatory parameters
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
A computer implemented method for automatic speech analysis comprising setting a target phrase, the target phrase comprising a target phoneme, the target phoneme having corresponding target values of a set of phonological features associated therewith, wherein each target value is one of: an indication that the phonological feature should be present, an indication that the phonological feature should be absent, or an indication that the presence of the phonological feature is not specified, receiving a speech signal, wherein the speech signal comprises a user' s attempt to say the target phrase, analysing the speech signal to determine the phonological features present within a portion of the speech signal corresponding to the target phoneme and assigning a probability to each of the set of phonological features, comparing the probability assigned to each phonological feature based on the speech signal to the expected target value of that phonological feature within the phrase, determining a deviation from the comparison; and outputting the deviation to provide feedback on the closeness of the speech signal to the phrase.