Automated Feedback for Non-Native Speech via Key Point Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated feedback generation for non-native spontaneous speech is challenging due to errors from automated speech recognition systems and the variability of spontaneous speech, which complicates the detection of content completeness and accuracy in language learners' responses.
Innovation Solution
An automated algorithm that uses trained content scoring models to determine the absence of key points in spoken responses by calculating similarity scores through word-embedding and n-gram based features, providing targeted feedback to improve content completeness, and is robust to ASR errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated speech recognition systems are used to process spontaneous speech, then productivity is improved, but measurement precision deteriorates due to ASR errors
Solution Approach 1:
The patent introduces an intermediary processing layer between ASR output and content scoring. This layer includes normalization to remove disfluencies and stop words, segmentation into n-grams with overlaps, and similarity calculation against key points using word-mover's distance. This intermediary processing mitigates ASR errors while maintaining automated efficiency.
Solution Approach 2:
The system performs preliminary normalization and segmentation of ASR output before content scoring. By pre-processing the speech text to remove disfluencies, stop words, and segment into overlapping n-grams, the system prepares cleaner input for the scoring model, reducing the impact of ASR errors on final accuracy.
2Measurement precision
If traditional feedback methods are used for spontaneous speech, then measurement precision is maintained through manual evaluation, but productivity deteriorates due to time-consuming processes
Solution Approach 1:
The system implements self-service automated feedback generation that processes spontaneous speech independently without requiring manual transcription or evaluation. The trained content scoring model automatically compares normalized speech content against key points, generating feedback at scale while maintaining precision comparable to manual methods.
Solution Approach 2:
The patent replaces the mechanical manual evaluation system with an automated content scoring model. This model uses machine learning algorithms to perform content comparison and feedback generation, substituting human labor with computational processes that maintain accuracy while dramatically improving productivity.
3Measurement precision
If detailed content analysis is performed on spontaneous speech, then measurement precision is improved, but device complexity increases due to multiple processing steps
Solution Approach 1:
The patent segments the content analysis process into distinct modular steps: normalization to remove disfluencies, segmentation into overlapping n-grams, similarity calculation against key points using word-mover's distance, and final scoring. This segmentation enables detailed analysis while managing complexity through clear separation of concerns.
Solution Approach 2:
The system transforms the content analysis from direct text comparison to a multi-dimensional approach using n-gram segmentation and word embedding spaces. By representing speech content in vector space and calculating distances in this transformed dimension, the system achieves precise key point detection while simplifying the comparison operation.
4Measurement precision
If normalization and segmentation are applied to speech text, then measurement precision is improved, but loss of information increases due to removal of stop words and disfluencies
Solution Approach 1:
The system extracts and removes elements that interfere with content measurement: stop words and disfluencies are taken out during normalization. This extraction improves content similarity measurement by focusing on meaningful content words while discarding noise that would otherwise distort the similarity calculation.
Solution Approach 2:
The system applies partial normalization by selectively removing only stop words and disfluencies while preserving content-bearing words. This partial action is sufficient to improve measurement precision without excessive removal that would cause information loss, striking an optimal balance between cleaning and preservation.
Data Source
AI summary
An electronic audio file is received that comprises spontaneous speech responsive to a prompt in a non-native language of a speaker. Thereafter, the electronic audio file is parsed into a plurality of spoken words. The spoken words are then normalized to remove stop words and disfluencies. At least one trained content scoring model is then used to determine an absence of pre-defined key points associated with the prompt in the normalized spoken words. A list of the determined absent key points can be generated. This list can then be displayed/caused to be displayed in a graphical user interface along with feedback to improve content completeness. Related apparatus, systems, techniques and articles are also described.