Content-Based Speech Scoring via Semantic Vector Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated speech assessment systems are limited in scoring non-scripted speech samples based on content features due to inaccuracies in automatic speech recognition, typically relying on non-content metrics like fluency, pronunciation, and prosody.
Innovation Solution
A system and method that extract content features from automatically recognized words in non-scripted speech samples and use a scoring model to generate a content-based score, incorporating multiple sub-models and an ontology source to adjust and expand content vectors for improved similarity analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If automatic speech recognition is used to extract content features from non-scripted speech samples, then content-based scoring capability is improved, but scoring accuracy deteriorates due to recognition inaccuracies
Solution Approach 1:
The patent introduces an intermediary processing layer between automatic speech recognition and content scoring. This intermediary includes a scoring model that receives recognized words and applies multiple sub-models (pointwise mutual information, content vector analysis, latent semantic analysis) to compute content scores. The intermediary processes recognize inaccuracies by using contextual analysis and semantic relationships to compensate for recognition errors, thereby maintaining scoring accuracy while enabling content-based assessment.
Solution Approach 2:
The patent changes the parameters used for content representation from simple word matches to enriched feature vectors that incorporate pointwise mutual information scores, content vector analyses, and latent semantic analysis results. By transforming raw recognized words into multi-dimensional feature representations that capture semantic meaning and contextual relationships, the system can tolerate recognition inaccuracies while maintaining high scoring precision.
2Reliability
If traditional non-content metrics (fluency, pronunciation, prosody) are used for scoring, then scoring reliability is maintained, but scoring comprehensiveness deteriorates
Solution Approach 1:
The patent merges multiple scoring dimensions into a unified content scoring framework. It combines traditional non-content metrics (fluency, pronunciation, prosody) with new content-based metrics (semantic relevance, conceptual accuracy, contextual appropriateness) into a comprehensive scoring system. The scoring model integrates results from multiple sub-models that analyze different aspects of content quality, creating a holistic evaluation that maintains reliability while significantly improving comprehensiveness.
Solution Approach 2:
The patent creates a universal scoring framework that can evaluate multiple aspects of speech quality through a single integrated system. The scoring model is designed to handle both traditional non-content metrics and new content-based metrics using the same processing architecture, allowing the system to comprehensively assess speech samples across multiple dimensions without requiring separate specialized systems for each metric type.
Data Source
AI summary
Systems and methods are provided for scoring a non-scripted speech sample. A system includes one or more data processors and one or more computer-readable mediums. The computer-readable mediums are encoded with a non-scripted speech sample data structure, where the non-scripted speech sample data structure includes: a speech sample identifier that identifies a non-scripted speech sample, a content feature extracted from the non-scripted speech sample, and a content-based speech score for the non-scripted speech sample. The computer-readable mediums further include instructions for commanding the one or more data processors to extract the content feature from a set of words automatically recognized in the non-scripted speech sample and to score the non-scripted speech sample by providing the extracted content feature to a scoring model to generate the content-based speech score.