Automated Speech Content Scoring via Clustering and Vector Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual scoring of spoken responses is costly and resource-intensive, especially when evaluating large numbers of responses, as it requires human feedback on both mechanical and content aspects of verbal communication.
Innovation Solution
A computer-implemented system generates a scoring model for spoken responses by clustering training responses, selecting a subset for scoring, and using content training vectors to assign scores based on similarity metrics, reducing the need for extensive human scoring through automated processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual scoring is used for spoken responses, then accurate assessment of content quality is achieved, but time consumption and cost increase significantly
Solution Approach 1:
The system performs preliminary actions by generating transcripts and creating content training vectors from scored training responses before actual scoring occurs. The clustering analysis and vector generation are prepared in advance, enabling rapid scoring of new responses without requiring manual review of each new response's full content quality assessment process
Solution Approach 2:
The system creates content training vectors as simplified representations (copies) of the actual spoken responses. These vectors capture the essential content characteristics and can be used for comparison and scoring without requiring human scorers to review the full original responses, thereby maintaining assessment accuracy while reducing time consumption
2Measurement precision
If manual scoring is used for spoken responses, then accurate feedback on verbal communication quality is provided, but resource requirements increase
Solution Approach 1:
The system replaces the mechanical human scoring process with an automated computational system. The computer-implemented system performs transcript generation, clustering analysis, and similarity computation to assess verbal communication quality, eliminating the need for human scorers while maintaining measurement precision and enabling high-volume processing of spoken responses
Solution Approach 2:
The system transforms the complex, multi-dimensional assessment of verbal communication quality into comparable parameter representations through content training vectors and similarity metrics. By converting qualitative content assessment into quantitative parameter comparisons, the system achieves both accurate evaluation and high processing throughput
3Reliability
If all training responses are scored manually, then comprehensive scoring model accuracy is achieved, but cost and time consumption increase
Solution Approach 1:
The system segments the training responses into clusters based on their content characteristics and transcripts. By dividing the training data into meaningful groups, the system can generate scoring models more efficiently while maintaining reliability, as each cluster can be processed independently and representative samples from each cluster can be used to build comprehensive scoring models
Data Source
AI summary
Computer-implemented systems and methods are provided for scoring content of a spoken response to a prompt. A scoring model is generated for a prompt, where generating the scoring model includes generating a transcript for each of a plurality of training responses to the prompt, dividing the plurality of training responses into clusters based on the transcripts of the training responses, selecting a subset of the training responses in each cluster for scoring, scoring the selected subset of training responses for each cluster, and generating content training vectors using the transcripts from the scored subset. A transcript is generated for a received spoken response to be scored, and a similarity metric is computed between the transcript of the spoken response to be scored and the content training vectors. A score is assigned to the spoken response based on the determined similarity metric.