Mathematical Language Processing for Open Response Grading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large-scale online education platforms, grading open response mathematical questions is inefficient due to the need for instructors to trace logical flows and identify errors in numerous solutions, limiting the capacity to serve a large number of learners.
Innovation Solution
A data-driven framework for mathematical language processing (MLP) that converts solutions into numerical features, clusters them to identify correct, partially correct, and incorrect solutions, and automatically grades submissions based on instructor-provided grades, providing feedback on error locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If instructors manually grade open response mathematical questions by tracing logical flows and identifying errors, then grading accuracy is improved, but the capacity to serve a large number of learners deteriorates
Solution Approach 1:
The patent segments the grading task into multiple components: (1) extracting mathematical expressions and their logical relationships from student solutions, (2) identifying the sequence of expressions that constitute the logical flow, (3) comparing each expression against a knowledge base of correct mathematical steps, and (4) assigning grades based on the point of first error. This segmentation allows the system to automate the complex process of tracing logical flows while maintaining grading accuracy, thereby increasing grading capacity without sacrificing precision.
2Productivity
If the number of learners in an online course is increased, then educational scalability is improved, but the difficulty of grading all learner-submitted answers deteriorates
Solution Approach 1:
The patent implements a self-service grading system where the automated grading mechanism independently processes student submissions without requiring manual instructor intervention for each answer. The system automatically extracts mathematical expressions, traces logical flows, identifies errors, and assigns grades, enabling the course to scale to large numbers of learners while keeping grading complexity manageable through full automation of the grading process.
3Loss of information
If instructors grade every learner submission individually, then feedback quality is improved, but the time required for grading deteriorates
Solution Approach 1:
The patent implements an automated feedback mechanism that provides timely, personalized feedback to each student by identifying the specific point where their logical flow diverges from correct mathematical reasoning. The system analyzes each submission, traces the logical sequence of expressions, compares them against known correct solutions, and generates feedback indicating where the first error occurs. This automated feedback process maintains high feedback quality while dramatically reducing the time required compared to manual grading of every submission.
Data Source
AI summary
Mechanisms for automatically grading a large number of solutions provided by learners in response to an open response mathematical question. Each solution is mapped to a corresponding feature vector based on the mathematical expressions occurring in the solution. The feature vectors are clustered using a conventional clustering method, or alternatively, using a presently-disclosed Bayesian nonparametric clustering method. A representative solution is selected from each solution cluster. An instructor supplies a grade for each of the representative solutions. Grades for the remaining solutions are automatically generated based on their cluster membership and the instructor supplied grades. The Bayesian method may also automatically identify the location of an error in a given solution. The error location may be supplied to the learner as feedback. The error location may also be used to extract information from correct solutions. The extracted information may be supplied to a learner as a solution hint.