Automated Grading System for Multi-Step Problem Solving Assessment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated grading systems for multi-step problems in mathematics, science, and technology lack the capability to assess the problem-solving skill level of individuals, as they require answers to be submitted in a pre-assigned sequence and do not account for different solution paths or proficiency levels.
Innovation Solution
A computer-based system that allows users to submit answers in any order, provides flexibility in answer entry, and assesses skill level by tracking the number of submissions, prior-answer credit, and use of strategy-describing formulas, enabling the identification of efficient problem-solving methods and error location.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If answers are submitted in a pre-assigned sequence, then grading is simplified, but the ability to assess true problem-solving skill and efficiency is limited
Solution Approach 1:
The problem-solving process is segmented into multiple independent steps that can be submitted in any order. Each step is treated as a separate assessable unit, allowing the system to track individual step correctness and the sequence in which steps were submitted, thereby assessing both accuracy and problem-solving efficiency without requiring a fixed submission sequence
Solution Approach 2:
The submission sequence is made dynamic rather than static. Users can submit steps in any order they choose, and the system adapts to accommodate this flexibility by tracking the actual submission sequence and using it as an assessment criterion. This dynamic approach captures the true problem-solving process while maintaining grading capability
2Adaptability or versatility
If multiple valid solution paths are allowed, then assessment of problem-solving skill becomes possible, but grading complexity increases
Solution Approach 1:
The grading system is designed to be universal and multi-functional, capable of handling multiple valid solution paths and submission sequences through a single unified framework. The system evaluates each step independently and determines correctness based on the specific problem context, allowing it to accommodate various solution approaches without requiring separate grading mechanisms for each path
Solution Approach 2:
The assessment is extended into a new dimension by evaluating not only the correctness of answers but also the sequence and efficiency of step submission. This additional dimensional assessment of problem-solving process efficiency allows the system to handle multiple valid paths while maintaining manageable complexity through structured evaluation criteria
3Measurement precision
If manual grading is used to fairly assign partial credit, then assessment accuracy improves, but efficiency and scalability decrease
Solution Approach 1:
The system implements automated feedback mechanisms that track which steps are correct, which are incorrect, and the sequence of submission. This feedback information is used to automatically assign partial credit according to predefined rules, eliminating the need for manual review while maintaining the accuracy and fairness of partial credit assignment that would otherwise require human graders
Solution Approach 2:
The grading system performs self-service by automatically evaluating submitted steps, determining correctness, and assigning credits without human intervention. The system uses the tracked submission data and correctness information to autonomously compute grades and provide feedback, achieving both high efficiency and accurate partial credit assignment through automated decision-making algorithms
Data Source
AI summary
A process for using a computer to assess problem-solving skill includes a computer program that grades multi-step problems and is made aware of several different but equally valid formulas or methods that use answers to one or more prior steps to arrive at a correct answer to any problem step for which such choices exist. The user is allowed to submit answers, repeatedly if necessary, to any or all of a problem's steps, in any desired order, until all steps are either graded as correct or the correct answer was requested and given. A user's problem-solving skill level is assessed by compiling how many times answers to each step were submitted and on which Tries; if graded as correct, the Try on which credit was given and whether credit was given for a submitted answer, a submitted formula, or an unsubmitted answer for which prior-answer credit was given; if the correct answer was requested, the Try for which the request was made; and if hints are provided, a summary of their use. This data is stored and used to assess a user's skill level.