Automated Assessment of Constrained Constructed Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional test assessment mechanisms are time-consuming, laborious, and limited in scope, prone to human error, and unable to accurately assess diverse student knowledge and skills, particularly for constructed response items like graphics, markups, and multimedia responses.
Innovation Solution
The development of automated systems and methods for assessing constrained constructed response (CCR) items, including a CCR response analyzer, assessment engine, and action/log engine, which analyze and score student responses automatically, using absolute and substantial correctness criteria, to provide efficient, objective, and reliable evaluations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated assessment systems are implemented, then productivity and assessment speed are improved, but device complexity increases
Solution Approach 1:
The automated assessment system is divided into distinct functional modules: a response analyzer component that processes student responses, an assessment engine that applies evaluation criteria, and an action/log engine that records results. This segmentation allows each component to be optimized independently while maintaining overall system productivity.
Solution Approach 2:
The patent introduces an intermediary processing layer between raw student responses and final assessment results. This intermediary layer includes buffers, normalization components, and criterion-mapping mechanisms that simplify the complexity of direct assessment while maintaining high processing speed.
2Reliability
If manual grading is used, then measurement precision and reliability are maintained, but loss of time increases
Solution Approach 1:
The patent replaces manual mechanical grading processes with automated electronic assessment mechanisms. The system uses computer-based response analysis, algorithmic criterion application, and automated scoring that maintains reliability while eliminating the time-consuming nature of hand-grading.
Solution Approach 2:
The system creates and uses standardized assessment criteria templates and response evaluation models that can be replicated across multiple assessments. This copying approach ensures consistent, reliable evaluation across different students and assessors while significantly reducing the time required for each individual grading process.
3Ease of operation
If selected answer testing is used, then ease of operation is improved, but adaptability decreases
Solution Approach 1:
The automated assessment system is designed with multi-functional capability that handles multiple response types including selected answer, constructed response, graphic responses, markup annotations, and multimedia submissions. This universality allows the system to maintain the ease of automated processing while adapting to diverse assessment formats and educational objectives.
Solution Approach 2:
The system incorporates dynamic response analysis capabilities that can adapt evaluation criteria based on the type of response received. The assessment engine dynamically adjusts its processing approach for different media types (text, graphics, audio, video) while maintaining automated operation, thus preserving ease of use while expanding adaptability.
4Ease of manufacture
If hand grading is used, then ease of manufacture is maintained, but productivity decreases
Solution Approach 1:
The assessment system incorporates self-service capabilities where the automated response analyzer and assessment engine independently process and evaluate student submissions without requiring manual intervention for each grading decision. This self-service approach maintains system simplicity while dramatically improving grading productivity and throughput.
Data Source
AI summary
A system and method for automated assessment of constrained constructed responses provides for automatic, e.g., programmatic, assessing of a test subject response to a constrained constructed response item or question. In one embodiment, paper test materials with test subject markings thereon are converted to electronic form. Test subject markings are further isolated, for example, by registration, sizing and removal of markings other than a graphic response. The subject markings are further compared with a baseline response, correctness deviation criteria and incorrectness deviation criteria (e.g., correctness, incorrectness, substantial correctness, substantial incorrectness, and so on) from which one or more of a score, further learning or other assessment results may be determined. Cluster analysis or other post-evaluation or post-assessment processing may also be conducted, or re-evaluation or re-assessment may also be conducted.