String Interaction Assessment System for Granular Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional multiple-choice assessments are inadequate for evaluating complex concepts, leading to inefficiencies in grading and feedback provision, especially in large-scale educational settings like MOOCs, where teachers struggle to provide meaningful feedback to thousands of students without overwhelming themselves.
Innovation Solution
An assessment system that allows students to provide edited or marked-up text strings, enabling semi-automatic grading and feedback through interactive graphical user interfaces, which includes an authoring subsystem for creating string interaction problems, a grading subsystem for applying criteria, and a feedback subsystem for associating feedback objects with student answers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional multiple-choice assessments are used, then grading is simple and fast, but they are inadequate for evaluating complex concepts and providing meaningful feedback
Solution Approach 1:
The patent segments the grading task by identifying and extracting specific edit locations and types from student answers. The system divides feedback into discrete edit objects that can be independently evaluated and matched against criteria, enabling automated assessment of complex concepts while maintaining grading efficiency.
Solution Approach 2:
The patent introduces an intermediary processing layer that automatically analyzes student edits, extracts meaningful feedback objects, and matches them against predefined criteria. This intermediary system bridges the gap between complex assessment requirements and automated grading, providing both accuracy and efficiency.
2Loss of information
If manual grading is performed to provide granular feedback, then feedback quality is high, but the assessor becomes overwhelmed in large-scale settings
Solution Approach 1:
The patent implements self-service by enabling automated extraction of feedback objects from student answers and automatic matching against criteria. The system serves itself by processing large volumes of student work without requiring proportional increases in assessor time, maintaining feedback quality while scaling to large student populations.
Solution Approach 2:
The patent changes the parameters of feedback delivery by transforming detailed editorial feedback into structured, machine-processable objects with defined properties. This parameter transformation enables automated handling of feedback while preserving its granularity and quality, allowing the system to manage large numbers of students effectively.
3Productivity
If automated grading is implemented, then grading efficiency increases, but feedback becomes less granular and meaningful
Solution Approach 1:
The patent applies preliminary action by pre-defining criteria and feedback objects before student assessment. This preparation enables the automated system to efficiently match student edits against predetermined standards, maintaining both grading speed and feedback meaningfulness through advance structuring of assessment parameters.
Solution Approach 2:
The patent adds another dimension to automated grading by introducing structured feedback objects with multiple properties (location, type, severity, suggested corrections). This dimensional enrichment transforms simple automated matching into a nuanced assessment system that preserves feedback granularity while maintaining high processing efficiency.
4Loss of information
If detailed editorial feedback is provided manually, then feedback is meaningful, but the time required for assessment becomes excessive
Solution Approach 1:
The patent substitutes the mechanical process of manual feedback creation with an automated computational system. The system automatically extracts edit information, creates feedback objects, and matches them against criteria, replacing the time-consuming manual process while preserving feedback meaningfulness through structured data processing.
Solution Approach 2:
The patent uses copying by replicating proven feedback patterns and criteria across multiple student answers. Once feedback objects are created and validated against criteria, they can be efficiently copied and applied to similar edits in other student work, maintaining feedback quality while dramatically reducing assessment time.
Data Source
AI summary
An assessment system may present one or more string interaction problems to each of a plurality of students. Students may provide answers to string interaction problems in the form of string edits that, for example, identify deletions, additions, or revisions to a text string of each respective string interaction problem. The assessment system may normalize each string edit by converting each string edit into a student range edit. Each student range edit defines a replacement text string (which could be an empty text string) to be inserted within a range of reference locations relative to an original text string of the string interaction problem (which likewise could be an empty text string). An assessor may associate feedback objects with student range edits that match or, in some embodiments, are within a predetermined distance of an assessor-defined range edit.