LLM Writing Feedback Using Developmental Rubrics for Timely Depth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated writing feedback systems fail to provide timely, comprehensive, and contextually relevant feedback to students, either offering superficial immediate feedback or requiring significant teacher intervention, thus hindering effective literacy development.
Innovation Solution
A computer-based system that utilizes a large language model (LLM) to generate feedback aligned with a developmental rubric, integrating natural language processing and machine learning to analyze student writing, providing immediate and actionable feedback on claims, evidence, and reasoning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If fully automated scoring systems are used, then immediate feedback is provided, but the feedback becomes superficial and lacks depth
Solution Approach 1:
The patent introduces an intermediary layer between automated scoring and human teaching. A trained machine learning model serves as a mediator that processes student writing samples and generates detailed, contextually relevant feedback incorporating pedagogical frameworks, thereby bridging the gap between speed and depth in feedback delivery.
Solution Approach 2:
The system changes the parameters of automated feedback by incorporating multiple evaluation dimensions (claim clarity, evidence relevance, reasoning quality) and contextualizing feedback within developmental frameworks. This transforms basic automated scoring into comprehensive evaluation while maintaining speed through automated processing.
2Loss of information
If teacher-mediated feedback systems are used, then deep and comprehensive feedback is provided, but the speed of feedback delivery is sacrificed
Solution Approach 1:
The system enables self-service feedback delivery where the machine learning model autonomously evaluates student writing and generates personalized feedback without requiring teacher intervention. This maintains the depth of teacher-mediated feedback while achieving the speed of automated systems.
Solution Approach 2:
The patent replaces the mechanical process of manual grading with an automated machine learning system that uses natural language processing and pedagogical frameworks to generate feedback. This substitution maintains feedback quality while dramatically improving delivery speed.
3Extent of automation
If generic definitions are used for writing elements, then automated feedback can be provided, but the feedback lacks contextual relevance to developmental frameworks
Solution Approach 1:
The system applies local quality by tailoring feedback to specific developmental contexts and student needs. The machine learning model analyzes individual writing samples and generates feedback aligned with grade-level expectations and pedagogical frameworks, making each feedback instance contextually relevant rather than generic.
Solution Approach 2:
The feedback system is dynamic, adapting to different writing genres, student levels, and pedagogical contexts. The machine learning model processes varied input and generates contextually appropriate feedback, transforming static automated feedback into a dynamic, adaptable system.
Data Source
AI summary
A computer-based system and method provides feedback to a student in response to a student text for a provided lesson. An evaluation request including student text response to a lesson passage, a writing prompt, a student identifier, and a lesson identifier is received for evaluation of the student text response. An evaluation rubric is selected from rubrics indexed by the student grade level, student the lesson objective, and/or the lesson identifier. An evaluation data structure corresponding to the request for evaluation is initialized. A large language model (LLM) prompt is formulated based on the evaluation request and the evaluation rubric and provided to an LLM. The evaluation data structure is updated with the response from the LLM and converted to feedback text strings which are presented to the student in the context of the student text response.