LLM Essay Feedback With Human-in-the-Loop Tutor Review
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Tutoring students in essay writing is a labor-intensive task that requires timely, constructive, and encouraging feedback, which existing systems struggle to provide efficiently.
Innovation Solution
A computerized system using a Large Language Model (LLM) generates AI-generated feedback that is reviewed by human tutors, allowing for human-in-the-loop oversight and refinement, ensuring the feedback is encouraging, inquiry-based, and specific to the student's work.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If human tutors review student essays manually, then feedback quality and tone are high, but time consumption and labor intensity increase significantly
Solution Approach 1:
An AI system serves as an intermediary between the student's essay and the human tutor. The AI performs initial analysis of the essay, generating draft feedback on grammar, structure, and content, which the tutor then reviews and refines. This mediator approach allows the tutor to focus on high-value feedback tasks while the AI handles routine analysis, reducing time consumption while maintaining feedback quality.
Solution Approach 2:
The feedback generation process is segmented into multiple stages: AI-generated initial feedback, tutor review, tutor editing/refinement, and final delivery. This segmentation allows different components of feedback to be handled by different agents (AI vs. human), optimizing the balance between automation efficiency and human quality assurance.
2Productivity
If AI systems generate feedback automatically, then time efficiency improves, but feedback quality and pedagogical appropriateness deteriorate
Solution Approach 1:
The system implements a feedback loop where AI-generated feedback is submitted to human tutors for review. Tutors evaluate the AI's feedback quality, make corrections, and provide refined feedback to students. This feedback mechanism ensures that AI efficiency gains do not compromise feedback quality, as human tutors continuously monitor and correct AI outputs.
Solution Approach 2:
The human tutor acts as an intermediary quality assurance layer between the AI system and the student. The tutor reviews AI-generated feedback, ensures it meets pedagogical standards, and makes necessary corrections. This intermediary approach maintains feedback quality while leveraging AI efficiency.
3Speed
If more human tutors are hired to provide timely feedback, then feedback timeliness improves, but operational costs and system complexity increase
Solution Approach 1:
The AI system performs self-service by automatically analyzing essays and generating initial feedback without requiring human tutor intervention for every submission. This automation reduces the number of tutors needed to maintain timely feedback, thereby reducing system complexity and operational costs while preserving feedback speed.
Solution Approach 2:
The AI system performs partial feedback generation, handling routine aspects like grammar and structure, while human tutors focus on higher-level pedagogical feedback. This partial automation approach achieves timely feedback with fewer human resources, reducing system complexity while maintaining speed.
4Productivity
If AI generates all feedback without human review, then labor costs decrease, but feedback appropriateness and student encouragement deteriorate
Solution Approach 1:
Human tutors serve as an intermediary quality assurance layer that reviews AI-generated feedback before it reaches students. This ensures feedback is pedagogically appropriate, encouraging, and suitable for individual student needs, maintaining reliability while benefiting from AI cost efficiency.
Solution Approach 2:
A feedback mechanism allows human tutors to review and correct AI-generated feedback, ensuring appropriateness. This feedback loop maintains high feedback quality while leveraging AI for cost-efficient initial analysis, balancing automation savings with human quality assurance.
Data Source
AI summary
A non-transitory computer-readable medium stores code which when executed by one or more processors of one or more computing devices causes the one or more computing devices to assist a human tutor to assess an essay written by a student by analyzing the essay using a Large Language Model (LLM) to output AI-generated suggested written corrective feedback to the human tutor via a user interface to enable human-in-the-loop (HITL) review of the AI-generated suggested written corrective feedback. Input is received from the human tutor via the user interface to accept, reject or edit the AI-generated suggested written corrective feedback to thereby constitute HITL-AI written corrective feedback. The HITL-AI written corrective feedback is communicated to the student.