Authorship Verification Using LLM-Based Comprehension Questions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI detection methods fail to effectively assess a student's comprehension of their assignments, allowing students to plagiarize by using AI-generated content without demonstrating understanding.
Innovation Solution
A comprehension-based authorship verification system that uses Large Language Models to generate knowledge-testing questions and evaluate student responses to determine their understanding of submitted work.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional AI detection methods are used to identify plagiarism, then the detection process is simple and quick, but the detection accuracy is low and cannot effectively assess student comprehension
Solution Approach 1:
The patent introduces an intermediary assessment mechanism - a comprehension evaluation system that generates questions based on the submitted work and evaluates student responses. This intermediary process mediates between the simple AI detection need and the comprehensive comprehension assessment requirement, transforming direct plagiarism detection into an indirect verification through student understanding evaluation.
Solution Approach 2:
The system performs preliminary actions by generating comprehension questions and evaluating student responses before making a final plagiarism determination. This preliminary assessment of student understanding provides additional evidence that complements traditional AI detection, improving overall detection accuracy through a multi-stage process.
2Measurement precision
If comprehension-based verification is implemented to accurately detect AI usage, then the detection accuracy improves, but the time required for assessment increases
Solution Approach 1:
The system applies partial action by selectively generating a limited number of targeted comprehension questions rather than comprehensive testing. This allows the system to obtain sufficient evidence of student understanding without requiring excessive time investment, achieving a balance between detection accuracy and time efficiency.
Solution Approach 2:
The system implements skipping by allowing students to answer comprehension questions in a time-efficient manner and using automated evaluation to quickly process responses. This rushes through the assessment process efficiently while still maintaining adequate verification of student comprehension, reducing overall assessment time.
3Ease of operation
If automated question generation and evaluation is used to verify authorship, then the ease of operation improves, but the reliability of detection may be compromised without proper evaluation of student responses
Solution Approach 1:
The system implements feedback mechanisms by evaluating student responses to comprehension questions and using this feedback to inform the final authorship verification determination. This closed-loop feedback process ensures that automated operation does not compromise reliability, as the system continuously adjusts its assessment based on actual student understanding demonstrated through responses.
Data Source
AI summary
The method and system for comprehension-based authorship verification evaluates a user's response to determine how well the user comprehends their own submitted assignment. The user submits their assignment into a validation system where it is analyzed by a Large Language Model (LLM) to generate a question based on the assignment. The generated question is then presented to the user wherein the user provides a user response with a microphone, camera or keyboard. The user response is then analyzed by the LLM again to generate a score that ranks the level of comprehension the user has based on their submitted assignment. Each user response, question, score, and assignment are stored within a database and visually presented to an administrator, wherein the database highlights users that require more attention due to low scores.