Collusion Detection via Normalized Identity Scores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting cheating in student academic exams are limited to specific exam formats and require modifications for each class, making them ineffective for a wide array of exams and difficult to implement universally, especially in online proctored settings where cheating has increased.
Innovation Solution
A system and method for detecting collusion by calculating identity scores and normalized collusion scores from student exam data, which identifies pairs and groups of students with unusually identical question scores, using statistical calculations and thresholds to determine likely collusion groups, and provides a false positive rate estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If statistical methods are used to detect cheating after an exam, then cheating detection capability is improved, but the methods are only applicable to multiple choice exams and require tuning for specific exam formats
Solution Approach 1:
The patent creates a universal cheating detection system that works across multiple exam formats (multiple choice, true/false, essay, short answer) by using a standardized statistical approach based on answer pattern similarity. The system processes different question types through a common algorithm that calculates similarity metrics between student responses, making the detection method adaptable and versatile rather than format-specific.
Solution Approach 2:
The system adjusts detection parameters dynamically based on exam characteristics. It calculates similarity thresholds and statistical significance levels that account for variations in exam difficulty, question types, and answer distributions. This allows the same core methodology to be applied effectively across diverse exam formats by modifying parameters rather than requiring entirely different approaches for each format.
2Reliability
If proctors are present in the room to prevent cheating, then cheating prevention is improved, but online remote exams make strict proctoring challenging
Solution Approach 1:
The patent replaces the mechanical system of human proctors with an automated computational system that analyzes exam responses statistically. Instead of requiring physical presence and manual monitoring, the system uses algorithms to detect cheating patterns in student submissions, making remote proctoring feasible and reducing operational complexity while maintaining or improving detection reliability.
Solution Approach 2:
The cheating detection system operates autonomously without requiring human intervention during or after exams. It automatically processes student responses, calculates similarity metrics, identifies suspicious patterns, and generates detection reports. This self-service capability eliminates the need for proctors while maintaining effective cheating detection in remote exam settings.
3Ease of operation
If a general method to detect cheating is developed, then ease of implementation across different classes is improved, but must account for variation in number and difficulties of questions, scoring and grading methods, and class size
Solution Approach 1:
The system employs dynamic parameter adjustment that adapts to each exam's specific characteristics. It automatically calculates detection thresholds based on the actual distribution of student responses, question difficulties, and class performance patterns. This dynamic approach allows a single general methodology to handle variations in exam structure and class composition without requiring manual configuration for each specific exam scenario.
Solution Approach 2:
The patent segments the cheating detection process into modular statistical operations that can be applied independently to different aspects of exam data. It separates analysis into question-level similarity calculations, overall response pattern analysis, and statistical significance testing. This segmentation allows the system to handle diverse exam formats and class variations through a structured, reusable framework that maintains implementation simplicity.
Data Source
AI summary
Systems and methods for determining collusion in student academic exams are described. One embodiment includes receiving an input electronic file with academic test data that includes student identifiers, question scores associated with each student identifier, and the total score for each student, calculating an identity score for each pair of students, determining a maximum identity score for each student, generating a normalized collusion score for each student by subtracting from the maximum identity score an average identity score to generate an identity metric, and dividing the identity metric by a local average identity metric to generate a normalized collusion score, where the local average identity metric is an estimated average of a set of identity metrics for a subset of students adjacent to one another on the total test score ranked list, writing student identifiers and normalized collusion scores associated with student identifiers to an output electronic file.