Automated Proctoring Risk Scoring System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional automated proctoring systems face high false positive rates and inefficiencies in identifying true exam violations during online examinations, leading to unnecessary administrative burdens and difficulties in ranking exam sessions by risk level.
Innovation Solution
A system and method that processes and analyzes data from online exam sessions to assign point values to flagged events based on their type and severity, applying weights to adjust these values, and generating an overall risk level for each exam session, enabling prioritization of sessions for review by administrators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If automated proctoring systems flag proctoring events during online exams, then suspicious activities are detected, but false positive rates increase to as much as 90%
Solution Approach 1:
The patent introduces an intermediary risk score calculation mechanism that mediates between raw proctoring event data and final violation determination. Instead of directly flagging events as violations, the system calculates risk scores based on multiple factors including event type, frequency, duration, and contextual patterns, then uses these scores to prioritize review. This intermediary layer filters out false positives by requiring threshold exceedance and pattern consistency before flagging actual violations.
Solution Approach 2:
The system transforms the binary classification approach (flagged/not flagged) into a multi-parameter risk assessment model. It changes parameters such as event severity weights, temporal patterns, frequency thresholds, and contextual factors to generate a continuous risk score. This parameter transformation allows nuanced differentiation between genuine violations and false positives, improving measurement precision while maintaining reliability.
2Reliability
If automated proctoring systems monitor all exam sessions, then potential cheating incidents are identified, but administrative time and resources are wasted on low-risk sessions
Solution Approach 1:
The patent applies local quality by differentiating the level of scrutiny applied to different exam sessions based on their risk profiles. High-risk sessions receive intensive manual review, medium-risk sessions receive automated analysis, and low-risk sessions receive minimal or no human review. This localized quality approach ensures that administrative time and resources are concentrated where they are most needed, reducing waste on low-risk sessions while maintaining reliable detection of cheating incidents.
Solution Approach 2:
The system implements feedback loops where outcomes from manual reviews are fed back into the risk scoring algorithm. When reviewers confirm or dismiss flagged events, this information adjusts future risk score calculations and thresholds. This feedback mechanism continuously improves the system's ability to identify high-risk sessions, progressively reducing administrative time spent on false positives while maintaining reliable cheating detection.
3Quantity of substance
If automated proctoring systems use simple tallies of flagged events, then the number of violations is counted, but the overall risk assessment becomes meaningless due to varying severity of events
Solution Approach 1:
The patent segments the overall risk assessment into multiple weighted components: event type severity, duration of violation, frequency of occurrence, contextual factors, and temporal patterns. Each flagged event is broken down into these segments and scored individually, then aggregated into a comprehensive risk score. This segmentation preserves information about severity and context while maintaining a quantifiable measure of total risk, making the assessment meaningful despite the varying nature of different violations.
Data Source
AI summary
Systems and methods for processing and analyzing data collected in connection with online examinations are described herein. The methods may be implemented by one or more computing devices and may include flagging one or more proctoring events indicated by data that was obtained in connection with an exam session of a test taker, the one or more flagged proctoring events being potentially associated with one or more exam rule violations. A point value may be assigned to each of the one or more flagged proctoring events. The point value of at least one of the one or more flagged proctoring events may be adjusted with a weight, the weight being obtained based on data other than mere occurrence of the one or more flagged proctoring events. An overall risk level may be generated based on an overall score obtained by tallying together the adjusted point value or values and unadjusted point value or values, if there are any, of all of the one or more flagged proctoring events, the overall risk level indicating likelihood of exam rule violation in connection with the exam session.