Risk Scoring System for Exam Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Online and testing center-based candidate evaluation systems face challenges in preventing cheating and identifying fraudulent activities, as existing measures can be circumvented by individuals or organizations.
Innovation Solution
A system comprising a data store server, a model server, and a resource management server that utilizes machine learning models to generate risk scores for candidates, test centers, proctors, and exams, and triggers predefined actions when aggregate risk scores exceed a threshold, such as initiating investigations or enhancing identity verification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional online and testing center-based candidate evaluation systems are used, then convenience and efficiency are improved, but security against cheating and fraudulent activities deteriorates
Solution Approach 1:
The system performs preliminary risk assessment and monitoring during the exam registration event and throughout the exam delivery event. Machine learning models analyze candidate data, test center data, proctor data, and real-time exam delivery event data to generate risk scores before and during the examination, enabling preventive actions rather than reactive measures.
Solution Approach 2:
The system continuously monitors exam delivery event data in real-time and periodically updates risk scores based on this feedback. The machine learning models process ongoing data streams from proctored exams, and when risk scores exceed thresholds, the system automatically triggers predefined actions such as enhanced monitoring or investigation, creating a closed-loop feedback system.
2Reliability
If enhanced monitoring and risk assessment measures are implemented, then security against cheating is improved, but system complexity increases
Solution Approach 1:
The system segments the monitoring and assessment function into distinct machine learning models for different entities: candidate machine learning models, test center machine learning models, proctor machine learning models, and exam delivery event machine learning models. Each model processes specific data types and generates entity-specific risk scores, which are then aggregated by an aggregate machine learning model.
Solution Approach 2:
The machine learning models automatically process data, generate risk scores, and trigger predefined actions without requiring manual intervention. The system self-manages the complex risk assessment process by autonomously analyzing multi-source data, comparing risk scores against thresholds, and executing appropriate responses such as initiating investigations or enhancing monitoring.
3Measurement precision
If real-time monitoring and periodic updates of risk scores are performed, then detection precision of fraudulent activities is improved, but processing time and computational resources increase
Solution Approach 1:
The system performs periodic updates of risk scores during exam delivery events rather than continuous real-time processing. The exam delivery event machine learning model periodically generates updated risk scores based on accumulated exam delivery event data, balancing detection precision with computational efficiency by processing data at strategic intervals throughout the examination.
Data Source
AI summary
Systems and methods may involve processing of entity data by machine learning models to produce one or more entity and/aggregate risk scores and/or aggregate anticipated risk scores, which may be compared to one or more thresholds to determine when one or more predefined actions should be taken. The entity data may be collected for various entities related to an exam registration and delivery process, which may include a candidate, an exam, a test center, an exam registration event, a proctor, and an exam delivery event. The exam registration and delivery process may include multiple states—each being associated with a different set of entities. Aggregate risk scores for a given state may be calculated using only entity data for the set of entities associated with that state. The predetermined actions taken may also be dependent on the current state.