Continuous Probabilistic Assessment for Extendible Adaptive Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing examination systems lack the ability to create standardized and fair tests that are scalable, flexible, and adaptable across diverse educational levels and institutions while maintaining integrity and consistency.
Innovation Solution
A method and system for extendible adaptive testing using continuous probabilistic assessment, which includes a test delivery server computing system that continuously estimates candidate performance (THETA) and updates probabilistic performance bands, determining whether to extend testing based on a predetermined cutoff score, and assigning a conclusive pass or fail status.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If standardized test evaluation criteria are implemented, then grading consistency and fairness are improved, but system complexity and adaptability to diverse educational levels deteriorate
Solution Approach 1:
The system dynamically adjusts test parameters including question difficulty, test length, and item selection based on real-time candidate performance assessment. The adaptive testing engine modifies the test structure during administration to maintain appropriate challenge levels while ensuring standardized evaluation metrics are met.
Solution Approach 2:
The system changes multiple parameters simultaneously including item difficulty levels, test duration, number of questions, and question types based on the candidate's demonstrated ability level. This allows the same standardized test to be effectively administered across diverse educational levels by adjusting these parameters while maintaining consistent grading criteria.
2Adaptability or versatility
If manual test delivery and evaluation are used, then flexibility in administration is improved, but bias and inconsistency in grading increase
Solution Approach 1:
The system implements continuous feedback loops where candidate responses are immediately scored and used to adjust subsequent item selection and test parameters. This automated feedback mechanism eliminates human bias while maintaining flexibility through real-time adaptations based on performance data.
Solution Approach 2:
The adaptive testing engine autonomously makes decisions about item selection, test continuation, and difficulty adjustment without human intervention. The system serves itself by automatically adjusting to each candidate's performance while maintaining standardized evaluation, eliminating the need for manual test delivery and reducing human bias.
3Device complexity
If fixed test length is used, then administrative simplicity is improved, but assessment accuracy and individualization deteriorate
Solution Approach 1:
The test length is dynamically determined based on when the candidate's ability level can be confidently estimated. The system continues administering items until the performance band is sufficiently precise or a maximum limit is reached, allowing each candidate to take a different number of questions while maintaining assessment accuracy.
Solution Approach 2:
The system administers more items than a fixed-length test would require for candidates whose ability is not yet precisely determined, and fewer items for those who reach confidence thresholds quickly. This partial action approach ensures adequate assessment accuracy while avoiding unnecessary test length for competent candidates.
Data Source
AI summary
A method and system of adaptive testing via continuous probabilistic assessment. The method comprises continuously estimating a candidate performance measure (“THETA”) based on scoring a sequence of test performance submissions acquired from a candidate computing device in response to a sequence of items presented to the candidate computing device, determining, responsive to the continuously assessing, a probabilistic performance band that is associated with the THETA, generating, based at least in part on the probabilistic performance band and a predetermined cutoff score, a decision to extend the testing in accordance with an extended sequence of items presented to the candidate computing device, and assigning, based on the extended testing, a conclusive fail status or a conclusive pass status based on cumulatively updating the probabilistic performance band.