Adaptive Test Scoring With Real-Time Item Difficulty Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing examination systems face challenges in accurately, consistently, and objectively evaluating remotely located candidates while ensuring fair and scalable deployment, particularly in online proctoring contexts, without compromising examination integrity.
Innovation Solution
A distributed computer network system for adaptive testing via real-time continuous scoring, which customizes item selection based on candidate proficiency, using an item bank with pre-classified difficulty levels, continuously recalculates performance profiles, and concludes tests with a conclusive pass or fail based on real-time scoring.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional fixed-question exams are used, then deployment simplicity is maintained, but assessment accuracy and candidate customization are reduced
Solution Approach 1:
The patent implements dynamic item selection where the test adapts in real-time based on candidate responses. The system continuously adjusts question difficulty and selection based on performance data, transforming a static exam into a dynamic assessment that personalized to each candidate's ability level, thereby improving measurement precision without requiring complex manual configuration
Solution Approach 2:
The system incorporates continuous feedback loops where candidate responses are immediately scored and used to adjust subsequent item selection. This real-time feedback mechanism enables the test to respond to candidate performance, selecting easier questions when performance is strong and harder questions when performance declines, thus enhancing assessment accuracy through adaptive feedback-driven item selection
2Measurement precision
If more questions are included in the exam, then assessment thoroughness improves, but testing time and candidate fatigue increase
Solution Approach 1:
The patent applies partial action by selecting only the necessary subset of questions required to achieve adequate assessment precision. Through adaptive algorithms, the system determines the minimum number of items needed to confidently evaluate candidate ability, stopping question administration once sufficient precision is reached, thereby avoiding unnecessary time consumption from excessive questions
Solution Approach 2:
The system dynamically changes the parameter of question quantity based on real-time performance data. As candidates answer questions, the system monitors performance and adjusts the number and difficulty of subsequent questions accordingly, optimizing the balance between assessment thoroughness and testing time by adapting the exam length to individual candidate needs rather than using a fixed predetermined length
3Reliability
If item exposure is increased, then assessment reliability improves, but examination integrity and security are compromised
Solution Approach 1:
The patent implements dynamic item selection that adapts to candidate performance in real-time, continuously adjusting which questions are presented based on response patterns and performance data. This dynamic approach ensures that items are selected optimally for assessment reliability while preventing predictable exposure patterns that could compromise security
Solution Approach 2:
The system uses feedback from candidate responses to continuously refine item selection, monitoring performance trends and adjusting subsequent question choices accordingly. This feedback-driven mechanism ensures reliable assessment by selecting the most informative questions while maintaining security through adaptive selection that prevents memorization and item leakage concerns
Data Source
AI summary
A method and system of adaptive testing via continuous real time scoring. The method comprises scoring a first performance submission received from a candidate computing device in accordance with a first item sourced from the item bank that is presented to the candidate computing device, selecting, from the plurality of unique test items of the item bank, one of a more and a less difficult at least a second item, scoring an at least a second performance submission received from the candidate computing device, generating a candidate performance profile based at least on the first and the at least a second performance submissions received, and assigning either a conclusive fail or a conclusive pass based on the candidate performance profile as generated.