Bayesian Knowledge Tracing for Mastery Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current educational systems face challenges in predicting student concept mastery leading up to summative assessments, as they often rely on summative assessments that measure mastery at a specific point in time, failing to account for changes over time and instructional gaps that may not be identified until it's too late.
Innovation Solution
A system and method that continually assess and update a student's probability of concept mastery by aggregating test results from formative and summative assessments, using a Bayesian Knowledge Tracing formula modified to consider all answers related to a concept, accounting for false positives and negatives, and iteratively updating the probability of mastery over time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If summative assessments are used to measure concept mastery at a specific point in time, then measurement precision is improved, but the ability to track changes over time is lost
Solution Approach 1:
The system continuously updates the probability of concept mastery by incorporating results from multiple formative assessments administered throughout the instructional period, rather than relying on a single summative assessment. This creates a continuous trajectory of mastery probability that captures changes over time and provides ongoing information about student learning progress.
Solution Approach 2:
The system performs preliminary assessments through multiple formative tests before the final summative assessment to build a probabilistic model of concept mastery. By gathering and analyzing assessment data in advance, the system can predict mastery probability and identify instructional gaps before the summative assessment occurs, allowing for targeted intervention.
2Device complexity
If only the most recent test score is used to compute mastery probability, then computational simplicity is improved, but prediction accuracy deteriorates
Solution Approach 1:
The system pre-computes and stores the probability of concept mastery after each formative assessment using Bayesian Knowledge Tracing. These pre-computed probabilities serve as inputs for subsequent updates, eliminating the need to re-process all raw assessment data from scratch and reducing computational complexity while maintaining accuracy.
Solution Approach 2:
The system uses the previously computed mastery probability as a prior belief and updates it with new assessment evidence to generate an updated mastery probability. This feedback loop incorporates historical performance information systematically, improving prediction accuracy by considering the entire assessment trajectory rather than isolated scores.
3Measurement precision
If multiple formative assessments are administered and aggregated, then prediction accuracy is improved, but instructional time is consumed
Solution Approach 1:
The system can adapt the number of formative assessments administered based on the computed mastery probability. When the probability indicates sufficient mastery, fewer additional assessments are needed. When mastery is uncertain or low, the system can trigger targeted formative assessments on specific concepts, optimizing the balance between assessment accuracy and instructional time usage.
Solution Approach 2:
The system changes the parameters of assessment administration by adapting the frequency, timing, and focus of formative assessments based on the computed mastery probability and identified knowledge gaps. This dynamic adjustment optimizes the trade-off between gathering sufficient data for accurate prediction and preserving instructional time for concept teaching.
Data Source
AI summary
A system and a method for estimating readiness for passing an associated summative assessment includes, after receiving test results for at least one test, generating a score Cn for each concept tested on the test. Using the score, the system computes a probability of mastery p(Mn) of the at least one concept and associates the probability of mastery p(Mn) as a prior for updating the probability after receiving future test results. The system iteratively updates the probability of mastery p(Mn) of the at least one concept after each test using for the new test score and the prior.