DP-MTL Model for Student Assessment via Multi-Task Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing knowledge tracing models fail to effectively utilize the reasoning behind answer choices, ignoring valuable information that could enhance student assessment accuracy, as they primarily rely on binary classification and do not account for the skill level of students when selecting incorrect answers.
Innovation Solution
A DP-MTL model is developed, combining dichotomous and polytonomous learning approaches to predict a student's score based on correct answer probability, incorrect answer selection, and skill level, using a weighted objective function that minimizes the negative log likelihood of correct and incorrect choices, incorporating both KT and OT tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If binary classification (KT model) is used for answer accuracy assessment, then the assessment process is simple, but valuable reasoning information behind answer choices is ignored
Solution Approach 1:
The assessment process is segmented into two distinct tasks: Knowledge Tracing (KT) for binary correctness classification and Option Tracing (OT) for analyzing choice selection among multiple options. This segmentation allows each task to focus on specific aspects of student reasoning, preserving information that would be lost in a single binary classification model.
Solution Approach 2:
The model transitions from a single-dimension binary classification (correct/incorrect) to a multi-dimensional assessment framework that simultaneously evaluates correctness and option selection. This dimensional expansion captures reasoning information by analyzing which specific wrong options students choose, providing insights into their understanding gaps.
2Ease of manufacture
If traditional KT model is used, then implementation is straightforward, but student skill level information is not utilized
Solution Approach 1:
The multi-task learning model serves multiple functions simultaneously: it performs knowledge tracing for correctness prediction, option tracing for choice analysis, and skill level estimation. This multi-functionality allows the system to leverage skill level information to improve assessment accuracy while maintaining a unified model structure that is relatively easy to implement.
Solution Approach 2:
The model incorporates skill level parameters (θ) that dynamically adjust the assessment based on individual student capabilities. By changing the parameters to include skill levels and integrating them into the prediction functions, the model achieves more precise measurements of student understanding without significantly complicating the implementation.
3Device complexity
If only correct answer probability is predicted, then the model is simple, but incorrect answer selection information is ignored
Solution Approach 1:
The prediction process is segmented into two components: correct answer probability prediction and incorrect option selection prediction. The OT task specifically models the probability of selecting each wrong option, capturing valuable information about student reasoning errors while maintaining manageable model complexity through shared underlying structures.
Solution Approach 2:
The model merges the correct answer prediction and incorrect option prediction into a unified multi-task learning framework. By combining these tasks with shared representations and parameters, the model efficiently utilizes incorrect answer selection information without proportionally increasing complexity, as the same student and item representations serve both prediction goals.
Data Source
AI summary
The present disclosure relates to a method of predicting a user's score on a question by an electronic device. The method includes: training a DP-multi tasking learning (DP-MTL) model; verifying the DP-MTL model; receiving choice selection information related to the question from the user through the terminal, and predicting 1) a probability that the user answers the question correctly and 2) the user's score related to the question using the verified DP-MTL model based on the choice selection information, and the DP-MTL model may be a model for predicting the user's score based on 1) information on whether the user answers the question correctly, 2) information on which incorrect answer is selected among choices of the question when the user selects an incorrect answer, and 3) a skill level of the user.