Method for dynamically evaluating student knowledge level based on double attention mechanism

Abstract

The present application relates to the technical field of knowledge tracking, and relates to a method for dynamically evaluating student knowledge level based on a double attention mechanism, comprising the following steps: S1. obtaining interactive information of a student learning process, and grouping the interactive information into a sequence; S2. dividing the interactive sequence into three parts, namely a dynamic problem level sequence, an average skill level sequence and an additional feature sequence; S3. inputting different sequences into corresponding modules for training, and obtaining a student knowledge state through a long short-term memory network and multiple attention mechanisms; S4. inputting the knowledge state into an interpretability module to evaluate the knowledge level; and S5. recording training model evaluation indexes, verifying the interactive sequence of the student through a model updated by parameters, and evaluating the knowledge level of the student. The present application fully mines the interactive information of the student, evaluates the knowledge state of the student from different angles, and improves the accuracy of predicting the future performance of the student.

Description

Technical Field

[0001] This invention relates to the field of knowledge tracking technology, and more specifically, to a method for dynamically assessing students' knowledge levels based on a dual attention mechanism. Background Technology

[0002] Knowledge tracing is a key technology that predicts students' future learning performance by observing their historical learning processes. Deep learning holds great potential in solving knowledge tracing problems because deep neural networks can better represent and process complex data. When combined with high-quality learning materials and guidance, this predictive ability can help students learn better and faster. Therefore, knowledge tracing models have been widely used in intelligent tutoring systems. However, current knowledge tracing models do not fully mine information from student interactions, and their accuracy and fairness are insufficient. Summary of the Invention

[0003] The present invention provides a method for dynamically assessing students' knowledge level based on a dual attention mechanism. This method fully explores students' interactive information, assesses students' knowledge status from different perspectives, and improves the accuracy of predicting students' future performance.

[0004] The method for dynamically assessing students' knowledge level based on a dual-attention mechanism according to the present invention includes the following steps:

[0005] S1. Obtain interactive information about students' learning process from online education platforms, filter out valid values, and form an interactive sequence;

[0006] S2. Divide the interaction sequence into three parts: dynamic question level sequence, average skill level sequence, and additional feature sequence. The dynamic question level sequence is divided according to the dynamic knowledge level, the average skill level sequence is divided according to the average contribution of the knowledge points corresponding to the question, and the additional feature sequence includes the time spent on the question, the hint rate used by the question, and the difficulty of the question.

[0007] S3. Input different sequences into the corresponding modules for training, and obtain the student's knowledge state through the long short-term memory network and multiple attention mechanisms, that is, obtain the information hidden in each part of the interaction sequence;

[0008] S4. Input the knowledge status into the interpretability module to assess the knowledge level, that is, to predict the student's future performance;

[0009] S5. Record the evaluation metrics of the training model, verify the student's interaction sequence through the updated model, and assess their knowledge level.

[0010] As a preferred method, in S1, the online education platform obtains interactive information about the student's learning process, including question ID, answer, time spent, number of hints used, and number of hints set for the question; the interactive information is organized into a sequence, the sequence is cleaned, and invalid data of each feature is removed.

[0011] As a preferred approach, the dynamic question-level sequence includes question ID, answer, and student category.

[0012] As a preferred approach, student classification is based on students' performance in answering questions within their respective skill levels, dynamically clustered according to their knowledge level.

[0013] As a preferred approach, dynamic clustering of student knowledge levels involves dividing interactions over a period of time into several fixed-length segments, and calculating the knowledge level of each segment based on the student's correct answer rate and the time spent answering questions within that segment. The calculation uses the following formula:

[0014]

[0015]

[0016] R(s j ) 1:z =Correct(s j ) 1:z -Incorrect(s j ) 1:z

[0017]

[0018] Student i's knowledge level vector:

[0019] Among them, s j Let j be skill j, 1:z be the time interval from 1 to z, and N be the time interval from 1 to z jt Skill s before time interval t j The total number of exercises; To increase the accuracy of answering questions, Let qs_col be the time spent / total time; R represent learning ability; let qs_col be the list of skills corresponding to problem q, and qs_dict be the dictionary of problem-skill connections, where len(qs_col) is the length of the list, and qs_col x For the element at index x in the skill list, Correct(s) j ) 1:z To represent student i's response to skill s within time intervals 1 to z j The correct performance Let z be a vector representing the learning ability of student i in each skill from time interval 1 to z.

[0020] As a preferred approach, average skill refers to the skill level associated with a problem. The average contribution of the associated skills is taken and averaged. Average skill is obtained by embedding and averaging the knowledge points associated with the problem. Specifically, for each problem, the relevant knowledge points are embedded and averaged to obtain a vector representing the skill level of that problem.

[0021] As a preferred approach, the difficulty of a question is calculated based on the accuracy rate of students' answers and the rate at which hints are used. If a question is answered less than three times, it means that the number of interactions is insufficient to calculate the difficulty, and therefore half the difficulty is assigned.

[0022] As a preferred approach, in S3, the segmented sequence is input into the model, which employs deep learning and is trained on the PyTorch framework. The average skill sequence acquires hidden information between questions through an attention mechanism, calculates the similarity between questions using the average skill sequence, and acquires the student's knowledge state through a multi-head attention mechanism with the dynamic question sequence.

[0023] Preferably, in S4, the additional features are input into a multilayer perceptron (MLP) for computation; through the nonlinear transformation of multiple hidden layers of the MLP, the additional features are mapped to a new high-level representation; then, the additional features processed by the MLP are concatenated with the original knowledge state to obtain a comprehensive knowledge state vector; finally, this comprehensive knowledge state vector is input into the prediction layer to predict the student's future performance.

[0024] As a preferred method, in S5, all AUCs and ACCs during the training process are recorded, the one with the best AUC is selected, and the training model is saved; the model parameters of the best batch are used to update the model, and the model is used to assess the knowledge level and predict the performance of students in the test set; by inputting the data from the test set into the updated model, the knowledge level assessment results and predictions of future performance of students are obtained.

[0025] This invention observes students' learning performance to uncover the one-to-many relationship between questions and skills, thereby calculating the average knowledge point contribution for each question. This process utilizes a self-attention mechanism to capture the implicit relationships between questions and determines the weight of each question based on its importance. By calculating the importance of student-question interactions, a more focused knowledge state on key questions can be obtained, leading to a more accurate representation of students' knowledge levels. This invention achieves good results in terms of accuracy and fairness. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a method for dynamically assessing students' knowledge levels based on a dual-attention mechanism, as described in this embodiment. Detailed Implementation

[0027] To further understand the content of this invention, a detailed description of the invention will be provided in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0028] Example

[0029] like Figure 1 As shown, this embodiment provides a method for dynamically assessing students' knowledge level based on a dual attention mechanism, which includes the following steps:

[0030] S1. Obtain interactive information about students' learning process from online education platforms, filter out valid values, and form an interactive sequence;

[0031] In S1, the online education platform obtains interactive information about the student's learning process, including question ID, answer, time spent, number of hints used, and number of hints set for the question; the interactive information is organized into a sequence, the sequence is cleaned, and invalid data of each feature is removed.

[0032] S2. Divide the interaction sequence into three parts: dynamic question level sequence, average skill level sequence, and additional feature sequence. The dynamic question level sequence is divided according to the dynamic knowledge level, the average skill level sequence is divided according to the average contribution of the knowledge points corresponding to the question, and the additional feature sequence includes the time spent on the question, the hint rate used by the question, and the difficulty of the question.

[0033] The dynamic question-level sequence includes question ID, answer, and student category. Student category is dynamically clustered based on students' performance in the corresponding segment, according to their knowledge level. Dynamic clustering of student knowledge level involves dividing interactions over a period of time into several fixed-length segments, and calculating the knowledge level of each segment based on the student's correct answer rate and time spent answering questions; the calculation uses the following formula:

[0034]

[0035]

[0036] R(s j ) 1:z =Correct(s j ) 1:z -Incorrect(s j ) 1:z

[0037]

[0038] Student i's knowledge level vector:

[0039] Among them, s j Let j be skill j, 1:z be the time interval from 1 to z, and N be the time interval from 1 to zjt Skill s before time interval t j The total number of exercises; To increase the accuracy of answering questions, Let qs_col be the time spent / total time; R represent learning ability; let qs_col be the list of skills corresponding to problem q, and qs_dict be the dictionary of problem-skill connections, where len(qs_col) is the length of the list, and qs_col x For the element at index x in the skill list, Correct(s) j ) 1:z To represent student i's response to skill s within time intervals 1 to z j The correct performance Let z be a vector representing the learning ability of student i in each skill from time interval 1 to z.

[0040] In the context of real-world education, a single question corresponds to multiple knowledge points, meaning that a student's answer to a question affects their mastery of multiple knowledge points. Average skill refers to the skill level associated with a question, which is obtained by averaging the contribution of all related skills. Specifically, for each question, the relevant knowledge points are averaged to obtain a vector representing the skill level of that question.

[0041] The difficulty of a question is calculated based on the accuracy rate of students' answers and the rate at which they use hints. If a question is answered less than three times, it means that the number of interactions is insufficient to calculate the difficulty, and therefore half the difficulty is assigned.

[0042] S3. Input different sequences into the corresponding modules for training, and obtain the student's knowledge state through the long short-term memory network and multiple attention mechanisms, that is, obtain the information hidden in each part of the interaction sequence;

[0043] In S3, the segmented sequence is input into the model, which adopts deep learning and is trained on the PyTorch framework. The average skill sequence obtains hidden information between questions through an attention mechanism, calculates the similarity between questions using the average skill sequence, and obtains the student's knowledge state through a multi-head attention mechanism with the dynamic question sequence.

[0044] The specific steps for S3 are as follows:

[0045] 1) Based on the one-to-many relationship between problems and skills, the average skill contribution corresponding to each problem is determined, and the importance relationship between problems is calculated. The calculation process is shown in the following formula:

[0046]

[0047]

[0048]

[0049] In the above formula, Embedding is an encoding method in torch, and k q For question q, the corresponding skill k is... This represents the average skill level corresponding to question q. The multiple skills corresponding to question q include both direct and implicit skills exposed in the dataset, with their weight ratios adjusted via hyperparameters. N is the total number of skills k. Let Q be the average skill contribution for problem q, where Q is the query matrix, K is the keyword matrix, V is the value matrix, and d is the value matrix. k The dimensions are the query matrix and the keyword matrix. LSTM stands for Long Short-Term Memory Recurrent Neural Network. Using LSTM can effectively transmit and represent information in long-term sequences without ignoring useful information from long ago. Attention is a self-attention mechanism that calculates the importance of each question. The code is as follows:

[0050]

[0051]

[0052] 2) Based on the question weights calculated in step 1), the importance of student question interactions is calculated using a self-attention mechanism. That is, student individual abilities are calculated based on their time spent answering questions, accuracy rate, and hint rate, and the importance of student question interactions is calculated based on the question weights. Furthermore, question interactions are divided into multiple equal-length interaction sequences based on student individual abilities. The calculation process is shown in the following formula:

[0053] q ic =Embedding(q,a,c)

[0054] Q,K = LSTM(q) ic V = q i

[0055]

[0056] In the above formula, the Embedding, LSTM, and Attention structures are the same as those in step 1), and have the same effect. Here, a represents the question answer, and q... ic For dynamic question interaction, q a q represents the problem weights. c is a category derived from clustering students' individual abilities, used to identify students with similar abilities; therefore, its contribution differs across different segments. i The importance of student question interactions is calculated based on the weights of the questions, as determined in step 1). The code is as follows:

[0057]

[0058]

[0059] S4. Input the knowledge status into the interpretability module to assess the knowledge level, that is, to predict the student's future performance;

[0060] In S4, the additional features are input into a multilayer perceptron (MLP) for computation. Through nonlinear transformation of multiple hidden layers of the MLP, the additional features are mapped to a new high-level representation. Then, the additional features processed by the MLP are concatenated with the original knowledge state, that is, the weighted question interaction features calculated in step 2) are aggregated with the additional features to obtain a comprehensive knowledge state vector. Finally, this comprehensive knowledge state vector is input into the prediction layer to predict the student's future performance.

[0061] S5. Record the evaluation metrics of the training model, verify the student's interaction sequence through the updated model, and assess their knowledge level.

[0062] In S5, all AUCs and ACCs during the training process are recorded, the best AUC is selected, and the training model is saved. The model parameters of the best batch are used to update the model, and the model is used to assess the knowledge level and predict the performance of students in the test set. By inputting the test set data into the updated model, the knowledge level assessment results and future performance predictions of students are obtained.

[0063] This embodiment obtains scores of students' knowledge status and designs a loss function to calculate the loss. After training, the model is tested to verify whether it meets the requirements in terms of accuracy and fairness. The results demonstrate that the model achieves good results on multiple datasets.

[0064] This embodiment clusters students based on their individual abilities and optimizes learning outcomes by identifying groups of students with similar abilities. In this scenario, even if two students receive the same learning sequence, their acquired knowledge may differ due to variations in their ability to grasp new information. For example, regarding the ability to grasp new information, if two students have different abilities in this area, their performance in understanding and absorbing knowledge may differ even if they receive the same learning content. One student may be able to understand and apply new knowledge more quickly, while another student may require more time and guidance to reach the same level.

[0065] This embodiment cleverly utilizes the self-attention mechanism to determine the importance of each question, thereby revealing the significance of student question interactions. For example, if a student has a weak grasp of a certain knowledge point, the student can choose to practice more related questions, focusing their attention on the weak areas and thus revealing the student's true knowledge status.

[0066] This embodiment removes invalid data from the dataset, such as negative answer times or invalid responses. Furthermore, when calculating question difficulty, questions with fewer than 3 interactions are removed and assigned a default difficulty level, increasing the reasonableness of the question difficulty.

[0067] The datasets used in the experiment were ASSIST2009 and ASSIST2017, two widely used datasets for researching and developing intelligent assisted education systems. These datasets were collected by Ryan Baker et al. from Carnegie Mellon University. ASSIST2009, in particular, is highly interactive, containing 552,535 interactions, 17,751 questions, 123 skills, and 4,163 users, and was used to train the model.

[0068] In summary, this embodiment has achieved good results in terms of accuracy and fairness in the field of education.

[0069] The present invention and its embodiments have been described above illustratively. This description is not restrictive, and the figures shown are only one embodiment of the present invention; the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for dynamically assessing students' knowledge level based on a dual attention mechanism, characterized by: Includes the following steps: S1. Obtain interactive information about students' learning process from online education platforms, filter out valid values, and form an interactive sequence; S2. Divide the interaction sequence into three parts: dynamic question level sequence, average skill level sequence, and additional feature sequence. The dynamic question level sequence is divided according to the dynamic knowledge level, the average skill level sequence is divided according to the average contribution of the knowledge points corresponding to the question, and the additional feature sequence includes the time spent on the question, the hint rate used by the question, and the difficulty of the question. S3. Different sequences are input into the corresponding modules for training. The student's knowledge state is obtained through a long short-term memory network and multiple attention mechanisms, that is, the information hidden in each part of the interaction sequence is obtained. Specifically, the segmented sequences are input into the model, the model adopts a deep learning approach, and the model is trained on the PyTorch framework; the average skill sequence obtains the hidden information between questions through the attention mechanism, the similarity between questions is calculated based on the average skill sequence, and the student's knowledge state is obtained through a multi-head attention mechanism with the dynamic question sequence. S4. The knowledge state is input into the interpretability module to evaluate its knowledge level. Specifically, additional features are input into a multilayer perceptron (MLP) for computation; through nonlinear transformation of multiple hidden layers of the MLP, the additional features are mapped to a new high-level representation; then, the additional features processed by the MLP are concatenated with the original knowledge state to obtain a comprehensive knowledge state vector; finally, this comprehensive knowledge state vector is input into the prediction layer to evaluate its knowledge level. S5. Record the evaluation metrics of the training model, verify the student's interaction sequence through the updated model, and assess their knowledge level.

2. The method for dynamically assessing students' knowledge level based on a dual attention mechanism according to claim 1, characterized in that: In S1, the online education platform obtains interactive information about the student's learning process, including question ID, answer, time spent, number of hints used, and number of hints set for the question; the interactive information is organized into a sequence, the sequence is cleaned, and invalid data of each feature is removed.

3. The method for dynamically assessing students' knowledge level based on a dual attention mechanism according to claim 2, characterized in that: The dynamic question-level sequence includes question ID, answer, and student category.

4. The method for dynamically assessing students' knowledge level based on a dual attention mechanism according to claim 3, characterized in that: Student classification is based on students' performance in answering questions within their respective skill levels, dynamically clustered according to their knowledge level.

5. The method for dynamically assessing students' knowledge level based on a dual attention mechanism according to claim 4, characterized in that: Dynamic clustering of student knowledge levels involves dividing interactions over a period of time into several fixed-length segments, and calculating the student's knowledge level for each segment based on the student's correct answer rate and time spent answering questions within that segment. The calculation uses the following formula: ； ； ； ； Student i's knowledge level vector: ; in, For skills , The time interval is from 1 to z. Skills before time interval t The total number of exercises; To increase the accuracy of answering questions, Let R represent the time spent / total time; R represents learning ability; let The skill list corresponding to question q For the question-skill connection dictionary, The length of the list. The skill list index is elements, To represent student i's response to the skill during time intervals 1 to z The correct performance Let z be a vector representing the learning ability of student i in each skill from time interval 1 to z.

6. The method for dynamically assessing students' knowledge level based on a dual attention mechanism according to claim 5, characterized in that: Average skill refers to the skill level associated with a problem. It is obtained by averaging the contribution of the associated skills. Specifically, for each problem, the relevant knowledge points are averaged to obtain a vector representing the skill level of that problem.

7. The method for dynamically assessing students' knowledge level based on a dual attention mechanism according to claim 6, characterized in that: The difficulty of a question is calculated based on the accuracy rate of students' answers and the rate at which they use hints. If a question is answered less than three times, it means that the number of interactions is insufficient to calculate the difficulty, and therefore half the difficulty is assigned.

8. The method for dynamically assessing students' knowledge level based on a dual attention mechanism according to claim 7, characterized in that: In S5, all AUCs and ACCs during the training process are recorded, the best AUC is selected, and the training model is saved. The model parameters of the best batch are used to update the model, and the model is used to assess the knowledge level and predict the performance of students in the test set. By inputting the test set data into the updated model, the knowledge level assessment results and future performance predictions of students are obtained.

Images