Deep knowledge tracing and exercise recommendation methods that integrate multiple features

Abstract

This invention discloses a deep knowledge tracking method and a question recommendation method that integrates multiple features. The deep knowledge tracking method includes the following steps: S01. Pre-training an initial question representation vector matrix by inputting the absolute difficulty matrix of the questions and the similarity relationship between questions; S02. Using the student's answer interaction sequence set and the pre-trained initial representation vector matrix as input, constructing a student answer interaction sequence matrix, a historical relevance matrix, and a multi-knowledge point answer accuracy matrix, and concatenating them to form a comprehensive information matrix, which is then passed through a gating mechanism GLU to obtain the G matrix output; S03. Extracting the student's learning state matrix from the G matrix using multiple one-dimensional convolutional layers; S04. Obtaining the prediction result of the relative difficulty based on the question representation vector to be predicted and the learning state matrix. This invention has the advantages of simple implementation, high prediction accuracy and efficiency, and the ability to effectively handle questions of multiple knowledge points simultaneously.

Description

Technical Field

[0001] This invention relates to the field of intelligent teaching technology, and in particular to a deep knowledge tracking method and exercise recommendation method that integrates multiple features. Background Technology

[0002] With the rise of various intelligent tutoring systems, the relative difficulty of questions plays a crucial role in the field of intelligent education, providing effective support for core tasks such as personalized exercise recommendation and student performance prediction. Therefore, the relative difficulty of questions holds a significant position in education, and predicting the relative difficulty of questions has become an important issue of current concern and research.

[0003] The relative difficulty of a question reflects an individual student's knowledge level. Different students, due to differences in their individual knowledge levels, will exhibit different relative difficulties with the same question. Relative difficulty prediction mainly includes static cognitive diagnosis and knowledge tracing methods. Static cognitive diagnosis assumes that a student's knowledge level remains constant over a certain period. It uses students' answer data to assess their knowledge level and predict the relative difficulty of a question for that student. However, traditional cognitive diagnosis methods cannot obtain the relative difficulty of a massive number of questions in a timely manner, and as the number of questions and students requiring prediction continues to grow, traditional cognitive diagnosis methods can no longer meet current prediction needs. Knowledge tracing technology is currently the mainstream method for predicting relative difficulty. This method aims to track changes in a student's knowledge level in real time based on their historical answer sequences, thereby predicting the relative difficulty or other performance of each question for different students. Compared to static cognitive diagnosis, knowledge tracing dynamically tracks a student's knowledge level over time using their answer sequences. In other words, knowledge tracing focuses more on the student's mastery of knowledge points over time, thus predicting the relative difficulty of questions at different times, and therefore, it is receiving more attention. However, regardless of whether it is a cognitive diagnostic method or a knowledge tracing method, the existing technology for predicting the relative difficulty of a question can usually only achieve prediction for a specific question, and cannot achieve general prediction.

[0004] Among various knowledge tracing models, the CKT (convolutional knowledge tracing) model boasts the highest accuracy in predicting the relative difficulty of questions and the shortest training time. However, the CKT model is a single-knowledge-point model, meaning it assumes that a question contains only one knowledge point, treating the question and the knowledge point as equivalent. Therefore, when using the CKT model, questions with multiple knowledge points can only be simplified to one knowledge point or treated as a single question, thus failing to effectively handle questions with multiple knowledge points. Furthermore, the CKT model ignores rich information contained within the question, such as the absolute difficulty of the question and the similarity relationships between questions, thereby reducing the accuracy of predicting the relative difficulty of questions. Summary of the Invention

[0005] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a deep knowledge tracking method that integrates multiple features, which is simple to implement, has high prediction accuracy and efficiency, and can effectively handle questions with multiple knowledge points at the same time, as well as a question recommendation method with high recommendation accuracy.

[0006] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0007] A deep knowledge tracing method integrating multiple features, comprising the following steps:

[0008] S01. Input the absolute difficulty matrix of the questions and the similarity relationship between the questions to pre-train and obtain the initial representation vector matrix of the questions. The absolute difficulty matrix of the questions is obtained by transforming the representation vector of the questions using a fully connected layer. The similarity relationship between the questions is used to represent whether different questions contain the same knowledge points.

[0009] S02. Using the current student's question-answering interaction sequence set and the pre-trained initial representation vector matrix as input, construct the student question-answering interaction sequence matrix, the historical correlation matrix, and the multi-knowledge point answering accuracy matrix, and then concatenate them to form a comprehensive information matrix F. The comprehensive information matrix F is then processed through a gating mechanism GLU to obtain the output matrix G. The question-answering interaction sequence set includes a question set and a knowledge point set. The student question-answering interaction sequence matrix is ​​used to record the student's historical question-answering interaction sequence. The historical correlation matrix is ​​used to record the correlation between the question that the student needs to predict now and the questions that were answered in the past. The multi-knowledge point answering accuracy matrix is ​​used to record the number of times each knowledge point was answered correctly.

[0010] S03. Extract the student's learning state matrix from the G matrix using multiple one-dimensional convolutional layers;

[0011] S04. Based on the learning state matrix and the question representation vector of the question to be predicted, the prediction result of the relative difficulty of the question to be predicted is output.

[0012] Furthermore, the absolute difficulty matrix of the problem is calculated using the following formula:

[0013]

[0014] Among them, e i Represents the topic p i The representation vector, W diff ∈R K This represents the weight matrix indicating the absolute difficulty of the problem, i.e., the weight matrix of the fully connected layer.

[0015] The similarity between the questions is calculated using the following formula:

[0016]

[0017] in, Represents the topic p i p j The similarity relationship between them, e j Represents the topic p j The representation vector.

[0018] Furthermore, this also includes defining a loss function for the pre-training process:

[0019]

[0020] Among them, R ij The label representing the similarity between questions i and j, d i λ1 and λ2 represent the absolute difficulty label of question i, and are preset coefficients used to adjust the influence of different question information on the question representation vector.

[0021] Furthermore, when constructing the historical relevance matrix, the Masking operation is used to mask the relevance between questions answered after time t and the current time, and the softmax function is used to calculate the relevance between the current question and historically answered questions. The relevance between questions is multiplied by the question-answering interaction sequence vector and then summed to obtain the historical relevance matrix HRP. The calculation formula is as follows:

[0022] w t (i) = Softmax(r) t (i)), i∈(1,N)

[0023] r t (i) = Masking(e i ·e t ), i∈(t,N)

[0024]

[0025] Among them, w t (i) represents the correlation between the question to be predicted at time t and question i, x i Let i represent the representation vector of the question-answering interaction sequence.

[0026] Furthermore, each column in the multi-knowledge-point answer accuracy matrix represents a knowledge point in the dataset. If the current time is t, the predicted question contains a set C of knowledge points. t For the knowledge point set C t Knowledge point c i Multi-knowledge point answering rate matrix MCPC t (c i The calculation formula is:

[0027]

[0028] Among them, c ij This indicates that for knowledge point c i Let's discuss the relationship between student j's answers before time t and their answers at time t. ij == 1 indicates that the knowledge point c at time t i Let j represent the number of times student j answered correctly before time t, where j represents the time sequence of student j's answers.

[0029] Furthermore, in step S02, the student answer interaction sequence matrix LIS, the historical relevance matrix HRP, and the multi-knowledge point correct answer rate matrix MCPC are concatenated according to the following formula to obtain the comprehensive information matrix F;

[0030]

[0031] The comprehensive information matrix F is nonlinearly transformed and dimensionality reduced using a gated GLU mechanism to obtain the G matrix, calculated as follows:

[0032]

[0033] Where W1 and W2 are weight matrices, and b1 and b2 are bias parameters.

[0034] Furthermore, in step S03, a sliding window is set up to take d consecutive answer interactions as input and map each answer interaction to a single output element. Then, a gating mechanism is implemented on the output of the convolutional layer using GLU to simulate the forgetting behavior of students during the learning process. A residual network is added to the convolutional layer to connect the input and output of the convolutional layer. Finally, L identical convolutional layers are stacked to form a hierarchical convolutional layer, where the lower layer is used to capture learning rates that are later than a preset threshold, and the higher layer is used to monitor learning rates that are earlier than the preset threshold.

[0035] Furthermore, in step S03, in step S04, the question representation vector et+1 of the question to be predicted is multiplied by the student's current learning state zt, and then normalized using the softmax function to obtain the relative difficulty of the question relative to the current student. The calculation formula is as follows:

[0036] o t+1 =σ(z) t ·e t+1 ).

[0037] Furthermore, step S03 also includes constructing the objective function as a binary cross-entropy loss function:

[0038]

[0039] Among them, a t This indicates the student's actual answer result, o t This represents the student's predicted answer result from the model.

[0040] A method for recommending practice problems, comprising the following steps:

[0041] When a request for exercise recommendation is received, the system obtains the current student's historical answer information and uses the aforementioned deep knowledge tracing method to predict the relative difficulty of each exercise.

[0042] Based on the prediction results, recommended exercises of the required relative difficulty are selected.

[0043] Compared with existing technologies, the advantages of this invention are as follows: This invention improves upon the traditional deep knowledge tracing model CKT to form a new knowledge tracing model DMCKT. By introducing the absolute difficulty of questions and the similarity relationship between questions, it can accurately model the student's answering interaction process, effectively improving the accuracy of predicting the relative difficulty of questions. At the same time, considering that some questions may correspond to multiple knowledge points, multiple knowledge points are used as input. The knowledge tracing model is trained using the relevant information of the questions and the answering interaction sequence of each student. It can handle the interaction sequence between students and questions with multiple knowledge points, thereby making full use of the multiple knowledge points contained in the questions and quickly and accurately predicting the relative difficulty of the questions for different students. Attached Figure Description

[0044] Figure 1 This is a schematic diagram illustrating the implementation process of the deep knowledge tracking method that integrates multiple features in this embodiment.

[0045] Figure 2 This is a comparative diagram of the DMCKT model in this embodiment and the traditional CKT model.

[0046] Figure 3 This is a schematic diagram of the structural principle of the DMCKT module in this embodiment. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0048] like Figure 1 As shown, the steps of the deep knowledge tracking method that integrates multiple features in this embodiment include:

[0049] S01. Input the absolute difficulty matrix of the questions and the similarity relationship between the questions to pre-train and obtain the initial representation vector matrix of the questions. The absolute difficulty matrix of the questions is obtained by transforming the representation vector of the questions using a fully connected layer. The similarity relationship between the questions is used to represent whether different questions contain the same knowledge points.

[0050] S02. Using the student's question-answering interaction sequence set and the initial representation vector matrix obtained from pre-training as input, construct the student question-answering interaction sequence matrix, the historical correlation matrix, and the multi-knowledge point answering rate matrix, and then concatenate them to form a comprehensive information matrix F. The comprehensive information matrix F is then passed through the gating mechanism GLU to obtain the output matrix G. The question-answering interaction sequence set includes the question set and the knowledge point set. The student question-answering interaction sequence matrix is ​​used to record the student's historical question-answering interaction sequence. The historical correlation matrix is ​​used to record the correlation between the question that the student needs to predict now and the question that was answered in the past. The multi-knowledge point answering rate matrix is ​​used to record the number of times each knowledge point was answered correctly.

[0051] S03. Extract the student's learning state matrix from the G matrix using multiple one-dimensional convolutional layers;

[0052] S04. Based on the learning state matrix and the question representation vector of the question to be predicted, the prediction result of the relative difficulty of the question to be predicted is output.

[0053] This embodiment improves upon the traditional Deep Knowledge Tracing (CKT) model to form a new knowledge tracing model, DMCKT (Absolute Difficulty and Multiple Knowledge Points Enhanced Convolutional Knowledge Tracing). The DMCKT model introduces the absolute difficulty of questions and the similarity relationships between questions, enabling accurate modeling of students' answering interactions and effectively improving the accuracy of predicting the relative difficulty of questions. Simultaneously, considering the possibility that some questions may correspond to multiple knowledge points, the CKT model, which assumes one question corresponds to one knowledge point, is improved. A knowledge tracing model is trained using relevant information about the questions (including absolute difficulty, similarity relationships between questions, and all knowledge points included in the question) and each student's answering interaction sequence. This allows the model to predict the relative difficulty of questions at different times. Therefore, the improved DMCKT knowledge tracing model can fully utilize the multiple knowledge points contained in a question, enabling rapid and accurate prediction of the relative difficulty of a question for different students.

[0054] The DMCCT model in this embodiment is compared with the traditional CKT model, for example... Figure 2 As shown, the traditional CKT model takes question q as input. i Answer result a i Or enter knowledge point s i、 Answer result a i Furthermore, CKT is a single-knowledge-point model (each question contains only one knowledge point). In CKT, questions and knowledge points are treated as equivalent. For some datasets, CKT uses the knowledge point ID as the question ID; for other datasets, CKT directly uses the question ID. In this embodiment, the DMCKT model simultaneously inputs the question q. i Knowledge Points i Answer result a i Specifically, the set of questions is defined as follows: The students gathered as The knowledge points are Where, p i Representing the problem i, s j Let j and c represent students. k Representing knowledge points k, M, and N s and N cThese represent the number of questions, the number of students, and the number of knowledge points, respectively. Furthermore, the student's question-answering interaction sequence is defined as H. j ={(s j ,p1,c1,a1),…,(s j ,p t ,c t ,a t ),…,(s j ,p N ,c N ,a N )}. Where p t c represents the question that student j answered at time t; t This represents the knowledge points included in the questions answered by the student at time t; a t This represents the student's answer at time t, and its value can be either 1 or 0. t =1 means that student j answered the question correctly, a t =0 indicates that student j answered the question incorrectly; N is the maximum length of the student-answer interaction sequence; the total number of student-answer interaction sequences H in the student set S includes the question set P and the knowledge point set C. Assume that each student learns independently, without inter-student interaction. The model is trained using the question-answer interaction sequence set H, and the model can predict the relative difficulty of the questions at time t+1.

[0055] In actual teaching, a single problem may involve multiple knowledge points (for example, an SQL programming problem may involve join queries, aggregate functions, sorting, etc.), and the intelligent teaching system needs to accurately and promptly predict the relative difficulty of the problems. Traditional models used to predict the relative difficulty of problems either have high prediction accuracy but excessively long training times, or high prediction accuracy and relatively short training times but cannot support problems with multiple knowledge points. Although the CKT model has high prediction accuracy, it equates problems with knowledge points. For the three input matrices constructed by the input module, they must either be uniformly represented by problems or uniformly represented by knowledge points (e.g., ...). Figure 2 As shown), it is impossible to fully utilize the multiple knowledge points contained in the question. This embodiment of DMCKT enriches the question input by utilizing the absolute difficulty of the question and the similarity between questions. Furthermore, considering that a question contains multiple knowledge points, this embodiment completely separates the question and the knowledge points (i.e., using different representations in different matrices, such as...). Figure 2 As shown in the figure, this enables the DMCKT model to support the situation where a question corresponds to multiple knowledge points, and to make full use of the multiple knowledge points contained in the question, thereby improving the accuracy of predicting the relative difficulty of the question.

[0056] The DMCKT model can make full use of information about the questions (including the absolute difficulty of the questions and the similarity between questions), making the model's input richer and thus improving the accuracy of predicting the relative difficulty of the questions. At the same time, considering that in real teaching environments, a question may correspond to multiple knowledge points, and traditional CKT models cannot handle multiple knowledge points in a question, the DMCKT model in this embodiment takes multiple knowledge points as input at the same time, and can handle the interaction sequence between students and questions with multiple knowledge points.

[0057] like Figure 3 As shown, the DMCKT model in this embodiment specifically includes the following four modules:

[0058] (1) Pre-training module, corresponding to step S01, is used for pre-training. In order to enrich the question information in the DMCKT model, the initial representation vector of the question is obtained by pre-training using the absolute difficulty of the question and the similarity relationship between the questions.

[0059] (2) Input Module: Corresponding to step S02, in order to utilize multiple knowledge points in the question, in addition to the question labels and corresponding answer results in the student's answer interaction sequence, the DMCKT model in this embodiment also uses the knowledge points corresponding to the question as input to the model. Specifically, the DMCKT model does not equate the question with the knowledge points, but rather uses the multiple knowledge points contained in the question as part of the input. Therefore, the DMCKT model needs to input the question in the student's answer sequence, the knowledge points contained in the question, and the answer sequence of the question. Then, three matrices are constructed in the input module: student answer history matrix, historical correlation matrix, and multi-knowledge point answer rate matrix. The student answer history matrix is ​​used to record the student's past answer interaction sequences, the historical correlation matrix is ​​used to record the correlation between the question that the student needs to predict now and the questions answered in the past, and the multi-knowledge point answer rate matrix is ​​used to record the number of times all knowledge points are answered correctly. Finally, the above three matrices are concatenated to obtain an F matrix containing student answer information, question information, and knowledge point information. The GLU mechanism can perform non-linear processing on the F matrix, thereby controlling the flow of effective information into the subsequent student learning rate modeling process. Therefore, after passing the F matrix through the GLU gate mechanism, the G matrix is ​​obtained, which contains information about the questions and students during the student's interactive response process.

[0060] (3) Student learning rate modeling module, corresponding to step S03, is used to extract the learning status of students from the G matrix by using multiple one-dimensional convolutional layers to extract learning rate information of different students.

[0061] (4) Output module, corresponding to step S04, is used to perform a dot product between the student's learning state and the question representation vector to be predicted, and output the probability that the student can answer the question correctly, that is, the relative difficulty of the question. If the student can answer the question correctly, it means that the question is easy for the student, that is, the relative difficulty is small; conversely, if the student cannot answer the question correctly, the question is difficult for the student, that is, the relative difficulty is large.

[0062] The detailed implementation process of each model in this embodiment of the DMCKT model is as follows:

[0063] Step S01: Pre-training

[0064] To enrich the vector representation of the questions and thus improve the accuracy of predicting the relative difficulty of the questions, this embodiment first introduces two types of question information during pre-training: the absolute difficulty of the questions and the similarity relationship between the questions.

[0065] (1) Question Information

[0066] The absolute difficulty of a question is one of its key pieces of information. Different questions have different absolute difficulty values ​​due to their own characteristics (such as the knowledge points they contain and the descriptive language in the question stem), and the absolute difficulty of different questions has a significant impact on learning outcomes. The similarity between questions is also important information. The similarity between two questions is defined as whether they share the same knowledge points. Therefore, the similarity between questions can be captured by examining the knowledge points they contain. A question may contain multiple knowledge points; therefore, the similarity between two questions can be defined by the number of knowledge points that both questions contain, as shown in Formula 1.

[0067]

[0068] Among them, R ij This indicates the similarity between problem i and problem j. and They represent the topic p respectively i Number of knowledge points and questions included j Number of knowledge points included Then it means that the question p i and p j The number of knowledge points included.

[0069] (2) Pre-training

[0070] This embodiment introduces the absolute difficulty of questions and the similarity relationship between questions into the CKT model to enrich the question information input and thus improve the accuracy of predicting the relative difficulty of questions. The goal of pre-training is to improve the accuracy of predicting the relative difficulty of questions by inputting the absolute difficulty D∈R. M The similarity relationship between the problem and the problem is R∈RM×M The training yields a rich problem representation matrix E∈R M ×K , where K is the dimension of the problem representation vector.

[0071] Specifically, given that the absolute difficulty of a question is a reflection of the question itself, this embodiment uses a fully connected layer to transform the question's representation vector into the question's absolute difficulty. The specific calculation is shown in formula (2).

[0072]

[0073] Among them, e i Represents the topic p i The representation vector, W diff ∈R K This represents the absolute difficulty weight matrix of the problem, i.e., the weight matrix of the fully connected layer.

[0074] In this embodiment, cosine similarity is used to calculate the similarity between the representation vectors of two questions i and j. The specific calculation is shown in formula (3):

[0075]

[0076] To simultaneously incorporate the relationship between absolute difficulty and the questions, this embodiment further defines the loss function of the pre-training process as the sum of two squared loss functions, as shown in Formula 4:

[0077]

[0078] Among them, R ij The label representing the similarity between questions i and j, d i This represents the absolute difficulty label of question i. λ1 and λ2 are coefficients in this module, used to adjust the influence of different question information on the question representation vector. Specifically, when λ1 > λ2, the absolute difficulty of the question has a greater impact on the question representation vector than the similarity relationship between questions; conversely, the similarity relationship between questions has a greater impact on the question representation vector.

[0079] Step S02. Input data into the input module.

[0080] After pre-training, the student's question-answering interaction sequence set H and the initial question representation vector matrix E∈R obtained during the pre-training process are first combined. M×K As input to the DMCKT model, the student response interaction sequence matrix (LIS), historical relevance matrix (HRP), and multi-concept percent correct (MCPC) matrix are then constructed:

[0081] (1) Student answer interaction sequence matrix

[0082] Taking a student as an example, if the student answers a question correctly at a certain moment, then the interaction vector x at that moment is... t =[e t ,a t Otherwise x t =[a t ,e t ]. Among them, e t ∈R K Let a represent the representation vector of this problem. t ∈R K This indicates the answer to the question, a. t All values ​​are 0. Furthermore, if a student's answer sequence length is greater than N, then a truncated answer sequence of length N is taken; conversely, if a student's answer sequence length is less than N, it is padded with 0s. Therefore, the student's answer sequence matrix LIS∈R N×2K .

[0083] (2) Historical correlation matrix

[0084] When constructing the historical correlation matrix, taking a student as an example, assuming the current time is t, in order to avoid the question sequence after time t from affecting the historical correlation matrix, the Masking operation is used to mask the correlation between the questions answered after time t and the current time. The calculation formula is shown in formula (5).

[0085] r t (i) = Masking(e i ·e t ),i∈(t,N) (5)

[0086] Among them, e i This represents the question the student answered after time t, e t This represents the question the student answered at time t.

[0087] The softmax function is then used to calculate the correlation between the current question and the historical answers, as shown in formula (6). Finally, the correlation between questions and the question-answering interaction sequence vector are multiplied and summed to obtain the matrix HRP∈R. N×2K As shown in formula (7).

[0088] w t (i) = Softmax(r) t (i)), i∈(1,N) (6)

[0089]

[0090] Among them, wt (i) represents the correlation between the question to be predicted at time t and question i, x i This represents the representation vector of the question-answering interaction sequence i.

[0091] (3) Multi-knowledge point answer rate matrix

[0092] To fully utilize the multiple knowledge points in the questions, this embodiment includes the knowledge points contained in the questions as input in the input module, and constructs a knowledge point correct answer rate matrix MCPC∈R. N×Nc Each column of this matrix represents a knowledge point in the dataset. The specific operation for updating the multi-knowledge-point accuracy matrix is ​​as follows: For a given student, assuming the current time is t, the predicted question is e. t And e t Includes a set of knowledge points C t For the knowledge point set C t A certain knowledge point c i The calculation formula is shown in formula (8).

[0093]

[0094] Among them, c ij This indicates that for knowledge point c i Let's discuss the relationship between student j's answers before time t and their answers at time t. ij == 1 indicates that the knowledge point c at time t i Let j represent the number of times student j answered correctly before time t, where j represents the time sequence of student j's answers.

[0095] After obtaining the three matrices mentioned above, they are concatenated to obtain matrix F∈R. N×(4K+Nc) The calculation formula is shown in Formula 9. This matrix contains various information such as the questions and knowledge points answered by the students.

[0096]

[0097] The F matrix is ​​further transformed nonlinearly and reduced in dimension by the gating mechanism GLU

[84] to obtain G∈R N×K The matrix is ​​calculated using the formula shown in Formula 10.

[0098]

[0099] Where W1∈R (4K+Nc)×K and W2∈R (4K+Nc)×K Both are the weight matrices of the model, b1∈R K and b2∈R K This represents the bias of the model.

[0100] Step S03. Modeling Student Learning Rate

[0101] To obtain the learning status of different students, the student learning rate modeling module in this embodiment uses a one-dimensional convolutional network suitable for processing sequence data to extract the student learning status matrix Z∈R from the G matrix. N×K Specifically, the sliding window is set to W. s ∈R 2d×K b s ∈R K To prevent convolutional operations involving subsequent question-and-answer interactions, the latter half of the sliding window is masked. In this embodiment, a sliding window takes d consecutive question-and-answer interactions as input and maps these interactions to a single output element. The number of feature maps is also set to K. Then, a simple gating mechanism is implemented on the output of the convolutional layer using GLU to simulate the forgetting behavior of students during the learning process.

[0102] To accelerate the training process, this embodiment further adds a residual network to the convolutional layers, connecting the input and output of the convolutional layers. Finally, L identical convolutional layers are stacked to form a hierarchical convolutional layer, where lower layers (less than a preset layer threshold) are used to capture recent (later than a preset time threshold) learning rates, and higher layers (more than a preset layer threshold) are used to monitor learning rates over more distant (earlier than a preset time threshold) periods. The student learning rate modeling module ultimately outputs the student's learning state matrix Z∈R. N×K .

[0103] Step S04. Output module outputs results.

[0104] This embodiment focuses on a student in the learning state at time t, and assigns the question representation vector e of the question to be predicted. t+1 and the student's current learning status z t Perform a dot product, then normalize using the softmax function to obtain the relative difficulty of the question relative to the student. The specific calculation is shown in formula (11). Specifically, if o t+1 If the value is greater than 0.5, the student can answer the question correctly, indicating that the question is easy for the student; otherwise, the question is difficult for the student.

[0105] o t+1 =σ(z) t ·e t+1 (11)

[0106] Furthermore, in order to learn all the parameters in the DMCKT model, this embodiment constructs the objective function of the DMCKT model as the binary cross-entropy loss function, as shown in formula (12).

[0107]

[0108] Among them, a t This indicates the student's actual answer result, o t This represents the student's predicted answer result from the model.

[0109] This embodiment uses the knowledge tracing method described above to construct a new knowledge tracing model, DMCKT, by introducing the absolute difficulty of the question, the similarity between questions, and the knowledge points contained in the question. This model can be used to predict the relative difficulty of the question. It can make full use of information such as the absolute difficulty of the question, the similarity between questions, and multiple knowledge points to achieve fast and accurate prediction of the relative difficulty of the question.

[0110] To verify the effectiveness of this invention, experiments were conducted on a self-collected SQL dataset and an open-source student answer interaction sequence dataset, EdNet2 (this embodiment only uses the KT1 dataset). The SQL dataset underwent preprocessing, removing students whose answer interaction sequences were no longer than 3 characters. The resulting SQL dataset contained 318 SQL programming questions, 283 students, 10952 student answer records, and 27 knowledge points, as shown in Table 1. Since the EdNet dataset lacks absolute difficulty labels for questions, this embodiment uses the student's error rate on a question as the absolute difficulty value. Furthermore, similar to the SQL dataset, the EdNet dataset was also preprocessed before the experiment, removing students whose answer interaction sequences were no longer than 3 characters and removing questions with problematic knowledge point labels. The final processed EdNet dataset contained 5002 students with 222141 answer records, covering 188 knowledge points and 10799 questions, as shown in Table 1.

[0111] Table 1. Preprocessed student response sequence dataset

[0112]

[0113] This embodiment specifically uses the evaluation metrics AUC (Area under the ROC) and accuracy (ACC) to evaluate the effectiveness of the invention in predicting the relative difficulty of the questions, wherein:

[0114] (1) AUC

[0115] In binary classification problems, a result where the classifier predicts a positive example and the true label is positive is denoted as TP (Truepositive); while a result where the classifier predicts a positive example but the true label is negative is denoted as FP (False positive). Similarly, a result where the classifier predicts a negative example and the true label is negative is denoted as TN (True negative); while a result where the classifier predicts a negative example but the true label is positive is denoted as FN (False negative).

[0116] AUC is the area under the Receiver Operating Characteristic (ROC) curve. The horizontal axis of the ROC curve represents the False Positive Rate (FPR), as shown in formula (13), and the vertical axis represents the True Positive Rate (TPR), as shown in formula (14). The ROC curve can be obtained by continuously calculating the false positive rate and the true positive rate of the samples. However, since the number of samples is generally limited, a smooth ROC curve cannot be obtained. Therefore, in binary classification problems, the area under the ROC curve, AUC, is generally used as an evaluation index, where a higher AUC value indicates a better prediction result.

[0117]

[0118]

[0119] (2) Accuracy

[0120] ACC refers to the proportion of correctly classified samples out of the total number of samples. Similar to the evaluation metric AUC, a higher ACC value indicates a higher accuracy in model prediction and a better model performance. The specific calculation of this evaluation metric is shown in formula (15).

[0121]

[0122] Where r represents the number of correctly classified samples, and total represents the total number of samples.

[0123] To test the effectiveness of the DMCKT model of this invention, this embodiment compares the DMCKT model with the most typical and popular knowledge tracing models, including the DKT model, the DKVMN model, and the CKT model. The DKT model is the first deep knowledge tracing model to apply deep learning to the field of knowledge tracing, using an LSTM model to model the student's question-answering interaction process. The DKVMN model borrows the idea of ​​the MANN model to model the student's question-answering interaction process. The CKT model is the first to introduce a CNN model into a knowledge tracing model, enabling it to model the student's question-answering interaction process. Compared with other typical knowledge tracing models, the CKT model outperforms the aforementioned deep knowledge tracing models in both prediction accuracy and training time.

[0124] In the specific experimental setup, to ensure fairness, 60% of the data in the dataset was randomly selected as the training set, 20% as the test set, and the remaining data as the validation set. The performance of the models DKT, DKVMN, and CKT used for comparison on both datasets was optimized. Because the SQL dataset and the EdNet dataset differ significantly in size, different DKVMN and CKT model parameters were set for each dataset; specific parameter settings are detailed in Table 2. The software environment used in this embodiment was: Python 3 programming language, Tensorflow deep learning framework, and an 8-core CPU, 64GB of RAM, a 1.5TB hard drive, and a 64-bit Ubuntu operating system.

[0125] Table 2 Parameter settings of the DMCKT model on different datasets

[0126]

[0127]

[0128] To verify the effectiveness of the CKT model of this invention in predicting the relative difficulty of questions, the CKT model of this invention was compared with traditional typical knowledge tracing models (namely DKT model, DKVMN model and CKT model). The specific experimental results are shown in Tables 3 and 4.

[0129] Table 3. AUC results of knowledge tracing

[0130]

[0131] Table 4. Analysis of ACC Results in Knowledge Tracking

[0132]

[0133] As shown in Tables 3 and 4, on the SQL dataset, the DMCKT model of this invention has the best RMSE and MAE results, followed by the CKT model, and then the DKVMN model. The DKT model has the worst results in both evaluation metrics. These results indicate that the DMCKT model of this invention can effectively utilize information about the questions (including the absolute difficulty of the questions, the similarity between questions, and the knowledge points contained in the questions), thereby further improving the accuracy of the relative difficulty prediction. The CKT model only utilizes the student's answer interaction sequence and ignores the rich information contained in the questions. The DKT model also only utilizes the student's answer interaction sequence, and since this model automatically learns all the potential knowledge points in the question set, it cannot directly utilize the knowledge points contained in the questions like the DMCKT model, and therefore cannot obtain good results in predicting the relative difficulty of the questions. Compared with the DKT model, the DKVMN model extends the DKT model by two matrices to store potential knowledge points, so its RMSE and MAE structures are better than those of the DKT model. However, this model also only uses the student's answer interaction sequence to predict the accuracy of the relative difficulty of the questions, and therefore cannot fully utilize the rich information contained in the questions.

[0134] Furthermore, on the EdNet dataset, the DMCKT model of this invention achieves the best RMSE and MAE results, followed by the DK model, then the DKVMN model and the DKT model, with little difference between the two. However, the CKT model performs the worst in both RMSE and MAE on this dataset. This is because the CKT model assumes that questions and knowledge points are equivalent, leading to two problems: 1) When the CKT model uses the question labels from this dataset as the final question numbers, the knowledge point answer rate matrix in the input module becomes sparse due to the much larger number of questions than knowledge points in the EdNet dataset, thus affecting the accuracy of relative question prediction; 2) When the CKT model uses knowledge point labels as the final question numbers, it ignores the characteristics of different questions and therefore cannot accurately predict the relative difficulty of questions. In contrast, the DKT and DKVMN models use student answer interaction sequences as input, and the knowledge points are automatically learned during the training phase. Therefore, the performance of these two models on the EdNet dataset is not significantly different from their performance on the SQL dataset. In summary, the experimental results in Tables 3 and 4 show that, regardless of whether the SQL dataset or the EdNet dataset is used, the DMCKT model of this invention outperforms the three traditional deep knowledge tracing models DKT, DKVMN, and CKT on both the RMSE and MAE evaluation metrics. That is, compared with traditional knowledge tracing models, the DMCKT model of this invention not only uses the student's answer interaction sequence as input for predicting the relative difficulty of questions, but also fully utilizes the question information in the EdNet dataset to further improve the accuracy of predicting the relative difficulty of questions, thus achieving superior prediction results.

[0135] This invention improves the traditional CKT model, enhancing its accuracy in predicting the relative difficulty of questions. To highlight the contributions of different improvements in the DMCKT model, this embodiment further conducted ablation experiments, the results of which are shown in Tables 5 and 6. Before analyzing the experimental results, the symbols defined in Tables 5 and 6 are explained for ease of analysis. In Tables 5 and 6, ① represents the original CKT model; ①+② represents the improved knowledge tracking model supporting multiple knowledge points based on the original CKT model; and ①+②+③ represents the improved DMCKT model based on the original CKT model, which supports multiple knowledge points and can incorporate various question information.

[0136] Table 5. AUC results of DMCKT model ablation experiments

[0137]

[0138] Table 6. ACC results of DMCKT model ablation experiments

[0139]

[0140] As shown in Tables 5 and 6, for the SQL dataset, the experimental results of ①+② on both evaluation metrics AUC and ACC are superior to those of ①, indicating that the improved knowledge tracing model supporting multiple knowledge points performs better than the original CKT model. This is because the original CKT model assumes that questions and knowledge points are equivalent, thus ignoring the information of multiple knowledge points contained in a question. The improved DMCKT model of this invention, supporting multiple knowledge points, can effectively utilize the knowledge point information contained in the question, thereby improving the accuracy of predicting the relative difficulty of the question. Similarly, on the EdNet dataset, the experimental results of ①+② on both evaluation metrics AUC and ACC are also superior to those of ①, fully demonstrating the effectiveness of supporting multiple knowledge points in the DMCKT model of this invention and highlighting the contribution of supporting multiple knowledge points to the experimental results of the DMCKT model.

[0141] As can be seen from the ①+②+③ examples of the SQL and EdNet datasets, the addition of a pre-training process to the improved knowledge tracking model supporting multiple knowledge points improves the performance on both datasets in terms of evaluation metrics AUC and ACC. However, ①+②+③ on the SQL dataset does not show a significant improvement over ①+②, while ①+②+③ on the EdNet dataset shows a more significant improvement over ①+②. This indicates that the pre-training process introduced in the DMCKT model of this invention is effective, but the improvement is more pronounced on the EdNet dataset than on the SQL dataset. This is because the average number of knowledge points per question in the SQL dataset is less than that in the EdNet dataset, resulting in a sparser matrix recording the similarity relationships between questions during pre-training using the SQL dataset. Consequently, the input for pre-training is not as rich as that used with the EdNet dataset, indicating that pre-training has a more significant improvement effect on datasets with a larger average number of knowledge points per question.

[0142] This invention improves upon the high-performance traditional knowledge tracing model CKT by introducing the absolute difficulty of questions, the similarity relationships between questions, and the knowledge points contained in the questions to construct a new knowledge tracing model, DMCKT, for predicting the relative difficulty of questions. Experimental results consistently demonstrate that the DMCKT model of this invention effectively improves the accuracy of predicting the relative difficulty of questions. Specifically, on the SQL dataset, the AUC of the DMCKT model of this invention is improved by an average of 7.59% compared to the state-of-the-art deep knowledge tracing model CKT, and on the open-source EdNet dataset, the AUC is improved by an average of 8.51% compared to CKT and by an average of 6.55% compared to the best-performing DKT model in comparative experiments.

[0143] Knowledge tracking can be used to output the difficulty value of questions relative to different students in real time. Furthermore, this knowledge tracking can be used to achieve personalized exercise recommendations. This embodiment further provides an exercise recommendation method, the steps of which include:

[0144] When a request for exercise recommendations is received, the system obtains the current student's historical answer information and uses the aforementioned deep knowledge tracing method to predict the relative difficulty of each exercise.

[0145] Based on the prediction results, recommended exercises of the required relative difficulty can be selected, allowing for precise recommendations of exercises of suitable difficulty for different students.

[0146] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A deep knowledge tracing method fusing multiple features, characterized by the steps of Comprising: S01. Input the absolute difficulty matrix of the questions and the similarity relationship between the questions to obtain the initial representation vector matrix of the questions, the absolute difficulty matrix of the questions is obtained by converting the representation vector of the questions by using a full connection layer, and the similarity relationship between the questions is used to represent whether the different questions contain the same knowledge points; S02. Using the current set of students' answer interaction sequences and the pre-trained initial representation vector matrix as input, construct the student answer interaction sequence matrix, the historical relevance matrix, and the multi-knowledge point answer accuracy matrix, and then concatenate them to form a comprehensive information matrix. F The comprehensive information matrix F After passing through the gating mechanism GLU, the following is obtained: G The output matrix includes a set of questions and a set of knowledge points. The student question-answering interaction sequence matrix records the student's historical question-answering interaction sequences. The historical correlation matrix records the correlation between the question the student currently needs to predict and the questions answered in the past. When constructing the historical correlation matrix, a masking operation is used to mask the correlation between questions answered after time t and the current time, and a softmax function is used to calculate the correlation between the current question and the questions answered in the past. The historical correlation matrix is ​​obtained by multiplying the correlation between questions and the question-answering interaction sequence vector and then summing the results. HRP The multi-knowledge point answering rate matrix is ​​used to record the number of times each knowledge point is answered correctly. Each column in the multi-knowledge point answering rate matrix represents a knowledge point in the dataset. If the current time is... t At any given time, the predicted questions contain a set of knowledge points. C t For knowledge point sets C t Knowledge points c i Multi-knowledge point answering rate matrix The calculation formula is: wherein, represents the relationship between the answer results of the student c i for the knowledge point j before the time point t and the answer results at the time point t , c ij ==1 represents the relationship between the answer results of the student t for the knowledge point c at the time point i , and the number of times of correct answers of the student j before the time point t , j represents the answer sequence time point of the student j . S03. extracting a student learning state matrix from the matrix using a multi-layer one-dimensional convolution layer G a student learning state matrix; S04. Obtain the prediction result output of the relative difficulty of the required prediction question according to the learning state matrix and the question representation vector of the required prediction question. 2.The fusion multi-feature deep knowledge tracking method according to claim 1, characterized in that, The absolute difficulty matrix of the question is calculated according to the following formula: wherein, e i denotes the topic p i representation vector, W diff R K denotes the topic absolute difficulty weight matrix, i.e. the weight matrix of the fully connected layer; The similarity relationship between the questions is calculated according to the following formula: wherein represents a title p i , p j a similarity relationship between e j represents a title p j a representation vector. 3.The fusion multi-feature deep knowledge tracking method according to claim 2, characterized in that, In the step S01, a loss function is further defined for the pre-training process: wherein, R ij a similarity label of the question, i and j an absolute difficulty label of the question, d i a similarity label of the question, i an absolute difficulty label of the question, and are preset coefficients for adjusting the influence size of different question information on the question representation vector. 4.The fusion multi-feature deep knowledge tracking method according to claim 1, characterized in that, The calculation formula for constructing the historical correlation matrix is: wherein, w t i represents t the relevance between the topic and the question required for prediction at the time instant, i x i represents a representation vector of the question answering interaction sequence, i represents the time instant, t N represents the length of the question answering interaction sequence.​​​ 5.The fusion multi-feature deep knowledge tracking method according to any one of claims 1-4, characterized in that, In the step S02, the student answer interaction sequence matrix LIS , the history correlation matrix HRP and the multi-knowledge point answer rate matrix MCPC are spliced to obtain the comprehensive information matrix F ; The comprehensive information matrix F The nonlinear transformation and dimension reduction are performed through a gating mechanism GLU to obtain the G matrix, and the calculation formula is: wherein, , are weight matrices, respectively, , are bias parameters, respectively. 6.The method according to any one of claims 1-4, wherein, In the step S03, one sliding window is set to take d a continuous answering interaction as input, and each answering interaction is mapped to a single output element, and then a GLU is used to realize a gating mechanism on the output of the convolution layer to simulate the forgetting behavior of the student in the learning process. The residual network is added to the convolutional layer to connect the input and output of the convolutional layer, and finally L The same convolutional layers are stacked to form a hierarchical convolutional layer, wherein the lower layers are used to capture the learning rate later than the preset threshold, and the higher layers are used to monitor the learning rate earlier than the preset threshold.

7. The multi-faceted deep knowledge tracing method of any one of claims 1-4, wherein, In the step S04, the question representation vector et+1 of the required prediction question and the student learning state zt of the current moment of the student are dot multiplied, and then normalized by a softmax function to obtain the relative difficulty of the question to the current student, and the calculation formula is: 。 8. The multi-faceted deep knowledge tracking method according to any one of claims 1-4, wherein, In the step S03, a target function is further constructed as a binary cross-entropy loss function: wherein, a t denotes the actual answer of the student, o t denotes the predicted answer of the student.

9. A problem recommendation method characterized by comprising the steps of Comprising: When receiving a problem recommendation request, the historical answering information of the current student is obtained, and the prediction result of the relative difficulty of each problem is predicted by using the deep knowledge tracking method in any one of claims 1-8; According to the prediction result, the required relative difficulty of the question is selected for recommendation.

Images