A Temporal Knowledge Tracing Method and Device Based on Attention Enhancement

Abstract

The present invention provides a temporal knowledge tracking method and device based on attention enhancement, which relates to the field of computer technology, and includes: fusing a problem embedding vector and a concept embedding vector by means of connection and dimensionality reduction to obtain an exercise representation, and combining the exercise representation, the response result corresponding to the real-time problem, and the position encoding to obtain an interaction representation; using a multi-layer perceptron (MLP) to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, obtaining a global interaction feature, correcting the learner interaction information by using the answering response time and the answering interval time, and outputting the obtained target interaction information to a preset prediction model to determine the learner's knowledge state at the next moment. By integrating the global relationship modeling of the attention mechanism and the local time dynamic characteristics of the temporal model, the present invention captures complex learning behavior patterns and improves the tracking accuracy of the model for the student knowledge state.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a temporal knowledge tracking method and device based on attention enhancement. Background Art

[0002] The development of artificial intelligence technology helps to carry out personalized and precise teaching models. Knowledge tracking can model the trajectory and results of learners' answering questions, analyze the changes in learners' knowledge states during the learning process, and predict learners' performance in answering questions at future moments. It models based on the student behavior sequence and predicts the degree of students' mastery of knowledge. Deep knowledge tracking is to apply deep learning technology to the knowledge tracking task, and capture the potential rules and patterns in students' answering behaviors by training a deep neural network model, so as to achieve accurate prediction of students' knowledge mastery.

[0003] In recent years, there have been many studies on deep knowledge tracking. In these studies, many have used self-attention mechanisms or considered temporal issues. However, these attention-based knowledge tracking models often have insufficient capture of temporal dependencies and cannot effectively capture long-term dependencies in time series when dealing with temporal data. And the temporal-based knowledge tracking models often encounter difficulties when facing complex data patterns because they usually rely on fixed feature selection and linear models and cannot effectively handle non-linear dynamic changes in the data. Existing studies only focus on the coarse-grained response results, which cannot effectively explore the fine-grained changes in learners' knowledge states during the answering process. Summary of the Invention

[0004] The purpose of the present invention is to provide a temporal knowledge tracking method and device based on attention enhancement to solve the problem of incomplete tracking in the current knowledge tracking field.

[0005] The technical solution of the embodiment of the present application is realized as follows:

[0006] The first aspect of the embodiment of the present application provides a temporal knowledge tracking method based on attention enhancement, including:

[0007] Obtain the question embedding vector and concept embedding vector of historical exercise data; the question embedding vector represents the real-time question answered by the learner, and the concept embedding vector represents the concept corresponding to the real-time question;

[0008] Fuse the question embedding vector and the concept embedding vector by means of connection and dimensionality reduction to obtain an exercise representation, and combine the exercise representation, the response result corresponding to the real-time question, and the position encoding to obtain an interaction representation;

[0009] Replace the dot product operation with the Hadamard product, and use a multi-layer perceptron (MLP) to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, so as to obtain the global interaction features; the global interaction features include the learner interaction information at each moment.

[0010] Use the answering response time and the answering interval time to correct the learner interaction information, obtain the target interaction information, and output the target interaction information to a preset prediction model to determine the learner's knowledge state at the next moment.

[0011] Optionally, the exercise representation is: , and the interaction representation is ; where is the question embedding vector, is the concept embedding vector, is the response result, is the positional encoding.

[0012] Optionally, the step of replacing the dot product operation with the Hadamard product and using a multi-layer perceptron (MLP) to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, so as to obtain the global interaction features, includes:

[0013] Concatenate the exercise representation and the interaction representation to obtain the exercise representation matrix and the interaction representation matrix , and the calculation logic is as follows:

[0014] ;

[0015] where M is the number of interaction records and d is the hidden layer dimension.

[0016] Optionally, the step of replacing the dot product operation with the Hadamard product and using a multi-layer perceptron (MLP) to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, so as to obtain the global interaction features, includes:

[0017] Use the exercise representation as the query item, the interaction representation as the key and value, and use a causal mask for constraint to obtain the global interaction features; where the transformation principle of the query item, the key, and the value constrained by the causal mask is:

[0018] ;

[0019] where are the projection matrices of the query item, the key, and the value respectively, and is the corresponding bias term.

[0020] Optionally, replacing the dot product operation with the Hadamard product and using a multi-layer perceptron (MLP) to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, obtaining the global interaction features, includes:

[0021] Using a multi-layer perceptron (MLP) to learn the correlation between the query term and the key:

[0022] ; where is the Hadamard product operation, is the feature correlation;

[0023] Controlling the numerical scaling of the correlation through the Sigmoid function:

[0024]

[0025] The formula for the Sigmoid function transformation is: ;

[0026] Calculating the attention weights using the softmax function:

[0027] ; where is the weight matrix.

[0028] Optionally, correcting the learner interaction information using the answering response time and the answering interval time to obtain the target interaction information, includes: inputting the learner interaction information into a forgetting gate to simulate the forgetting behavior of the learner's knowledge mastery:

[0029] ;

[0030] where is the non-linear activation function sigmoid, and are weight matrices, and are bias terms, is the knowledge state at the previous moment, represents the concatenation operation, is the interaction embedding at time t, is the concept similarity, is the parameter for adjusting the influence of the concept similarity.

[0031] Optionally, correcting the learner interaction information using the answering response time and the answering interval time to obtain the target interaction information, includes: The learner's knowledge acquisition process is as follows:

[0032] ;

[0033] where tanh is a non - linear activation function, and are weight matrices, and are bias terms, , representing the latent knowledge obtained by the learner at moment;

[0034] The knowledge acquisition ability of the learner is:

[0035] ;

[0036] where is the non - linear activation function sigmoid, and are weight matrices, and are bias terms, representing the regulation of the degree of knowledge acquisition;

[0037] Combining the knowledge acquisition process, the knowledge acquisition ability and the forgetting gate mechanism to determine the knowledge state at the next moment:

[0038] .

[0039] The second aspect of the embodiments of the present application provides a temporal knowledge tracking device based on attention enhancement, including: a vector acquisition module, a vector fusion module, a weight adjustment module and a state determination module, where,

[0040] The vector acquisition module is configured to acquire a question embedding vector and a concept embedding vector of historical exercise data; the question embedding vector represents the real - time question answered by the learner, and the concept embedding vector represents the concept corresponding to the real - time question;

[0041] The vector fusion module is configured to fuse the question embedding vector and the concept embedding vector by means of connection and dimensionality reduction to obtain an exercise representation, and combine the exercise representation, the response result corresponding to the real - time question and the position encoding to obtain an interaction representation;

[0042] The weight adjustment module is configured to replace the dot - product operation with a Hadamard product and use a multi - layer perceptron MLP to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, so as to obtain a global interaction feature; the global interaction feature includes the learner interaction information at each moment;

[0043] The state determination module is configured to correct the learner interaction information by using the answering response time and the answering interval time to obtain target interaction information, and output the target interaction information to a preset prediction model to determine the knowledge state of the learner at the next moment.

[0044] A third aspect of the embodiments of the present application provides an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the method for attention-enhanced temporal knowledge tracking described in the first aspect.

[0045] A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0046] Compared with the prior art, the beneficial effects brought by the technical solution provided by the present application are:

[0047] The present invention provides a method and device for attention-enhanced temporal knowledge tracking. By fusing the question embedding vector and the concept embedding vector in a connection and dimensionality reduction manner, an exercise representation is obtained, and an interaction representation is obtained by combining the exercise representation, the response result corresponding to the real-time question, and the position encoding; a multi-layer perceptron (MLP) is used to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, so as to obtain a global interaction feature. The learner interaction information is corrected by using the answering response time and the answering interval time, and the obtained target interaction information is output to a preset prediction model to determine the knowledge state of the learner at the next moment. By integrating the global relationship modeling of the attention mechanism and the local time dynamic characteristics of the temporal model, complex learning behavior patterns are captured, and the tracking accuracy of the model for the student's knowledge state is improved. Description of the Drawings

[0048] Figure 1 It is a schematic flowchart of a method for attention-enhanced temporal knowledge tracking provided by an embodiment of the present application;

[0049] Figure 2 It is a schematic structural diagram of a device for attention-enhanced temporal knowledge tracking provided by an embodiment of the present application;

[0050] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0051] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present application. In the following detailed description, for the sake of explanation, numerous specific details are set forth in order to provide a comprehensive understanding of the embodiments of the present application. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.

[0052] The terms used herein are merely for describing specific embodiments and are not intended to limit the present application. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0053] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0054] Some block diagrams and / or flowcharts are shown in the accompanying drawings. It should be understood that some blocks or combinations of blocks in the block diagrams and / or flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when executed by the processor, these instructions can create a device for implementing the functions / operations illustrated in these block diagrams and / or flowcharts.

[0055] In some embodiments, please refer to Figure 1 , Figure 1 which is a schematic flowchart of a temporal knowledge tracking method based on attention enhancement provided by an embodiment of the present application; the temporal knowledge tracking method based on attention enhancement provided by an embodiment of the present application includes:

[0056] S110, obtaining a question embedding vector and a concept embedding vector of historical exercise data; the question embedding vector represents the real-time question answered by the learner, and the concept embedding vector represents the concept corresponding to the real-time question.

[0057] In this embodiment. Let represent the set of learners, where N represents the total number of learners. The answer response sequence of each learner is denoted as , where , They respectively represent the question answered by the learner at the t-th moment, the concept corresponding to the question, the response to the question (1 indicates correct, 0 indicates incorrect), the answering time of this question, and the time interval from the last answer when answering this question. The probability that the learner correctly answers a new question is After obtaining the historical exercise data, preprocess the historical exercise data and modify the learner's answering response sequence to , where s is the sequence length we set and t <= s. If the actual answering sequence length is less than s, padding content is added to expand the sequence until the length is s. Conversely, if the actual answering sequence length is greater than s, the sequence is truncated. At this time, we will obtain the question sequence answered by the learner, the concept sequence , the response time sequence , the answering time sequence , and the interval time sequence .

[0058] S120, fuse the question embedding vector and the concept embedding vector by means of connection and dimensionality reduction to obtain the exercise representation, and combine the exercise representation, the response result corresponding to the real-time question, and the position encoding to obtain the interaction representation.

[0059] Randomly initialize the embedding vector of the question , and let it learn automatically during the training process. All questions can be transformed into an embedding matrix , where N is the number of exercises and d is the dimension of the hidden layer. Similarly, denote the embedding vector of the knowledge concept as . For the answer to a question, according to whether the answer is correct, it can be expanded into a vector of all zeros or all ones . Since directly connecting the question embedding and the concept may trigger high-dimensional problems, the interaction term is obtained by connecting the linear layer through concat for dimensionality reduction to obtain the exercise representation of, so as to maximize the retention of effective information. The knowledge state of the student develops gradually and should show a smooth transition over time, rather than a sudden change. To adapt to this natural development process, position encoding is introduced. The position encoding is learned during the training process, combined with all the previous interaction situations of the student, and then introduced into the interaction representation through splicing.

[0060] S130, replace the dot product operation with the Hadamard product, and use the multi-layer perceptron MLP to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, and obtain the global interaction feature; the global interaction feature includes the learner interaction information at each moment.

[0061] In some embodiments, the exercise representation is: , and the interaction representation is ; where is the problem embedding vector, is the concept embedding vector, is the response result, is the positional encoding.

[0062] In some embodiments, in S130, the dot product operation is replaced with the Hadamard product, and a multi-layer perceptron (MLP) is used to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, obtaining the global interaction features, including:

[0063] The exercise representation and the interaction representation are concatenated to obtain the exercise representation matrix and the interaction representation matrix , and the calculation logic is as follows:

[0064] ;

[0065] where M is the number of interaction records and d is the hidden layer dimension.

[0066] Here, the number of learners is N, the number of interaction records obtained after processing is M, ERs is the exercise representation of a certain interaction record, and IRs is the interaction representation of a certain interaction record. The exercise representations and interaction representations of all interactions are concatenated together to form the exercise representation matrix and the interaction representation matrix.

[0067] In some embodiments, in S130, the dot product operation is replaced with the Hadamard product, and a multi-layer perceptron (MLP) is used to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, obtaining the global interaction features, including:

[0068] The exercise representation is used as the query term, the interaction representation is used as the key and value, and a causal mask is used for constraint to obtain the global interaction features; where the transformation principle of the query term, key, and value constrained by the causal mask is:

[0069] ;

[0070] where are the projection matrices of the query term, key, and value respectively, is the corresponding bias term.

[0071] In the knowledge tracing task, through the parallel calculation of multiple attention heads, richer representations can be learned, which helps to more accurately capture the learning dynamics of learners at different tasks and time points. Therefore, the attention mechanism here is extended to multi-head parallel calculation, and eight attention heads are used in the model:

[0072] ;

[0073] 。

[0074] In some embodiments, in S130, the dot product operation is replaced with a Hadamard product, and a multi-layer perceptron (MLP) is used to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, obtaining global interaction features, including:

[0075] Using a multi-layer perceptron (MLP) to learn the correlation between the query term and the key:

[0076] ; where is the Hadamard product operation, is the feature correlation;

[0077] Controlling the numerical scaling of the correlation through a Sigmoid function:

[0078]

[0079] The formula for the Sigmoid function transformation is: ;

[0080] Calculating the attention weights using a softmax function:

[0081] ; where is the weight matrix.

[0082] In knowledge tracing, the interaction between the learner and the question may involve complex interactions of multiple features. There is a large amount of information in the learner's interaction representation. By multiplying the interaction representation by the interaction representation after Sigmoid, the model is allowed to learn which interaction information should be focused on in different situations and to what extent, and at the same time, introducing non-linear changes in this way helps to improve the generalization ability of the model. The specific transformation formula is as follows: . Subsequently, the softmax function is applied to highlight important information and calculate the attention weights. It should be noted that the attention output can be obtained by performing batch matrix multiplication with, selectively aggregating and extracting information.

[0083] In S140, the learner interaction information is corrected using the answering response time and the answering interval time to obtain target interaction information, and the target interaction information is output to a preset prediction model to determine the learner's knowledge state at the next moment.

[0084] In some embodiments, in S140, the learner interaction information is corrected using the answering response time and the answering interval time to obtain the target interaction information, including: inputting the learner interaction information into a forgetting gate to simulate the forgetting behavior of the learner's knowledge mastery:

[0085] ;

[0086] wherein, is the non-linear activation function sigmoid, and are weight matrices, and are bias terms, is the knowledge state at the previous moment, represents the concatenation operation, is the interaction embedding at time t, is the concept similarity, is a parameter for adjusting the influence of the concept similarity.

[0087] In some embodiments, in S140, the learner interaction information is corrected using the answering response time and the answering interval time to obtain the target interaction information, including: the learner's knowledge acquisition process is as follows:

[0088] ;

[0089] wherein, tanh is the non-linear activation function, and are weight matrices, and are bias terms, , representing the potential knowledge acquired by the learner at time;

[0090] The learner's knowledge acquisition ability is:

[0091] ;

[0092] wherein, is the non-linear activation function sigmoid, and are weight matrices, and are bias terms, represents the regulation of the knowledge acquisition degree;

[0093] Combining the knowledge acquisition process, the knowledge acquisition ability and the forgetting gate mechanism to determine the knowledge state at the next moment:

[0094] .

[0095] For the prediction part, the response to the exercise at the (t + 1)-th moment is predicted based on the learner's knowledge state at the t-th moment. The prediction is carried out by concatenating the knowledge state with the exercise embedding vector at the (t + 1)-th moment, then passing it through a fully-connected layer, and finally activating it with a function:

[0096]

[0097] where, is the weight matrix, , and the output result represents the predicted score for the next exercise , and its value range is (0, 1). The preset prediction model is trained using the binary cross-entropy loss function to minimize the objective function between the predicted value and the true value:

[0098] .

[0099] In an optional embodiment, in S140, the learner interaction information is corrected using the answer response time and the answer interval time to obtain the target interaction information, including correcting the learner interaction information using the semantic-aware temporal module. The semantic-aware temporal module receives the weighted sum from the attention mechanism module based on interaction enhancement, and these weighted sums represent the refined global interaction features, which contain the student interaction information at each moment. The knowledge state of the learner at this moment is obtained through processing by the semantic-aware temporal module , and then is used to predict the current exercise , and the prediction result is obtained. This module consists of two gating mechanisms, namely the forgetting gate and the input gate. In the forgetting gate, the answer interval and the concept similarity between two adjacent answers are considered, and in the input gate, the student's answer time is considered to control the input.

[0100] Specifically, the response time can reflect the student's knowledge mastery, and the interval time affects the learner's knowledge forgetting process. Therefore, according to the characteristics of the time data, first, the former is discretized into seconds on the time scale, and the latter is discretized into minutes. Then, these two time features are represented by the embedding matrices and , where M is the number of interaction records and d is the hidden layer dimension. Subsequently, the two time features corresponding to the learning interaction at time t can be represented as and .

[0101] The problems corresponding to each interaction are usually different, and the updates of the corresponding knowledge states are also different. Therefore, a Q matrix is maintained. The Q matrix is an exercise-knowledge concept correlation matrix, and the elements of this matrix are binary codes. Among them, if the exercise ei contains the knowledge concept kj, then Qij = 1, otherwise it is 0. Whether a learner can correctly answer a certain question is determined by the learner's knowledge state on this question, that is, the higher the learner's knowledge state level on this question, the more likely the learner is to answer this question correctly.

[0102] To match the learner's knowledge state for the exercise at time t we first multiply by the knowledge-related vector related to the current question , which is equivalent to a row in the Q matrix, representing the exercise and its corresponding knowledge concept. The learner's knowledge state at time t - 1 is The corrected knowledge state is:

[0103]

[0104] where represents the Hadamard product operation.

[0105] In the forget gate, consider the time interval between two answers and the concept similarity of these two answer records. The higher the concept similarity between two answers, the more likely it is to enhance the learner's mastery level on this concept, and forgetting should be reduced. This concept similarity is obtained by generating concept texts. Specifically, through the word2vec method, the word embeddings of concept texts are obtained. When this concept consists of multiple words, the word embeddings are calculated for each word separately and then the average value is taken. The measurement of similarity uses cosine similarity.

[0106] Specifically, assume that there are two concept texts C1 and C2, which are composed of word sets and respectively. We calculate the average value of all word embeddings. Let and be the word embedding vectors of words and respectively, then the average word embedding vectors of concept texts C1 and C2 are:

[0107] ;

[0108] Finally, we use the cosine similarity formula to calculate the similarity between two concept texts. The formula for cosine similarity is:

[0109]

[0110] Here, the word2vec model is selected to more effectively process the semantic information of word groups. For the measurement of similarity, Euclidean distance, Jaccard similarity, and cosine similarity are compared. Cosine similarity is selected as the best measurement method because of its effectiveness in evaluating semantic similarity and its insensitivity to changes in text length.

[0111] The present invention fuses the problem embedding vector and the concept embedding vector by using connection and dimensionality reduction methods to obtain an exercise representation, and combines the exercise representation, the response result corresponding to the real-time problem, and the position encoding to obtain an interaction representation; uses a multi-layer perceptron (MLP) to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, obtains a global interaction feature, corrects the learner's interaction information by using the answering response time and the answering interval time, and outputs the obtained target interaction information to a preset prediction model to determine the learner's knowledge state at the next moment. By integrating the global relationship modeling of the attention mechanism and the local time dynamic characteristics of the time series model, complex learning behavior patterns are captured, and the tracking accuracy of the model for the student's knowledge state is improved.

[0112] In an optional embodiment, the following three real-world datasets widely used in knowledge tracking research are used. The basic statistics of these datasets are summarized in Table 1. Both ASSIST12 and ASSIST17 are from the ASSISTments intelligent tutoring system, and these datasets record the answering situations of learners in the primary school mathematics curriculum. The ASSIST12 dataset was collected from September 2012 to October 2013 and is the largest version in the ASSISTments dataset. The ASSIST17 dataset was released in 2017 and covers the answering records of the 2004 - 2005 and 2005 - 2006 school years. Junyi is an interactive dataset in the education field, which was collected from an education platform from November 2010 to March 2015, and this dataset details the answering behaviors of students on the intelligent tutoring system.

[0113] Table 1 Statistical Information of Three Datasets

[0114]

[0115] To ensure the reusability of the experiments, all experiments were implemented on a Linux system using a 4090D graphics card and the PyTorch framework. The random seed was set to 42 to ensure the repeatability of the results. For the dataset division, referring to the standard practice of the PYKT library, the training set and the test set were divided in a ratio of 8:2, and 5-fold cross-validation was adopted on the training set for further division into the training set and the validation set. During the model training process, a learning rate of 0.005, a dropout rate of 0.3, a batch size of 64 were used, and the Adam optimizer was adopted, with the hidden layer dimension set to 64. The total number of training epochs was 200, but if the AUC of the validation set did not improve within 10 epochs, the training would be stopped early to avoid overfitting.

[0116] All baseline models and IETKT use the same dataset, experimental settings, and evaluation metrics. In addition, the shared parameters of each model are also kept consistent to ensure the fairness and reliability of the comparison. DKT uses an LSTM layer to encode the knowledge state of students to simulate students' mastery of knowledge concepts to predict students' future response performance. DKVMN introduces two external memory modules to store knowledge concepts and dynamically update the knowledge mastery state of learners. SAKT uses a self-attention mechanism to capture the relationship between exercises and students' responses, strengthening the ability to model the long-term dependence between students' learning interactions. SAINT uses a Transformer-based encoder-decoder architecture to split the interaction sequence of students into a question embedding sequence and an answer embedding sequence, which are respectively sent as inputs to the encoder and the decoder to generate a prediction sequence. AKT uses a monotonic attention mechanism to characterize the time distance between questions and students' past interactions, and reduces the attention weight at an exponentially decaying rate according to the time distance in the interaction sequence. IEKT estimates students' responses to cognitive questions by introducing an individualized estimation mechanism and updates the knowledge state before evaluating students' knowledge acquisition sensitivity. simpleKT extracts time-aware information in students' learning interactions through a common dot product attention function to predict students' future performance. LPKT-S is an extended model of LPKT. By introducing student embeddings, it clearly distinguishes different learning progress rates of students, and considers individual differences of students in the learning gain module, forgetting module, and prediction module to better evaluate students' knowledge states and predict performance.

[0117] Table 2 Performance Comparison

[0118]

[0119] QIKT is a problem-centered interpretable knowledge tracing model that learns problem-centered representations through a knowledge acquisition module and a problem-solving module. stableKT designs a multi-head aggregation module that combines dot-product attention and hyperbolic attention to capture the complex relationships between problems and knowledge components. TCKT solves the problems of data bias and learning process simulation through a causal self-attention mechanism and a learning behavior simulation module, respectively, improving the consistency of students' knowledge state representations.

[0120] In the research of knowledge tracing, common evaluation metrics include the area under the ROC curve (Area Under Curve, AUC) and prediction accuracy (Accuracy, ACC) in classification tasks. The larger the AUC and ACC, the better the model's prediction performance. When considering the knowledge tracing task as a regression problem, the root mean square error (Root Mean Square Error, RMSE) is usually used to measure the performance of the model. The lower the value, the better the model's performance. Here, AUC, ACC, and RMSE are used as evaluation metrics for IETKT to comprehensively measure the performance of the model.

[0121] According to the experimental results in Table 2, the IETKT model outperforms other baseline models on all three datasets, demonstrating its excellent performance in the knowledge tracing task. Specifically, on the ASSIST12 dataset, the AUC of IETKT reached 0.7790, significantly higher than 0.7755 of the second-best model. At the same time, the ACC and RMSE metrics reached 0.7578 and 0.4065 respectively, both being the best performances. On the ASSIST17 dataset, the AUC of IETKT was 0.7989, also higher than 0.7941 of the second-best model. In addition, its ACC and RMSE metrics were 0.7387 and 0.4166 respectively, further verifying the stability and prediction ability of the model. On the Junyi dataset, IETKT achieved the second-best result with an AUC of 0.8016, slightly lower than 0.8018 of AKT, but the ACC and RMSE metrics were 0.8558 and 0.3309 respectively, showing a leading advantage.

[0122] Here, a set of ablation experiments was also designed. By gradually removing the key modules in the IETKT model and evaluating its performance. The following variants were used to conduct ablation experiments on the IETKT model (w / o means removing a certain module):

[0123] ·IETKT w / o ATT: Remove the interaction-enhanced attention module.

[0124] ·IETKT w / o CR: Do not consider the concept relevance in the knowledge forgetting part.

[0125] ·IETKT w / o IT: The time interval is not considered for the part of knowledge forgetting.

[0126] ·IETKT w / o AT: The answering time is not considered for the part of knowledge gain.

[0127] Table 3 Comparison results of IETKT and its ablation variants in terms of the AUC metric.

[0128]

[0129] AUC is one of the most commonly used evaluation metrics in knowledge tracing models, which is used to measure the ability of the model to distinguish between correct and incorrect student answers. The experimental results show that the complete IETKT model has better AUC on all datasets than each ablation variant, verifying the important contribution of each module to the overall performance of the model.

[0130] Furthermore, by analyzing the data in Table 3, it can be found that removing the AT module (w / o AT) leads to the most significant performance decline on the three datasets. Especially on the ASSIST17 and Junyi datasets, the AUC drops by 0.0221 and 0.0111 respectively, indicating the significant impact of the answering time on the model performance. In contrast, removing the IT module (w / o IT) results in the second-largest performance decline, indicating the importance of the time interval in simulating the knowledge forgetting process. In addition, removing the CR module (w / o CR) and the ATT module (w / o ATT) has relatively less impact on the performance, but still indicates that the relevance of concept texts and the interactive enhanced attention mechanism play an auxiliary role in the overall performance of the model. These results fully verify the necessity of each module of the IETKT model and its synergistic effect in the knowledge tracing task.

[0131] In some embodiments, please refer to Figure 2 , Figure 2 which is a schematic structural diagram of an attention-enhanced temporal knowledge tracing device provided by an embodiment of the present application; An embodiment of the present application provides an attention-enhanced temporal knowledge tracing device 200, including: a vector acquisition module 210, a vector fusion module 220, a weight adjustment module 230, and a state determination module 240, where,

[0132] The vector acquisition module 210 is configured to acquire a question embedding vector and a concept embedding vector of historical exercise data; the question embedding vector represents the real-time question answered by the learner, and the concept embedding vector represents the concept corresponding to the real-time question;

[0133] The vector fusion module 220 is configured to fuse the question embedding vector and the concept embedding vector by means of concatenation and dimensionality reduction to obtain an exercise representation, and combine the exercise representation, the response result corresponding to the real-time question, and the position encoding to obtain an interaction representation;

[0134] The weight adjustment module 230 is configured to replace the dot product operation with a Hadamard product and use a multi-layer perceptron (MLP) to learn the correlation between the problem representation and the interaction representation to adjust the attention weights corresponding to the problem representation and the interaction representation to obtain a global interaction feature; the global interaction feature includes the learner interaction information at each moment;

[0135] The state determination module 240 is configured to modify the learner interaction information using the answer response time and the answer interval time to obtain target interaction information, and output the target interaction information to the preset prediction model to determine the learner's knowledge state at the next moment.

[0136] In some embodiments, the problem representation is: , the interaction is represented as ;in, Embedding vector for the question, is the concept embedding vector, To respond to the results, Encode the position.

[0137] In some embodiments, the weight adjustment module 230 is specifically configured to:

[0138] The exercise representation and the interaction representation are concatenated to obtain the exercise representation matrix and interaction representation matrix , the calculation logic is as follows:

[0139] ;

[0140] Where M is the number of interaction records and d is the hidden layer dimension.

[0141] In some embodiments, the weight adjustment module 230 is specifically configured to:

[0142] The problem representation is used as the query item, the interaction representation is used as the key and value, and the causal mask is used for constraint to obtain the global interaction feature. The transformation principle of the query item, key, and value constrained by the causal mask is as follows:

[0143] ;

[0144] in, are the projection matrices of query items, keys, and values, respectively. is the corresponding bias term.

[0145] In some embodiments, the weight adjustment module 230 is specifically configured to:

[0146] Use Multilayer Perceptron (MLP) to learn the correlation between query terms and keys:

[0147] ; where, is the Hadamard product operation, is the feature correlation;

[0148] Control the numerical scaling of the correlation through the Sigmoid function:

[0149]

[0150] The formula for the Sigmoid function transformation is: ;

[0151] Calculate the attention weights using the softmax function:

[0152] ; where, is the weight matrix.

[0153] In some embodiments, S140, correct the learner interaction information using the answering response time and the answering interval time to obtain the target interaction information, including: input the learner interaction information into the forgetting gate to simulate the forgetting behavior of the learner's knowledge mastery:

[0154] ;

[0155] where, is the non - linear activation function sigmoid, and are the weight matrices, and are the bias terms, is the knowledge state at the previous moment, represents the concatenation operation, [[ID=5x1]]is the interaction embedding at time t, is the concept similarity, is the parameter for adjusting the influence of the concept similarity, .

[0156] In some embodiments, the state determination module 240 is configured as follows: The learner's knowledge acquisition process is as follows:

[0157] ;

[0158] where, tanh is the non - linear activation function, and are the weight matrices, and are the bias terms, , represents the potential knowledge acquired by the learner at time;

[0159] It should be noted that there may be a small error in the translation of "[[ID=5x1]]" as there seems to be a potential typo in the original text. If this is an important part, it may need to be confirmed with the original source for a more accurate translation.The knowledge acquisition ability of the learner is as follows:

[0160] ;

[0161] Among them, is the non-linear activation function sigmoid, and are weight matrices, and are bias terms, represents the regulation of the degree of knowledge acquisition;

[0162] Determine the knowledge state at the next moment by combining the knowledge acquisition process, knowledge acquisition ability and forgetting gate mechanism:

[0163] .

[0164] The temporal knowledge tracking device based on attention enhancement provided by the embodiments of the present application can implement each process in the corresponding embodiments of the above-mentioned temporal knowledge tracking method based on attention enhancement. To avoid repetition, it will not be elaborated here.

[0165] It should be noted that the temporal knowledge tracking device based on attention enhancement provided by the embodiments of the present application and the temporal knowledge tracking method based on attention enhancement provided by the embodiments of the present application are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the above-mentioned temporal knowledge tracking method based on attention enhancement, and the repeated parts will not be elaborated.

[0166] In some embodiments, please refer to Figure 3 , Figure 3 which is a schematic structural diagram of an electronic device provided by the embodiments of the present application. An electronic device 300 provided by the embodiments of the present application includes a processor 310 and a memory 320; the memory 320 stores a computer program, and when the computer program is executed by the processor, the above-mentioned temporal knowledge tracking method based on attention enhancement is implemented.

[0167] Specifically, the processor 310 may include, for example, a general microprocessor, an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application specific integrated circuit (ASIC)), and so on. The processor 310 may also include on-board memory for caching purposes. The processor 310 may be a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiments of the present application.

[0168] The memory 320 can be, for example, any medium capable of containing, storing, transmitting, propagating, or transporting instructions. For example, the memory 320 can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, components, or propagation media. Specific examples of the memory 320 include: magnetic storage devices, such as magnetic tapes or hard disk drives (HDDs); optical storage devices, such as compact discs (CD-ROMs); it can also be, for example, random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0169] The present application also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for attention-enhanced temporal knowledge tracking. The computer-readable medium can be included in the device / apparatus / system described in the above embodiments; or it can exist separately and not be assembled into the device / apparatus / system. The above computer-readable medium carries one or more programs, which, when the one or more programs are executed, implement the method according to the embodiments of the present application.

[0170] According to an embodiment of the present application, the computer-readable medium can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. And in the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transport a program for use by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination of the above.

[0171] Those skilled in the art will understand that the features recited in the various embodiments and / or claims of the present application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly recited in the present application. In particular, without departing from the spirit and teachings of the present application, the features recited in the various embodiments and / or claims of the present application can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present application. Therefore, the scope of the present application should not be limited to the above embodiments, but should be determined not only by the appended claims, but also by equivalents of the appended claims.

Claims

1. A temporal knowledge tracking method based on attention enhancement, characterized in that including: obtaining a problem embedding vector and a concept embedding vector of historical exercise data; the problem embedding vector represents a real-time problem answered by a learner, and the concept embedding vector represents a concept corresponding to the real-time problem; fusing the problem embedding vector and the concept embedding vector by means of connection and dimensionality reduction to obtain an exercise representation, and combining the exercise representation, a response result corresponding to the real-time problem, and a positional encoding to obtain an interaction representation; replacing a dot product operation with a Hadamard product, and using a multi-layer perceptron MLP to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, so as to obtain a global interaction feature; the global interaction feature includes the learner interaction information at each moment; correcting the learner interaction information by using the answer response time and the answer interval time to obtain target interaction information, and outputting the target interaction information to a preset prediction model to determine the learner's knowledge state at the next moment.

2. The method for temporal knowledge tracking based on attention enhancement according to claim 1, wherein The exercise is characterized as: , and the interaction is characterized as ; where is the problem embedding vector, is the concept embedding vector, is the response result, is the positional encoding.

3. The attention-enhanced temporal knowledge tracking method according to claim 1, wherein The replacing the dot product operation with a Hadamard product, and using a multi-layer perceptron MLP to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, so as to obtain a global interaction feature, includes: Concatenate the exercise representation and the interaction representation to obtain an exercise representation matrix and an interaction representation matrix , and the calculation logic is as follows: ； where M is the number of interaction records and d is the hidden layer dimension.

4. The method for tracking temporal knowledge based on attention enhancement according to claim 3, wherein The replacing the dot product operation with a Hadamard product, and using a multi-layer perceptron MLP to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, so as to obtain a global interaction feature, includes: using the exercise representation as a query term, the interaction representation as keys and values, and constraining with a causal mask to obtain a global interaction feature; wherein, the transformation principle of the query term and the keys and values constrained by the causal mask is: ； Among them, are the projection matrices of the query item, key, and value respectively, is the corresponding bias term.

5. The method for tracking temporal knowledge based on attention enhancement according to claim 4, wherein The replacing the dot product operation with a Hadamard product, and using a multi-layer perceptron MLP to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, so as to obtain a global interaction feature, includes: using a multi-layer perceptron MLP to learn the correlation between the query term and the keys; ; wherein, is a Hadamard product operation, is the feature correlation; controlling the numerical scaling of the correlation through a Sigmoid function; The formula for the sigmoid function transformation is: ; calculating the attention weights by using a softmax function; ; wherein, is the weight matrix.

6. The method for tracking temporal knowledge based on attention enhancement according to claim 1, wherein The correcting the learner interaction information by using the answer response time and the answer interval time to obtain target interaction information includes: inputting the learner interaction information into a forgetting gate to simulate the forgetting behavior of the learner's knowledge mastery: ； Among them, is the non-linear activation function sigmoid, and are weight matrices, and are bias terms, is the knowledge state at the previous moment, represents the concatenation operation, is the interaction embedding at time t, is the concept similarity, is a parameter that adjusts the influence of the concept similarity.

7. The method for tracking temporal knowledge based on attention enhancement according to claim 6, wherein The correcting the learner interaction information by using the answer response time and the answer interval time to obtain target interaction information includes: the learner's knowledge acquisition process is as follows: ； where tanh is a non-linear activation function, and are weight matrices, and are bias terms, , representing the latent knowledge obtained by the learner at time; The learner's knowledge acquisition ability is: ； Among them, is the non-linear activation function sigmoid, and is the weight matrix, and is the bias term, represents the regulation of the degree of knowledge acquisition; combining the knowledge acquisition process, the knowledge acquisition ability, and the forgetting gate mechanism to determine the knowledge state at the next moment: 。 8. A temporal knowledge tracking device based on attention enhancement, characterized in that including: a vector acquisition module, a vector fusion module, a weight adjustment module, and a state determination module, wherein, The vector acquisition module is configured to acquire a question embedding vector and a concept embedding vector of historical exercise data; the question embedding vector represents a real-time question answered by a learner, and the concept embedding vector represents a concept corresponding to the real-time question; The vector fusion module is configured to fuse the question embedding vector and the concept embedding vector by means of concatenation and dimensionality reduction to obtain an exercise representation, and combine the exercise representation, the response result corresponding to the real-time question, and the positional encoding to obtain an interaction representation; The weight adjustment module is configured to replace the dot product operation with a Hadamard product, and use a multi-layer perceptron (MLP) to learn the correlation between the exercise representation and the interaction representation to adjust the attention weights corresponding to the exercise representation and the interaction representation, so as to obtain a global interaction feature; the global interaction feature includes the learner interaction information at each moment; The state determination module is configured to correct the learner interaction information by using the answering response time and the answering interval time to obtain target interaction information, and output the target interaction information to a preset prediction model to determine the knowledge state of the learner at the next moment.

9. An electronic device, comprising a processor and a memory; the memory stores a computer program, wherein, The computer program, when executed by the processor, implements the attention-enhanced temporal knowledge tracking method according to any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images