Knowledge tracking method and system based on noise reduction attention and problem enhancement representation

Abstract

The invention belongs to the technical field of knowledge tracking, and particularly relates to a knowledge tracking method and system based on noise reduction attention and problem enhancement representation, and the method comprises the steps: obtaining original interaction data; on the basis of the original interaction data, obtaining a problem enhancement representation and an interaction enhancement representation, the problem enhancement representation including an enhancement representation of a current problem; obtaining a query vector, a key vector and a value vector based on the question enhancement representation and the interaction enhancement representation; performing weighted adjustment on the query vector, the key vector and the value vector by using a weight factor matrix to obtain knowledge state representation of the current time step; and inputting the enhanced representation and the knowledge state representation of the current question into a prediction model, and outputting a probability value that the student answers correctly on the current question, the prediction model being obtained by constructing a full-connection neural network. According to the method, the problem of attention noise of an attention-based knowledge tracking model can be solved.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge tracking technology, and in particular to a knowledge tracking method and system based on noise reduction attention and problem enhanced representation. Background Art

[0002] Knowledge tracking (KT) is a critical task in online education systems. Its core objective is to predict future student performance by analyzing past learning interactions. Currently, mainstream KT models fall into two categories: deep sequence models and attention-based models. Deep sequence models employ autoregressive structures such as LSTM and GRU to model the temporal dependencies of student interaction sequences through iterative updates of hidden states. Typical methods include DKT, DKT-F, and KQN. Attention-based models utilize the Transformer's multi-head attention mechanism to directly model global dependencies across historical interaction sequences. Representative methods include SAKT, SAINT, and AKT. These two approaches, through their distinct architectural designs, have jointly advanced the development of knowledge tracking technology.

[0003] Although the attention-based KT model has shown good results, it still faces the problem of "attention noise" in practical applications: the model will assign non-negligible attention weights to information that is essentially not cognitively relevant, thereby generating interference signals, causing information that is truly cognitively relevant to be ignored. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes a knowledge tracking method and system based on noise reduction attention and problem enhanced representation, which can solve the "attention noise" problem of the attention-based KT model.

[0005] To achieve the above objectives, the present invention provides a knowledge tracking method based on noise reduction attention and question enhancement representation, comprising:

[0006] Get raw interaction data;

[0007] Based on the original interaction data, obtaining a question enhancement representation and an interaction enhancement representation, wherein the question enhancement representation includes an enhanced representation of the current question;

[0008] Obtaining a query vector, a key vector, and a value vector based on the question enhanced representation and the interaction enhanced representation;

[0009] Using a weight factor matrix to perform weighted adjustment on the query vector, key vector, and value vector to obtain a knowledge state representation of the current time step;

[0010] The enhanced representation of the current question and the knowledge state representation are input into the prediction model, and the probability value of the student answering the current question correctly is output, wherein the prediction model is constructed by a fully connected neural network.

[0011] Optionally, obtaining the question enhanced representation and the interaction enhanced representation based on the original interaction data includes:

[0012] Obtaining question-enhanced latent representation based on the original interaction data;

[0013] Based on the question set in the original interaction data, obtaining a difficulty potential representation and an answer potential representation;

[0014] Acquire potential representations of knowledge points based on a set of knowledge points in the original interaction data;

[0015] Obtaining the problem enhancement representation based on the problem enhancement potential representation, the difficulty potential representation, and the knowledge point potential representation;

[0016] The interaction enhancement representation is obtained based on the question enhancement potential representation and the answer potential representation.

[0017] Optionally, obtaining the enhanced potential representation of the question based on the original interaction data includes:

[0018] Based on the original interaction data, construct a bipartite graph, and obtain an adjacency matrix and a question embedding matrix of the bipartite graph;

[0019] Based on the question embedding matrix and the adjacency matrix, combined with the linear transformation matrix and the bias vector, an updated question embedding matrix is ​​obtained;

[0020] Based on the updated question embedding matrix and the corresponding one-hot encoding vector, we can obtain the enhanced latent representation of the question.

[0021] Optionally, obtaining the potential representation of the answer based on the question set in the original interaction data includes:

[0022] Obtaining answer results based on the question set in the original interaction data;

[0023] The answer result is one-hot encoded to obtain the potential representation of the answer.

[0024] Optionally, a weight factor matrix is ​​used to perform weighted adjustment on the query vector, key vector, and value vector to obtain the knowledge state representation of the current time step:

[0025] ,

[0026] in, is the knowledge state representation of the current time step, BF is the weight factor, Q, K, and V are the query vector, key vector, and value vector respectively. is the square root of the embedding dimension and T is the transpose.

[0027] Optionally, obtaining the weight factor matrix includes:

[0028] Get similarity measures between interactions at different time steps;

[0029] Based on the similarity measure, obtaining a matrix element;

[0030] The weight factor matrix is ​​obtained according to the matrix elements.

[0031] Optionally, the method for obtaining the matrix elements is:

[0032] ,

[0033] Among them, i is the row of the BF matrix, j is the column of the BF matrix, for and Similarity measure between interactions, and are the questions corresponding to the i-th row and the j-th row respectively, is a decay coefficient between 0 and 1, which is an adjustable hyperparameter. Represents the weight factor matrix No. Rank Elements of the column, express and The value is 1 when there are the same knowledge points.

[0034] Optionally, the knowledge tracking method further includes: optimizing the prediction model using a binary cross entropy loss function.

[0035] Optionally, the binary cross entropy loss function is:

[0036] ,

[0037] in, is the predicted value, is the true value, is the binary cross entropy loss function.

[0038] The present invention also provides a knowledge tracking system based on noise reduction attention and problem enhanced representation, comprising: an interactive representation module, a noise reduction attention module and a prediction module;

[0039] The interaction representation module is configured to obtain original interaction data, and based on the original interaction data, obtain a question enhancement representation and an interaction enhancement representation, wherein the question enhancement representation includes an enhanced representation of the current question;

[0040] The denoising attention module is configured to obtain a question-enhanced representation and an interaction-enhanced representation based on the original interaction data, obtain a query vector, a key vector, and a value vector based on the question-enhanced representation and the interaction-enhanced representation, and perform weighted adjustment on the query vector, the key vector, and the value vector using a weight factor matrix to obtain a knowledge state representation of the current time step;

[0041] The prediction module is used to input the enhanced representation of the current question and the knowledge state representation into the prediction model, and output the probability value of the student answering the current question correctly, wherein the prediction model is constructed by a fully connected neural network.

[0042] Compared with the prior art, the present invention has the following advantages and technical effects:

[0043] Through the question-enhancing representation module, this paper can effectively capture the complex relationships between questions and accurately model the difficulty of each question, thereby achieving more detailed feature representation and significantly improving the model's prediction accuracy. Through the noise-reducing attention mechanism module, the model can effectively reduce the allocation of attention to irrelevant information, effectively solving the attention noise problem of attention-based knowledge tracking models and further improving prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0045] Figure 1 This is a flow chart of a knowledge tracking method based on noise reduction attention and question enhancement representation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] This embodiment proposes a knowledge tracking method based on noise reduction attention and question enhancement representation, such as Figure 1As shown, the specific steps include:

[0049] Get raw interaction data;

[0050] Based on the original interaction data, obtain the enhanced representation of the problem and the enhanced representation of the interaction, where the enhanced representation of the problem includes the enhanced representation of the current problem;

[0051] Based on the question-enhanced representation and the interaction-enhanced representation, the query vector, key vector, and value vector are obtained;

[0052] Use the weight factor matrix to perform weighted adjustment on the query vector, key vector, and value vector to obtain the knowledge state representation of the current time step;

[0053] The enhanced representation of the current question and the knowledge state representation are input into the prediction model, and the probability value of the student's correct answer to the current question is output. The prediction model is constructed by a fully connected neural network.

[0054] Specifically, this embodiment utilizes a question enhancement representation module to capture the complex correlation between questions and effectively model the difficulty of questions, so as to better represent the questions in a refined manner; a noise reduction attention mechanism is designed to reduce the attention allocation to irrelevant information by introducing a weight factor, thereby alleviating the attention noise problem from the root.

[0055] Furthermore, based on the original interaction data, obtaining question-enhanced representation and interaction-enhanced representation includes:

[0056] Based on the original interaction data, obtain the problem-enhanced latent representation;

[0057] Based on the question set in the original interaction data, obtain the difficulty latent representation and the answer latent representation;

[0058] Based on the knowledge point set in the original interaction data, the potential representation of the knowledge points is obtained;

[0059] Obtaining problem enhancement representation based on problem enhancement potential representation, difficulty potential representation and knowledge point potential representation;

[0060] Based on the question-enhanced latent representation and the answer-enhanced latent representation, an interaction-enhanced representation is obtained.

[0061] Specifically, the interaction representation module of this embodiment is designed to extract feature information from students' historical learning behaviors to construct a structured representation for subsequent cognitive modeling. More specifically, the historical interaction sequence of each student is set to ,in, represents the question answered by the student at time step t; Representation and Problems A collection of related knowledge points; Indicates that students have The answer results, including =1 means the answer is correct, =0 means the answer is wrong.

[0062] In order to model the complex relationship between problems and enhance the problem representation, this embodiment designs a problem-enhanced Rasch model. Specifically, first, the present invention constructs a problem set and knowledge point collection The bipartite graph formed , where the node set is the union of the question set and the knowledge point set, that is , edge set It is defined as the set of all question-knowledge point pairs with associated relationships, that is, Then, we introduce a graph convolutional network to perform feature propagation and fusion on the above bipartite graph structure to obtain enhanced problem representation. Specifically, based on the adjacency matrix of the graph , the first layer of graph convolution output of the question embedding matrix is ​​calculated as follows: First, the initial question embedding matrix With the adjacency matrix Multiply and then apply the linear transformation matrix and the bias vector , get the updated question embedding matrix Then, for any time step t, , by its one-hot encoding vector Multiply to extract its question-enhanced latent representation The formula is as follows:

[0063] ,

[0064] in, It is a bipartite graph The adjacency matrix of is the problem embedding matrix, represents the original one-hot vector, and is a learnable linear transformation operation.

[0065] In getting the problem Problem-enhancing latent representation Finally, we further introduce the problem difficulty information and the related knowledge point features to construct a more comprehensive problem semantic representation. Specifically, we use a linear transformation matrix The problem The one-hot vector of the latent representation of its difficulty At the same time, in response to the problem Related knowledge point collection , the unique heat vector of each knowledge point is transformed through another linear transformation matrix Mapping to knowledge point potential vector representation, and taking the average of these vectors to obtain the knowledge point potential representation corresponding to the question In addition, in order to model the actual answer behavior of students to the question, a representation method of the answer result is introduced. The answer result Perform one-hot encoding and pass the mapping matrix Convert to answer potential representation , thereby obtaining the student's behavior feedback vector at this time step. The specific formula is as follows:

[0066] ,

[0067] in, Indicates a problem The potential representation of the difficulty of and express The corresponding knowledge point potential representation and answer potential representation, represents the potential representation of the knowledge point in the jth column.

[0068] After integrating the above information, we get two feature representations for subsequent modeling. The first is the problem enhanced representation , which enhances the latent representation of the problem by Potential representation of its difficulty and knowledge point latent representation The second one is the interactive enhancement representation , by enhancing the latent representation of the problem and answer potential representation Calculated by element-by-element operation. The specific formula is as follows:

[0069] ,

[0070] Among them, x t Representation problem enhancement representation, y t Represents interactive representation.

[0071] Furthermore, based on the original interaction data, we obtain the enhanced potential representation of the question, including:

[0072] Based on the original interaction data, a bipartite graph is constructed and the adjacency matrix and question embedding matrix of the bipartite graph are obtained;

[0073] Based on the question embedding matrix and the adjacency matrix, combined with the linear transformation matrix and the bias vector, an updated question embedding matrix is ​​obtained;

[0074] Based on the updated question embedding matrix and the corresponding one-hot encoding vector, we can obtain the enhanced latent representation of the question.

[0075] Furthermore, based on the question set in the original interaction data, the potential representation of the answer is obtained, including:

[0076] Obtain answer results based on the question set in the original interaction data;

[0077] Perform one-hot encoding on the answer results to obtain the potential representation of the answer.

[0078] Furthermore, obtaining the weight factor matrix includes:

[0079] Get similarity measures between interactions at different time steps;

[0080] Based on the similarity measure, obtain the matrix elements;

[0081] Get the weight factor matrix based on the matrix elements.

[0082] Specifically, this embodiment also designs a noise reduction attention mechanism, which aims to dynamically adjust the attention score by introducing a weight factor, thereby suppressing the interference of redundant or irrelevant information to a certain extent, so that the model can more accurately focus on the historical fragments that truly contribute information to the current prediction task. Let the current time step be t+1, to obtain the knowledge state representation of the student at this moment First, we need to construct the query, key, and value vectors used in the attention mechanism. Enhanced representation by the problem at the current time step Through a set of learnable linear transformation functions Mapping obtains; key vector Enhanced representation of all questions from time step 1 to t Obtained by the same linear transformation; value vector The interaction representation of the corresponding time step is Obtained by mapping with the same transformation function. 、 and After that, the relevance of each historical interaction to the current query is calculated through the self-attention mechanism. The calculation of the attention score follows the classic scaled dot product attention form, that is, and Perform matrix multiplication on the transposed matrix of , and then divide by the square root of the embedding dimension To avoid numerical explosion, the softmax function is then applied to the result to obtain the normalized attention distribution. However, unlike the traditional attention mechanism, this embodiment introduces a weight factor matrix based on the above dot product attention. , weighted adjustment is performed on the original attention score by element-by-element multiplication, so that the model can effectively gather truly relevant information. The specific formula is as follows:

[0083] ,

[0084] Among them, x t+1 For an enhanced representation of the current problem, represents the student’s knowledge state at the current time step t+1.

[0085] The weight factor matrix Each element of It is used to express the correlation strength adjustment factor between historical time step i and time step j. Specifically, we first define is the similarity measure between the interactions of two time steps, which is calculated as the time step difference |ij| and the knowledge point overlap index The value of this index function is in two problems and If there is at least one common knowledge point, it is 1, indicating that the two have similar knowledge structures, otherwise it is 0. The value of :like If it is not equal to 0, that is, the two problems are similar and not in the same time step, then assign it a value ,in is a decay coefficient between 0 and 1, which is an adjustable hyperparameter; if is equal to 0, then the corresponding Taking 1 means keeping the original attention score unchanged. The specific formula is as follows:

[0086] ,

[0087] Among them, i is the row of the BF matrix, j is the column of the BF matrix, for and Similarity measure between interactions, and are the questions corresponding to the i-th row and the j-th row respectively, is a decay coefficient between 0 and 1, which is an adjustable hyperparameter. Represents the weight factor matrix No. Rank Elements of the column, express and The value is 1 when there are the same knowledge points.

[0088] Furthermore, the knowledge tracking method also includes: optimizing the prediction model using a binary cross entropy loss function.

[0089] Specifically, after using the denoising attention mechanism to model historical learning behaviors and obtain a representation of the current knowledge state, this embodiment employs a neural network module for predicting student responses. This module aims to predict the student's answer to the current question, that is, the probability of a correct answer, based on their knowledge state and the enhanced representation of the current question.

[0090] The core structure of the prediction module is a two-layer feedforward neural network. The input part consists of two main components: the knowledge state representation of the student at the current time step t+1 obtained by the denoising attention mechanism ; Second, the enhanced representation of the current problem . After the two vectors are concatenated, they are fed into the prediction module as the input of the neural network. The module first maps the input features to the intermediate dimension through a linear transformation layer, and applies the ReLU nonlinear activation function after this layer to enhance the model's expressive power and nonlinear modeling capabilities. Subsequently, the intermediate representation is again mapped to the scalar output space through a set of linear transformations, and the Sigmoid activation function is used to limit the prediction result to the range of 0 to 1, thereby outputting the predicted value of the student's correct answer to the current question. The specific formula is as follows:

[0091] ,

[0092] in, and Refers to the Sigmoid function and the ReLU function respectively. and Both are learnable parameter matrices in neural networks.

[0093] In addition, this embodiment minimizes and The binary cross entropy loss is used to optimize our prediction function. The specific loss formula is as follows:

[0094] ,

[0095] in, is the predicted value, is the true value, is the binary cross entropy loss function.

[0096] The core idea of ​​the above loss function is: when the model predicts the value Approaching the true value When the two deviate greatly, the loss increases, prompting the model to adjust parameters to reduce the error.

[0097] This embodiment also provides a knowledge tracking system based on noise reduction attention and question enhanced representation, including: an interactive representation module, a noise reduction attention module and a prediction module;

[0098] An interaction representation module is used to obtain original interaction data and, based on the original interaction data, obtain a question enhancement representation and an interaction enhancement representation, wherein the question enhancement representation includes an enhanced representation of the current question;

[0099] The denoising attention module is used to obtain question-enhanced representation and interaction-enhanced representation based on the original interaction data. Based on the question-enhanced representation and interaction-enhanced representation, the module obtains the query vector, key vector, and value vector. The module uses the weight factor matrix to perform weighted adjustment on the query vector, key vector, and value vector to obtain the knowledge state representation of the current time step.

[0100] The prediction module is used to input the enhanced representation of the current question and the knowledge state representation into the prediction model and output the probability value of the student answering the current question correctly, wherein the prediction model is constructed by a fully connected neural network.

[0101] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A knowledge tracking method based on noise reduction attention and question-enhanced representation, characterized by: include: Get raw interaction data; Based on the original interaction data, obtaining a question enhancement representation and an interaction enhancement representation, wherein the question enhancement representation includes an enhanced representation of the current question; Obtaining a query vector, a key vector, and a value vector based on the question enhanced representation and the interaction enhanced representation; Using a weight factor matrix to perform weighted adjustment on the query vector, key vector, and value vector to obtain a knowledge state representation of the current time step; The enhanced representation of the current question and the knowledge state representation are input into the prediction model, and the probability value of the student answering the current question correctly is output, wherein the prediction model is constructed by a fully connected neural network.

2. The knowledge tracking method based on noise reduction attention and question enhancement representation according to claim 1 is characterized in that: Based on the original interaction data, obtaining the question enhanced representation and the interaction enhanced representation includes: Obtaining question-enhanced latent representation based on the original interaction data; Obtaining a potential representation of difficulty based on a set of questions in the original interaction data; Obtaining a potential representation of an answer based on a set of answers in the original interaction data; Acquire potential representations of knowledge points based on a set of knowledge points in the original interaction data; Obtaining the problem enhancement representation based on the problem enhancement potential representation, the difficulty potential representation, and the knowledge point potential representation; The interaction enhancement representation is obtained based on the question enhancement potential representation and the answer potential representation.

3. The knowledge tracking method based on noise reduction attention and question enhancement representation according to claim 2 is characterized in that Based on the original interaction data, obtaining the enhanced potential representation of the question includes: Based on the original interaction data, construct a bipartite graph, and obtain an adjacency matrix and a question embedding matrix of the bipartite graph; Based on the question embedding matrix and the adjacency matrix, combined with the linear transformation matrix and the bias vector, an updated question embedding matrix is ​​obtained; Based on the updated question embedding matrix and the corresponding one-hot encoding vector, we can obtain the enhanced latent representation of the question.

4. The knowledge tracking method based on noise reduction attention and question enhancement representation according to claim 2 is characterized in that Based on the question set in the original interaction data, obtaining the potential representation of the answer includes: Obtaining answer results based on the question set in the original interaction data; The answer result is one-hot encoded to obtain the potential representation of the answer.

5. The knowledge tracking method based on noise reduction attention and question enhanced representation according to claim 1 is characterized in that: The method of weighting the query vector, key vector, and value vector using the weight factor matrix to obtain the knowledge state representation of the current time step is: ， in, is the knowledge state representation of the current time step, BF is the weight factor, Q, K, and V are the query vector, key vector, and value vector respectively. is the square root of the embedding dimension and T is the transpose.

6. The knowledge tracking method based on noise reduction attention and question enhancement representation according to claim 1 is characterized in that Obtaining the weight factor matrix includes: Get similarity measures between interactions at different time steps; Based on the similarity measure, obtaining a matrix element; The weight factor matrix is ​​obtained according to the matrix elements.

7. The knowledge tracking method based on noise reduction attention and question enhanced representation according to claim 6 is characterized in that: The method for obtaining the matrix elements is: ， Among them, i is the row of the BF matrix, j is the column of the BF matrix, for and Similarity measure between interactions, and are the questions corresponding to the i-th row and the j-th row respectively, is a decay coefficient between 0 and 1, which is an adjustable hyperparameter. Represents the weight factor matrix No. Rank Elements of the column, express and The value is 1 when there are the same knowledge points.

8. The knowledge tracking method based on noise reduction attention and question enhancement representation according to claim 1 is characterized in that: The knowledge tracking method further includes optimizing the prediction model using a binary cross entropy loss function.

9. The knowledge tracking method based on noise reduction attention and question enhanced representation according to claim 8, characterized in that: The binary cross entropy loss function is: ， in, is the predicted value, is the true value, is the binary cross entropy loss function.

10. Knowledge tracking system based on noise reduction attention and question enhanced representation, characterized by: include: Interaction representation module, noise reduction attention module and prediction module; The interaction representation module is configured to obtain original interaction data, and based on the original interaction data, obtain a question enhancement representation and an interaction enhancement representation, wherein the question enhancement representation includes an enhanced representation of the current question; The denoising attention module is configured to obtain a question-enhanced representation and an interaction-enhanced representation based on the original interaction data, obtain a query vector, a key vector, and a value vector based on the question-enhanced representation and the interaction-enhanced representation, and perform weighted adjustment on the query vector, the key vector, and the value vector using a weight factor matrix to obtain a knowledge state representation of the current time step; The prediction module is used to input the enhanced representation of the current question and the knowledge state representation into the prediction model, and output the probability value of the student answering the current question correctly, wherein the prediction model is constructed by a fully connected neural network.

Images