Hierarchical contrastive graph knowledge tracing method and system based on multi-level feature learning

Abstract

The present application belongs to the technical field of knowledge tracking, and particularly relates to a hierarchical comparative graph knowledge tracking method and system based on multi-level feature learning, the tracking method comprising: obtaining an education question-answer knowledge point dataset; based on the education question-answer knowledge point dataset, combining a question-knowledge point adjacency matrix to obtain a question representation; adding disturbance to the question representation, and obtaining a target question representation according to the question representation and the question representation after adding disturbance; based on the education question-answer knowledge point dataset and the target question representation, obtaining knowledge embedding and interaction representation; based on the interaction representation and the knowledge embedding, obtaining a hidden state; inputting the hidden state and the knowledge embedding into a knowledge tracking prediction model to obtain a prediction value. The present application uses a hierarchical graph filtering attention mechanism to solve the problem of over-smoothing by gradually extracting multi-level feature representations of students.

Description

Technical Field

[0001] This invention belongs to the field of knowledge tracing technology, and particularly relates to a hierarchical comparison graph knowledge tracing method and system based on multi-level feature learning. Background Technology

[0002] Knowledge tracing (KT) is a fundamental task in educational data mining, aiming to estimate learners' knowledge states and predict their future performance by modeling historical interaction data. With billions of learning interactions generated daily on global education platforms, KT-based applications help identify learners with learning difficulties and support personalized learning path planning, thereby improving learning outcomes. With the rapid development of deep learning technology, the field of knowledge tracing has made groundbreaking progress. Deep sequence models, such as RNNs, can effectively capture the dynamic evolution of learners' knowledge states. Recently, attention-based knowledge tracing models, such as SAINT, AKT, and simpleKT, have gradually become mainstream solutions in this field by establishing direct associations within sequences. These models are well-suited for modeling long-distance dependencies in student interactions.

[0003] Limitations and shortcomings of existing technologies: Despite the significant success of attention-based knowledge tracking models, analysis reveals the following technical limitations:

[0004] 1) Inability to accurately distinguish knowledge mastery levels: Existing attention-based KT models are prone to "oversmoothing," resulting in a uniform distribution of attention weights and an inability to effectively distinguish learners' different levels of knowledge mastery. This severely limits the model's ability to provide accurate and personalized learning support.

[0005] 2) Difficulty in handling real-world environmental noise interference: Real-world educational scenarios contain a large amount of noisy data, such as abnormal interactions caused by learners' random answers, carelessness, and fatigue. Existing models lack effective noise identification and filtering mechanisms, resulting in a significant decrease in prediction accuracy in real-world environments, affecting the system's practicality.

[0006] 3) Failure to fully integrate semantic and structural information: Existing methods mainly rely on the ID features of questions and knowledge components (KCs), ignoring rich textual semantic information. Although there have been attempts to introduce semantic information, joint modeling with graph structures is still imperfect, failing to fully capture the complex relationships between knowledge points and reducing the accuracy of knowledge state representation. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention proposes a hierarchical comparative graph knowledge tracking method and system based on multi-level feature learning, which can solve the problem of multi-level semantic knowledge tracking in noisy environments.

[0008] To achieve the above object, the application provides a hierarchical contrastive graph knowledge tracing method based on multi-level feature learning, comprising:

[0009] Obtaining an education question-answer knowledge point dataset;

[0010] Based on the education question-answer knowledge point dataset, combining a question-knowledge point adjacency matrix, obtaining a question representation;

[0011] Adding perturbation to the question representation, and obtaining a target question representation according to the question representation and the question representation after adding perturbation;

[0012] Based on the education question-answer knowledge point dataset and the target question representation, obtaining a knowledge embedding and an interaction representation;

[0013] Based on the interaction representation and the knowledge embedding, obtaining a hidden state;

[0014] Inputting the hidden state and the knowledge embedding into a knowledge tracing prediction model to obtain a prediction value.

[0015] Optionally, based on the education question-answer knowledge point dataset, combining a question-knowledge point adjacency matrix, obtaining a question representation comprises:

[0016] For each question in the education question-answer knowledge point dataset, performing mean pooling operation on all knowledge point text embeddings corresponding to the question to obtain an aggregated representation of the question;

[0017] According to the aggregated representation, obtaining an aggregated question representation matrix;

[0018] Based on the aggregated question representation matrix and the question-knowledge point adjacency matrix, obtaining the question representation.

[0019] Optionally, according to the question representation and the question representation after adding perturbation, obtaining a target question representation comprises:

[0020] Performing contrastive learning on the question representation and the question representation after adding perturbation, and combining a contrastive learning objective function to obtain a target question representation.

[0021] Optionally, the contrastive learning objective function is:

[0022]

[0023] wherein, is a contrastive learning objective function, is a similarity measurement function, is a multi-layer perceptron for projecting the representation into a space for calculating the contrastive loss, is a question representation, The target question representation is obtained after contrastive learning is performed on the problem representation of the disturbance.

[0024] Optionally, based on the education question-answer knowledge point dataset and the target question representation, obtaining the knowledge embedding and the interaction representation comprises:

[0025] Based on the education question-answer knowledge point dataset, obtain a response embedding and a knowledge concept latent embedding;

[0026] Based on the response embedding and the knowledge concept latent embedding, obtain an interaction representation;

[0027] Based on the knowledge concept latent embedding and the target question representation, obtain a knowledge embedding.

[0028] Optionally, based on the interaction representation and the knowledge embedding, obtaining the hidden state comprises:

[0029] Projecting the interaction representation and the knowledge embedding to obtain a query vector, a key vector and a value vector;

[0030] Based on the query vector and the key vector, obtain a first-order attention score and a second-order attention score;

[0031] Based on the first-order attention score and the second-order attention score, obtain a high-order attention interaction value;

[0032] Based on the high-order attention interaction value, obtain an attention score;

[0033] According to the attention score and the value vector, obtain the hidden state.

[0034] Optionally, based on the high-order attention interaction value, obtain an attention score:

[0035]

[0036] wherein P is an attention score, is a learnable parameter that controls the contribution of different interaction orders, I is an identity matrix, A is an initial attention score, and A K is a high-order interaction value.

[0037] Optionally, inputting the hidden state and the knowledge embedding into a knowledge tracking prediction model to obtain a prediction value comprises:

[0038]

[0039] wherein σ and φ represent Sigmoid function and ReLU function respectively, is a trainable parameter, is the final prediction value of the model, a probability value ranging from 0 to 1, indicating the probability of the student getting this question right, is the hidden state, is the knowledge embedding.

[0040] The application also provides a hierarchical contrastive graph knowledge tracing system based on multi-level feature learning, comprising a fusion module, a contrastive learning module, a hierarchical feature distillation module and a knowledge tracing prediction module.

[0041] The fusion module is configured to obtain an education question-answer knowledge point dataset, obtain question representations based on the education question-answer knowledge point dataset and in combination with a question-knowledge point adjacency matrix.

[0042] The contrastive learning module is configured to add perturbations to the question representations, and obtain target question representations according to the question representations and the question representations after adding the perturbations.

[0043] The hierarchical feature distillation module is configured to obtain knowledge embeddings and interaction representations based on the education question-answer knowledge point dataset and the target question representations, and obtain hidden states based on the interaction representations and the knowledge embeddings.

[0044] The knowledge tracing prediction module is configured to input the hidden states and the knowledge embeddings into a knowledge tracing prediction model, and obtain prediction values.

[0045] Compared with the prior art, the application has the following advantages and technical effects:

[0046] 1. The hierarchical graph filtering attention mechanism effectively solves the problem of over-smoothing in existing knowledge tracing models. Traditional attention mechanisms tend to cause attention distribution to be uniform, making it difficult to distinguish between different levels of knowledge and learning ability. The hierarchical feature distillation module of the application can capture multi-level cognitive patterns from surface understanding to deep understanding, achieving fine-grained modeling of student knowledge states.

[0047] 2. The application innovatively combines adversarial perturbation and contrastive learning technology, simulates complex education data scenarios by injecting random perturbations during training, and effectively improves the robustness of the model to noisy data. Experiments show that compared with the baseline model, the application performs more stably when dealing with students' random answering, fatigue answering and other noisy behaviors, with a significantly reduced standard deviation.

[0048] 3. The application deeply integrates the semantic information of knowledge components and the structural relationship between questions and knowledge components. By using the BGE model to extract semantic embeddings of knowledge components and combining graph convolution network modeling structural relationships, the application achieves a comprehensive understanding of the relationship between questions and knowledge components, making up for the shortcomings of existing methods that rely only on ID features. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings, which are a part of this application, are included to provide a further understanding of the application, and are incorporated in and constitute a part of this application. The illustrations, together with the description, serve to explain the application, but do not limit the application. In the drawings:

[0050] Figure 1 is a hierarchical contrastive graph knowledge tracing method based on multi-level feature learning of an embodiment of the application. DETAILED DESCRIPTION

[0051] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0052] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0053] The present embodiment proposes a hierarchical contrastive graph knowledge tracing method based on multi-level feature learning, as shown in Figure 1 which specifically includes the following steps:

[0054] Obtain an education question-answer knowledge point dataset;

[0055] Based on the education question-answer knowledge point dataset, obtain a question representation in combination with a question-knowledge point adjacency matrix;

[0056] Add perturbation to the question representation, and obtain a target question representation according to the question representation and the question representation after adding perturbation;

[0057] Based on the education question-answer knowledge point dataset and the target question representation, obtain a knowledge embedding and an interaction representation;

[0058] Based on the interaction representation and the knowledge embedding, obtain a hidden state;

[0059] Input the hidden state and the knowledge embedding into a knowledge tracing prediction model to obtain a prediction value.

[0060] Specifically, the present embodiment uses a hierarchical graph filtering attention mechanism to solve the over-smoothing problem by gradually extracting multi-level feature representations of students. The contrastive learning is combined with the adversarial perturbation to process the noisy student interaction data, and the random perturbation is injected in the training process to simulate complex education data scenarios and extract robust features. A graph convolutional network (GCN) is used to jointly model the semantic information and structural relationship between the question and the knowledge concept (KC).

[0061] Further, based on the education question-answer knowledge point dataset, combined with the question-knowledge point adjacency matrix, the question representation includes:

[0062] For each question in the education question-answer knowledge point dataset, the mean pooling operation is performed on all knowledge point text embeddings corresponding to the question to obtain the aggregated representation of the question;

[0063] According to the aggregated representation, an aggregated question representation matrix is obtained;

[0064] Based on the aggregated question representation matrix and the question-knowledge point adjacency matrix, the question representation is obtained.

[0065] Specifically, given a text description of a knowledge concept , the semantic representation thereof is obtained by using a BGE model , wherein represents the semantic embedding of the knowledge concept, is a d-dimensional real vector space. In addition, a directed graph is constructed to model the relationship between questions and knowledge concepts. For a dataset with questions and knowledge concepts, a question-knowledge point adjacency matrix M is defined , wherein represents an Nq row Nc column real matrix space, if question i contains knowledge concept j, then is equal to 1, otherwise it is equal to 0. Then, for each question , the semantic embeddings of its related knowledge concepts are aggregated by mean pooling: , represents the semantic embedding obtained by aggregating the knowledge concepts related to question i, represents the set of knowledge concepts connected to question i, represents the size of the set of knowledge concepts connected to question i, represents the semantic embedding of knowledge concept j. Each question has an embedding r i , and finally R is the embedding of all questions, that is, all r i , the dimension is N q xd, wherein N q is the number of questions in the dataset, and then input into the GCN to further integrate the graph structure information together with the question-knowledge point adjacency matrix M:

[0066]

[0067] wherein is the matrix of aggregated question representations, and ​are learnable parameters, and ReLU is a rectified linear unit activation function. The final output denotes the enhanced question representation, which integrates semantic and structural information.

[0068] Further, the target question representation is obtained according to the question representation and the question representation with added perturbation, and the target question representation comprises:

[0069] The target question representation is obtained by contrastive learning on the question representation and the question representation with added perturbation, and the contrastive learning objective function is combined.

[0070] Specifically, in order to simulate the potential noise in the educational data and enhance the robustness of the model, the adversarial perturbation is generated from a uniform distribution , that is , where ε controls the amplitude of the perturbation. This perturbation method is simple and efficient, and it makes the smallest modification to the original data compared to other methods, while still creating meaningful changes in the input space while ensuring stability. Then, for the specific question embedding of R and its relationship matrix M, two complementary representations are designed: the original question representation z p and the target question representation z a obtained by contrastive learning on the question representation and the question representation with added perturbation. Robust features are captured by different GCN encoders, which are defined as follows:

[0071]

[0072] Then, the contrastive learning objective aims to maximize the consistency between the original representation and the perturbed representation of the same question:

[0073]

[0074] where, is the contrastive learning objective function, is a similarity measure function, is a multi-layer perceptron used to project the representation into a space where the contrastive loss is calculated, z p is the original question representation, z a is the target question representation obtained by contrastive learning on the question representation and the question representation with added perturbation.

[0075] Further, based on the educational question-answer knowledge point dataset and the target question representation, the knowledge embedding and the interaction representation are obtained, which comprises:

[0076] Based on the educational question-answer knowledge point dataset, the response embedding and the knowledge concept latent embedding are obtained.

[0077] Based on the response embedding and the knowledge concept latent embedding, the interaction representation is obtained.

[0078] Based on the latent embedding of knowledge concepts and the representation of the target problem, obtain a knowledge embedding.

[0079] Specifically, based on the interactive encoding scheme of simpleKT and its extension, the input layer is designed to effectively model the student learning trajectory. Instead of using a problem difficulty vector, the enhanced problem representation learned from the contrastive learning module is utilized.

[0080] Specifically, the interactive sequence encoding is as follows:

[0081]

[0082] wherein represents the latent embedding of knowledge concepts, represents the response embedding at the t-th time step. and are the original one-hot vectors corresponding to knowledge concepts and responses (correct / incorrect), respectively is an s-dimensional real number vector space, is a two-dimensional real number vector space. and are learnable transformation matrix embeddings combines knowledge concept information and problem representation obtained from the contrastive learning module wherein represents element-wise addition, is a real number matrix space of size d x s, is a real number matrix space of size d x 2. Finally, the final interactive representation combines knowledge concept embedding with response information, enabling the model to capture the complete interactive context.

[0083] Further, based on the interactive representation and the knowledge embedding, the hidden state is obtained, including:

[0084] Project the interactive representation and the knowledge embedding to obtain a query vector, a key vector, and a value vector;

[0085] Based on the query vector, the key vector, and the value vector, obtain a high-order interaction value;

[0086] Based on the high-order interaction value, obtain an attention score;

[0087] According to the attention score and the value vector, obtain the hidden state.

[0088] Specifically, first, given the problem representation x and the interactive representation y, project x to the query and key space, and project y to the value space, and calculate the initial attention score through multi-head attention. The first-order attention score A is calculated using scaled dot-product attention, i.e., A = where Q, K are the query vector and key vector obtained by linearly projecting the problem representation x, V is the value vector projected from the interaction representation y, T is the transpose, dk is the dimension of the key vector, and is a numerical value. Next, to capture high-order interaction patterns, a hierarchical graph filtering attention mechanism is proposed, defined as follows:

[0089]

[0090] where, to model the second-order relationship, to aggregate high-order interactions (K ≥ 2 is a hyperparameter), is a learnable parameter that controls the contribution of different interaction orders. The identity matrix I helps to maintain local information while learning global patterns.

[0091] Then, the next hidden state is calculated as: where V represents the value vector, FFN is a two-layer feedforward network with ReLU activation, and P is the final attention score, which includes the first-order and second-order attention scores.

[0092] Finally, this hierarchical feature processing enables the model to capture both fine-grained knowledge state transitions and long-term learning patterns.

[0093] Further, to make the final prediction, a two-layer fully connected network is used, taking the concatenation of the hidden state and the input embedding as input:

[0094]

[0095] where σ, represent the Sigmoid function and the ReLU function, respectively, is a trainable parameter, is the final prediction value of the model, a probability value ranging from 0 to 1, representing the probability of the student getting the question right, is the hidden state, is the knowledge embedding.

[0096] The embodiment also provides a hierarchical contrastive graph knowledge tracing system based on multi-level feature learning, including a fusion module, a contrastive learning module, a hierarchical feature distillation module, and a knowledge tracing prediction module.

[0097] The fusion module is used to obtain an educational question-answer knowledge point dataset, and based on the educational question-answer knowledge point dataset, a problem representation is obtained in combination with a question-knowledge point adjacency matrix.

[0098] The contrast learning module is configured to add a perturbation to the question representation, and obtain a target question representation according to the question representation and the question representation after the perturbation is added;

[0099] The hierarchical feature distillation module is configured to obtain a knowledge embedding and an interaction representation based on the educational question-answer knowledge point dataset and the target question representation, and obtain a hidden state based on the interaction representation and the knowledge embedding.

[0100] The knowledge tracing prediction module is configured to input the hidden state and the knowledge embedding into a knowledge tracing prediction model, and obtain a prediction value.

[0101] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A hierarchical contrast map knowledge tracing method based on multi-level feature learning, characterized in that, The method comprises the following steps: obtaining an education question-answer knowledge point data set; based on the education question-answer knowledge point data set, obtaining a question representation in combination with a question-knowledge point adjacency matrix; adding perturbation to the question representation, and obtaining a target question representation according to the question representation and the question representation after adding perturbation; obtaining a target question representation according to the question representation and the question representation after adding perturbation comprises: performing contrastive learning on the question representation and the question representation after adding perturbation, and obtaining the target question representation in combination with a contrastive learning objective function; the contrastive learning objective function is: wherein, is a contrastive learning objective function, is a similarity measure function, is a multi-layer perceptron for projecting the representation into a space where the contrastive loss is computed, is a problem representation, is a target problem representation after contrastive learning of the augmented perturbed problem representation; based on the education question-answer knowledge point data set and the target question representation, obtaining a knowledge embedding and an interaction representation; based on the interaction representation and the knowledge embedding, obtaining a hidden state; based on the interaction representation and the knowledge embedding, obtaining a hidden state comprises: projecting the interaction representation and the knowledge embedding to obtain a query vector, a key vector and a value vector; based on the query vector and the key vector, obtaining a first-order attention score and a second-order attention score; based on the first-order attention score and the second-order attention score, obtaining a high-order attention interaction value; based on the high-order attention interaction value, obtaining an attention score: where P is the attention score, is a learnable parameter that controls the contribution of different interaction orders, I is an identity matrix, A is the initial attention score, A K is the high-order attention score; obtaining the hidden state according to the attention score and the value vector; inputting the hidden state and the knowledge embedding into a knowledge tracking prediction model to obtain a prediction value. 2.The hierarchical contrast map knowledge tracing method based on multi-level feature learning according to claim 1, characterized in that, based on the education question-answer knowledge point data set, obtaining a question representation in combination with a question-knowledge point adjacency matrix comprises: for each question in the education question-answer knowledge point data set, performing mean pooling operation on all knowledge point text embeddings corresponding to the question to obtain an aggregated representation of the question; obtaining an aggregated question representation matrix according to the aggregated representation; based on the aggregated question representation matrix and the question-knowledge point adjacency matrix, obtaining the question representation. 3.The hierarchical contrastive graph knowledge tracing method based on multi-level feature learning according to claim 1, characterized in that, based on the education question-answer knowledge point data set and the target question representation, obtaining a knowledge embedding and an interaction representation comprises: based on the education question-answer knowledge point data set, obtaining a response embedding and a knowledge concept latent embedding; based on the response embedding and the knowledge concept latent embedding, obtaining an interaction representation; based on the knowledge concept latent embedding and the target question representation, obtaining a knowledge embedding. 4.The hierarchical contrast map knowledge tracing method based on multi-level feature learning according to claim 1, characterized in that, inputting the hidden state and the knowledge embedding into a knowledge tracking prediction model to obtain a prediction value comprises: where σ, φ represent Sigmoid function and ReLU function respectively, are trainable parameters, is the final prediction of the model, a probability value ranging from 0-1, indicating the probability of the student getting the question right, is the hidden state, is the knowledge embedding.

5. The hierarchical contrastive graph knowledge tracing system based on multi-level feature learning achieved by the method according to any one of claims 1-4, characterized in that, The method comprises the following steps: a fusion module, a contrastive learning module, a hierarchical feature distillation module and a knowledge tracking prediction module; the fusion module is used to obtain an education question-answer knowledge point data set, and obtain a question representation based on the education question-answer knowledge point data set in combination with a question-knowledge point adjacency matrix; the contrastive learning module is used to add perturbation to the question representation, and obtain a target question representation according to the question representation and the question representation after adding perturbation; the hierarchical feature distillation module is used to obtain a knowledge embedding and an interaction representation based on the education question-answer knowledge point data set and the target question representation, and obtain a hidden state based on the interaction representation and the knowledge embedding; The knowledge tracking prediction module is configured to input the hidden state and the knowledge embedding into a knowledge tracking prediction model to obtain a predicted value.

Images