Hypergraph attention knowledge tracking method, model and system and readable storage medium

Abstract

The invention relates to the technical field of knowledge tracking, in particular to a hypergraph attention knowledge tracking method, model and system and a readable storage medium. The method comprises the following steps: taking a to-be-predicted task as an input feature in the form of a hypergraph star-shaped expansion graph, and enhancing the input feature by using a hypergraph incidence matrix; constructing a full connection graph among nodes of the hypergraph star expansion graph by using an attention mechanism, carrying out sparsification on the full connection graph by adopting a Gumbel-Softmax structure, and screening out key connections after sparsification; performing regularization constraint on the key connection by adopting hypergraph structure regularization; and predicting a potential knowledge state result in the to-be-predicted task. The objective of the invention is to solve the problem of how to improve the prediction performance of a hypergraph attention knowledge tracking model.

Description

Technical Field

[0001] The present application relates to the field of knowledge tracking technology, and in particular to a hypergraph attention knowledge tracking method, model, system and readable storage medium. Background Art

[0002] Knowledge tracking is a core task in the field of intelligent education. It aims to model students' knowledge status through their learning behavior and dynamically predict their mastery of specific knowledge points, thereby inferring students' responses to new problems in future learning processes.

[0003] The existing knowledge tracking model based on Hypergraph Neural Networks (HGNN) uses hypergraph structures to model multiple relationships such as students and problems, problems and concepts, etc., significantly improving the ability to represent students' knowledge status.

[0004] In the process of conceiving and implementing this application, the inventors discovered that the existing hypergraph-based knowledge tracking models mostly focus on learning local relationship information from the prior hypergraph structure, and lack an effective mechanism to mine the global semantic structure contained in the problem-concept hypergraph; and current research on combining hypergraphs with Transformers mainly focuses on updating the node features within a single hyperedge through the domain attention mechanism, or on modeling the temporal dynamics in student interaction sequences, failing to fully mine the global structural information contained in the hypergraph, which to a certain extent limits the model's ability to express high-order relationships and the depth of semantic modeling.

[0005] In view of this, this application proposes a hypergraph attention knowledge tracking method that can capture global structural information to improve the prediction performance of the hypergraph attention knowledge tracking model. Summary of the Invention

[0006] The main purpose of this application is to provide a hypergraph attention knowledge tracking method, aiming to solve the problem of how to improve the prediction performance of the hypergraph attention knowledge tracking model.

[0007] To achieve the above objectives, this application provides a hypergraph attention knowledge tracking method, which includes the following steps:

[0008] S10, taking the task to be predicted as input features in the form of a hypergraph star-shaped expansion graph, and using the hypergraph association matrix to enhance the input features;

[0009] S20, using an attention mechanism to construct a fully connected graph between the nodes of the star-shaped expansion graph of the hypergraph, using a Gumbel-Softmax structure to sparsify the fully connected graph, and screening out key connections after the sparsification;

[0010] S30, using hypergraph structure regularization to perform regularization constraints on the key connections;

[0011] S40: predicting the potential knowledge state result in the task to be predicted.

[0012] Optionally, in S10, using the hypergraph association matrix to enhance input features specifically includes:

[0013] S11, let the expression of input feature X be:

[0014]

[0015] Where, , Represents the characteristics of the node, Represents the hyperedge e j=1，2，3...m The feature of the hyperedge is obtained by averaging the features of the nodes within the hyperedge;

[0016] S12, introducing sparse linear transformation layers into the star-shaped expansion graph and its dual graph respectively to extract their structural information and map them into a low-dimensional embedding space:

[0017]

[0018] Where, is a learnable weight matrix, is the incidence matrix, is the transposed matrix of the incidence matrix H;

[0019] S13, embedding the prior hypergraph structure through learnable linear projection to obtain enhanced input features ;

[0020]

[0021] Where, , .

[0022] Optionally, step S20 specifically includes:

[0023] S21, the attention weight of the enhanced input feature is calculated using the following formula :

[0024]

[0025] Where, are two different learnable weight matrices for the attention layer, represents the feature embedding of node i, represents the feature embedding of node j, represents the feature embedding of node k;

[0026] S22, after calculating the attention weights between any two inputs, the representation of the i-th node will be updated based on the weighted combination of all relevant inputs, thereby integrating its global dependencies with other nodes to obtain a new node representation vector :

[0027]

[0028] Where, is another learnable weight matrix of the attention layer, n represents the number of question nodes, and m represents the number of concept nodes;

[0029] S23, Gumbel-Softmax is introduced to extract sparse structure subgraphs from the fully connected graph and obtain the sparse key connections:

[0030]

[0031] Where, is the key connection after sparsification, is the activation function, is the temperature variable, is the attention weight of node i and node j in the fully connected graph, There are two independent Gumbel noise terms, adding random perturbations to the two categories respectively. Refers to the Goebel distribution.

[0032] Optionally, the step S30 specifically includes:

[0033] S31, introduces the hypergraph star-shaped expansion graph as a structural prior to supervise and guide the attention matrix. The hypergraph star-shaped expansion graph structure expression is:

[0034]

[0035] Where, is the incidence matrix of the hypergraph star expansion graph, is the transposed matrix of the incidence matrix H, The dimension is The all-zero vector, The dimension is An all-zero vector of ;

[0036] S32, determining a probability transfer matrix according to the hypergraph star-shaped extended graph structure:

[0037]

[0038] Where, is a diagonal matrix, represents the probability of transferring the i-th instance to the j-th instance, n represents the number of question nodes, and m represents the number of concept nodes;

[0039] S33, according to the probability transfer matrix and the sparse key connection The cross entropy of the hypergraph is used to calculate the regularization loss function of the star-shaped expansion graph of the hypergraph :

[0040]

[0041] Where L is the number of attention layers, d i is the degree of instance i of the hypergraph star-shaped extended graph structure;

[0042] S34: Perform regularization constraints on the key connections based on the regularization loss.

[0043] Optionally, the S40 specifically includes:

[0044] S41, inputting the task to be predicted into a recurrent neural network including a gated recurrent unit:

[0045]

[0046] Where X is the input feature, a represents the student's answer sequence, [,] represents the connection operation, and h t-1 represents the student’s knowledge state at time t-1, and GRU represents a recurrent neural network containing a gated recurrent unit;

[0047] S42, the characteristics of a problem qi and the student’s knowledge state h at time t-1 t-1 Connect to get the student's knowledge status of qi :

[0048]

[0049] S43, will Mapped to the probability that the student will answer the question qi correctly in the next step t , as the underlying knowledge state results:

[0050]

[0051] in, ;

[0052] Where, is the unnormalized log probability, and b o are weight and bias parameters.

[0053] Optionally, before executing S43, the method further includes:

[0054] S44, determine the minimum predicted probability y i and the true response r t The cross entropy loss function between :

[0055]

[0056] Where, is the Dirac function;

[0057] S45, according to the cross entropy loss function and the regularization loss function , determine the total loss function L:

[0058]

[0059] Where, is the loss weight coefficient;

[0060] S46, updating the trainable parameters with the goal of minimizing the total loss function L.

[0061] In addition, to achieve the above objectives, the present application also provides a hypergraph attention knowledge tracking model, which includes:

[0062] A structural embedding module based on the question-concept association matrix is ​​used to take the task to be predicted as input features in the form of a hypergraph star-shaped expansion graph and enhance the input features using the hypergraph association matrix;

[0063] Based on Gumbel-Softmax structure learning and hypergraph structure regularization module, it is used to use the attention mechanism to construct a fully connected graph between the nodes of the star-shaped expansion graph of the hypergraph, and to use the Gumbel-Softmax structure to sparsify the fully connected graph, and to screen out the key connections after sparsification, and to use hypergraph structure regularization to regularize the key connections;

[0064] The prediction module is used to predict the potential knowledge state results in the task to be predicted.

[0065] Optionally, the embedding dimension of each layer output in the hypergraph attention knowledge tracking model is between 128-512, and the controller uses a logarithmic grid with a step size of 10 to perform hyperparameter search in the range of 0.1 to 1.

[0066] In addition, to achieve the above-mentioned purpose, the present application also provides a computer system, which includes: a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the computer program is executed by the processor, the steps of the hypergraph attention knowledge tracking method as described in any one of the above items are implemented.

[0067] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the hypergraph attention knowledge tracking method as described in any one of the above items are implemented.

[0068] This application has at least the following beneficial effects:

[0069] 1. Use the hypergraph association matrix to enhance input features, so that the model can learn local relationship information between questions and concepts;

[0070] 2. Use the attention mechanism to build a fully connected graph between the nodes of the star-shaped expansion graph, thereby capturing the global high-order relationships between nodes;

[0071] 3. Use Gumbel-Softmax to sparsify the fully connected graph and dynamically screen out key connections. Further, use hypergraph structure regularization to regularize the sparse structure. While learning information outside the hypergraph structure, it does not deviate from the original hypergraph structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a flow chart of a first embodiment of a hypergraph attention knowledge tracking method according to an embodiment of the present application;

[0073] Figure 2 Constructing a hypergraph relationship diagram for the relationship between the issues and concepts involved in the embodiments of the present application;

[0074] Figure 3 Schematic diagram of the hypergraph star expansion graph and global message delivery mechanism involved in the embodiment of the present application;

[0075] Figure 4 Schematic diagram of the architecture of the hypergraph attention knowledge tracking model involved in the embodiments of the present application;

[0076] Figure 5 A comparative line chart showing the impact of different embedding dimensions on model performance involved in the embodiments of this application;

[0077] Figure 6 A line chart comparing the impact of different lamda values ​​on model performance involved in the embodiments of this application;

[0078] Figure 7This is a schematic diagram of the architecture of the hardware operating environment of the computer system involved in the embodiments of the present application.

[0079] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0080] To better understand the above technical solutions, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0081] First embodiment

[0082] Reference Figure 1 In this embodiment, a hypergraph attention knowledge tracking method is provided, and the method includes the following steps:

[0083] S10, taking the task to be predicted as input features in the form of a hypergraph star-shaped expansion graph, and using the hypergraph association matrix to enhance the input features;

[0084] In this step, the prediction task is first formally defined, with a hypergraph star-shaped expansion graph as the input feature, and the hypergraph association matrix is ​​used to enhance the input feature.

[0085] Hypergraph star expansion refers to converting a hypergraph into its corresponding star expansion graph.

[0086] In some optional implementations, how to convert a hypergraph into its corresponding star-shaped extended graph is exemplified below:

[0087] Some key symbols and definitions used in this embodiment are listed in Table 1:

[0088] Table 1. Key symbols and definitions

[0089]

[0090] Definition 1 (Student Performance Prediction) Given each student’s answer record from time step 1 to t-1, our goal is to predict the probability pt that the student will correctly answer the next question at time step t.

[0091] Definition 2 (Hypergraph) In this implementation, a hypergraph is represented as a triple ,in is a node set, , ={e1, e2, ..., em} is a hyperedge set, . is an incidence matrix, where Indicates that hyperedge j contains node i, otherwise .

[0092] Reference Figure 2 The diagram shows a hypergraph relationship constructed from the relationship between questions and concepts. In the hypergraph, each concept is represented as a hyperedge, and the questions contained in the concept are represented as nodes in the hyperedge. Therefore, the concept hyperedge c1 contains the question nodes q2 and q3. The question node q1 belongs to both hyperedge c2 and hyperedge c3. A hyperedge is a subset of the question nodes.

[0093] Definition 3 (Hypergraph Star Extension Graph) Given a hypergraph , whose star-shaped expansion graph is a bipartite graph ,in , the vertex set of the graph includes all the vertices in the original graph , and each hyperedge is considered a new vertex. , for each hyperedge vertex and vertex in the original graph, an edge is added to the expanded graph. Each hyperedge becomes a "center point" in the expanded graph, which forms a star structure with all the vertices it connects.

[0094] Definition 4 (Hypergraph Dual Graph) Given a hypergraph , its dual graph is another hypergraph constructed by exchanging the roles of vertices and hyperedges in the original hypergraph, denoted as .

[0095] Reference Figure 3 Figure 2 shows a hypergraph star-shaped expansion diagram and a schematic diagram of the global message passing mechanism. The upper part shows how to convert a hypergraph into its corresponding star-shaped expansion diagram. In this expansion diagram, hyperedges are modeled as central nodes connecting their associated nodes, forming a star structure. The solid lines in the lower part of the figure reflect the local message passing mechanism of the hypergraph neural network, where information follows the node → hyperedge → node transmission path, which propagates slowly and is prone to oversmoothing problems, without considering the interaction between hyperedges. The fusion of the dashed and solid lines reflects the global message passing mechanism of our model, which allows information to be transmitted between all nodes and hyperedges, effectively modeling global dependencies and deep semantic connections between concepts, between problems, and between problems and concepts.

[0096] In some optional implementations, the specific steps of using the hypergraph association matrix to enhance input features are as follows:

[0097] S11, let the expression of input feature X be:

[0098]

[0099] Where, , Represents the characteristics of the node, Represents the hyperedge e j=1，2，3...m The feature of the hyperedge is obtained by averaging the features of the nodes within the hyperedge;

[0100] S12, introducing sparse linear transformation layers into the star-shaped expansion graph and its dual graph respectively to extract their structural information and map them into a low-dimensional embedding space:

[0101]

[0102] Where, is a learnable weight matrix, is the incidence matrix, is the transposed matrix of the incidence matrix H;

[0103] S13, embedding the prior hypergraph structure through learnable linear projection to obtain enhanced input features ;

[0104]

[0105] Where, , .

[0106] It should be noted that through structural embedding, our model can effectively utilize the prior structural information of the hypergraph, thereby enhancing the ability to perceive the relative position of nodes in the overall structure.

[0107] S20, using an attention mechanism to construct a fully connected graph between the nodes of the star-shaped expansion graph of the hypergraph, using a Gumbel-Softmax structure to sparsify the fully connected graph, and screening out key connections after the sparsification;

[0108] In this step, the attention mechanism and Gumbel-Softmax are introduced to deeply explore the high-order correlations between questions and concepts in the hypergraph and capture global semantic information.

[0109] In some optional implementations, step S20 specifically includes:

[0110] S21, the attention weight of the enhanced input feature is calculated using the following formula :

[0111]

[0112] Where, are two different learnable weight matrices for the attention layer, represents the feature embedding of node i, represents the feature embedding of node j, represents the feature embedding of node k;

[0113] In this step, this formula is used to build a dense, fully connected graph from any node / hyperedge to any hyperedge / node, enabling efficient information exchange between questions and concepts. This design fully considers the need for joint modeling between questions and concepts, significantly enhancing global structural modeling capabilities.

[0114] In previous hypergraph-based KT models, the interaction between questions usually relies on indirect connections through the "question-concept-question" path. This message passing mechanism based on local structure limits the efficiency of information dissemination and easily leads to over-smoothing problems. In contrast, the GS-HAKT model introduces a full connection mechanism between questions and concepts, allowing direct pairwise interactions between all instances, effectively improving the breadth and depth of information dissemination. In addition, existing methods generally ignore the potential relationships between concepts and only focus on the one-way information flow from questions to concepts. To this end, we further introduce an attention mechanism between hyperedges to capture the semantic associations between concepts, thereby optimizing the representation of concepts, and through it, enhance the representation ability of nodes, significantly improving the overall representation learning effect.

[0115] S22, after calculating the attention weights between any two inputs, the representation of the i-th node will be updated based on the weighted combination of all relevant inputs, thereby integrating its global dependencies with other nodes to obtain a new node representation vector :

[0116]

[0117] Where, is another learnable weight matrix of the attention layer, n represents the number of question nodes, and m represents the number of concept nodes;

[0118] Furthermore, the weight calculation formula in step S21 suffers from oversmoothing. This stems from the fact that message passing in this stage is performed on a fully connected graph, but in reality, only some edges are critical for semantic modeling. In other words, redundant connections not only dilute useful information but may also introduce noise, affecting the accuracy of node representation. To this end, we introduce Gumbel-Softmax into the model to automatically extract sparse structured subgraphs from the fully connected graph. This preserves key interactions, reduces redundant connections, further alleviates the oversmoothing problem, and maintains efficient gradient propagation to optimize the model's learning process as shown below:

[0119] S23, Gumbel-Softmax is introduced to extract sparse structure subgraphs from the fully connected graph and obtain the sparse key connections:

[0120]

[0121] Where, is the key connection after sparsification, is the activation function, is the temperature variable, is the attention weight of node i and node j in the fully connected graph, There are two independent Gumbel noise terms, adding random perturbations to the two categories respectively. Refers to the Goebel distribution.

[0122] S30, using hypergraph structure regularization to perform regularization constraints on the key connections;

[0123] In this embodiment, we introduce hypergraph structure regularization to integrate local structural information. The key idea is to use the prior relationship between questions and concepts to guide the training of the attention matrix.

[0124] In some optional implementations, step S30 specifically includes:

[0125] S31, introduces the hypergraph star-shaped expansion graph as a structural prior to supervise and guide the attention matrix. The hypergraph star-shaped expansion graph structure expression is:

[0126]

[0127] Where, is the incidence matrix of the hypergraph star expansion graph, is the transposed matrix of the incidence matrix H, The dimension is The all-zero vector, The dimension is An all-zero vector of ;

[0128] S32, determining a probability transfer matrix according to the hypergraph star-shaped extended graph structure:

[0129]

[0130] Where, is a diagonal matrix, represents the probability of transferring the i-th instance to the j-th instance, n represents the number of question nodes, and m represents the number of concept nodes;

[0131] S33, according to the probability transfer matrix and the sparse key connection The cross entropy of the hypergraph is used to calculate the regularization loss function of the star-shaped expansion graph of the hypergraph :

[0132]

[0133] Where L is the number of attention layers, d i is the degree of instance i of the hypergraph star-shaped extended graph structure;

[0134] S34: Perform regularization constraints on the key connections based on the regularization loss.

[0135] S40: predicting the potential knowledge state result in the task to be predicted.

[0136] In this step, the task to be predicted is input into the deep learning network to capture its potential knowledge state that changes over time.

[0137] In some optional implementations, the S40 specifically includes:

[0138] S41, inputting the task to be predicted into a recurrent neural network including a gated recurrent unit:

[0139]

[0140] Where X is the input feature, a represents the student's answer sequence, [,] represents the connection operation, and h t-1 represents the student’s knowledge state at time t-1, and GRU represents a recurrent neural network containing a gated recurrent unit;

[0141] S42, the characteristics of a problem qi and the student’s knowledge state h at time t-1 t-1 Connect to get the student's knowledge status of qi :

[0142]

[0143] S43, will Mapped to the probability that the student will answer the question qi correctly in the next step t , as the underlying knowledge state results:

[0144]

[0145] in, ;

[0146] Where, is the unnormalized log probability, and b o are weight and bias parameters.

[0147] In the technical solution provided in this embodiment, a hypergraph association matrix is ​​used to enhance input features, enabling the model to learn local relationship information between questions and concepts. An attention mechanism is used to construct a fully connected graph between nodes in the star-shaped expansion graph, thereby capturing global high-order relationships between nodes. Subsequently, the fully connected graph is sparsified using Gumbel-Softmax to dynamically filter out key connections. Hypergraph structure regularization is further used to regularize the sparse structure, aiming to learn information outside the hypergraph structure while maintaining the original hypergraph structure.

[0148] Second embodiment

[0149] Based on the first embodiment, in this embodiment, in order to achieve joint learning of all parameters of the model, the minimization of the prediction probability y is introduced. i and the true response r t The cross entropy loss function between , optimizes the regularization loss function proposed in the first embodiment. Specifically, before executing step S43, the following actions are also performed:

[0150] S44, determine the minimum predicted probability y i and the true response r t The cross entropy loss function between :

[0151]

[0152] Where, is the Dirac function;

[0153] S45, according to the cross entropy loss function and the regularization loss function , determine the total loss function L:

[0154]

[0155] Where, is the loss weight coefficient;

[0156] S46, updating the trainable parameters with the goal of minimizing the total loss function L.

[0157] Third embodiment

[0158] As an implementation scheme, based on the method in any of the above embodiments, refer to Figure 4 , a hypergraph attention knowledge tracking model GS-HAKT is proposed in this embodiment, and the hypergraph attention knowledge tracking model includes:

[0159] A structural embedding module based on the question-concept association matrix is ​​used to take the task to be predicted as input features in the form of a hypergraph star-shaped expansion graph and enhance the input features using the hypergraph association matrix;

[0160] Based on Gumbel-Softmax structure learning and hypergraph structure regularization module, it is used to use the attention mechanism to construct a fully connected graph between the nodes of the star-shaped expansion graph of the hypergraph, and to use the Gumbel-Softmax structure to sparsify the fully connected graph, and to screen out the key connections after sparsification, and to use hypergraph structure regularization to regularize the key connections;

[0161] The prediction module is used to predict the potential knowledge state results in the task to be predicted.

[0162] Comparative Example 1

[0163] Based on the model proposed in the third embodiment, this comparative example considers the model's embedding dimension d and the trade-off controller lamda between the cross entropy loss and the hypergraph structure regularization loss. The experimental results are shown in Figure 5 Comparative line charts and graphs showing the impact of different embedding dimensions on model performance Figure 6 A line chart comparing the impact of different lamda values ​​on model performance is shown.

[0164] Depend on Figure 5 As can be seen, the model's AUC shows a significant decrease when the embedding dimension is 64. Otherwise, as the embedding dimension increases, the AUC generally increases. However, when the dimension increases to 512, the performance improvement tends to saturate and even shows a slight decrease. Therefore, choosing an embedding dimension between 128 and 512 for each layer output in the model is more beneficial.

[0165] Depend on Figure 6 As can be seen, using a logarithmic grid with a step size of 10 for hyperparameter search within the range of 10-5 to 100, the model AUC remains relatively stable when the value is between 10-5 and 10-2. Further increases in the value of lamda lead to a significant improvement in the model AUC. However, further increases in lamda lead to a decrease in model performance, likely due to overly strong regularization that inhibits global feature learning. Therefore, using a logarithmic grid with a step size of 10 for hyperparameter search within the range of 0.1 to 1 for the controller yields better results.

[0166] Fourth embodiment

[0167] As a verification example, this example evaluates the method proposed in the first example or the model GS-HAKT proposed in the third example.

[0168] (1) Dataset design

[0169] The proposed method is evaluated on four widely used knowledge tracing benchmark datasets, namely ASSIST2009, ASSIST2017, EdNet, and algebra05.

[0170] Table 2. Statistics of the dataset

[0171]

[0172] ASSIST09 is derived from the ASSISTments online learning platform and covers student learning records from the 2009-2010 academic year. To improve data quality and ensure the effectiveness of model training, this example removes questions without concept annotations and excludes student records that have been studied less than three times.

[0173] The ASSIST17 dataset comes from the 2017 ASSISTments data mining competition. In this example, the same preprocessing as ASSIST09 is performed.

[0174] The EdNe dataset, constructed by Choi et al., contains over 130 million student interaction records on an intelligent learning platform, making it one of the largest educational datasets available. Given the massive amount of data collected, this example randomly selected 5,000 student learning records as research subjects to improve computational efficiency and ensure data processing feasibility.

[0175] The algebra05 dataset was originally released by the KDD Cup 2010 Educational Data Mining Challenge, recording students' answering behavior on algebra problems between 2005 and 2006.

[0176] Table 2 summarizes the basic statistics of the four datasets. To maintain consistent model length, we set the maximum length of each student's learning sequence to 50 across all datasets. When a student's answer record exceeds this length, their learning sequence is divided into multiple non-overlapping subsequences. During the experiment, we used 80% of the dataset for training and validation, and the remaining 20% ​​for testing.

[0177] (2) Comparative experiment

[0178] In GS-HAKT, the embedding dimension of questions and concepts is uniformly set to 256. All model parameters are randomly initialized and dynamically updated through end-to-end optimization during training. Furthermore, to fully capture the potential evolution of the student's knowledge state, the GRU model's hidden state dimension is set to 512, with one hidden layer. The learning rate is set to 0.0001. To prevent overfitting, regularization techniques are introduced with a penalty term of 0.01, a batch size of 128, and a hypergraph structure regularization loss of 0.1 in the loss function.

[0179] This experiment was conducted on a server equipped with an Nvidia RTX 4090 (24GB) graphics card. We used the PyTorch framework (version 3.9) for training and testing, using the Adam optimizer for training. The results are shown in Table 3 below:

[0180] Table 3. Performance comparison on four benchmark datasets

[0181]

[0182] In the above models, BKT (Bayesian Knowledge Tracing) is a Bayesian network, DKT (Deep Knowledge Tracing) is deep knowledge tracing, DKVMN (Dynamic Key-Value Memory Networks) is a dynamic key-value memory network, SAKT (Self-Attentive Knowledge Tracing) is self-attention knowledge tracing, AKT (Adaptive Knowledge Tracing) is adaptive knowledge tracing, and DGEKT (Directed Graph Enhanced Knowledge Tracing) is directed graph enhanced knowledge tracing.

[0183] Results show that GS-HAKT outperforms all baseline models on three datasets (ASSIST09, EdNet, and algebra05), achieving AUC improvements of 2.76%, 1.44%, and 0.37%, respectively, compared to the best baseline model. However, its performance on the ASSIST2017 dataset slightly lags behind the best baseline model. Further analysis reveals that this dataset contains a large number of student interaction records, rich in temporal information. Traditional RNN models struggle to fully capture the long-term dependencies within a student's learning path when processing long sequences. By constructing a directed graph-based representation of the student's learning path, the model's ability to model long-term dependencies is enhanced.

[0184] In contrast, traditional deep knowledge tracing models (such as DKT, DKVMV, and AKT) outperformed GA-HAKT and DGEKT on various datasets, but their performance was relatively stable. Notably, despite incorporating a self-attention mechanism, SAKT performed poorly on the EdNet dataset, likely due to its more complex structure and excessive noise.

[0185] Furthermore, the traditional BKT model performed significantly worse than deep learning methods across all datasets, demonstrating the advantages of deep learning's end-to-end optimization in automatic feature extraction and complex pattern modeling. Models that incorporate structure-aware mechanisms (such as DGEKT and GS-HAKT) exhibit significant advantages in knowledge tracking tasks, particularly on large-scale, complex datasets. This demonstrates the importance of structural information in improving model reasoning and generalization capabilities. GS-HAKT's superior performance to DGEKT reflects the fact that the introduction of a more sophisticated structural modeling mechanism can more effectively capture high-level relationships between questions and concepts, thereby improving the model's accuracy in predicting students' knowledge status.

[0186] Fifth embodiment

[0187] As a verification example, to further analyze the contribution of different modules to GS-HAKT, we designed three variants of GS-HAKT and conducted a series of ablation studies:

[0188] GS-HAKT&HGNN: Only use hypergraph neural network to capture the local relationship between questions and concepts.

[0189] GS-HAKT&SE: The structural embedding module based on the question-concept hypergraph association matrix is ​​removed from GS-HAKT, which means only global relations are captured to learn question embeddings.

[0190] GS-HAKT & Gumbel: Replace the Gumbel-Softmax module in GS-HAKT with Softmax for deterministic propagation.

[0191] Table 4. Performance comparison of GS-HAKT and its variants

[0192]

[0193] Table 4 shows the performance comparison of GS-HAKT and its variants. Experiments show that using only HGNN to capture question-concept relationships significantly underperforms GS-HAKT on the ASSIST17 and algebra05 datasets. This phenomenon may be due to the large scale and complex interactions of the ASSIST17 dataset, making it difficult for HGNN to fully capture the complex underlying global structure. Furthermore, the algebra05 dataset has a small number of concepts, and the prior structure constructed based on the dataset lacks expressive power, making it difficult to accurately model the true relationship between questions and concepts, which in turn affects HGNN's representation of question features. These results further demonstrate the limitations of relying solely on static prior hypergraph structures for structural modeling, while the Gumbel-Softmax-based learnable structural mechanism introduced by GS-HAKT offers significant advantages in capturing dynamic semantic relationships.

[0194] When the structural embedding mechanism based on the question-concept association matrix was removed from GS-HAKT, the model's performance on all datasets showed a certain degree of decline. Although the decline was not significant, it was still lower than the full GS-HAKT model. This result shows that relying solely on modeling global structure while ignoring the local structural information present in the data may lead to a decrease in the accuracy of the model in representing question features. The local connection between questions and concepts usually contains more direct and clear semantic relationships. If this structural information is not fully utilized, the model may find it difficult to capture the key cognitive paths in the learning process. Therefore, the introduction of the structural embedding mechanism helps to enhance the model's ability to comprehensively model fine-grained structural information, thereby improving the accuracy and generalization ability of knowledge tracking.

[0195] When the Gumbel-Softmax sampling mechanism was removed from GS-HAKT and Softmax was used instead for deterministic structural propagation, model performance decreased across all datasets. This result demonstrates that the differentiable discrete sampling mechanism introduced by Gumbel-Softmax is important for capturing key interactions. Unlike Softmax, Gumbel-Softmax introduces controllable randomness during training, explicitly modeling potentially important connections in the graph structure while maintaining differentiability and reducing redundant connections, effectively alleviating the over-smoothing problem common in graph neural networks. This mechanism enables the model to more accurately distinguish between important and unimportant structural relationships, thereby improving its ability to model the learner's knowledge state and its predictive performance.

[0196] In addition, as an implementation solution, Figure 7 This is a schematic diagram of the architecture of the hardware operating environment of the computer system involved in the embodiment of the present application.

[0197] like Figure 7 As shown, the computer system may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. The communication bus 1002 is used to implement communication between these components. The user interface 1003 may include a display and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as a disk storage device. The memory 1005 may also be a storage device independent of the processor 1001.

[0198] Those skilled in the art will understand that Figure 7 The computer system architecture shown in the figure does not constitute a limitation of the computer system, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0199] like Figure 7 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module and a computer program. Among them, the operating system is a program that manages and controls the hardware and software resources of the computer system, the operation of the computer program and other software or programs.

[0200] exist Figure 7 In the computer system shown, the user interface 1003 is mainly used to connect to the terminal and communicate data with the terminal; the network interface 1004 is mainly used to communicate data with the background server; the processor 1001 can be used to call the computer program stored in the memory 1005.

[0201] In this embodiment, the computer system includes: a memory 1005, a processor 1001, and a computer program stored in the memory and executable on the processor, wherein:

[0202] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0203] S10, taking the task to be predicted as input features in the form of a hypergraph star-shaped expansion graph, and using the hypergraph association matrix to enhance the input features;

[0204] S20, using an attention mechanism to construct a fully connected graph between the nodes of the star-shaped expansion graph of the hypergraph, using a Gumbel-Softmax structure to sparsify the fully connected graph, and screening out key connections after the sparsification;

[0205] S30, using hypergraph structure regularization to perform regularization constraints on the key connections;

[0206] S40: predicting the potential knowledge state result in the task to be predicted.

[0207] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0208] S11, let the expression of input feature X be:

[0209]

[0210] Where, , Represents the characteristics of the node, Represents the hyperedge e j=1，2，3...m The feature of the hyperedge is obtained by averaging the features of the nodes within the hyperedge;

[0211] S12, introducing sparse linear transformation layers into the star-shaped expansion graph and its dual graph respectively to extract their structural information and map them into a low-dimensional embedding space:

[0212]

[0213] Where, is a learnable weight matrix, is the incidence matrix, is the transposed matrix of the incidence matrix H;

[0214] S13, embedding the prior hypergraph structure through learnable linear projection to obtain enhanced input features ;

[0215]

[0216] Where, , .

[0217] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0218] S21, the attention weight of the enhanced input feature is calculated using the following formula :

[0219]

[0220] Where, are two different learnable weight matrices for the attention layer, represents the feature embedding of node i, represents the feature embedding of node j, represents the feature embedding of node k;

[0221] S22, after calculating the attention weights between any two inputs, the representation of the i-th node will be updated based on the weighted combination of all relevant inputs, thereby integrating its global dependencies with other nodes to obtain a new node representation vector :

[0222]

[0223] Where, is another learnable weight matrix of the attention layer, n represents the number of question nodes, and m represents the number of concept nodes;

[0224] S23, Gumbel-Softmax is introduced to extract sparse structure subgraphs from the fully connected graph and obtain the sparse key connections:

[0225]

[0226] Where, is the key connection after sparsification, is the activation function, is the temperature variable, is the attention weight of node i and node j in the fully connected graph, There are two independent Gumbel noise terms, adding random perturbations to the two categories respectively. Refers to the Goebel distribution.

[0227] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0228] S31, introduces the hypergraph star-shaped expansion graph as a structural prior to supervise and guide the attention matrix. The hypergraph star-shaped expansion graph structure expression is:

[0229]

[0230] Where, is the incidence matrix of the hypergraph star expansion graph, is the transposed matrix of the incidence matrix H, The dimension is The all-zero vector, The dimension is An all-zero vector of ;

[0231] S32, determining a probability transfer matrix according to the hypergraph star-shaped extended graph structure:

[0232]

[0233] Where, is a diagonal matrix, represents the probability of transferring the i-th instance to the j-th instance, n represents the number of question nodes, and m represents the number of concept nodes;

[0234] S33, according to the probability transfer matrix and the sparse key connection The cross entropy of the hypergraph is used to calculate the regularization loss function of the star-shaped expansion graph of the hypergraph :

[0235]

[0236] Where L is the number of attention layers, d i is the degree of instance i of the hypergraph star-shaped extended graph structure;

[0237] S34: Perform regularization constraints on the key connections based on the regularization loss.

[0238] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0239] S41, inputting the task to be predicted into a recurrent neural network including a gated recurrent unit:

[0240]

[0241] Where X is the input feature, a represents the student's answer sequence, [,] represents the connection operation, and h t-1 represents the student’s knowledge state at time t-1, and GRU represents a recurrent neural network containing a gated recurrent unit;

[0242] S42, the characteristics of a problem qi and the student’s knowledge state h at time t-1 t-1 Connect to get the student's knowledge status of qi :

[0243]

[0244] S43, will Mapped to the probability that the student will answer the question qi correctly in the next step t , as the underlying knowledge state results:

[0245]

[0246] in, ;

[0247] Where, is the unnormalized log probability, and b o are weight and bias parameters.

[0248] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0249] S44, determine the minimum predicted probability y i and the true response r t The cross entropy loss function between :

[0250]

[0251] Where, is the Dirac function;

[0252] S45, according to the cross entropy loss function and the regularization loss function , determine the total loss function L:

[0253]

[0254] Where, is the loss weight coefficient;

[0255] S46, updating the trainable parameters with the goal of minimizing the total loss function L.

[0256] Furthermore, those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is a computer-readable storage medium. The program instructions are executed by at least one processor in a computer system to implement the steps in the process of the above-described method embodiment.

[0257] Therefore, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various steps of the hypergraph attention knowledge tracking method described in the above embodiment.

[0258] The computer-readable storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0259] It should be noted that since the storage medium provided in the embodiments of this application is the storage medium used to implement the method of the embodiments of this application, based on the method described in the embodiments of this application, those skilled in the art will be able to understand the specific structure and deformation of the storage medium, and therefore will not be described in detail here. All storage media used in the method of the embodiments of this application fall within the scope of protection to be provided by this application.

[0260] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0261] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0262] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0263] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0264] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several distinct components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0265] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0266] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A hypergraph attention knowledge tracking method, characterized by: The method comprises the following steps: S10, taking the task to be predicted as input features in the form of a hypergraph star-shaped expansion graph, and using the hypergraph association matrix to enhance the input features; S20, using an attention mechanism to construct a fully connected graph between the nodes of the star-shaped expansion graph of the hypergraph, using a Gumbel-Softmax structure to sparsify the fully connected graph, and screening out key connections after the sparsification; S30, using hypergraph structure regularization to perform regularization constraints on the key connections; S40: predicting the potential knowledge state result in the task to be predicted.

2. The method according to claim 1, wherein In S10, using the hypergraph association matrix to enhance input features specifically includes: S11, let the expression of input feature X be: ； Where, , Represents the characteristics of the node, Represents the hyperedge e j=1，2，3...m The feature of the hyperedge is obtained by averaging the features of the nodes within the hyperedge; S12, introducing sparse linear transformation layers into the star-shaped expansion graph and its dual graph respectively to extract their structural information and map them into a low-dimensional embedding space: ； Where, is a learnable weight matrix, is the incidence matrix, is the transposed matrix of the incidence matrix H; S13, embedding the prior hypergraph structure through learnable linear projection to obtain enhanced input features ; ； Where, , .

3. The method according to claim 1, wherein The step S20 specifically includes: S21, the attention weight of the enhanced input feature is calculated using the following formula : ； Where, are two different learnable weight matrices for the attention layer, represents the feature embedding of node i, represents the feature embedding of node j, represents the feature embedding of node k; S22, after calculating the attention weights between any two inputs, the representation of the i-th node will be updated based on the weighted combination of all relevant inputs, thereby integrating its global dependencies with other nodes to obtain a new node representation vector : ； Where, is another learnable weight matrix of the attention layer, n represents the number of question nodes, and m represents the number of concept nodes; S23, Gumbel-Softmax is introduced to extract sparse structure subgraphs from the fully connected graph and obtain the sparse key connections: ； Where, is the key connection after sparsification, is the activation function, is the temperature variable, is the attention weight of node i and node j in the fully connected graph, There are two independent Gumbel noise terms, adding random perturbations to the two categories respectively. Refers to the Goebel distribution.

4. The method according to claim 1, wherein The step S30 specifically includes: S31, introduces the hypergraph star-shaped expansion graph as a structural prior to supervise and guide the attention matrix. The hypergraph star-shaped expansion graph structure expression is: ； Where, is the incidence matrix of the hypergraph star expansion graph, is the transposed matrix of the incidence matrix H, The dimension is The all-zero vector, The dimension is An all-zero vector of ; S32, determining a probability transfer matrix according to the hypergraph star-shaped extended graph structure: ； Where, is a diagonal matrix, represents the probability of transferring the i-th instance to the j-th instance, n represents the number of question nodes, and m represents the number of concept nodes; S33, according to the probability transfer matrix and the sparse key connection The cross entropy of the hypergraph is used to calculate the regularization loss function of the star-shaped expansion graph of the hypergraph : ； Where L is the number of attention layers, d i is the degree of instance i of the hypergraph star-shaped extended graph structure; S34: Perform regularization constraints on the key connections based on the regularization loss.

5. The method according to claim 1, wherein The S40 specifically includes: S41, inputting the task to be predicted into a recurrent neural network including a gated recurrent unit: ； Where X is the input feature, a represents the student's answer sequence, [,] represents the connection operation, and h t-1 represents the student’s knowledge state at time t-1, and GRU represents a recurrent neural network containing a gated recurrent unit; S42, the characteristics of a problem qi and the student’s knowledge state h at time t-1 t-1 Connect to get the student's knowledge status of qi : ； S43, will Mapped to the probability that the student will answer the question qi correctly in the next step t , as the underlying knowledge state results: ； in, ; Where, is the unnormalized log probability, and b o are weight and bias parameters.

6. The method according to claim 5, wherein Before executing S43, the method further includes: S44, determine the minimum predicted probability y i and the true response r t The cross entropy loss function between : ； Where, is the Dirac function; S45, according to the cross entropy loss function and the regularization loss function , determine the total loss function L: ； Where, is the loss weight coefficient; S46, updating the trainable parameters with the goal of minimizing the total loss function L.

7. A hypergraph attention knowledge tracking model, characterized by: The hypergraph attention knowledge tracking model includes: A structural embedding module based on the question-concept association matrix is ​​used to take the task to be predicted as input features in the form of a hypergraph star-shaped expansion graph and enhance the input features using the hypergraph association matrix; Based on Gumbel-Softmax structure learning and hypergraph structure regularization module, it is used to use the attention mechanism to construct a fully connected graph between the nodes of the star-shaped expansion graph of the hypergraph, and to use the Gumbel-Softmax structure to sparsify the fully connected graph, and to screen out the key connections after sparsification, and to use hypergraph structure regularization to regularize the key connections; The prediction module is used to predict the potential knowledge state results in the task to be predicted.

8. The hypergraph attention knowledge tracking model according to claim 7, wherein: The embedding dimension of each layer output in the hypergraph attention knowledge tracking model is between 128-512, and the controller uses a logarithmic grid with a step size of 10 for hyperparameter search in the range of 0.1 to 1.

9. A computer system, characterized in that: The computer system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the hypergraph attention knowledge tracking method as described in any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the hypergraph attention knowledge tracking method as described in any one of claims 1 to 6.

Images