Multi-granularity sparse attention knowledge tracking method

Abstract

The invention relates to a multi-granularity sparse attention knowledge tracking method, and belongs to the technical field of dynamic tracking of knowledge state evolution of learners. The invention discloses a multi-granularity sparse attention knowledge tracking method. The method comprises the following steps: S1, carrying out data mining and analysis on student learning behavior data; s2, a result processed in the step S1 is fused with exercise answering conditions and exercise answering time; s3, grain size feature extraction and feature fusion; s4, performing feature fusion on a result processed in the step S3 and a position code; s5, performing consistency characterization of the knowledge state and the learning situation; and S6, predicting and constructing a loss function. In the application, a three-level feature extraction module is constructed, short-term learning behavior features, long-term knowledge relevance and potential knowledge migration paths are effectively identified, and the ability of the model to acquire various dependencies is enhanced.

Description

Technical Field

[0001] This invention relates to a knowledge tracking method with multi-granularity sparse attention, belonging to the field of dynamic tracking technology for the evolution of learners' knowledge states. Background Technology

[0002] Knowledge tracing (KT), a core technology in intelligent education, dynamically infers knowledge states by analyzing learning records. Based on this dynamic modeling capability, it can accurately predict students' performance in future interactions and generate personalized learning paths, supporting personalized teaching and effectively improving teaching quality. However, existing methods have limitations in deeply capturing the multi-granular temporal dependencies and cognitive evolution patterns in complex learning behaviors, thus restricting the performance of accurately simulating and predicting learners' knowledge state evolution and future performance. Summary of the Invention

[0003] To address the problems existing in the prior art, this application provides a multi-granularity sparse attention-based knowledge tracking method. The technical solution of this application significantly improves the modeling ability of multi-granularity dependencies in complex learning behaviors and enhances the consistency between knowledge state and learning context representation.

[0004] The present invention solves the above-mentioned technical problems through the following technical solution:

[0005] A knowledge tracking method based on multi-granularity sparse attention, characterized by the following steps:

[0006] S1. Data mining and analysis of student learning behavior data; including three parts: question embedding, global feature enhancement of knowledge point-question, and question difficulty encoding. The analysis results of the three parts are enhanced by the latent information of the questions to construct a comprehensive semantic representation for each question.

[0007] S2. Integrate the results of step S1 with the exercise answer status and exercise answer time to obtain interactive embedding;

[0008] S3. Granularity-level feature extraction and feature fusion; Extract coarse-grained, fine-grained, and supplementary-grained features from the results of step S2, and perform dynamic fusion of multi-granularity features;

[0009] S4. Combine the result of dynamic fusion of multi-granularity features in step S3 with the location encoding and obtain the final dynamic feature consistent representation.

[0010] S5. Encode the time interval between the results of step S2 and the exercises using a multi-perspective context, and then define the acquisition factor and forgetting factor through a gating mechanism to represent the consistency between the knowledge state and the learning context.

[0011] S6. Predict and construct the loss function: Update the processing results of steps S4 and S5 using the knowledge state, and use the dynamically updated knowledge state. Based on this, predictions of future learning performance are generated by combining the current learning task with the current learning task.

[0012] In order to better achieve the purpose of this invention, the following adjustments and improvements are made to the above technical solution:

[0013] Furthermore, in step S1, the problem embedding, knowledge point-problem global feature enhancement, and problem difficulty encoding are described through the following modeling:

[0014] S11, Exercise Embedded:

[0015] Exercise number e t These are mapped to high-dimensional embedding vectors, representing the basic features of the exercises. The specific mapping is accomplished through the embedding matrix:

[0016] ,

[0017] Among them, E t Exercise number e t The high-dimensional embedding vector, For d e 3D real matrix space, For embedded functions, For the time period; this feature captures the basic information of the exercise;

[0018] S12, Global Feature Enhancement for Knowledge Points and Exercises:

[0019] Each exercise is numbered e t Link one or more knowledge points c t The single knowledge point embedding vector is calculated using the following formula:

[0020] ,

[0021] Wherein, S(c t ) indicates the relationship with exercise number e t Related single-knowledge-point embedding vectors, For d k 3D real matrix space;

[0022] The definition of a multi-knowledge point embedding vector is as follows:

[0023] ,

[0024] in, This indicates the exercise number e t The feature vector obtained by averaging multiple related knowledge points, N cIndicates the relationship with exercise number e t The number of related knowledge points is not zero, S(c t ) is related to the exercise number e t The embedded vectors of associated single knowledge points; this mechanism captures global interaction information between knowledge points and exercises;

[0025] S13, Exercise Difficulty Code:

[0026] Based on the characteristics of exercises in the student's learning process, exercise number e t The difficulty level of the exercises is d et The definition is as follows:

[0027] ,

[0028] in:

[0029] The number of times the exercises were answered correctly;

[0030] Total number of times the exercises were answered;

[0031] SD: Standard deviation of the correct answer rate, indicating the degree of dispersion of the answer results;

[0032] S: Smoothing parameter, used to avoid the influence of sparse data;

[0033] Exercise difficulty d et The embedding layer maps the vector representation D. et , For d k 3D real matrix space:

[0034] ;

[0035] S14, Integrating Potential Information:

[0036] The final latent representation of the exercises is defined as follows, through the fusion of exercise embeddings, global feature enhancement of knowledge points and exercises, and difficulty information:

[0037] ,

[0038] Where, q enhanced This represents the enhanced latent representation of the exercise, which incorporates the features E of the exercise itself. t Characteristics of knowledge points and difficulty information D et , ⊕ represents the vector concatenation operation.

[0039] Furthermore, in step S2, the result of processing in S1 is integrated with the exercise answer status and exercise answer time to obtain interactive embedding, which is achieved as follows:

[0040] S21, Exercises - Interactive Answering:

[0041] Student's answer results ∈{0, 1}, 1 for correct and 0 for incorrect, mapped to a high-dimensional vector through the embedding matrix:

[0042] ,

[0043] For d a 3D real matrix space, For the embedded function, the answer behavior is embedded a. t It provides information about the student's answer at the current moment;

[0044] Enhanced latent representation of exercises q enhanced Embedded with response behavior a t The interaction is defined as follows:

[0045] ,

[0046] Where ⊕ represents the vector concatenation operation, qa t This indicates the combined characteristics of the exercise and the act of answering it;

[0047] S22, Interactive Embedding:

[0048] Answering time Mapping to a high-dimensional embedding vector via the embedding matrix:

[0049] ,

[0050] in:

[0051] r t A high-dimensional embedding vector representing the time taken by a student to complete the current question. For d k 3D real matrix space;

[0052] Further combining the response time characteristics, interactive embedding Represented as:

[0053] ,

[0054] Here, MLP stands for Multilayer Perceptron, used to extract high-order nonlinear relationships of joint features; the final interactive embedding It integrates information on knowledge points, the difficulty characteristics of exercises, students' answering behavior, and the dynamic characteristics of answering time.

[0055] Furthermore, step S3 includes the following steps:

[0056] S31, extracting fine-grained information from local instantaneous response modeling:

[0057] By employing a sliding window mechanism, we can focus on the contextual information of the current time step and its neighboring windows to capture short-term dependencies in the sequence; for time step 𝑡, we define a sliding window w t w t ={x t-τ , ..., x t}, where τ is the cognitive span, and τ takes a value of 4; the attention weight within the window is h. L The calculation is as follows:

[0058] ,

[0059] Among them, h L This represents the attention weight within the window, where L represents local information. This indicates the degree of attention that time step i pays to time step j; This indicates the sliding window w t The level of attention paid to all time steps within the period; among them, As the key value of time step j As the key value for time step k, q i As the query value for time step i This represents a scaling factor, effectively modeling students' short-term learning patterns over consecutive time steps;

[0060] S32, extracting coarse-grained information from global knowledge construction and aggregation:

[0061] Core Knowledge Points , The core knowledge points correspond to the course content, where m represents the number of core knowledge points and their attention weights. The calculation is as follows:

[0062] ,

[0063] in, Indicates time step The degree of attention paid to time step j; This indicates the degree of attention paid to all time steps within the core knowledge point set G; where, As the key value of time step j As a time step The query value, Representing the scaling factor, the core knowledge set G is expressed through h. G The weights continuously influence the subsequent learning trajectory, providing a holistic perspective on the impact on students' knowledge status;

[0064] S33, supplementary granular information is extracted by stochastic cognitive transition enhancement:

[0065] Introducing Bernoulli random variables Controlling attention, its attention weight h R The calculation is as follows:

[0066] ,

[0067] Among them, h R Attention weights are used to capture supplementary granular information. This represents supplementary granularity information. This indicates the degree of attention that time step v pays to time step j;

[0068] This represents the degree of attention paid to all time steps within a set of n random variables; where, As the key value of time step j As the query value for time step v Represents the scaling factor; through random probability The probability of 0.1 is used to randomly activate historical interactions, promoting transfer learning between non-adjacent knowledge points.

[0069] S34, Dynamic Fusion of Multi-Granularity Features:

[0070] ,

[0071] ,

[0072] Among them, h L h G h R These represent fine-grained, coarse-grained, and supplementary granularity information, respectively; λ m To integrate information, h m Representing one of the three granularities of information, σ is the activation function, and w n D is a trainable parameter matrix. final It is a contextual representation that is integrated into a whole to achieve dynamic allocation of different cognitive resources.

[0073] Furthermore, step S4 includes the following steps:

[0074] S41, Feature Extension:

[0075] To model the sequential relationships in the input sequence, positional encoding is introduced. The calculation formula is as follows:

[0076] , ,

[0077] Where, p t,2z p represents the position code at even times. t,2z+1This represents the position code for odd-numbered time points, where sin represents the sine function, cos represents the cosine function, t is the time point, z is the flag symbol, and d represents the dimension.

[0078] S42, the final consistent representation of dynamic features:

[0079] context representation With position encoding The final dynamic feature consistency representation is generated through residual connections and nonlinear transformations. :

[0080] ,

[0081] in, For regularization, For activation function, This is the weight matrix.

[0082] Furthermore, step S5 includes the following steps:

[0083] S51, Multi-view learning context encoding:

[0084] The time interval between a student's previous answer and their current answer is mapped to a high-dimensional vector using an embedding matrix:

[0085] ,

[0086] Among them, b t For the high-dimensional vector of the time interval after embedding, Δ t This represents the raw time interval data between answering exercises. For embedded functions, For a moment, For d k 3D real matrix space;

[0087] The features learned from multiple perspectives after encoding are expressed as follows:

[0088] ,

[0089] in, For regularization, C t To learn the encoded features from multiple perspectives, x t For the final interactive embedding, b t w is a high-dimensional vector representing the time interval after embedding. c These are learnable parameters;

[0090] S52, Obtaining the factor μ through the gating definition. l Knowledge acquisition status (KA) and forgetting factor (μ) f To characterize students' knowledge absorption and forgetting behavior:

[0091] ,

[0092] ,

[0093] ,

[0094] Where, μ l To obtain the factors, KA represents the knowledge acquisition status, and μ f Forgetting factor, C t To learn the encoded features from multiple perspectives, F t To integrate features that combine contextual and location information, h t Let be the knowledge state at time t. w is a learnable parameter l w is a learnable parameter f Here are the learnable parameters, σ is the activation function, tanh is the activation function, and ⊙ represents element-wise multiplication;

[0095] S53, the final knowledge state update formula is:

[0096] ,

[0097] Among them, h t+1 Let h be the knowledge state at time t+1. t Let KA be the knowledge state at time t. t Let μ represent the knowledge acquired at time t. f The forgetting factor is represented by ⊙, which indicates element-wise multiplication.

[0098] By integrating these learning behaviors from multiple perspectives, this method is able to capture changes in students' knowledge status in complex learning scenarios.

[0099] Furthermore, step S6 includes the following steps:

[0100] S61, Prediction:

[0101] S611, Knowledge State Focus:

[0102] ,

[0103] in, Let Q be the knowledge state focused on by the exercise at time t+1, and let e be the knowledge point embedding matrix. t+1 For the exercise at time t+1, h t+1 This represents the knowledge state at time t+1;

[0104] S612, Learning Performance Prediction:

[0105] ,

[0106] Among them, W p Here are the learnable parameters, σ is the activation function, and e is the activation parameter. t+1 The exercise is for time t+1. The knowledge state focused on by the exercise at time t+1, and the final output y t+1 This represents the accuracy of the prediction;

[0107] S62, Loss Function:

[0108] The training objective function optimizes the predicted student performance by minimizing the cross-entropy loss function, maximizing the match between the predicted probability and the true label for each sample, thereby reducing the overall error. The specific formula is as follows:

[0109] ,

[0110] Where L is the loss function, y t+1 a represents the prediction accuracy at time t+1. t+1 The label represents the actual answer at time t+1, and n represents the predicted number of questions.

[0111] This application has the following advantages:

[0112] This application first performs data mining and analysis on students' learning data to construct a comprehensive semantic representation for each exercise; second, it obtains interactive embeddings; then, it extracts and fuses features using the granular feature extraction method proposed in this invention; next, it fuses the results of dynamic fusion of multi-granular features with positional encoding to obtain a final dynamic feature consistent representation; then, it performs consistent representation of knowledge state and learning context to make the method more in line with the laws of educational cognition; finally, it updates the knowledge state to form a multi-granular knowledge tracking network containing student behavioral information, thereby improving the performance of predicting student performance.

[0113] This application not only preserves information at various granularities of student learning behavior data but also further explores the relationship between learning and forgetting through multi-perspective contextual coding and gating mechanisms to achieve a two-way alignment between educational principles and this method. This makes this application more consistent with educational principles. Attached Figure Description

[0114] Figure 1 This is a flowchart illustrating a knowledge tracking method using multi-granularity sparse attention proposed in this application;

[0115] Figure 2 The flowchart is for steps S1 and S2.

[0116] Figure 3 The flowchart is for steps S2 to S5.

[0117] Figure 4 This is a flowchart of steps S5 to S6. Detailed Implementation

[0118] The following embodiments, in conjunction with the accompanying drawings, are merely for illustrating the technical solutions described in the claims and are not intended to limit the scope of protection of the claims.

[0119] Combined with appendix Figure 1-4 A knowledge tracking method based on multi-granularity sparse attention.

[0120] First, data mining and analysis are performed on students' learning data; second, complete interactive embeddings are obtained; then, feature extraction and feature fusion are performed using the granular feature extraction method proposed in this invention; then, the final dynamic feature consistent representation is obtained; then, the consistency representation of knowledge state and learning context is performed; finally, the knowledge state is updated to form a multi-granularity knowledge tracking network containing student behavior information to improve the performance of predicting student performance.

[0121] Specifically, the steps include the following:

[0122] S1. Data mining and analysis of student learning behavior data; including three parts: question embedding, global feature enhancement of knowledge point-question, and question difficulty encoding. The analysis results of the three parts are enhanced by the latent information of the questions to construct a comprehensive semantic representation for each question.

[0123] S11, Exercise Embedded:

[0124] Exercise number e t These are mapped to high-dimensional embedding vectors, representing the basic features of the exercises. The specific mapping is accomplished through the embedding matrix:

[0125] ,

[0126] Among them, E t Exercise number e t The high-dimensional embedding vector, For d e 3D real matrix space, For embedded functions, For the time period; this feature captures the basic information of the exercise;

[0127] S12, Global Feature Enhancement for Knowledge Points and Exercises:

[0128] Each exercise is numbered e t Link one or more knowledge points c t The single knowledge point embedding vector is calculated using the following formula:

[0129] ,

[0130] Wherein, S(c t ) indicates the relationship with exercise number e t Related single-knowledge-point embedding vectors, For d k 3D real matrix space;

[0131] The definition of a multi-knowledge point embedding vector is as follows:

[0132] ,

[0133] in, This indicates the exercise number e t The feature vector obtained by averaging multiple related knowledge points, N c Indicates the relationship with exercise number e t The number of related knowledge points is not zero, S(c t ) is related to the exercise number e t The embedded vectors of associated single knowledge points; this mechanism captures global interaction information between knowledge points and exercises;

[0134] S13, Exercise Difficulty Code:

[0135] Based on the characteristics of exercises in the student's learning process, exercise number e t The difficulty level of the exercises is d et The definition is as follows:

[0136] ,

[0137] in:

[0138] The number of times the exercises were answered correctly;

[0139] Total number of times the exercises were answered;

[0140] SD: Standard deviation of the correct answer rate, indicating the degree of dispersion of the answer results;

[0141] S: Smoothing parameter, used to avoid the influence of sparse data;

[0142] Exercise difficulty d et The embedding layer maps the vector representation D. et , For d k 3D real matrix space:

[0143] ;

[0144] S14, Integrating Potential Information:

[0145] The final latent representation of the exercises is defined as follows, through the fusion of exercise embeddings, global feature enhancement of knowledge points and exercises, and difficulty information:

[0146] ,

[0147] Where, q enhanced This represents the enhanced latent representation of the exercise, which incorporates the features E of the exercise itself. t Characteristics of knowledge points and difficulty information D et , ⊕ represents the vector concatenation operation.

[0148] S2. Integrate the results of step S1 with the exercise answer status and exercise answer time to obtain interactive embedding;

[0149] S21, Exercises - Interactive Answering:

[0150] Student's answer results ∈{0, 1}, 1 for correct and 0 for incorrect, mapped to a high-dimensional vector through the embedding matrix:

[0151] ,

[0152] For d a 3D real matrix space, For the embedded function, the answer behavior is embedded a. t It provides information about the student's answer at the current moment;

[0153] Enhanced latent representation of exercises q enhanced Embedded with response behavior a t The interaction is defined as follows:

[0154] ,

[0155] Where ⊕ represents the vector concatenation operation, qa t This indicates the combined characteristics of the exercise and the act of answering it;

[0156] S22, Interactive Embedding:

[0157] Answering time Each is mapped to a high-dimensional embedding vector through an embedding matrix:

[0158] ,

[0159] in:

[0160] r t A high-dimensional embedding vector representing the time taken by a student to complete the current question. For d k 3D real matrix space;

[0161] Further combining the response time characteristics, interactive embedding Represented as:

[0162] ,

[0163] Here, MLP stands for Multilayer Perceptron, used to extract high-order nonlinear relationships of joint features; the final interactive embedding It integrates information on knowledge points, the difficulty characteristics of exercises, student answering behavior, and dynamic characteristics of answering time;

[0164] S3. Granularity-level feature extraction and feature fusion; Extract coarse-grained, fine-grained, and supplementary-grained features from the results of step S2, and perform dynamic fusion of multi-granularity features;

[0165] S31, extracting fine-grained information from local instantaneous response modeling:

[0166] By employing a sliding window mechanism, we can focus on the contextual information of the current time step and its neighboring windows to capture short-term dependencies in the sequence; for time step 𝑡, we define a sliding window w t w t ={x t-τ , ..., x t}, where τ is the cognitive span, and τ takes a value of 4; the attention weight within the window is h. L The calculation is as follows:

[0167] ,

[0168] Among them, h L This represents the attention weight within the window, where L represents local information. This indicates the degree of attention that time step i pays to time step j; This indicates the sliding window w t The level of attention paid to all time steps within the period; among them, As the key value of time step j As the key value for time step k, q i As the query value for time step i This represents a scaling factor, effectively modeling students' short-term learning patterns over consecutive time steps;

[0169] S32, extracting coarse-grained information from global knowledge construction and aggregation:

[0170] Core Knowledge Points , The core knowledge points correspond to the course content, where m represents the number of core knowledge points and their attention weights. The calculation is as follows:

[0171] ,

[0172] in, Indicates time step The degree of attention paid to time step j; This indicates the degree of attention paid to all time steps within the core knowledge point set G; where, As the key value of time step j As a time step The query value, Representing the scaling factor, the core knowledge set G is expressed through h. G The weights continuously influence the subsequent learning trajectory, providing a holistic perspective on the impact on students' knowledge status;

[0173] S33, supplementary granular information is extracted by stochastic cognitive transition enhancement:

[0174] Introducing Bernoulli random variables Controlling attention, its attention weight h R The calculation is as follows:

[0175] ,

[0176] Among them, h R Attention weights are used to capture supplementary granular information. This represents supplementary granularity information. This indicates the degree of attention that time step v pays to time step j;

[0177] This represents the degree of attention paid to all time steps within a set of n random variables; where, As the key value of time step j As the query value for time step v Represents the scaling factor; through random probability The probability of 0.1 is used to randomly activate historical interactions, promoting transfer learning between non-adjacent knowledge points.

[0178] S34, Dynamic Fusion of Multi-Granularity Features:

[0179] ,

[0180] ,

[0181] Among them, h L h G h R These represent fine-grained, coarse-grained, and supplementary granularity information, respectively; λ m To integrate information, h m Representing one of the three granularities of information, σ is the activation function, and wn D is a trainable parameter matrix. final It is a contextual representation that is integrated into a whole to achieve dynamic allocation of different cognitive resources;

[0182] S4. Combine the result of dynamic fusion of multi-granularity features in step S3 with the location encoding and obtain the final dynamic feature consistent representation.

[0183] S41, Feature Extension:

[0184] To model the sequential relationships in the input sequence, positional encoding is introduced. The calculation formula is as follows:

[0185] , ,

[0186] Where, p t,2z p represents the position code at even times. t,2z+1 This represents the position code for odd-numbered time points, where sin represents the sine function, cos represents the cosine function, t is the time point, z is the flag symbol, and d represents the dimension.

[0187] S42, the final consistent representation of dynamic features:

[0188] context representation With position encoding The final dynamic feature consistency representation is generated through residual connections and nonlinear transformations. :

[0189] ,

[0190] in, For regularization, For activation function, S5. Encode the time interval between the results of step S2 and the exercises using a multi-perspective contextualization method. Then, define the acquisition factor and forgetting factor through a gating mechanism to represent the consistency between the knowledge state and the learning context.

[0191] S51, Multi-view learning context encoding:

[0192] The time interval between a student's previous answer and their current answer is mapped to a high-dimensional vector using an embedding matrix:

[0193] ,

[0194] Among them, b t For the high-dimensional vector of the time interval after embedding, Δ t This represents the raw time interval data between answering exercises. For embedded functions, For a moment, For d k 3D real matrix space;

[0195] The features learned from multiple perspectives after encoding are expressed as follows:

[0196] ,

[0197] in, For regularization, C t To learn the encoded features from multiple perspectives, x t For the final interactive embedding, b t w is a high-dimensional vector representing the time interval after embedding. c These are learnable parameters;

[0198] S52, Obtaining the factor μ through the gating definition. l Knowledge acquisition status (KA) and forgetting factor (μ) f To characterize students' knowledge absorption and forgetting behavior:

[0199] ,

[0200] ,

[0201] ,

[0202] Where, μ l To obtain the factors, KA represents the knowledge acquisition status, and μ f Forgetting factor, C t To learn the encoded features from multiple perspectives, F t To integrate features that combine contextual and location information, h t Let be the knowledge state at time t. w is a learnable parameter l w is a learnable parameter f Here are the learnable parameters, σ is the activation function, tanh is the activation function, and ⊙ represents element-wise multiplication;

[0203] S53, the final knowledge state update formula is:

[0204] ,

[0205] Among them, h t+1 Let h be the knowledge state at time t+1. t Let KA be the knowledge state at time t. t Let μ represent the knowledge acquired at time t. f The forgetting factor is represented by ⊙, which indicates element-wise multiplication.

[0206] By integrating these learning behaviors from multiple perspectives, this method is able to capture changes in students' knowledge status in complex learning scenarios.

[0207] S6. Predict and construct the loss function: Update the processing results of steps S4 and S5 through the knowledge state, using the dynamically updated knowledge state h. t Based on this, and combined with the current learning task, predictive values ​​for future learning performance are generated;

[0208] S61, Prediction:

[0209] S611, Knowledge State Focus:

[0210] ,

[0211] in, Let Q be the knowledge state focused on by the exercise at time t+1, and let e be the knowledge point embedding matrix. t+1 For the exercise at time t+1, h t+1 This represents the knowledge state at time t+1;

[0212] S612, Learning Performance Prediction:

[0213] ,

[0214] Among them, W p Here are the learnable parameters, σ is the activation function, and e is the activation parameter. t+1 The exercise is for time t+1. The knowledge state focused on by the exercise at time t+1, and the final output y t+1 This represents the accuracy of the prediction;

[0215] S62, Loss Function:

[0216] The training objective function optimizes the predicted student performance by minimizing the cross-entropy loss function, maximizing the match between the predicted probability and the true label for each sample, thereby reducing the overall error. The specific formula is as follows:

[0217] ,

[0218] Where L is the loss function, y t+1 a represents the prediction accuracy at time t+1. t+1 The label represents the actual answer at time t+1, and n represents the predicted number of questions.

[0219] Method validation was performed, using the area under the curve and accuracy as evaluation metrics.

[0220] This embodiment uses three benchmark datasets from real-world educational scenarios for method validation: ASSISTments2012, ASSISTments2017, and the large-scale online learning interaction dataset EdNet-KT1 (in this method, 20,000 students and their problem-solving records are randomly selected to form the dataset).

[0221] This method sets all input sequences to a fixed length of 200 based on the average sequence length of the dataset. For sequences exceeding the fixed length, they are segmented into several unique subsequences according to the fixed length. For sequences shorter than the fixed length, they are padded to the fixed length using zero vectors. For all datasets, this method performs standard 5x cross-validation on all methods. 80% of the data is used for training and validation (training to validation ratio of 8:2), and the remaining 20% ​​is used as an independent test set. During training, all parameters are randomly initialized in a uniform distribution. Furthermore, the learning rate is set to 0.003, a Dropout layer is added to the network, and the dropout rate is set to 0.1. The size parameter embedding feature is set to 64.

[0222] The results of this method were compared with those of other methods on three benchmark datasets. The AUC (Area Under Curve) and ACC (Accuracy) results are shown in Table 1.

[0223] Table 1 compares the results of this method with other methods on three benchmark datasets.

[0224]

[0225] Notes: Hawkeskt (Temporal Cross-Effects in Knowledge Tracing): A knowledge tracing method that considers temporal cross-effects; Lpkt (Learning Process-consistent Knowledge Tracing): A knowledge tracing method that considers the consistency of the learning process; Dtransformer (Tracing Knowledge Instead of Patterns: Stable Knowledge Tracing with Diagnostic Transformer): A stable knowledge tracing method based on Diagnostic Transformer; Sparsekt (Towards Robust Knowledge Tracing Models via k-Sparse Attention): A robust knowledge tracing method based on k-sparse attention; Stablekt (Enhancing Length Generalization for Attention Based Knowledge Tracing Models with Linear Biases): An attention-based knowledge tracing method with linear bias; Extrakt (Extending Context Window of Attention Based Knowledge Tracing Models via Length Extrapolation): An attention-based knowledge tracing method that extends the context window through length extrapolation; csKT (csKT: Addressing cold-start problem in knowledge tracing via kernel bias and Coneattention: A knowledge tracking method that solves the cold start problem through kernel bias and cone attention.

[0226] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A knowledge tracking method using multi-granularity sparse attention, characterized in that, Includes the following steps: S1. Data mining and analysis of student learning behavior data; including three parts: question embedding, global feature enhancement of knowledge point-question, and question difficulty encoding. The analysis results of the three parts are enhanced by the latent information of the questions to construct a comprehensive semantic representation for each question. S2. Integrate the results of step S1 with the exercise answer status and exercise answer time to obtain interactive embedding; S3. Granular-level feature extraction and feature fusion; Extract the three-level features of coarse-grained, fine-grained, and supplementary granularity from the results of step S2, and perform dynamic fusion of multi-granularity features; S4. Combine the result of dynamic fusion of multi-granularity features in step S3 with the location encoding and obtain the final dynamic feature consistent representation. S5. Encode the time interval between the results of step S2 and the exercises using a multi-perspective context, and then define the acquisition factor and forgetting factor through a gating mechanism to represent the consistency between the knowledge state and the learning context. S6. Predict and construct the loss function: Update the processing results of steps S4 and S5 using the knowledge state, and use the dynamically updated knowledge state h. t Based on this, predictions of future learning performance are generated by combining the current learning task with the current learning task.

2. The knowledge tracking method for multi-granularity sparse attention according to claim 1, characterized in that, In step S1, the problem embedding, global feature enhancement of knowledge point-problem, and problem difficulty encoding are described by the following modeling: S11, Exercise Embedded: Exercise number e t These are mapped to high-dimensional embedding vectors, representing the basic features of the exercises. The specific mapping is accomplished through the embedding matrix: ， Among them, E t Exercise number e t The high-dimensional embedding vector, For d e 3D real matrix space, For embedded functions, For the time period; this feature captures the basic information of the exercise; S12, Global Feature Enhancement for Knowledge Points and Exercises: Each exercise is numbered e t Link one or more knowledge points c t The single knowledge point embedding vector is calculated using the following formula: ， Wherein, S(c t ) indicates the relationship with exercise number e t Related single-knowledge-point embedding vectors, For d k 3D real matrix space; The definition of a multi-knowledge point embedding vector is as follows: ， Among them, C avg This indicates the exercise number e t The feature vector obtained by averaging multiple related knowledge points, N c Indicates the relationship with exercise number e t The number of related knowledge points is not zero, S(c t ) is related to the exercise number e t The embedded vectors of associated single knowledge points; this mechanism captures global interaction information between knowledge points and exercises; S13, Exercise Difficulty Code: Based on the characteristics of exercises in the student's learning process, exercise number e t The difficulty level of the exercises is d et The definition is as follows: ， in: The number of times the exercises were answered correctly; Total number of times the exercises were answered; SD: Standard deviation of the correct answer rate, indicating the degree of dispersion of the answer results; S: Smoothing parameter, used to avoid the influence of sparse data; Exercise difficulty d et The embedding layer maps the vector representation D. et , For d k 3D real matrix space: ； S14, Integrating Potential Information: The final latent representation of the exercises is defined as follows, through the fusion of exercise embeddings, global feature enhancement of knowledge points and exercises, and difficulty information: ， Where, q enhanced This represents the enhanced latent representation of the exercise, which incorporates the features E of the exercise itself. t Knowledge Point Characteristics C avg and difficulty information D et , ⊕ represents the vector concatenation operation.

3. The knowledge tracking method for multi-granularity sparse attention according to claim 2, characterized in that, In step S2, the result of processing in S1 is integrated with the exercise answer status and the exercise answer time to obtain interactive embedding, which is achieved as follows: S21, Exercises - Interactive Answering: The student's answer result a' t ∈{0, 1}, 1 for correct and 0 for incorrect, mapped to a high-dimensional vector through the embedding matrix: ， For d a 3D real matrix space, For the embedded function, the answer behavior is embedded a. t It provides information about the student's answer at the current moment; Enhanced latent representation of exercises q enhanced Embedded with response behavior a t The interaction is defined as follows: ， Where ⊕ represents the vector concatenation operation, qa t This indicates the combined characteristics of the exercise and the act of answering it; S22, Interactive Embedding: Answering time Mapping to a high-dimensional embedding vector via the embedding matrix: ， in: r t A high-dimensional embedding vector representing the time taken by a student to complete the current question. For d k 3D real matrix space; Further combining the response time characteristics, interactive embedding Represented as: ， Here, MLP stands for Multilayer Perceptron, used to extract high-order nonlinear relationships of joint features; the final interactive embedding It integrates information on knowledge points, the difficulty characteristics of exercises, students' answering behavior, and the dynamic characteristics of answering time.

4. The knowledge tracking method for multi-granularity sparse attention according to claim 3, characterized in that, Step S3 includes the following steps: S31, extracting fine-grained information from local instantaneous response modeling: By employing a sliding window mechanism, we can focus on the contextual information within the current time step and its neighboring windows to capture short-term dependencies in the sequence; for time t, we define a sliding window w. t w t ={x t-τ , ..., x t }, where τ is the cognitive span, and τ takes a value of 4; the attention weight within the window is h. L The calculation is as follows: ， Among them, h L This represents the attention weight within the window, where L represents local information. This indicates the degree of attention that time step i pays to time step j; This indicates the sliding window w t The level of attention paid to all time steps within the period; among them, As the key value of time step j As the key value for time step k, q i As the query value for time step i Represents the scaling factor; S32, extracting coarse-grained information from global knowledge construction and aggregation: Core Knowledge Points , The core knowledge points correspond to the course content, where m represents the number of core knowledge points and their attention weights. The calculation is as follows: ， in, Indicates time step The degree of attention paid to time step j; This indicates the degree of attention paid to all time steps within the core knowledge point set G; where, As the key value of time step j As a time step The query value, Represents the scaling factor; S33, supplementary granular information is extracted by stochastic cognitive transition enhancement: Introducing Bernoulli random variables Controlling attention, its attention weight h R The calculation is as follows: ， Among them, h R Attention weights are used to capture supplementary granular information. This represents supplementary granularity information. This indicates the degree of attention that time step v pays to time step j; This indicates the degree of attention paid to all time steps within a set of n random variables; in, As the key value of time step j As the query value for time step v Represents the scaling factor; S34, Dynamic Fusion of Multi-Granularity Features: ， ， Among them, h L h G h R These represent fine-grained, coarse-grained, and supplementary granularity information, respectively; λ m To integrate information, h m Representing one of the three granularities of information, 𝜎 is the Sigmoid function, w n D is a trainable parameter matrix. final It is a contextual representation that is integrated into a whole to achieve dynamic allocation of different cognitive resources.

5. The knowledge tracking method for multi-granularity sparse attention according to claim 2, characterized in that, Step S4 includes the following steps: S41, Feature Extension: To model the sequential relationships in the input sequence, positional encoding is introduced. The calculation formula is as follows: ， ， Where, p t,2z p represents the position code at even times. t,2z+1 This represents the position code for odd-numbered time points, where sin represents the sine function, cos represents the cosine function, t is the time point, z is the flag symbol, and d represents the dimension. S42, the final consistent representation of dynamic features: context representation With position encoding The final dynamic feature consistency representation is generated through residual connections and nonlinear transformations. : ， in, For regularization, For activation function, This is the weight matrix.

6. The knowledge tracking method for multi-granularity sparse attention according to claim 1, characterized in that, Step S5 includes the following steps: S51, Multi-view learning context encoding: The time interval between a student's previous answer and their current answer is mapped to a high-dimensional vector using an embedding matrix: ， Among them, b t For the high-dimensional vector of the time interval after embedding, Δ t This represents the raw time interval data between answering exercises. For embedded functions, For a moment, For d k 3D real matrix space; The features learned from multiple perspectives after encoding are expressed as follows: ， Among them, C t To learn the encoded features from multiple perspectives, For regularization, x t For the final interactive embedding, b t w is a high-dimensional vector representing the time interval after embedding. c These are learnable parameters; S52, Obtaining the factor μ through the gating definition. l Knowledge acquisition status (KA) and forgetting factor (μ) f To characterize students' knowledge absorption and forgetting behavior: ， ， ， Where, μ l To obtain the factors, KA represents the knowledge acquisition status, and μ f Forgetting factor, C t To learn the encoded features from multiple perspectives, F t To integrate features that combine contextual and location information, h t Let be the knowledge state at time t. w is a learnable parameter l w is a learnable parameter f Let σ be a learnable parameter, and tan σ be the activation function. The circle represents the activation function, and ⊙ denotes element-wise multiplication. S53, the final knowledge state update formula is: ， Among them, h t+1 Let h be the knowledge state at time t+1. t Let KA be the knowledge state at time t. t Let μ represent the knowledge acquired at time t. f ⊙ represents the forgetting factor, and ⊙ denotes element-wise multiplication.

7. The knowledge tracking method for multi-granularity sparse attention according to claim 1, characterized in that, Step S6 includes the following steps: S61, Prediction: S611, Knowledge State Focus: ， in, Let Q be the knowledge state focused on by the exercise at time t+1, and let e be the knowledge point embedding matrix. t+1 For the exercise at time t+1, h t+1 This represents the knowledge state at time t+1; S612, Learning Performance Prediction: ， Among them, W p Here are the learnable parameters, σ is the activation function, and e is the activation parameter. t+1 The exercise is for time t+1. The knowledge state focused on by the exercise at time t+1, and the final output y t+1 This represents the accuracy of the prediction; S62, Loss Function: The training objective function optimizes the predicted student performance by minimizing the cross-entropy loss function, maximizing the match between the predicted probability and the true label for each sample. The specific formula is as follows: ， Where L is the loss function, y t+1 a represents the prediction accuracy at time t+1. t+1 The label represents the actual answer at time t+1, and n represents the predicted number of questions.

Images