Knowledge tracing data augmentation method based on adaptive diffusion model

Abstract

The application discloses a kind of adaptive data enhancement methods based on diffusion model, comprising:1 constructs knowledge tracking network, by embedding layer student interactive triple is mapped into vector, and output the predicted probability of next question;2 construct conditional diffusion model, and by Markov conversion original interactive sequence is converted into noise sequence, combine the context condition extracted by bidirectional converter to generate positive / negative interactive sequence, to be discretized output by rounding step;3 construct contrast learning network, mix original sequence and generated sequence to construct enhanced view, based on question similarity and response consistency calculation adaptive weight to mine difficult sample;4 jointly optimize knowledge tracking loss, diffusion model loss and contrast loss, end-to-end training network.The application generates enhanced data in line with knowledge state dependence by conditional diffusion, and improves the robustness of model to sparse noise data by combining adaptive weighting strategy, can significantly improve prediction accuracy and representation quality.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to data augmentation methods in knowledge tracing (KT) technology. Background Technology

[0002] Knowledge tracing is a crucial task in educational data mining, aiming to predict future learning performance based on students' historical learning records. However, in practical applications, student data often exhibits sparsity and imbalance, severely limiting the performance of knowledge tracing models. To address this issue, data augmentation techniques are widely used to generate more diverse training data. However, most existing data augmentation methods rely on randomness, neglecting the importance of contextual information and knowledge state, leading to deviations between the generated data and the actual data distribution. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a knowledge tracking data augmentation method based on an adaptive diffusion model. This method aims to leverage the advantages of diffusion models in condition generation, combining contextual information and knowledge state to generate more reasonable and diverse augmented data. Simultaneously, it mines hard samples in contrastive learning, thereby improving the predictive performance of the knowledge tracking model for future learning capabilities based on historical interactions.

[0004] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0005] The adaptive data augmentation method based on a diffusion model of the present invention is characterized by the following steps:

[0006] Step 1: Define the student's historical interaction sequence X = {x1, x2, ..., x...} t ,…,x T}, where x t Let x be the triple at time t, and x t =(q t ,c t ,r t ), where q t This represents the problem at time t; and q t ∈Q; Q represents the problem set; c t Let c represent the concept at time t, and c t ∈C; C represents the concept set; r t Let r represent the answer at time t, and r t ∈{0, 1}, when r t When q = 1, it means q t The answer is correct, when r t When = 0, it means q t The answer is incorrect; T represents the total time.

[0007] Step 2: Construct a knowledge tracing network, including: a first embedding layer, a knowledge tracing model, and a prediction layer, and then process x... t After processing, we obtain the problem q at time t+1. t+1 Predicted probability Objective function for constructing knowledge tracing networks

[0008] Step 3: Construct a conditional diffusion model and apply it to x. a After processing, the positive interaction sequence X is obtained. + and negative interaction sequence X - Thus, the training objective of the conditional diffusion model is established.

[0009] Step 4: Construct a contrastive learning network, including a second embedding layer and an encoder, to predict the student's knowledge state at each time step; thereby constructing the contrastive learning loss function.

[0010] Step 5, based on and Construct the overall objective function for joint optimization This is used to train a tracking-learning network consisting of a knowledge tracking network, a conditional diffusion model, and a contrastive learning network, to obtain the optimal tracking-learning network, which is then used to process the input student triples to obtain prediction results and high-quality representations.

[0011] The adaptive data augmentation method based on a diffusion model described in this invention is also characterized in that step two includes the following steps:

[0012] Step 2.1, the first embedding layer will q t c t and r t Mapped to the problem embedding vector at time t respectively Concept embedding vector at time t The embedding vector of the answer at time t This generates the joint embedding vector of the problem and concept at time t. and interactive embedding vectors Among them, W q Embedding matrix W for randomly initialized problems c For randomly initialized concept embedding matrix, W r The answer embedding matrix is ​​randomly initialized, ⊕ represents the concatenation operation, W1 and W2 are the two weight matrices of the embedding layer, and b1 and b2 are the two bias vectors of the embedding layer;

[0013] Step 2.2, the knowledge tracing model for x tThe data is processed to obtain the student's knowledge characteristics h at time t. t ;

[0014] Step 2.3, the first embedding layer for x t The process is performed to obtain the original interactive embedding vector at time t. Thus, the original interactive embedding sequence is obtained.

[0015] Step 2.4: The prediction layer uses equation (1) to obtain q. t+1 Predicted probability

[0016]

[0017] In equation (1), f represents the problem embedding vector at time t+1; θ This represents a multilayer perceptron model; σ is the sigmoid function.

[0018] Step 2.5: Construct the objective function of the knowledge tracing network using equation (2).

[0019]

[0020] Furthermore, step three includes the following steps:

[0021] Step 3.1: Use equation (3) to evaluate x. a Perform masking to obtain the masked interactive embedding sequence.

[0022]

[0023] In equation (3), M = {m1, m2, ... m} t …m T} represents a randomly generated binary mask sequence, where m t Let m represent the state at time t, and m t ∈{0,1}, ⊙ represents element-wise multiplication;

[0024] Step 3.2, The input is processed by a bidirectional converter to obtain... Context conditions at time t

[0025] Concept embedding vector at time t As Conceptual conditions at time t

[0026] Step 3.3: Use equation (4) to obtain the context conditions for reversal at time t.

[0027]

[0028] Step 3.4: Use equation (5) to evaluate x. a Perform a Markov transformation to obtain the initial noise sequence. in, This represents the initial noise vector at time t;

[0029]

[0030] In equation (5), σ0 represents the standard deviation of Z0, I represents the unit covariance matrix with the same dimension as Z0, and φ represents the parameters of the forward noise addition process; q φ This represents the probability distribution of the forward noise addition process; Represents a normal distribution;

[0031] Step 3.5: Starting from t=1, use equation (6) to process the noise sequence Z at time t-1. t-1 Add T variances {β1,β2,…,β} respectively t ,…,β T Gaussian noise is used to obtain the noise sequence at time t. This allows us to iterate up to t=T and obtain the pure noise sequence at time T. Where, β t Let t represent the variance. This represents the noise vector at time t. This represents the pure noise vector at time t;

[0032]

[0033] In equation (6), Let represent the parameter that controls the addition of noise amplitude for the t-th control, and α s Let α represent the s-th hyperparameter, and let α be the hyperparameter. s =1-β s ,β s Let s represent the variance of the s-th variable. Indicates adding to Z T The t-th Gaussian white noise in Z follows a standard normal distribution. t-1 This represents the noise sequence at time t-1;

[0034] Step 3.6, in and Next to Z T Denoising is performed to obtain the initial conditional noise sequence. At the same time, and Next to Z T Denoising is performed to obtain the inverted initial conditional noise sequence. in, Let represent the initial conditional noise vector at time t. The initial conditional noise vector represents the reversal at time t;

[0035] Step 3.7: Use equation (11) to obtain the t-th positive cross-embedding vector. Thus, the positive interactive embedding sequence is obtained.

[0036]

[0037] In equation (11), W k b k W q or W c or W r The training parameters are denoted as k, which represents the class index of each element in the triplet, and softmax represents the activation function.

[0038] The t-th negative cross-embedding vector is obtained using equation (12). Thus, the negative interactive embedding sequence is obtained.

[0039]

[0040] Step 3.8 Construct the training loss function of the conditional diffusion model using equation (13)

[0041]

[0042] In equation (13), q(Z) t |Z0) represents the result of gradually adding noise to Z0 during the forward noise addition process to obtain Z. t The conditional probability distribution, Represents q(Z) t The expectation of |Z0), μ θ (Z t ,t) is the conditional diffusion model for Z t The predicted mean, || || 2 This indicates taking the square of the Euclidean distance. Indicates at Z0 and Z t Given the condition, the mean of the posterior distribution is obtained, and we have:

[0043]

[0044] In equation (14), This represents the parameter that controls the amplitude of the added noise at the (t-1)th time step.

[0045] Step 3.9 Construct the training objective of the conditional diffusion model using equation (15)

[0046]

[0047] In equation (15), Indicates the relationship from x a Noise is gradually added to generate Z0 to Z T The expectation, μ θ (Z1,1) is the denoised mean predicted by the diffusion model for the noise sequence Z1 at time 1, θ represents the parameters of the reverse denoising process, and p θ This represents the probability distribution of the reverse denoising process.

[0048] Furthermore, step 3.6 includes the following steps:

[0049] Step 3.6.1: Initialize t=T, initialize the conditional noise sequence Z at time t. t,1 =Z T And starting from t=T, using equation (7) in and Next to Z t,1 After denoising, the conditional noise sequence Z at time t-1 is obtained. t-1,1 This allows us to iterate up to t=1 and obtain the initial conditional noise sequence Z. 0,1 ;

[0050]

[0051] In equation (7), α t This represents the t-th hyperparameter. Is and Conditional diffusion model for Z t,1 The predicted mean; and we have:

[0052] In equation (8), Is The conditional diffusion model for the conditional noise Z at time t t,1 The predicted mean Is Conditional diffusion model for Z t,1 The predicted mean, μ θ (Z t,1 ,t) is the conditional diffusion model for Z without conditions. t,1 The predicted mean, s1 is the control The influence weights on the inverse denoising process, s2 is the control Weighting of the impact on the reverse denoising process;

[0053] Step 3.6.2: Initialize t=T, initialize the inverted conditional noise sequence Z at time t. t,2 =Z T And starting from t=T, using equation (9) in and Controlled Z t,2 After denoising, the conditional noise sequence Z, which is reversed at time t-1, is obtained. t-1,2 This allows us to iterate up to t=1 and obtain the inverted initial condition noise sequence Z. 0,2 ;

[0054]

[0055] In equation (9), Is and Conditional diffusion model for Z t,2 The predicted mean is:

[0056]

[0057] In equation (10), Is The conditional diffusion model for the inverted conditional noise sequence Z at time t t,2 The predicted mean Is Conditional diffusion model for Z t,2 The predicted mean, μ θ (Z t,2 ,t) is the conditional diffusion model for Z without conditions. t,2 The predicted mean, s3 is the control The influence weights on the inverse denoising process of the inversion, s4 is the control The weighting of the impact on the inverse denoising process of the inversion.

[0058] Further steps include the following:

[0059] Step 4.1 Obtain x using the mixture probability p a Two front views;

[0060] Let the first mixture probability be p1. When p = p1, let... When p = 1 - p1, let Thus, x is obtained a The first front view Where, x' t This represents the first positive interactive embedding vector or the original interactive embedding vector at time t under the first mixed probability.

[0061] Let the second mixture probability be p2. When p = p2, let... When p = 1 - p2, let Thus, x is obtained a The second front view Where, x″ t This represents the second positive interactive embedding vector or the original interactive embedding vector at time t under the second mixed probability.

[0062] X - As x a Negative view;

[0063] Step 4.2: Utilize the second embedding layer and encoder to... X - Processing is performed, and the corresponding results are obtained. First time features Second time feature X - The third time feature

[0064] Step 4.3, using equation (16) to obtain The mean cosine similarity qs of all problems within the total time T. 1,2 Thus, by using equation (17), we can obtain and The weight ω at total time T 1,2 :

[0065]

[0066] ω 1,2 =exp(-r 1,2 ·qs 1,2 ) / n 1,2 (17)

[0067] In equation (16), sim is the cosine similarity. express The problem embedding vector at time t, express The problem embedding vector at time t;

[0068] In equation (17), r 1,2 yes The ratio of the number of identical responses within a total time period T, n 1,2 Within the total time T, respectively at The average probability of answering correctly is given by exp, which is an exponential function.

[0069] Step 4.4: Using equation (18) to obtain With X -The mean cosine similarity qs of all problems within the total time T. 1,- Thus, by using equation (19), we can obtain the result in With X - The weight ω within the total time T 1,- :

[0070]

[0071] ω 1,- =exp(-1+r 1,- ·qs 1,- ) / n 1,- (19)

[0072] In equation (18), X represents - The problem embedding vector at time t;

[0073] In equation (19), r 1,- yes With X - The ratio of the number of identical responses within a total time period T, n 1,- Within the total time T, respectively at X - The average probability of answering correctly.

[0074] Step 4.5: Construct the contrastive learning loss function using equation (20).

[0075]

[0076] In equation (20), τ represents the temperature parameter, and <,> represents the cosine similarity.

[0077] Furthermore, step five includes the following steps:

[0078] Step 5.1 Construct the overall objective function for joint optimization using equation (21)

[0079]

[0080] In equation (21), λ1 and λ2 are two hyperparameters;

[0081] Step 5.2 The tracking-learning network, consisting of a knowledge tracking network, a conditional diffusion model, and a contrastive learning network, is trained end-to-end using gradient descent, and the results are calculated. To update network parameters until The process continues until convergence, thus yielding the optimal tracking-learning model.

[0082] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program supporting the processor in executing the knowledge tracing data augmentation method, and the processor is configured to execute the program stored in the memory.

[0083] The present invention provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, performs the steps of the knowledge tracing data augmentation method.

[0084] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0085] 1. This invention obtains context conditions by employing a bidirectional converter. Capture the temporal dependencies of historical answer sequences, and then introduce concept embedding conditions. By constraining the consistency between the generated data and the current learning concept, and finally dynamically adjusting the conditional weights s1 and s2, the generated data is ensured to conform to the law of knowledge state evolution. This improves the similarity between the data generated by this invention and the real data, thereby solving the problem of data generation deviating from the real distribution and enhancing the rationality of the teaching logic of augmented data.

[0086] 2. This invention reverses the context conditions. Adversarial negative samples are generated, and then an adaptive weighting mechanism is used to dynamically adjust the loss weights based on question similarity (qs) and response consistency (r): the weights of simple samples (ω) are reduced. 1,2 This amplifies the value of low-frequency, high-value samples ω 1,- Finally, by constructing a hybrid view, the exposure rate of difficult samples is increased. This improves the prediction accuracy of the model for low-frequency questions, thereby breaking through the bottleneck of difficult sample mining under sparse data and enhancing the model's ability to identify key learning bottlenecks.

[0087] 3. This invention utilizes a joint loss function. Integrated Knowledge Tracking Diffusion generation Comparative learning Three objective functions, along with a shared embedding layer between the diffusion model generator and the knowledge tracking network, ensure alignment between augmented data and the prediction task, enabling end-to-end training to improve the student's knowledge state representation h. t The information entropy is effectively improved, and the discriminative power of the representation space is enhanced, thereby unifying the end-to-end training of knowledge tracing and data augmentation, and improving the representation quality and prediction accuracy.

[0088] 4. This invention injects controllable noise into the original data through conditional diffusion Markov noise addition, and adds a negative view mechanism in contrastive learning to force the model to learn noise-invariant features, thereby enhancing the model's robustness to noisy data and reducing the risk of overfitting. Attached Figure Description

[0089] Figure 1 This is a diagram of the overall framework of DiffDAKT proposed in this invention. Detailed Implementation

[0090] In this embodiment, an adaptive data augmentation method based on a diffusion model includes the following steps:

[0091] Step 1: Define the student's historical interaction sequence X = {x1, x2, ..., x...} t ,…,x T}, where x t Let x be the triple at time t, and x t =(q t ,c t ,r t ), where q t This represents the problem at time t; and q t ∈Q; Q represents the problem set; c t Let c represent the concept at time t, and c t ∈C; C represents the concept set; r t Let r represent the answer at time t, and r t ∈{0, 1}, when r t When q = 1, it means q t The answer is correct, when r t When = 0, it means q t The answer is incorrect; T represents the total time.

[0092] Step 2: Construct a knowledge tracing network, including: a first embedding layer, a knowledge tracing model, and a prediction layer, and then process x... t After processing, we obtain the problem q at time t+1. t+1 Predicted probability Objective function for constructing knowledge tracing networks

[0093] Step 2.1: The first embedding layer will embed q t c t and r t Mapped to the problem embedding vector at time t respectively Concept embedding vector at time t The embedding vector of the answer at time t This generates the joint embedding vector of the problem and concept at time t. and interactive embedding vectors Among them, W q Embedding matrix W for randomly initialized problems c For randomly initialized concept embedding matrix, W r The embedding matrix is ​​initialized randomly, ⊕ denotes the concatenation operation, W1 and W2 are the two weight matrices of the embedding layer, and b1 and b2 are the two bias vectors of the embedding layer; q t (Question), c t (concept), r t The embedding vector of (the answer) is obtained through a randomly initialized matrix (W). q W c W r Mapping to a low-dimensional space facilitates model processing. Embedded representations can capture the semantic relationship between a problem and a concept (such as the mathematical relationship between "Pythagorean theorem" and "trigonometric functions"), improving the accuracy of subsequent knowledge state modeling.

[0094] Step 2.2, Knowledge Trace (KT) model for x t The data is processed to obtain the student's knowledge characteristics h at time t. t ;

[0095] Step 2.3, the first embedding layer for x t The process is performed to obtain the original interactive embedding vector at time t. Thus, the original interactive embedding sequence is obtained.

[0096] Step 2.4: The prediction layer uses equation (1) to obtain q. t+1 Predicted probability

[0097]

[0098] In equation (1), f represents the problem embedding vector at time t+1; θ This represents a multilayer perceptron model; σ is the sigmoid function.

[0099] Step 2.5: Construct the objective function of the knowledge tracing network using equation (2).

[0100]

[0101] Step 3: Construct a conditional diffusion model and apply it to x. a After processing, the positive interaction sequence X is obtained. + and negative interaction sequence X - ;

[0102] Step 3.1: Use equation (3) to evaluate x. aPerform masking to obtain the masked interactive embedding sequence.

[0103]

[0104] In equation (3), M = {m1, m2, ... m} t …m T} represents a randomly generated binary mask sequence, where m t Let m represent the state at time t, and m t ∈{0,1}, ⊙ represents element-wise multiplication; in this implementation, T=1000.

[0105] Step 3.2, The input is processed by a bidirectional converter to obtain... Context conditions at time t

[0106] Concept embedding vector at time t As Conceptual conditions at time t

[0107] Step 3.3: Use equation (4) to obtain the context conditions for reversal at time t.

[0108]

[0109] Step 3.4: Use equation (5) to evaluate x. a Perform a Markov transformation to obtain the initial noise sequence.

[0110]

[0111] In equation (5), σ0 represents the standard deviation of Z0. In this embodiment, σ0 = 0.01, I represents the unit covariance matrix with the same dimension as Z0, and φ represents the parameters of the forward noise addition process; q φ This represents the probability distribution of the forward noise addition process; This indicates a normal distribution.

[0112] Step 3.5: Starting from t=1, use equation (6) to process the noise sequence Z at time t-1. t-1 Add T variances {β1,β2,…,β} respectively t ,…,β T Gaussian noise is used to obtain the noise sequence at time t. This allows us to iterate up to t=T and obtain the pure noise sequence at time T. Where, β t This represents the t-th variance;

[0113]

[0114] In equation (6), Let represent the parameter that controls the addition of noise amplitude for the t-th control, and α s Let α represent the s-th hyperparameter, and let α be the hyperparameter. s =1-β s ,β s Let s represent the variance of the s-th variable. Indicates adding to Z T The t-th Gaussian white noise in Z follows a standard normal distribution. t-1 This represents the pure noise sequence at time t-1.

[0115] Step 3.6, in and Next to Z T Denoising is performed to obtain the initial conditional noise sequence. At the same time, and Next to Z T Denoising is performed to obtain the inverted initial conditional noise sequence.

[0116] Step 3.6.1: Initialize t=T, initialize the conditional noise sequence Z at time t. t,1 =Z T And starting from t=T, using equation (7) in and Next to Z t,1 After denoising, the conditional noise sequence Z at time t-1 is obtained. t-1,1 This allows us to iterate up to t=1 and obtain the initial conditional noise sequence Z. 0,1 ;

[0117]

[0118] In equation (7), α t This represents the t-th hyperparameter. Is and Conditional diffusion model for Z t,1 The predicted mean; and we have:

[0119] In equation (8), Is The conditional diffusion model for the conditional noise Z at time t t,1 The predicted mean Is Conditional diffusion model for Z t,1 The predicted mean, μθ (Z t,1 ,t) is the conditional diffusion model for Z without conditions. t,1 The predicted mean, s1 is the control The influence weights on the inverse denoising process, s2 is the control The influence weights on the reverse denoising process are determined; the denoising process employs a classifier-free approach, and the intensity of the conditional influence is controlled by adjusting weights s1 and s2. For example, increasing s1 results in generated data that better fits the historical context.

[0120] Step 3.6.2: Initialize t=T, initialize the inverted conditional noise sequence Z at time t. t,2 =Z T And starting from t=T, using equation (9) in and Controlled Z t,2 After denoising, the conditional noise sequence Z, which is reversed at time t-1, is obtained. t-1,2 This allows us to iterate up to t=1 and obtain the inverted initial condition noise sequence Z. 0,2 ;

[0121]

[0122] In equation (9), Is and Conditional diffusion model for Z t,2 The predicted mean is:

[0123]

[0124] In equation (10), Is The conditional diffusion model for the inverted conditional noise sequence Z at time t t,2 The predicted mean Is Conditional diffusion model for Z t,2 The predicted mean, μ θ (Z t,2 ,t) is the conditional diffusion model for Z without conditions. t,2 The predicted mean, s3 is the control The influence weights on the inverse denoising process of the inversion, s4 is the control The weighting of the impact on the inverse denoising process of the inversion.

[0125] Step 3.7: Use equation (11) to obtain the t-th positive cross-embedding vector. Thus, the positive interactive embedding sequence is obtained.

[0126]

[0127] In equation (11), W k b k Trainable parameters representing the question / concept / answer embedding matrix. Z represents 0,1 The noise vector at time t in the vector is represented by softmax, where softmax represents the softmax distribution.

[0128] The t-th negative cross-embedding vector is obtained using equation (12). Thus, the negative interactive embedding sequence is obtained.

[0129]

[0130] In equation (12), Z represents 0,2 The noise vector at time t in the data.

[0131] Step 3.8 Construct the training loss function of the conditional diffusion model using equation (13)

[0132]

[0133] In equation (13), q(Z) t |Z0) represents the result of gradually adding noise to Z0 during the forward noise addition process to obtain Z. t The conditional probability distribution, Represents q(Z) t The expectation of |Z0), μ θ (Z t ,t) is the conditional diffusion model for Z t The predicted mean, || || 2 This indicates taking the square of the Euclidean distance. Indicates at Z0 and Z t Given the condition, the mean of the posterior distribution is obtained, and we have:

[0134]

[0135] In equation (14), This represents the parameter that controls the amplitude of the added noise at the (t-1)th time step.

[0136] Step 3.9 Construct the training objective of the conditional diffusion model using equation (15)

[0137]

[0138] In equation (15), Indicates the relationship from x a Noise is gradually added to generate Z0 to ZT The expectation, μ θ (Z1,1) is the denoised mean predicted by the diffusion model for the noise sequence Z1 at time 1, θ represents the parameters of the reverse denoising process, and p θ This represents the probability distribution of the reverse denoising process.

[0139] Step 4: Construct a contrastive learning network, including a second embedding layer and an encoder, to predict the student's knowledge state at each time step;

[0140] Step 4.1 Obtain x using the mixture probability p a Two front views;

[0141] Let the first mixture probability be p1. When p = p1, let... When p = 1 - p1, let Thus, x is obtained a The first front view Where, x' t This represents the positive interactive embedding vector or the original interactive embedding vector at time t; the original sequence and the diffusion-generated sequence are mixed in the positive samples, and the effect is best when the mixing probability p = 0.4.

[0142] Let the second mixture probability be p2. When p = p2, let... When p = 1 - p2, let Thus, x is obtained a The second front view Where, x″ t Represents the positive interactive embedding vector or the original interactive embedding vector at time t;

[0143] X - As x a Negative view;

[0144] Step 4.2: Utilize the second embedding layer and encoder to... X - Processing is performed, and the corresponding results are obtained. First time features Second time feature X - The third time feature

[0145] Step 4.3, using equation (16) to obtain The mean cosine similarity qs of all problems within the total time T. 1,2 Thus, by using equation (17), we can obtain and The weight ω at total time T 1,2 :

[0146]

[0147] ω 1,2 =exp(-r 1,2 ·qs 1,2 ) / n 1,2 (17)

[0148] In equation (16), sim is the cosine similarity. express The problem embedding vector at time t, express The problem embedding vector at time t;

[0149] In equation (17), r 1,2 yes The ratio of the number of identical responses within a total time period T, n 1,2 Within the total time T, respectively at The average probability of the answer being 1 is given by exp, which is an exponential function.

[0150] Step 4.4: Using equation (18) to obtain With X - The mean cosine similarity qs of all problems within the total time T. 1,- Thus, by using equation (19), we can obtain the result in With X - The weight ω within the total time T 1,- :

[0151]

[0152] ω 1,- =exp(-1+r 1,- ·qs 1,- ) / n 1,- (19)

[0153] In equation (18), X represents - The problem embedding vector at time t;

[0154] In equation (19), r 1,- yes With X - The ratio of the number of identical responses within a total time period T, n 1,- Within the total time T, respectively at

[0155] X - The average probability of the answer being 1.

[0156] Step 4.5: Construct the contrastive learning loss function using equation (20).

[0157]

[0158] In equation (20), τ represents the temperature parameter. In this embodiment, τ = 0.07, and <,> represents the cosine similarity. The adaptive weight dynamically adjusts the loss weight ω based on the similarity between the problem and the response. 1,- We should focus on low-frequency but important samples (such as questions that students repeatedly get wrong).

[0159] Step 5: Train the optimal tracking network;

[0160] Step 5.1 Construct the overall objective function for joint optimization using equation (21)

[0161]

[0162] In equation (21), λ1 and λ2 are two hyperparameters; in this embodiment, the optimal balance is achieved when λ1 = 0.1 and λ2 = 0.3.

[0163] Step 5.2 The tracking-learning network, consisting of a knowledge tracking network, a conditional diffusion model, and a contrastive learning network, is trained end-to-end using gradient descent, and the results are calculated. To update network parameters until The process continues until convergence, yielding the optimal tracking-learning network, which processes the input student triples to obtain prediction results. It can also generate augmented data X + and X - and high-quality feature representation h t .

[0164] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0165] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

[0166] Experimental example:

[0167] In this embodiment, four diverse real-world datasets were used to evaluate the effectiveness of DiffCl4KT, including algebra05, assistant09, assistant12, and slepemapy. Students with fewer than 5 interactions were excluded. Since a problem may be labeled with multiple skills, each unique combination of skills was converted into a new skill, and duplicate interactions caused by multiple skills were removed. To verify that the framework proposed in this invention can significantly enhance the performance of different knowledge tracing (KT) models, in the specific implementation, not only were three representative KT models selected as backbone models, including DKT, AKT, and LPKT, but it was also compared with the current state-of-the-art (SOTA) KT models, including DKT converter and QIKT (Chen et al., 2023).

[0168] The maximum length of the input sequence for the knowledge tracing model was set to 200, and DKT, AKT, and LPKT were implemented. An enhancement method of CL4KT was implemented within the contrastive learning framework to support contrast. The embedding dimension and hidden layer dimension of all models were fixed at d = 128. In this embodiment, a bidirectional transformer with one layer and two attention heads was used. The total number of steps for the diffusion model was fixed at 1000. The loss weights λ1 and λ2 for the contrastive learning and diffusion models were searched from [0.05, 0.1, 0.2, 0.4, 0.6]. The learning rate, bootstrapping scale, and replacement ratio p were searched from [5e-4, 1e-3, 2e-3, 5e-3, 1e-2], [1, 2, 3, 4], and [0.1, 0.2, 0.4, 0.6, 0.8], respectively.

[0169] To verify the effectiveness of the present invention framework, the performance of the present invention method was compared on three KT models and their baseline models. As shown in Table 1, the overall prediction performance of the present invention method, all baseline models, and several state-of-the-art (SOTA) models on four real-world datasets is compared. The best results are shown in bold.

[0170] Table 1:

[0171]

[0172] In Table 1, Datasets indicates the names of the datasets used, including algebra05, assistant09, assistant12, and slepemapy.

[0173] Metric: This refers to the performance evaluation metrics, which include two metrics: AUC and ACC. The performance evaluation metrics are AUC (Area Under the Curve) and ACC (Accuracy).

[0174] DTransformer: Applied to knowledge tracing tasks, it comes from the DeepKnowledge Tracing with Transformers released by Scott P. Pu in June 2020;

[0175] DKT: DKT pioneered the application of deep learning in the field of knowledge tracing, originating from DeepKnowledge Tracing released in 2015;

[0176] AKT: The first model in the field of knowledge tracing to simultaneously achieve high predictive performance and strong interpretability, derived from Context-Aware Attentive Knowledge Tracing released in 2020;

[0177] LPKT: Became the first knowledge tracing model to explicitly incorporate learning process consistency, originating from Learning Process-consistent Knowledge Tracing released in 2021;

[0178] As shown in Table 1: First, it is evident that the diffusion-based data augmentation model of this invention delivers significant performance improvements across all datasets. Second, this invention observes that applying the CL4KT augmentation method to the KT model within the framework of this invention also improves performance. However, the method of this invention shows even greater improvement, further demonstrating the effectiveness of this invention in data augmentation methods. Third, this invention compares its method with state-of-the-art models (e.g., DTransformer and QIKT). The method of this invention outperforms these models on all four datasets, highlighting its superiority in learning high-quality representations. Unlike methods that modify the KT model, the method of this invention exhibits greater flexibility and scalability, adapting to various KT models.

[0179] This invention achieves targeted data augmentation and hard sample mining by fusing diffusion models and contrastive learning, and its superiority has been verified in multiple KT tasks (e.g. Figure 1 (As shown in Table 1), this provides an innovative solution to the problem of sparsity in educational data.

[0180] Table 2: Results of the ablation experiment.

[0181]

[0182] In Table 2, Datasets indicates the names of the datasets used, including algebra05, assistant09, assistant12, and slepemapy.

[0183] Metric: This represents the performance evaluation metric. There are two metrics here: AUC and ACC. The values ​​for AUC and ACC are listed separately for each dataset.

[0184] DiffDAKT: A complete adaptive data augmentation method based on diffusion models;

[0185] w / o HN: A DiffDAKT variant after removing HN (which may be a specific component or module, depending on the context of the original text);

[0186] w / o KC: A DiffDAKT variant after removing KC (which may be related to knowledge components, etc.);

[0187] w / o Context: A DiffDAKT variant with context information removed;

[0188] w / o RW: A DiffDAKT variant after removing RW (which may be some weight or rule-related part);

[0189] w / o CL: A DiffDAKT variant after removing CL (which may be related to contrastive learning, etc.);

[0190] Table 2 shows the performance comparison of the diffusion-based adaptive data augmentation method (DiffDAKT) and its different variants (versions after removing different components) on different datasets (algebra05, assistant09, assistant12, slepemapy).

Claims

1. An adaptive data augmentation method based on a diffusion model, characterized in that, Includes the following steps: Step 1: Define the student's historical interaction sequence X = {x1, x2, ..., x...} t ,…,x T },in, Let represent the triple at time t, and ,in, This refers to the problem at time t; and ∈Q; Q represents the problem set; This represents the concept of time t, and ∈C; C represents the concept set; Denotes the answer at time t, and ∈{0, 1}, when r t When =1, it means The answer is correct, when =0 indicates The answer is incorrect; T represents the total time. Step 2: Construct a knowledge tracing network, including: a first embedding layer, a knowledge tracing model, and a prediction layer, and then... Processing yields the problem at time t+1. Predicted probability The objective function used to construct a knowledge tracing network ; Step 3: Construct a conditional diffusion model and analyze it. Processing yields a positive interaction sequence. and negative interaction sequence Thus, the training objective of the conditional diffusion model is established. ; Step 3.1, using equation (3) to... Perform masking to obtain the masked interactive embedding sequence. : (3) In equation (3), M = Represents a randomly generated binary mask sequence, where, Describes the state at time t, and ∈{0,1}, This indicates element-wise multiplication; Step 3.2, The input is processed by a bidirectional converter to obtain... exist Contextual conditions at any time ; Concept embedding vector at time t As exist The concept of time and its conditions ; Step 3.3, using equation (4) to obtain Contextual conditions for time reversal : (4) Step 3.4, using equation (5) to... Perform a Markov transformation to obtain the initial noise sequence. Z 0= ,in, This represents the initial noise vector at time t; (5) In equation (5), express standard deviation Indicates and Having unit covariance matrices of the same dimension Parameters representing the forward noise addition process; This represents the probability distribution of the forward noise addition process; Represents a normal distribution; Step 3.5: Starting from t=1, use equation (6) to process the noise sequence at time t-1. Add separately T variance Gaussian noise is used to obtain the noise sequence at time t. This allows us to iterate up to t=T and obtain the pure noise sequence at time T. ,in, Let t represent the variance. This represents the noise vector at time t. This represents the pure noise vector at time t; (6) In equation (6), Indicates the first A parameter that controls the added noise amplitude, and , Indicates the first One hyperparameter, and , Indicates the first One variance, Indicates adding to Z T The first in It contains Gaussian white noise that follows a standard normal distribution. This represents the noise sequence at time t-1; Step 3.6, in and Below Denoising is performed to obtain the initial conditional noise sequence. At the same time, and Below Denoising is performed to obtain the inverted initial conditional noise sequence. ;in, Let represent the initial conditional noise vector at time t. The initial conditional noise vector represents the reversal at time t; Step 3.7: Use equation (11) to obtain the t-th positive cross-embedding vector. Thus, the positive interactive embedding sequence is obtained. ={ }; (11) In equation (11), , express or or The training parameters are denoted as k, which represents the class index of each element in the triplet, and softmax represents the activation function. The t-th negative cross-embedding vector is obtained using equation (12). Thus, the negative interactive embedding sequence is obtained. ={ }; (12) Step 3.8 Construct the training loss function of the conditional diffusion model using equation (13). ; (13) In equation (13), Indicates the process of adding noise in the forward direction. Z 0 gradually adds noise to obtain The conditional probability distribution, express Expectations It is a conditional diffusion model for The predicted mean This indicates taking the square of the Euclidean distance. Indicates in and Given the condition, the mean of the posterior distribution is obtained, and we have: (14) In equation (14), This represents the parameter that controls the amplitude of the added noise at the (t-1)th time step. Step 3.9 Construct the training objective of the conditional diffusion model using equation (15) : (15) In equation (15), Indicates to from Gradually add noise generation arrive Expectations It is the noise sequence at time 1 of the diffusion model. Predicted denoised mean The parameters represent the inverse denoising process. This represents the probability distribution of the reverse denoising process; Step 4: Construct a contrastive learning network, including a second embedding layer and an encoder, to predict the student's knowledge state at each time step; thereby constructing the contrastive learning loss function. cl ; Step 5, based on , and Construct the overall objective function for joint optimization. It is used to train a tracking-learning network consisting of a knowledge tracking network, a conditional diffusion model, and a contrastive learning network, to obtain the optimal tracking-learning network, which is used to process the input student triples to obtain prediction results and high-quality representations.

2. The adaptive data augmentation method based on a diffusion model according to claim 1, characterized in that, Step two includes the following steps: Step 2.1, the first embedding layer will , and Mapped to the problem embedding vector at time t respectively Concept embedding vector at time t The embedding vector of the answer at time t This generates the joint embedding vector of the problem and concept at time t. and interactive embedding vectors ,in, Embedding matrices for randomly initialized problems Embedding matrix for randomly initialized concepts The answer embedding matrix is ​​randomly initialized, and ⊕ denotes the concatenation operation. and These are the two weight matrices of the embedding layer. and These are the two bias vectors of the embedding layer; Step 2.2, the knowledge tracing model for x t The data is processed to obtain the student's knowledge characteristics at time t. ; Step 2.3, the first embedding layer for x t The process is performed to obtain the original interactive embedding vector at time t. Thus, the original interactive embedding sequence is obtained. ={ , ,…, ,…, }; Step 2.4: The prediction layer obtains the result using equation (1). Predicted probability : (1) In equation (1), This represents the problem embedding vector at time t+1; Represents a multilayer perceptron model; It is the Sigmoid function; Step 2.5: Construct the objective function of the knowledge tracing network using equation (2). : (2)。 3. The adaptive data augmentation method based on a diffusion model according to claim 2, characterized in that, Step 3.6 includes the following steps: Step 3.6.1: Initialize t=T, initialize the conditional noise sequence at time t. And starting from t=T, using equation (7) in and Below Denoising is performed to obtain the conditional noise sequence at time t-1. This allows us to iterate up to t=1 and obtain the initial conditional noise sequence Z. 0,1 ; (7) In equation (7), This represents the t-th hyperparameter. Is and Conditional diffusion model The predicted mean; and we have: (8) In equation (8), Is Conditional diffusion model for conditional noise at time t The predicted mean Is Conditional diffusion model The predicted mean It is an unconditional diffusion model for... The predicted mean It is control The weighting of the impact on the reverse denoising process It is control Weighting of the impact on the reverse denoising process; Step 3.6.2: Initialize t=T, initialize the conditional noise sequence that is reversed at time t. And starting from t=T, using equation (9) in and Under control After denoising, the conditional noise sequence inverted at time t-1 is obtained. This allows us to iterate up to t=1 and obtain the reversed initial condition noise sequence Z. 0,2 ; (9) In equation (9), Is and Conditional diffusion model The predicted mean is: (10) In equation (10), Is The conditional diffusion model for the inverted conditional noise sequence at time t The predicted mean Is Conditional diffusion model The predicted mean It is an unconditional diffusion model for... The predicted mean It is control The weighting of the impact on the inverse denoising process. It is control The weighting of the impact on the inverse denoising process of the inversion.

4. The adaptive data augmentation method based on a diffusion model according to claim 3, characterized in that, Includes the following steps: Step 4.1 Obtain the mixture probability p Two front views; Let the first mixture probability be When p= season = When p= season = Thus obtain The first front view ;in, This represents the first positive interactive embedding vector or the original interactive embedding vector at time t under the first mixed probability. Let the second mixture probability be When p= season = When p= season = Thus obtain The second front view ;in, This represents the second positive interactive embedding vector or the original interactive embedding vector at time t under the second mixed probability. Will As Negative view; Step 4.2: Utilize the second embedding layer and encoder to... , , Processing is performed, and the corresponding results are obtained. First time features , Second time feature , The third time feature ; Step 4.3, using equation (16) to obtain , The mean cosine similarity of all problems within the total time T Thus, by using equation (17), we can obtain and Weight at total time T : (16) (17) In equation (16), sim is the cosine similarity. express The problem embedding vector at time t, express The problem embedding vector at time t; In equation (17), yes , The ratio of identical responses within a total time period T. Within the total time T, respectively at , The average probability of answering correctly. It is an exponential function; Step 4.4: Using equation (18) to obtain and The mean cosine similarity of all problems within the total time T Thus, by using equation (19), we can obtain the result in and Weights within the total time T : (18) (19) In equation (18), express The problem embedding vector at time t; In equation (19), yes and The ratio of identical responses within a total time period T. Within the total time T, respectively at , The average probability of answering correctly. Step 4.5: Construct the contrastive learning loss function using equation (20). cl : (20) In equation (20), This represents the temperature parameter. This represents the cosine similarity.

5. The adaptive data augmentation method based on a diffusion model according to claim 4, characterized in that, Step five includes the following steps: Step 5.1 Construct the overall objective function for joint optimization using equation (21). : (21) In equation (21), , There are two hyperparameters; Step 5.2 Train the tracking-learning network (comprising the knowledge tracking network, the conditional diffusion model, and the contrastive learning network) end-to-end using gradient descent, and calculate... To update network parameters until The process continues until convergence, thus yielding the optimal tracking-learning model.

6. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing the adaptive data augmentation method according to any one of claims 1-5, the processor being configured to execute the program stored in the memory.

7. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the steps of the adaptive data augmentation method according to any one of claims 1-5.

Images