A cold-start method for knowledge tracking based on cone attention and kernel bias

Abstract

The present invention relates to the field of knowledge tracking technology, and in particular to a knowledge tracking cold start method based on cone attention and kernel bias, comprising: obtaining answer record information of a target student; inputting the answer record information into a preset knowledge tracking model to predict the target student's next answer situation, wherein the knowledge tracking model is obtained based on training of a training set, and the training set includes the answer record information of several students; the knowledge tracking model is used to embed the answer record information, and then use cone attention to perform spatial mapping on the embedded representation, and then perform kernel bias attenuation processing on the output of the cone attention, and perform knowledge state enhancement through the processed embedded representation, thereby predicting the next answer situation. The present invention utilizes the cone attention mechanism and introduces kernel bias to capture the hierarchical relationship between knowledge points, and can provide more accurate knowledge state prediction under the conditions of short sequence data in cold start scenarios.

Description

Technical Field

[0001] The present invention relates to the field of knowledge tracing technology, and in particular to a knowledge tracing cold start method based on cone attention and kernel bias. Background Art

[0002] Knowledge tracking (KT) is a crucial component of intelligent education systems. Its purpose is to leverage students' past learning interaction data to predict their future performance. Driven by the rapid development of deep learning, various deep knowledge tracking (DLKT) models have emerged, including autoregressive deep sequential KT models, memory-enhanced KT models, attention-based KT models, and graph-based KT models.

[0003] Although the existing DLKT model shows good results, it still has the following problems when dealing with cold start scenarios (students have short sequence interaction data):

[0004] 1) Unable to adapt to changes in sequence length: In cold-start scenarios, when the model is trained on short sequences and applied to longer sequences, the prediction performance of the knowledge tracking model will drop significantly, which limits the application of the model in real-world scenarios.

[0005] 2) It is difficult to capture the complex relationships between knowledge concepts: In the cold start scenario, students' interaction data is limited. Existing knowledge tracking models mainly rely on the point product attention mechanism in Euclidean space, which makes it difficult to effectively model the hierarchical relationships between knowledge points.

[0006] To solve the above technical problems, the present invention proposes a knowledge tracking cold start method based on cone attention and kernel bias. Summary of the Invention

[0007] The purpose of this invention is to provide a knowledge tracking cold start method based on cone attention and kernel bias, which utilizes the cone attention mechanism and introduces kernel bias to better capture the hierarchical relationship between knowledge points, improve the generalization ability of the model on sequences of different lengths, and provide more accurate knowledge state prediction under short sequence data conditions.

[0008] To achieve the above object, the present invention provides the following solutions:

[0009] A knowledge tracking cold-start method based on cone attention and kernel bias, including:

[0010] Obtain the target student's answer record information;

[0011] The answer record information is input into a preset knowledge tracking model to predict the next answer situation of the target student, wherein the knowledge tracking model is obtained based on training of a training set, and the training set includes the answer record information of several students; the knowledge tracking model is used to embed the answer record information, and then use cone attention to spatially map the embedded representation, and then perform kernel bias attenuation processing on the output of the cone attention, and enhance the knowledge state through the processed embedded representation, so as to predict the next answer situation.

[0012] Optionally, the answer record information includes: the question, the knowledge points associated with the question, the answer result of the question, and the answer time of the question.

[0013] Optionally, the knowledge tracking model includes: an answer information embedding module, an attention processing module, and an answer status prediction module, wherein the answer information embedding module is used to obtain the embedded representation by processing the answer record information; the attention processing module is used to spatially map the embedded representation, calculate the attention score with kernel bias attenuation and perform knowledge state enhancement to obtain knowledge state output; the answer status prediction module is used to predict the next answer status through the knowledge state output.

[0014] Optionally, the answer information embedding module is specifically configured to process the answer record information to obtain the embedded representation, including:

[0015] Calculate the two-dimensional one-hot encoding of the knowledge points and the answers to the questions where the knowledge points are located in the answer record information;

[0016] Calculating the implicit representation of the knowledge point and the implicit representation of the answer to the question where the knowledge point is located based on the two-dimensional one-hot encoding;

[0017] By introducing the learnable problem difficulty factor and the embedding of the problem-related knowledge points into the implicit representation, the embedding representation of the knowledge points and the embedding representation of the answers to the questions where the knowledge points are located are calculated.

[0018] Optionally, the attention processing module includes: an attention mapping unit, an attention decay unit, and an attention output unit, wherein the attention mapping unit is used to map the embedded representation from the Euclidean space to the Poincare hemisphere space, and calculate the similarity attention score of the hyperbolic space; the attention decay unit is used to introduce a kernel bias decay matrix and a causal relationship mask matrix to obtain an attention score with kernel bias decay; the attention output unit is used to perform weighted aggregation on the attention score and the question answer to obtain the knowledge state output.

[0019] Optionally, the method for calculating the similarity attention score in the hyperbolic space is:

[0020]

[0021] Among them, A(q, k) is the similarity attention score, exp(·) is the exponential function, φ(q) d Indicates that the last dimension of the query vector q in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k) d The last dimension of the key vector k in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(q): d-1 Indicates that the first d-1 dimensions of the query vector q in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k): d-1 Indicates that the first d-1 dimensions of the key vector k in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, 7 is the scale transformation coefficient, r is the hyperbolic radius, and ||·|| represents the L2 norm distance.

[0022] Optionally, the method for obtaining the attention score with kernel bias decay is:

[0023]

[0024] Among them, A bias is the attention score, ψ is the softmax function, is the kernel bias attenuation matrix, is the causal relationship mask matrix.

[0025] Optionally, the method for obtaining the knowledge state output is:

[0026] h t+1 =A bias ·v;

[0027] Among them, h t+1 is the knowledge state output of the hidden layer in the attention mechanism, and V represents the value vector in the attention mechanism.

[0028] Optionally, the answer situation prediction module is specifically configured to predict the next answer situation based on the knowledge state output, including:

[0029] The knowledge state output is input into the fully connected neural network to calculate the predicted probability of the student's next answer under the condition of minimum binary cross entropy loss. The calculation method is:

[0030]

[0031] in, is the predicted probability of the student's next answer, σ is the Sigmoid function, ReLU is the ReLU function, W1 and W2 are the learnable parameter matrices in the fully connected neural network, and h t+1 is the knowledge state output of the hidden layer in the attention mechanism, x t+1 For the next question to be practiced, b1 and b2 are bias terms. is the binary cross entropy loss, r t+1 is the true probability of the student's next answer.

[0032] The beneficial effects of the present invention are:

[0033] The present invention effectively solves the cold start problem in the field of knowledge tracking by training on short sequence training set data in cold start scenarios and maintaining stable performance when predicting long sequences. It does not require additional auxiliary information or external data such as knowledge graphs, and can achieve better prediction results by relying only on limited interaction data. The present invention has strong sequence length generalization ability and proposes a kernel bias mechanism to enable the model to maintain stable performance on sequences of different lengths. The hierarchical relationship between knowledge points is modeled in hyperbolic space through the cone attention mechanism, which is more effective than the point product attention in traditional Euclidean space. It can automatically discover and utilize the implicit hierarchical structure between knowledge points, thereby improving prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 Schematic diagram of the knowledge tracking model structure and workflow according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] This embodiment provides a knowledge tracking cold start method based on cone attention and kernel bias, including:

[0039] Obtain the target student's answer record information;

[0040] The answer record information is input into a preset knowledge tracking model to predict the next answer situation of the target student, wherein the knowledge tracking model is obtained based on training of a training set, and the training set includes the answer record information of several students; the knowledge tracking model is used to embed the answer record information, and then use cone attention to spatially map the embedded representation, and then perform kernel bias attenuation processing on the output of the cone attention, and enhance the knowledge state through the processed embedded representation, so as to predict the next answer situation.

[0041] The answer record information includes: the question, the knowledge points associated with the question, the answer result of the question, and the answer time of the question.

[0042] The knowledge tracking model includes: an answer information embedding module, an attention processing module, and an answer status prediction module, wherein the answer information embedding module is used to obtain the embedded representation by processing the answer record information; the attention processing module is used to spatially map the embedded representation, calculate the attention score with kernel bias attenuation and perform knowledge state enhancement to obtain knowledge state output; the answer status prediction module is used to predict the next answer status through the knowledge state output.

[0043] Specifically, this embodiment effectively solves the cold start problem in the field of knowledge tracking by training on short sequence data and maintaining stable performance when predicting long sequences. It does not require additional auxiliary information or external data such as knowledge graphs, and can achieve good prediction results only by relying on limited interaction data. This embodiment has strong sequence length generalization capabilities and proposes a kernel bias mechanism to enable the model to maintain stable performance on sequences of different lengths. It uses a cone-shaped attention mechanism to model the hierarchical relationship between knowledge points in hyperbolic space, which is more effective than the point product attention in traditional Euclidean space. It can automatically discover and utilize the implicit hierarchical structure between knowledge points, thereby improving prediction accuracy.

[0044] The following combination Figure 1 The knowledge tracking model structure and workflow proposed in this embodiment are described in detail, including:

[0045] (1) Question answer information embedding module:

[0046] The answer information embedding module is specifically configured to process the answer record information to obtain the embedded representation, including:

[0047] Calculate the two-dimensional unique-hot encoding of several knowledge points in the answer record information and the answers to the questions where the knowledge points are located; calculate the implicit representation of the knowledge points and the implicit representation of the answers to the questions where the knowledge points are located based on the two-dimensional unique-hot encoding; and calculate the embedded representation of the knowledge points and the embedded representation of the answers to the questions where the knowledge points are located by introducing the embedding of the learnable problem difficulty factor and the problem-associated knowledge points in the implicit representation.

[0048] Specifically, this embodiment obtains each interaction record from the original data, which contains: the student answer sequence is {q, c, r, t}, where q is the question; c is the knowledge point associated with the question; r is the answer to the question, 1 indicates a correct answer, 0 indicates an incorrect answer; t is the time step of the answer. When a question contains multiple knowledge points, it is split into multiple knowledge point-level interaction records. First, the 2D one-hot encoding ec of the knowledge point and the answer to the question at time step t is calculated. t and Then calculate the implicit representation of the knowledge point and the implicit representation of the answer to the question By introducing the learnable problem difficulty factor Embedding of knowledge points associated with the question Calculate the embedded representation of the enhanced knowledge point and embedding representations containing question answers

[0049] (2) Attention processing module:

[0050] The attention processing module includes: an attention mapping unit, an attention decay unit, and an attention output unit, wherein the attention mapping unit is used to map the embedded representation from the Euclidean space to the Poincare hemisphere space and calculate the similarity attention score of the hyperbolic space; the attention decay unit is used to introduce the kernel bias decay matrix and the causal relationship mask matrix to obtain the attention score with kernel bias decay; the attention output unit is used to perform weighted aggregation on the attention score and the question answer to obtain the knowledge state output.

[0051] Specifically, in this embodiment, the attention mapping unit maps the embedding representation from the Euclidean space to the Poincare hemisphere space and calculates the similarity attention score in the hyperbolic space as follows:

[0052] First, the embedding representation of the student's answer sequence is used as the input of the attention mechanism. Specifically, the query vector q=x in the attention mechanism t+1 , key vector k={x1,…,x t} and value vector v = {y1, ...y t}. From Euclidean space to Poincare hemisphere space through mapping function φ, x d Represents the last dimension of vector x, X:d-1 The first d-1 dimensions of the pointing quantity x, the mapping rule is:

[0053] φ(x): d-1 =x: d-1 ·exp(x d ) φ(x) d =exp(x d );

[0054] Where exp(·) is an exponential function. Then, the query vector q and key vector k in the attention mechanism are mapped to the Poincare hemisphere space:

[0055] q=[φ(q): d-1 ,φ(q)d]

[0056] k=[φ(k): d-1 ,φ(k) d ];

[0057] Finally, the hierarchical perception similarity attention score A(q,k) in the hyperbolic space is calculated, that is:

[0058]

[0059] Among them, A(q,k) is the similarity attention score, exp(·) is the exponential function, φ(q) d Indicates that the last dimension of the query vector q in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k) d The last dimension of the key vector k in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(q): d-1 Indicates that the first d-1 dimensions of the query vector q in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k): d-1 Indicates that the first d-1 dimensions of the key vector k in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, γ is the scale transformation coefficient, r is the hyperbolic radius, and ||·|| represents the L2 norm distance.

[0060] Specifically, the attention decay unit in this embodiment introduces a kernel bias decay matrix and a causality mask matrix. The process of obtaining the attention score with kernel bias decay is as follows:

[0061] In hyperbolic space, the attention score of the self-attention mechanism is calculated as follows:

[0062] A bias =ψ(A(q, k));

[0063] Where ψ is the softmax function. By introducing the kernel bias attenuation matrix into the attention score and the causal relationship mask matrix Get the attention score A based on kernel bias decay bias :

[0064]

[0065] Among them, each element of the attenuation matrix is:

[0066]

[0067] Causality mask matrix Each element of is:

[0068]

[0069] Specifically, in this embodiment, the attention output unit is used to perform weighted aggregation on the attention score and the question answer to obtain the knowledge state output as follows:

[0070] The output of the cone attention mechanism and the interaction of the students are weightedly aggregated by matrix multiplication to obtain the implicit knowledge state output, as follows:

[0071] h t+1 =A bias ·v;

[0072] Among them, v represents the value vector in the attention mechanism, h t+1 is the output of the hidden layer in the attention mechanism, that is, the knowledge state output.

[0073] (3) Question answer prediction module:

[0074] The answering situation prediction module is used to predict the next answering situation through the knowledge state output, specifically including: inputting the knowledge state output into a fully connected neural network, and calculating the predicted probability of the student's next answering situation under the condition of minimum binary cross entropy loss.

[0075] Specifically, this embodiment uses a two-layer fully connected neural network to predict the student's answer situation, takes the output of the knowledge state enhancement module as the input of the fully connected neural network, and calculates the binary cross entropy loss Under the minimum condition, the predicted probability of students' answers for:

[0076]

[0077] in, is the predicted probability of the student's next answer, σ is the Sigmoid function, ReLU is the ReLU function, W1 and W2 are the learnable parameter matrices in the fully connected neural network, and ht+1 is the knowledge state output of the hidden layer in the attention mechanism, x t+1 For the next question to be practiced, b1 and b2 are bias terms. is the binary cross entropy loss, r t+1 is the true probability of the student's next answer.

[0078] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A knowledge tracking cold start method based on cone attention and kernel bias, characterized by: include: Obtain the target student's answer record information; Inputting the answer record information into a preset knowledge tracking model to predict the next answer of the target student, wherein the knowledge tracking model is trained based on a training set, and the training set includes the answer record information of several students; the knowledge tracking model is used to embed the answer record information, spatially map the embedded representation using cone attention, perform kernel bias attenuation processing on the output of the cone attention, and enhance the knowledge state through the processed embedded representation, thereby predicting the next answer; The knowledge tracking model includes: an answer information embedding module, an attention processing module, and an answer status prediction module, wherein the answer information embedding module is used to obtain the embedded representation by processing the answer record information; the attention processing module is used to perform spatial mapping on the embedded representation, calculate the attention score with kernel bias attenuation and perform knowledge state enhancement to obtain knowledge state output; the answer status prediction module is used to predict the next answer status based on the knowledge state output; The answer information embedding module is specifically configured to process the answer record information to obtain the embedded representation, including: Calculate the two-dimensional one-hot encoding of the knowledge points and the answers to the questions where the knowledge points are located in the answer record information; Calculating the implicit representation of the knowledge point and the implicit representation of the answer to the question where the knowledge point is located based on the two-dimensional one-hot encoding; By introducing the learnable problem difficulty factor and the embedding of the problem-related knowledge points into the implicit representation, the embedded representation of the knowledge points and the embedded representation of the answers to the questions where the knowledge points are located are calculated; The attention processing module includes: an attention mapping unit, an attention decay unit, and an attention output unit, wherein the attention mapping unit is used to map the embedding representation from the Euclidean space to the Poincare hemisphere space and calculate the similarity attention score of the hyperbolic space; the attention decay unit is used to introduce a kernel bias decay matrix and a causal relationship mask matrix to obtain an attention score with kernel bias decay; the attention output unit is used to perform weighted aggregation on the attention score and the question answer to obtain the knowledge state output.

2. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 1, characterized in that The answer record information includes: the question, the knowledge points associated with the question, the answer result of the question, and the answer time of the question.

3. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 1, characterized in that The method for calculating the similarity attention score in hyperbolic space is: Among them, A(q, k) is the similarity attention score, exp(·) is the exponential function, φ(q) d Indicates that the last dimension of the query vector q in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k) d Indicates that the last dimension of the key vector k in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(q) ：d-1 Indicates that the first d-1 dimensions of the query vector q in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k) :d-1 Indicates that the first d-1 dimensions of the key vector k in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, γ is the scale transformation coefficient, r is the hyperbolic radius, and ||·|| represents the L2 norm distance.

4. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 3, characterized in that The method for obtaining the attention score with kernel bias decay is: Among them, A bias is the attention score, ψ is the softmax function, is the kernel bias attenuation matrix, is the causal relationship mask matrix.

5. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 4, characterized in that The method for obtaining the knowledge state output is: h t+1 =A bias ·V; Among them, h t+1 is the knowledge state output of the hidden layer in the attention mechanism, and V represents the value vector in the attention mechanism.

6. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 1, characterized in that The answer situation prediction module is specifically used to predict the next answer situation based on the knowledge state output, including: The knowledge state output is input into the fully connected neural network to calculate the predicted probability of the student's next answer under the condition of minimum binary cross entropy loss. The calculation method is: in, is the predicted probability of the student's next answer, σ is the Sigmoid function, ReLU is the ReLU function, W1 and W2 are the learnable parameter matrices in the fully connected neural network, and h t+1 is the knowledge state output of the hidden layer in the attention mechanism, x t+1 For the next question to be practiced, b1 and b2 are bias terms. is the binary cross entropy loss, r t+1 is the true probability of the student's next answer.

Images