Knowledge tracking cold start method based on cone attention and kernel bias

Abstract

The invention relates to the technical field of knowledge tracking, in particular to a knowledge tracking cold start method based on cone attention and kernel bias, and the method comprises the steps: obtaining the answer record information of a target student; the answer record information is input into a preset knowledge tracking model, the next answer condition of the target student is predicted, the knowledge tracking model is obtained based on training of a training set, and the training set comprises answer record information of a plurality of students; the knowledge tracking model is used for performing embedded representation on the answer record information, performing spatial mapping on the embedded representation by adopting conical attention, performing kernel bias attenuation processing on the output of the conical attention, performing knowledge state enhancement through the processed embedded representation, and further predicting the next answer condition. According to the method, the hierarchical relationship between knowledge points is captured by using a conical attention mechanism and introducing kernel bias, so that more accurate knowledge state prediction can be provided under the condition of short sequence data of a cold start scene.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge tracing, and in particular to a knowledge tracing cold start method based on cone attention and kernel bias. Background Art

[0002] Knowledge tracking (KT) is an important part of the intelligent education system. Its purpose is to use students' past learning interaction data to predict their future answer performance. Driven by the rapid development of deep learning, various deep knowledge tracking (DLKT) models have emerged, such as: autoregression-based deep sequence KT model, memory-enhanced KT model, attention-based KT model, and graph-based KT model.

[0003] Although the existing DLKT model shows good results, it still has the following problems when dealing with cold start scenarios (students have short sequence interaction data):

[0004] 1) Unable to adapt to changes in sequence length: In the cold start scenario, when the model is trained on short sequences and applied to longer sequences, the prediction performance of the knowledge tracking model will drop significantly. This limits the application of the model in practical scenarios.

[0005] 2) It is difficult to capture the complex relationship between knowledge concepts: In the cold start scenario, students’ interaction data is limited, and the existing knowledge tracking model mainly relies on the point product attention mechanism in Euclidean space, which makes it difficult to effectively model the hierarchical relationship between knowledge points.

[0006] To solve the above technical problems, the present invention proposes a knowledge tracking cold start method based on cone attention and kernel bias. Summary of the invention

[0007] The purpose of the present invention is to provide a knowledge tracking cold start method based on cone attention and kernel bias, which utilizes the cone attention mechanism and introduces kernel bias to better capture the hierarchical relationship between knowledge points, improve the generalization ability of the model on sequences of different lengths, and provide more accurate knowledge state prediction under short sequence data conditions.

[0008] To achieve the above object, the present invention provides the following solutions:

[0009] A knowledge tracking cold start method based on cone attention and kernel bias, including:

[0010] Obtain the answer record information of the target students;

[0011] The answer record information is input into a preset knowledge tracking model to predict the next answer situation of the target student, wherein the knowledge tracking model is trained based on a training set, and the training set includes the answer record information of several students; the knowledge tracking model is used to embed the answer record information, and then use cone attention to spatially map the embedded representation, and then perform kernel bias attenuation processing on the output of the cone attention, and enhance the knowledge state through the processed embedded representation, so as to predict the next answer situation.

[0012] Optionally, the answer record information includes: the question, the knowledge points associated with the question, the answer result of the question, and the answer time of the question.

[0013] Optionally, the knowledge tracking model includes: an answer information embedding module, an attention processing module, and an answer status prediction module, wherein the answer information embedding module is used to obtain the embedded representation by processing the answer record information; the attention processing module is used to spatially map the embedded representation, calculate the attention score with kernel bias attenuation and perform knowledge state enhancement to obtain knowledge state output; the answer status prediction module is used to predict the next answer status through the knowledge state output.

[0014] Optionally, the answer information embedding module is specifically configured to process the answer record information to obtain the embedded representation, including:

[0015] Calculate the two-dimensional unique hot codes of the several knowledge points and the answers to the questions where the knowledge points are located in the answer record information;

[0016] Calculating the implicit representation of the knowledge point and the implicit representation of the answer to the question where the knowledge point is located based on the two-dimensional one-hot encoding;

[0017] By introducing the learnable problem difficulty factor and the embedding of the problem-related knowledge points into the implicit representation, the embedded representation of the knowledge points and the embedded representation of the answer to the question where the knowledge points are located are calculated.

[0018] Optionally, the attention processing module includes: an attention mapping unit, an attention decay unit, and an attention output unit, wherein the attention mapping unit is used to map the embedded representation from the Euclidean space to the Poincare hemisphere space, and calculate the similarity attention score of the hyperbolic space; the attention decay unit is used to introduce a kernel bias decay matrix and a causal relationship mask matrix to obtain an attention score with kernel bias decay; the attention output unit is used to perform weighted aggregation on the attention score and the question answer to obtain the knowledge state output.

[0019] Optionally, the method for calculating the similarity attention score in the hyperbolic space is:

[0020]

[0021] Among them, A(q, k) is the similarity attention score, exp(·) is the exponential function, φ(q) d Indicates that the last dimension of the query vector q in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k) d Represents that the last dimension of the key vector k in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(q): d-1 It represents that the first d-1 dimensions of the query vector q in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k): d-1 It indicates that the first d-1 dimensions of the key vector k in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, 7 is the scale transformation coefficient, r is the hyperbolic radius, and ||·|| represents the L2 norm distance.

[0022] Optionally, the method for obtaining the attention score with kernel bias decay is:

[0023]

[0024] Among them, A bias is the attention score, ψ is the softmax function, is the kernel bias attenuation matrix, is the causal relationship mask matrix.

[0025] Optionally, the method for obtaining the knowledge state output is:

[0026] h t+1 =A bias ·v;

[0027] Among them, h t+1 is the knowledge state output of the hidden layer in the attention mechanism, and V represents the value vector in the attention mechanism.

[0028] Optionally, the answer situation prediction module is specifically used to predict the next answer situation through the knowledge state output, including:

[0029] The knowledge state output is input into the fully connected neural network to calculate the predicted probability of the student's next answer under the condition of minimum binary cross entropy loss. The calculation method is:

[0030]

[0031] in, is the predicted probability of the student's next answer, σ is the Sigmoid function, ReLU is the ReLU function, W1 and W 2 is the learnable parameter matrix in the fully connected neural network, h t+1 is the knowledge state output of the hidden layer in the attention mechanism, x t+1 For the next exercise, b 1 and b 2 is the bias term, is the binary cross entropy loss, r t+1 is the actual probability of the student's next answer.

[0032] The beneficial effects of the present invention are:

[0033] The present invention effectively solves the cold start problem in the field of knowledge tracking by training on short sequence training set data in a cold start scenario and maintaining stable performance when predicting long sequences. It does not require additional auxiliary information or external data such as knowledge graphs, and can achieve better prediction results by relying only on limited interaction data. The present invention has strong generalization ability for sequence length and proposes a kernel bias mechanism, which enables the model to maintain stable performance on sequences of different lengths. The hierarchical relationship between knowledge points is modeled in hyperbolic space through a cone attention mechanism, which is more effective than the point product attention in traditional Euclidean space, and can automatically discover and utilize the implicit hierarchical structure between knowledge points to improve prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 The figure is a schematic diagram of the structure and workflow of the knowledge tracking model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] This embodiment provides a knowledge tracking cold start method based on cone attention and kernel bias, including:

[0039] Obtain the answer record information of the target students;

[0040] The answer record information is input into a preset knowledge tracking model to predict the next answer situation of the target student, wherein the knowledge tracking model is obtained based on training of a training set, and the training set includes the answer record information of several students; the knowledge tracking model is used to embed the answer record information, and then use cone attention to spatially map the embedded representation, and then perform kernel bias attenuation processing on the output of the cone attention, and enhance the knowledge state through the processed embedded representation, so as to predict the next answer situation.

[0041] The answer record information includes: the question, the knowledge point associated with the question, the answer result of the question, and the answer time of the question.

[0042] The knowledge tracking model includes: an answer information embedding module, an attention processing module, and an answer status prediction module, wherein the answer information embedding module is used to obtain the embedded representation by processing the answer record information; the attention processing module is used to spatially map the embedded representation, calculate the attention score with kernel bias attenuation and perform knowledge state enhancement to obtain knowledge state output; the answer status prediction module is used to predict the next answer status through the knowledge state output.

[0043] Specifically, this embodiment effectively solves the cold start problem in the field of knowledge tracking by training on short sequence data and maintaining stable performance when predicting long sequences. It does not require additional auxiliary information or external data such as knowledge graphs, and can achieve good prediction results only by relying on limited interaction data. This embodiment has strong generalization ability for sequence length and proposes a kernel bias mechanism, which enables the model to maintain stable performance on sequences of different lengths. The hierarchical relationship between knowledge points is modeled in hyperbolic space through the cone attention mechanism, which is more effective than the point product attention in traditional Euclidean space, and can automatically discover and utilize the implicit hierarchical structure between knowledge points to improve prediction accuracy.

[0044] Combine the following Figure 1 The knowledge tracking model structure and workflow proposed in this embodiment are described in detail, including:

[0045] (1) Question answer information embedding module:

[0046] The answer information embedding module is specifically used to process the answer record information to obtain the embedded representation, including:

[0047] Calculate the two-dimensional unique hot encoding of several knowledge points in the answer record information and the answers to the questions where the knowledge points are located; calculate the implicit representation of the knowledge points and the implicit representation of the answers to the questions where the knowledge points are located based on the two-dimensional unique hot encoding; calculate the embedded representation of the knowledge points and the embedded representation of the answers to the questions where the knowledge points are located by introducing the embedding of the learnable problem difficulty factor and the problem-related knowledge points in the implicit representation.

[0048] Specifically, this embodiment obtains each interaction record from the original data, including: the student answer sequence is {q, c, r, t}, q is the question; c is the knowledge point associated with the question; r is the answer to the question, 1 indicates a correct answer, 0 indicates an incorrect answer; t is the time step of the answer. When a question contains multiple knowledge points, it is split into multiple knowledge point-level interaction records. First, the 2D unique hot encoding ec of the knowledge point and the answer to the question at time step t is calculated. t and Then calculate the implicit representation of the knowledge point and the implicit representation of the answer to the question By introducing the learnable problem difficulty factor Embedding of knowledge points associated with the question Compute the embedded representation of enhanced knowledge points and an embedding representation containing the question answer

[0049] (2) Attention processing module:

[0050] The attention processing module includes: an attention mapping unit, an attention decay unit, and an attention output unit, wherein the attention mapping unit is used to map the embedded representation from the Euclidean space to the Poincare hemisphere space, and calculate the similarity attention score of the hyperbolic space; the attention decay unit is used to introduce a kernel bias decay matrix and a causal relationship mask matrix to obtain an attention score with kernel bias decay; the attention output unit is used to perform weighted aggregation on the attention score and the question answer to obtain the knowledge state output.

[0051] Specifically, in this embodiment, the attention mapping unit maps the embedding representation from the Euclidean space to the Poincare hemisphere space, and calculates the similarity attention score of the hyperbolic space as follows:

[0052] First, the embedding representation of the student's answer sequence is used as the input of the attention mechanism. Specifically, the query vector q=x in the attention mechanism t+1 , key vector k = {x 1 , …, x t} and value vector v = {y 1 ,…y t}. From Euclidean space to Poincare hemisphere space through mapping function φ, xd Represents the last dimension of vector x, X: d-1 The mapping rule for the first d-1 dimensions of the pointing quantity x is:

[0053] φ(x): d-1 =x: d-1 ·exp(x d ) φ(x) d =exp(x d );

[0054] Among them, exp(·) is an exponential function. Then the query vector q and key vector k in the attention mechanism are mapped to the Poincare hemisphere space:

[0055] q=[φ(q): d-1 ,φ(q)d]

[0056] k=[φ(k): d-1 ,φ(k) d ];

[0057] Finally, the hierarchical perceived similarity attention score A(q,k) in the hyperbolic space is calculated, namely:

[0058]

[0059] Among them, A(q,k) is the similarity attention score, exp(·) is the exponential function, φ(q) d Indicates that the last dimension of the query vector q in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k) d Represents that the last dimension of the key vector k in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(q): d-1 It represents that the first d-1 dimensions of the query vector q in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k): d-1 It means that the first d-1 dimensions of the key vector k in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, γ is the scale transformation coefficient, r is the hyperbolic radius, and ||·|| represents the L2 norm distance.

[0060] Specifically, in this embodiment, the attention decay unit introduces a kernel bias decay matrix and a causal relationship mask matrix, and the process of obtaining the attention score with kernel bias decay is as follows:

[0061] In hyperbolic space, the attention score of the self-attention mechanism is calculated as follows:

[0062] A bias =ψ(A(q, k));

[0063] Where ψ is the softmax function. By introducing the kernel bias attenuation matrix into the attention score and the causality mask matrix Get the attention score A based on kernel bias decay bias :

[0064]

[0065] Among them, each element of the attenuation matrix is:

[0066]

[0067] Causality Mask Matrix Each element of is:

[0068]

[0069] Specifically, in this embodiment, the attention output unit is used to perform weighted aggregation on the attention score and the question answer, and the process of obtaining the knowledge state output is as follows:

[0070] The output of the cone attention mechanism and the interaction of the students are weighted and aggregated by matrix multiplication to obtain the implicit knowledge state output, as follows:

[0071] h t+1 =A bias ·v;

[0072] Among them, v represents the value vector in the attention mechanism, h t+1 is the output of the hidden layer in the attention mechanism, that is, the knowledge state output.

[0073] (3) Question answer prediction module:

[0074] The answer situation prediction module is used to predict the next answer situation through the knowledge state output, specifically including: inputting the knowledge state output into a fully connected neural network, and calculating the predicted probability of the student's next answer situation under the condition of minimum binary cross entropy loss.

[0075] Specifically, this embodiment uses a two-layer fully connected neural network to predict the student's answer situation, takes the output of the knowledge state enhancement module as the input of the fully connected neural network, and calculates the binary cross entropy loss Under the minimum condition, the predicted probability of students' answers for:

[0076]

[0077] in, is the predicted probability of the student's next answer, σ is the Sigmoid function, ReLU is the ReLU function, W 1 and W 2 is the learnable parameter matrix in the fully connected neural network, h t+1 is the knowledge state output of the hidden layer in the attention mechanism, x t+1 For the next exercise, b 1 and b 2 is the bias term, is the binary cross entropy loss, r t+1 is the actual probability of the student's next answer.

[0078] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A knowledge tracking cold start method based on cone attention and kernel bias, characterized in that: include: Obtain the answer record information of the target students; The answer record information is input into a preset knowledge tracking model to predict the next answer situation of the target student, wherein the knowledge tracking model is trained based on a training set, and the training set includes the answer record information of several students; the knowledge tracking model is used to embed the answer record information, and then use cone attention to spatially map the embedded representation, and then perform kernel bias attenuation processing on the output of the cone attention, and enhance the knowledge state through the processed embedded representation, so as to predict the next answer situation.

2. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 1, characterized in that: The answer record information includes: the question, the knowledge points associated with the question, the answer result of the question, and the answer time of the question.

3. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 1, characterized in that: The knowledge tracking model includes: an answer information embedding module, an attention processing module, and an answer status prediction module, wherein the answer information embedding module is used to obtain the embedded representation by processing the answer record information; the attention processing module is used to spatially map the embedded representation, calculate the attention score with kernel bias attenuation and perform knowledge state enhancement to obtain knowledge state output; the answer status prediction module is used to predict the next answer status through the knowledge state output.

4. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 3, characterized in that: The answer information embedding module is specifically used to obtain the embedded representation by processing the answer record information, including: Calculate the two-dimensional unique hot codes of the several knowledge points and the answers to the questions where the knowledge points are located in the answer record information; Calculating the implicit representation of the knowledge point and the implicit representation of the answer to the question where the knowledge point is located based on the two-dimensional one-hot encoding; By introducing the learnable problem difficulty factor and the embedding of the problem-related knowledge points into the implicit representation, the embedded representation of the knowledge points and the embedded representation of the answer to the question where the knowledge points are located are calculated.

5. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 3, characterized in that: The attention processing module includes: an attention mapping unit, an attention decay unit, and an attention output unit, wherein the attention mapping unit is used to map the embedded representation from the Euclidean space to the Poincare hemisphere space, and calculate the similarity attention score of the hyperbolic space; the attention decay unit is used to introduce a kernel bias decay matrix and a causal relationship mask matrix to obtain an attention score with kernel bias decay; the attention output unit is used to perform weighted aggregation on the attention score and the question answer to obtain the knowledge state output.

6. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 5, characterized in that: The method for calculating the similarity attention score in hyperbolic space is: Among them, A(q,k) is the similarity attention score, exp(·) is the exponential function, φ(q) d Indicates that the last dimension of the query vector q in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k) d The last dimension of the key vector k in the attention mechanism is mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(q) ：d-1 The first d-1 dimensions of the query vector q in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, φ(k) ：d-1 It means that the first d-1 dimensions of the key vector k in the attention mechanism are mapped from the Euclidean space to the Poincare hemisphere space through the mapping function φ, γ is the scale transformation coefficient, r is the hyperbolic radius, and ||·|| represents the L2 norm distance.

7. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 6, characterized in that: The method for obtaining the attention score with kernel bias decay is: Among them, A bias is the attention score, ψ is the softmax function, is the kernel bias attenuation matrix, is the causal relationship mask matrix.

8. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 7, characterized in that: The method for obtaining the knowledge state output is: h t+1 =A bias ·in; Among them, h t+1 is the knowledge state output of the hidden layer in the attention mechanism, and V represents the value vector in the attention mechanism.

9. The knowledge tracking cold start method based on cone attention and kernel bias according to claim 3, characterized in that: The answer situation prediction module is specifically used to predict the next answer situation through the knowledge state output, including: The knowledge state output is input into the fully connected neural network to calculate the predicted probability of the student's next answer under the condition of minimum binary cross entropy loss. The calculation method is: in, is the predicted probability of the student's next answer, σ is the Sigmoid function, ReLU is the ReLU function, W1 and W2 are the learnable parameter matrices in the fully connected neural network, and h t+1 is the knowledge state output of the hidden layer in the attention mechanism, x t+1 This is the next exercise, b1 and b2 are bias terms, is the binary cross entropy loss, r t+1 is the actual probability of the student's next answer.

Images