A knowledge tracking method and system based on reinforcement learning and negative sampling optimization

Abstract

The present invention provides a knowledge tracking method based on reinforcement learning and negative sampling optimization, which relates to the field of knowledge tracking technology. The method comprises: obtaining a set of knowledge points from a learner's historical answer sequence to obtain a knowledge point representation of an exercise; generating an initial exercise graph based on the knowledge point representation, and dynamically adjusting the edge weights of the initial exercise graph to obtain an adjusted exercise graph; using the exercise graph to obtain a set of historical positive samples and a set of historical negative samples associated with the current exercise; calculating similarity based on the historical positive sample set and the historical negative sample set, and obtaining a knowledge point vector for the current exercise through sample loss function optimization; dynamically updating the learner's knowledge state based on the knowledge point vector, and outputting the learner's future answer status based on the updated knowledge state. The present invention effectively captures the potential correlation between exercises, enhances predictive reasoning capabilities, and improves prediction accuracy.

Description

Technical Field

[0001] The present invention relates to the field of knowledge tracking technology, and in particular to a knowledge tracking method and system based on reinforcement learning and negative sampling optimization. Background Art

[0002] The development of personalized education has made knowledge tracing (KT) an important research area in intelligent online education. The core task of knowledge tracing is to predict a learner's future performance based on problem representations and their past responses.

[0003] Currently, existing knowledge tracking usually relies on multimodal feature fusion to represent exercises. The advantage of this method is that the overall knowledge state is generated by the fusion of the states of each knowledge point. The representation of the overall knowledge state contains richer information, and the prediction effect is better overall.

[0004] However, the relationship between the states of various knowledge points will affect the aggregate representation of the knowledge point states and the representation of the overall knowledge state, resulting in certain limitations in capturing the correlation between positive and negative samples in the current answer exercises and historical answer exercises, which leads to a decline in the predictive performance of the knowledge tracking method and low prediction precision and accuracy. Summary of the Invention

[0005] The present invention provides a knowledge tracking method and system based on reinforcement learning and negative sampling optimization, which are used to solve the problem of low precision and accuracy of prediction of knowledge tracking methods.

[0006] In one aspect, the present invention provides a knowledge tracking method based on reinforcement learning and negative sampling optimization, the method comprising:

[0007] Obtain the knowledge point set of the learner's historical answer sequence and obtain the knowledge point representation of the exercise;

[0008] generating an initial exercise graph according to the knowledge point representation, and dynamically adjusting the edge weights of the initial exercise graph to obtain an adjusted exercise graph;

[0009] Using the exercise graph, a historical positive sample set and a historical negative sample set associated with the current exercise are obtained;

[0010] Calculate similarity based on historical positive sample sets and historical negative sample sets, and optimize the sample loss function to obtain the knowledge point vector of the current exercise;

[0011] The learner's knowledge state is dynamically updated according to the knowledge point vector, and the learner's future answer status is output based on the updated knowledge state.

[0012] Furthermore, the knowledge point representation is obtained by:

[0013] Obtain a set of knowledge points of an exercise, perform one-hot encoding on each knowledge point, and obtain a one-hot encoding vector of each knowledge point; simultaneously obtain a knowledge point weight of each knowledge point in the exercise; and obtain a knowledge point representation of the exercise based on the one-hot encoding vector and the knowledge point weight.

[0014] Furthermore, the knowledge point represents It is obtained by weighted summation of knowledge points, and the calculation formula is:

[0015] ;

[0016] in, Indicates exercises; Indicates knowledge points, Representation Exercises The corresponding knowledge point set, ; Represents the weight of the knowledge point, satisfying ,and ; represents a one-hot encoded vector, , , and knowledge points The corresponding position is marked as 1, and the rest of the positions are marked as 0; Indicates the size of the knowledge point set.

[0017] Furthermore, edge weights may be obtained by:

[0018] Each edge The edge weight is initialized to 1, and the two nodes connected by the edge and The features are linearly transformed and activated, and two new nodes are obtained. ; Calculate new nodes The attention between them is calculated and the edge weight is obtained based on the attention.

[0019] Furthermore, the method for obtaining the historical negative sample set includes:

[0020] S301: traverse the exercise graph based on edge weights to obtain an exercise node sequence;

[0021] S302: Randomly select a central node in the exercise node sequence as the initial node of the Markov chain;

[0022] S303: For each central node, calculate the current state space corresponding to the current central node, and select the next central node according to the reinforcement learning algorithm function;

[0023] S304: Update the corresponding state space according to the next central node, calculate the reward function, and update the reinforcement learning algorithm function according to the reward function;

[0024] S305: Repeat steps S303 to S304 until a node is sampled from the exercise node sequence. negative samples, and thus obtain the historical negative sample set.

[0025] Furthermore, the reward function is:

[0026] ;

[0027] in, Indicates the current center node, Indicates the next central node; Indicates the current central node With the next central node The edge weights between express The set of neighbor nodes of Represents an adjustment factor used to balance the rewards of positive and negative samples; Indicates the weighted shortest path length between the current node and the next node.

[0028] Furthermore, the update formula of the reinforcement learning algorithm function is:

[0029] ;

[0030] in, Represents the current state space; represents the next state space; represents the learning rate; represents the discount factor; Indicates that in the next state space The next action; Indicates execution of an action Rewards received afterward.

[0031] Furthermore, the calculation formula of the similarity is:

[0032] ;

[0033] in, Represents a vector With vector The dot product of and Represents vectors and vector The L2 norm of Indicates the current exercise 's knowledge point vector; Indicates the negative sample set The node vector of the history exercise.

[0034] Furthermore, the learner's future answer status is output based on the updated knowledge state vector, including:

[0035]

[0036] in, represents the weight matrix; Represents the updated knowledge state; represents the bias term.

[0037] In a second aspect, the present invention provides a knowledge tracking system based on reinforcement learning and negative sampling optimization, comprising a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of any of the above methods.

[0038] In general, the present invention provides a knowledge tracking method and system based on reinforcement learning and negative sampling optimization. The technical solution conceived by the present invention can achieve the following beneficial effects compared with the existing technology:

[0039] (1) The present invention adjusts the exercise graph and samples the positive and negative sample sets based on the added exercise graph. It adopts the negative sampling strategy of weighted probability priority traversal to sample in the exercise graph, and optimizes the knowledge point vectors represented by the exercise based on the positive and negative sample sets, so that the sample set approaches the target distribution, improves the clustering effect of the positive sample exercise nodes, effectively distinguishes the negative sample exercise nodes, effectively mines the potential relationship between exercises, captures the correlation between positive and negative samples, enhances the prediction performance, and improves the precision and accuracy of the prediction.

[0040] (2) The present invention dynamically adjusts the edge weights between nodes in the exercise graph through the graph attention network, further exploring the similarities between historical exercises and currently answered exercises, thereby enabling the selection of historical exercise samples with a strong correlation with the central node in the exercise graph, enhancing the clustering effect of positive sample exercises, and effectively distinguishing negative sample exercises, optimizing the exercise representation, and thus improving the prediction performance.

[0041] (3) Based on Markov Chain Monte Carlo, the present invention performs negative sampling on the state space of the exercise graph, which can not only approximate the target distribution and further optimize the exercise representation; but also simulate the cognitive process of learners relying on historical exercise reasoning when solving new problems, thereby improving their performance in complex learning situations; and further effectively capture the potential correlation between exercises and improve the prediction accuracy in knowledge tracking tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 This is a flow chart of a knowledge tracking method and system based on reinforcement learning and negative sampling optimization provided by the present invention;

[0044] Figure 2 This is a logical architecture diagram of a knowledge tracking method and system based on reinforcement learning and negative sampling optimization provided by the present invention. DETAILED DESCRIPTION

[0045] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0046] It should be noted that, in the description of the embodiments of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a method, step, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such method, step, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the method, step, or apparatus comprising the element.

[0047] In order to improve the predictive performance of the knowledge tracking method and enhance the precision and accuracy of the prediction, the present invention provides a knowledge tracking method and system based on reinforcement learning and negative sampling optimization. The learner's historical answer sequence is input into the established knowledge tracking model to obtain the learner's future answer situation. That is: the input of the knowledge tracking model is the learner's historical answer sequence, and the output is the learner's future answer situation.

[0048] On the one hand, if Figure 1 and 2 As shown, the method includes:

[0049] S101: Constructing a knowledge point representation module for the exercise. That is, obtaining a set of knowledge points from the learner's historical answer sequence and obtaining a knowledge point representation for the exercise.

[0050] The knowledge point representation of an exercise is derived from the knowledge points contained in the exercise and the relationships between them. It should be noted that an exercise contains multiple knowledge points, each of which plays a different role within the exercise. In the knowledge tracking model, the implicit features of an exercise are captured by extracting the characteristic information of the knowledge points contained in the exercise. The knowledge point representation of an exercise includes the historical knowledge point representation of previous exercises and the current knowledge point representation of the current exercise.

[0051] Define the learner's historical answer sequence as ;in, Indicates the learner's current answer record. Indicates the answer record at the last moment.

[0052] Define the learner's current answer record as ;in, Indicates the exercise the learner is currently answering; Indicates the learner's answer to the current exercise, 0 indicates an incorrect answer, and 1 indicates a correct answer; Indicates all the knowledge points contained in the current exercise. The number of knowledge points included is , then the current exercise The corresponding knowledge point set is .

[0053] By constructing the knowledge point encoding of the exercise and combining it with the attention mechanism to calculate the knowledge point weight of each knowledge point, we can finally obtain the knowledge point representation of the exercise.

[0054] As an embodiment, obtaining the knowledge point representation includes: obtaining a set of knowledge points of an exercise, performing one-hot encoding on each knowledge point, and obtaining a one-hot encoding vector for each knowledge point; simultaneously obtaining the knowledge point weight of each knowledge point in the exercise; and obtaining the knowledge point representation of the exercise based on the one-hot encoding vector and the knowledge point weight.

[0055] It should be noted that after obtaining the one-hot encoded vector, in order to reflect the importance of each knowledge point in the exercise, it is also necessary to assign a knowledge point weight to each knowledge point. These knowledge point weights are gradually optimized through the backpropagation algorithm during the model training process.

[0056] More specifically, knowledge points represent It is obtained by weighted summation of knowledge points, and its calculation formula is: ;in, Indicates exercises; Indicates knowledge points, Representation Exercises The corresponding knowledge point set, ; Represents the weight of the knowledge point, satisfying ,and ; represents a one-hot encoded vector, , , and knowledge points The corresponding position is marked as 1, and the rest of the positions are marked as 0; Indicates the size of the knowledge point set.

[0057] S102: Constructing a problem graph module. That is, generating an initial problem graph based on the knowledge point representation, and dynamically adjusting the edge weights of the initial problem graph to obtain an adjusted problem graph.

[0058] It's important to note that as learners answer exercises, analogies from historical exercises are used to assist with their answers. This process reveals characteristic correlations between exercises and reflects the relevance of exercises in learners' responses. Therefore, we construct an exercise graph by representing historical and current knowledge points. We use a graph attention network to calculate edge weights between nodes, and then adjust the exercise graph based on these edge weights.

[0059] Exercise graphs consist of nodes and edges, where nodes represent exercises and edges represent connections between them. Constructing an exercise graph can more accurately and comprehensively capture and express the complex relationships between exercises. It not only preserves the knowledge points of the exercises but also reflects the underlying connections between them through edge weight adjustments.

[0060] As an embodiment, the construction of the initial exercise graph includes: And the current knowledge point representation , with all exercises As nodes, the relationships between different exercises are used as edges to construct the initial exercise graph .in, ; Represents a collection of nodes, ; represents the edge set, , and Represents exercises respectively and exercises A collection of knowledge points, Representation Exercises and exercises Shared knowledge point set, if the shared knowledge point set is a non-empty set, then in the exercise and exercises Add an edge between the corresponding two nodes .

[0061] In order to further enhance the expressive power of the exercise graph, the present invention introduces a graph attention mechanism to dynamically adjust the edge weights of the initial exercise graph to obtain the adjusted exercise graph. ,in, represents the edge set in the adjusted exercise graph, ; Represents edge weight.

[0062] Specifically, initially, since these edges are established based on shared knowledge points, the edge weights can be obtained by: first, The edge weight is initialized to 1, and the two nodes connected by the edge and The features are linearly transformed and activated, and two new nodes are obtained. ; Then, calculate the new node The attention between them is calculated and the edge weight is obtained based on the attention.

[0063] use The function normalizes the attention of all neighbor nodes to obtain edge weights. As an embodiment, the edge weight calculation formula is:

[0064] ;

[0065] in, Represents a new node attention between; Represents a new node and neighbor nodes attention between; Representation node The set of neighbor nodes.

[0066] Furthermore, the calculation formula of attention is:

[0067] ;

[0068] in, represents the parameter vector of the attention mechanism, represents the concatenation operation of feature vectors, Represents the activation function.

[0069] Step 103: Construct a graph sampling module. That is, use the exercise graph to obtain a set of historical positive samples and a set of historical negative samples associated with the current exercise.

[0070] The graph sampling module uses a neighbor node sampling strategy to extract a set of historical positive samples. That is, by sampling the neighbor nodes of the current exercise, historical exercise samples with a high correlation with the current exercise are selected.

[0071] As an embodiment, obtaining the historical positive sample set includes:

[0072] Find the problem that matches the current problem in the problem map Directly connected historical exercises, get the set of neighbor nodes , and then get the edge set in the adjusted exercise graph , and based on the edge set Get the adjusted edge weights ;

[0073] Multiple edge weights Arrange from high to low, select The historical exercises of the neighbor nodes corresponding to the edge weights are used as positive samples to obtain the positive samples of the current exercises. A set of historical positive samples with high correlation: At the same time, the historical positive sample set Knowledge point representation Extract it.

[0074] In the exercise graph, the edge weights between exercise nodes reflect the similarity between exercises. Similar exercise nodes have similar sampling distributions. Therefore, during the negative sample sampling process, nodes with greater similarity can be selected, which helps accelerate the convergence of the knowledge tracking model.

[0075] Exercise Chart The nodes in the Perform weighted probability-first traversal. The weighted probability-first traversal can be based on the traditional depth-first search (DFS) strategy and introduce edge weights between nodes.

[0076] As an example, during each traversal, the traversal order is determined based on the edge weights between nodes, ensuring that nodes with greater similarity are traversed first. Specifically, for the current node, all nodes connected to the current node are calculated and recorded as the neighbor node set. From the neighbor node set, the node most similar to the current node is selected as a positive sample and added to the historical positive sample set.

[0077] However, as the traversal proceeds, the similarity between nodes gradually decreases, resulting in differences in the sampling distribution of samples, which affects the model training effect.

[0078] In order to improve the model training effect, this paper introduces reinforcement learning to optimize the negative sample sampling strategy and proposes a negative sample sampling strategy that combines the Markov chain Monte Carlo method (MCMC) with reinforcement learning. The Markov chain sampling strategy is optimized through reinforcement learning, thereby improving the sampling efficiency of negative samples.

[0079] After obtaining the problem graph, we traverse the problem graph based on edge weights to obtain a problem node sequence, where each node is a central node. We use the last node of the previous central node in the Markov chain as the initial node of the Markov chain for historical problems similar to the current problem. We perform reinforcement learning sampling to obtain a set of historical negative samples that are highly relevant to the current problem.

[0080] In the negative sampling process, a Markov chain is used to generate negative samples. Each time, an action is selected through the reinforcement learning algorithm function based on the current state space to determine the next node.

[0081] As an embodiment, a method for obtaining a historical negative sample set includes:

[0082] S301: Traverse the exercise graph based on edge weights to obtain an exercise node sequence.

[0083] It should be noted that the central node is obtained by traversing the current exercise's node and the last node of the previous exercise in the historical answer sequence. The last node of the previous exercise is the starting node, and the node of the current exercise is the ending node. The exercise node sequence is obtained by traversing the edge weights in between. Each node in the exercise node sequence is a central node.

[0084] S302: Randomly select a central node in the exercise node sequence as the initial node of the Markov chain.

[0085] S303: For each central node, calculate the current state space corresponding to the current central node, and select the next central node according to the reinforcement learning algorithm function.

[0086] It should be noted that, at the first sampling, the initial node Can be used as the initial state space , so that the next central node is selected The process is a reinforcement learning problem.

[0087] The reinforcement learning algorithm function is the Q-learning algorithm function. Q-learning is a reinforcement learning algorithm based on value iteration, which is updated by maximizing the expected return of the current strategy.

[0088] As an embodiment, the update formula of the reinforcement learning algorithm function is:

[0089] ;

[0090] in, Represents the current state space; represents the next state space; represents the learning rate; represents the discount factor; Indicates that in the next state space The next action; Indicates execution of an action Rewards received afterward.

[0091] S304: Update the corresponding state space according to the next central node, calculate the reward function, and update the reinforcement learning algorithm function according to the reward function;

[0092] The reward function is a function that evaluates the sampling quality based on the similarity between the central nodes and the edge weights. Nodes with higher correlation will be given higher rewards; Irrelevant nodes are given negative rewards.

[0093] As an embodiment, the reward function is calculated based on the edge weight between the current central node and the next central node and the weighted shortest path length between the two central nodes.

[0094] Specifically, the reward function can be expressed as:

[0095] ;

[0096] in, Indicates the current center node, Indicates the next central node; Indicates the current central node With the next central node The edge weights between express The set of neighbor nodes of Represents an adjustment factor used to balance the rewards of positive and negative samples; Indicates the weighted shortest path length between the current node and the next node.

[0097] S305: Repeat the above steps until a node is sampled from the exercise node sequence. negative samples, and thus obtain the historical negative sample set.

[0098] S104: Construct a contrastive learning module. That is, calculate similarity based on the historical positive sample set and the historical negative sample set, and obtain the knowledge point vector of the current exercise through sample loss function optimization.

[0099] The graph sampling module obtains a set of positive and negative samples associated with the current problem. The contrastive learning module clusters the positive sample problems and distinguishes the negative sample problems, generating a vector representation of knowledge points that fully reveals the inherent structure and patterns of the current problem.

[0100] Therefore, the present invention first calculates the positive similarity between the current knowledge point representation and the historical positive sample set, as well as the negative similarity between the current knowledge point representation and the historical negative sample set; then compares the positive similarity and the negative similarity, optimizes the exercise sample through the sample loss function, obtains the exercise representation of the current exercise, and reduces the historical exercise samples that are irrelevant to the current exercise.

[0101] It should be noted that similarity calculation is used to evaluate the similarity between the current exercise and the positive and negative sample sets of historical exercises, so that the knowledge tracking model can capture the characteristics of the exercises and their correlations, thereby enhancing the ability to distinguish; at the same time, by analyzing a large amount of historical data, the generalization ability and predictive performance of the knowledge tracking model are improved.

[0102] In order to calculate the similarity between the positive and negative sample sets of the current exercise and the previous exercises, the cosine similarity measurement method is adopted. As a specific embodiment, the similarity calculation formula is:

[0103] ;

[0104] in, Represents a vector With vector The dot product of and Represents vectors and vector The L2 norm of . Indicates the current exercise 's knowledge point vector; Indicates the negative sample set The node vector of the history exercise.

[0105] In order to calculate the sample loss function, the similarity results of all positive and negative sample sets of the current exercise and the historical exercises are stored in the similarity matrix.

[0106] Specifically, the similarity matrix is ​​calculated as:

[0107] ;

[0108] in, Indicates the current exercise 's knowledge point vector; Indicates the first Each historical exercise is represented by a node in the exercise graph. ; Indicates the negative sample set Each historical exercise is represented by a node in the exercise graph. .

[0109] The sample loss function plays a key role in optimizing the sample set. By reducing the similarity between the positive samples of the current exercise and the past exercises, and increasing the similarity between the negative samples of the current exercise and the past exercises, the knowledge tracking model can more accurately capture and distinguish the characteristics of different exercises.

[0110] As an example, the sample loss function is obtained by summing the losses of all exercises, specifically:

[0111] ;

[0112] in, ; Indicates the current exercise 's knowledge point vector; Indicates the first Each historical exercise is represented by a node in the exercise graph. ; Indicates the negative sample set Each historical exercise is represented by a node in the exercise graph. ; represents the negative sample set.

[0113] S105: Constructing a prediction module. That is, dynamically updating the learner's knowledge state based on the knowledge point vector and outputting the learner's future answer status.

[0114] When predicting a learner's response, the core task is to reasonably predict their future responses based on their knowledge state. By dynamically updating the learner's knowledge state through optimized knowledge point vectors, we can determine their future responses.

[0115] This paper introduces a gated recurrent unit (GRU) network structure to help the knowledge tracking model capture the learner's current memory of historical answer exercises and the degree of dependence on previously learned content, update the learner's knowledge state, and predict the learner's answer at the next moment based on the learner's knowledge state. By capturing the hidden state to represent the learner's knowledge mastery, it is mapped to the probability of answering the next question.

[0116] During the learner's knowledge state updating phase, the goal is to gradually update the learner's knowledge state by inputting knowledge points from exercises. Therefore, the present invention introduces a gated recurrent unit network structure to capture the learner's knowledge state changes over the course of a historical answer sequence. Furthermore, this is captured by capturing the learner's memory of historically answered exercises and their reliance on previously learned content.

[0117] In order to obtain the learner's memory of historical answer exercises, the present invention controls the retention degree of the previous time step information in the current time step through the update gate, reflecting the learner's dependence on historical knowledge when processing new knowledge.

[0118] As an example, update gate The calculation method is:

[0119] ;

[0120] in, , Represents the Sigmoid activation function; and represents the weight matrix; Indicates the current time step Input; Represents the hidden state of the current time step, which is also the final hidden state; represents the bias term.

[0121] In order to adjust the learner's dependence on the previously learned content, so that when dealing with new exercises, the learner's knowledge state can be effectively updated without being disturbed by historical exercise factors; the present invention uses a reset gate to control the degree of ignoring the hidden state of the previous time step.

[0122] As an example, the reset gate The calculation method is:

[0123] ;

[0124] in, , Represents the Sigmoid activation function; and represents the weight matrix; Indicates the current time step Input; Represents the hidden state of the current time step, which is also the final hidden state; represents the bias term.

[0125] It should be noted that in order to better represent the current time step Input , the knowledge point vector of the current exercise is taken as the current time step Input , that is: ;in, , indicating the The knowledge point vector of the exercise, Represents the vector dimension.

[0126] As an example, the final hidden state The hidden state of the previous time step can be and candidate hidden states Weighted fusion is obtained.

[0127] Furthermore, the final hidden state The calculation method can be:

[0128] ;

[0129] in, represents the update gate; represents the hidden state of the previous time step; represents a candidate hidden state.

[0130] As an embodiment, the candidate hidden state can be calculated based on the reset gate, and the specific calculation formula is:

[0131] ;

[0132] in, represents the candidate hidden state, which is the combination of the hidden state of the current time step and the adjusted hidden state of the previous time step; represents element-wise multiplication; Represents the Sigmoid activation function; and represents the weight matrix; Indicates the current time step Input; Reset gate. represents the hidden state of the previous time step; represents the bias term.

[0133] It should be noted that after the learner's knowledge state is dynamically updated, the learner's answer to the next exercise is predicted based on the updated knowledge state. The prediction module of the present invention only depends on the final hidden state of the current time step. , it is possible to avoid using the learner’s historical answer sequence, thereby improving the objectivity of the prediction.

[0134] The prediction of the learner's answer to the next exercise can be calculated through the output of the fully connected layer and the Sigmoid activation function.

[0135] In order to optimize the performance of the prediction module, the present invention adopts the cross-entropy loss function to measure the difference between the predicted results and the actual answer results. By minimizing the cross-entropy loss function, the actual answer situation can be more accurately fitted when predicting the learner's answer situation, thereby improving the accuracy of the prediction.

[0136] As an example, the cross entropy loss function is:

[0137] ;

[0138] in, represents the total number of samples; , represents the real answer result, , which represents the predicted probability that the learner answers correctly.

[0139] It should be noted that the predicted probability It can be calculated by the fully connected layer and Sigmoid activation function. Further, the predicted probability The calculation formula is:

[0140] ;

[0141] in, Represents the Sigmoid activation function; Represents the weight matrix of the fully connected layer; represents the final hidden state; represents the bias term;

[0142] The prediction module predicts the learner's future answers through optimized knowledge point vectors.

[0143] Considering that different examination weights of exercises for knowledge points will lead to different difficulties of exercises, the present invention will Knowledge point vectors related to the distributed exercises to be answered Splicing is performed and the updated knowledge state is obtained through a fully connected layer with an activation function , used to represent the future state of knowledge points and the relationship between the states of each future knowledge point, as well as information such as learning behavior.

[0144] Further, the updated knowledge state The calculation formula is:

[0145] ;

[0146] in, Represents the Sigmoid activation function; represents the weight matrix; Represents the knowledge point vector to be answered; Represents the current state of the knowledge point, which is equal to the final hidden state; represents the bias term.

[0147] Output the learner's future answer based on the updated knowledge state vector, including:

[0148]

[0149] in, represents the weight matrix; Represents the updated knowledge state; represents the bias term.

[0150] In addition, in order to verify that the prediction of the present invention is more accurate, the present invention verifies the performance of the method through experiments, and compares the prediction accuracy of the present invention with that of other existing methods. The comparison results are shown in Table 1:

[0151] Table 1 Comparison of prediction results between the present invention and other existing methods

[0152]

[0153] It can be seen that the knowledge tracking method based on negative sampling of the improved Markov Chain Monte Carlo method proposed in the present invention has a higher prediction accuracy, which proves the superiority of this method in the knowledge tracking task.

[0154] In a second aspect, the present invention provides a knowledge tracking system based on reinforcement learning and negative sampling optimization, comprising a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of any of the above methods. The technical features of the system are consistent with those of the above methods and are not further detailed here.

[0155] In summary, the present invention adjusts the exercise graph and samples positive and negative sample sets based on the added exercise graph. It adopts a negative sampling strategy of weighted probability priority traversal to optimize the knowledge point vector represented by the exercise, so that the sample set approaches the target distribution, improves the clustering effect of positive sample exercise nodes, effectively distinguishes negative sample exercise nodes, effectively captures the potential correlation between exercises, enhances the predictive reasoning ability, and improves the prediction accuracy.

[0156] It should be noted that for the aforementioned embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0157] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed methods or systems can be implemented in other ways. For example, the embodiments described above are merely illustrative, and the division of the units described is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or ignoring or not implementing certain features.

[0159] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0160] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0161] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application.

[0162] Those skilled in the art will appreciate that all or part of the various circuits in the above embodiments may be implemented by instructing related hardware through a program. The program may be stored in a computer-readable memory, which may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0163] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure herein, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

[0164] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0165] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A knowledge tracking method based on reinforcement learning and negative sampling optimization, characterized in that: The method comprises: Obtain the knowledge point set of the learner's historical answer sequence and obtain the knowledge point representation of the exercise; Generating an initial exercise graph according to the knowledge point representation, and dynamically adjusting the edge weights of the initial exercise graph to obtain an adjusted exercise graph; the exercise graph includes nodes and edges, wherein the nodes represent exercises and the edges represent the relationships between exercises; Using the exercise graph, a historical positive sample set and a historical negative sample set associated with the current exercise are obtained; Calculate similarity based on historical positive sample sets and historical negative sample sets, and optimize the sample loss function to obtain the knowledge point vector of the current exercise; Dynamically updating the learner's knowledge state according to the knowledge point vector, and outputting the learner's future answer status based on the updated knowledge state; The method for obtaining the historical negative sample set includes: S301: traverse the exercise graph based on edge weights to obtain an exercise node sequence; S302: Randomly select a central node in the exercise node sequence as the initial node of the Markov chain; S303: For each central node, calculate the current state space corresponding to the current central node, and select the next central node according to the reinforcement learning algorithm function; S304: Update the corresponding state space according to the next central node, calculate the reward function, and update the reinforcement learning algorithm function according to the reward function; S305: Repeat steps S303 to S304 until a node is sampled from the exercise node sequence. negative samples, and thus obtain the historical negative sample set.

2. The knowledge tracking method based on reinforcement learning and negative sampling optimization according to claim 1, characterized in that: The knowledge point representation is obtained by: Obtain a set of knowledge points of an exercise, perform one-hot encoding on each knowledge point, and obtain a one-hot encoding vector of each knowledge point; simultaneously obtain a knowledge point weight of each knowledge point in the exercise; and obtain a knowledge point representation of the exercise based on the one-hot encoding vector and the knowledge point weight.

3. The knowledge tracking method based on reinforcement learning and negative sampling optimization according to claim 2, characterized in that: The knowledge point represents It is obtained by weighted summation of knowledge points, and the calculation formula is: in, Indicates exercises; Indicates knowledge points, Representation Exercises The corresponding knowledge point set, ; Represents the weight of the knowledge point, satisfying ,and ; represents a one-hot encoded vector, , , and knowledge points The corresponding position is marked as 1, and the rest of the positions are marked as 0; Indicates the size of the knowledge point set.

4. The knowledge tracking method based on reinforcement learning and negative sampling optimization according to claim 1, characterized in that: Edge weights can be obtained by: Each edge The edge weight is initialized to 1, and the two nodes connected by the edge and The features are linearly transformed and activated, and two new nodes are obtained. ; Calculate new nodes The attention between them is calculated and the edge weight is obtained based on the attention.

5. The knowledge tracking method based on reinforcement learning and negative sampling optimization according to claim 1, characterized in that: The reward function is: in, Indicates the current center node, Indicates the next central node; Indicates the current central node With the next central node The edge weights between express The set of neighbor nodes of Represents an adjustment factor used to balance the rewards of positive and negative samples; Indicates the weighted shortest path length between the current node and the next node.

6. The knowledge tracking method based on reinforcement learning and negative sampling optimization according to claim 1, characterized in that: The update formula of the reinforcement learning algorithm function is: ； in, Represents the current state space; represents the next state space; represents the learning rate; represents the discount factor; Indicates that in the next state space The next action; Indicates execution of an action Rewards received afterward.

7. The knowledge tracking method based on reinforcement learning and negative sampling optimization according to claim 1 is characterized in that: The calculation formula of the similarity is: ； in, Represents a vector With vector The dot product of and Represents vectors and vector The L2 norm of Indicates the current exercise 's knowledge point vector; Indicates the negative sample set The node vector of the history exercise.

8. The knowledge tracking method based on reinforcement learning and negative sampling optimization according to claim 1, characterized in that: Output the learner's future answer based on the updated knowledge state vector, including: in, represents the weight matrix; Represents the updated knowledge state; represents the bias term.

9. A knowledge tracking system based on reinforcement learning and negative sampling optimization, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

Images