Methods and electronic devices for tracking dialogue-related knowledge points

Abstract

This application provides a method and electronic device for tracking dialogue-related knowledge points. This embodiment employs multiple methods, such as a dialogue knowledge point prediction model, retrieval of augmented generation, and hierarchical prompting using a large language model (LLM), to predict dialogue-related knowledge points. The predicted knowledge points from each method are then fused to track dialogue-related knowledge points, reducing the bias of individual methods and improving the confidence of dialogue tracking. Furthermore, this embodiment utilizes the fusion results of knowledge point sets from each dialogue, along with historical dialogues and questions from each dialogue, to train an autoregressive target knowledge tracking model. This ensures that when using the target knowledge tracking model to track knowledge in a conversation, changes in knowledge points across multiple rounds of dialogue are captured in a timely manner, improving the accuracy of knowledge tracking.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and in particular to methods and electronic devices for tracking dialogue-related knowledge points. Background Technology

[0002] In scenarios such as personalized learning in intelligent education platforms, AI teacher tutoring, online education, and adaptive learning recommendations, it is common to track the knowledge points (also known as dialogue-related knowledge points) involved in a dialogue, such as a teacher-student conversation. However, current conventional knowledge point tracking methods rely heavily on manual annotation, resulting in low efficiency, high cost, and an inability to handle massive amounts of dialogue at scale. Furthermore, this single method of manual annotation also has limitations and may introduce biases. Summary of the Invention

[0003] This application provides a method and electronic device for tracking dialogue-related knowledge points, which enables the tracking of dialogue-related knowledge points by performing knowledge annotation on multiple dialogues.

[0004] This application provides a method for tracking dialogue-related knowledge points, the method comprising:

[0005] Based on the dialogue knowledge point prediction model, predict the first set of knowledge points involved in each round of dialogue in the dialogue text; the first set of knowledge points includes at least one knowledge point; a round of dialogue includes at least a question and the content of the answer to the question.

[0006] The Retrieval Enhancement Generates (RAG) matching process is used to match each round of dialogue with knowledge points in the knowledge base to predict the second set of knowledge points involved in each round of dialogue; the second set of knowledge points includes at least one knowledge point.

[0007] Based on LLM and hierarchical prompt words, hierarchical prediction is performed starting from the first layer of the knowledge point tree structure to output a third set of knowledge points involved in each round of dialogue through the LLM; the third set of knowledge points includes at least one knowledge point; the number of knowledge point nodes increases at each layer starting from the first layer in the knowledge point tree structure.

[0008] The first, second, and third sets of knowledge points involved in each round of dialogue are merged to obtain the fourth set of knowledge points involved in each round of dialogue; and the mastery status of each knowledge point in the fourth set of knowledge points involved in each round of dialogue is obtained.

[0009] Generate a dialogue training sample and a knowledge point label for each round of dialogue. The dialogue training sample for any round of dialogue includes the question in that round of dialogue and the previous historical dialogue. The knowledge point label for each round of dialogue includes at least the fourth set of knowledge points for that round of dialogue.

[0010] Based on each dialogue training sample and the knowledge point labels of each dialogue training sample, a target knowledge tracking model is trained. The target knowledge tracking model is used to perform knowledge tracking on the target dialogue text. Based on the fusion result of the knowledge point set involved in each round of dialogue in the target dialogue text predicted by the dialogue knowledge point prediction model, RAG, LLM and Prompt respectively, the model predicts the degree to which the knowledge points in each round of dialogue in the target dialogue text are mastered and / or whether the content of the response in each round of dialogue is correct.

[0011] A method for tracking dialogue-related knowledge points, the method comprising:

[0012] Based on the dialogue knowledge point prediction model, predict the set of the fifth knowledge points involved in each round of dialogue in the current target dialogue text;

[0013] Based on RAG, each round of dialogue is matched with knowledge points in the knowledge base to predict the set of sixth knowledge points involved in each round of dialogue.

[0014] Based on LLM and hierarchical prompt words, hierarchical prediction is performed starting from the first layer of the knowledge point tree hierarchy to output the seventh set of knowledge points involved in each round of dialogue through the LLM.

[0015] The sets of fifth, sixth, and seventh knowledge points involved in each round of dialogue are merged to obtain the set of eighth knowledge points involved in each round of dialogue.

[0016] For each round of dialogue, the dialogue, the set of eighth knowledge points involved in the dialogue, and the historical dialogues preceding the dialogue are input into the target knowledge tracking model to predict the degree of mastery of each knowledge point in the set of eighth knowledge points involved in the dialogue and / or to predict whether the response in the dialogue is correct.

[0017] This application also provides an electronic device. The electronic device includes: a processor and a machine-readable storage medium;

[0018] The machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0019] The processor is used to execute machine-executable instructions to implement the steps of the disclosed method.

[0020] As can be seen from the above technical solutions, this embodiment uses multiple methods, such as dialogue knowledge point prediction model, retrieval enhancement generation and large language model LLM hierarchical prompts, to predict dialogue-related knowledge points. Then, the knowledge points predicted by each method are fused to track dialogue-related knowledge points, reduce the bias of a single method, and improve the confidence of dialogue tracking.

[0021] Furthermore, in this embodiment, based on multiple methods such as dialogue knowledge point prediction model, three methods—enhanced generation and hierarchical prompting of large language model LLM—are retrieved to predict dialogue-related knowledge points. Then, the knowledge points predicted by each method are fused to obtain a fourth set of knowledge points involved in the dialogue. Using the fourth set of knowledge points of each dialogue, as well as the historical dialogues before each dialogue and the questions in each dialogue, an autoregressive target knowledge tracking model is trained to ensure that when the target knowledge tracking model is used to track knowledge of the conversation in the future, the changes of knowledge points in multiple rounds of dialogue are captured in a timely manner, thereby improving the accuracy of knowledge tracking.

[0022] Furthermore, this embodiment is based on multiple methods such as dialogue knowledge point prediction model, retrieves three methods to predict dialogue-related knowledge points, namely augmented generation and hierarchical prompting of large language model LLM, and then fuses the knowledge points predicted by each method to obtain a set of knowledge points involved in the dialogue. By using each dialogue, the knowledge point set of each dialogue, such as the eighth knowledge point set mentioned above, and the historical dialogues before each dialogue, the changes of knowledge points in multiple rounds of dialogue are captured in a timely manner when performing knowledge tracking of the conversation, thereby improving the accuracy of knowledge tracking. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0024] Figure 1 A flowchart illustrating the method provided in this application embodiment;

[0025] Figure 2 A schematic diagram of the framework provided for an embodiment of this application;

[0026] Figure 3 This is a schematic diagram illustrating the implementation of step 102 in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the knowledge point tree-like hierarchical structure provided in the embodiments of this application;

[0028] Figure 5 Another method flowchart provided for embodiments of this application;

[0029] Figure 6 This is a structural diagram of the device provided in the embodiments of this application;

[0030] Figure 7 Another device structure diagram provided for embodiments of this application;

[0031] Figure 8 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0033] See Figure 1 , Figure 1 This is a flowchart illustrating a method provided in an embodiment of this application. This process can be applied to electronic devices such as hosts or servers, and is not specifically limited to any particular device in this embodiment.

[0034] This process begins by labeling the knowledge points involved in each round of dialogue, then by labeling the mastery of each knowledge point and / or the accuracy of the responses in each round of dialogue. Next, it generates dialogue training samples for each round of dialogue, along with the knowledge point labels and / or mastery of the relevant knowledge points for each training sample. Finally, it trains the target knowledge tracking model by leveraging the knowledge point labels and / or mastery of the relevant knowledge points from each dialogue training sample. Figure 2 A structural diagram has been provided. A detailed description follows:

[0035] like Figure 1 As shown, the process may include the following steps:

[0036] Step 101: Based on the dialogue knowledge point prediction model, predict the first set of knowledge points involved in each round of dialogue in the dialogue text; the first set of knowledge points includes at least one knowledge point; a round of dialogue includes at least a question and the answer to that question.

[0037] Optionally, this embodiment will annotate the knowledge points involved in each round of dialogue (such as the question-and-answer session between the AI ​​teacher and student in each round of dialogue) to train a dialogue knowledge point prediction model based on the knowledge points involved in each round of dialogue. Taking a teacher-student dialogue as an example, the specific questions and knowledge points involved in this round of dialogue are reflected through the teacher asking questions and the student answering. Based on this, this embodiment uses the existing knowledge point system and the online smart education platform to collect students' historical dialogues, and manually annotates the knowledge points involved in each round of dialogue in the collected dialogue set. For example, for each round of dialogue, one or more of the most relevant knowledge points in the knowledge system or knowledge points not involved in this round are first annotated. Then, LLM (Large Language Model) models such as LLaMa and Qwen are used to perform supervised SFT (Supervised Fine-Tuning) training to obtain the dialogue knowledge point prediction model.

[0038] Optionally, the dialogue knowledge point prediction model may include a recall model and a ranking model.

[0039] First, in this embodiment, for each round of dialogue, if there are previous dialogues in the dialogue text, then: the current dialogue, the previous dialogue, and the knowledge points in the knowledge base are input together into the recall model to obtain N knowledge points. Conversely, if there are no previous dialogues in the dialogue text, then the current dialogue and the knowledge points in the knowledge base are input together into the recall model. That is, the recall model is used to predict the top N knowledge points most relevant to the current round of dialogue from the knowledge base (hereinafter, the top 10 knowledge points are used as an example) as a preliminary screening. It should be noted that the reason this embodiment expands the historical dialogue to the current dialogue when there are previous dialogues in the dialogue text is to enhance the sample richness of multi-round dialogues. By using historical dialogues to select the top N knowledge points most relevant to the current round of dialogue, it ensures that the true knowledge points are among the recalled N knowledge points, such as the top 10 knowledge points, and achieves a considerable accuracy rate, such as 90% or even 95%.

[0040] Secondly, since each knowledge point in the knowledge base also possesses its own specific meaning explanation information, after recalling N knowledge points through the recall model, the number of knowledge points recalled by the recall model is limited, for example, no more than 10. Under this premise, if the above dialogue text contains historical dialogues before the current round of dialogue, this embodiment can input the N knowledge points and their meaning explanation information together with the current round of dialogue and the historical dialogues into the fine ranking model. Of course, if the above dialogue text does not contain historical dialogues before the current round of dialogue, this embodiment can also input the N knowledge points and their meaning explanation information together with the current round of dialogue into the fine ranking model. Here, the fine ranking model will, based on the understanding of the input dialogue and the input knowledge points, hit at least one of the most accurate knowledge points from the knowledge points recalled by the recall model and output it, striving for an accuracy rate of over 85%. The hit at least one knowledge point is the knowledge point in the above-mentioned first knowledge point set.

[0041] Step 102: Match each round of dialogue with knowledge points in the knowledge base based on retrieval augmented generation (RAG) to predict the second set of knowledge points involved in each round of dialogue; the second set of knowledge points includes at least one knowledge point.

[0042] The reason for executing step 102 in this embodiment is that the above-mentioned dialogue knowledge point prediction model may have prediction errors, such as not being able to completely predict all the knowledge points involved in a certain round or multiple rounds of dialogue, and lacking diversity in knowledge point prediction. For example, some dialogue rounds involve three knowledge points, but the above-mentioned dialogue knowledge point prediction model only predicts one or two of them.

[0043] In this embodiment, RAG enhances the model's ability to handle knowledge-intensive tasks by retrieving data from a knowledge base and combining the retrieved results with prompts (Prompts). Specifically, for example... Figure 3 As shown, step 102 may include the following in its specific implementation: for each round of dialogue, using a vector retrieval tool such as the Faiss vector retrieval tool to retrieve the embedding vector that matches the round of dialogue from the vector knowledge base; inputting the round of dialogue and the embedding vector into the Large Language Model (LLM) to obtain the knowledge points involved in the round of dialogue output by the LLM, wherein at least one of the knowledge points output is a knowledge point in the aforementioned second set of knowledge points.

[0044] It should be noted that step 102 retrieves the matching embedding vector of a single dialogue turn as a unit, rather than the entire dialogue text. The reason is that experimental analysis in this embodiment revealed that the entire dialogue text is too long, resulting in a less effective retrieval than a single dialogue turn. Therefore, this embodiment retrieves the matching embedding vector of a single dialogue turn as a unit to improve the retrieval effect.

[0045] In this embodiment, the vector knowledge base includes the embedding vector corresponding to each knowledge point in the knowledge base; the embedding vector corresponding to any knowledge point is obtained by inputting the Chunks text block obtained by concatenating the meaning explanation information of the knowledge point into the embedding model.

[0046] In addition, in this embodiment, the aforementioned vector retrieval tool can be an existing text embedding model, or it can be a model fine-tuned from an existing embedding model through contrastive learning. This existing embedding model could be the bge model released by BAAI or the gte model released by the Qwen team; this embodiment is not specifically limited to these models. The specific contrastive learning method involves constructing triplet data (q, a+, a-), where q represents a round of dialogue (referred to as sample dialogue), a+ represents knowledge point samples matching q (denoted as positive samples), and a- represents knowledge point samples not matching q (denoted as negative samples). The triplet data (q, a+, a-) is input into the existing embedding model, and the existing embedding model, such as bge / gte, is fine-tuned based on the output results and the contrastive loss function. This narrows the spatial distance between q and a+, and widens the distance between q and a-, allowing the fine-tuned model to learn the knowledge point system and dialogue data distribution information, thereby improving the overall knowledge point recall rate.

[0047] In this embodiment, the semantic relevance of the retrieved embedding vectors matching the current dialogue round can be further reordered. Specifically, this embodiment can input the current dialogue round and the embedding vectors matching the current dialogue round into an applied ranking model to predict the reordering score of each embedding vector, and then sort them according to the reordering scores of each embedding vector. This applied ranking model is RankGPT (an open-source GitHub project that specifically uses carefully designed prompts to reorder the input current dialogue round and the embedding vectors matching the current dialogue round).

[0048] Based on this, the above-mentioned input of the dialogue and the matching embedding vector of the dialogue into the large language model LLM may include: inputting the dialogue, the prompt word, the reordered embedding vector, and the explanation information of the knowledge point corresponding to the embedding vector into the LLM, so that the LLM can select the embedding vector corresponding to the knowledge point most relevant to the dialogue based on the prompt word.

[0049] As can be seen, this embodiment ultimately uses vector retrieval tools and LLM in combination to determine the above-mentioned second set of knowledge points, so as to improve the accuracy of the final determined set of second set of knowledge points.

[0050] Step 103: Based on LLM and hierarchical prompts, predict hierarchically starting from the first layer of the knowledge point tree structure to output the third set of knowledge points involved in each round of dialogue through the LLM; the third set of knowledge points includes at least one knowledge point; the number of knowledge point nodes in each layer of the knowledge point tree structure increases from the first layer.

[0051] In this embodiment, the knowledge system, such as K12, has a hierarchical structure (specifically, a knowledge point tree-like hierarchical structure). This knowledge point tree-like hierarchical structure is derived from the chapter order and the order of knowledge progression. For example, it progresses from the knowledge point nodes at the first level (denoted as the domain level, where each knowledge point node represents a knowledge point) to the knowledge point nodes at the next level (denoted as the cluster level) connected to the knowledge point nodes at the first level, and then from the knowledge point nodes at the cluster level to the knowledge point nodes at the next level (denoted as the standard level) connected to the knowledge point nodes at the cluster level. The number of knowledge point nodes increases at each level starting from the first level in the knowledge point tree-like hierarchical structure. Figure 4 An example is shown in the diagram of a hierarchical tree structure of knowledge points. In this embodiment, the hierarchical tree structure is obtained by pruning the existing knowledge system. The purpose is to reduce erroneous or overly long and complex child nodes, thereby enabling the most accurate annotation of knowledge points in the dialogue.

[0052] In this embodiment, step 103 may include the following steps:

[0053] For each round of dialogue, if the above dialogue text contains historical dialogues preceding that round, then:

[0054] Candidate knowledge points are determined from the first layer of the knowledge point tree structure, and the first layer of the knowledge point tree structure is taken as the current layer;

[0055] The LLM model is input into the current dialogue, historical dialogues, the meaning explanations of each candidate knowledge point at the current layer, and the prompt words matching the current layer. This allows the LLM model to select candidate knowledge points matching the current dialogue when the current layer is not the last layer of the knowledge point tree structure; when the current layer is the last layer of the knowledge point tree structure, it outputs knowledge points according to the format requirements, which are the knowledge points in the third knowledge point set; when the current layer is the last layer, the prompt words matching the current layer prompt the output of knowledge points that meet the format requirements; when the current layer is not the last layer, the prompt words matching the current layer prompt the output of candidate knowledge points in the current layer that match the current dialogue.

[0056] When the current layer is not the last layer of the knowledge point tree hierarchy, for each selected candidate knowledge point, extract the node on the next layer connected to the candidate knowledge point from the knowledge point tree hierarchy; determine the next layer as the current layer, and take the extracted node on the next layer as the candidate knowledge point on the current layer. Then return to the step of inputting the dialogue round and historical dialogue, the meaning explanation information of each candidate knowledge point on the current layer, and the hierarchical prompt words into the LLM model.

[0057] Of course, if there is no previous dialogue in the dialogue text, the following input of the current dialogue and the previous dialogue, the meaning explanation information of each candidate knowledge point in the current layer, and the prompt words matched in the current layer into the LLM model may include: inputting the current dialogue, the meaning explanation information of each candidate knowledge point in the current layer, and the prompt words matched in the current layer into the LLM model.

[0058] Step 104: Merge the first set of knowledge points, the second set of knowledge points, and the third set of knowledge points involved in each round of dialogue to obtain the fourth set of knowledge points involved in each round of dialogue; and obtain the mastery status of each knowledge point in the fourth set of knowledge points involved in each round of dialogue.

[0059] Optionally, in this embodiment, the first set of knowledge points, the second set of knowledge points, and the third set of knowledge points involved in each round of dialogue can be fused according to the union, intersection, voting, or weighting method, and the fourth set of knowledge points involved in each round of dialogue can be determined based on the fusion result.

[0060] Taking the union method as an example, the union of the first set of knowledge points, the second set of knowledge points, and the third set of knowledge points involved in each round of dialogue is determined as the fourth set of knowledge points involved in that round of dialogue.

[0061] Taking the intersection method as an example, the intersection of the first set of knowledge points, the second set of knowledge points, and the third set of knowledge points involved in each round of dialogue is determined as the fourth set of knowledge points involved in that round of dialogue.

[0062] Taking a weighted approach as an example, hyperparameters are used to assign weights to each method. For instance, the dialogue knowledge point prediction model is assigned the first weight, the RAG method the second weight, and the LLM and hierarchical prompt word Prompt method the third weight. Then, based on the weights assigned to each method, the predicted knowledge points are selected from each method to generate a fourth set of knowledge points involved in this round of dialogue. Here, the weight assigned to each method indicates the number of knowledge points predicted under that method.

[0063] Optionally, in step 103 above, LLM will further output the mastery status of each knowledge point in the third knowledge point set and the dialogue label for each round of dialogue. The dialogue label for any round of dialogue indicates whether the response in that round of dialogue is correct. Here, the mastery status of any knowledge point indicates whether the knowledge point has been mastered (which is determined based on the response content). This indicates that the student's response to the question related to the j-th round of dialogue is... The level of understanding of each knowledge point. For example, This indicates that the student has grasped the knowledge involved in the j-th round of dialogue. One knowledge point.

[0064] Define a dialogue sequence of teacher (t) – student (s) rounds, assuming the student initiates the dialogue, forming the following sequence. ,in, This represents the question raised by the teacher in the j-th dialogue. Let M represent the student's response in the j-th round of the dialogue, where M is the total number of rounds, and M is the number of rounds in the dialogue if the student initiates the conversation. It exists. Each round of dialogue has a dialogue tag. ,in ,2,...,M. mean right A correct response means that the student's answer in this round of dialogue is correct. This indicates that the student's response in that round of dialogue was vague, such as the topic deviating from the main point or the teacher providing emotional support. Each round of dialogue is also associated with the following set of KC (Knowledge Points): ,in It refers to the number of knowledge points covered in this round of dialogue. This represents all the knowledge points involved in the j-th round of dialogue.

[0065] Optionally, students need to master all the knowledge points involved in a round of dialogue in order to correctly answer the questions posed by the teacher in that round of dialogue. That is, formally, the probability of a student answering correctly in the j-th round of dialogue... It depends on the student's grasp of the various knowledge points involved in this round of dialogue.

[0066] Step 105: Generate dialogue training samples and knowledge point labels for each round of dialogue. The dialogue training samples for any round of dialogue include the questions in that round of dialogue and the previous historical dialogues.

[0067] In this embodiment, the knowledge point labels of the dialogue training sample corresponding to each round of dialogue include at least the fourth set of knowledge points of that round of dialogue (that is, the set of knowledge points involved in the dialogue training sample).

[0068] In this embodiment, as described below, the knowledge point labels of the dialogue training samples corresponding to each round of dialogue also include the dialogue labels of that round of dialogue, which are used to indicate whether the response content in that round of dialogue is correct, as specifically as described above.

[0069] Step 106: Train a target knowledge tracking model based on each dialogue training sample and the knowledge point labels of each dialogue training sample; the target knowledge tracking model is used to predict the degree to which knowledge points are mastered in each round of dialogue in the target dialogue text and / or whether the response content in each round of dialogue is correct when tracking knowledge in the target dialogue text.

[0070] In this embodiment, the target knowledge tracking model can be trained according to the KC fusion training method and based on the knowledge point labels of each dialogue training sample.

[0071] Here, the KC fusion training method involves inputting each dialogue training sample, along with the knowledge points involved (i.e., the set of knowledge points in the knowledge point tags of the dialogue training sample, such as the fourth knowledge point set), into a pre-trained LLM to obtain the mastery status of each knowledge point. The mastery status of each knowledge point is then aggregated to obtain the aggregation result corresponding to that dialogue training sample. Based on the aggregation result corresponding to each dialogue training sample and the dialogue tags of each dialogue training sample, a loss is calculated. The pre-trained LLM is then adjusted based on the loss to obtain the target knowledge tracking model. The following is a description:

[0072] In the KC fusion training method, for each dialogue training sample, the probability that the dialogue label of that training sample indicates the correctness of the response is modeled as the product of the mastery levels of all knowledge points in the knowledge point set involved in that dialogue training sample, that is: .

[0073] Under this premise, this embodiment utilizes supervised fine-tuning (SFT) of the pre-trained LLM using dialogue training samples to obtain the aforementioned target knowledge tracking model. Specifically, firstly, the probability of mastering each knowledge point in the knowledge point set involved in each dialogue training sample is modeled as follows:

[0074] .

[0075] Where θ represents the model parameters. and These represent the probabilities (logit) of the model returning "True" (indicating the knowledge point has been mastered) and "False" (indicating the knowledge point has not been mastered), respectively. In practice, to improve model efficiency, each knowledge point in the set of knowledge points involved in each dialogue training sample can be packaged into a prompt and input into the model. Attention masks and positional embeddings are used to ensure that the input knowledge points do not pay attention to each other, and that the probability of mastering each knowledge point is estimated independently.

[0076] Next, the probability of mastering each knowledge point in the knowledge point set involved in each dialogue training sample is aggregated to obtain the output labels for each round of dialogue, such as... .

[0077] Finally, loss is calculated using maximum likelihood estimation to fine-tune the pre-trained LLM, for example:

[0078] .

[0079] It should be noted that this method of fine-tuning a pre-trained LLM is relatively simple and has many advantages. The reason is that although it starts with a powerful pre-trained LLM, no new parameters are added to the pre-trained model, reducing the need for a large amount of training data. In addition, the sample data input during training in this embodiment is text content, rather than questions / KC embeddings that need to be learned from scratch, which is more suitable for new knowledge points that cannot be seen during training.

[0080] Ultimately, a target knowledge tracking model will be trained based on the KC fusion training method.

[0081] As another embodiment, this embodiment can also train the target knowledge tracking model according to the PRM training method and based on the knowledge point labels of each dialogue training sample.

[0082] Specifically, in this embodiment, a dialogue sequence of teacher (denoted by t) -- student (denoted by s) rounds is defined, assuming the original dialogue text is as follows: The resulting dialogue training samples are as follows:

[0083] t1 + kc11 + •+ kc12 + •+ ……

[0084] t1 + s1 + t2 + kc21 + •+ kc22 + •+ ……

[0085] t1 + s1 + t2 + s2 + t3 + kc31 + •+ kc32 + •+ ……

[0086] ...

[0087] Here, • represents the probability that each knowledge point to be predicted has been mastered. In each dialogue training sample, • can be empty or a specified value.

[0088] In the PRM training method, for each dialogue training sample, the dialogue training sample and the knowledge points involved in the dialogue training sample are input into a pre-trained LLM to obtain the mastery status of each knowledge point. Based on the mastery status of each knowledge point and the dialogue label in the knowledge point label of the dialogue training sample, a loss is calculated. The pre-trained LLM is adjusted based on this loss to obtain the target knowledge tracking model. Finally, the target knowledge tracking model can be trained according to the PRM training method.

[0089] It should be noted that the above PRM training method mainly focuses on splitting the dialogue to generate dialogue training samples corresponding to each round of dialogue. This embodiment can also focus on the full dialogue format to generate only one overall dialogue training sample. For example, t1 + kc11 + • + kc12 + • + …… + s1 + t2 + kc21 + • + kc22 + • + …… + s2 + t3 + ……. Then, this dialogue training sample is input into the pre-trained LLM to obtain the mastery status of each knowledge point involved in each round of dialogue in the dialogue training sample, such as the probability of mastering each knowledge point. Then, based on the mastery status of each knowledge point involved in each round of dialogue in the dialogue training sample and the dialogue labels of each round of dialogue, the loss is calculated. Based on the calculated losses, the pre-trained LLM is adjusted to obtain the target knowledge tracking model.

[0090] After training the target knowledge tracking model, when tracking knowledge in a target dialogue text, the model can be fused based on the knowledge point sets involved in each round of dialogue in the target dialogue text predicted by the dialogue knowledge point prediction model, RAG, LLM, and Prompt, respectively. Based on the fusion results, the model can predict the degree to which knowledge points are mastered in each round of dialogue and / or whether the responses in each round are correct. Examples will be provided below.

[0091] This concludes the process. Figure 1 The process is shown below.

[0092] pass Figure 1 As can be seen from the process shown, this embodiment uses multiple methods, such as dialogue knowledge point prediction model, retrieval enhancement generation and large language model LLM hierarchical prompts, to predict dialogue-related knowledge points. Then, the knowledge points predicted by each method are fused to track dialogue-related knowledge points, reduce the bias of a single method, and improve the confidence of dialogue tracking.

[0093] Furthermore, in this embodiment, based on multiple methods such as dialogue knowledge point prediction model, three methods—enhanced generation and hierarchical prompting of large language model LLM—are retrieved to predict dialogue-related knowledge points. Then, the knowledge points predicted by each method are fused to obtain a fourth set of knowledge points involved in the dialogue. Using the fourth set of knowledge points of each dialogue, as well as the historical dialogues before each dialogue and the questions in each dialogue, an autoregressive target knowledge tracking model is trained to ensure that when the target knowledge tracking model is used to track knowledge of the conversation in a timely manner, the changes of knowledge points in multiple rounds of dialogue are captured in time, thereby improving the accuracy of knowledge tracking.

[0094] After training the target knowledge tracing model, knowledge tracing can be performed based on the target knowledge tracing model, specifically as follows: Figure 5 As shown:

[0095] See Figure 5 , Figure 5 Another method flowchart provided for an embodiment of this application. (See attached flowchart.) Figure 5 As shown, the method may include the following steps:

[0096] Step 501: Based on the dialogue knowledge point prediction model, predict the set of fifth knowledge points involved in each round of dialogue in the current target dialogue text.

[0097] Step 502: Match each round of dialogue with knowledge points in the knowledge base based on RAG to predict the set of sixth knowledge points involved in each round of dialogue.

[0098] Step 503: Based on LLM and hierarchical prompt words Prompt, hierarchical prediction is performed starting from the first layer of the knowledge point tree hierarchy to output the seventh set of knowledge points involved in each round of dialogue through the LLM.

[0099] Step 504: Merge the sets of fifth, sixth, and seventh knowledge points involved in each round of dialogue to obtain the set of eighth knowledge points involved in each round of dialogue.

[0100] Steps 501 to 504 are similar to steps 101 to 104 described above.

[0101] Step 505: For each round of dialogue, input the dialogue, the set of eighth knowledge points involved in the dialogue, and the historical dialogues before the dialogue into the target knowledge tracking model to predict the degree of mastery of each knowledge point in the set of eighth knowledge points involved in the dialogue and / or predict whether the content of the response in the dialogue is correct.

[0102] Ultimately, through Figure 5 It enables the tracking of knowledge points related to the dialogue.

[0103] pass Figure 5 As can be seen from the process shown, in this embodiment, based on multiple methods such as dialogue knowledge point prediction model, retrieval enhancement generation and large language model LLM hierarchical prompts, the knowledge points related to the dialogue are predicted. Then, the knowledge points predicted by each method are fused to obtain the knowledge point set involved in the dialogue. By using each dialogue, the knowledge point set of each dialogue, such as the eighth knowledge point set mentioned above, and the historical dialogues before each dialogue, the changes in knowledge points in multiple rounds of dialogue are captured in a timely manner when the knowledge of the conversation is tracked, thereby improving the accuracy of knowledge tracking.

[0104] The methods provided in the embodiments of this application have been described above. The apparatus provided in the embodiments of this application is described below:

[0105] See Figure 6 , Figure 6A structural diagram of a device provided in an embodiment of this application. The device includes:

[0106] The first prediction unit is used to predict the first set of knowledge points involved in each round of dialogue in the dialogue text based on the dialogue knowledge point prediction model; the first set of knowledge points includes at least one knowledge point; a round of dialogue includes at least a question and the answer to the question.

[0107] The second prediction unit matches each round of dialogue with knowledge points in the knowledge base based on retrieval enhancement-generated RAG to predict the second set of knowledge points involved in each round of dialogue; the second set of knowledge points includes at least one knowledge point.

[0108] The third prediction unit, based on LLM and hierarchical prompt words, makes hierarchical predictions starting from the first layer of the knowledge point tree structure, so as to output the third set of knowledge points involved in each round of dialogue through the LLM; the third set of knowledge points includes at least one knowledge point; the number of knowledge point nodes in each layer of the knowledge point tree structure increases from the first layer.

[0109] The fusion unit is used to fuse the first set of knowledge points, the second set of knowledge points, and the third set of knowledge points involved in each round of dialogue to obtain the fourth set of knowledge points involved in each round of dialogue; and to obtain the mastery status of each knowledge point in the fourth set of knowledge points involved in each round of dialogue.

[0110] A processing unit is configured to generate a dialogue training sample corresponding to each round of dialogue and knowledge point labels for that dialogue training sample. The dialogue training sample for any round of dialogue includes the question in that round of dialogue and previous historical dialogues. The knowledge point labels for each round of dialogue training sample include at least the fourth set of knowledge points from that round of dialogue.

[0111] Based on each dialogue training sample and the knowledge point labels of each dialogue training sample, a target knowledge tracking model is trained. The target knowledge tracking model is used to perform knowledge tracking on the target dialogue text. Based on the fusion result of the knowledge point set involved in each round of dialogue in the target dialogue text predicted by the dialogue knowledge point prediction model, RAG, LLM and Prompt respectively, the model predicts the degree to which the knowledge points in each round of dialogue in the target dialogue text are mastered and / or whether the content of the response in each round of dialogue is correct.

[0112] Optionally, the dialogue knowledge point prediction model includes a recall model and a ranking model; the first set of knowledge points predicted in each round of dialogue text based on the dialogue knowledge point prediction model includes:

[0113] For each round of dialogue, if the dialogue text contains historical dialogues preceding that round, then:

[0114] The current dialogue, historical dialogues, and knowledge points from the knowledge base are all input into the recall model to obtain N knowledge points;

[0115] The current dialogue and historical dialogues, as well as N knowledge points and their meaning explanations, are input into the fine-ranking model so that the fine-ranking model outputs at least one accurate knowledge point based on its understanding of the dialogue content and the input knowledge points. The output at least one knowledge point is a knowledge point in the first set of knowledge points.

[0116] Optionally, the step of matching each round of dialogue with knowledge points in the knowledge base based on retrieval enhancement to predict the second set of knowledge points involved in each round of dialogue includes:

[0117] For each round of dialogue, a vector retrieval tool is used to retrieve the embedding vector that matches the dialogue in the vector knowledge base. The vector knowledge base includes the embedding vector corresponding to each knowledge point in the knowledge base. The embedding vector corresponding to any knowledge point is obtained by inputting the Chunks text block obtained by concatenating the meaning explanation information of the knowledge point into the embedding model.

[0118] The dialogue round and the embedding vector are input into the Large Language Model (LLM) to obtain the knowledge points involved in the dialogue round as output by the LLM. At least one knowledge point in the output is a knowledge point in the second knowledge point set.

[0119] Optionally, the vector retrieval tool is obtained by fine-tuning an existing embedding model through contrastive learning as follows:

[0120] Input the triplet sample data (q, a+, a-) into the existing embedding model, where q is a round of sample dialogue, a+ is the knowledge point sample that matches q, and a- is the knowledge point sample that does not match q. Fine-tune the existing embedding model based on the output results and the contrastive loss function so that the fine-tuned embedding model meets the following requirements: narrow the spatial distance between q and a+, and widen the distance between q and a-.

[0121] Optionally, the LLM-based and hierarchical prompt word Prompt method, which predicts hierarchically starting from the first layer of the knowledge point tree structure, to output a third set of knowledge points involved in each round of dialogue through the LLM, includes:

[0122] For each round of dialogue, if the dialogue text contains historical dialogues preceding that round, then:

[0123] Candidate knowledge points are determined from the first layer of the knowledge point tree structure, and the first layer of the knowledge point tree structure is taken as the current layer;

[0124] The LLM model is input into the current dialogue, historical dialogues, the meaning explanations of each candidate knowledge point at the current layer, and the prompt words matching the current layer. This allows the LLM model to select candidate knowledge points matching the current dialogue when the current layer is not the last layer of the knowledge point tree structure; when the current layer is the last layer of the knowledge point tree structure, it outputs knowledge points according to the format requirements, which are the knowledge points in the third knowledge point set; when the current layer is the last layer, the prompt words matching the current layer prompt the output of knowledge points that meet the format requirements; when the current layer is not the last layer, the prompt words matching the current layer prompt the output of candidate knowledge points in the current layer that match the current dialogue.

[0125] When the current layer is not the last layer of the knowledge point tree hierarchy, for each selected candidate knowledge point, extract the node on the next layer connected to the candidate knowledge point from the knowledge point tree hierarchy; determine the next layer as the current layer, and take the extracted node on the next layer as the candidate knowledge point on the current layer. Then return to the step of inputting the dialogue round and historical dialogue, the meaning explanation information of each candidate knowledge point on the current layer, and the hierarchical prompt words into the LLM model.

[0126] Optionally, the step of outputting the third set of knowledge points involved in each round of dialogue through the LLM further includes: the LLM further outputting the mastery status of each knowledge point in the third set of knowledge points and the dialogue tags of that round of dialogue;

[0127] The process of obtaining the mastery status of each knowledge point in the fourth knowledge point set involved in each round of dialogue and the dialogue tags for that round of dialogue includes:

[0128] For each knowledge point in the fourth set of knowledge points involved in each round of dialogue, if the knowledge point also exists in the third set of knowledge points involved in the same round of dialogue, the mastery status of the knowledge point and the dialogue tag of the same round of dialogue are obtained from the output of the LLM.

[0129] Optionally, the fusion of the first set of knowledge points, the second set of knowledge points, and the third set of knowledge points involved in each round of dialogue to obtain the fourth set of knowledge points involved in each round of dialogue includes:

[0130] The first, second, and third sets of knowledge points involved in each round of dialogue are merged using union, intersection, voting, or weighting methods. Based on the fusion results, the fourth set of knowledge points involved in each round of dialogue is determined.

[0131] Optionally, the knowledge point labels of the dialogue training samples corresponding to each round of dialogue also include the dialogue labels of that round of dialogue, which are used to indicate whether the content of the response in that round of dialogue is correct;

[0132] The training target knowledge tracking model based on the knowledge point labels of each dialogue training sample includes:

[0133] The target knowledge tracking model is trained using the KC fusion training method and based on the knowledge point labels of each dialogue training sample. The KC fusion training method is as follows: for each dialogue training sample, the dialogue training sample and the knowledge points involved in the dialogue training sample are input into the pre-trained LLM to obtain the mastery status of each knowledge point. The mastery status of each knowledge point is aggregated to obtain the aggregation result corresponding to the dialogue training sample. The loss is calculated based on the aggregation result corresponding to each dialogue training sample and the dialogue labels in the knowledge point labels of each dialogue training sample. The pre-trained LLM is adjusted based on the loss to obtain the target knowledge tracking model.

[0134] Alternatively, the target knowledge tracking model can be trained using the PRM training method, based on the knowledge point labels of each dialogue training sample. The PRM training method involves inputting the dialogue training sample and the knowledge points involved in the dialogue training sample into a pre-trained LLM to obtain the mastery status of each knowledge point. Based on the mastery status of each knowledge point and the dialogue labels in the knowledge point labels of the dialogue training sample, a loss is calculated. Based on the loss, the pre-trained LLM is adjusted to obtain the target knowledge tracking model.

[0135] This concludes the process. Figure 6 Structural description of the device shown.

[0136] This application also provides another device structure diagram in its embodiments. See Figure 7 , Figure 7 Another device structural diagram provided for an embodiment of this application. This device corresponds to… Figure 5 The process is shown below.

[0137] like Figure 7 As shown, the device may include:

[0138] A knowledge fusion unit is used to predict the set of fifth knowledge points involved in each round of dialogue in the current target dialogue text based on a dialogue knowledge point prediction model; and,

[0139] Based on RAG, each round of dialogue is matched with knowledge points in the knowledge base to predict the set of sixth knowledge points involved in each round of dialogue; and,

[0140] Based on LLM and hierarchical prompts, prediction is performed hierarchically starting from the first layer of the knowledge point tree structure to output the seventh set of knowledge points involved in each round of dialogue through the LLM; and,

[0141] The sets of fifth, sixth, and seventh knowledge points involved in each round of dialogue are merged to obtain the set of eighth knowledge points involved in each round of dialogue.

[0142] The knowledge tracking unit is used to input the current dialogue, the set of eighth knowledge points involved in the current dialogue, and the historical dialogues preceding the current dialogue into the target knowledge tracking model for each round of dialogue, in order to predict the degree to which each knowledge point in the set of eighth knowledge points involved in the current dialogue is mastered, and / or predict whether the content of the response in the current dialogue is correct.

[0143] This concludes the process. Figure 7 Structural description of the device shown.

[0144] This application also provides embodiments that... Figure 6 or Figure 7 The hardware structure of the device shown. See also Figure 8 , Figure 8 This is a structural diagram of an electronic device provided in an embodiment of this application. Figure 8 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0145] Based on the same application concept as the above method, this application embodiment also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.

[0146] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0147] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A method for tracking a dialog-related knowledge point, characterized by, The method comprises: predicting, based on a dialogue knowledge point prediction model, a first knowledge point set involved in each round of dialogue in dialogue text; one round of dialogue at least comprises a question and a reply content of the question; generating, based on retrieval enhancement, a RAG to match each round of dialogue and a knowledge point in a knowledge base to predict a second knowledge point set involved in each round of dialogue; For each round of dialogue, if there is a historical dialogue before the round of dialogue in the dialogue text, then: determine candidate knowledge points from the first layer of a knowledge point tree structure, the number of knowledge point nodes on each layer of the knowledge point tree structure starting from the first layer increases; take the first layer of the knowledge point tree structure as the current layer; input the round of dialogue and the historical dialogue, the meaning explanation information of each candidate knowledge point on the current layer, and the matching prompt word of the current layer into the LLM model, so that the LLM model outputs a third knowledge point set according to the format requirement when the current layer is the last layer of the knowledge point tree structure; when the current layer is not the last layer of the knowledge point tree structure, select a candidate knowledge point matching the round of dialogue from the current layer; for each selected candidate knowledge point, extract the nodes on the next layer connected to the candidate knowledge point from the knowledge point tree structure; determine the next layer as the current layer, and take the extracted nodes on the next layer as the candidate knowledge points on the current layer, and return to the step of inputting the round of dialogue and the historical dialogue, the meaning explanation information of each candidate knowledge point on the current layer, and the hierarchical prompt word into the LLM model; fuse the first knowledge point set, the second knowledge point set and the third knowledge point set involved in each round of dialogue to obtain a fourth knowledge point set involved in each round of dialogue; obtain the mastery of each knowledge point in the fourth knowledge point set involved in each round of dialogue; and generate a dialogue training sample corresponding to each round of dialogue and a knowledge point label of the dialogue training sample, wherein the dialogue training sample corresponding to any round of dialogue comprises a question in the round of dialogue and a previous historical dialogue; the knowledge point label of the dialogue training sample corresponding to each round of dialogue at least comprises the fourth knowledge point set of the round of dialogue; train a target knowledge tracking model based on the dialogue training samples and the knowledge point labels of the dialogue training samples; the target knowledge tracking model is used to predict the degree of mastery of knowledge points in each round of dialogue and / or whether the reply content in each round of dialogue is correct based on the fusion result of the knowledge point sets involved in each round of dialogue in the target dialogue text predicted by the dialogue knowledge point prediction model, the RAG and the LLM and Prompt respectively when performing knowledge tracking on the target dialogue text.

2. The method of claim 1, wherein, The dialogue knowledge point prediction model comprises a recall model and a precision model; the dialogue knowledge point prediction model comprises: For each round of dialogue, if there is a historical dialogue before the round of dialogue in the dialogue text, then: input the round of dialogue and the historical dialogue, and the knowledge points in the knowledge base into the recall model to obtain N knowledge points; inputting the round of dialogue and the historical dialogue, and the N knowledge points and the meaning explanation information of the N knowledge points into a fine arrangement model, so that the fine arrangement model outputs accurate at least one knowledge point based on understanding of the dialogue content and the input knowledge points, and the output at least one knowledge point is a knowledge point in the first knowledge point set.

3. The method of claim 1, wherein, The RAG based on retrieval enhancement matches each round of dialogue and the knowledge points in the knowledge base to predict a second knowledge point set involved in each round of dialogue, which includes: For each round of dialogue, a vector retrieval tool is used to retrieve an embedding vector matching the round of dialogue in a vector knowledge base; the vector knowledge base includes an embedding vector corresponding to each knowledge point in the knowledge base; and the embedding vector corresponding to any knowledge point is obtained by inputting a Chunks text block obtained by concatenating the meaning explanation information of the knowledge point into an embedding model; inputting the round of dialogue and the embedding vector into a large language model (LLM) to obtain a knowledge point involved in the round of dialogue output by the LLM, and the output at least one knowledge point is a knowledge point in the second knowledge point set.

4. The method of claim 3, wherein, The vector retrieval tool is obtained by fine-tuning an existing embedding model in a contrast learning manner as follows: inputting a triple sample data (q, a+, a-) into the existing embedding model, where q is a round of sample dialogue, a+ is a knowledge point sample matching q, and a- is a knowledge point sample not matching q; and fine-tuning the existing embedding model based on an output result and a contrast loss function, so that the fine-tuned embedding model meets the following requirements: narrowing the spatial distance between q and a+, and widening the distance between q and a-.

5. The method of claim 1, wherein, When the current layer is the last layer, the prompt word matched by the current layer prompts output of a knowledge point meeting the format requirement; and when the current layer is not the last layer, the prompt word matched by the current layer prompts output of a candidate knowledge point matching the round of dialogue in the current layer.

6. The method according to claim 1 or 5, characterized in that, The LLM further outputs a mastery situation of each knowledge point in the third knowledge point set and a dialogue label of the round of dialogue. The obtaining of the mastery situation of each knowledge point in the fourth knowledge point set involved in each round of dialogue includes: For each knowledge point in the fourth knowledge point set involved in each round of dialogue, if the knowledge point also exists in the third knowledge point set involved in the round of dialogue, the mastery situation of the knowledge point and the dialogue label of the round of dialogue are obtained from the output result of the LLM.

7. The method of claim 1, wherein, The fusion of the first knowledge point set, the second knowledge point set and the third knowledge point set involved in each round of dialogue to obtain the fourth knowledge point set involved in each round of dialogue includes: The first knowledge point set, the second knowledge point set and the third knowledge point set involved in each round of dialogue are fused in a union, intersection, voting or weighted manner, and the fourth knowledge point set involved in each round of dialogue is determined based on the fusion result.

8. The method of claim 1, wherein, The knowledge point label of the dialogue training sample corresponding to each round of dialogue further includes a dialogue label of the round of dialogue, used to indicate whether the reply content in the round of dialogue is correct; The training of the target knowledge tracking model based on the knowledge point labels of the dialogue training samples includes: According to a KC fusion training manner, and based on the knowledge point labels of the dialogue training samples, the target knowledge tracking model is trained; the KC fusion training manner is that, for each dialogue training sample, the dialogue training sample and each knowledge point involved in the dialogue training sample are input into a pre-trained LLM to obtain a mastering situation of each knowledge point being mastered, and the mastering situation of each knowledge point being mastered is aggregated to obtain an aggregation result corresponding to the dialogue training sample; a loss is calculated based on the aggregation result corresponding to each dialogue training sample and the dialogue label in the knowledge point label of each dialogue training sample, and the pre-trained LLM is adjusted based on the loss to obtain the target knowledge tracking model; Alternatively, according to a PRM training manner, and based on the knowledge point labels of the dialogue training samples, the target knowledge tracking model is trained; the PRM training manner is that, for each dialogue training sample, the dialogue training sample and each knowledge point involved in the dialogue training sample are input into a pre-trained LLM to obtain a mastering situation of each knowledge point being mastered, a loss is calculated based on the mastering situation of each knowledge point being mastered and the dialogue label in the knowledge point label of the dialogue training sample, and the pre-trained LLM is adjusted based on the loss to obtain the target knowledge tracking model.

9. A method for tracking dialogue-related knowledge points, characterized in that, The method includes: Based on the dialogue knowledge point prediction model, a fifth knowledge point set involved in each round of dialogue in the current target dialogue text is predicted; Based on the RAG, each round of dialogue and the knowledge points in the knowledge base are matched to predict a sixth knowledge point set involved in each round of dialogue; For each round of dialogue, if there is a historical dialogue before the round of dialogue in the dialogue text, a candidate knowledge point is determined from a first layer of a knowledge point tree-like hierarchical structure, the number of knowledge point nodes on each layer of the knowledge point tree-like hierarchical structure increases from the first layer; the first layer of the knowledge point tree-like hierarchical structure is taken as a current layer; the dialogue text, the meaning explanation information of each candidate knowledge point on the current layer, and the prompt words matched in the current layer are input into the LLM model, so that the LLM model outputs a seventh knowledge point set according to the format requirement when the current layer is the last layer of the knowledge point tree-like hierarchical structure; when the current layer is not the last layer of the knowledge point tree-like hierarchical structure, a candidate knowledge point matched with the round of dialogue is selected from the current layer; for each selected candidate knowledge point, a node on a next layer connected with the candidate knowledge point is extracted from the knowledge point tree-like hierarchical structure; the next layer is determined as the current layer, and the extracted node on the next layer is taken as a candidate knowledge point on the current layer, and the step of inputting the dialogue text, the meaning explanation information of each candidate knowledge point on the current layer, and the hierarchical prompt words into the LLM model is returned; Fifth, sixth and seventh knowledge point sets involved in each round of dialogue are fused to obtain an eighth knowledge point set involved in each round of dialogue; For each round of dialogue, the round of dialogue, the eighth knowledge point set involved in the round of dialogue and the historical dialogue before the round of dialogue are input into the target knowledge tracking model to predict the degree of mastery of each knowledge point in the eighth knowledge point set involved in the round of dialogue and / or predict whether the reply content in the round of dialogue is correct.

10. An electronic device, comprising: The electronic device includes a processor and a machine-readable storage medium; The machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is configured to execute the machine-executable instructions to implement the method of any one of claims 1-9.

Images