Knowledge content source tracing method, device and equipment and readable storage medium

Abstract

The invention discloses a root tracing method and device for knowledge content, equipment and a readable storage medium. The method comprises the following steps: positioning at least one piece of difference information from target answer content uploaded by a target user; the root cause classification of each piece of difference information is determined, the root cause classification is the error type to which the information corresponding to the point losing point belongs, and the error type comprises information known and information unknown; and / or the error type comprises information missing and information redundancy; the root cause tracing result of the difference information under the error type is determined through the large language model, and the root cause tracing result of the difference information can be displayed to the user, so that the user can perform improvement based on the root cause tracing result of the difference information, and the analysis ability of the user is helped to be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and specifically to a method, apparatus, device, and readable storage medium for tracing the origins of knowledge content. Background Technology

[0002] Subjective questions are those without a single correct answer, requiring users to express their opinions or analyze problems independently. They reflect the comprehensive qualities, logical thinking, language organization, and innovation abilities of users who answer subjective questions, and are an indispensable part of evaluating users who answer subjective questions.

[0003] Currently, intelligent analysis systems can proactively analyze answers to subjective questions. However, these systems can only perform grammatical correction and automatic scoring, but cannot provide analysis or feedback. This prevents users from improving their answers based on the system's output, thus affecting the system's practical application effectiveness. Summary of the Invention

[0004] This application provides a method, apparatus, device, and readable storage medium for tracing the origins of knowledge content. It can display the root cause tracing results of discrepancies to users, enabling them to make improvements based on these results and enhance their analytical capabilities.

[0005] Firstly, embodiments of this application provide a method for tracing the origins of knowledge content, the method comprising: Identify at least one discrepancy from the target answer content uploaded by the target user, wherein the discrepancy is the information corresponding to the point deduction in the target answer content; Determine the root cause classification for each of the above-mentioned discrepancies, wherein the root cause classification is the error type to which the information corresponding to the above-mentioned point loss belongs, and the error type includes known information and unknown information; and / or, the error type includes missing information and redundant information, wherein the above-mentioned known information indicates that the target user knows the information corresponding to the above-mentioned point loss, the above-mentioned unknown information indicates that the target user does not know the information corresponding to the above-mentioned point loss, the above-mentioned missing information indicates that the information corresponding to the above-mentioned point loss is missing information in the target answer content, and the above-mentioned redundant information indicates that the information corresponding to the above-mentioned point loss is redundant information in the target answer content. The root cause tracing results of the above-mentioned discrepancies under the above-mentioned error types were determined by using a large language model.

[0006] Optionally, at least one discrepancy information is located from the target user's uploaded target answer content, including: Based on the above target answer content, generate the first cognitive expression path of the above target user on the question content corresponding to the above target answer content; By comparing the cognitive differences between the first cognitive expression path and the second cognitive expression path of the reference content, the above difference information is obtained. The cognitive expression path is a logical reasoning chain consisting of entity nodes corresponding to each knowledge point and logical edges connecting each of the above entity nodes.

[0007] Optionally, determining the target user's first cognitive expression path regarding the question content corresponding to the target answer content, based on the target answer content, includes: Extract at least one primary knowledge point from the above target answer content and the logical relationship between each of the above primary knowledge points; Based on the first knowledge point and the above logical relationship, we obtain the first cognitive expression path, which includes the entity nodes corresponding to each of the first knowledge points and the logical edges corresponding to each logical relationship.

[0008] Optionally, the root cause tracing results of determining the aforementioned discrepancies under the aforementioned error types using a large language model include: If the root cause classification of the above discrepancy information includes known information and missing information, a content supplementation instruction is generated to obtain the supplementary content input by the target user in response to the above content supplementation instruction. Using a large language model, based on the above supplementary information, we determine the root cause tracing results for the information corresponding to the points lost, both when the above information is known and when the above information is missing.

[0009] Optionally, the above-mentioned supplementary instructions include target questions and / or prompts, and the questioning method for the target questions includes counterfactual questions.

[0010] Optionally, the root cause tracing results of determining the aforementioned discrepancies under the aforementioned error types using a large language model include: The target data features of the target user are determined by using a large language model based on historical learning data and at least one of the target answer content. The root cause tracing results of the information corresponding to the lost points under the error type are then determined based on the target data features. The target data features are used to characterize the cognitive state and / or thinking pattern defects of the target user.

[0011] Optionally, the fine-tuning process of the aforementioned large language model includes: Obtain training samples, which include sample response content and difference labels of the sample response content; Based on the above sample answer content, the large language model generates sample difference information for the above sample answer content and generates the sample root cause tracing result of the above sample difference information under the sample root cause classification. The above sample difference information is the information corresponding to the points lost in the above sample answer content, and the above sample root cause classification is the error type to which the sample difference information belongs. Based on the above difference labels and the above sample difference information, the first loss function value corresponding to the difference information identification task is determined, and the second loss function value corresponding to the root cause tracing task is determined based on the above sample root cause tracing results. The large language model is fine-tuned based on the first loss function value, the first weight corresponding to the first loss function value, the second loss function value, and the second weight corresponding to the second loss function value.

[0012] Optionally, the aforementioned second cognitive expression path is obtained through a knowledge graph, wherein the knowledge graph includes entity nodes corresponding to knowledge points in at least one knowledge domain.

[0013] Optionally, this embodiment also includes: Based on the above discrepancy information and / or the above root cause tracing results, a learning path is generated for the above target users.

[0014] Secondly, embodiments of this application also provide a knowledge tracing device, the device comprising: The positioning module is used to locate at least one difference information from the target answer content uploaded by the target user, wherein the difference information is the information corresponding to the point loss in the target answer content; The determination module is used to determine the root cause classification of each of the above-mentioned discrepancies, wherein the root cause classification is the error type to which the information corresponding to the above-mentioned point loss belongs, and the error type includes known information and unknown information; and / or, the error type includes missing information and redundant information, wherein known information indicates that the target user knows the information corresponding to the above-mentioned point loss, unknown information indicates that the target user does not know the information corresponding to the above-mentioned point loss, missing information indicates that the information corresponding to the above-mentioned point loss is missing information in the target answer content, and redundant information indicates that the information corresponding to the above-mentioned point loss is redundant information in the target answer content; and the root cause tracing result of the above-mentioned discrepancies under the above-mentioned error types is determined by a large language model.

[0015] Thirdly, embodiments of this application also provide an electronic device, including a memory storing multiple instructions; the processor loads instructions from the memory to execute any of the knowledge content tracing methods provided in embodiments of this application.

[0016] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute any of the knowledge content tracing methods provided in embodiments of this application.

[0017] In this embodiment, at least one discrepancy information is located from the target answer content uploaded by the target user, wherein the discrepancy information is the information corresponding to the point deduction in the target answer content; the root cause classification of each discrepancy information is determined, wherein the root cause classification is the error type to which the information corresponding to the point deduction belongs, and the error type includes known information and unknown information; and / or, the error type includes missing information and redundant information, wherein known information indicates that the target user knows the information corresponding to the point deduction, unknown information indicates that the target user does not know the information corresponding to the point deduction, missing information indicates that the information corresponding to the point deduction is missing information in the target answer content, and redundant information indicates that the information corresponding to the point deduction is redundant information in the target answer content; the root cause tracing result of the discrepancy information under the error type is determined through a large language model, and the root cause tracing result of the discrepancy information is displayed so that users can make improvements based on the root cause tracing result of the discrepancy information, thereby helping to improve the user's analytical capabilities. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the knowledge point processing system provided in an embodiment of this application; Figure 2 This is a schematic flowchart of one embodiment of the method for tracing the source of knowledge content provided in this application. Figure 3 This is a schematic diagram of root cause classification provided in the embodiments of this application; Figure 4 This is another schematic diagram of root cause classification provided in the embodiments of this application; Figure 5 This is a schematic diagram of the first cognitive expression path and differential information provided in the embodiments of this application; Figure 6 This is a schematic diagram of the first cognitive expression path and the second cognitive expression path provided in the embodiments of this application; Figure 7 This is another schematic diagram of the first cognitive expression path and the second cognitive expression path provided in the embodiments of this application; Figure 8 This is a schematic diagram of the phrase structure tree provided in the embodiments of this application; Figure 9 This is a schematic diagram of the dependency tree provided in the embodiments of this application; Figure 10This is a schematic diagram of the data characteristics of the root cause tracing results provided in the embodiments of this application; Figure 11 This is a schematic diagram of the sub-knowledge graph provided in the embodiments of this application; Figure 12 This is a schematic diagram of the structure of the knowledge tracing device provided in the embodiments of this application; Figure 13 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Before providing a detailed explanation of the embodiments of this application, some terms involved in the embodiments of this application will be explained.

[0022] In the description of the embodiments of this application, the terms "first," "second," etc., may be used herein to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0023] This application provides a method, apparatus, device, and readable storage medium for tracing the source of knowledge content. Specifically, the method for tracing the source of knowledge content in this application can be executed by an electronic device, which can be a terminal or a server. The terminal can be a smartphone, tablet, laptop, touch screen, game console, personal computer (PC), personal digital assistant (PDA), or other terminal device. The terminal can also include a client, which can be a game application client, a browser client carrying a game program, or an instant messaging client. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0024] For example, such as Figure 1 As shown, the electronic device is illustrated using terminal 10 and server 20 as an example. Terminal 10 can display a content interface including the target question. In response to input operations on the content interface, it obtains the target answer content for the target question input by the target user. In response to an answer upload operation for at least one target answer content of the target user, it sends the target answer content to server 20. Server 20 determines a first cognitive expression path of the target answer content based on the target answer content. Based on the first cognitive expression path and a second cognitive expression path of the reference content, it determines the difference information and returns the difference information to terminal 10. Terminal 10 displays the difference information. In response to a linkage operation for any difference information, terminal 10 sends a linkage instruction to server 20. Server 20 determines the root cause tracing result of the difference information and returns the root cause tracing result to terminal 10. Terminal 10 displays the root cause tracing result of the difference information.

[0025] With the development of artificial intelligence, intelligent analysis systems can proactively correct grammatical errors and automatically score answers to subjective questions. However, due to limitations in user thinking, educational background, learning situation, and / or limited information sources, users may have cognitive limitations. This can lead to user-submitted opinions or answers lacking certain knowledge points, which the user may not be aware of, or even if the user knows certain knowledge points, they may not know how to apply them. Consequently, the user may not understand why their input differs from the reference answer. If the intelligent analysis system only performs grammatical error correction and automatic scoring, it cannot help users identify knowledge points they haven't covered, are unaware of, or are unaware of but are crucial to supporting their conclusions, or it cannot teach them how to use certain knowledge points. This prevents users from understanding why their input differs from the reference answer, hindering their self-improvement and impacting the practical application effectiveness of the intelligent analysis system.

[0026] To address this technical problem, this embodiment identifies at least one discrepancy information from the target answer content uploaded by the target user. This discrepancy information corresponds to the information that resulted in point deductions in the target answer content. The root cause classification of each discrepancy information is determined. This root cause classification is based on the error type to which the information corresponding to the point deduction belongs. Error types include known information and unknown information; and / or, error types include missing information and redundant information. Known information indicates that the target user is aware of the information corresponding to the point deduction; unknown information indicates that the target user is unaware of the information corresponding to the point deduction; missing information indicates that the information corresponding to the point deduction is missing from the target answer content; and redundant information indicates that the information corresponding to the point deduction is redundant from the target answer content. The root cause tracing results of the discrepancy information under each error type are determined using a large language model. This visualizes the root cause tracing results of the discrepancy information, allowing users to intuitively view them. This enables users to identify knowledge points they are unaware of or unaware of, and / or knowledge points they know but haven't used, as well as the reasons for the errors. This allows users to improve their abilities based on the discrepancy information and the reasons for the errors, thereby enhancing the practical application effect of the intelligent analysis system.

[0027] The following is a detailed description in conjunction with the accompanying drawings. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the drawings.

[0028] In this embodiment, a terminal is used as an example for illustration. This embodiment provides a method for tracing the origins of knowledge content, such as... Figure 2 As shown, the specific process of tracing the origins of this knowledge content can be as follows: 201. Locate at least one discrepancy information from the target answer content uploaded by the target user, wherein the discrepancy information is the information corresponding to the points lost in the target answer content.

[0029] It is understood that the target user can interact with the system on the graphical user interface. In response to the upload operation of at least one target answer content for the target user, the system displays at least one difference information located. Alternatively, after the target user uploads the target answer content, in response to the difference analysis operation of the target answer content, at least one difference information is located from the target answer content uploaded by the target user. This embodiment does not limit this.

[0030] The target user can be any user who answers the target question. The target question can be either an objective question or a subjective question; this embodiment does not limit the scope of the question.

[0031] Subjective questions are those that do not have a single correct answer and require the user to express their own viewpoints or analyze the problem.

[0032] Optionally, subjective questions can be traditional essay questions, such as short answer questions, discussion questions, problem-solving questions, composition questions, or case analysis questions; short answer questions can be such as "Why can a refrigerator cool?"

[0033] Alternatively, subjective questions can refer to content published on an interactive platform or messages sent by other users to the target user.

[0034] The interactive platform refers to a system through which different users can interact. Its type can be set according to the actual situation. For example, the interactive platform can be a video platform, a shopping platform, or an instant messaging platform, etc. This embodiment does not limit it.

[0035] The type of content published on the interactive platform can be set according to the actual situation. For example, the content published on the interactive platform may be a video or a post. The content of the video and the content of the post can be set according to the actual situation. This embodiment does not limit this.

[0036] The target answer content refers to the target user's personal understanding of the target question. When the target question is a subjective question and the subjective question is a traditional question-and-answer question, the target answer content can be the answer entered by the target user. When the subjective question is content posted on an interactive platform, the target answer can be a comment entered by the target user (the comment can be used to indicate the target user's opinion); when the subjective question is a message sent to the target user by other users, the target answer can be the target user's reply to that message.

[0037] The type of answer upload operation can be set according to the actual situation. For example, the answer upload operation can be a click operation or a double click operation. This embodiment does not limit it here.

[0038] The information corresponding to the points deducted refers to the content in the target answer that leads to the deduction of points. Specifically, it can be information that differs between the target answer and the reference content, where the reference content refers to the reference answer to the target question.

[0039] Optionally, the difference information can be information that exists in the target answer but not in the reference content. In this case, the difference information is redundant information. Alternatively, the difference information can be information that does not exist in the target answer but exists in the reference content. In this case, the difference information is missing information.

[0040] When the difference information is redundant, it can be content unrelated to the target question, or it can be content related to the target question.

[0041] Since the reference content may omit certain knowledge points or logical relationships, the discrepancy information may also be related to the target question. In this case, it can be used to improve the reference content of the target question.

[0042] Optionally, the discrepancy information may include discrepancy knowledge points and / or discrepancy inference results. Specifically, discrepancy knowledge points indicate that the discrepancy information belongs to a difference in knowledge points; for example, if the reference content mentions that the evaporator absorbs heat, but the user's answer does not mention this content, then the discrepancy information is a discrepancy knowledge point; Differential reasoning results can be understood as differing logical relationships. For example, the reference content mentions that evaporator heat absorption is a key aspect of refrigerator cooling; if the user's answer also involves both evaporator heat absorption and refrigerator cooling, but does not mention any connection between them, this missing connection represents a deficiency in the reasoning process and can be categorized as a differential reasoning result.

[0043] When the difference information is missing, the difference knowledge point is a missing knowledge point and / or the difference reasoning result is a missing reasoning result. When the difference information is redundant, the difference knowledge point can be a redundant knowledge point and / or the difference reasoning result can be a redundant reasoning result.

[0044] The missing knowledge points can also be referred to as the target user's cognitive blind spots or unknown knowledge points regarding the target question.

[0045] Cognitive blind spots refer to knowledge points in the reference content that the target user is unaware of or unaware should exist.

[0046] When the knowledge gap is a missing knowledge gap, visualizing the gap information can help target users intuitively learn about knowledge gaps they do not know or have not been aware of, thereby breaking through their cognitive limitations and improving their abilities.

[0047] In this embodiment, difference information for the target answer content is displayed so that users can learn about the difference knowledge points and / or difference reasoning results through the difference information, thereby determining their unknown knowledge points, and / or known but unused knowledge points, and / or unknown logical relationships, and / or known but unused logical relationships under the question. Compared with the method of only outputting scores, this can enhance interpretability and improve interactivity.

[0048] 202. Determine the root cause classification of each discrepancy information, wherein the root cause classification is the error type to which the information corresponding to the point of loss belongs, and the error type includes known information and unknown information; and / or, the error type includes missing information and redundant information, wherein known information indicates that the target user knows the information corresponding to the point of loss, unknown information indicates that the target user does not know the information corresponding to the point of loss, missing information indicates that the information corresponding to the point of loss is missing information in the target answer content, and redundant information indicates that the information corresponding to the point of loss is redundant information in the target answer content; The error types can be set according to the actual situation. For example, error types include known information and unknown information; and / or, error types include missing information and redundant information, that is, root cause classification includes known information and unknown information; and / or, root cause classification includes missing information and redundant information.

[0049] "Information is known" indicates that the target user is aware of the information corresponding to the points where points were lost, that is, the target user knows the knowledge point or knows the relationship between at least two knowledge points.

[0050] "Unknown information" indicates that the target user is unaware of the information corresponding to the point where points were lost, meaning that the user is unaware of the existence of that knowledge point or its related relationships.

[0051] The missing information indicates that the missing information is the missing information in the target answer content. In other words, the target answer content is missing the knowledge points or reasoning processes that exist in the reference content, meaning that a certain knowledge point or reasoning process is completely absent.

[0052] The information that indicates a loss of points is information that is superfluous to the target answer. This superfluous information can be irrelevant to the target question, or it can be relevant to the target question.

[0053] When redundant information is irrelevant to the target question, it is considered erroneous. This means that the target answer contains incorrect knowledge points or flawed reasoning processes. For example, it may introduce incorrect knowledge points, present incorrect causal relationships, make writing errors, or copy errors.

[0054] When the redundant information is irrelevant to the target question, it can be called information error; when the redundant information is relevant to the target question, it can be called information without error.

[0055] When the error type includes known information and unknown information, the root cause classification of each difference information can be determined based on historical learning data. If the historical learning data includes the information corresponding to the lost points, it means that the target user knows the information corresponding to the lost points, and the root cause classification is determined to be known information. If the historical learning data does not include the information corresponding to the lost points, it means that the target user does not know the information corresponding to the lost points, and the root cause classification is determined to be unknown information.

[0056] Historical learning data refers to data recorded by the target user in the past, which may include information about the courses the target user has taken and / or the target user's historical test-taking data.

[0057] When the error type includes missing information and redundant information, the root cause classification can be determined based on the difference information. If the difference information is information that exists in the target answer but not in the reference content, the root cause classification is redundancy. If the difference information is information that does not exist in the target answer but exists in the reference content, the root cause classification is missing information.

[0058] When the error type includes known information, unknown information, missing information, and redundant information, if the error type is known information and missing information, it means that the information is known but not used. If the error type is known information and redundant information with incorrect information, it means that the information is known but misused. If the root cause classification is unknown information and missing information, it means that the information is unknown and not used. If the error type is unknown information and redundant information with incorrect information, it means that the information is unknown and misused.

[0059] Understandably, when error types include known information, unknown information, missing information, and redundant information, the terminal can, after receiving the discrepancy information, determine whether the discrepancy information is missing or redundant, and determine whether the discrepancy information exists in the historical learning data. If the discrepancy information exists and it is missing, the root cause classification of the discrepancy information is "known but not used," meaning the discrepancy information is a knowledge point or logic that the target user has learned but has not used in this question. If the discrepancy information exists and it belongs to the category of "information error" under "information redundancy," the root cause classification of the discrepancy information is "information misuse," meaning the discrepancy information is a knowledge point or logic that the user has learned but has misused in this question. If the discrepancy information does not exist in the historical learning data and it is missing, the root cause classification of the discrepancy information is "unknown and not used," meaning the target user has not seen the knowledge point or logic and has not used it in this question. If the discrepancy information does not exist in the historical learning data and it belongs to the category of "information error" under "information redundancy," the root cause classification of the discrepancy information is "information misuse," meaning the target user has not seen the knowledge point or logic but has misused it in this question.

[0060] Understandably, this application will also categorize users' mastery of certain information in historical learning data to achieve a more accurate classification and positioning effect.

[0061] Understandably, the root cause classification of discrepancy information can be determined based on the user's historical learning data. Generally, the root cause classification is the path result indicated by the classification path to which the discrepancy information belongs. For example, in the case where information redundancy is considered information error, the process of determining the root cause classification can be as follows: Determine whether the discrepancy information is a discrepancy knowledge point or a discrepancy inference result; If it is a knowledge point with discrepancies, then determine whether the knowledge point with discrepancies is a missing knowledge point or an incorrect knowledge point; If the discrepancy knowledge point is a missing knowledge point, then the historical learning data is used to determine whether the discrepancy knowledge point is a known knowledge point or an unknown knowledge point; If the knowledge point that is discrepancy is a known knowledge point, then the root cause classification of the discrepancy information is determined as information that is known but not used; If the knowledge point that is discrepancy is unknown, then the root cause classification of the discrepancy information is determined as unknown and unused information; If the discrepancy information indicates an incorrect knowledge point, then the historical learning data is used to determine whether the discrepancy knowledge point is a known knowledge point or an unknown knowledge point. If the knowledge point that is discrepancy is a known knowledge point, then the root cause classification of the discrepancy information is determined to be known misuse of information; If the knowledge point that is discrepancy is unknown, then the root cause of the discrepancy is classified as misuse of unknown information. If the result is a differential reasoning result, then determine whether the differential reasoning result is a missing reasoning result or an incorrect reasoning result; If the differential reasoning result is a missing reasoning result, then it is determined whether the differential reasoning result is a known reasoning result or an unknown reasoning result based on historical learning data. If the result of the difference inference is a known inference result, then the root cause classification of the difference information is determined to be known but unused information; If the result of the differential reasoning is an unknown reasoning result, then the root cause classification of the differential information is determined to be unknown and unused information; If the discrepancy information is an incorrect reasoning result, then the historical learning data is used to determine whether the discrepancy reasoning result is a known reasoning result or an unknown reasoning result. If the result of the differential reasoning is a known reasoning result, then the root cause classification of the differential information is determined to be known misuse of information; If the result of the differential reasoning is an unknown reasoning result, then the root cause classification of the differential information is determined to be information unknown misuse.

[0062] Understandably, this application could first identify whether the difference information is known or unknown to the user, then determine whether it is redundant or missing information; finally, identify whether the difference information belongs to a difference knowledge point or a difference inference result, ultimately obtaining a root cause classification. Alternatively, it could first determine whether the difference information belongs to a difference knowledge point or a difference inference result, then identify whether the difference information is known or unknown to the user, and finally identify whether the difference information is missing or redundant information, obtaining a root cause classification. As long as the purpose of identifying the root cause classification of the difference information can be achieved, the above steps do not need to be ordered.

[0063] Understandably, it is also possible to label the difference information based on the user's historical learning data to determine the root cause category to which the difference information belongs. For example, it may belong to reasoning, logical difference, the difference belongs to the reasoning process, or logical deficiency, that is, the reasoning process is not reflected, the knowledge point and the reasoning process corresponding to the knowledge point are not reflected, or the logic belongs to the content that has been mastered. Finally, the root cause category of the difference information is logical deficiency - known but not used.

[0064] For example, when information redundancy is considered information error, the classification path of the discrepancy information and the corresponding root cause classification can be as follows: Figure 3 or Figure 4 As shown.

[0065] Optionally, after obtaining the difference information, the difference information can be displayed, and then, in response to the linkage operation of the difference information, the root cause classification of each difference information can be determined. The type of linkage operation can be set according to the actual situation; for example, the linkage operation can be a click operation or a voice operation, which is not limited in this embodiment.

[0066] 203. Determine the root cause tracing results of the difference information under the error type through the large language model.

[0067] The root cause analysis of the discrepancy information refers to the reason for the discrepancy, i.e., the cause of the error. Identifying and displaying the root cause analysis results allows the target user to clearly understand the cause of the error, thereby helping them identify limitations in their thinking, further clarify their learning direction, expand their cognition, and ultimately improve their abilities.

[0068] It is understood that in this embodiment, the root cause classification of each difference information is determined, wherein the root cause classification is the error type to which the information corresponding to the point of loss belongs; the root cause tracing result of the difference information under the error type is determined by the large language model, so as to realize the classification tracing, refine the root cause tracing result, and improve the accuracy of the root cause tracing result.

[0069] It is understandable that the application scenarios of the knowledge content tracing method provided in this application can be set according to the actual situation. For example, the application scenarios of the knowledge content tracing method provided in this application can be educational scenarios, legal scenarios, economic analysis scenarios, or argument analysis scenarios.

[0070] When applied to educational settings, the knowledge content tracing method provided in this application enables students to receive immediate reinforcement based on the root cause tracing results of differential information, thereby improving students' academic performance.

[0071] When analyzing a scenario for an argument, the method for tracing the source of knowledge content provided in this application allows users to learn and enrich their arguments based on the root cause tracing results of the difference information, thereby learning to understand the essence of things. This embodiment does not limit this.

[0072] It is understandable that, since this embodiment can help users discover unknown knowledge points, which can be knowledge points in areas unknown to the user, thereby helping the target user to think innovatively, this embodiment can also be applied to higher education scenarios, such as to cultivate the interdisciplinary thinking ability of graduate students and researchers.

[0073] Optionally, the method for tracing the source of knowledge content provided in this application can be integrated into an intelligent analysis system. The type of intelligent analysis system can be set according to the actual situation. For example, the intelligent analysis system can be an intelligent grading system or an intelligent argument analysis system. This embodiment does not limit this.

[0074] As can be seen from the above, in this embodiment, at least one discrepancy information is located from the target answer content uploaded by the target user. This discrepancy information corresponds to the information that resulted in point deductions in the target answer content. The root cause classification of each discrepancy information is determined. The root cause classification is the error type to which the information corresponding to the point deduction belongs. Error types include known information and unknown information; and / or, error types include missing information and redundant information. Known information indicates that the target user knows the information corresponding to the point deduction; unknown information indicates that the target user does not know the information corresponding to the point deduction; missing information indicates that the information corresponding to the point deduction is missing information in the target answer content; and redundant information indicates that the information corresponding to the point deduction is redundant information in the target answer content. The root cause tracing results of the discrepancy information under the error type are determined through a large language model. The root cause tracing results of the discrepancy information are then displayed, allowing users to intuitively understand the root cause tracing results of knowledge blind spots and / or known but unused knowledge points that they were never aware of or knew, which are necessary for solving or analyzing a problem. This allows for improvements based on the root cause tracing results of the discrepancy information, thereby enhancing analytical capabilities.

[0075] It is understandable that, in some embodiments, the method for determining the difference information can be set according to the actual situation. For example, the target answer content can be compared with the reference content to obtain the difference information, or the large language model can locate the difference information in the target answer content by using prompts generated based on the scoring rules. The comparison standard for the difference can be the reference answer generated by the large language model according to the scoring rules, or it can be the reference content generated by the large language model after training on the training data, which is more comprehensive on the current basis.

[0076] It is understandable that the process of locating at least one discrepancy from the target user's uploaded target answer content can also be described as follows: Generate the target user's first cognitive expression path on the question content corresponding to the target answer content; By comparing the cognitive differences between the first cognitive expression path and the second cognitive expression path of the reference content, the difference information is obtained. The cognitive expression path is a logical reasoning chain consisting of entity nodes corresponding to each knowledge point and logical edges connecting each entity node.

[0077] Here, the content of the question corresponding to the target answer refers to the content of the target question. Logical edges are used to indicate the logical relationships between the knowledge points corresponding to each entity node. The first cognitive expression path refers to the logical reasoning chain composed of the entity nodes corresponding to the first knowledge point involved in the target answer content, and the logical edges used to connect the entity nodes and represent the logical relationships between the first knowledge points.

[0078] For example, if the target answer is "The refrigerator can cool because it constantly blows out cold air. The compressor cools the air, and then the cold air is sent into the refrigerator, which creates a cooling effect," the first knowledge points extracted include compressor, air cooling, and cooling. The logical relationship between the extracted first knowledge points is "makes" and "creates," and the first cognitive expression path is "compressor → air cooling → creating a cooling effect."

[0079] The second cognitive expression path refers to the logical reasoning chain consisting of entity nodes corresponding to the second knowledge points involved in the reference content, and logical edges used to connect the entity nodes and represent the logical relationship between the second knowledge points.

[0080] At this point, the difference information can refer to the different content in the first cognitive expression path and the second cognitive expression path. This can be different knowledge points and / or different logical edges. Different logical edges can lead to different understandings of logical relationships. When the difference information refers to different knowledge points, it can include at least one different knowledge point. When the difference information refers to different logical edges, it can include at least one different reasoning result, and the different reasoning result includes at least one different logical edge.

[0081] Among them, the difference knowledge points can refer to knowledge points that exist in the first cognitive expression path but not in the second cognitive expression path. That is, the difference knowledge points can be redundant knowledge points. Redundant knowledge points can be knowledge points related to the target question or knowledge points unrelated to the target question. And / or, the difference knowledge points can refer to knowledge points that do not exist in the first cognitive expression path but exist in the second cognitive expression path.

[0082] The differential reasoning result may include logical edges that exist in the first cognitive expression path but not in the second cognitive expression path. That is, the differential reasoning result may be redundant logical edges, which may be logical edges related to the target question or logical edges unrelated to the target question. And / or, the differential reasoning result may refer to logical edges that do not exist in the first cognitive expression path but exist in the second cognitive expression path.

[0083] Understandably, the display method of the difference information can be set according to the actual situation. For example, the difference information can be displayed directly, such as... Figure 5 As shown, or, the differences can be highlighted by comparing the first and second cognitive expression paths, for example... Figure 6 As shown.

[0084] Optionally, when the difference information is highlighted by comparing the first cognitive expression path and the second cognitive expression path, the display style of the difference information between the first cognitive expression path and the second cognitive expression path is the first display style, and the display style of the same information between the first cognitive expression path and the second cognitive expression path is the second display style, so as to highlight the difference information, make it easier for the target user to notice the difference information, and further reduce the difficulty for the target user to obtain the difference information.

[0085] The first display style is used to indicate the display effect of the difference information, and the second display style is used to indicate the display effect of the same information. The first display style and the second display style are different display styles, which can be set according to the actual situation.

[0086] For example, the first display style is bold, and the second display style is non-bold.

[0087] For example, the first display style is red text, and the second display style is black text, such as... Figure 7 As shown, this embodiment is not limited here.

[0088] Alternatively, the first display style can be a backlit display, and the second display style can be a non-backlit display.

[0089] For ease of description, the entity node in the first cognitive expression path is called the first entity node, the logical edge in the first cognitive expression path is called the first logical edge, the knowledge point in the second cognitive expression path is called the second entity node, and the logical edge in the second cognitive expression path is called the second logical edge. That is, the entity node corresponding to the first knowledge point is called the first entity node, and the entity node corresponding to the second knowledge point is called the second entity node.

[0090] Optionally, the terminal can compare the first entity node and the second entity node to obtain the difference knowledge points, and can compare the first logical edge between the first entity nodes and the second logical edge between the second entity nodes to obtain the difference reasoning result.

[0091] In this embodiment, a first cognitive expression path of the target user on the question content corresponding to the target answer content is generated based on the target answer content; the cognitive difference between the first cognitive expression path and the second cognitive expression path of the reference content is compared to obtain difference information. The cognitive expression path is a logical reasoning chain consisting of entity nodes corresponding to each knowledge point and logical edges connecting each entity node, so as to obtain difference information through path comparison and improve the efficiency of obtaining difference information.

[0092] Understandably, determining the target user's first cognitive expression path regarding the question content corresponding to the target answer content, based on the target answer content, includes: Extract at least one primary knowledge point from the target answer content and the logical relationship between each primary knowledge point; Based on the first knowledge point and logical relationship, the first cognitive expression path is obtained, which includes the entity nodes corresponding to each first knowledge point and the logical edges corresponding to each logical relationship.

[0093] The method for extracting the first knowledge point and the logical relationship between the first knowledge points can be set according to the actual situation.

[0094] For example, the first knowledge point and logical relationship can be extracted from the target answer content by extracting templates or natural language processing models; among them, the natural language processing model can be a large language model (LLM), such as BERT or GPT, which is not limited in this embodiment.

[0095] Optionally, the process of extracting at least one primary knowledge point and the logical relationships between primary knowledge points from the target answer content using a natural language processing model can be as follows: Using a natural language processing model, the target answer content is analyzed syntactically according to phrase structure rules (CFG, Context-Free Grammar) to obtain the hierarchical structure of the first knowledge point. Based on the hierarchical structure, the target answer content is analyzed for grammatical dependency to obtain the logical relationships between the first knowledge points.

[0096] Optionally, the hierarchical structure of the first knowledge point can be represented by a phrase structure tree. In the process of performing grammatical dependency analysis, a dependency tree can be obtained first through the dependency classifier and attention mechanism layer (Transformer) in the natural language processing model, and then the logical relationship between the first knowledge points can be obtained based on the dependency tree.

[0097] For example, if the target answer is "The reason refrigerators cool is that the compressor cools the air," the resulting phrase structure tree could be as follows: Figure 8 As shown, the resulting dependency tree can be represented as follows: Figure 9 As shown, the main body (the reason why the refrigerator cools) and core information (the compressor makes the air cold) can be obtained through the phrase structure tree. Subject-verb-object parsing can be performed through the dependency tree, and then the logical relationship between the first knowledge points can be obtained based on the subject-verb-object parsing results.

[0098] Optionally, the method of connecting the first entity nodes corresponding to the extracted first knowledge points according to the logical relationship between the first knowledge points to obtain the first cognitive expression path can be set according to the actual situation.

[0099] For example, the first entity nodes corresponding to the first knowledge points can be connected according to the logical relationship between the first knowledge points through natural language processing models and / or graph structure generation algorithms to obtain the first cognitive expression path. This embodiment does not limit this.

[0100] Alternatively, the process of determining the logical relationships between the first knowledge points can also be as follows: Logical detection processing is performed based on the target answer content to obtain the logical relationships between the first knowledge points.

[0101] The method of performing logical detection processing based on the target answer content can be set according to the actual situation. For example, logical detection processing of the target answer content can be performed by preset logical rules, or logical detection processing of the target answer content can be performed by a natural language processing model. This embodiment does not limit this.

[0102] Optionally, when performing grammatical dependency analysis on the target answer content to obtain the logical relationship between the first knowledge points, the results of the grammatical dependency analysis can be transformed into a logical structure to construct a propositional representation based on the subject-verb-object (SVO) framework, so that natural language can be converted into formal logical expression for easier reasoning and analysis. Then, logical detection processing can be performed on the formal logical expression to obtain the logical detection result.

[0103] For example, the propositional representation of a subject-verb-object framework can be: Subject (S): The compressor cools the air. Predicate (V): leading to Object (O): Refrigeration; The causal relationship expressed in formal logic can be: a compressor cools the air. The refrigerator is cooling.

[0104] It is understood that a second cognitive expression path for the reference content can be determined. The process of determining the second cognitive expression path for the reference content can refer to the process of determining the first cognitive expression path based on the target answer content. This embodiment does not limit this process.

[0105] Alternatively, it can be understood that the second cognitive expression path can also be obtained through a knowledge graph. In this case, the process of determining the second cognitive expression path can be as follows: By using a natural language processing model and calling a knowledge graph, the target question is identified and processed to obtain the second knowledge point involved in the target question; By using a knowledge graph, the second cognitive expression path is determined based on the second entity node corresponding to the second knowledge point.

[0106] The knowledge graph is a pre-constructed multi-level directed graph. Entity nodes in the knowledge graph represent knowledge points. Entity nodes can include at least one of concept nodes and fact nodes. Concept nodes represent core knowledge points. For example, concept nodes include concept node n1, concept node n2, and concept node n3. Concept node n1 represents "refrigerator", concept node n2 represents "evaporator", and concept node n3 represents "refrigerant". Fact nodes refer to specific events, theories, or related background information. For example, fact nodes include fact node n4, fact node n5, and fact node n6. Fact node n4 represents "endothermic evaporation", fact node n5 represents "condensation and exothermic", and fact node n6 represents "endothermic sublimation".

[0107] In a knowledge graph, logical edges between entity nodes represent logical relationships between knowledge points. These logical relationships can be causal, hierarchical, or parallel, for example: A → B means that A causes B, and "→" indicates that there is a causal relationship between A and B. For example, A represents the evaporator absorbing heat, and B represents the refrigerator's internal cooling; For example, A B indicates that A is a subconcept of B. This indicates that there is a hierarchical relationship between A and B; For example, A represents a means of transportation, and B represents a bicycle; For example, AB means that A and B are at the same level. " indicates that there is a parallel relationship between A and B; For example, A represents traffic emissions and B represents industrial emissions (both of which are causes of air pollution).

[0108] Optionally, entity nodes in a knowledge graph may include entity nodes corresponding to knowledge points in at least one knowledge domain. A knowledge domain refers to a set of facts, concepts, principles, rules, skills, and / or experiences that are widely recognized and organized within a certain scope. It can be divided according to topics, with one topic corresponding to one knowledge domain.

[0109] The division of topics can be set according to the actual situation. For example, topics can be divided according to the subject system, with one topic corresponding to one subject, that is, one subject corresponding to one knowledge domain. For example, mathematics corresponds to one knowledge domain, Chinese language corresponds to one knowledge domain, and English corresponds to one knowledge domain.

[0110] For example, themes can be divided according to social sciences, and themes can correspond to economics, psychology, sociology, and political science, respectively.

[0111] For example, themes can be divided according to medical health, and these themes can correspond to clinical medicine, pharmacy, public health, and nursing, etc. For example, themes can be divided according to multiple systems at the same time. For example, literature and rhetoric correspond to a knowledge domain, psychology and affective theory correspond to a knowledge domain, and philosophy and aesthetics correspond to a knowledge domain.

[0112] It is understandable that when the discrepancy information includes unknown knowledge points, these unknown knowledge points usually exist at the intersection of different knowledge domains, and may even cross from one knowledge domain to another.

[0113] Therefore, in this embodiment, when the entity nodes in the knowledge graph can include entity nodes corresponding to knowledge points in at least one knowledge domain, a second cognitive expression path can be obtained through cross-domain association detection to further discover unknown knowledge points, thereby helping users to know the knowledge points they do not know, and further helping users to improve their comprehensive abilities.

[0114] In this embodiment, a natural language processing model is used to call a knowledge graph to identify the target question and obtain the second knowledge point involved in the question. Based on the second entity node corresponding to the second knowledge point, the second cognitive expression path is determined through the knowledge graph. This eliminates the need to store the second cognitive expression path for each question or determine the reference content for each question, thus reducing the number of second cognitive expression paths that need to be stored and saving the storage resources consumed by storing the second cognitive expression paths.

[0115] Understandably, the root cause analysis of discrepancies in error types, determined through large language models, includes: If the root cause classification of the discrepancy information includes known information and missing information, a content supplementation instruction is generated to obtain the supplementary content input by the target user in response to the content supplementation instruction; By using a large language model, the root cause tracing results of the information corresponding to the points lost are determined based on the supplementary content, under the conditions of known information and missing information.

[0116] Specifically, whether to first determine whether the information is known or unknown, and then determine whether the information is missing or redundant, or to first determine whether the information is missing or redundant, can be referred to in the above process of first determining whether the information is known or unknown, or first determining whether the information is redundant or missing. This embodiment will not repeat the details here.

[0117] In cases where the classification includes known information and missing information, it indicates that the target user is aware of the discrepancy information, but may not have used it in the question for some reason. Therefore, a content supplementation instruction can be generated to prompt the target user to input supplementary content. The reason why the target user did not use the discrepancy information in the question can be determined by the supplementary content input by the target user in response to the content supplementation instruction.

[0118] The types of content supplementation instructions can be set according to the actual situation. For example, content supplementation instructions may include target questions and / or prompting actions. The questioning method of the target question may include counterfactual questions. The target question is used to guide the target user to use the difference information from the opposite perspective. The prompting action is used to prompt the target user to use the difference information from the positive perspective.

[0119] Counterfactual questioning refers to a questioning method that negates or modifies known facts, constructing a hypothetical situation contrary to the facts. For example, the target question could be "Can a refrigerator still cool without an evaporator?" When the target question includes counterfactual questioning, it can imply causal relationships, help users distinguish between "relevance" and "causation," and encourage users to reflect.

[0120] Supplementary content can be obtained by modifying the target answer, or it can be added to the target answer. If the supplementary content can be corrected, i.e., the cognitive expression path of the supplementary content is the same as the second cognitive expression path, or the supplementary content includes discrepancy information, then the root cause tracing result corresponding to the root cause classification is determined to include retrieval failure or motivational / emotional factors. If the cognitive expression path of the supplementary content still differs from the second cognitive expression path, or the supplementary content does not include discrepancy information, then it is determined whether the discrepancy information was used in the original context. If so, then the root cause tracing result corresponding to the root cause classification is determined to include scenario mapping failure. If the target question is a subjective question, the content supplementation instruction includes multiple-choice questions, and the cognitive expression path of the supplementary content is the same as the second cognitive expression path, or the supplementary content includes discrepancy information, then the root cause tracing result corresponding to the root cause classification is determined to include insufficient proceduralization.

[0121] In this embodiment, if the root cause classification of the discrepancy information includes known information and missing information, a content supplementation instruction is generated to obtain the supplementary content input by the target user in response to the content supplementation instruction; through a large language model, the root cause tracing results of the information corresponding to the lost points under known information and missing information are determined based on the supplementary content, so as to further improve the accuracy of the root cause tracing results.

[0122] Understandably, large language models can be used to determine the root causes of differences in user responses under corresponding root cause categories. The process includes: The target data features of the target user are determined by using a large language model based on at least one of historical learning data and target answer content. The root cause tracing results of the information corresponding to the lost points under the error type are determined based on the target data features. The target data features are used to characterize the target user's cognitive state and / or thinking pattern defects.

[0123] Among them, the cognitive state of the target user refers to the real-time operational status of the target user in cognitive processes such as perception, attention, memory, understanding, reasoning, and judgment within a specific time period. The target user's thinking pattern defects refer to the systematic deviations from rational or realistic thinking patterns formed in the target user's habits, leading to repeated errors in judgment or decision-making.

[0124] Optionally, the target answer content may include not only the target user's personal understanding of the target question, but also the target answer process characteristics of the target user during the answering process. Target answer process characteristics are used to indicate the answer characteristics exhibited by the target user during the answering of the target question, which may include at least one of the following: answering time, word count, editing trajectory (deletion and / or pauses), interaction characteristics during the same period (interaction characteristics such as network differences and / or device differences), emotional words in the answer content, and subtotal statements in the answer content (subtotal statements may refer to summarizing, refining, or transitional statements).

[0125] Optionally, target data characteristics of target users can also be determined based on difference information.

[0126] For example, such as Figure 10 As shown, the difference information is the difference in knowledge points, and the root cause of the difference in knowledge points is classified as known information and missing information. The root cause tracing results of known information and missing information include psychological factors, insufficient organizational ability, lack of attention, heuristic bias, and failure of knowledge point transfer.

[0127] Among them, the data features corresponding to psychological factors can include emotional features, such as ideological preferences, social background preferences, and economic preferences. If the target data features include target emotional features, the target emotional features can be determined by the analytical perspective of the historical learning data and the analytical perspective of the first knowledge point. For example, if the target user's historical learning data indicates that the target user has learned about traffic emissions, industrial emissions, and volcanic eruptions, the analytical perspective corresponding to "traffic emissions" is the human cause perspective, the analytical perspective corresponding to "industrial emissions" is the human cause perspective, and the analytical perspective corresponding to "volcanic eruptions" is the natural cause perspective. If the difference information includes volcanic eruptions and the first knowledge point only includes traffic emissions and industrial emissions, then it is determined that the target user has ideological preferences. The inclusion of ideological preferences in the target data features means that there is a problem with the target user's cognitive state, and the root cause tracing results corresponding to the root cause classification include psychological factors.

[0128] Insufficient organizational ability refers to the target user's difficulty in integrating knowledge points into a logical chain in a structured, clear, and coherent manner. Its data characteristics include at least one of the following: knowledge points are not missing, but logical connections between knowledge points in the answer are lacking, or knowledge points are not missing, but the conjunctions between knowledge points in the answer are weak. If the target data characteristics include no missing knowledge points (i.e., no different knowledge points) but logical connections between the first entity nodes and / or weak conjunctions between the first knowledge points in the target answer, it means the target user is in a confused cognitive state or has a flawed thinking pattern. The root cause tracing results corresponding to the root cause classification include insufficient organizational ability.

[0129] Understandably, a lack of organizational skills usually appears in similar types of questions and can be reproduced across different questions.

[0130] Among them, the data characteristics of attention deficit include a complete logical reasoning chain in historical questions but an incomplete cognitive expression path in the answer content. If it is determined based on historical learning data that the target user's logical reasoning chain in historical questions is complete and the difference information indicates that the first cognitive expression path is incomplete, that is, the target data characteristics include that the target user's logical reasoning chain in historical questions is complete but the first cognitive expression path is incomplete, it means that the target user is in a scattered cognitive state. The root cause tracing results corresponding to the root cause classification include attention deficit.

[0131] Heuristic bias refers to systematic errors caused by relying on certain mental shortcuts. Its data characteristics include users answering based on frequently used knowledge and / or based on common phenomena. If, based on historical learning data and target answer content, it is determined that the target user answers based on frequently used knowledge points or common phenomena—that is, the target data characteristics include the target user answering based on frequently used knowledge points or common phenomena—it means that the target user has a flawed thinking pattern, and the root cause tracing results corresponding to the root cause classification include heuristic bias.

[0132] Among them, the data characteristics of knowledge point transfer failure include that the user uses the knowledge point and / or logical edge in the questions of the neighboring context, but does not use the knowledge point and / or logical edge in the questions of the current context. If it is determined based on historical learning data that the target user uses the difference information in the questions of the neighboring context and / or in the questions of the original context, that is, if the target data characteristics include the target user using the difference information in the questions of the neighboring context and / or in the questions of the original context, it means that the target user is in a state of ambiguous cognition, and the root cause tracing result corresponding to the root cause classification includes knowledge point transfer failure.

[0133] For example, such as Figure 10As shown, the difference information is the difference in knowledge points, and the root cause of the difference in knowledge points is classified as unknown information and missing information. The root cause tracing results of unknown information and missing information include missing prerequisite knowledge points and knowledge points that have not been encountered before.

[0134] Among these, the data characteristic of missing prerequisite knowledge points includes knowing a certain knowledge point but not understanding its specific content or background knowledge. For example, if the target question is to explain the process of refrigerator refrigeration, the second cognitive expression path is: evaporator absorbs heat - refrigerator interior cools - compressor compresses refrigerant - condenser releases heat to the outside - refrigeration cycle continues. The first cognitive expression path is also: refrigerator interior cools - compressor compresses refrigerant - condenser releases heat to the outside - refrigeration cycle continues. The difference is that the evaporator absorbs heat. If historical learning data determines that the target user knows about the evaporator but does not understand its working principle, that is, the target data characteristic includes the target user knowing about the evaporator but not understanding its working principle, it means that the target user is in a state of ambiguous cognition. Therefore, the root cause tracing result corresponding to the root cause classification is determined to include missing prerequisite knowledge points.

[0135] Among them, the data feature of "knowledge point not encountered" includes users who have never learned the knowledge point.

[0136] For example, such as Figure 10 As shown, the difference information is the difference knowledge point, and the root cause classification of the difference knowledge point is that the information is known but the information is wrong. The root cause tracing results of the known but misused information include incorrect understanding of concepts or boundary errors, incorrect writing of facts or terms, and going off-topic.

[0137] Among them, data features of conceptual misunderstanding or boundary error include at least one of the following: low semantic alignment between the answer content and the question; answer content deviating from the question's wording; confusion of synonymous or adjacent concepts (confusion of adjacent concepts refers to the confusion of knowledge points indicated by adjacent entity nodes in the knowledge graph); and confusion of logical relationship types. Optionally, when there is supplementary content for supplementary instructions, the data features of conceptual misunderstanding or boundary error may also include the ability to be corrected after restating the question stem. If the target data features include at least one of the following: low semantic alignment between the target answer content and the target question; target answer content deviating from the question's wording; confusion of synonymous or adjacent concepts; confusion of logical relationship types; and the ability to be corrected after restating the question stem, it means that the target user is in a state of confusion and uncertainty. The root cause tracing result corresponding to the root cause classification includes conceptual misunderstanding or boundary error.

[0138] Among them, irrelevant data features include misunderstandings of the topic concept. If the target data features include misunderstandings of the target topic concept, it means that the target user is in a state of ambiguous cognition, and the root cause tracing results corresponding to the root cause classification include irrelevant data features.

[0139] For example, such as Figure 10 As shown, the differential information is the differential knowledge points, and the root cause of the differential knowledge points is classified as unknown information and incorrect information. The root cause tracing results of unknown information and incorrect information include misunderstanding of concepts and suspected fabrication and plagiarism.

[0140] Conceptual misunderstanding refers to a user having an abstract noun concept in their mind but not knowing its correct meaning. If conceptual misunderstanding is determined based on historical learning data, it means that the target user is in a state of ambiguous cognition, and the root cause tracing results corresponding to the root cause classification include conceptual misunderstanding.

[0141] For example, such as Figure 10 As shown, the differential information is the result of differential reasoning, and the root cause classification of the differential reasoning result is information known and information missing. The root cause tracing results of information known and information missing include emotional motivation factors, retrieval failure, situation mapping failure, mapping binding error, insufficient proceduralization, and metacognitive failure.

[0142] Among them, emotional motivation factors refer to situations where users possess relevant knowledge but fail to correctly utilize or elaborate on it during answering due to emotions such as anxiety or frustration, or insufficient motivation such as perfunctory work or fatigue. Its data characteristics can include the user's answering process characteristics, which indicate that the user possesses relevant knowledge and meets preset characteristic conditions. When the target answering process characteristics meet the preset characteristic conditions, it suggests that the user may have poor emotions or insufficient motivation, meaning the user is in a state of fatigue or frustration. When the target data characteristics indicate that the target answering process characteristics of the target question meet the preset characteristic conditions, and it is determined based on historical learning data that the target user possesses relevant knowledge, the root cause tracing results include emotional motivation factors.

[0143] It is understandable that a user possessing relevant knowledge can be interpreted as the user having studied the relevant knowledge, or as the user being able to answer the question correctly in previous quizzes.

[0144] Optionally, the preset feature conditions can be set according to the actual situation. For example, when the target answering process features include the answering time, the preset feature condition can be that the difference between the answering time and the historical answering time is greater than the preset time. Or, when the target answering process features include the editing trajectory (delete and / or pause), the preset feature condition can be that the number of deletions and / or pauses is greater than the preset number. This embodiment does not limit this.

[0145] Optionally, when supplementary content exists, the data characteristics of emotional motivation factors may also include that the supplementary content can be modified, that is, the cognitive expression path of the supplementary content is the same as the second cognitive expression path, or the supplementary content includes difference information. In this case, the target data characteristics can also be determined based on the supplementary content. If the target data characteristics include that the target user has relevant knowledge, the target answer process characteristics of the target question meet the preset characteristic conditions, and the supplementary content can be modified, it indicates that the target data characteristics are data characteristics of emotional motivation factors. Then, the root cause tracing results corresponding to the root cause classification include emotional motivation factors.

[0146] For example, in past questions, users can write out the complete cycle of evaporator absorbing heat - refrigerator cooling - compressor compressing refrigerant - condenser releasing heat to the outside - and the refrigeration cycle continuing. However, in the target question, they cannot write this out, their answer time is only 1 / 4 of their usual answer time, and there are a lot of deletions and "too difficult" messages before submitting the target answer. After generating the content supplementation instruction, the supplemented content can be corrected, and the root cause tracing results corresponding to the root cause classification include emotional motivation factors.

[0147] Among them, retrieval failure refers to the user having learned relevant knowledge but being unable to trigger a valid retrieval based on the question. Its data characteristics include that the user has learned relevant knowledge and the supplementary content can be corrected. In this case, the target data characteristics can be determined based on the target user's historical learning data and the target user's supplementary content for the target question. If the target data characteristics indicate that the target user has learned relevant knowledge and the target user's supplementary content for the target question can be corrected, it means that the target data characteristics are the data characteristics of retrieval failure, which means that the target user is in a scattered cognitive state. The root cause tracing results corresponding to the root cause classification include retrieval failure.

[0148] For example, the target user may have learned about the components needed for refrigerator cooling in class and answered the question, but the target question has been changed to "What components are needed for equipment cooling?" The target user did not answer "evaporator." The content supplementation instruction is then executed, which includes a prompt about refrigerator cooling. The supplementary content entered by the target user in response to the content supplementation instruction includes "evaporator." This can determine that the root cause tracing result corresponding to the root cause classification includes "search failure."

[0149] Among them, the data feature of scenario mapping failure includes that the user can answer the relevant knowledge correctly in the original context of the question, but cannot answer the relevant knowledge in different contexts or in variant questions (variant questions include questions with different question types, subjects, or expressions). If the target data feature indicates that the user cannot answer the relevant knowledge in different contexts of the target question or in the target question as a variant question, but the target user can answer the relevant knowledge correctly in the original context of the target question or in the original question, it means that the target data feature is a data feature of scenario mapping failure, which means that the target user has a vague cognitive state. The root cause tracing result corresponding to the root cause classification includes scenario mapping failure.

[0150] For example, target users might use the phrase "evaporator absorbs heat - refrigerator interior cools - compressor compresses refrigerant - condenser releases heat to the outside - refrigeration cycle continues" when describing the refrigeration process in a refrigerator. However, when faced with the question "If you were the refrigerant, please describe your journey in the refrigerator for a day," they would not use the phrase "evaporator absorbs heat - refrigerator interior cools - compressor compresses refrigerant - condenser releases heat to the outside - refrigeration cycle continues."

[0151] Optionally, when the problem cannot be corrected by supplementing the content, an original supplementing operation can be performed. The original supplementing operation includes displaying the target question in the original context or displaying the original question, receiving supplementing content input for the original supplementing operation, and when the supplementing content input for the original supplementing operation includes knowledge of the target question, determining that the target user can correctly answer the relevant knowledge in the original context or original question of the target question.

[0152] Among them, the data characteristics of mapping binding errors include related knowledge point binding errors. For example, the object or direction binding error of the concept (the direction of the relationship is reversed or the object is reversed) and the target or recipient binding error of the concept. For example, "good grades make students study hard", which treats the result as the cause and reverses the direction of the relationship. Another example is attributing good grades to the wrong event, which is a target binding error.

[0153] Among them, the data characteristics of insufficient proceduralization include being able to answer objective questions correctly but not subjective questions correctly. If the target question is a subjective question, cannot be answered correctly, and the target data characteristics indicate that the target user can answer the customer question correctly, it means that the user's cognitive state has problems or there is a defect in thinking pattern. The root cause tracing results corresponding to the root cause classification include insufficient proceduralization.

[0154] Optionally, if the supplementary content cannot be corrected, the objective question corresponding to the target question can be displayed, and the answer content for the objective question can be received. If the answer content of the customer question includes relevant knowledge of the target question, then the root cause tracing result is determined to include procedural insufficiency. Alternatively, it can be determined based on historical learning data whether there is an objective question corresponding to the target question in the historical question-solving process. If the answer content of the customer question includes relevant knowledge of the target question, then the root cause tracing result corresponding to the root cause classification is determined to include procedural insufficiency.

[0155] Metacognitive failure refers to knowing knowledge points but not being able to convert them into reasoning steps. For example, knowing the "law of universal gravitation" but not being able to perform step-by-step force analysis on an object based on the "law of universal gravitation". Its data characteristics include being able to answer objective questions correctly or verbally, but not being able to answer subjective questions correctly, or being able to answer correctly after being given a demonstration.

[0156] Optionally, if the supplementary content cannot be corrected, the objective question corresponding to the target question can be displayed, and the answer content for the objective question can be received. If the answer content for the objective question includes relevant knowledge of the target question, then the root cause tracing result is determined to include metacognitive failure. Alternatively, based on historical learning data, it can be determined whether there is an objective question corresponding to the target question in the historical question-solving process. If the answer content for the target question includes relevant knowledge of the target question, then the root cause tracing result corresponding to the root cause classification is determined to include metacognitive failure.

[0157] Optionally, the content supplementation instruction may also include a demonstration, and then determine whether the supplementary content input for the content supplementation instruction can be corrected. If it can be corrected, then determine that the root cause tracing result corresponding to the root cause classification includes metacognitive failure.

[0158] Optionally, the data features of metacognitive failure may also include historical answer features that meet the answer step-by-step condition, which is used to indicate that the user performed poorly in historical answers that are either step-by-step or coherent in reasoning.

[0159] For example, such as Figure 10 As shown, the difference information is the result of difference reasoning, and the root cause classification of the difference reasoning result is known information and incorrect information. The root cause tracing results of known information and incorrect information include knowledge fragmentation and disconnection. The root cause classification of the difference reasoning result is unknown information and incorrect information. The root cause tracing results of unknown information and incorrect information include suspected fabrication and plagiarism.

[0160] Fragmented and disconnected knowledge refers to users' lack of self-verification mechanisms, their inability or habit of reviewing, verifying, or correcting their reasoning (making it impossible to output correctly even if they know the relevant knowledge). Its data characteristics include at least one of the following: lack of review / modification in the editing trajectory (or many modifications without substantial correction), contradictory answers, ignoring system prompts, and lack of improvement trend in the user's history.

[0161] For example, if a target user writes "Industrial emissions cause air pollution" and "Industrial emissions prevent air pollution," which are contradictory statements, and the user still submits the statement after the system prompts "Please check for contradictions," then it is determined that the root cause tracing results corresponding to the root cause classification include fragmented and disconnected knowledge.

[0162] Optionally, target data features can be determined based on at least one of historical learning data, difference information, target response content, target response content, and supplementary content.

[0163] In this embodiment, a large language model is used to determine the target user's target data features based on at least one of historical learning data and target answer content. The root cause tracing results of the information corresponding to the lost points under the error type are then determined based on the target data features. The target data features are used to characterize the target user's cognitive state and / or thinking pattern defects, thereby realizing the determination of root cause tracing results based on multiple data and further improving the accuracy of the root cause tracing results.

[0164] It is understandable that, in response to the answer upload operation, logical problems existing in the first cognitive expression path can also be displayed, or, in response to logical operations on difference information, logical problems existing in the first cognitive expression path can be displayed. In this case, logical problems existing in the first cognitive expression path can be detected, and the detection process can include: Logical detection processing is performed on the first cognitive expression path to obtain the logical detection results.

[0165] The logic detection process refers to the analysis and verification of logical relationships. The logic detection result indicates whether there is a logical problem or not in the first cognitive expression path. The type of logical problem can be set according to the actual situation. For example, logical problems can be causal relationship problems, logical jump problems, overgeneralization problems, self-contradiction, and neutrality. This embodiment does not limit the scope of the problem.

[0166] The method of logic detection processing can be set according to the actual situation. For example, logic detection processing can be performed by preset logic rules, or by knowledge graph, or by language model. This embodiment does not limit the specific method.

[0167] It is understandable that logical detection processing includes causal relationship detection processing. Logical detection processing is performed based on the target answer content to obtain logical detection results, including: By using knowledge graphs, causal relationship detection is performed on the first cognitive expression path to obtain logical detection results.

[0168] Specifically, the first entity node in the first cognitive expression path can be mapped to the knowledge graph. It can be determined whether there are direct connecting edges between the first entity nodes in the knowledge graph. If there are no direct connecting edges, it indicates that the causal reasoning is too simple, lacks intermediate links, and is incomplete, thus indicating that there is a causal relationship logic problem.

[0169] For example, the first cognitive expression path and the sub-knowledge graphs corresponding to evaporator heat absorption in the knowledge graph are as follows: Figure 11 As shown, since there is no direct connection between the evaporator absorbing heat and the condenser releasing heat to the outside, the causal reasoning for determining the first cognitive expression path is too simplistic and has a causal relationship logic problem.

[0170] In this embodiment, the logic detection processing includes causal relationship detection processing. By using a knowledge graph, causal relationship detection processing is performed on the first cognitive expression path to obtain the logic detection result. This realizes the causal relationship detection processing through the knowledge graph, thereby improving the efficiency of obtaining the logic detection result.

[0171] It is understandable that logic detection processing includes logic jump detection processing, which performs logic detection processing on the first cognitive expression path to obtain logic detection results, including: Determine the conditional probability value of the first knowledge point corresponding to each first entity node in the first cognitive expression path. The conditional probability value is used to indicate the degree of influence between the first knowledge points. Based on the conditional probability value, logical jump detection processing is performed on the first cognitive expression path to obtain the logical detection result.

[0172] Here, the conditional probability value represents the probability that one first knowledge point exists or does not exist given that one first knowledge point exists. A lower conditional probability value indicates a lower influence of one first knowledge point on another, while a higher conditional probability value indicates a higher influence of one first knowledge point on another. After obtaining the conditional probability value, the method for performing logical jump detection processing on the first cognitive expression path based on the conditional probability value can be set according to the actual situation; this embodiment does not limit this. In one possible implementation, the difference between the first and second conditional probability values ​​can be determined. If the difference is less than a preset threshold, it means that the causal relationship between the knowledge points is weak, indicating a logical jump problem. The first conditional probability value indicates the probability that one first knowledge point exists given that one first knowledge point exists, and the second conditional probability value indicates the probability that one first knowledge point does not exist given that one first knowledge point exists. For example, the first conditional probability value is P(A|B), and the second conditional probability value is P(A|-B), where A and B represent two first knowledge points, and -B represents the case where the first knowledge point does not exist. In another possible implementation, the target conditional probability value can be divided by the prior probability value, and 1 can be subtracted from the result to obtain the change value. If the difference between the absolute value of the change value and 0 is less than the preset difference, it means that the change value is closer to 0, which means that the probability of another first knowledge point existing when the first knowledge point exists is the prior probability value of the other first knowledge point. For example, the target conditional probability value is P(A|B), and the prior probability value is P(A).

[0173] Optionally, the method for calculating the conditional probability value of the first knowledge point can be set according to the actual situation. For example, the conditional probability value of the first knowledge point can be calculated by Bayes' theorem or probability calculation model. This embodiment does not limit this.

[0174] In this embodiment, the conditional probability value of the first knowledge point corresponding to each first entity node in the first cognitive expression path is determined. The conditional probability value is used to indicate the degree of influence between the first knowledge points. Based on the conditional probability value, logical jump detection processing is performed on the first cognitive expression path to obtain logical detection results, so as to enrich the logical detection processing method and improve the diversity of the obtained logical detection results.

[0175] Understandably, logic detection processing includes propositional logic relation detection, performing logic detection processing on the first cognitive expression path to obtain logic detection results, including: Determine the conclusion information and the prerequisite information corresponding to the target answer content based on the target answer content; Determine the attention weight between the first word in the conclusion information and the second word in the premise information. The attention weight is used to indicate the degree of matching between the first word and the second word. Based on attention weights, propositional logic relationships are detected in the first cognitive expression path, and logic detection results are obtained.

[0176] In this approach, the target answer can be defined as the conclusion information, while the background information related to the target question can be used as the premise information corresponding to the target answer. For example, if the target answer is "Schools use a certain learning model to improve student performance, therefore, all high-performing schools must have adopted this learning model," and the background information is "Many high-performing schools still use traditional lecture methods or a combination of various teaching strategies," then the conclusion information is "Schools use a certain learning model to improve student performance, therefore, all high-performing schools must have adopted this learning model," and the premise information is "Many high-performing schools still use traditional lecture methods or a combination of various teaching strategies."

[0177] Propositional logic relationship detection, also known as implication analysis, is used to analyze premise and conclusion information to infer the logical relationship between them. The logical relationship between premise and conclusion can be one of the following: implication, contradiction, or neutrality. An implication indicates that the premise supports the conclusion; a contradiction indicates that the premise and conclusion conflict; and neutrality indicates that the premise cannot support or refute the conclusion.

[0178] For example, if the target answer is "Schools use a certain learning model to improve student grades, therefore, all high-achieving schools must have adopted this learning model," and the premise is "Many high-achieving schools still use traditional lecture methods or a combination of teaching strategies," the logic check result is self-contradictory. Another example is if the target answer is "Students use tablets for learning, leading to improved grades," and the premise is "Many schools have indeed observed improved grades in some students after introducing tablet teaching; however, research also shows that improved academic performance is more related to factors such as instructional design, teacher guidance, and student engagement," the logic check result is neutral.

[0179] Optionally, the conclusion information and premise information can be encoded first to obtain semantic features, and then the attention weight between the first word and the second word can be determined based on the semantic features. Optionally, the conclusion information and premise information can be encoded using formula (1) to obtain semantic features: (1) in, Input Represents semantic features, [CLS] Indicates input to transformerThe global representation of the entire sequence. [SEP] Used to distinguish between premise information and conclusion information. Premise Indicates prerequisite information, Hypothesis This indicates the conclusion information.

[0180] After obtaining semantic features, then through transformer Based on semantic features, the attention weights between the first and second words are determined. Then, the attention weights are normalized to obtain the cross-sentence semantic similarity between the first and second words. Finally, the logical detection result is determined based on the cross-sentence semantic similarity. Optionally, the attention weights can be substituted into formula (2) for normalization to obtain the cross-sentence semantic similarity: (2) in, a i,j Indicates the first i The first word and the first j Cross-sentence semantic similarity between the second words e i,j Indicates the first i The first word and the first j Attention weights between each second word exp() Indicates exponentiation. k This indicates the number of the second word in the premise information. Formula (2) can be understood as... transformer In Softmax The calculation process of the function.

[0181] After obtaining the cross-sentence semantic similarity, information features that incorporate contextual information can be generated based on the cross-sentence semantic similarity (for example, multiplying the cross-sentence semantic similarity by the V vector to obtain information features). These information features are... transformer The output can optionally be expressed by formula (3). transformer Output: (3) After obtaining the information features, the information features can be classified to obtain the logic detection results. Specifically, the information features can be input into formula (4) for calculation to obtain the logic detection results: (4) in, y Indicates logical relationships, P Probability representing a logical relationship softmax Represents a classifier. W Indicates weight, b Indicates bias.

[0182] In this embodiment, the logic detection process includes propositional logic relationship detection, determining conclusion information and premise information corresponding to the target answer content based on the target answer content; determining the attention weight between the first word in the conclusion information and the second word in the premise information, the attention weight being used to indicate the degree of matching between the first word and the second word; performing propositional logic relationship detection on the first cognitive expression path based on the attention weight to obtain logic detection results, thereby enriching the logic detection processing methods and improving the diversity of the obtained logic detection results.

[0183] It is understandable that, in response to an upload operation or a linked operation for any discrepancy information, the completeness of the first cognitive expression path can also be displayed. In this case, this embodiment further includes: Determine the similarity between the second cognitive expression path and the first cognitive expression path to obtain the path completeness of the first cognitive expression path.

[0184] The method for calculating the similarity between the second cognitive expression path and the first cognitive expression path can be set according to the actual situation. For example, the similarity between the second cognitive expression path and the first cognitive expression path can be calculated by the Euclidean distance formula or the cosine similarity formula. This embodiment does not limit this method.

[0185] In this embodiment, the similarity between the second cognitive expression path and the first cognitive expression path is determined, and the similarity is used as the path completeness of the first cognitive expression path, so as to quantitatively evaluate the coverage of the knowledge points involved in the target question by the first cognitive expression path, and improve the convenience for the target user to understand the degree of lack of information about the differences.

[0186] It is understood that this embodiment also includes: This displays analytical information during the process of determining the root cause tracing results.

[0187] Among them, analyzing information refers to the deductive process of obtaining the root cause tracing results. For example, if the target question is to explain the process of refrigerator refrigeration, the first cognitive expression path is "evaporator absorbs heat → refrigerator interior cools", and the second cognitive expression path is: "evaporator absorbs heat - refrigerator interior cools - compressor compresses refrigerant - condenser releases heat to the outside - refrigeration cycle continues".

[0188] The difference information refers to the differences in knowledge points, including the compressor compressing the refrigerant, the condenser releasing heat to the outside environment, and the continuous operation of the refrigeration cycle.

[0189] The process by which the terminal determines the root cause analysis result can be as follows, that is, the information analysis can be as follows: Based on the target user's historical learning data, it was determined that the target user had previously answered a complete logical chain: "Evaporator absorbs heat - refrigerator interior cools - compressor compresses refrigerant - condenser releases heat to the outside - refrigeration cycle continues"; Therefore, the root cause analysis revealed a lack of attention.

[0190] In this embodiment, the process analysis information for determining the root cause tracing results is displayed so that the target user can understand the cause of the error in more detail, further helping the target user to clarify the learning direction, thereby further improving the user's ability.

[0191] Understandably, the fine-tuning process of a large language model includes: Obtain training samples, which include differential labels for sample response content and sample response content; Using a large language model, based on the sample answer content, sample difference information is generated for the sample answer content, and sample root cause tracing results of the sample difference information under the sample root cause classification are generated. The sample difference information is the information corresponding to the points lost in the sample answer content, and the sample root cause classification is the error type to which the sample difference information belongs. Based on the difference labels and sample difference information, the first loss function value corresponding to the difference information identification task is determined, and the second loss function value corresponding to the root cause tracing task is determined based on the sample root cause tracing results. The large language model is trained based on the first loss function value, the first weight corresponding to the first loss function value, the second loss function value, and the second weight corresponding to the second loss function value.

[0192] Specifically, through a large language model, based on the sample answer content, a first cognitive expression path and a second cognitive expression path for the sample answer content are generated. Based on the first cognitive expression path and the second cognitive expression path, sample difference information is determined. Alternatively, sample difference information can be determined based on the sample answer content and sample reference content.

[0193] After obtaining the sample difference information, the sample root cause classification corresponding to the sample difference information can be determined. Then, based on at least one of the user's sample historical learning data, sample answer content, sample difference information, and sample supplementary content, the sample root cause tracing result under the sample root cause classification can be generated.

[0194] Optionally, the difference label is used to indicate specific label difference information or the difference label can be used to indicate whether it is sample difference information. For example, the difference label is 0 or 1, where 1 indicates that the information is sample difference information and 0 indicates that the information is not sample difference information.

[0195] The difference information recognition task is used to learn to identify difference information, while the root cause tracing task is used to learn to determine the root cause tracing result. Based on the difference labels and sample difference information, the loss function type for determining the first loss function value corresponding to the difference information recognition task can be set according to the actual situation. For example, it can be determined using the cross-entropy loss function or hinge loss. This embodiment does not limit this specific approach.

[0196] When calculating using the cross-entropy loss function, it can be done using formula (5): (5) in, L gap This represents the value of the first loss function. y q Indicates the first q Difference tags for individual information, This indicates the probability that the information belongs to the sample difference information.

[0197] Optionally, the loss function type for determining the second loss function value corresponding to the root cause tracing task can be set according to the actual situation. For example, the second loss function can be a cross-entropy loss function, a hinge loss function, or a mean absolute error loss function. This embodiment does not limit this.

[0198] After obtaining the first loss function value and the second loss function value, the first loss function value can be multiplied by the first weight to obtain the first loss result. The second loss function value can be multiplied by the second weight to obtain the second loss result. The first loss result and the second loss result can be added together to obtain the target loss function value. If the target loss function value indicates that the large language model has converged, then fine-tuning stops. If the target loss function value indicates that the large language model has not converged, then the large language model is updated based on the target loss function value, and the process returns to the steps of generating sample difference information based on the sample answer content through the large language model and generating sample root cause tracing results of the sample difference information under the sample root cause classification.

[0199] The first weight controls the degree of attention the large language model pays to the differential information recognition task. If the first weight is large, the large language model tends to optimize the accuracy of differential information recognition, that is, to focus on whether the user is missing a certain knowledge point, missing a certain differential reasoning result, or misusing a certain knowledge point or misusing a certain differential reasoning result. The second weight controls the degree of optimization of the large language model for the root cause tracing task. If the second weight is large, the large language model pays more attention to optimizing the completeness and rationality of the root cause tracing results.

[0200] Specifically, the target loss function value can be obtained by multiplying the first loss function value, the first weight, the second loss function value, and the second weight and substituting them into formula (6): (6) in, L This represents the target loss function value. Indicates the first weight. This represents the first weight. The sum of the first and second weights is 1.

[0201] Optionally, the first weight and the second weight can be pre-set weights, or the first weight and the second weight can be weights selected during the training process. This embodiment does not limit this.

[0202] Optionally, the training samples may also include sample reference content for the sample questions, so that the sample second knowledge point can be obtained from the sample reference content, and the sample second cognitive expression path can be generated based on the sample second knowledge point.

[0203] Optionally, the training samples may also include rating labels for the sample answers, so that the large language model can learn to output ratings for the sample answers. Optionally, the training samples may also include users' historical learning data, which may include users' educational background, major, grades, learning ability, course information, and / or test-taking data.

[0204] Optionally, when determining the similarity between the first and second cognitive expression paths, a third loss function can be determined based on the first and second cognitive expression paths of the samples. The large language model is then trained based on the first, second, and third loss functions. The third loss function can be set according to the actual situation. For example, the third loss function can be a contrastive learning function, as shown in formula (7): (7) in, L bias This represents the value of the third loss function. v m,n Indicates the first n The first cognitive expression path and the second sample m The label between the second cognitive expression paths of the samples can be 1 or 0. A value of 1 indicates that the first and second cognitive expression paths of the samples are similar, while a value of 0 indicates that the first and second cognitive expression paths of the samples are not similar. d（） Represents cosine similarity. f(x) m ） Indicates the firstm The second cognitive expression path of each sample f(x) n ） Indicates the first n The first cognitive expression path of each sample.

[0205] Optionally, before fine-tuning the large language model, it can be trained using data from a specific vertical domain to enable the trained large language model to have general language comprehension and reasoning abilities, and then the trained large language model can be fine-tuned.

[0206] For example, when the vertical field is education, the data in the vertical field can be the user's historical learning data. Historical learning data can include test-taking data, such as the first question, the standard answer to the first question, and the first response to the first question.

[0207] Optionally, historical learning data may also include the user's course information, such as the courses the user has taken, their academic background, and their learning trajectory.

[0208] Optionally, historical learning data may also include blind spot data, which may include users' error records on different knowledge points, knowledge forgetting, and interdisciplinary knowledge gaps.

[0209] In this embodiment, training samples are obtained, including sample answer content and difference labels for the sample answer content. Using a large language model, based on the sample answer content, sample difference information and root cause tracing results of the sample difference information under the sample root cause classification are generated. The sample difference information corresponds to the points lost in the sample answer content, and the sample root cause classification is the error type to which the sample difference information belongs. Based on the difference labels and sample difference information, a first loss function value corresponding to the difference information recognition task is determined, and a second loss function value corresponding to the root cause tracing task is determined based on the sample root cause tracing results. The large language model is fine-tuned based on the first loss function value, the first weight corresponding to the first loss function value, the second loss function value, and the second weight corresponding to the second loss function value, so as to improve the accuracy of the difference information and root cause tracing results in determining the difference information and root cause tracing results through the fine-tuned large language model.

[0210] It is understood that this embodiment also includes: A weight combination is determined from a first weight set and a second weight set, the weight combination including the first weight and the second weight; The large language model is fine-tuned based on the first loss function value, the first weight corresponding to the first loss function value, the second loss function value, and the second weight corresponding to the second loss function value, including: Based on the first weight, the first loss function value, the second weight, and the second loss function value, the large language model is trained to obtain a candidate large language model; Based on the validation samples, determine the performance indicators of candidate large language models; If the performance indicator information does not meet the preset performance conditions or the number of weight updates does not reach the preset number, then based on the performance indicator information, a new weight combination is determined from the first weight set and the second weight set, and the candidate large language model is used as the new large language model. Then, the process is returned to execute the steps of generating sample difference information and sample root cause tracing results based on the sample answer content through the large language model. If the performance indicator information meets the preset performance conditions or the number of weight updates reaches the preset number, a new weight combination is determined from the weight combination, the new weight combination is fixed, and the candidate large language model is used as the new large language model. The process is then returned to execute the large language model. Based on the sample answer content, sample difference information and sample root cause tracing results are generated for the sample answer content until the candidate large language model converges, and the fine-tuned large language model is obtained.

[0211] The first weight set and the second weight set are pre-set sets. For example, the first weight set is [0.1, 0.5, 1.0] and the second weight set is [0.1, 0.5, 1.0]. Since different weight combinations have different effects on the model, selecting weight combinations during the training process can improve the model's performance.

[0212] The large language model is trained based on the first weight, the first loss function value, the second weight, and the second loss function value to obtain a candidate large language model. Alternatively, the large language model can be trained for a first preset number of times based on the first weight, the first loss function value, the second weight, and the second loss function value to obtain a candidate large language model.

[0213] Performance indicator information is used to indicate the performance of candidate large language models in the difference information recognition task and root cause tracing task. Its type can be set according to the actual situation. For example, performance indicator information can be the loss function value, the accuracy of candidate large language models or the F1 score. This embodiment does not limit it.

[0214] When the performance indicator information is a loss function value, the preset performance condition can be that the decrease in the loss function value is less than a preset threshold. When the performance indicator information is accuracy, the preset performance condition can be that the increase in accuracy is less than a first preset value or the accuracy is less than a preset accuracy. When the performance indicator information is an F1 score, the preset performance condition can be that the increase in accuracy is less than a first preset value or the F1 score is less than a preset score. The embodiments of this application are not limited here.

[0215] Optionally, the method for determining a new weight combination from the first weight set and the second weight set based on performance indication information can be set according to the actual situation. For example, a Bayesian optimization network or a genetic algorithm can be used to determine a new weight combination from the first weight set and the second weight set based on performance indication information. This embodiment does not limit this method.

[0216] The new weight combinations determined from the first weight set and the second weight set are different from the previously selected weight combinations. For example, the first weight set is [0, 0.1, 0.5, 0.9, 1.0], and the second weight set is [0, 0.1, 0.5, 0.9, 1.0]. The weight combination obtained the first time is (0.1, 0.9), and the weight combination obtained the second time is (0.5, 0.5).

[0217] Determining a new weight combination from the existing weight combinations can be understood as selecting one of the chosen weight combinations as the new weight combination. For example, if the selected weight combinations include (0.1, 0.9), (0.5, 0.5), and (0.9, 0.1), the new weight combination could be (0.5, 0.5). Optionally, the weight combination corresponding to the best performance indicator among the existing weight combinations can be determined as the new weight combination. Fixing the new weight combination can be understood as not updating the weight combinations during the training of the candidate large language model.

[0218] For example, a weight combination (0.1, 0.9) is determined from the first weight set and the second weight set. The weight combination includes the first weight and the second weight. The following steps are performed by the large language model for a first preset number of times: Based on the sample answer content, sample difference information and sample root cause tracing results are generated for the sample answer content; based on the difference label and sample difference information, the first loss function value corresponding to the difference information recognition task is determined; based on the sample root cause tracing results, the second loss function value corresponding to the root cause tracing task is determined; the large language model is trained based on the first loss function value, the first weight corresponding to the first loss function value, the second loss function value, and the second weight corresponding to the second loss function value to obtain a candidate large language model.

[0219] Based on the validation samples, performance indicator information of the candidate large language model is determined. If the performance indicator information does not meet the preset performance conditions, a new weight combination (0.5, 0.5) is determined from the first weight set and the second weight set based on the performance indicator information. The following steps are then performed on the candidate large language model for the first preset number of times: Based on the sample answer content, sample difference information and sample root cause tracing results are generated for the sample answer content. Based on the difference label and sample difference information, the first loss function value corresponding to the difference information recognition task is determined. Based on the sample root cause tracing results, the second loss function value corresponding to the root cause tracing task is determined. The candidate large language model is trained based on the first loss function value, the first weight corresponding to the first loss function value, the second loss function value, and the second weight corresponding to the second loss function value to obtain a new candidate large language model.

[0220] Based on the validation samples, performance indicators for new candidate large language models are determined. If the performance indicators meet the preset performance conditions, a new weight combination is selected from (0.1, 0.9) and (0.5, 0.5). If the new weight combination is (0.5, 0.5), the process returns to generate sample difference information and sample root cause tracing results based on the sample answer content. Based on the knowledge point tags and sample difference information, the first loss function value corresponding to the difference information recognition task is determined, and the second loss function value corresponding to the root cause tracing task is determined based on the sample root cause tracing results. The new candidate large language model is trained based on the first loss function value, the first weight corresponding to the first loss function value, the second loss function value, and the second weight corresponding to the second loss function value until the new candidate large language model converges, resulting in a fine-tuned large language model.

[0221] It is understandable that the convergence of the candidate large language model can be interpreted as the target loss function value indicating that the candidate large language model has converged, or as the number of training iterations reaching a second preset number indicating that the candidate large language model has converged.

[0222] In this embodiment, a weight combination is determined from a first weight set and a second weight set. The weight combination includes the first weight and the second weight. Based on the first weight, the first loss function value, the second weight, and the second loss function value, the large language model is trained to obtain a candidate large language model. Based on the validation samples, performance indicator information of the candidate large language model is determined. If the performance indicator information does not meet the preset performance conditions or the number of weight updates does not reach the preset number, a new weight combination is determined from the first weight set and the second weight set based on the performance indicator information, and the candidate large language model is used as the new large language model. The process is then returned to execute the approved large language model. Based on the sample response content, the steps include generating sample difference information and root cause tracing results for the sample response content; if the performance indicator information meets the preset performance conditions or the number of weight updates reaches the preset number, a new weight combination is determined from the weight combination, the new weight combination is fixed, and the candidate large language model is used as the new large language model. The process is then returned to execute. Based on the sample response content, the large language model generates sample difference information and root cause tracing results for the sample response content until the candidate large language model converges, resulting in a fine-tuned large language model. This achieves automatic search to obtain the best weight combination and improves the performance of the fine-tuned large language model.

[0223] It is understandable that, in the process of training the candidate large language model based on determining a new weight combination from the weight combination, this embodiment also includes: Based on the first accuracy rate corresponding to the difference information identification task and the second accuracy rate corresponding to the root cause tracing task, the first weight and the second weight in the new weight combination are adjusted. Alternatively, based on the first and second loss function values, the first and second weights in the new weight combination can be adjusted.

[0224] The first and second loss function values ​​are obtained during the training of the candidate large language model. When the first and second accuracies indicate that the candidate large language model is overly focused on the difference information recognition task and neglects the root cause tracing task, the first weight can be reduced and the second weight increased. Similarly, when the first and second loss function values ​​indicate that the candidate large language model is overly focused on the difference information recognition task and neglects the root cause tracing task, the first weight can be increased and the second weight increased. Finally, when the first and second loss function values ​​indicate that the candidate large language model is overly focused on the root cause tracing task and neglects the difference information recognition task, the first weight can be increased and the second weight decreased.

[0225] For example, if the first accuracy rate is less than the first preset accuracy rate and the second accuracy rate is greater than the second preset accuracy rate, it indicates that the root cause tracing task is being overemphasized while the difference information identification task is being ignored. In this case, the first weight is increased and the second weight is decreased. If the first accuracy rate is greater than the second preset accuracy rate and the second accuracy rate is less than the first preset accuracy rate, it indicates that the difference information identification task is being overemphasized while the root cause tracing task is being ignored. In this case, the first weight is decreased and the second weight is increased.

[0226] For example, if the first loss function value is much smaller than the second loss function value, it indicates an overemphasis on the root cause tracing task and a neglect of the differential information identification task. In this case, the first weight can be increased and the second weight decreased. If the first loss function value is much larger than the second loss function value, it indicates an overemphasis on the differential information identification task and a neglect of the root cause tracing task. In this case, the first weight can be decreased and the second weight increased.

[0227] In this embodiment, during the training of the candidate large language model based on the new weight combination determined from the weight combination, the first weight and the second weight in the new weight combination are adjusted based on the first accuracy corresponding to the difference information recognition task and the second accuracy corresponding to the root cause tracing task; or, the first weight and the second weight in the new weight combination are adjusted based on the first loss function value and the second loss function value, so as to realize the dynamic adjustment of the first weight and the second weight, so as to balance the optimization direction of the difference information recognition task and the root cause tracing task, and further improve the performance of the knowledge point processing model.

[0228] It is understood that this embodiment also includes: Based on the difference information and / or root cause tracing results, a learning path is generated for the target user.

[0229] Specifically, by using discrepancy information, we can identify knowledge points unknown to the target user, known but unused knowledge points, unknown reasoning results, and / or known but unused reasoning results. Based on this discrepancy information, we generate relevant materials, and based on the names of these materials, we generate a learning path. Each node on the learning path can represent the name of the relevant material. For example, if the target answer is "The evaporator absorbs heat, so the inside of the refrigerator can get cold," the discrepancy information includes the compressor compressing the refrigerant, the condenser releasing heat to the outside, and the continuous refrigeration cycle. The generated learning path could be: reading the working principle of the compressor → the working principle of the condenser → the closed loop of heat absorption and release.

[0230] Optionally, after obtaining the difference information, relevant materials can also be directly recommended to the target user. This embodiment does not limit this.

[0231] In this embodiment, a learning path is generated for the target user based on the difference information and / or root cause tracing results. This eliminates the need for the target user to plan the learning path themselves, improves the convenience for the target user to learn relevant information on the difference information or root cause tracing results, and provides the user with "cognitive upgrade" rather than just "knowledge reinforcement", thereby expanding their thinking.

[0232] Understandably, when the difference information includes different knowledge points, a learning path is generated for the target user based on this difference information, including: Based on the difference information, the knowledge entropy of the target user in the knowledge domain of the difference knowledge points is determined. The knowledge entropy is positively correlated with the target user's mastery of the knowledge points in the knowledge domain. Based on knowledge entropy, a learning path for the target user is generated, where knowledge points in knowledge domains with lower knowledge entropy are placed earlier in the learning path.

[0233] Knowledge entropy measures a target user's mastery of knowledge points within a knowledge domain. A lower knowledge entropy indicates a lower level of mastery, while a higher entropy indicates a higher level of mastery. Knowledge entropy can be calculated using Shannon's formula for information entropy. Optionally, the knowledge entropy for the target user's specific knowledge domain related to the differing knowledge points can be determined based on their existing responses. Alternatively, a large language model can be used to determine the knowledge entropy for the target user's specific knowledge domain related to the differing knowledge points, based on the difference information, and a learning path for the target user can be generated based on this knowledge entropy.

[0234] In this embodiment, based on the difference information, the knowledge entropy of the knowledge domain in which the target user has the difference knowledge points is determined. The knowledge entropy is positively correlated with the target user's mastery of the knowledge points in the knowledge domain. Based on the knowledge entropy, a learning path for the target user is generated. In this path, the knowledge points in the knowledge domain with lower knowledge entropy are placed earlier, so that the knowledge points with lower mastery are recommended to the target user first, thereby improving the efficiency and effectiveness of the target user in learning the missing knowledge points.

[0235] To better implement the above methods, this application also provides a knowledge tracing device, which can be integrated into an electronic device, such as a computer device, which can be a terminal, server or other device.

[0236] For example, in this embodiment, the method of this application embodiment will be described in detail by taking the knowledge tracing device specifically integrated into the terminal as an example. This embodiment provides a knowledge tracing device, such as... Figure 12 As shown, the device for tracing the origins of this knowledge may include: The positioning module 1201 is used to locate at least one difference information from the target answer content uploaded by the target user, wherein the difference information is the information corresponding to the point loss in the target answer content; The determination module 1202 is used to determine the root cause classification of each discrepancy information. The root cause classification is the error type to which the information corresponding to the point of loss belongs. The error type includes known information and unknown information; and / or, the error type includes missing information and redundant information. Here, known information indicates that the target user knows the information corresponding to the point of loss, unknown information indicates that the target user does not know the information corresponding to the point of loss, missing information indicates that the information corresponding to the point of loss is missing from the target answer content, and redundant information indicates that the information corresponding to the point of loss is redundant from the target answer content. The root cause tracing result of the discrepancy information under the error type is determined by the large language model.

[0237] It is understandable that the positioning module 1201 is specifically used for: Generate the target user's first cognitive expression path on the question content corresponding to the target answer content; By comparing the cognitive differences between the first cognitive expression path and the second cognitive expression path of the reference content, the difference information is obtained. The cognitive expression path is a logical reasoning chain consisting of entity nodes corresponding to each knowledge point and logical edges connecting each entity node.

[0238] It is understandable that the positioning module 1201 is specifically used for: Extract at least one primary knowledge point from the target answer content and the logical relationship between each primary knowledge point; Based on the first knowledge point and logical relationship, the first cognitive expression path is obtained, which includes the entity nodes corresponding to each first knowledge point and the logical edges corresponding to each logical relationship.

[0239] It is understandable that module 1202 is used to perform: If the root cause classification of the discrepancy information includes known information and missing information, a content supplementation instruction is generated to obtain the supplementary content input by the target user in response to the content supplementation instruction; By using a large language model, the root cause tracing results of the information corresponding to the points lost are determined based on the supplementary content, under the conditions of known information and missing information.

[0240] It is understandable that the content supplementation instructions include target questions and / or prompts, and the target questions can be asked in ways that include counterfactual questions.

[0241] It is understandable that module 1202 is used to perform: The target data features are determined by using a large language model based on at least one of historical learning data and target answer content. The root cause tracing results of the information corresponding to the lost points under the error type are then determined based on the target data features. The target data features are used to characterize the cognitive state and / or thinking pattern defects of the target user.

[0242] It is understood that the above-mentioned device also includes a training module, which is used to perform: Obtain training samples, which include sample response content and differential labels of sample response content; Using a large language model, based on the sample answer content, sample difference information is generated for the sample answer content, and sample root cause tracing results of the sample difference information under the sample root cause classification are generated. The sample difference information is the information corresponding to the points lost in the sample answer content, and the sample root cause classification is the error type to which the sample difference information belongs. Based on the difference labels and sample difference information, the first loss function value corresponding to the difference information identification task is determined, and the second loss function value corresponding to the root cause tracing task is determined based on the sample root cause tracing results. The large language model is fine-tuned based on the first loss function value, the first weight corresponding to the first loss function value, the second loss function value, and the second weight corresponding to the second loss function value.

[0243] It is understandable that the second cognitive expression path is obtained through a knowledge graph, which includes entity nodes corresponding to knowledge points in at least one knowledge domain.

[0244] Understandably, the determination module is also used for execution: Based on the difference information and / or root cause tracing results, a learning path is generated for the target user.

[0245] In practice, each of the above modules can be implemented as an independent entity or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation methods and corresponding beneficial effects of each of the above modules, please refer to the previous method embodiments, which will not be repeated here.

[0246] Accordingly, embodiments of this application also provide an electronic device, which can be a terminal, such as a smartphone, tablet computer, laptop computer, touch screen, game console, personal computer (PC), personal digital assistant (PDA), or other terminal device. Figure 13 As shown, Figure 13This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1300 includes a processor 1301 with one or more processing cores, a memory 1302 with one or more computer-readable storage media, and a computer program stored on the memory 1302 and executable on the processor. The processor 1301 and the memory 1302 are electrically connected. Those skilled in the art will understand that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0247] The processor 1301 is the control center of the electronic device 1300. It connects various parts of the electronic device 1300 through various interfaces and lines. By running or loading software programs and / or modules stored in the memory 1302, and calling data stored in the memory 1302, it performs various functions of the electronic device 1300 and processes data, thereby monitoring the electronic device 1300 as a whole.

[0248] In this embodiment, the processor 1301 in the electronic device 1300 loads the instructions corresponding to the processes of one or more applications into the memory 1302 according to the following steps, and the processor 1301 runs the applications stored in the memory 1302 to realize various functions, such as: Identify at least one discrepancy from the target answer content uploaded by the target user, wherein the discrepancy is the information corresponding to the point deduction in the target answer content; Determine the root cause classification of each discrepancy information. The root cause classification is the error type to which the information corresponding to the point of loss belongs. The error type includes known information and unknown information; and / or, the error type includes missing information and redundant information. Here, known information indicates that the target user knows the information corresponding to the point of loss, unknown information indicates that the target user does not know the information corresponding to the point of loss, missing information indicates that the information corresponding to the point of loss is missing from the target answer content, and redundant information indicates that the information corresponding to the point of loss is redundant from the target answer content. The root cause of the discrepancies was determined using a large language model under the error type.

[0249] For details on the specific implementation methods of each of the above operations and their corresponding beneficial effects, please refer to the detailed description of the method for tracing the origins of knowledge content above, which will not be repeated here.

[0250] Optional, such as Figure 13As shown, the electronic device 1300 also includes: a touch display screen 1303, a radio frequency circuit 1304, an audio circuit 1305, an input unit 1306, and a power supply 1307. The processor 1301 is electrically connected to the touch display screen 1303, the radio frequency circuit 1304, the audio circuit 1305, the input unit 1306, and the power supply 1307. Those skilled in the art will understand that... Figure 13 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0251] The touch display screen 1303 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 1303 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1301. It can also receive and execute commands from the processor 1301. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 1301 to determine the type of touch event. Subsequently, the processor 1301 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the touch display screen 1303 to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen 1303 can be implemented as two independent components to achieve input and output functions. That is, the touch display screen 1303 can also be used as part of the input unit 1306 to achieve input functions.

[0252] The radio frequency circuit 1304 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other electronic devices, and to transmit and receive signals with network devices or other electronic devices.

[0253] Audio circuit 1305 can be used to provide an audio interface between a user and an electronic device via a speaker and a microphone. Audio circuit 1305 can convert received audio data into electrical signals and transmit them to the speaker, where the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuit 1305, converted back into audio data, and then processed by processor 1301 before being transmitted via radio frequency circuit 1304 to, for example, another electronic device, or output to memory 1302 for further processing. Audio circuit 1305 may also include an earphone jack to provide communication between peripheral headphones and electronic devices.

[0254] The input unit 1306 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0255] Power supply 1307 is used to supply power to various components of electronic device 1300. Optionally, power supply 1307 can be logically connected to processor 1301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 1307 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0256] although Figure 13 As not shown in the diagram, the electronic device 1300 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0257] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0258] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0259] Therefore, embodiments of this application provide a computer-readable storage medium storing multiple computer programs that can be loaded by a processor to execute any of the knowledge content tracing methods provided in embodiments of this application. For example, the computer program can perform the following steps: Identify at least one discrepancy from the target answer content uploaded by the target user, wherein the discrepancy is the information corresponding to the point deduction in the target answer content; Determine the root cause classification of each discrepancy information. The root cause classification is the error type to which the information corresponding to the point of loss belongs. The error type includes known information and unknown information; and / or, the error type includes missing information and redundant information. Here, known information indicates that the target user knows the information corresponding to the point of loss, unknown information indicates that the target user does not know the information corresponding to the point of loss, missing information indicates that the information corresponding to the point of loss is missing from the target answer content, and redundant information indicates that the information corresponding to the point of loss is redundant from the target answer content. The root cause of the discrepancies was determined using a large language model under the error type.

[0260] For details on the specific implementation methods of each of the above operations and their corresponding beneficial effects, please refer to the detailed description of the method for tracing the origins of knowledge content above, which will not be repeated here.

[0261] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0262] Since the computer program stored in the computer-readable storage medium can execute any of the knowledge content tracing methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the knowledge content tracing methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0263] The foregoing has provided a detailed description of a method, apparatus, device, and readable storage medium for tracing the source of knowledge content provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method of tracing the origin of knowledge content, characterized by, The method comprises: locating at least one difference information from target answer content uploaded by a target user, wherein the difference information is information corresponding to a missed point of the target answer content; determining a root cause classification of each of the difference information, wherein the root cause classification is an error type to which the information corresponding to the missed point belongs, the error type including information known and information unknown, and / or the error type including information missing and information redundant, wherein the information known indicates that the target user knows the information corresponding to the missed point, the information unknown indicates that the target user does not know the information corresponding to the missed point, the information missing indicates that the information corresponding to the missed point is missing information of the target answer content, and the information redundant indicates that the information corresponding to the missed point is redundant information of the target answer content; determining a root cause tracing result of the difference information under the error type by a large language model.

2. The method of tracing the origin of knowledge content according to claim 1, wherein, The locating at least one difference information from target answer content uploaded by a target user comprises: generating a first cognitive expression path of the target user on a question content corresponding to the target answer content according to the target answer content; comparing a cognitive difference between the first cognitive expression path and a second cognitive expression path of reference content to obtain the difference information, wherein the cognitive expression path is a logical reasoning chain composed of entity nodes corresponding to each knowledge point and logical edges connecting each entity node.

3. The method of tracing the origin of knowledge content according to claim 2, wherein, The generating a first cognitive expression path of the target user on a question content corresponding to the target answer content according to the target answer content comprises: extracting at least one first knowledge point in the target answer content and a logical relationship between each of the first knowledge points; obtaining a first cognitive expression path including entity nodes corresponding to each of the first knowledge points and logical edges corresponding to each of the logical relationships according to the first knowledge points and the logical relationships.

4. The method of claim 1, wherein, The determining a root cause tracing result of the difference information under the error type by a large language model comprises: if the root cause classification of the difference information includes information known and information missing, generating a content supplement instruction to obtain supplement content input by the target user for the content supplement instruction; determining a root cause tracing result of the information corresponding to the missed point under the information known and the information missing based on the supplement content by a large language model.

5. The method of tracing the origin of knowledge content according to claim 4, wherein, The content supplement instruction includes a target question and / or a prompt operation, and a question mode of the target question includes a counterfactual question.

6. The method of claim 1, wherein, The determining a root cause tracing result of the difference information under the error type by a large language model comprises: determining a target data feature of the target user according to at least one of historical learning data and the target answer content by a large language model, to determine a root cause tracing result of the information corresponding to the missed point under the error type based on the target data feature, wherein the target data feature is used to represent a cognitive state and / or a thinking mode defect of the target user.

7. The method of claim 1, wherein, The fine-tuning process of the large language model comprises: obtain a training sample, the training sample comprising sample answer content and a difference label of the sample answer content; generate, by a large language model, sample difference information corresponding to the sample answer content based on the sample answer content, and generate a sample root cause tracing result of the sample difference information under a sample root cause classification, wherein the sample difference information is information corresponding to a missed point in the sample answer content, and the sample root cause classification is an error type to which the sample difference information belongs; determine a first loss function value corresponding to a difference information identification task based on the difference label and the sample difference information, and determine a second loss function value corresponding to a root cause tracing task based on the sample root cause tracing result; fine-tune the large language model based on the first loss function value, a first weight corresponding to the first loss function value, the second loss function value, and a second weight corresponding to the second loss function value.

8. The method of tracing the origin of knowledge content of claim 2, wherein, The second cognitive expression path is obtained through a knowledge graph, and the knowledge graph comprises entity nodes corresponding to knowledge points in at least one knowledge field.

9. A method of tracing the origin of knowledge content according to any one of claims 1-8, characterized in that, The method further comprises: generating a learning path for the target user based on the difference information and / or the root cause tracing result.

10. A provenance-of-knowledge apparatus, comprising: The device comprises: a positioning module configured to locate at least one difference information from target answer content uploaded by a target user, wherein the difference information is information corresponding to a missed point in the target answer content; a determination module configured to determine a root cause classification of each of the difference information, wherein the root cause classification is an error type to which the information corresponding to the missed point belongs, the error type comprises known information and unknown information, and / or the error type comprises missing information and redundant information, wherein the known information indicates that the target user knows the information corresponding to the missed point, the unknown information indicates that the target user does not know the information corresponding to the missed point, the missing information indicates that the information corresponding to the missed point is missing information of the target answer content, and the redundant information indicates that the information corresponding to the missed point is redundant information of the target answer content; and a large language model is used to determine a root cause tracing result of the difference information under the error type.

11. An electronic device, comprising: The device comprises a processor and a memory, and the memory stores a plurality of instructions; the processor loads the instructions from the memory to execute the root cause tracing method of knowledge content according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions, and the instructions are adapted to be loaded by a processor to execute the root cause tracing method of knowledge content according to any one of claims 1 to 9.

Images