A knowledge base correction method, electronic device and computer readable storage medium

Abstract

The application discloses a knowledge base correction method, an electronic device and a computer readable storage medium. The method comprises the following steps: obtaining user feedback information corresponding to a target answer; wherein the target answer is obtained by replying to a target question based on a target knowledge base; performing intention recognition on the user feedback information to obtain a user feedback intention; wherein the user feedback intention is used to represent the evaluation feedback of the user on the target answer; and correcting the target knowledge base based on the user feedback intention. Through the above method, the knowledge base correction based on the user feedback information can be realized.

Description

Technical Field

[0001] This application relates to the field of knowledge base optimization technology, and in particular to a knowledge base correction method, electronic device, and computer-readable storage medium. Background Technology

[0002] Current knowledge bases largely rely on manual correction or maintenance, lacking the ability to perceive and adaptively learn from user feedback during actual use. When using question-answering systems or knowledge retrieval platforms, users often experience poor performance due to outdated knowledge, inaccurate answers, and semantic misunderstandings. Existing systems cannot automatically extract effective signals from user feedback and drive the self-evolution of the knowledge base. Summary of the Invention

[0003] The main technical problem addressed by this application is to provide a knowledge base correction method, electronic device, and computer-readable storage medium that enable knowledge base correction based on user feedback information.

[0004] To address the aforementioned technical problems, this application provides a knowledge base correction method in its first aspect. The method includes: obtaining user feedback information corresponding to a target answer; wherein the target answer is obtained by responding to a target question based on a target knowledge base; performing intent recognition on the user feedback information to obtain user feedback intent; wherein the user feedback intent is used to characterize the user's evaluation feedback on the target answer; and correcting the target knowledge base based on the user feedback intent.

[0005] The target knowledge base includes several target knowledge blocks; the modification of the target knowledge base includes: in response to the negative evaluation feedback represented by the user feedback intent, modifying the source information of at least one target knowledge block in the target knowledge base, or modifying the knowledge block segmentation strategy of at least one target document, wherein the several target knowledge blocks are obtained by segmenting several target documents into knowledge blocks.

[0006] The modification of the source information of at least one target knowledge block in the target knowledge base is performed when the evaluation feedback belongs to a specific negative evaluation type; the modification of the knowledge block segmentation strategy of at least one target document is performed when the evaluation feedback belongs to a non-specific negative evaluation type; wherein, the specific negative evaluation type indicates that the user points out the type of error in the target answer, and the non-specific negative evaluation type indicates that the user only points out that the target answer is wrong.

[0007] The knowledge base correction method further includes, before correcting the source information of at least one target knowledge block in the target knowledge base, the correction method further includes: generating correction guidance based on the user feedback information; the correction of the source information of at least one target knowledge block in the target knowledge base includes: based on the correction guidance, identifying at least one knowledge block to be corrected from a plurality of target knowledge blocks in the target knowledge base; and correcting the source information of the knowledge block to be corrected.

[0008] The negative evaluation feedback includes the need for expansion; the step of finding at least one knowledge block to be corrected from a plurality of target knowledge blocks in the target knowledge base based on the correction guidance includes: finding at least one target knowledge block related to the correction guidance from the target knowledge base as a first knowledge block to be corrected; for each first knowledge block to be corrected, finding at least one recall knowledge block that meets a first similarity requirement with the first knowledge block to be corrected from a plurality of recall knowledge blocks in the target knowledge base as a second knowledge block to be corrected corresponding to the first knowledge block to be corrected; wherein, the recall knowledge block is a target knowledge block in the target knowledge base used to generate the target answer; the step of correcting the source information of the knowledge block to be corrected includes: adding the corresponding knowledge block identifier of each second knowledge block to be corrected to the source information of the first knowledge block to be corrected; and adding the knowledge block identifier of the first knowledge block to be corrected to the source information of each second knowledge block to be corrected corresponding to the first knowledge block to be corrected.

[0009] Wherein, the recalled knowledge block that satisfies the first similarity requirement with the first knowledge block to be corrected is: the recalled knowledge block corresponding to the maximum cosine similarity; and / or, the source information includes hook information, and the knowledge block identifier is added to the hook information of the knowledge block to be corrected.

[0010] The negative evaluation feedback includes outdated information; the step of identifying at least one knowledge block to be corrected from several target knowledge blocks in the target knowledge base based on the correction guidance includes: identifying at least one recall knowledge block that meets the correction requirements from several recall knowledge blocks in the target knowledge base based on the correction guidance, as the third knowledge block to be corrected; wherein the recall knowledge block is the target knowledge block in the target knowledge base used to generate the target answer; the step of correcting the source information of the knowledge block to be corrected includes: adding an expiration identifier to the source information of the third knowledge block to be corrected.

[0011] The recalled knowledge block that meets the correction requirements is: a recalled knowledge block in which there is a second entity of the target entity type and the entity value of the second entity and the entity value of the third entity meet the second similarity requirement, wherein the third entity is an entity existing in the correction guidance, and the target entity type is the entity type of the third entity; and / or, the source information includes timeliness information, and the expiration identifier is added to the timeliness information of the third knowledge block to be corrected.

[0012] The negative evaluation feedback includes a mix of true and false feedback. The step of correcting the source information of at least one target knowledge block in the target knowledge base includes: based on the correction guidelines and the target question, performing knowledge block type identification on several recalled knowledge blocks in the target knowledge base to obtain type identification results for each recalled knowledge block; wherein the type identification results of the recalled knowledge blocks are used to characterize whether the recalled knowledge block is a distracting knowledge block or a related knowledge block; based on the knowledge content of the several recalled knowledge blocks, determining the target information type that can distinguish between the distracting knowledge block and the related knowledge block; for each recalled knowledge block, finding the target information corresponding to the target information type from the knowledge content of the recalled knowledge block, and adding the target information to the source information of the recalled knowledge block.

[0013] The target information type includes at least one of the following: the document name of the target document and the corresponding hierarchical title; and / or, the source information includes identifier information, and the target information is added to the identifier information of the recalled knowledge block; and / or, the knowledge base correction method further includes: for each recalled knowledge block, determining the first position of the recalled knowledge block in the target document; obtaining a fourth entity existing in the target knowledge block corresponding to the second position; wherein, the second position is a position in the target document to which the recalled knowledge block belongs that is adjacent to the first position; and adding the fourth entity to the source information of the recalled knowledge block.

[0014] The negative evaluation feedback includes both needs for expansion and a mix of true and false feedback. Before correcting the knowledge block segmentation strategy for at least one of the target documents, the knowledge base correction method further includes: obtaining statistical information on the negative evaluation feedback corresponding to each target document; correcting the knowledge block segmentation strategy for at least one of the target documents includes: using the negative evaluation feedback statistics, selecting at least one target document from the plurality of target documents that belongs to high-frequency negative evaluation feedback as the document to be corrected; correcting the knowledge block segmentation strategy for each document to be corrected, so as to use the corrected knowledge block segmentation strategy to segment the corresponding document to be corrected into knowledge blocks.

[0015] The negative feedback statistics include the percentage of negative feedback occurrences. Obtaining the negative feedback statistics for each target document includes: obtaining the first negative feedback occurrence for each target document; wherein the first negative feedback occurrence is the sum of the second negative feedback occurrences for each target knowledge block of the target document; and obtaining the total number of feedback occurrences for the target knowledge base; obtaining the ratio of each first negative feedback occurrence to the total number of feedback occurrences as the percentage of negative feedback occurrences for each target document; and / or, the target document belonging to the high-frequency negative feedback situation is: a target document whose negative feedback occurrence percentage is greater than or equal to a preset percentage threshold.

[0016] To address the aforementioned technical problems, a second aspect of this application provides an electronic device comprising a memory and a processor, wherein the memory stores program instructions and the processor executes the program instructions to implement the aforementioned knowledge base correction method.

[0017] To address the aforementioned technical problems, a third aspect of this application provides a computer-readable storage medium for storing program instructions that can be executed to implement the aforementioned knowledge base correction method.

[0018] In the above technical solution, user feedback intent represents the user's evaluation feedback on the target answer. Therefore, by using user feedback intent, it is possible to determine whether the target answer generated based on the target knowledge base is accurate and appropriate. Thus, by modifying the target knowledge base based on user feedback intent, the target knowledge base can be corrected in a timely manner, thereby improving the accuracy of the target answer generated based on the target knowledge base.

[0019] Furthermore, the user feedback intent is determined based on the user feedback information corresponding to the target answer. Therefore, the knowledge base correction method provided in this application can sense the user's feedback information on the target answer in real time and automatically correct the target knowledge base based on the user's feedback information on the target answer, thereby realizing knowledge base correction based on user feedback information. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating an embodiment of the knowledge base correction method provided in this application;

[0021] Figure 2 This is a flowchart illustrating an embodiment of modifying the source information of at least one target knowledge block in a target knowledge base, as provided in this application.

[0022] Figure 3 This is a schematic diagram of the framework of an embodiment of the knowledge base correction device provided in this application;

[0023] Figure 4 This is a schematic diagram of the framework of an embodiment of the electronic device provided in this application;

[0024] Figure 5 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0025] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0026] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0027] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper means two or more.

[0028] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the knowledge base correction method provided in this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this embodiment includes:

[0029] Step S11: Obtain user feedback information corresponding to the target answer.

[0030] In this embodiment, user feedback information corresponding to the target answer is obtained; wherein, the target answer is obtained by responding to the target question based on the target knowledge base.

[0031] In one implementation, the user feedback information corresponding to the target answer can be user input, and the user input feedback information can be regarded as implicit feedback of user feedback information.

[0032] Of course, in other implementations, the user feedback information corresponding to the target answer can also be selected by the user from several options on the display interface. The user selecting an option from the display interface as their feedback information on the target answer can be considered explicit feedback. For example, if the display interface shows "like" and "dislike" options, the user can choose either option as their feedback information on the target answer. Another example is a feedback information dropdown menu displayed on the display interface, where the user can click to select an option as their feedback information on the target answer.

[0033] Step S12: Perform intent recognition on the user feedback information to obtain the user feedback intent.

[0034] In this embodiment, user feedback information is subjected to intent recognition to obtain user feedback intent; wherein, user feedback intent is used to characterize the user's evaluation feedback on the target answer.

[0035] In one implementation, the evaluation feedback representing the user's feedback intent can be either negative or positive.

[0036] In one specific implementation, positive evaluation feedback can be "completely correct." For example, the interface displays "like" and "dislike" options. The user selects the "like" option as their feedback to the target answer. Therefore, intent recognition is performed on the user feedback to obtain the user's feedback intent, and the evaluation feedback represented by this user's feedback intent is the positive evaluation feedback "completely correct." As another example, the user feedback is input by the user, such as "This target answer is perfect," "The answer is very clear," or "This solved my problem." Since "This target answer is perfect," "The answer is very clear," and "This solved my problem" all explicitly express the user's belief that the current knowledge item is accurate, intent recognition is performed on the user feedback to obtain the user's feedback intent, and the evaluation feedback represented by this user's feedback intent is the positive evaluation feedback "completely correct."

[0037] In one specific implementation, negative feedback can include a specific negative feedback type, which indicates the type of error the user points out in the target answer. A specific negative feedback type can be mixed true / false, outdated information, needs expansion, or completely wrong. Mixed true / false can be understood as the target answer being partially correct, partially incorrect, or ambiguous; that is, the target answer contains both correct and incorrect / ambiguous parts. Outdated information can be understood as the target answer being no longer applicable to the current time or environment. Needs expansion can be understood as the target answer being insufficiently comprehensive or in-depth. Completely wrong can be understood as the target answer being incorrect.

[0038] For example, user feedback is input by the user, and the user input could be something like "The first half of the target answer is correct, but the second half is problematic" or "The data in the target answer is correct, but the interpretation is wrong." Since the user pointed out that part of the target answer is correct and part is wrong in "The first half of the target answer is correct, but the second half is problematic" or "The data in the target answer is correct, but the interpretation is wrong," the user's feedback is used to identify the user's intention. The evaluation feedback represented by the user's feedback intention is a negative evaluation feedback that is "half true and half false."

[0039] For example, user feedback information is input by the user, such as "The target answer contains data from 2020, which has changed now" or "The product in the target answer is discontinued." Since the user indicates that the target answer is no longer applicable to the current time or environment in statements like "The target answer contains data from 2020, which has changed now" or "The product in the target answer is discontinued," the user's feedback information is analyzed for intent, resulting in the user's feedback intent. The evaluation feedback represented by the user's feedback intent is the negative evaluation feedback "information is outdated."

[0040] For example, user feedback information is input by the user, such as "Can you provide more examples?", "Please explain in more detail", or "What are the other steps?". Since the user pointed out in "Can you provide more examples?", "Please explain in more detail", or "What are the other steps?", the user indicated that the target answer was not comprehensive or deep enough. Therefore, the user feedback information is subjected to intent recognition to obtain the user feedback intent. The evaluation feedback represented by the user feedback intent is negative evaluation feedback "needs to be expanded".

[0041] For example, user feedback information is input by the user, and the user input could be "This target answer is wrong", "The data in this target answer is incorrect", "This target answer is misleading information", etc. Since the user points out that the target answer is fundamentally wrong in "This target answer is wrong", "The data in this target answer is incorrect", "This target answer is misleading information", etc., the user feedback information is used to identify the user's intention, and the evaluation feedback represented by the user's feedback intention is negative evaluation feedback "completely wrong".

[0042] In other specific implementations, negative feedback can include non-specific negative feedback types, which indicate that the user only points out that the target answer is wrong. For example, the interface displays "like" and "dislike" options. The user selects the "dislike" option as their feedback to the target answer. Since the user selected "dislike," they must believe that the target answer is wrong. Therefore, the user feedback information is subjected to intent recognition to obtain the user feedback intent. The evaluation feedback represented by the user feedback intent is negative feedback.

[0043] In one implementation, a natural language processing model can be used to identify the intent of user feedback information to obtain the user's feedback intent.

[0044] Step S13: Based on user feedback intent, revise the target knowledge base.

[0045] In this embodiment, the target knowledge base is modified based on user feedback intent. User feedback intent represents the user's evaluation of the target answer. Therefore, by understanding user feedback intent, it can be determined whether the target answer generated based on the target knowledge base is accurate and appropriate. Thus, modifying the target knowledge base based on user feedback intent allows for timely correction of the target knowledge base, improving the accuracy of the target answers generated based on it. Furthermore, since user feedback intent is determined based on user feedback information corresponding to the target answer, the knowledge base modification method provided in this application can sense user feedback information on the target answer in real time and automatically modify the target knowledge base based on this feedback information, achieving knowledge base modification based on user feedback information.

[0046] In one embodiment, the target knowledge base includes several target knowledge blocks. The modification of the target knowledge base specifically involves: responding to negative evaluation feedback representing user feedback intent, modifying the source information of at least one target knowledge block in the target knowledge base, or modifying the knowledge block segmentation strategy of at least one target document. The several target knowledge blocks are obtained by segmenting several target documents into knowledge blocks. Negative evaluation feedback is a clear and high-value signal that directly indicates the target answer generated based on the target knowledge base is inaccurate or inappropriate. Therefore, it is necessary to modify the source information of at least one target knowledge block in the target knowledge base, or modify the knowledge block segmentation strategy of at least one target document, to improve the accuracy of the target answer generated based on the target knowledge base.

[0047] In one specific implementation, the modification of the source information of at least one target knowledge block in the target knowledge base is performed when the evaluation feedback belongs to a specific negative evaluation type; the modification of the knowledge block segmentation strategy of at least one target document is performed when the evaluation feedback belongs to a non-specific negative evaluation type. Here, a specific negative evaluation type indicates that the user points out the type of error in the target answer, while a non-specific negative evaluation type indicates that the user only points out that the target answer is wrong. A specific negative evaluation type indicates that the user points out the type of error in the target answer, therefore, the user feedback on the target answer contains biased opinions or guidance. In this case, modifying the source information of at least one knowledge block in the target knowledge base means maintaining or adjusting the source information of at least one knowledge block in the target knowledge base, achieving maintenance of the knowledge block source information of the knowledge base based on user feedback. A non-specific negative evaluation type indicates that the user only points out that the target answer is wrong, therefore, the user feedback on the target answer does not contain biased opinions or guidance. In this case, modifying the knowledge block segmentation strategy of at least one target document means that the target knowledge base performs self-correction, achieving self-correction of the knowledge base based on user feedback.

[0048] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of modifying source information of at least one target knowledge block in a target knowledge base, as provided in this application. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily replace it. Figure 2 The illustrated process sequence is limited. For example... Figure 2 As shown, before correcting the source information of at least one target knowledge block in the target knowledge base, correction guidance is generated based on user feedback. This embodiment includes:

[0049] Step S21: Based on the correction guidelines, identify at least one knowledge block to be corrected from several target knowledge blocks in the target knowledge base.

[0050] In this embodiment, based on the correction guidelines, at least one knowledge block to be corrected is identified from several target knowledge blocks in the target knowledge base.

[0051] In one implementation, a large model is used to generate corrective guidance based on user feedback.

[0052] In one specific implementation, the following prompt is provided to the large model: "You are a professional user feedback analyst, skilled at accurately identifying core issues and potential optimization opportunities from user feedback, and extracting valuable improvement suggestions.\n##Instructions:\n 1. Understand the user's real needs In-depth analysis of the actual intentions and core needs behind user expressions, distinguishing between surface feedback and deep-seated requirements. 2. Extract the points of dissatisfaction from the feedback. Based on user feedback on the answers, accurately identify the specific aspects of their dissatisfaction. 3. Generate optimization guidance If user feedback shows a clear trend, clear and actionable improvement guidelines should be provided, focusing on improving the quality of responses and user satisfaction.

[0053] The format of the correction guidelines generated by the large model can be JSON. In addition, the generated correction guidelines can include the type of operation to be performed and the specific goal or content of the operation. The type of operation to be performed can be one of merge, separate, or calibrate; for example, the generated correction guidelines are "{"operation":"separate","content":"operate content"}.

[0054] Step S22: Correct the source information of the knowledge block to be corrected.

[0055] In this embodiment, the source information of the knowledge block to be corrected is corrected.

[0056] In one embodiment, negative evaluation feedback includes the need for expansion; based on the correction guidelines, identifying at least one knowledge block to be corrected from a plurality of target knowledge blocks in the target knowledge base, specifically: identifying at least one target knowledge block related to the correction guidelines from the target knowledge base as a first knowledge block to be corrected; for each first knowledge block to be corrected, identifying at least one recall knowledge block from a plurality of recall knowledge blocks in the target knowledge base that meets a first similarity requirement with the first knowledge block to be corrected as a second knowledge block to be corrected corresponding to the first knowledge block to be corrected; wherein, the recall knowledge block is a target knowledge block in the target knowledge base used to generate the target answer; correcting the source information of the knowledge block to be corrected includes: adding the knowledge block identifier of each corresponding second knowledge block to be corrected to the source information of the first knowledge block to be corrected; and adding the knowledge block identifier of the first knowledge block to be corrected to the source information of each second knowledge block to be corrected corresponding to the first knowledge block to be corrected.

[0057] Specifically, relevant target knowledge blocks are retrieved from the target database using the correction guidelines, serving as the first knowledge blocks to be corrected; for each first knowledge block to be corrected, recall knowledge blocks that meet the first similarity requirement with the first knowledge block to be corrected are identified, serving as the second knowledge blocks to be corrected; knowledge block identifiers of the corresponding second knowledge blocks to be corrected are added to the source information of the first knowledge blocks to be corrected, and knowledge block identifiers of the first knowledge blocks to be corrected are added to the source information of each second knowledge block to be corrected corresponding to the first knowledge block to be corrected, that is, the first knowledge blocks to be corrected and the corresponding second knowledge blocks to be corrected are clustered and bound together.

[0058] When user feedback on the target answer indicates a need for expansion, it suggests that the target answer generated based on the retrieved knowledge blocks (related knowledge blocks retrieved from the target knowledge base based on the target question) is not comprehensive or lacks depth. This implies omissions in the retrieved knowledge blocks. When user feedback on the target answer requires expansion, the generated correction guidance must include areas where the target answer is currently incomplete or lacks depth. Therefore, retrieving relevant target knowledge blocks (the first knowledge block to be corrected) from the target database using the correction guidance must identify the omitted knowledge blocks that caused the target answer's incompleteness or lack of depth. Thus, clustering the first knowledge block to be corrected with its corresponding strongly related second knowledge block to be corrected—that is, adding each other's knowledge block identifiers to their respective source information—allows that subsequent retrieval of any knowledge block based on the user question can identify its strongly related knowledge blocks through its source information and retrieve them as retrieved knowledge blocks. This results in a more comprehensive or in-depth answer generated based on the retrieved knowledge blocks from the target knowledge base.

[0059] In one specific implementation, the recalled knowledge block that satisfies the first similarity requirement with the first knowledge block to be corrected is the recalled knowledge block corresponding to the maximum similarity.

[0060] In one specific implementation, the recalled knowledge block that satisfies the first similarity requirement with the first knowledge block to be corrected is the recalled knowledge block corresponding to the maximum cosine similarity.

[0061] In one specific implementation, from a plurality of recalled knowledge blocks in the target knowledge base, at least one recalled knowledge block that satisfies a first similarity requirement with the first knowledge block to be corrected is selected as the second knowledge block to be corrected corresponding to the first knowledge block to be corrected. Specifically, the plurality of recalled knowledge blocks in the target knowledge base and the first knowledge block to be corrected are vectorized using embedding; then, the similarity between each recalled knowledge block after embedding vectorization and the first knowledge block to be corrected after embedding vectorization is calculated, and at least one recalled knowledge block that satisfies the first similarity requirement with the first knowledge block to be corrected is selected as the second knowledge block to be corrected corresponding to the first knowledge block to be corrected.

[0062] In one specific implementation, the source information includes hook information, and the knowledge block identifier is added to the hook information of the knowledge block to be corrected. The hook information of knowledge block A stores the knowledge block identifier of knowledge block B, which is logically strongly related to knowledge block A. The knowledge block identifier of a knowledge block can be the ID of the knowledge block.

[0063] In one specific implementation, the generated correction guidance is "{"operation":"merge","content":"merge content"}". Since operation is "merge", it indicates that the user's evaluation feedback on the target answer needs to be expanded. In this case, based on the content field, at least one knowledge block to be corrected and its subsequent steps are identified from several target knowledge blocks in the target knowledge base.

[0064] In one embodiment, negative evaluation feedback includes information being outdated; based on correction guidelines, identifying at least one knowledge block to be corrected from several target knowledge blocks in the target knowledge base, specifically: based on correction guidelines, identifying at least one recall knowledge block that meets the correction requirements from several recall knowledge blocks in the target knowledge base, as the third knowledge block to be corrected; wherein, the recall knowledge block is the target knowledge block in the target knowledge base used to generate the target answer; correcting the source information of the knowledge block to be corrected, specifically: adding an expiration marker to the source information of the third knowledge block to be corrected.

[0065] When a user's feedback on the target answer indicates that the information is outdated, it means that the target answer generated based on the recalled knowledge blocks retrieved from the target knowledge base is no longer applicable to the current time or environment. In other words, the target answer generated based on the recalled knowledge blocks retrieved from the target knowledge base contains outdated content, meaning the recalled knowledge blocks retrieved from the target knowledge base are no longer applicable to the current time or environment, or have expired and need updating. When a user's feedback on the target answer indicates that the information is outdated, the generated correction guidance will inevitably include the outdated content in the target answer. Therefore, the recalled knowledge block that meets the correction requirements, i.e., the third knowledge block to be corrected, must be the recalled knowledge block that caused the generated target answer to contain outdated content. Therefore, an expiration flag is added to the source information of the third knowledge block to be corrected, so that it is possible to know whether the recalled knowledge blocks retrieved based on the user's question are outdated, and thus to mark the outdated reminder in the answer generated based on the outdated recalled knowledge block.

[0066] In one specific implementation, a recalled knowledge block that meets the correction requirement is: a recalled knowledge block in which a second entity of the target entity type exists and the entity value of the second entity satisfies the second similarity requirement with the entity value of the third entity, wherein the third entity is an entity existing in the correction guidance, and the target entity type is the entity type of the third entity. Specifically, the second entity and its corresponding entity type are extracted from the recalled knowledge block, and the third entity and its corresponding entity type are extracted from each recalled knowledge block; the similarity between the entity values ​​of the second entity and the third entity of the same entity type is calculated, and recalled knowledge blocks that satisfy the second similarity requirement are identified.

[0067] In one specific implementation, the similarity between entity values ​​is evaluated using edit distance, and the recalled knowledge block that meets the second similarity requirement is the recalled knowledge block corresponding to the maximum edit distance.

[0068] In one specific implementation, NER technology is used to extract entities and entity types from the recalled knowledge blocks and correction guidelines.

[0069] In one specific implementation, the source information includes expiration information, and an expiration flag is added to the expiration information of the third knowledge block to be corrected. The expiration information of the knowledge block is used to mark whether the knowledge block has expired and needs to be updated.

[0070] In one specific implementation, the generated correction guidance is "{"operation":"calibrate","content":"calibrate content"}". Since operation is "calibrate", it indicates that the user's evaluation feedback on the target answer is outdated. In this case, based on the content field, at least one recall knowledge block that meets the correction requirements is selected from several recall knowledge blocks in the target knowledge base, and this block is designated as the third knowledge block to be corrected and subsequent steps.

[0071] In one embodiment, negative evaluation feedback includes a mix of true and false feedback; the source information of at least one target knowledge block in the target knowledge base is corrected, specifically: based on the correction guidelines and the target question, several recalled knowledge blocks in the target knowledge base are identified by knowledge block type to obtain the type identification result of each recalled knowledge block; wherein, the type identification result of the recalled knowledge block is used to characterize whether the recalled knowledge block is a distracting knowledge block or a related knowledge block; based on the knowledge content of several recalled knowledge blocks, the target information type that can distinguish between distracting knowledge blocks and related knowledge blocks is determined; for each recalled knowledge block, the target information corresponding to the target information type is found from the knowledge content of the recalled knowledge block, and the target information is added to the source information of the recalled knowledge block.

[0072] Specifically, based on the revised guidelines and target questions, knowledge block type identification is performed on several recalled knowledge blocks in the target knowledge base to determine whether each recalled knowledge block is a distracting knowledge block that causes the target answer to be half true and half false or a related knowledge block; target information that can distinguish whether it is a distracting knowledge block or a related knowledge block is added to the source information of each recalled knowledge block.

[0073] When user feedback on the target answer is mixed (half true, half false), it indicates that the target answer generated based on the recalled knowledge blocks retrieved from the target knowledge base is partially correct and partially incorrect. This means that some of the recalled knowledge blocks are relevant to the target question, while others interfere with generating the correct answer. When user feedback on the target answer is mixed, the generated correction guidance must include both correct and incorrect content from the target answer. Therefore, using the correction guidance and the target question, we can identify interfering and relevant knowledge blocks within several recalled knowledge blocks. Interfering and relevant knowledge blocks exhibit local semantic similarity; for example, "product release process" and "project release process" are both release process knowledge in their local semantics. When the recalled knowledge blocks lack global information about the product and project, it can lead to confusing answers. Therefore, by adding target information to the source information of the recalled knowledge blocks, which can distinguish whether they are interfering or related knowledge blocks, we can know whether the recalled knowledge blocks retrieved based on the user question are related to the user question. This allows us to generate the answer to the user question based on the recalled knowledge blocks that are related to the user question, thereby improving the accuracy of the generated answer.

[0074] In one specific implementation, the cross_encoder model is used to identify the knowledge block types of several recalled knowledge blocks in the target knowledge base based on the revised guidance and the target question, so as to obtain the type identification results of each recalled knowledge block.

[0075] In one specific implementation, the target information type includes at least one of the following: the document name of the target document and the corresponding hierarchical title.

[0076] In one specific implementation, the source information includes identifier information, and the target information is added to the identifier information of the recalled knowledge block. The identifier information of the knowledge block stores unique identification information, such as: the document name where the knowledge block is located, title information, specific objects, etc.

[0077] In one specific implementation, the generated correction guidance is "{"operation":"separate","content":"separate content"}". Since operation is "separate", it indicates that the user's evaluation feedback on the target answer is mixed. In this case, based on the content field and the target question query, knowledge block type identification is performed on several recalled knowledge blocks in the target knowledge base to obtain the type identification results of each recalled knowledge block and its subsequent steps.

[0078] In one specific implementation, for each recalled knowledge block, a first position of the recalled knowledge block in its target document is determined; a fourth entity existing in the target knowledge block corresponding to the second position is obtained; wherein, the second position is the position in the target document to which the recalled knowledge block belongs that is adjacent to the first position; and the fourth entity is added to the source information of the recalled knowledge block. That is, for each recalled knowledge block, the fourth entity is extracted from the knowledge blocks above and below the recalled knowledge block, and the unique fourth entity corresponding to the recalled knowledge block is added to the source information of the recalled knowledge block.

[0079] In one embodiment, negative evaluation feedback includes both need for expansion and a mix of true and false feedback. Before modifying the knowledge block segmentation strategy for at least one target document, the statistical data of negative evaluation feedback for each target document is obtained. The modification of the knowledge block segmentation strategy for at least one target document specifically involves: using the statistical data of negative evaluation feedback, selecting at least one target document from several target documents that belongs to the high-frequency negative evaluation feedback situation as the document to be modified; modifying the knowledge block segmentation strategy for each document to be modified, so as to use the modified knowledge block segmentation strategy to perform knowledge block segmentation on the corresponding document to be modified.

[0080] Knowledge block segmentation strategies for target documents include rule-based segmentation, text structure-based segmentation, sliding window segmentation, and recursive segmentation. Different strategies have advantages for different document types. Inappropriate selection of a segmentation strategy can easily lead to a mix of true and false answers and / or require expansion. Target documents exhibiting high-frequency negative feedback indicate that users frequently provide negative feedback on answers generated based on the knowledge blocks in that document. This means that users often find the answers inaccurate, incomplete, or lacking in depth. In other words, the target document's knowledge block segmentation strategy is incompatible with the document and needs to be modified. The modified strategy should then be used to segment the corresponding document to ensure that subsequent answers generated based on the target document are accurate, comprehensive, and in-depth.

[0081] In one specific implementation, target documents that fall under the category of high-frequency negative feedback are defined as those whose negative feedback frequency accounts for a percentage greater than or equal to a preset percentage threshold. The preset percentage threshold is not fixed and can be set according to actual usage needs; for example, the preset percentage threshold = 1 / num. doc ,num doc This indicates the number of target documents in the target knowledge base.

[0082] In one specific implementation, the negative feedback statistics include the percentage of negative feedback occurrences. Specifically, obtaining the negative feedback statistics for each target document involves: obtaining the first negative feedback occurrence for each target document; wherein the first negative feedback occurrence is the sum of the second negative feedback occurrences corresponding to each target knowledge block of the target document; and obtaining the total number of feedback occurrences for the target knowledge base; and obtaining the ratio of each first negative feedback occurrence to the total number of feedback occurrences as the percentage of negative feedback occurrences for each target document. The specific formula is as follows:

[0083]

[0084] Where, r n This represents the percentage of negative feedback responses for the nth target document in the target knowledge base; n doc This represents the number of the first negative feedback for the nth target document in the target knowledge base; n base This represents the total number of evaluation feedbacks for the target knowledge base.

[0085] Please see Figure 3 , Figure 3 This is a schematic diagram of an embodiment of the knowledge base correction device provided in this application. The knowledge base correction device 30 includes an acquisition module 31, an intent recognition module 32, and a correction module 33. The acquisition module 31 is used to acquire user feedback information corresponding to the target answer; wherein, the target answer is obtained by replying to the target question based on the target knowledge base; the intent recognition module 32 is used to perform intent recognition on the user feedback information to obtain the user feedback intent; wherein, the user feedback intent is used to characterize the user's evaluation feedback on the target answer; the correction module 33 is used to correct the target knowledge base based on the user feedback intent.

[0086] The aforementioned target knowledge base includes several target knowledge blocks; the correction module 33 is used to correct the target knowledge base by: in response to the user feedback intent representation being negative evaluation feedback, correcting the source information of at least one target knowledge block in the target knowledge base, or correcting the knowledge block segmentation strategy of at least one target document, wherein several target knowledge blocks are obtained by segmenting several target documents into knowledge blocks.

[0087] Specifically, the aforementioned modification of the source information of at least one target knowledge block in the target knowledge base is performed when the evaluation feedback belongs to a specific negative evaluation type; the modification of the knowledge block segmentation strategy of at least one target document is performed when the evaluation feedback belongs to a non-specific negative evaluation type; wherein, a specific negative evaluation type means that the user points out the type of error in the target answer, and a non-specific negative evaluation type means that the user only points out that the target answer is wrong.

[0088] The correction module 33 is used to correct the source information of at least one target knowledge block in the target knowledge base before correcting it, including: generating correction guidance based on user feedback information; the correction module 33 is used to correct the source information of at least one target knowledge block in the target knowledge base, including: identifying at least one knowledge block to be corrected from several target knowledge blocks in the target knowledge base based on the correction guidance; and correcting the source information of the knowledge block to be corrected.

[0089] The aforementioned negative evaluation feedback includes the need for expansion; the correction module 33 is used to find at least one knowledge block to be corrected from several target knowledge blocks in the target knowledge base based on the correction guidance, including: finding at least one target knowledge block related to the correction guidance from the target knowledge base as the first knowledge block to be corrected; for each first knowledge block to be corrected, finding at least one recall knowledge block that meets the first similarity requirement from several recall knowledge blocks in the target knowledge base as the second knowledge block to be corrected corresponding to the first knowledge block to be corrected; wherein, the recall knowledge block is the target knowledge block in the target knowledge base used to generate the target answer; the correction module 33 is used to correct the source information of the knowledge block to be corrected, including: adding the knowledge block identifier of each second knowledge block to be corrected to the source information of the first knowledge block to be corrected; and adding the knowledge block identifier of the first knowledge block to be corrected to the source information of each second knowledge block to be corrected corresponding to the first knowledge block to be corrected.

[0090] Wherein, the recalled knowledge block that meets the first similarity requirement with the first knowledge block to be corrected is: the recalled knowledge block corresponding to the maximum cosine similarity; and / or, the above source information includes hook information, and the knowledge block identifier is added to the hook information of the knowledge block to be corrected.

[0091] The aforementioned negative evaluation feedback includes outdated information; the correction module 33 is used to identify at least one knowledge block to be corrected from several target knowledge blocks in the target knowledge base based on the correction guidelines, including: identifying at least one recall knowledge block that meets the correction requirements from several recall knowledge blocks in the target knowledge base based on the correction guidelines, as the third knowledge block to be corrected; wherein, the recall knowledge block is the target knowledge block in the target knowledge base used to generate the target answer; the correction module 33 is used to correct the source information of the knowledge block to be corrected, including: adding an expiration mark to the source information of the third knowledge block to be corrected.

[0092] Among them, the recall knowledge block that meets the requirements for correction is: a recall knowledge block in which there is a second entity of the target entity type and the entity value of the second entity and the entity value of the third entity meet the second similarity requirement, wherein the third entity is an entity existing in the correction guidance and the target entity type is the entity type of the third entity; and / or, the above source information includes timeliness information, and the expiration mark is added to the timeliness information of the third knowledge block to be corrected.

[0093] The aforementioned negative evaluation feedback includes a mix of true and false feedback. The correction module 33 is used to correct the source information of at least one target knowledge block in the target knowledge base, including: based on the correction guidelines and target questions, performing knowledge block type identification on several recalled knowledge blocks in the target knowledge base to obtain the type identification results of each recalled knowledge block; wherein, the type identification results of the recalled knowledge blocks are used to characterize whether the recalled knowledge block is a distracting knowledge block or a related knowledge block; based on the knowledge content of several recalled knowledge blocks, determining the target information type that can distinguish between distracting knowledge blocks and related knowledge blocks; for each recalled knowledge block, finding the target information corresponding to the target information type from the knowledge content of the recalled knowledge block, and adding the target information to the source information of the recalled knowledge block.

[0094] The target information types mentioned above include at least one of the following: the document name of the target document and the corresponding hierarchical title; and / or, the source information includes identifier information, and the target information is added to the identifier information of the recalled knowledge block; and / or, the correction module 33 is further used to determine the first position of each recalled knowledge block in the target document; obtain the fourth entity existing in the target knowledge block corresponding to the second position; wherein, the second position is the position in the target document to which the recalled knowledge block belongs that is adjacent to the first position; and add the fourth entity to the source information of the recalled knowledge block.

[0095] The aforementioned negative feedback includes feedback requiring expansion and feedback that is a mix of true and false. Before modifying the knowledge block segmentation strategy for at least one target document, the correction module 33 includes: obtaining statistical information on the negative feedback for each target document; the correction module 33 is used to modify the knowledge block segmentation strategy for at least one target document, including: using the negative feedback statistics to select at least one target document from several target documents that exhibits high-frequency negative feedback, as the document to be modified; and modifying the knowledge block segmentation strategy for each document to be modified, so as to use the modified knowledge block segmentation strategy to segment the corresponding document to be modified into knowledge blocks.

[0096] The aforementioned negative feedback statistics include the percentage of negative feedback occurrences. The correction module 33 is used to obtain the negative feedback statistics for each target document, including: obtaining the first negative feedback occurrence for each target document; wherein the first negative feedback occurrence is the sum of the second negative feedback occurrences for each target knowledge block of the target document; and obtaining the total number of feedback occurrences for the target knowledge base; obtaining the ratio of each first negative feedback occurrence to the total number of feedback occurrences as the percentage of negative feedback occurrences for each target document; and / or, the target documents that fall under the category of high-frequency negative feedback are those whose percentage of negative feedback occurrences is greater than or equal to a preset percentage threshold.

[0097] Please see Figure 4 , Figure 4 This is a schematic diagram of a framework of an embodiment of the electronic device provided in this application. The electronic device 40 includes a memory 41 and a processor 42 coupled to each other. The memory 41 stores program instructions, and the processor 42 is used to execute the program instructions to implement the steps in any of the above-described knowledge base correction method embodiments. Specifically, the electronic device 40 may include, but is not limited to, desktop computers, laptops, servers, mobile phones, tablets, etc., and is not limited thereto.

[0098] Specifically, processor 42 controls itself and memory 41 to implement the steps in any of the above-described knowledge base correction method embodiments. Processor 42 can also be referred to as a CPU (Central Processing Unit). Processor 42 may be an integrated circuit chip with signal processing capabilities. Processor 42 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 42 can be implemented using integrated circuit chips.

[0099] Please see Figure 5 , Figure 5 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 50 stores program instructions 51 that can be executed by a processor. The program instructions 51 are used to implement the steps in any of the above-described embodiments of the knowledge base correction method.

[0100] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0101] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0103] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0104] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0107] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A knowledge base revision method, characterized in that, The method includes: Obtain user feedback information corresponding to the target answer; wherein, the target answer is obtained by responding to the target question based on the target knowledge base; The user feedback information is subjected to intent recognition to obtain the user feedback intent; wherein, the user feedback intent is used to characterize the user's evaluation feedback on the target answer; Based on the user feedback intent, the target knowledge base is revised; The target knowledge base includes several target knowledge blocks; the modification of the target knowledge base includes: The evaluation feedback that responds to the user's feedback intent is negative evaluation feedback. Based on the user feedback information, correction guidance is generated, and the negative evaluation feedback includes areas that need to be expanded. From the target knowledge base, identify at least one target knowledge block related to the revised guidance, and designate it as the first knowledge block to be revised; For each of the first knowledge blocks to be corrected, at least one recall knowledge block that meets the first similarity requirement with the first knowledge block to be corrected is found from a plurality of recall knowledge blocks in the target knowledge base, and is used as the second knowledge block to be corrected corresponding to the first knowledge block to be corrected. The recall knowledge block is the target knowledge block in the target knowledge base used to generate the target answer. In the source information of the first knowledge block to be corrected, add the corresponding knowledge block identifier of each of the second knowledge blocks to be corrected, and in the source information of each of the second knowledge blocks to be corrected corresponding to the first knowledge block to be corrected, add the knowledge block identifier of the first knowledge block to be corrected.

2. The method according to claim 1, characterized in that, The modification of the target knowledge base also includes: In response to negative evaluation feedback representing the user's feedback intent, the knowledge block segmentation strategy for at least one target document is modified, wherein the plurality of target knowledge blocks are obtained by segmenting the plurality of target documents into knowledge blocks.

3. The method according to claim 2, characterized in that, The modification of the source information of at least one target knowledge block in the target knowledge base is performed when the evaluation feedback belongs to a specific negative evaluation type; The modification of the knowledge block segmentation strategy for at least one of the target documents is performed when the evaluation feedback belongs to a non-specific negative evaluation type. The specific negative evaluation type refers to the type of error the user points out in the target answer, while the non-specific negative evaluation type refers to the user only pointing out that the target answer is wrong.

4. The method according to claim 1, characterized in that, The recalled knowledge block that satisfies the first similarity requirement with the first knowledge block to be corrected is: the recalled knowledge block corresponding to the maximum cosine similarity. And / or, the source information includes hook information, and the knowledge block identifier is added to the hook information of the knowledge block to be corrected.

5. The method according to claim 1, characterized in that, The negative evaluation feedback includes outdated information; the step of identifying at least one knowledge block to be corrected from several target knowledge blocks in the target knowledge base based on the correction guidance includes: Based on the aforementioned correction guidelines, at least one recall knowledge block that meets the correction requirements is identified from a plurality of recall knowledge blocks in the target knowledge base, and designated as the third knowledge block to be corrected; wherein, the recall knowledge block is the target knowledge block in the target knowledge base used to generate the target answer; The step of correcting the source information of the knowledge block to be corrected includes: An expiration flag is added to the source information of the third knowledge block to be corrected.

6. The method according to claim 5, characterized in that, The recalled knowledge block that meets the correction requirement is: a recalled knowledge block in which there is a second entity of the target entity type and the entity value of the second entity and the entity value of the third entity meet the second similarity requirement, wherein the third entity is the entity that exists in the correction guidance and the target entity type is the entity type of the third entity; And / or, the source information includes timeliness information, and the expiration identifier is added to the timeliness information of the third knowledge block to be corrected.

7. The method according to claim 2 or 3, characterized in that, The negative evaluation feedback includes a mix of true and false feedback; the correction of the source information of at least one target knowledge block in the target knowledge base includes: Based on the revised guidelines and the target problem, knowledge block type identification is performed on several recalled knowledge blocks in the target knowledge base to obtain the type identification result of each recalled knowledge block; wherein, the type identification result of the recalled knowledge block is used to characterize whether the recalled knowledge block is a distracting knowledge block or a related knowledge block; Based on the knowledge content of the aforementioned recall knowledge blocks, a target information type that can distinguish between the interfering knowledge block and the associated knowledge block is determined; for each recall knowledge block, target information corresponding to the target information type is found from the knowledge content of the recall knowledge block, and the target information is added to the source information of the recall knowledge block.

8. The method according to claim 7, characterized in that, The target information type includes at least one of the following: the document name of the target document and the corresponding level heading; And / or, the source information includes identifier information, and the target information is added to the identifier information of the recall knowledge block; And / or, the method further includes: For each of the recalled knowledge blocks, determine the first position of the recalled knowledge block in its respective target document; Obtain the fourth entity existing in the target knowledge block corresponding to the second position; wherein, the second position is the position in the target document to which the recalled knowledge block belongs that is adjacent to the first position; Add the fourth entity to the source information of the recalled knowledge block.

9. The method according to claim 2 or 3, characterized in that, The negative feedback includes feedback requiring expansion and a mix of true and false feedback; before revising the knowledge block segmentation strategy for at least one of the target documents, the method further includes: Obtain the negative evaluation feedback statistics for each of the target documents; The modification of the knowledge block segmentation strategy for at least one of the target documents includes: Using the negative evaluation feedback statistics, at least one target document belonging to the high-frequency negative evaluation feedback situation is selected from the plurality of target documents as the document to be corrected; The knowledge block segmentation strategy for each of the documents to be corrected is modified so that the corrected knowledge block segmentation strategy can be used to segment the corresponding documents to be corrected into knowledge blocks.

10. The method according to claim 9, characterized in that, The statistics on negative feedback include the percentage of negative feedback received, where... The step of obtaining the negative evaluation feedback statistics for each of the target documents includes: Obtain the number of first negative evaluation feedbacks corresponding to each of the target documents; wherein, the number of first negative evaluation feedbacks is the sum of the number of second negative evaluation feedbacks corresponding to each target knowledge block of the target document; And, obtain the total number of evaluation feedbacks for the target knowledge base; The ratio of each first negative feedback instance to the total number of feedback instances is obtained as the percentage of negative feedback instances for each target document; and / or, The target documents that fall under the category of high-frequency negative feedback are those whose negative feedback frequency accounts for a percentage greater than or equal to a preset percentage threshold.

11. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory being used to store program instructions, and the processor being used to execute the program instructions to implement the knowledge base correction method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions that can be executed to implement the knowledge base modification method as described in any one of claims 1-10.

Images