Method and device for managing QA knowledge base

Abstract

The invention relates to the field of artificial intelligence, in particular to a QA knowledge base management method and device. The method comprises the following steps: acquiring a preset knowledge base for RAG retrieval enhancement generation; the preset knowledge base comprises n pieces of QA knowledge, each piece of QA knowledge comprises questions and answers, and n is a positive integer greater than 1; calculating a first similarity set and a second similarity set; wherein the first similarity set comprises the similarity of problems of any two pieces of QA knowledge in the n pieces of QA knowledge; the second similarity set comprises the similarity of answers of any two pieces of QA knowledge in the n pieces of QA knowledge; and updating the preset knowledge base according to the first similarity set and the second similarity set. The method is applied to management of the QA knowledge base.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to a method and apparatus for managing a QA-type knowledge base. Background Art

[0002] Currently, queryanswer (QA) knowledge bases are widely used in retrieval augmented generation (RAG) due to their advantages such as concise structure and ease of finding answers.

[0003] RAG is an artificial intelligence (AI) framework that retrieves knowledge from a knowledge base and generates results using a large language model (LLM).

[0004] A QA-type knowledge base is a knowledge base that stores questions and answers. It's like a box full of knowledge cards, with each card having a specific question on one side and a detailed answer to that question on the other.

[0005] However, in RAG, the accuracy of the generated results is often affected by the low quality of the QA-type knowledge base used (for example, the QA-type knowledge base may contain a lot of erroneous or duplicate knowledge).

[0006] Therefore, how to manage the QA-type knowledge base applied to RAG in order to improve the content quality of the QA-type knowledge base is a problem that needs to be solved. Summary of the Invention

[0007] To address the aforementioned technical issues, this disclosure provides a management method and apparatus for QA-type knowledge bases.

[0008] Firstly, a method for managing a QA-type knowledge base is provided. The method includes: acquiring a preset knowledge base for RAG (Research and Application Group); the preset knowledge base includes n QA knowledge items, each QA knowledge item including a question and an answer, where n is a positive integer greater than 1; calculating a first similarity set and a second similarity set; wherein the first similarity set includes the similarity of the questions of any two QA knowledge items among the n QA knowledge items; the second similarity set includes the similarity of the answers of any two QA knowledge items among the n QA knowledge items; and updating the preset knowledge base based on the first similarity set and the second similarity set.

[0009] In some implementations, updating the preset knowledge base based on the first similarity set and the second similarity set includes: determining a first QA knowledge set from the n QA knowledge items based on the first similarity set and the second similarity set; wherein the similarity between the questions of any two QA knowledge items in the first QA knowledge set is higher than or equal to a first threshold, and the similarity between the answers of any two QA knowledge items in the first QA knowledge set is higher than or equal to a second threshold; updating the preset knowledge base; wherein some QA knowledge items in the first QA knowledge set are deleted from the preset knowledge base.

[0010] In some implementations, updating the preset knowledge base based on the first similarity set and the second similarity set includes: determining a second QA knowledge set from the n QA knowledge items based on the first similarity set and the second similarity set; wherein the similarity between the questions of any two QA knowledge items in the second QA knowledge set is higher than or equal to a third threshold, and the similarity between the answers of any two QA knowledge items in the second QA knowledge set is lower than or equal to a fourth threshold; generating a conflict message and receiving a user operation corresponding to the conflict message; the conflict message is used to display the QA knowledge items included in the second QA knowledge set to the user; the user operation is used to select a QA knowledge item from the second QA knowledge set; updating the preset knowledge base; wherein the preset knowledge base deletes other QA knowledge items from the second QA knowledge set except for the QA knowledge item selected by the user operation.

[0011] In some implementations, updating the preset knowledge base based on the first similarity set and the second similarity set includes: determining a third QA knowledge set from the n QA knowledge sets based on the first similarity set and the second similarity set; wherein the similarity between the questions of any two QA knowledge sets in the third QA knowledge set is lower than or equal to a fifth threshold, and the similarity between the answers of any two QA knowledge sets in the third QA knowledge set is higher than or equal to a sixth threshold; updating the preset knowledge base; wherein the preset knowledge base replaces the third QA knowledge set with QA knowledge sets that include multiple questions corresponding to the same answer; the multiple questions include questions from each QA knowledge set in the third QA knowledge set; the same answer is obtained by merging the answers from each QA knowledge set in the third QA knowledge set.

[0012] In some implementations, calculating the first similarity set and the second similarity set includes: performing embedding processing on the questions and answers of each of the n QA knowledge sets to obtain the feature vectors of the questions and the feature vectors of the answers of each of the n QA knowledge sets; determining the first similarity set based on the feature vectors of the questions of each of the n QA knowledge sets; and determining the second similarity set based on the feature vectors of the answers of each of the n QA knowledge sets.

[0013] In some implementations, the method further includes: obtaining the question to be answered; calculating a third similarity set. The third similarity set includes the similarity between questions in each QA knowledge in the preset knowledge base and the question to be answered; wherein, if a fifth QA knowledge in the preset knowledge base includes multiple candidate questions corresponding to the same answer, then the third similarity set includes the highest similarity value between the multiple candidate questions in the fifth QA knowledge and the question to be answered. Based on the third similarity set, a fourth QA knowledge set is determined from the preset knowledge base. The fourth QA knowledge set includes QA knowledge corresponding to similarities higher than or equal to a similarity threshold in the third similarity set; wherein, if the highest similarity value is higher than or equal to the similarity threshold, then the fourth QA knowledge set includes a sixth QA knowledge consisting of a recombined question and a recombined answer, wherein the recombined question is the question with the highest similarity to the question to be answered among the multiple candidate questions, and the recombined answer is the answer in the fifth QA knowledge. Based on the fourth QA knowledge set, a result for the question to be answered is generated.

[0014] In some implementations, generating the result for the unanswered question based on the fourth QA knowledge set includes: sorting the fourth QA knowledge set using reranking techniques to obtain a QA knowledge ranking; the QA knowledge ranking is an order indicating the relevance of the QA knowledge in the fourth QA knowledge set to the unanswered question. The result for the unanswered question is then generated using an LLM (Large Language Model) based on the QA knowledge ranking.

[0015] In some implementations, the method further includes: acquiring first QA knowledge; the first QA knowledge includes a first question and a first answer; determining a question similarity set and an answer similarity set based on the first QA knowledge; wherein the question similarity set includes: the similarity between the first question and questions in each QA knowledge in the preset knowledge base; the answer similarity set includes: the similarity between the first answer and answers in each QA knowledge in the preset knowledge base; and importing the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set.

[0016] In some implementations, importing the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set includes: if it is determined that there is a second QA knowledge in the preset knowledge base that satisfies a first condition based on the question similarity set and the answer similarity set, then select one of the first QA knowledge and the second QA knowledge and record it in the preset knowledge base; the first condition includes: the similarity between the question of the QA knowledge and the first question is higher than or equal to a first threshold, and the similarity between the answer of the QA knowledge and the first answer is higher than or equal to a second threshold.

[0017] In some implementations, importing the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set includes: if, based on the question similarity set and the answer similarity set, it is determined that there exists a third QA knowledge in the preset knowledge base that satisfies a second condition, then an import conflict message is generated and a user instruction operation corresponding to the import conflict message is received; wherein, the second condition includes: the similarity between the question of the QA knowledge and the first question is higher than or equal to a third threshold, and the similarity between the answer of the QA knowledge and the first answer is lower than or equal to a fourth threshold; the import conflict message is used to display the third QA knowledge and the first QA knowledge to the user; the user instruction operation is used to select one QA knowledge from the third QA knowledge and the first QA knowledge; and according to the user instruction operation, one of the first QA knowledge and the third QA knowledge is recorded in the preset knowledge base.

[0018] In some implementations, importing the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set includes: if, based on the question similarity set and the answer similarity set, it is determined that there exists a fourth QA knowledge in the preset knowledge base that satisfies a third condition, then in the preset knowledge base, the fourth QA knowledge and the first QA knowledge are merged into QA knowledge that includes multiple questions corresponding to the same answer; wherein, the third condition includes: the similarity between the question of the QA knowledge and the first question is lower than or equal to a fifth threshold, and the similarity between the answer of the QA knowledge and the first answer is higher than or equal to a sixth threshold; the multiple questions include: the question of the fourth QA knowledge and the question of the first QA knowledge; the same answer is obtained by merging the answers of the fourth QA knowledge and the first knowledge.

[0019] Secondly, a management method for a QA knowledge base is provided. This method includes: acquiring first QA knowledge; the first QA knowledge includes a first question and a first answer; determining a question similarity set and an answer similarity set based on the first QA knowledge; wherein the question similarity set includes the similarity between the first question and questions in various QA knowledge sets in a preset knowledge base; the answer similarity set includes the similarity between the first answer and answers in various QA knowledge sets in the preset knowledge base; the preset knowledge base includes n QA knowledge sets, each QA knowledge set including a question and an answer, where n is a positive integer greater than 1; and importing the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set.

[0020] In some implementations, importing the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set includes: if it is determined that there is a second QA knowledge in the preset knowledge base that satisfies a first condition based on the question similarity set and the answer similarity set, then select one of the first QA knowledge and the second QA knowledge and record it in the preset knowledge base; the first condition includes: the similarity between the question of the QA knowledge and the first question is higher than or equal to a first threshold, and the similarity between the answer of the QA knowledge and the first answer is higher than or equal to a second threshold.

[0021] In some implementations, importing the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set includes: if, based on the question similarity set and the answer similarity set, it is determined that there exists a third QA knowledge in the preset knowledge base that satisfies a second condition, then an import conflict message is generated and a user instruction operation corresponding to the import conflict message is received; wherein, the second condition includes: the similarity between the question of the QA knowledge and the first question is higher than or equal to a third threshold, and the similarity between the answer of the QA knowledge and the first answer is lower than or equal to a fourth threshold; the import conflict message is used to display the third QA knowledge and the first QA knowledge to the user; the user instruction operation is used to select one QA knowledge from the third QA knowledge and the first QA knowledge; and according to the user instruction operation, one of the first QA knowledge and the third QA knowledge is recorded in the preset knowledge base.

[0022] In some implementations, importing the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set includes: if, based on the question similarity set and the answer similarity set, it is determined that there exists a fourth QA knowledge in the preset knowledge base that satisfies a third condition, then in the preset knowledge base, the fourth QA knowledge and the first QA knowledge are merged into QA knowledge that includes multiple questions corresponding to the same answer; wherein, the third condition includes: the similarity between the question of the QA knowledge and the first question is lower than or equal to a fifth threshold, and the similarity between the answer of the QA knowledge and the first answer is higher than or equal to a sixth threshold; the multiple questions include: the question of the fourth QA knowledge and the question of the first QA knowledge; the same answer is obtained by merging the answers of the fourth QA knowledge and the first knowledge.

[0023] Thirdly, a knowledge base management device is provided, comprising: an acquisition unit for acquiring a preset knowledge base for RAG; the preset knowledge base includes n QA knowledge items, each QA knowledge item including a question and an answer, where n is a positive integer greater than 1; a calculation unit for calculating a first similarity set and a second similarity set; wherein the first similarity set includes the similarity of the questions of any two QA knowledge items among the n QA knowledge items; the second similarity set includes the similarity of the answers of any two QA knowledge items among the n QA knowledge items; and an update unit for updating the preset knowledge base according to the first similarity set and the second similarity set.

[0024] In some implementations, the update unit, used to update the preset knowledge base according to the first similarity set and the second similarity set, includes: an update unit specifically used to determine a first QA knowledge set from the n QA knowledge items according to the first similarity set and the second similarity set; wherein the similarity of the questions of any two QA knowledge items in the first QA knowledge set is higher than or equal to a first threshold, and the similarity of the answers of any two QA knowledge items in the first QA knowledge set is higher than or equal to a second threshold; and an update unit specifically used to update the preset knowledge base; wherein, some QA knowledge items in the first QA knowledge set are deleted from the preset knowledge base.

[0025] In some implementations, the update unit, configured to update the preset knowledge base based on the first similarity set and the second similarity set, includes: an update unit specifically configured to determine a second QA knowledge set from the n QA knowledge items based on the first similarity set and the second similarity set; wherein the similarity between the questions of any two QA knowledge items in the second QA knowledge set is higher than or equal to a third threshold, and the similarity between the answers of any two QA knowledge items in the second QA knowledge set is lower than or equal to a fourth threshold; an update unit specifically configured to generate a conflict message and receive a user operation corresponding to the conflict message; the conflict message is used to display the QA knowledge items included in the second QA knowledge set to the user; the user operation is used to select a QA knowledge item from the second QA knowledge set; and an update unit specifically configured to update the preset knowledge base; wherein the preset knowledge base deletes other QA knowledge items from the second QA knowledge set except for the QA knowledge item selected by the user operation.

[0026] In some implementations, the update unit, used to update the preset knowledge base based on the first similarity set and the second similarity set, includes: an update unit specifically used to determine a third QA knowledge set from the n QA knowledge items based on the first similarity set and the second similarity set; wherein the similarity between the questions of any two QA knowledge items in the third QA knowledge set is lower than or equal to a fifth threshold, and the similarity between the answers of any two QA knowledge items in the third QA knowledge set is higher than or equal to a sixth threshold; the update unit is specifically used to update the preset knowledge base; wherein the preset knowledge base replaces the third QA knowledge set with QA knowledge items that include multiple questions corresponding to the same answer; the multiple questions include: questions of each QA knowledge item in the third QA knowledge set; the same answer is obtained by merging the answers of each QA knowledge item in the third QA knowledge set.

[0027] In some implementations, the calculation unit, used to calculate the first similarity set and the second similarity set, includes: a calculation unit specifically used to perform embedding processing on the questions and answers of each of the n QA knowledge pieces to obtain the feature vectors of the questions and the feature vectors of the answers of each of the n QA knowledge pieces; and a calculation unit specifically used to determine the first similarity set based on the feature vectors of the questions of each of the n QA knowledge pieces, and to determine the second similarity set based on the feature vectors of the answers of each of the n QA knowledge pieces.

[0028] In some implementations, the acquisition unit is further configured to acquire the question to be answered; the calculation unit is further configured to calculate a third similarity set; wherein the third similarity set includes the similarity between questions in each QA knowledge in the preset knowledge base and the question to be answered; wherein, if the fifth QA knowledge in the preset knowledge base includes multiple candidate questions corresponding to the same answer, then the third similarity set includes: the highest similarity value between the multiple candidate questions in the fifth QA knowledge and the question to be answered; the calculation unit is further configured to determine a fourth QA question from the preset knowledge base based on the third similarity set. A knowledge set; wherein, the fourth QA knowledge set includes: QA knowledge corresponding to similarities higher than or equal to a similarity threshold in the third similarity set; wherein, if the highest value is higher than or equal to the similarity threshold, the fourth QA knowledge set includes a sixth QA knowledge consisting of a recombined question and a recombined answer, wherein the recombined question is the question with the highest similarity to the question to be answered among the plurality of candidate questions, and the recombined answer is the answer in the fifth QA knowledge; the knowledge base management device further includes: a generation unit, used to generate the result of the question to be answered based on the fourth QA knowledge set.

[0029] In some implementations, a generation unit is used to generate the result of the question to be answered based on a fourth QA knowledge set, including: the generation unit is specifically used to sort the fourth QA knowledge set using reranking technology to obtain a QA knowledge ranking; the QA knowledge ranking is a ranking that indicates the relevance of the QA knowledge in the fourth QA knowledge set to the question to be answered; the generation unit is specifically used to generate the result of the question to be answered based on the QA knowledge ranking using an LLM large language model.

[0030] In some implementations, the acquisition unit is further configured to acquire first QA knowledge; the first QA knowledge includes a first question and a first answer; the calculation unit is further configured to determine a question similarity set and an answer similarity set based on the first QA knowledge; wherein the question similarity set includes: the similarity between the first question and questions in each QA knowledge in the preset knowledge base; the answer similarity set includes: the similarity between the first answer and answers in each QA knowledge in the preset knowledge base. The update unit is further configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set.

[0031] In some implementations, the updating unit is further configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, including: the updating unit is further configured to select one of the first QA knowledge and the second QA knowledge and record it in the preset knowledge base if it is determined that there is a second QA knowledge in the preset knowledge base that satisfies a first condition based on the question similarity set and the answer similarity set; the first condition includes: the similarity between the question of the QA knowledge and the first question is higher than or equal to a first threshold, and the similarity between the answer of the QA knowledge and the first answer is higher than or equal to a second threshold.

[0032] In some implementations, the updating unit is further configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, including: the updating unit is further configured to generate an import conflict message and receive a user instruction operation corresponding to the import conflict message if it is determined that there is a third QA knowledge in the preset knowledge base that satisfies a second condition based on the question similarity set and the answer similarity set; wherein, the second condition includes: the similarity between the question of the QA knowledge and the first question is higher than or equal to a third threshold, and the similarity between the answer of the QA knowledge and the first answer is lower than or equal to a fourth threshold; the import conflict message is used to display the third QA knowledge and the first QA knowledge to the user; the user instruction operation is used to select one QA knowledge from the third QA knowledge and the first QA knowledge; the updating unit is further configured to record one of the first QA knowledge and the third QA knowledge in the preset knowledge base according to the user instruction operation.

[0033] In some implementations, the updating unit is further configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, including: the updating unit is further configured to, if it is determined from the question similarity set and the answer similarity set that there exists a fourth QA knowledge in the preset knowledge base that satisfies a third condition, then merge the fourth QA knowledge and the first QA knowledge in the preset knowledge base into QA knowledge that includes multiple questions corresponding to the same answer; wherein, the third condition includes: the similarity between the question of the QA knowledge and the first question is lower than or equal to a fifth threshold, and the similarity between the answer of the QA knowledge and the first answer is higher than or equal to a sixth threshold; the multiple questions include: the question of the fourth QA knowledge and the question of the first QA knowledge; the same answer is obtained by merging the answers of the fourth QA knowledge and the first knowledge.

[0034] Fourthly, a knowledge base management device is provided, comprising: an acquisition unit for acquiring first QA knowledge; the first QA knowledge including a first question and a first answer; a calculation unit for determining a question similarity set and an answer similarity set based on the first QA knowledge; wherein the question similarity set includes: the similarity between the first question and questions in each QA knowledge in a preset knowledge base; the answer similarity set includes: the similarity between the first answer and answers in each QA knowledge in the preset knowledge base; the preset knowledge base includes n QA knowledge items, each QA knowledge item including a question and an answer, where n is a positive integer greater than 1; and an update unit for importing the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set.

[0035] In some implementations, the updating unit, configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, includes: the updating unit, configured to select one of the first QA knowledge and the second QA knowledge and record it in the preset knowledge base if it is determined, based on the question similarity set and the answer similarity set, that there exists a second QA knowledge in the preset knowledge base that satisfies a first condition; the first condition includes: the similarity between the question of the QA knowledge and the first question is higher than or equal to a first threshold, and the similarity between the answer of the QA knowledge and the first answer is higher than or equal to a second threshold.

[0036] In some implementations, the update unit, configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, includes: an update unit specifically configured to, if it is determined based on the question similarity set and the answer similarity set that a third QA knowledge satisfying a second condition exists in the preset knowledge base, generate an import conflict message and receive a user instruction operation corresponding to the import conflict message; wherein, the second condition includes: the similarity between the QA knowledge question and the first question is higher than or equal to a third threshold, and the similarity between the QA knowledge answer and the first answer is lower than or equal to a fourth threshold; the import conflict message is used to display the third QA knowledge and the first QA knowledge to the user; the user instruction operation is used to select one QA knowledge from the third QA knowledge and the first QA knowledge; and the update unit specifically configured to, according to the user instruction operation, record one of the first QA knowledge and the third QA knowledge in the preset knowledge base.

[0037] In some implementations, the updating unit, configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, includes: If, based on the question similarity set and the answer similarity set, it is determined that a fourth QA knowledge satisfying a third condition exists in the preset knowledge base, then the updating unit merges the fourth QA knowledge and the first QA knowledge in the preset knowledge base into QA knowledge comprising multiple questions corresponding to the same answer; wherein, the third condition includes: the similarity between the QA knowledge question and the first question is lower than or equal to a fifth threshold, and the similarity between the QA knowledge answer and the first answer is higher than or equal to a sixth threshold; the multiple questions include: the questions of the fourth QA knowledge and the questions of the first QA knowledge; the same answer is obtained by merging the answers of the fourth QA knowledge and the first QA knowledge.

[0038] Fifthly, a knowledge base management device is provided, comprising: a memory and a processor, wherein the memory is used to store a computer program and the processor is used to, when executing the computer program, cause the knowledge base management device to implement the method described in the first aspect or any implementation thereof, or in the second aspect or any implementation thereof.

[0039] In a sixth aspect, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a computing device, causes the computing device to implement the method as described in the first aspect or any implementation thereof, or in the second aspect or any implementation thereof.

[0040] In a seventh aspect, a computer program product is provided, which, when run on a computer, causes the computer to implement the method described in the first aspect or any implementation thereof, or in the second aspect or any implementation thereof.

[0041] The technical solution provided in this application has the following advantages compared with the prior art:

[0042] In this embodiment of the application, it is considered that: among multiple QA knowledge sets, the similarity between the questions in one QA knowledge set (hereinafter referred to as QA1) and the questions in other QA knowledge sets, as well as the similarity between the answers in QA1 and the answers in other QA knowledge sets, can be used to determine whether there is a conflict or duplication between the multiple QA knowledge sets and QA1. For example, when multiple QA knowledge sets include QA2, where the similarity between the question in QA2 and the question in QA1 is higher than or equal to a first threshold and the similarity between the answer in QA2 and the answer in QA1 is higher than or equal to a second threshold, then QA1 and QA2 can be determined to be duplicate knowledge. As another example, when multiple QA knowledge sets include QA3, where the similarity between the question in QA3 and the question in QA1 is higher than or equal to a first threshold and the similarity between the answer in QA3 and the answer in QA1 is lower than or equal to a second threshold, then QA1 and QA3 can be determined to be conflicting, where either QA1 or QA3 is incorrect knowledge. Based on the above considerations, in the QA-type knowledge base (i.e., the preset knowledge base) applied to RAG, the management method of the QA-type knowledge base provided in this application embodiment can update the preset knowledge base according to the similarity between the questions of each QA knowledge and the questions of other QA knowledge, as well as the similarity between the answers of each QA knowledge and the answers of other QA knowledge, thereby improving the content quality of the QA-type knowledge base. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0044] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0045] Figure 1 One of the flowcharts for a QA-type knowledge base management method provided in this embodiment of the disclosure;

[0046] Figure 2 A second flowchart illustrating a method for managing a QA-type knowledge base, provided in an embodiment of this disclosure;

[0047] Figure 3 A third flowchart illustrating a method for managing a QA-type knowledge base, provided as an embodiment of this disclosure;

[0048] Figure 4 A fourth flowchart illustrating a method for managing a QA-type knowledge base, provided as an embodiment of this disclosure;

[0049] Figure 5 Fifth flowchart illustrating a method for managing a QA-type knowledge base provided in this embodiment of the disclosure;

[0050] Figure 6 A flowchart illustrating a method for managing a QA-type knowledge base, as provided in this embodiment of the disclosure, is shown in Figure 6.

[0051] Figure 7 This is the seventh flowchart illustrating a method for managing a QA-type knowledge base, as provided in this embodiment of the disclosure.

[0052] Figure 8 This is the eighth flowchart illustrating a method for managing a QA-type knowledge base, as provided in this embodiment of the disclosure.

[0053] Figure 9 A flowchart illustrating a method for managing a QA-type knowledge base, as provided in this embodiment of the disclosure, is shown in Figure 9.

[0054] Figure 10 A flowchart illustrating a method for managing a QA-type knowledge base, as provided in this embodiment of the disclosure, is shown in Figure 10.

[0055] Figure 11 This is one of the structural schematic diagrams of a knowledge base management device provided in an embodiment of this disclosure;

[0056] Figure 12 This is a schematic diagram of the structure of a knowledge base management device provided in an embodiment of the present disclosure. Detailed Implementation

[0057] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0058] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0059] The technical solutions provided in the embodiments of this application will be described below with reference to examples.

[0060] Currently, to improve the content quality of QA knowledge bases used in RAG, a common method is to manually maintain the knowledge base, checking it and deleting erroneous and outdated information. However, as the data volume of QA knowledge bases increases, manual maintenance becomes increasingly difficult. Therefore, how to manage QA knowledge bases used in RAG more efficiently and quickly to improve their content quality is a problem that needs to be solved.

[0061] To address the aforementioned technical problems, this application embodiment considers the following: In multiple QA knowledge sets, the similarity between a question in one QA knowledge set (hereinafter referred to as QA1) and questions in other QA knowledge sets, as well as the similarity between the answer in QA1 and the answers in other QA knowledge sets, can be used to determine whether there are any conflicts or overlaps between the multiple QA knowledge sets and QA1. For example, if multiple QA knowledge sets include QA2, and the similarity between the question in QA2 and the question in QA1 is higher than or equal to a first threshold, and the similarity between the answer in QA2 and the answer in QA1 is higher than or equal to a second threshold, then QA1 and QA2 can be determined to be overlapping knowledge. As another example, if multiple QA knowledge sets include QA3, and the similarity between the question in QA3 and the question in QA1 is higher than or equal to a first threshold, and the similarity between the answer in QA3 and the answer in QA1 is lower than or equal to a second threshold, then QA1 and QA3 can be determined to be conflicting, and either QA1 or QA3 is incorrect knowledge.

[0062] Based on the above considerations, in the QA-type knowledge base (hereinafter referred to as "preset knowledge base") applied to RAG, the management method of the QA-type knowledge base provided in this application embodiment can update the preset knowledge base according to the similarity between the questions of each QA knowledge and the questions of other QA knowledge, as well as the similarity between the answers of each QA knowledge and the answers of other QA knowledge, thereby improving the content quality of the QA-type knowledge base.

[0063] The implementation process of the QA-type knowledge base management method provided in this application embodiment is described below with reference to examples. Specifically, the executing entity of the QA-type knowledge base management method provided in this application embodiment can be a knowledge base management device. When the knowledge base management device is running, it can be used to execute all or part of the steps in the QA-type knowledge base management method provided in this application embodiment. In practical applications, the function of the knowledge base management device can be implemented by personal computers (including desktop computers, laptop computers, handheld computers, and notebook computers, etc.), ultra-mobile personal computers (UMPCs), or electronic devices such as smartphones and servers; or, the function of the above-mentioned knowledge base management device can also be implemented by some hardware / software devices in the above-mentioned electronic devices. This application embodiment does not impose any special limitations on the specific form of the knowledge base management device.

[0064] like Figure 1 As shown in the embodiments of this application, the management method for QA-type knowledge bases may include:

[0065] S101, The knowledge base management device acquires the preset knowledge base applied to RAG.

[0066] The preset knowledge base includes n QA knowledge items, each of which includes a question (query) and an answer (answer). Here, n is a positive integer greater than 1.

[0067] S102, The knowledge base management device calculates the first similarity set and the second similarity set.

[0068] The first similarity set includes the similarity between any two QA questions from n QA knowledge sets.

[0069] For example, assuming the preset knowledge base includes 5 QA knowledge items: QA1, QA2, QA3, QA4, and QA5, then the first similarity set includes the similarity of the questions of each pair of QA knowledge items among these 5 QA knowledge items.

[0070] The second similarity set includes the similarity between the answers to any two QA questions out of n QA questions.

[0071] Taking the example of a preset knowledge base containing 5 QA knowledge items: QA1, QA2, QA3, QA4, and QA5, the second similarity set includes the similarity of the answers to each pair of QA knowledge items within these 5 QA knowledge items.

[0072] In some implementations, such as Figure 2 As shown, S102 may specifically include:

[0073] S1021. The knowledge base management device performs embedding processing on the questions and answers of each QA knowledge in the n QA knowledge sets to obtain the feature vectors of the questions and the feature vectors of the answers of each QA knowledge set in the n QA knowledge sets.

[0074] For example, the knowledge base management device can utilize the Beijing Academy of Artificial Intelligence General Embedding (BGE) model to perform embedding processing on the questions and answers of each of the n QA knowledge sets. Specifically, in the knowledge base management device, the questions and answers of each of the n QA knowledge sets can be input into the BGE model respectively, thereby obtaining the feature vectors of the questions and the feature vectors of the answers of each of the n QA knowledge sets.

[0075] S1022. The knowledge base management device determines a first similarity set based on the feature vector of the question of each QA knowledge in the n QA knowledge sets, and determines a second similarity set based on the feature vector of the answer of each QA knowledge in the n QA knowledge sets.

[0076] For example, taking any two QA knowledge items, QA1 and QA2, from a preset knowledge base, the similarity between QA1 and QA2 can be determined using their feature vectors. Specifically, the similarity between QA1 and QA2 can be calculated as the cosine similarity of their feature vectors. Similarly, the similarity between the answers in QA1 and QA2 can be determined using their feature vectors. Again, the similarity can be calculated as the cosine similarity of their feature vectors. By sequentially arranging and combining the QA knowledge items in the preset knowledge base, and calculating the similarity between each pair of combined QA knowledge items in the above manner, a first similarity set and a second similarity set can be obtained.

[0077] S103. The knowledge base management device updates the preset knowledge base based on the first similarity set and the second similarity set.

[0078] The following describes the specific implementation process of updating the preset knowledge base in three ways.

[0079] In the first implementation, such as Figure 3 As shown, S103 may specifically include the following contents: S103a1-S103a2:

[0080] S103a1, The knowledge base management device determines the first QA knowledge set from n QA knowledge sets based on the first similarity set and the second similarity set.

[0081] Specifically, the similarity between any two QA questions in the first QA knowledge set is higher than or equal to a first threshold, and the similarity between the answers of any two QA questions in the first QA knowledge set is higher than or equal to a second threshold.

[0082] The specific values ​​of the first threshold and the second threshold can be determined in combination with the actual application scenario, and there is no restriction on this in the embodiments of this application.

[0083] S103a2, The knowledge base management device updates the preset knowledge base. Specifically, it deletes a portion of the QA knowledge from the first QA knowledge set from the preset knowledge base.

[0084] Specifically, since the similarity between any two QA questions in the first QA knowledge set is higher than or equal to a first threshold, and the similarity between the answers to any two QA questions in the first QA knowledge set is higher than or equal to a second threshold, meaning that the similarity between the questions and answers of the QA knowledge in the first QA knowledge set is high, it can be understood that the first QA knowledge set contains duplicate QA knowledge. Therefore, by deleting some QA knowledge from the first QA knowledge set from the preset knowledge base, the preset knowledge base can be streamlined, and the content quality of the preset knowledge base can be improved.

[0085] In some designs, the preset knowledge base removes some QA knowledge from the first QA knowledge set. Specifically, it retains the most recently imported QA knowledge from the first QA knowledge set in the preset knowledge base.

[0086] Specifically, when the first QA knowledge set contains duplicate QA knowledge, the effect of deleting outdated QA knowledge in the preset knowledge base can be achieved by retaining the most recently imported QA knowledge in the first QA knowledge set.

[0087] In the second implementation, such as Figure 3 As shown, S103 may specifically include the following contents: S103b1-S103b3:

[0088] S103b1, The knowledge base management device determines the second QA knowledge set from n QA knowledge sets based on the first similarity set and the second similarity set.

[0089] Specifically, the similarity between any two QA questions in the second QA knowledge set is higher than or equal to the third threshold, and the similarity between any two QA answers in the second QA knowledge set is lower than or equal to the fourth threshold.

[0090] The specific values ​​of the third and fourth thresholds can be determined based on the actual application scenario, and this embodiment of the application does not impose any restrictions on this.

[0091] S103b2, The knowledge base management device generates a conflict message and receives user operations corresponding to the conflict message.

[0092] The conflict message is used to display the QA knowledge included in the second QA knowledge set to the user. The user action is used to select a QA knowledge from the second QA knowledge set.

[0093] Specifically, if the similarity between any two QA questions in the second QA knowledge set is higher than or equal to the third threshold, and the similarity between the answers to any two QA questions in the second QA knowledge set is lower than or equal to the fourth threshold, meaning that there are cases where the same question in the second QA knowledge set has different answers, it indicates that there is incorrect knowledge in the second QA knowledge set. In this case, a conflict message can be generated to display the QA knowledge included in the second QA knowledge set to the user, allowing the user to select which QA knowledge to retain through user interaction.

[0094] S103b3, The knowledge base management device updates the preset knowledge base.

[0095] Specifically, the preset knowledge base removes all QA knowledge from the second QA knowledge set except for the QA knowledge selected by the user.

[0096] Through the above implementation, on the one hand, it is possible to detect QA knowledge in the preset knowledge base that may be incorrect knowledge. On the other hand, it is possible to determine the incorrect knowledge in the preset knowledge base by generating conflict messages and receiving user operations. Then, when updating the preset knowledge base, other QA knowledge in the second QA knowledge set except for the QA knowledge selected by the user operation are deleted from the preset knowledge base, thereby achieving the effect of deleting the incorrect QA knowledge in the preset knowledge base.

[0097] In the third implementation, such as Figure 3 As shown, S103 may specifically include the following contents: S103c1-S103c2:

[0098] S103c1, The knowledge base management device determines a third QA knowledge set from n QA knowledge sets based on the first similarity set and the second similarity set.

[0099] Specifically, the similarity between any two QA questions in the third QA knowledge set is lower than or equal to the fifth threshold, and the similarity between the answers of any two QA questions in the third QA knowledge set is higher than or equal to the sixth threshold.

[0100] The specific values ​​of the fifth and sixth thresholds can be determined in combination with the actual application scenario, and there is no restriction on this in the embodiments of this application.

[0101] S103c2, The knowledge base management device updates the preset knowledge base.

[0102] Specifically, the preset knowledge base replaces the third QA knowledge set with QA knowledge that includes multiple questions with the same answer. These multiple questions include questions from various QA knowledge sets within the third QA knowledge set. The "same answer" is obtained by merging the answers from all QA knowledge sets within the third QA knowledge set.

[0103] In the above implementation, considering that the similarity between any two QA questions in the third QA knowledge set is lower than or equal to the fifth threshold, and the similarity between the answers of any two QA questions in the third QA knowledge set is higher than or equal to the sixth threshold, it indicates that the questions in each QA knowledge set in the third QA knowledge set are different questions pointing to the same answer. In this case, the QA knowledge in the third QA knowledge set can be combined to form a QA knowledge that includes multiple questions corresponding to the same answer. In this way, during subsequent RAG processing (such as the reranking process or the process of generating results using LLM), the QA knowledge that includes multiple questions corresponding to the same answer can be treated as a single QA knowledge, occupying only one token, thereby improving the efficiency of token resource utilization.

[0104] In some implementations, after obtaining the first and second similarity sets, the QA knowledge in the preset knowledge base can be arranged and combined to obtain the similarity between the questions of any two QA knowledge items and the similarity between the answers of any two QA knowledge items in the preset knowledge base. This allows determination of whether the QA knowledge items constitute the first, second, and third QA knowledge sets. Then, the preset knowledge base is updated using the content in S103a1-S103a2, S103b1-S103b3, and S103c1-S103c2. Specifically, as shown... Figure 4 As shown, step S103 of this method may further include: determining, according to steps S1031-S1035 below, whether any two QA knowledge items in the preset knowledge base constitute the aforementioned first QA knowledge set, second QA knowledge set, or third QA knowledge set. Specifically:

[0105] S1031, The knowledge base management device determines whether the similarity between the questions of two QA knowledge is higher than the threshold T1, and determines whether the similarity between the answers of two QA knowledge is higher than the threshold T2.

[0106] The specific values ​​of thresholds T1 and T2 can be determined according to actual needs.

[0107] S1032. When the similarity between the questions of two QA knowledge items is not higher than the threshold T1, and the similarity between the answers of two QA knowledge items is higher than the threshold T2, the knowledge base management device determines that the two QA knowledge items constitute a third QA knowledge base.

[0108] Next, the third QA knowledge set can be replaced with QA knowledge that includes multiple questions with the same answer through the process of S103c1-S103c2.

[0109] At this point, threshold T1 is equivalent to the fifth threshold mentioned above, and threshold T2 is equivalent to the sixth threshold mentioned above.

[0110] S1033. When the similarity between the questions of two QA knowledge items is higher than the threshold T1, and the similarity between the answers of two QA knowledge items is higher than the threshold T2, the knowledge base management device determines that the two QA knowledge items constitute the first QA knowledge base.

[0111] At this point, threshold T1 is equivalent to the first threshold mentioned above, and threshold T2 is equivalent to the second threshold mentioned above.

[0112] Next, we can delete some QA knowledge from the first QA knowledge set through the process of S103a1-S103a2.

[0113] S1034. When the similarity between the questions of two QA knowledge items is higher than the threshold T1, and the similarity between the answers of the two QA knowledge items is not higher than the threshold T2, the knowledge base management device determines that the two QA knowledge items constitute a second QA knowledge base.

[0114] At this point, threshold T1 is equivalent to the third threshold mentioned above, and threshold T2 is equivalent to the fourth threshold mentioned above.

[0115] Next, the process of S103b1-S103b2 can be used to delete the QA knowledge in the second QA knowledge set, except for the QA knowledge selected by the user operation.

[0116] In addition, if the similarity between the questions of two QA knowledge items is not higher than the threshold T1 and the similarity between the answers of two QA knowledge items is not higher than the threshold T2, it means that the two QA knowledge items are two independent QA knowledge items, and therefore no further processing is required.

[0117] Furthermore, the method provided in this application can also be applied to scenarios where QA knowledge is imported into a QA-type knowledge base. Specifically, such as... Figure 5 As shown in the embodiments of this application, the management method for QA-type knowledge bases may further include:

[0118] S201, The knowledge base management device acquires the first QA knowledge.

[0119] The first QA knowledge includes the first question and the first answer.

[0120] S202. The knowledge base management device determines the question similarity set and the answer similarity set based on the first QA knowledge.

[0121] The question similarity set includes the similarity between the first question and questions in various QA knowledge sets within the preset knowledge base. The answer similarity set includes the similarity between the first answer and answers in various QA knowledge sets within the preset knowledge base.

[0122] In some implementations, S102 may specifically include:

[0123] S1021. The knowledge base management device performs embedding processing on the first question and the first answer to obtain the feature vector of the first question and the feature vector of the first answer.

[0124] For example, in a knowledge base management device, the first question and the first answer can be input into BGE respectively, thereby obtaining the feature vector of the first question and the feature vector of the first answer.

[0125] S1022. The knowledge base management device determines the question similarity set based on the feature vector of the first question, and determines the answer similarity set based on the first answer.

[0126] For example, on the one hand, the cosine similarity between the feature vector of the first question and the feature vector of the question of each QA knowledge in the preset knowledge base can be calculated to obtain a question similarity set; on the other hand, the cosine similarity between the feature vector of the first answer and the feature vector of the answer of each QA knowledge in the preset knowledge base can be calculated to obtain an answer similarity set.

[0127] S203. The knowledge base management device imports the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set.

[0128] The following section describes the specific implementation process of updating the preset knowledge base using four different methods.

[0129] In the first implementation, such as Figure 6 As shown, S203 may specifically include the following content of S203a:

[0130] S203a. If the knowledge base management device determines, based on the question similarity set and the answer similarity set, that there exists a second QA knowledge in the preset knowledge base that satisfies the first condition, then select one of the first QA knowledge and the second QA knowledge and record it in the preset knowledge base.

[0131] The first condition includes: the similarity between the QA knowledge question and the first question is higher than or equal to a first threshold, and the similarity between the QA knowledge answer and the first answer is higher than or equal to a second threshold. In other words, if both the question and answer of the first and second QA knowledge are highly similar, it can be understood that the first and second QA knowledge are duplicates. Therefore, by selecting one from the first and second QA knowledge and recording it in the preset knowledge base, the preset knowledge base can be streamlined and its content quality improved.

[0132] The specific values ​​of the first threshold and the second threshold can be determined in combination with the actual application scenario, and there is no restriction on this in the embodiments of this application.

[0133] In the second implementation, S203 may specifically include the following contents of S203b1-S203b2:

[0134] S203b1. If the knowledge base management device determines, based on the question similarity set and the answer similarity set, that there exists a third QA knowledge in the preset knowledge base that satisfies the second condition, then it generates an import conflict message and receives a user instruction operation corresponding to the import conflict message.

[0135] The second condition includes: the similarity between the QA question and the first question is higher than or equal to a third threshold, and the similarity between the QA answer and the first answer is lower than or equal to a fourth threshold. In other words, if there are cases where the same question in the first and third QA knowledge sets has different answers, it indicates that there is incorrect knowledge in both sets. In this case, a conflict message can be generated to display the first and third QA knowledge sets to the user, allowing the user to select which QA knowledge to retain through user interaction.

[0136] The import conflict message is used to display third-party QA knowledge and first-party QA knowledge to the user. The user instruction action is used to select one QA knowledge from the third-party QA knowledge and first-party QA knowledge.

[0137] S203b2. The knowledge base management device operates according to user instructions and records one of the first QA knowledge and the third QA knowledge in the preset knowledge base.

[0138] In the third implementation, S203 may specifically include the following content of S203c:

[0139] S203c: If the knowledge base management device determines, based on the question similarity set and the answer similarity set, that there exists a fourth QA knowledge in the preset knowledge base that satisfies the third condition, then in the preset knowledge base, the fourth QA knowledge and the first QA knowledge are merged into QA knowledge that includes multiple questions corresponding to the same answer.

[0140] The third condition includes: the similarity between the QA knowledge question and the first question is lower than or equal to the fifth threshold, and the similarity between the QA knowledge answer and the first answer is higher than or equal to the sixth threshold.

[0141] Several questions included both questions on the fourth QA knowledge level and questions on the first QA knowledge level. The same answer was obtained by combining the answers to the fourth and first QA knowledge questions.

[0142] In the fourth implementation, S203 may specifically include the following content of S203d:

[0143] S203d. If the knowledge base management device determines, based on the question similarity set and the answer similarity set, that there is no QA knowledge in the preset knowledge base that satisfies the first, second, and third conditions mentioned above, then it imports the first QA knowledge into the preset knowledge base.

[0144] In some implementations, after obtaining the question similarity set and answer similarity set mentioned above, such as... Figure 7 As shown, the existence of QA knowledge satisfying the first, second, and third conditions can be determined according to the following steps S2031-S2033. Specifically:

[0145] S2031. The knowledge base management device determines, based on the question similarity set and the answer similarity set, whether there is a second QA knowledge in the preset knowledge base that satisfies the first condition mentioned above.

[0146] If so, then one of the first QA knowledge and the second QA knowledge that meets the first condition can be selected and recorded in the preset knowledge through the process of S203a; if not, then S2032 is executed.

[0147] S2032. The knowledge base management device determines, based on the question similarity set and the answer similarity set, whether there is a third QA knowledge in the preset knowledge base that satisfies the above second condition.

[0148] If so, one of the QA knowledge points, the first QA knowledge and the third QA knowledge, can be recorded in the preset knowledge base through the process of S203b1-S203b2; if not, then S2033 is executed.

[0149] S2033. The knowledge base management device determines, based on the question similarity set and the answer similarity set, whether there is a fourth QA knowledge in the preset knowledge base that satisfies the third condition mentioned above.

[0150] If so, the fourth QA knowledge and the first QA knowledge can be merged into QA knowledge that includes multiple questions with the same answer through the process of S203c; if not, then the first QA knowledge is imported into the preset knowledge base by executing S203d.

[0151] Furthermore, the method provided in this application embodiment can also be applied to scenarios where results are generated for unanswered questions. Specifically, such as... Figure 8 As shown in the embodiments of this application, the management method for QA-type knowledge bases may further include:

[0152] S301, The knowledge base management device obtains the questions to be answered.

[0153] S302, The knowledge base management device calculates the third similarity set.

[0154] The third similarity set includes the similarity between questions in each QA knowledge in the preset knowledge base and the question to be answered. If a fifth QA knowledge in the preset knowledge base includes multiple candidate questions corresponding to the same answer, then the third similarity set includes the highest similarity value between each candidate question in the fifth QA knowledge and the question to be answered.

[0155] For example, such as Figure 9 As shown, taking a preset knowledge base containing four QA knowledge items (QA-a, QA-b, QA-c, and QA-d) as an example, QA-a includes three alternative questions (Q-a1, Q-a2, and Q-a3) with the same answer (Aa); QA-b includes three alternative questions (Q-b1, Q-b2, and Q-b3) with the same answer (Ab); QA-c includes a question (Qc) and an answer (Ac); and QA-d includes three alternative questions (Q-d1, Q-d2, and Q-d3) with the same answer (Ad).

[0156] Furthermore, assume that the similarity between the unsolved problem and Q-a1, Q-a2, and Q-a3 in QA-a is 0.9, 0.94, and 0.6, respectively; the similarity between the unsolved problem and Q-b1, Q-b2, and Q-b3 in QA-b is 0.7, 0.6, and 0.8, respectively; the similarity between the unsolved problem and Qc in QA-c is 0.85; and the similarity between the unsolved problem and Q-d1, Q-d2, and Q-d3 in QA-d is 0.7, 0.75, and 0.6, respectively.

[0157] The third similarity set includes the following similarities: the unsolved problem has a similarity of 0.94 with Q-a2, the unsolved problem has a similarity of 0.8 with Q-b3, the unsolved problem has a similarity of 0.85 with Qc, and the solved problem has a similarity of 0.75 with Q-d3.

[0158] S303. Based on the third similarity set, determine the fourth QA knowledge set from the preset knowledge base.

[0159] The fourth QA knowledge set includes QA knowledge corresponding to similarities above or equal to the similarity threshold in the third similarity set. If the highest similarity value is above or equal to the similarity threshold, the fourth QA knowledge set includes a sixth QA knowledge set consisting of a recombined question and a recombined answer. The recombined question is the question with the highest similarity to the question to be answered from among multiple alternative questions, and the recombined answer is the answer from the fifth QA knowledge set.

[0160] Continuing with the three similarity sets, the similarity between the unsolved problem and Q-a2 is 0.94, the similarity between the unsolved problem and Q-b3 is 0.8, the similarity between the unsolved problem and Qc is 0.85, and the similarity between the solved problem and Q-d3 is 0.75.

[0161] Assuming a similarity threshold of 0.8, the similarities in the third similarity set that are 0.8 or higher include: the similarity between the unsolved question and Q-a2 (0.94), the similarity between the unsolved question and Q-b3 (0.8), and the similarity between the unsolved question and Qc (0.85). Therefore, the fourth QA knowledge set includes: QA knowledge consisting of Q-a2 and Aa, QA knowledge consisting of Q-b3 and Ab, and QA knowledge consisting of Qc and Ac.

[0162] As can be seen, through the above process, on the one hand, the fourth QA knowledge set can be selected from the preset knowledge base if the similarity between the question and the current question to be answered meets the requirements (i.e., the similarity between the question and the current question to be answered is higher than or equal to the similarity threshold), so that the result of the question to be answered can be generated based on the fourth QA knowledge set.

[0163] On the other hand, in the fourth QA knowledge set, for the fifth QA knowledge that includes multiple candidate questions corresponding to the same answer, the sixth QA knowledge is added to the fourth QA knowledge set by selecting the question with the highest similarity to the question to be answered (i.e., the recombined question) from the multiple candidate questions and the answer from the fifth QA knowledge (i.e., the recombined answer). In this way, in subsequent processes, QA knowledge including multiple questions corresponding to the same answer is treated as a single QA knowledge item, occupying only one token, thereby improving the efficiency of token resource utilization.

[0164] S304. The knowledge base management device generates the results of the questions to be answered based on the fourth QA knowledge set.

[0165] In some implementations, such as Figure 10 As shown, S304 may specifically include the following contents of S3041-S3042:

[0166] S3041. The knowledge base management device uses rerank technology to sort the fourth QA knowledge set to obtain the QA knowledge ranking.

[0167] Among them, the QA knowledge ranking is a ranking that indicates the relevance of the QA knowledge in the fourth QA knowledge set to the question to be answered.

[0168] In the above implementation, since the fourth QA knowledge set used for sorting includes multiple QA knowledge points corresponding to the same answer for multiple questions, each occupying only one token, a more suitable QA knowledge sorting can be obtained. For example, taking QA-a as an example, according to existing technology, QA-a can be divided into three QA knowledge points. Therefore, when using rerank technology for sorting, multiple QA knowledge points corresponding to the three QA knowledge points of QA-a may be ranked at the top. For example, since Q-a1, Q-a2, and the question to be answered are highly similar, Q-a1 and Q-a2 may both be ranked at the top. However, since Q-a1 and Q-a2 correspond to the same answer, it is unreasonable to rank Q-a1 and Q-a2 at the top. The above implementation provided in the embodiments of this application avoids this problem.

[0169] S3042. The knowledge base management device uses LLM to sort QA knowledge and generate results for questions to be answered.

[0170] Since multiple questions corresponding to the same answer in QA knowledge ranking only occupy one token, LLM can be used to obtain more accurate results for the questions to be answered by ranking QA knowledge.

[0171] Based on the same inventive concept, as an implementation of the above method, this application also provides a knowledge base management device. This embodiment corresponds to the aforementioned method embodiment. For ease of reading, this embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the knowledge base management device in this embodiment can implement all the contents of the aforementioned method embodiment.

[0172] Specifically, such as Figure 11 As shown, the knowledge base management device 40 may include all or part of the following units: acquisition unit 401, calculation unit 402, update unit 403, and generation unit 404.

[0173] In some implementations:

[0174] The acquisition unit 401 is used to acquire a preset knowledge base for RAG; the preset knowledge base includes n QA knowledge, each QA knowledge includes a question and an answer, and n is a positive integer greater than 1.

[0175] The calculation unit 402 is used to calculate a first similarity set and a second similarity set; wherein, the first similarity set includes: the similarity of questions between any two QA knowledge items among the n QA knowledge items; the second similarity set includes: the similarity of answers between any two QA knowledge items among the n QA knowledge items.

[0176] The update unit 403 is used to update the preset knowledge base according to the first similarity set and the second similarity set.

[0177] In some implementations, the update unit 403 is used to update the preset knowledge base based on the first similarity set and the second similarity set, including:

[0178] The updating unit 403 is specifically used to determine a first QA knowledge set from the n QA knowledge sets based on the first similarity set and the second similarity set; wherein the similarity of the questions of any two QA knowledge sets in the first QA knowledge set is higher than or equal to a first threshold, and the similarity of the answers of any two QA knowledge sets in the first QA knowledge set is higher than or equal to a second threshold.

[0179] The update unit 403 is specifically used to update the preset knowledge base; wherein, the preset knowledge base deletes some QA knowledge from the first QA knowledge set.

[0180] In some implementations, the update unit 403 is used to update the preset knowledge base based on the first similarity set and the second similarity set, including:

[0181] The updating unit 403 is specifically used to determine a second QA knowledge set from the n QA knowledge sets based on the first similarity set and the second similarity set; wherein the similarity between the questions of any two QA knowledge sets in the second QA knowledge set is higher than or equal to a third threshold, and the similarity between the answers of any two QA knowledge sets in the second QA knowledge set is lower than or equal to a fourth threshold.

[0182] The update unit 403 is specifically used to generate a conflict message and receive a user operation corresponding to the conflict message; the conflict message is used to display the QA knowledge included in the second QA knowledge set to the user; the user operation is used to select a QA knowledge from the second QA knowledge set.

[0183] The update unit 403 is specifically used to update the preset knowledge base; wherein, the preset knowledge base deletes other QA knowledge in the second QA knowledge set except for the QA knowledge selected by the user operation.

[0184] In some implementations, the update unit 403 is used to update the preset knowledge base according to the first similarity set and the second similarity set, including:

[0185] The updating unit 403 is specifically used to determine a third QA knowledge set from the n QA knowledge sets based on the first similarity set and the second similarity set; wherein the similarity between the questions of any two QA knowledge sets in the third QA knowledge set is lower than or equal to a fifth threshold, and the similarity between the answers of any two QA knowledge sets in the third QA knowledge set is higher than or equal to a sixth threshold.

[0186] The update unit 403 is specifically used to update the preset knowledge base; wherein, the preset knowledge base replaces the third QA knowledge set with QA knowledge that includes multiple questions corresponding to the same answer; the multiple questions include: questions of each QA knowledge in the third QA knowledge set; the same answer is obtained by merging the answers of each QA knowledge in the third QA knowledge set.

[0187] In some implementations, the calculation unit 402 is used to calculate the first similarity set and the second similarity set, including:

[0188] The calculation unit 402 is specifically used to perform embedding processing on the questions and answers of each of the n QA knowledge points to obtain the feature vectors of the questions of each of the n QA knowledge points and the feature vectors of the answers of each of the n QA knowledge points.

[0189] The calculation unit 402 is specifically used to determine the first similarity set based on the feature vector of the question of each QA knowledge in the n QA knowledge, and to determine the second similarity set based on the feature vector of the answer of each QA knowledge in the n QA knowledge.

[0190] In some implementations, the acquisition unit 401 is also used to acquire the question to be answered.

[0191] The calculation unit 402 is also used to calculate the third similarity set.

[0192] The third similarity set includes the similarity between questions in each QA knowledge in the preset knowledge base and the question to be answered; wherein, if the fifth QA knowledge in the preset knowledge base includes multiple candidate questions corresponding to the same answer, then the third similarity set includes: the highest similarity value between the multiple candidate questions in the fifth QA knowledge and the question to be answered.

[0193] The calculation unit 402 is also configured to determine a fourth QA knowledge set from the preset knowledge base based on the third similarity set.

[0194] The fourth QA knowledge set includes QA knowledge corresponding to similarities higher than or equal to a similarity threshold in the third similarity set; wherein, if the highest value is higher than or equal to the similarity threshold, the fourth QA knowledge set includes sixth QA knowledge consisting of a recombined question and a recombined answer, wherein the recombined question is the question with the highest similarity to the question to be answered among the plurality of candidate questions, and the recombined answer is the answer in the fifth QA knowledge.

[0195] The generation unit 404 is used to generate the result of the question to be answered based on the fourth QA knowledge set.

[0196] In some implementations, generation unit 404 is used to generate a result for the question to be answered based on a fourth QA knowledge set, including:

[0197] The generation unit 404 is specifically used to sort the fourth QA knowledge set using reranking technology to obtain a QA knowledge ranking; the QA knowledge ranking is a ranking that indicates the relevance of the QA knowledge in the fourth QA knowledge set to the question to be answered.

[0198] The generation unit 404 is specifically used to generate the results of the question to be answered by using the LLM large language model and sorting the QA knowledge.

[0199] In some implementations, the acquisition unit 401 is also used to acquire first QA knowledge; the first QA knowledge includes a first question and a first answer.

[0200] The calculation unit 402 is further configured to determine a question similarity set and an answer similarity set based on the first QA knowledge; wherein the question similarity set includes: the similarity between the first question and the questions in each QA knowledge in the preset knowledge base; the answer similarity set includes: the similarity between the first answer and the answers in each QA knowledge in the preset knowledge base.

[0201] The updating unit 403 is further configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set.

[0202] In some implementations, the update unit 403 is further configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, including:

[0203] The updating unit 403 is further configured to, if it is determined from the question similarity set and the answer similarity set that there exists a second QA knowledge in the preset knowledge base that satisfies the first condition, select one of the first QA knowledge and the second QA knowledge and record it in the preset knowledge base; the first condition includes: the similarity between the question of the QA knowledge and the first question is higher than or equal to a first threshold, and the similarity between the answer of the QA knowledge and the first answer is higher than or equal to a second threshold.

[0204] In some implementations, the update unit 403 is further configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, including:

[0205] The updating unit 403 is further configured to generate an import conflict message and receive a user instruction operation corresponding to the import conflict message if it is determined, based on the question similarity set and the answer similarity set, that there is a third QA knowledge in the preset knowledge base that satisfies the second condition.

[0206] The second condition includes: the similarity between the QA knowledge question and the first question is higher than or equal to a third threshold, and the similarity between the QA knowledge answer and the first answer is lower than or equal to a fourth threshold; the import conflict message is used to show the third QA knowledge and the first QA knowledge to the user; the user instruction operation is used to select one QA knowledge from the third QA knowledge and the first QA knowledge.

[0207] The update unit 403 is also configured to record one of the first QA knowledge and the third QA knowledge in the preset knowledge base according to the user instruction operation.

[0208] In some implementations, the update unit 403 is further configured to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, including:

[0209] The updating unit 403 is further configured to, if it is determined from the question similarity set and the answer similarity set that there exists a fourth QA knowledge in the preset knowledge base that satisfies the third condition, then merge the fourth QA knowledge and the first QA knowledge in the preset knowledge base into QA knowledge that includes multiple questions corresponding to the same answer.

[0210] The third condition includes: the similarity between the QA knowledge question and the first question is lower than or equal to a fifth threshold, and the similarity between the QA knowledge answer and the first answer is higher than or equal to a sixth threshold; the multiple questions include: the fourth QA knowledge question and the first QA knowledge question; the same answer is obtained by merging the answers of the fourth QA knowledge question and the first QA knowledge question.

[0211] In other implementations:

[0212] The acquisition unit 401 is used to acquire first QA knowledge; the first QA knowledge includes a first question and a first answer.

[0213] The calculation unit 402 is used to determine a question similarity set and an answer similarity set based on the first QA knowledge; wherein, the question similarity set includes: the similarity between the first question and the questions in each QA knowledge in the preset knowledge base; the answer similarity set includes: the similarity between the first answer and the answers in each QA knowledge in the preset knowledge base; the preset knowledge base includes n QA knowledge, each QA knowledge includes a question and an answer, and n is a positive integer greater than 1.

[0214] The update unit 403 is used to import the first QA knowledge into the preset knowledge base according to the question similarity set and the answer similarity set.

[0215] In some implementations, the update unit 403 is used to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, including:

[0216] The updating unit 403 is configured to select one of the first QA knowledge and the second QA knowledge and record it in the preset knowledge base if it is determined that there is a second QA knowledge in the preset knowledge base that satisfies the first condition based on the question similarity set and the answer similarity set; the first condition includes: the similarity between the question of the QA knowledge and the first question is higher than or equal to a first threshold, and the similarity between the answer of the QA knowledge and the first answer is higher than or equal to a second threshold.

[0217] In some implementations, the update unit 403 is used to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, including:

[0218] The update unit 403 is specifically used to generate an import conflict message and receive a user instruction operation corresponding to the import conflict message if it is determined that there is a third QA knowledge in the preset knowledge base that satisfies the second condition based on the question similarity set and the answer similarity set.

[0219] The second condition includes: the similarity between the QA knowledge question and the first question is higher than or equal to a third threshold, and the similarity between the QA knowledge answer and the first answer is lower than or equal to a fourth threshold; the import conflict message is used to show the third QA knowledge and the first QA knowledge to the user; the user instruction operation is used to select one QA knowledge from the third QA knowledge and the first QA knowledge.

[0220] The update unit 403 is specifically used to record one of the first QA knowledge and the third QA knowledge in the preset knowledge base according to the user's instruction.

[0221] In some implementations, the update unit 403 is used to import the first QA knowledge into the preset knowledge base based on the question similarity set and the answer similarity set, including:

[0222] The updating unit 403 is configured to, if it is determined from the question similarity set and the answer similarity set that there exists a fourth QA knowledge in the preset knowledge base that satisfies the third condition, then merge the fourth QA knowledge and the first QA knowledge in the preset knowledge base into QA knowledge that includes multiple questions corresponding to the same answer.

[0223] The third condition includes: the similarity between the QA knowledge question and the first question is lower than or equal to a fifth threshold, and the similarity between the QA knowledge answer and the first answer is higher than or equal to a sixth threshold; the multiple questions include: the fourth QA knowledge question and the first QA knowledge question; the same answer is obtained by merging the answers of the fourth QA knowledge question and the first QA knowledge question.

[0224] The knowledge base management device 40 provided in this application embodiment can execute all or part of the steps in S101-S103, S201-S203 and S301-S304 described above. Its implementation principle and technical effect are similar, and will not be described again here.

[0225] Based on the same inventive concept Figure 12 Another schematic diagram of the knowledge base management device provided in the embodiments of this application is shown below. Figure 12 As shown, the knowledge base management device provided in this embodiment includes a memory 501 and a processor 502. The memory 501 is used to store computer programs, and the processor 502 is used to execute any of the methods provided in the above embodiment when executing the computer programs.

[0226] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the computing device to implement the method provided in the above embodiments.

[0227] Based on the same inventive concept, this application also provides a computer program product that, when run on a computer, enables the computing device to implement the method provided in the above embodiments.

[0228] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0229] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0230] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0231] Computer-readable media include both permanent and non-permanent, removable and non-removable storage media. Storage media can store information using any method or technology; the information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0232] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A management method for a QA-type knowledge base, characterized in that, The method includes: Obtain a preset knowledge base for RAG; the preset knowledge base includes n QA knowledge points, each QA knowledge point includes a question and an answer, where n is a positive integer greater than 1; Calculate a first similarity set and a second similarity set; wherein, the first similarity set includes: the similarity of questions between any two QA knowledge items from the n QA knowledge items; the second similarity set includes: the similarity of answers between any two QA knowledge items from the n QA knowledge items; The preset knowledge base is updated based on the first similarity set and the second similarity set.

2. The method according to claim 1, characterized in that, The step of updating the preset knowledge base based on the first similarity set and the second similarity set includes: Based on the first similarity set and the second similarity set, a first QA knowledge set is determined from the n QA knowledge sets; wherein, the similarity between the questions of any two QA knowledge sets in the first QA knowledge set is higher than or equal to a first threshold, and the similarity between the answers of any two QA knowledge sets in the first QA knowledge set is higher than or equal to a second threshold. Update the preset knowledge base; wherein, the preset knowledge base deletes some QA knowledge from the first QA knowledge set.

3. The method according to claim 1, characterized in that, The step of updating the preset knowledge base based on the first similarity set and the second similarity set includes: Based on the first similarity set and the second similarity set, a second QA knowledge set is determined from the n QA knowledge sets; wherein, the similarity between the questions of any two QA knowledge sets in the second QA knowledge set is higher than or equal to a third threshold, and the similarity between the answers of any two QA knowledge sets in the second QA knowledge set is lower than or equal to a fourth threshold. A conflict message is generated and a user action corresponding to the conflict message is received; the conflict message is used to display QA knowledge included in the second QA knowledge set to the user; the user action is used to select a QA knowledge from the second QA knowledge set. Update the preset knowledge base; wherein, delete all QA knowledge in the second QA knowledge set except for the QA knowledge selected by the user operation.

4. The method according to claim 1, characterized in that, The step of updating the preset knowledge base based on the first similarity set and the second similarity set includes: Based on the first similarity set and the second similarity set, a third QA knowledge set is determined from the n QA knowledge sets; wherein, the similarity between the questions of any two QA knowledge sets in the third QA knowledge set is lower than or equal to a fifth threshold, and the similarity between the answers of any two QA knowledge sets in the third QA knowledge set is higher than or equal to a sixth threshold. Update the preset knowledge base; wherein, the preset knowledge base replaces the third QA knowledge set with QA knowledge that includes multiple questions corresponding to the same answer; the multiple questions include: questions of each QA knowledge in the third QA knowledge set; the same answer is obtained by merging the answers of each QA knowledge in the third QA knowledge set.

5. The method according to any one of claims 1-4, characterized in that, The calculation of the first similarity set and the second similarity set includes: Embedding is performed on the questions and answers of each of the n QA knowledge sets to obtain the feature vectors of the questions and the feature vectors of the answers of each of the n QA knowledge sets. The first similarity set is determined based on the feature vector of the question in each of the n QA knowledge sets, and the second similarity set is determined based on the feature vector of the answer in each of the n QA knowledge sets.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: Get the questions to be answered; Calculate the third similarity set; The third similarity set includes the similarity between questions in each QA knowledge in the preset knowledge base and the question to be answered; wherein, if the fifth QA knowledge in the preset knowledge base includes multiple candidate questions corresponding to the same answer, then the third similarity set includes: the highest similarity value between the multiple candidate questions in the fifth QA knowledge and the question to be answered. Based on the third similarity set, a fourth QA knowledge set is determined from the preset knowledge base; The fourth QA knowledge set includes: QA knowledge corresponding to similarities higher than or equal to a similarity threshold in the third similarity set; wherein, if the highest value is higher than or equal to the similarity threshold, the fourth QA knowledge set includes sixth QA knowledge consisting of a recombined question and a recombined answer, wherein the recombined question is the question with the highest similarity to the question to be answered among the plurality of candidate questions, and the recombined answer is the answer in the fifth QA knowledge; Based on the fourth QA knowledge set, the results for the question to be answered are generated.

7. The method according to claim 6, characterized in that, The step of generating the result for the question to be answered based on the fourth QA knowledge set includes: The fourth QA knowledge set is sorted using the reranking technique to obtain a QA knowledge ranking; the QA knowledge ranking is a ranking that indicates the relevance of the QA knowledge in the fourth QA knowledge set to the question to be answered. Using the LLM large language model, the results of the questions to be answered are generated based on the sorting of the QA knowledge.

8. A management method for a QA-type knowledge base, characterized in that, The method includes: Acquire initial QA knowledge; the initial QA knowledge includes the initial question and the initial answer; Based on the first QA knowledge, a question similarity set and an answer similarity set are determined; wherein, the question similarity set includes: the similarity between the first question and the questions in each QA knowledge in the preset knowledge base; the answer similarity set includes: the similarity between the first answer and the answers in each QA knowledge in the preset knowledge base; the preset knowledge base includes n QA knowledge, each QA knowledge includes a question and an answer, where n is a positive integer greater than 1; Based on the question similarity set and the answer similarity set, the first QA knowledge is imported into the preset knowledge base.

9. A knowledge base management device, characterized in that, include: A memory and a processor, wherein the memory is used to store a computer program and the processor is used to cause the knowledge base management device to implement the management method of the QA-type knowledge base as described in any one of claims 1-7 or claim 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a computing device, causes the computing device to implement the management method for a QA-type knowledge base as described in any one of claims 1-7 or claim 8.

11. A computer program product, characterized in that, When the computer program product is run on a computer, it enables the computer to implement the management method of the QA-type knowledge base as described in any one of claims 1-7 or claim 8.

Images