Knowledge data processing method and device, equipment, medium and product
By determining the target knowledge retrieval results based on confidence and relevance in the knowledge base and reducing duplicate documents in the construction stage, the problem of knowledge conflict in the knowledge base is solved, and the accuracy and rationality of large language models are improved, and the Processing efficiency and user experience.
Patent Information
- Application Number
- CN202410838464.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-26
- Publication Date
- 2024-11-05
AI Technical Summary
In the prior art, the repetition and conflict of knowledge documents in the knowledge base lead to the reduction of the accuracy and rationality of the reply content generated by large language models, and the lack of automated processing methods, resulting in low processing efficiency.
By obtaining multiple candidate knowledge search results related to user requests from the knowledge base, determining the target knowledge search results based on confidence and correlation, generating reply content using a big model, and reducing duplicate documents through similarity analysis during the knowledge base construction stage Upload.
It improves the accuracy and rationality of the content of the big model reply, improves the efficiency of knowledge conflict detection, reduces knowledge conflicts, and improves user experience.
Smart Images

Figure CN118898293A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence, and in particular to a method, device, equipment, medium and product for processing knowledge data. Background Art
[0002] Retrieval-augmented Generation (RAG) refers to providing additional information to large language models (LLMs) through external knowledge bases, so that LLMs can generate more accurate and contextual answers while reducing model hallucinations.
[0003] In order to improve the accuracy and comprehensiveness of the additional information, the knowledge base needs to be continuously expanded. As the amount of knowledge in the knowledge base increases, knowledge documents will be repeated or knowledge in knowledge documents will conflict. Providing these conflicting knowledge to LLM will result in poor response content generated by LLM and fail to meet user needs.
[0004] At present, the conflicting knowledge is usually processed by manual screening, which makes the processing efficiency of knowledge conflict low. Therefore, a method for automatically processing knowledge conflict is urgently needed. Summary of the invention
[0005] Based on the above-mentioned technical status, the present application provides a method, device, equipment and storage medium for processing knowledge data, which can improve the accuracy and rationality of response content generated by large models.
[0006] In order to achieve the above technical objectives, this application specifically proposes the following technical solutions:
[0007] According to the first aspect of an embodiment of the present application, a method for processing knowledge data is provided, comprising: obtaining a plurality of candidate knowledge retrieval results related to a user's knowledge request from a knowledge base; determining a target knowledge retrieval result for responding to the knowledge request from the plurality of candidate knowledge retrieval results based on the confidence of the plurality of candidate knowledge retrieval results and the relevance between each of the plurality of candidate knowledge retrieval results and the knowledge request; the confidence is determined based on a result of evaluating the correctness of knowledge in a historical knowledge retrieval process; and generating reply content corresponding to the knowledge request based on the target knowledge retrieval result through the large model.
[0008] In some implementations, based on the confidence of the multiple candidate knowledge retrieval results and the relevance between each of the multiple candidate knowledge retrieval results and the knowledge request, a target knowledge retrieval result for responding to the knowledge request is determined from the multiple candidate knowledge retrieval results, including: determining, as an initial knowledge retrieval result, a candidate knowledge retrieval result whose relevance label with the knowledge request is a preset relevance label among the multiple candidate knowledge retrieval results; and determining, based on the confidence of the initial knowledge retrieval result, a target knowledge retrieval result for responding to the knowledge request from the initial knowledge retrieval results.
[0009] In some implementations, the target knowledge retrieval results for responding to the knowledge request are determined from the initial knowledge retrieval results based on the confidence of the initial knowledge retrieval results, including: sorting the initial knowledge retrieval results in order of confidence from high to low to obtain a confidence sorting result; for the two initial knowledge retrieval results ranked highest in the confidence sorting result, if there is a knowledge conflict between the two initial knowledge retrieval results, then when the difference in confidence between the two initial knowledge retrieval results is less than a preset difference, the two initial knowledge retrieval results are determined as the target knowledge retrieval results; or, when the difference in confidence between the two initial knowledge retrieval results is greater than or equal to a preset difference, the initial knowledge retrieval result with a higher confidence among the two initial knowledge retrieval results is determined as the target knowledge retrieval result.
[0010] In some implementations, the initial knowledge retrieval results are sorted in descending order according to confidence to obtain a sorting result, including: if any two initial knowledge retrieval results have the same confidence, the two initial knowledge retrieval results are sorted in descending order according to the relevance between them and the knowledge request to obtain the confidence sorting result.
[0011] In some implementations, the knowledge base includes multiple knowledge documents, the initial knowledge retrieval results include the knowledge documents, and when the difference in confidence between two initial knowledge retrieval results is less than a preset difference, after the two initial knowledge retrieval results are determined as target knowledge retrieval results, the method further includes: obtaining correctness evaluation information corresponding to each of the two initial knowledge retrieval results, the correctness evaluation information representing the degree of closeness between the initial knowledge retrieval results and the correct knowledge; based on the correctness evaluation information corresponding to each of the two initial knowledge retrieval results, updating the confidence of the initial knowledge retrieval result that is closest to the correct knowledge among the two initial knowledge retrieval results.
[0012] In some implementations, before obtaining multiple candidate knowledge retrieval results related to the user's knowledge request from the knowledge base, the method also includes: determining the similarity between the original document to be uploaded to the knowledge base and each knowledge document in the knowledge base; determining the knowledge documents in the knowledge documents whose similarity to the original document is greater than a preset similarity as similar documents corresponding to the original document; obtaining indication information, the indication information being used to indicate whether to upload the original document to the knowledge base; and based on the indication information, performing storage processing on the original document.
[0013] In some implementations, after determining the knowledge documents in the knowledge document whose similarity with the original document is greater than a preset similarity as similar documents corresponding to the original document, the method further includes: determining a comparison result between the original document and the similar document; and outputting the comparison result, wherein the comparison result is used to assist the user in determining the indication information.
[0014] In some implementations, the comparison result includes inconsistent content between the original document and the similar document, and the inconsistent content is highlighted.
[0015] According to the second aspect of an embodiment of the present application, a knowledge data processing device is provided, comprising: an acquisition unit, used to acquire multiple candidate knowledge retrieval results related to a user's knowledge request from a knowledge base; a determination unit, used to determine a target knowledge retrieval result for responding to the knowledge request from the multiple candidate knowledge retrieval results based on the confidence of the multiple candidate knowledge retrieval results and the relevance between each of the multiple candidate knowledge retrieval results and the knowledge request; the confidence is determined based on a result of evaluating the correctness of knowledge in a historical knowledge retrieval process; and a generation unit, used to generate a reply content corresponding to the knowledge request based on the target knowledge retrieval result through a large model.
[0016] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor; the memory is connected to the processor and is used to store programs; the processor is used to implement the knowledge data processing method as described in the first aspect by running the program in the memory.
[0017] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for processing knowledge data as described in the first aspect is implemented.
[0018] The embodiment of the present application provides a method, device, equipment, medium and product for processing knowledge data. The method obtains multiple candidate knowledge retrieval results related to the user's knowledge request from the knowledge base, and determines the target knowledge retrieval result for responding to the knowledge request from the multiple candidate knowledge retrieval results based on the confidence of the multiple candidate knowledge retrieval results and the relevance between the multiple candidate knowledge retrieval results and the knowledge request. The confidence is determined according to the correctness evaluation results of the knowledge in the historical knowledge retrieval process, and the reply content corresponding to the knowledge request is generated based on the target knowledge retrieval result through the large model. Since the target knowledge retrieval result for generating the reply content is determined by combining the confidence of the multiple candidate knowledge retrieval results and the relevance between the multiple candidate knowledge retrieval results and the knowledge request, the knowledge accuracy of the target knowledge retrieval result can be improved. In addition, since the confidence is determined based on the correctness evaluation results of the knowledge in the historical knowledge retrieval process, the knowledge accuracy of the target knowledge retrieval result can be further improved to assist the large model in generating more accurate and more reasonable reply content and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0020] Figure 1 A flowchart of an embodiment of a method for processing knowledge data provided in an embodiment of the present application;
[0021] Figure 2 A schematic diagram of a process for determining target knowledge retrieval results provided in an embodiment of the present application;
[0022] Figure 3 A schematic diagram of an interactive interface of a voting page provided in an embodiment of the present application;
[0023] Figure 4 A schematic diagram of another interactive interface of a voting page provided in an embodiment of the present application;
[0024] Figure 5 A flowchart of another embodiment of the method for processing knowledge data provided in an embodiment of the present application;
[0025] Figure 6 A schematic diagram of a double-column page provided in an embodiment of the present application;
[0026] Figure 7A flowchart of the overall solution of the method for processing knowledge data provided in the embodiment of the present application;
[0027] Figure 8 A schematic diagram of the structure of a knowledge data processing device provided in an embodiment of the present application;
[0028] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0029] The technical solution proposed in the embodiments of the present application is applicable to scenarios of multiple natural language processing tasks such as question-answering systems, document generation, and intelligent assistants. The technical solution of the embodiments of the present application can improve the efficiency of detecting knowledge conflicts and reduce knowledge conflicts, thereby improving the accuracy of the reply content generated by the large model.
[0030] The technical solution provided in the embodiments of the present application can be exemplarily applied to hardware devices such as processors, electronic devices, servers (including cloud servers), or packaged into software programs to be run. When the hardware device executes the processing of the technical solution of the embodiments of the present application, or the above-mentioned software program is run, the automatic splitting of the target task and the automatic calling of the application program interface required for the task can be achieved to complete the purpose of the target task. The embodiments of the present application only exemplarily introduce the specific processing of the technical solution of the present application, and do not limit the specific implementation form of the technical solution of the present application. Any technical implementation form that can execute the processing of the technical solution of the present application can be adopted by the embodiments of the present application.
[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0032] Before introducing this application solution, the relevant technologies are first introduced:
[0033] In question-answering and dialogue scenarios, a response can usually be obtained through two methods: retrieval and generation. Retrieval-based responses are to retrieve satisfactory responses from an external knowledge base. They are more reliable and controllable, but lack diversity. Generative responses rely on internal knowledge stored in a large language model. They are uncontrollable and have poor interpretability, but can generate richer response content. Combining retrieval and generation, we can obtain a generation model powered by external knowledge retrieval, namely a retrieval-generation model.
[0034] The retrieval generation model works by retrieving knowledge related to the query (knowledge request) from the knowledge base (large-scale document collection), and then providing this knowledge to the big model as reference knowledge to achieve more well-founded responses, alleviate big model hallucinations, and solve data privacy and leakage issues, thereby enhancing its ability to generate accurate and relevant answers.
[0035] RAG consists of three parts: retrieval, utilization, and generation. In the retrieval phase, relevant knowledge is retrieved from the knowledge base according to the knowledge request, and in the utilization phase, the retrieved relevant knowledge is used to fill the text or answer the question, and in the generation phase, the final text content is generated according to the retrieved relevant knowledge as the response content.
[0036] Knowledge retrieval is inseparable from the construction of a knowledge base. The knowledge base is built by relying on a large number of knowledge documents uploaded by users. Since the knowledge base is relatively large and the review of uploaded documents is relatively loose, it is inevitable that there will be problems such as duplicate knowledge documents or knowledge conflicts in knowledge documents. When a large model refers to conflicting knowledge to respond, it will lead to incorrect responses or even nonsense. Therefore, solving the problem of knowledge conflicts becomes particularly critical.
[0037] Currently, manual screening is usually used to remove duplicate or conflicting knowledge in a knowledge base. However, as the amount of knowledge in the knowledge base increases, the need for an automated method to identify and handle knowledge conflicts becomes urgent.
[0038] Usually, during the knowledge base construction phase, users may upload different versions of the same knowledge document due to omissions when uploading knowledge documents. For example, most of the text content in the two versions of the knowledge document is the same, but some values in the document are modified or the data content is updated. This will result in knowledge documents with high similarity in the knowledge base. In addition, there may be knowledge with different expressions for the same problem in the knowledge base. During the knowledge retrieval phase, the knowledge in the different versions of the knowledge document or the knowledge with different expressions is sent to the big model as the knowledge retrieval results. The big model generates reply content based on these conflicting knowledge, which will affect the accuracy and rationality of the reply content, and thus cause a bad user experience.
[0039] In view of this, the embodiments of the present application are committed to providing a method, device, equipment and storage medium for processing knowledge data, which can automatically process knowledge conflicts and effectively resolve knowledge conflicts, thereby improving the accuracy and rationality of the large model reply content, which are described in detail one by one in the following embodiments.
[0040] Exemplary Methods
[0041] Figure 1A flowchart of a method for processing knowledge data provided in an embodiment of the present application. Figure 1 As shown, the knowledge data processing method provided in this embodiment includes steps S101 to S103:
[0042] S101. Acquire multiple candidate knowledge retrieval results related to the user's knowledge request from a knowledge base.
[0043] A knowledge request (query) refers to a question or request that a user asks to the big model. It is a way of communication between the user and the big model. The user can express his or her need to obtain information through a query. The user can enter a query in the question box of the big model to ask questions and get the answers they want. For example, "What is the height of Mount Everest?", "How many countries are there in the world?", "How many ethnic groups are there in China?", etc.
[0044] After receiving the user's knowledge request, knowledge search can be performed in the knowledge base based on the knowledge request to obtain multiple candidate knowledge search results corresponding to the knowledge request. The process of performing knowledge search in the knowledge base based on the knowledge request to obtain multiple candidate knowledge search results corresponding to the knowledge request includes the following steps a1 to a4:
[0045] a1. Embed the query through the representation model to obtain the vector representation V corresponding to the query.
[0046] a2. Use the vector representation V to perform knowledge retrieval in the knowledge base and obtain the original knowledge retrieval result set.
[0047] The knowledge base includes multiple chapter libraries, each of which includes chapters obtained by segmenting a knowledge document. Segmenting knowledge documents and storing knowledge in the form of chapter libraries can facilitate knowledge retrieval, help reduce the burden of large models, and improve the accuracy and efficiency of information retrieval.
[0048] The knowledge base is a vector database, and the knowledge in each chapter library in the knowledge base is stored in the form of vectors. The knowledge in the knowledge base corresponds to a storage vector. The vector representation V is used to perform knowledge retrieval in the knowledge base to obtain a set of original knowledge retrieval results, including: vectorizing the knowledge request to obtain the query vector corresponding to the knowledge request; calculating the similarity between the query vector and the storage vector corresponding to each knowledge in the knowledge base, and returning the knowledge corresponding to the Topn storage vectors with the highest similarity as the original knowledge retrieval result set. The similarity can be obtained based on similarity calculation methods such as cosine similarity, Euclidean distance, Manhattan distance, etc. n is an integer greater than or equal to 0.
[0049] Since the knowledge in different chapter libraries may be the same, there may be duplicate knowledge in the original knowledge retrieval result set. Sending this duplicate knowledge to the large model will limit the knowledge received by the large model. In order to increase the breadth of knowledge received by the large model, this embodiment can also remove duplicates from the original knowledge retrieval result set, thereby obtaining multiple candidate knowledge retrieval results. The specific process of removing duplicates can be found in the introduction of step a3 and step a4:
[0050] a3. Sort the original knowledge retrieval results in descending order of similarity to obtain a similarity sorting result.
[0051] The original knowledge search result set includes multiple original knowledge search results. There are many ways to determine the relevance between each original knowledge search result and the knowledge request in step a3, which can be as follows:
[0052] In some embodiments, determining the similarity between each original knowledge retrieval result and the knowledge request includes: for each original knowledge retrieval result, based on the similarity between the query vector corresponding to the knowledge request when performing knowledge retrieval in step a2 and the storage vector corresponding to each knowledge in the knowledge base, determining the similarity between the storage vector corresponding to the original knowledge retrieval result and the query vector corresponding to the knowledge request. The similarity here can be understood as the degree of matching between the original knowledge retrieval result and the knowledge request.
[0053] In some embodiments, determining the similarity between each original knowledge retrieval result and the knowledge request includes: using a similarity prediction model to determine the similarity between each candidate knowledge retrieval result and the knowledge request.
[0054] The similarity prediction model is a neural network model obtained by training a preset similarity prediction network using a knowledge training sample set (a plurality of knowledge samples and their corresponding knowledge requests) and their corresponding real similarities. The preset similarity prediction network may be Resnet or the like.
[0055] After the similarity prediction model is trained, each original knowledge retrieval result and the corresponding knowledge request can be input into the similarity prediction model to obtain the similarity between each original knowledge retrieval result and the knowledge request.
[0056] a4. The original knowledge retrieval results corresponding to the similarities whose similarity difference between two adjacent similarities in the similarity sorting result is less than a preset difference are deleted from the original knowledge retrieval result set, thereby obtaining multiple candidate knowledge retrieval results.
[0057] Through steps a1 to a4, the original knowledge retrieval result set obtained from the preliminary retrieval can be deduplicated, thereby improving the knowledge breadth of the knowledge retrieval results received by the large model, and further improving the accuracy and rationality of the response content generated by the large model.
[0058] Continue reading Figure 1 After step S101, the method for processing knowledge data of the embodiment of the present application further includes step S102.
[0059] S102: Determine a target knowledge retrieval result for responding to the knowledge request from the plurality of candidate knowledge retrieval results based on the confidences of the plurality of candidate knowledge retrieval results and the relevance between the plurality of candidate knowledge retrieval results and the knowledge request.
[0060] The confidence level is determined based on the correctness evaluation results of the knowledge in the historical knowledge retrieval process. The confidence level of each of the multiple candidate knowledge retrieval results represents the degree of closeness between each of the multiple candidate knowledge retrieval results and the correct knowledge.
[0061] The relevance represents the relevance between each of the multiple candidate knowledge retrieval results and the knowledge request. The greater the relevance between the two, the greater the relevance between the candidate knowledge retrieval results and the knowledge required by the knowledge request; conversely, the smaller the relevance between the two, the smaller the relevance between the candidate knowledge retrieval and the knowledge required by the knowledge request.
[0062] The relevance may include a relevance tag, which indicates whether the candidate knowledge retrieval result is relevant to the knowledge request.
[0063] The specific implementation of step S102 is introduced below in conjunction with the accompanying drawings:
[0064] Figure 2 The following is a flow chart of determining target knowledge retrieval results provided in an embodiment of the present application. Figure 2 As shown, step S102 includes step S201 and step S202:
[0065] S201 . Determine, among multiple candidate knowledge retrieval results, candidate knowledge retrieval results whose relevance tags with the knowledge request are preset relevance tags as initial knowledge retrieval results.
[0066] Before step S201, it is necessary to determine the relevance tags between each of the plurality of candidate knowledge retrieval results and the knowledge request. There are many ways to implement the determination of the relevance tags between each of the plurality of candidate knowledge retrieval results and the knowledge request, which may be as follows:
[0067] In some embodiments, the relevance labels between each of the multiple candidate knowledge retrieval results and the knowledge request can be determined by the big model. Specifically, the multiple candidate knowledge retrieval results and prompt information (prompt) can be input into the big model so that the big model can judge the relevance labels between each of the multiple candidate knowledge retrieval results and the knowledge request, and output the relevance labels between each of the multiple candidate knowledge retrieval results and the knowledge request.
[0068] The relevance tag may include a first tag and a second tag. The first tag indicates that the candidate knowledge retrieval result is relevant to the knowledge request, and the second tag indicates that the candidate knowledge retrieval result is irrelevant to the knowledge request. For example, the first tag may be represented by a value of "1", and the second tag may be represented by a value of "0".
[0069] For example, the prompt can be set to "You are now a conflict detection assistant. You are now given a user query and multiple candidate knowledge retrieval results. Please judge the correlation between the query and each of the multiple candidate knowledge retrieval results. If they are correlated, label them "1", otherwise label them "0". Then for each candidate knowledge retrieval result in the multiple candidate knowledge retrieval results, if the candidate knowledge retrieval result is correlated with the knowledge request, the big model will label the candidate knowledge retrieval result with a correlation label of "1"; if the candidate knowledge retrieval result is not correlated with the knowledge request, the big model will label the candidate knowledge retrieval result with a correlation label of "0".
[0070] In some embodiments, a relevance label prediction model may also be used to determine the relevance labels between each of the plurality of candidate knowledge retrieval results and the knowledge request. For example, a preset relevance prediction network is trained using a knowledge training sample set (a plurality of knowledge samples and their corresponding knowledge requests) and their corresponding real relevance labels to obtain a relevance label prediction model. The preset relevance prediction network may be a classification network.
[0071] After the relevance label prediction model is trained, multiple candidate knowledge retrieval results and corresponding knowledge requests can be input into the relevance label prediction model to obtain the relevance classification results between each of the multiple candidate knowledge retrieval results and the knowledge request. The relevance classification result is the relevance label, which indicates whether each of the multiple candidate knowledge retrieval results is relevant to the knowledge request.
[0072] After obtaining the relevance tags between each of the plurality of candidate knowledge retrieval results and the knowledge request, the candidate knowledge retrieval result whose relevance tag between the plurality of candidate knowledge retrieval results and the knowledge request is a preset relevance tag can be selected as the initial knowledge retrieval result. The preset relevance tag can be the first tag.
[0073] Continue reading Figure 2 , after step S201, step S202 may also be included.
[0074] S202: Based on the confidence of the initial knowledge retrieval result, determine a target knowledge retrieval result for responding to the knowledge request from the initial knowledge retrieval result.
[0075] Among them, the confidence of the initial knowledge retrieval result represents the degree of closeness between the initial knowledge retrieval result and the correct knowledge. The higher the confidence, the closer the initial knowledge retrieval result is to the correct knowledge, that is, the closer the initial knowledge retrieval result is to the correct knowledge; conversely, the lower the confidence, the lower the degree of closeness between the initial knowledge retrieval result and the correct knowledge, that is, the further the initial knowledge retrieval result is from the correct knowledge. Therefore, based on the confidence of the initial knowledge retrieval result, the target knowledge retrieval result used to respond to the knowledge request can be determined from the initial knowledge retrieval results to improve the accuracy and rationality of the reply content.
[0076] The number of initial knowledge search results may also affect the determination of the target knowledge search results. Specifically, if the number of initial knowledge search results is a preset number, the initial knowledge search results are directly used as the target knowledge search results.
[0077] If the number of initial knowledge retrieval results is greater than the preset number, it is necessary to further select the initial knowledge retrieval results with higher accuracy from the initial knowledge retrieval results as the target knowledge retrieval results to assist the large model in generating more accurate and more reasonable response content. The preset number is 1.
[0078] In some implementations, step S202 specifically includes steps b1 to b3:
[0079] b1. If the number of initial knowledge retrieval results is greater than a preset number, the initial knowledge retrieval results are sorted in descending order of confidence to obtain a confidence sorting result.
[0080] During the sorting process, it may be possible that the confidence of two initial knowledge retrieval results is the same. In view of this situation, in order to provide a more accurate and reasonable knowledge retrieval result for the large model. This embodiment may also include the following steps: if the confidence of any two initial knowledge retrieval results during the sorting process is the same, the two initial knowledge retrieval results are sorted in descending order according to the similarity between them and the knowledge request to obtain a confidence sorting result.
[0081] The similarity between the two initial knowledge search results and the knowledge request is the similarity determined in step a3. For details, please refer to the introduction of step a3, which will not be repeated here.
[0082] b2. For the two initial knowledge retrieval results ranked highest in the confidence sorting results, if there is a knowledge conflict between the two initial knowledge retrieval results, then when the difference in confidence between the two initial knowledge retrieval results is less than a preset difference, the two initial knowledge retrieval results are determined as target knowledge retrieval results; or, when the difference in confidence between the two initial knowledge retrieval results is greater than or equal to the preset difference, the initial knowledge retrieval result with higher confidence among the two initial knowledge retrieval results is determined as the target knowledge retrieval result.
[0083] Before step b2, it is necessary to determine whether there is a knowledge conflict between the two initial knowledge retrieval results ranked higher in the confidence ranking results. Specifically, the large model can still be used to perform conflict detection on the two initial knowledge retrieval results to determine whether there is a knowledge conflict between the two. For example, the prompt can be set to "You are now a conflict detection assistant. Now you are given two initial knowledge retrieval results. Please determine whether there is a conflict between the two initial knowledge retrieval results."
[0084] Among them, knowledge conflict refers to the disagreement between different knowledge on the same question. For example, for the question "What is the height of Mount Everest?", there may be knowledge A that is "The height of Mount Everest is 8,844 meters" and knowledge B that is "The height of Mount Everest is 8,848 meters". Then knowledge A and knowledge B are in knowledge conflict.
[0085] In addition, due to the diversity of textual expressions, knowledge conflicts may also occur. For example, for the question "What is the height of Mount Everest?", there may be knowledge C that is "The height of Mount Everest is 8,844 meters" and knowledge D that is "The height of Mount Everest is 8,844 meters". Then knowledge C and knowledge D are knowledge conflicts.
[0086] For the two initial knowledge retrieval results, the final target knowledge retrieval result may be determined based on a comparison result of a difference in confidence between the two initial knowledge retrieval results and a preset difference.
[0087] When the confidence difference between the two initial knowledge retrieval results is smaller than the preset difference, it indicates that the two initial knowledge retrieval results are relatively close to the correct knowledge. In order to further improve the accuracy and rationality of the reply content, the present embodiment can determine the two initial knowledge retrieval results as the target knowledge retrieval results.
[0088] When the confidence difference between the two initial knowledge retrieval results is significantly different from the preset difference, it indicates that the initial knowledge retrieval result with a higher confidence is closer to the correct knowledge. In order to further improve the accuracy and rationality of the reply content, this embodiment can determine the initial knowledge retrieval result with a higher confidence as the target knowledge retrieval result.
[0089] Continue reading Figure 1 After step S102, the knowledge data processing method of this embodiment further includes step S103:
[0090] S103. Generate reply content corresponding to the knowledge request based on the target knowledge retrieval results through the big model.
[0091] Specifically, this step is to combine the target knowledge retrieval results and the knowledge request, construct a prompt template, and input the prompt template into the big model, so as to obtain the reply content corresponding to the knowledge request.
[0092] Referring to step b2, when the initial knowledge search result with a higher confidence level among the two initial knowledge search results is determined as the target knowledge search result, step S103 is to generate a reply content corresponding to the knowledge request based on the initial knowledge search result with a higher confidence level among the two initial knowledge search results. For example, if the confidence level of "the peak height of Mount Everest is 8,844 meters" is higher, the reply content is generated based on "the peak height of Mount Everest is 8,844 meters".
[0093] In the case where both of the two initial knowledge search results are determined as target knowledge search results, step S103 is to generate reply contents corresponding to the knowledge request based on the two initial knowledge search results. For example, if "the peak height of Mount Everest is 8844 meters" and "the peak height of Mount Everest is 8848 meters" are both target knowledge search results, then reply contents are generated based on "the peak height of Mount Everest is 8844 meters" and "the peak height of Mount Everest is 8848 meters".
[0094] In order to further improve the accuracy and rationality of the reply content, after the two initial knowledge retrieval results are determined as the target knowledge retrieval results, this embodiment can also use a voting mechanism to update the confidence of the two initial knowledge retrieval results to improve the accuracy and rationality of the subsequent reply content. The voting mechanism is introduced below in conjunction with the accompanying drawings:
[0095] Figure 3 A schematic diagram of an interactive interface of a voting page provided in an embodiment of the present application. Figure 3As shown, in the case where the two initial knowledge retrieval results are determined as the target knowledge retrieval results, step S103 generates two reply contents based on the two initial knowledge retrieval results. Still taking "What is the peak height of Mount Everest?" as an example, assuming that the confidence difference between "The peak height of Mount Everest is 8,844 meters" and "The peak height of Mount Everest is 8,848 meters" is less than the preset difference, two reply contents are generated based on "The peak height of Mount Everest is 8,844 meters" and "The peak height of Mount Everest is 8,848 meters", and the two reply contents are displayed through the user's terminal device, and a voting option is set below each reply content (as shown by the finger symbol in the figure), and the user votes to decide which of the two reply contents is more reasonable.
[0096] Considering that some users may not know which reply content is more reasonable, and vote arbitrarily, thereby affecting the correctness evaluation result of knowledge, this embodiment can also set prompt text in the reply page to remind users to vote carefully.
[0097] Figure 4 A schematic diagram of another interactive interface of a voting page provided in an embodiment of the present application. Figure 4 As shown, you can set prompt text such as "Your reply will affect the correctness evaluation results of this knowledge, and then affect the accuracy of subsequent reply content. Please vote carefully. If you are not sure about the correct knowledge, you can choose not to vote" on the voting page.
[0098] After the user generates two replies for the two initial knowledge retrieval results and evaluates their correctness, the user can obtain the correctness evaluation information corresponding to each of the two initial knowledge retrieval results. The correctness evaluation information represents the degree of closeness between the initial knowledge retrieval results and the correct knowledge. Based on the correctness evaluation information corresponding to each of the two initial knowledge retrieval results, the confidence of the two initial knowledge retrieval results is updated.
[0099] In some embodiments, updating the confidences of the two initial knowledge retrieval results includes: increasing the confidence of the initial knowledge retrieval result that is closest to the correct knowledge among the two initial knowledge retrieval results.
[0100] For example, when a user votes for reply A, the confidence of the knowledge corresponding to reply A is increased by +1; when a user votes for reply B, the confidence of the knowledge corresponding to reply B is increased by +1; when a user does not vote for either reply A or reply B, the confidence of the knowledge corresponding to reply A and reply B remains unchanged.
[0101] In some embodiments, updating the confidences of the two initial knowledge retrieval results includes: lowering the confidence of the initial knowledge retrieval result that is the closest to the correct knowledge among the two initial knowledge retrieval results.
[0102] For example, when a user votes for reply A, the confidence of the knowledge corresponding to reply B is -1; when a user votes for reply B, the confidence of the knowledge corresponding to reply A is -1; when a user does not vote for either reply A or reply B, the confidence of the knowledge corresponding to reply A and reply B remains unchanged.
[0103] In addition, in order to further improve the accuracy and rationality of the reply content, the present embodiment may also record the two initial knowledge retrieval results in the background, and professionals may evaluate the correctness and modify the confidence of the two initial knowledge retrieval results at a later stage.
[0104] The storage format for each piece of knowledge in the knowledge base can be shown in the following table:
[0105] Id vector Knowledge Documents Confidence 0x1c2f3d4e [0.23,-0.4,…] The height of Mount Everest is 8848 meters 1000 0x5c6f7d8e [0.63,-0.2,…] The height of Mount Everest is 8844 meters 200
[0106] In the above table, the confidence level represents the authenticity or correctness of the knowledge document, which will change in real time according to the user's voting results or the professional's modification results. Before there is a user's voting result, the confidence level of each knowledge document can be set to 0 or set by a professional.
[0107] As more and more users vote on knowledge documents, a good cycle mechanism will be formed to continuously update the confidence of each knowledge document. Therefore, when receiving new knowledge requests from users in the future, the target knowledge retrieval results used to respond to knowledge requests can be screened based on the user's evaluation results of the correctness of knowledge in the historical knowledge retrieval process, so as to assist the large model in generating more accurate and reasonable response content and improve user experience.
[0108] The above-mentioned embodiments reduce knowledge conflicts from the real-time request stage to improve the accuracy and rationality of the large model reply content. The following describes how to reduce knowledge conflicts from the knowledge base construction stage.
[0109] Figure 5 A flowchart of another method for processing knowledge data provided in an embodiment of the present application. Figure 5 As shown, the method for processing knowledge data also includes the following steps S501 to S504:
[0110] S501: Determine the similarity between the original document to be uploaded to the knowledge base and each knowledge document in the knowledge base.
[0111] Each knowledge document in the knowledge base has a corresponding hash signature. The hash signature is a 64-bit binary number. The similarity between the original document and each knowledge document in the knowledge base can be determined by the Hamming distance between the hash signatures.
[0112] For example, when a user uploads an original document in the knowledge base, the plain text file corresponding to the original document can be obtained by text-processing the original document to be stored. Among them, the original document may include PDF documents, WORD documents, TXT files, CVS data tables, EXCEL tables, pictures, videos and other non-text format documents. The plain text file is in markdown format. The original document can be converted into a plain text file through a document processing tool. For example, for PDF files, the text content can be extracted through a PDF extractor to obtain a plain text file; for pictures and videos, OCR (Optical Character Recognition) technology can be used to identify and convert the text information therein to obtain a plain text file.
[0113] After obtaining the plain text file, the simhash algorithm can be used to sign the plain text file to obtain the hash signature corresponding to the plain text file, and the similarity between the original document and each knowledge document in the knowledge base can be determined based on the Hamming distance between the hash signature of the original document and the hash signature corresponding to each knowledge document in the knowledge base.
[0114] S502: Determine the knowledge documents whose similarity with the original document is greater than a preset similarity as similar documents corresponding to the original document.
[0115] The similar documents refer to documents that are highly similar to the original documents to be stored. The similar documents may be knowledge documents whose Hamming distance is less than a preset distance among the knowledge documents. The preset distance may be 2, 3 or 4.
[0116] After obtaining the similar document, the comparison result between the original document and the similar document may also be determined, and the comparison result may be output to assist the user in determining the indication information.
[0117] Outputting the comparison result includes: displaying the comparison result on a terminal device of the user.
[0118] The comparison results include inconsistent contents between the original document and the similar document, and the inconsistent contents are highlighted. Inconsistent contents include modified contents, deleted contents, and added contents. For example, if a user modifies a value in the original document, the modified value is inconsistent content. For another example, if a user updates a part of the data in the original document, the updated data is inconsistent content. Highlighting includes highlighting, underlining, bolding, or special font color display, etc.
[0119] Figure 6 A schematic diagram of a double-column page provided in an embodiment of the present application. Figure 6 As shown in the figure, the comparison results of the original document and the similar document can be displayed in a double-column page. In the figure, "8844" and "8848" are highlighted with underline + bold text to allow users to view the inconsistent content between the original document and the similar document, so as to facilitate users to quickly decide whether to upload the original document to the knowledge base.
[0120] S503: Obtain instruction information, where the instruction information is used to indicate whether to upload the original document to the knowledge base.
[0121] After viewing the comparison result, the user may input instruction information through the terminal device to instruct the terminal device to upload the original document to the knowledge base or not to upload the original document to the knowledge base.
[0122] S504: Based on the instruction information, the original document is put into storage.
[0123] The instruction information includes uploading the original document to the knowledge base or not uploading the original document to the knowledge base. Accordingly, the storage processing includes uploading to the knowledge base or not uploading to the knowledge base.
[0124] When the instruction information is to upload the original document to the knowledge base, the original document is stored in the knowledge base.
[0125] When the instruction information is not to upload the original document to the knowledge base, the original document is not stored in the knowledge base.
[0126] The following is an introduction to the knowledge data processing method from the overall process with the help of the accompanying drawings and examples:
[0127] Figure 7 The flowchart of the overall solution of the method for processing knowledge data provided in the embodiment of the present application is as follows. Figure 7 As shown in Figure 1, the scheme consists of two parts: knowledge building phase and real-time request phase:
[0128] (1) Knowledge construction stage
[0129] In the knowledge construction stage, if an original document uploaded by a user is received, a text processing tool is used to convert the original document into a text file to obtain a plain text file.
[0130] Afterwards, a hash signature algorithm is used to calculate a hash signature for the plain text file, and based on the hash signature of the plain text file and the hash signature of each knowledge document in the knowledge base, the correlation between the original document and each knowledge document is calculated.
[0131] For each knowledge document in each knowledge document, if the correlation between the knowledge document and the original document is less than the preset correlation, the knowledge document is determined to be similar to the original document, and the knowledge document is determined as a similar document corresponding to the original document, and a text comparison is performed between the original document and the similar document to generate a comparison result between the original document and the similar document, and the user decides whether to put the original document into the database.
[0132] If the relevance between the knowledge document and the original document is greater than or equal to a preset relevance, it is determined that the knowledge document is not similar to the original document, and the original document is stored in the knowledge base.
[0133] When storing the original document in the knowledge base, the original document needs to be divided into multiple chapters, so as to obtain the chapter library corresponding to the original document.
[0134] (2) Real-time request stage In the real-time request stage, if a user's knowledge request is received, the knowledge request is vectorized to obtain the vector representation corresponding to the knowledge request. After that, the vector representation is used to perform knowledge retrieval in the knowledge base to obtain the knowledge recall results sorted from high to low according to the similarity score.
[0135] After obtaining the knowledge recall results, the knowledge conflict module can also be used to perform conflict detection on the knowledge recall results and record the conflicting knowledge. Subsequently, professionals will use the manual review platform to review and reset the confidence of the two conflicting knowledge. At the same time, based on the conflicting knowledge, responses and feedback are generated and returned to the user, who votes on the conflicting knowledge, and the confidence of the conflicting knowledge in the chapter library is updated according to the voting results.
[0136] Exemplary Devices
[0137] Corresponding to the above-mentioned method for processing knowledge data, an embodiment of the present application also provides a device for processing knowledge data. Figure 8 Schematic diagram of a knowledge data processing device provided in an embodiment of the present application. Figure 8 As shown, the knowledge data processing device provided in the embodiment of the present application includes: an acquisition unit 801, used to acquire multiple candidate knowledge retrieval results related to the user's knowledge request from the knowledge base; a determination unit 802, used to determine a target knowledge retrieval result for responding to the knowledge request from the multiple candidate knowledge retrieval results based on the confidence of the multiple candidate knowledge retrieval results and the correlation between each of the multiple candidate knowledge retrieval results and the knowledge request; the confidence is determined according to the correctness evaluation result of the knowledge in the historical knowledge retrieval process; a generation unit 803, used to generate the reply content corresponding to the knowledge request based on the target knowledge retrieval result through a large model.
[0138] Optionally, the determination unit 802 determines a target knowledge retrieval result for responding to the knowledge request from the multiple candidate knowledge retrieval results based on the confidence of the multiple candidate knowledge retrieval results and the relevance between each of the multiple candidate knowledge retrieval results and the knowledge request, including: determining, as an initial knowledge retrieval result, a candidate knowledge retrieval result whose relevance label with the knowledge request is a preset relevance label among the multiple candidate knowledge retrieval results; and determining, based on the confidence of the initial knowledge retrieval result, a target knowledge retrieval result for responding to the knowledge request from the initial knowledge retrieval results.
[0139] Optionally, the determination unit 802 determines a target knowledge retrieval result for responding to the knowledge request from the initial knowledge retrieval results based on the confidence of the initial knowledge retrieval results, including: sorting the initial knowledge retrieval results in order of confidence from high to low to obtain a confidence sorting result; for the two initial knowledge retrieval results ranked highest in the confidence sorting result, if there is a knowledge conflict between the two initial knowledge retrieval results, then when the difference in confidence between the two initial knowledge retrieval results is less than a preset difference, the two initial knowledge retrieval results are determined as the target knowledge retrieval results; or, when the difference in confidence between the two initial knowledge retrieval results is greater than or equal to a preset difference, the initial knowledge retrieval result with higher confidence among the two initial knowledge retrieval results is determined as the target knowledge retrieval result.
[0140] Optionally, the determination unit 802 sorts the initial knowledge retrieval results in order of confidence from high to low to obtain a sorting result, including: if any two initial knowledge retrieval results have the same confidence, then the two initial knowledge retrieval results are sorted in order of relevance to the knowledge request from large to small to obtain the confidence sorting result.
[0141] Optionally, the knowledge base includes multiple knowledge documents, the initial knowledge retrieval result includes the knowledge document, and the device also includes: an updating unit 804, used to obtain correctness evaluation information corresponding to each of the two initial knowledge retrieval results, the correctness evaluation information characterizing the degree of closeness between the initial knowledge retrieval result and the correct knowledge; and based on the correctness evaluation information corresponding to each of the two initial knowledge retrieval results, updating the confidence of the two initial knowledge retrieval results.
[0142] Optionally, the device also includes: a warehousing unit 805, which is used to perform the following steps: determine the similarity between the original document to be uploaded to the knowledge base and each knowledge document in the knowledge base; determine the knowledge documents in the knowledge documents whose similarity with the original document is greater than a preset similarity as similar documents corresponding to the original document; obtain indication information, the indication information is used to indicate whether to upload the original document to the knowledge base; and based on the indication information, perform warehousing processing on the original document.
[0143] Optionally, the determination unit 802 is further used to: determine a comparison result between the original document and the similar document; and output the comparison result, wherein the comparison result is used to assist the user in determining the indication information.
[0144] Optionally, the comparison result includes inconsistent contents between the original document and the similar document, and the inconsistent contents are highlighted.
[0145] The knowledge data processing device provided in this embodiment belongs to the same application concept as the knowledge data processing method provided in the above embodiments of this application, and can execute the knowledge data processing method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the knowledge data processing method. For technical details not fully described in this embodiment, please refer to the specific processing content of the knowledge data processing method provided in the above embodiments of this application, and will not be repeated here.
[0146] The functions implemented by the above acquisition unit 801, determination unit 802, generation unit 803, update unit 804 and storage unit 805 can be implemented by the same or different processors respectively, and the embodiments of the present application are not limited thereto.
[0147] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.
[0148] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.
[0149] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.
[0150] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.
[0151] Exemplary Electronic Devices
[0152] The present application embodiment provides an electronic device, see Fig. 9 As shown, the electronic device includes:
[0153] Memory 200 and processor 210;
[0154] The memory 200 is connected to the processor 210 and is used to store programs;
[0155] The processor 210 is used to implement the knowledge data processing method disclosed in any of the above embodiments by running the program stored in the memory 200.
[0156] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .
[0157] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.
[0158] A bus may include a pathway that transfers information between components of a computer system.
[0159] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0160] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.
[0161] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.
[0162] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.
[0163] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.
[0164] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.
[0165] The processor 210 executes the program stored in the memory 200 and calls other devices, and can be used to implement each step of any knowledge data processing method provided in the above embodiments of the present application.
[0166] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the knowledge data processing method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the embodiment introduction of the above-mentioned knowledge data processing method.
[0167] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the knowledge data processing method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the embodiment introduction of the above-mentioned knowledge data processing method.
[0168] Exemplary computer program products and storage media
[0169] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method for processing knowledge data according to various embodiments of the present application described in any of the above embodiments of this specification.
[0170] The computer program product may be written in any combination of one or more programming languages to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0171] In addition, the embodiment of the present application may also be a storage medium on which a computer program is stored. The computer program is executed by a processor to execute the steps of the method for processing knowledge data according to various embodiments of the present application described in any of the above embodiments of this specification, and specifically the following steps may be implemented:
[0172] S101. Acquire multiple candidate knowledge retrieval results related to the user's knowledge request from a knowledge base.
[0173] S102. Based on the confidence of the multiple candidate knowledge retrieval results and the relevance between each of the multiple candidate knowledge retrieval results and the knowledge request, determine the target knowledge retrieval result for responding to the knowledge request from the multiple candidate knowledge retrieval results; the confidence is determined according to the correctness evaluation result of the knowledge in the historical knowledge retrieval process.
[0174] S103: Generate reply content corresponding to the knowledge request based on the target knowledge retrieval result through the large model.
[0175] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0176] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0177] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.
[0178] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.
[0179] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.
[0180] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0181] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.
[0182] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0183] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0184] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0185] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for processing knowledge data, It is characterized in that include: Acquire multiple candidate knowledge retrieval results related to the user's knowledge request from the knowledge base; Determining a target knowledge retrieval result for responding to the knowledge request from the multiple candidate knowledge retrieval results based on the confidences of the multiple candidate knowledge retrieval results and the relevance between each of the multiple candidate knowledge retrieval results and the knowledge request; The confidence level is determined based on the evaluation results of the correctness of knowledge during the historical knowledge retrieval process; Generate reply content corresponding to the knowledge request based on the target knowledge retrieval result through a large model.
2. The method according to claim 1, It is characterized in that The step of determining a target knowledge retrieval result for responding to the knowledge request from the plurality of candidate knowledge retrieval results based on the confidences of the plurality of candidate knowledge retrieval results and the relevance between the plurality of candidate knowledge retrieval results and the knowledge request comprises: Determine, among the plurality of candidate knowledge retrieval results, a candidate knowledge retrieval result whose relevance tag with the knowledge request is a preset relevance tag as an initial knowledge retrieval result; Based on the confidence of the initial knowledge retrieval result, a target knowledge retrieval result for responding to the knowledge request is determined from the initial knowledge retrieval result.
3. The method according to claim 2, It is characterized in that The step of determining a target knowledge retrieval result for responding to the knowledge request from the initial knowledge retrieval result based on the confidence of the initial knowledge retrieval result comprises: If the number of initial knowledge retrieval results is greater than a preset number, the initial knowledge retrieval results are sorted in descending order of confidence to obtain a confidence sorting result; For the two initial knowledge retrieval results ranked the highest in the confidence ranking results, if there is a knowledge conflict between the two initial knowledge retrieval results, then when the difference in confidence between the two initial knowledge retrieval results is less than a preset difference, the two initial knowledge retrieval results are determined as target knowledge retrieval results; or, When the difference between the confidences of the two initial knowledge retrieval results is greater than or equal to a preset difference, the initial knowledge retrieval result with a higher confidence among the two initial knowledge retrieval results is determined as the target knowledge retrieval result.
4. The method according to claim 3, It is characterized in that The initial knowledge search results are sorted in descending order of confidence to obtain sorting results, including: If the confidences of any two initial knowledge retrieval results are the same, the two initial knowledge retrieval results are sorted in descending order of relevance to the knowledge request to obtain the confidence sorting result.
5. The method according to claim 3, It is characterized in that The knowledge base includes a plurality of knowledge documents, the initial knowledge retrieval result includes the knowledge document, and when the difference between the confidences of two initial knowledge retrieval results is less than a preset difference, after the two initial knowledge retrieval results are determined as target knowledge retrieval results, the method further includes: Obtaining correctness evaluation information corresponding to each of the two initial knowledge retrieval results, wherein the correctness evaluation information represents the degree of closeness between the initial knowledge retrieval results and the correct knowledge; Based on the correctness evaluation information corresponding to each of the two initial knowledge retrieval results, the confidence levels of the two initial knowledge retrieval results are updated.
6. The method according to any one of claims 1 to 5, It is characterized in that Before obtaining a plurality of candidate knowledge retrieval results related to the user's knowledge request from the knowledge base, the method further includes: Determining the similarity between the original document to be uploaded to the knowledge base and each knowledge document in the knowledge base; Determine the knowledge documents whose similarity with the original document is greater than a preset similarity as similar documents corresponding to the original document; Acquire instruction information, where the instruction information is used to indicate whether to upload the original document to the knowledge base; Based on the indication information, the original document is put into storage.
7. The method according to claim 6, It is characterized in that After determining the knowledge documents whose similarity with the original document is greater than a preset similarity as similar documents corresponding to the original document, the method further includes: Determining a comparison result between the original document and the similar document; The comparison result is output, where the comparison result is used to assist the user in determining the indication information.
8. The method according to claim 7, It is characterized in that The comparison result includes inconsistent contents between the original document and the similar document, and the inconsistent contents are highlighted.
9. A knowledge data processing device, It is characterized in that include: An acquisition unit, used for acquiring a plurality of candidate knowledge retrieval results related to the user's knowledge request from the knowledge base; a determination unit, configured to determine a target knowledge retrieval result for responding to the knowledge request from the plurality of candidate knowledge retrieval results based on the confidences of the plurality of candidate knowledge retrieval results and the relevance between each of the plurality of candidate knowledge retrieval results and the knowledge request; The confidence level is determined based on the evaluation results of the correctness of knowledge during the historical knowledge retrieval process; A generating unit is used to generate reply content corresponding to the knowledge request based on the target knowledge retrieval result through a large model.
10. An electronic device, It is characterized in that including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the knowledge data processing method according to any one of claims 1 to 8 by running the program in the memory.
11. A storage medium, It is characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the method for processing knowledge data according to any one of claims 1 to 8 is implemented.
12. A computer program product, It is characterized in that The method comprises computer program instructions, which, when executed by a processor, enable the processor to implement the knowledge data processing method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Knowledge retrieval method, device and apparatus and computer readable storage medium
CN112445904A
Medical question and answer method and system based on large language model and storage medium
CN118152533A
Cited By
Retrieval-augmented generation method and apparatus, computer program product, and electronic device
WO2026108135A1
