Knowledge graph retrieval enhancement method and system based on large model hypothesis knowledge enhancement

Abstract

The invention discloses a knowledge graph retrieval enhancement method and system based on large model hypothesis knowledge enhancement, and belongs to the technical field of artificial intelligence and knowledge graphs. In order to solve the problem that accuracy and interpretability are insufficient when complex medical queries are processed, technical means such as hypothesis output generation, named entity recognition, knowledge graph inference chain search and granularity perception rearrangement are mainly adopted. According to the method, query insufficiency can be made up by generating hypothesis output, the relevance and logic consistency of retrieval are improved in combination with a knowledge graph reasoning chain, noise is reduced through refined rearrangement, and the retrieval efficiency, answer accuracy and robustness of medical query are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical fields of artificial intelligence and knowledge graphs, and specifically relates to a knowledge graph retrieval enhancement method and system based on large model assumed knowledge enhancement. Background Art

[0002] Knowledge Graph Retrieval Augmentation (KGRAG) refers to improving the accuracy and consistency of generated content by retrieving relevant structured information from external information sources such as knowledge graphs during the content generation process, capturing complex relationships between entities, so as to provide more accurate and logical knowledge support when generating answers. This retrieval augmentation method has currently been widely applied to various large model question answering tasks. However, the current knowledge graph retrieval augmentation models only simply retrieve based on the user query (Query), ignoring richer semantic information, resulting in the inability of existing retrieval augmentation methods to retrieve relevant documents and bringing a large amount of noise, which instead harms the downstream question answering tasks.

[0003] Existing graph retrieval enhancement methods mainly retrieve user queries to query corresponding triples in the knowledge graph (see Priyanka Sen, Sandeep Mavadia, and Amir Saffari. 2023. Knowledge Graph-augmented Language Models for Complex Question Answering. Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE) (2023).) and Karthik Soman, Peter W Rose, John H Morris, Rabia E Akbas, Brett Smith, Braian Peetoom, Catalina Villouta-Reyes, Gabriel Cerono, Yongmei Shi, Angela Rizk-Jackson, et al. 2023. Biomedical knowledge graph-enhanced prompt generation for large language models. arXiv preprint arXiv:2311.17330 (2023).), and inject the retrieved triples into the prompt as knowledge to enhance downstream task question answering.In addition, some studies have proposed many processing methods for retrieval enhancement: The query rewriting model (see Hiteshwar Kumar Azad and Akshay Deepak. 2019. Query expansion techniques for information retrieval: A survey. Information Processing & Management (Sept. 2019), 1698–1735.) enriches the user query by leveraging large language models during retrieval. There are also some models that utilize the idea of chain of thought. CoN (Chain-of-Note) (see Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, and Dong Yu. 2023. Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models. arXiv:2311.09210 [cs.CL].) and CoK (Chain-of-Knowledge) (see Xingxuan Li, Ruochen Zhao, Yew Ken Chia, Bosheng Ding, Shafiq Joty, Soujanya Poria, and Lidong Bing. 2023. Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources. arXiv:2305.13269 [cs.CL].) utilize the idea of interacting with large models multiple times, continuously retrieving and answering to obtain more accurate responses.

[0004] In the pre-retrieval stage, existing methods are difficult to effectively parse the user's intention, resulting in low-quality retrieved knowledge that contains redundant information and noise. Methods like CoN and CoK that enhance the reasoning ability through multiple rounds of interaction bring high time costs and cumulative errors. In the post-retrieval stage, existing methods are difficult to balance the relevance and diversity of knowledge, often resulting in single or repeated results, reducing the effectiveness of retrieval-augmented generation and preventing the full utilization of beneficial auxiliary information. Summary of the Invention

[0005] The object of the present invention is to solve the problems of insufficient accuracy and interpretability in dealing with complex medical queries. By enriching user queries with the hypothetical outputs of large language models (LLMs), and then performing knowledge graph retrieval, the quality of knowledge retrieval and answers is improved. Moreover, finer-grained re-ranking is utilized to reduce retrieval noise and enhance the accuracy and robustness of downstream question-answering tasks.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A method for enhancing knowledge graph retrieval based on large model hypothesis knowledge, comprising the following steps:

[0008] 1) Based on the user query, use a large language model to generate a hypothetical output;

[0009] 2) Extract medical entities from the user query and the hypothetical output;

[0010] 3) According to the extracted medical entities, search for inference chains in the knowledge graph;

[0011] 4) Segment the user query and the hypothetical output, and perform importance screening and re-ranking on the searched inference chains according to the segmentation results;

[0012] 5) Integrate the user query with the re-ranked inference chains and input them into a large language model to generate the final answer.

[0013] Further, in step 2), the W2NER model is used to identify medical entities, and the W2NER model is pre-trained with a medical NER dataset.

[0014] Further, the steps of using a dense vector retrieval method to search for inference chains in the knowledge graph in step 3) include:

[0015] Use an encoding model to encode medical entities and entities in the knowledge graph to generate embedding vectors;

[0016] Perform vector alignment on the embedding vectors of medical entities and entities in the knowledge graph;

[0017] The inference chains are composed of entities in the aligned knowledge graph.

[0018] Further, the method for performing vector alignment on the embedding vectors of medical entities and entities in the knowledge graph in step 3) is:

[0019] Calculate the inner product similarity of the embedding vectors of medical entities and entities in the knowledge graph;

[0020] Select the entity in the knowledge graph with the highest inner product similarity and exceeding the preset threshold.

[0021] Further, in step 3), the encoding model selects a GTE encoder, which is pre-trained with weak supervision on text pairs in a large-scale dataset and fine-tuned by contrastive learning with manually annotated data.

[0022] Further, the inference chains searched in the knowledge graph in step 3) include path chains, common ancestor chains, and co-occurrence chains.

[0023] Further, the steps of segmenting the user query and the hypothetical output in step 4) include: removing stop words from the user query and the hypothetical output, and then using a chunking method to perform segmentation processing according to the selected window size and overlap size to obtain a set of segmented segments.

[0024] Further, in step 4), the pre-fine-tuned and trained re-ranking model is used to perform importance screening and re-ranking on the searched inference chains. The steps include: mapping the segmentation results and the inference chain text to a low-dimensional dense vector space, and then sorting all the searched inference chains according to importance, and selecting the top K re-ranked inference chains.

[0025] Further, in step 4), the BAAI / bge-reranker-base model is selected as the re-ranking model.

[0026] A knowledge graph retrieval enhancement system based on large model hypothesis knowledge enhancement includes the following steps:

[0027] A hypothetical output module, responsible for generating a hypothetical output based on the user query using a large language model;

[0028] A named entity recognition module, responsible for extracting medical entities from the user query and the hypothetical output;

[0029] A knowledge graph retrieval module, responsible for searching for inference chains in the knowledge graph according to the extracted medical entities;

[0030] A noise knowledge filtering module, responsible for segmenting the user query and the hypothetical output, and performing importance screening and re-ranking on the searched inference chains according to the segmentation results;

[0031] An LLM reader, responsible for integrating the user query with the re-ranked inference chains and inputting them into the large language model to generate a final answer.

[0032] The beneficial effects obtained by the present invention are as follows:

[0033] 1. By introducing the Hypothetical Knowledge Graph Enhancement (HyKGE) framework, the present invention generates a hypothetical output through a large language model and combines it with the knowledge graph to search for inference chains, improving the logical consistency of knowledge and ensuring the interpretability and reference value of medical query answers.

[0034] 2. The present invention effectively filters noise through a refined rearrangement method, effectively reducing the interaction cost and cumulative errors, improving the relevance and diversity of knowledge, and being able to improve the efficiency, accuracy, and robustness of performance knowledge retrieval and generation by the medical large language model when processing complex medical queries.

[0035] 3. The present invention optimizes the process of medical entity extraction and hypothesis reasoning, further enhancing the precision and effectiveness of knowledge graph retrieval, and being able to make up for the disadvantages of high resource consumption and poor content quality in existing methods while ensuring efficient retrieval, effectively improving the quality and efficiency of queries. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flowchart of the knowledge graph retrieval enhancement method based on large model hypothesis knowledge enhancement of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] In order to make the technical features, advantages, or technical effects in the above technical solutions of the present invention more obvious and understandable, the following will be described in detail in conjunction with embodiments.

[0038] An embodiment of the present invention proposes a knowledge graph retrieval enhancement method based on large model hypothesis knowledge enhancement, as Figure 1 shown, mainly including the following four parts: pre-retrieval stage, knowledge graph retrieval stage, post-retrieval stage, and retrieval answer stage.

[0039] 1. Pre-retrieval stage:

[0040] This stage is responsible for execution by the Hypothetical Output Module (HOM) and the Named Entity Recognition Module (NER).

[0041] The Hypothetical Output Module generates a hypothetical output using a large language model according to the user query. By providing an inference direction, this module allows the large language model to answer questions exploratively, expands the semantic space by exploring possible answers, makes up for the incompleteness of the user query, and enhances the preliminary preparation for retrieval.

[0042] An example of the Prompt format of the Hypothetical Output Module is as follows:

[0043]

[0044] The Named Entity Recognition Module extracts medical entities from the user query and the hypothetical output using the NER method, providing an anchor point for knowledge graph retrieval and ensuring the precision and effectiveness of retrieval. Specifically, the Named Entity Recognition Module uses the W2NER model of the NER method to identify medical entities. The present invention prepares a medical NER dataset in advance and trains a proprietary medical W2NER model.

[0045] 2. Knowledge Graph Retrieval Phase:

[0046] This phase is executed by the knowledge graph retrieval module. Using the extracted medical entities as anchors, it searches for relevant inference chains in the knowledge graph (KG), which can obtain more relevant background information, provide high-quality knowledge with logical consistency, and improve the reliability and accuracy of retrieval results. In this phase, the Dense Retrieval method is used to search for key entities from the knowledge graph. By aligning entity vectors, medical entities are linked to the knowledge graph. The specific steps are as follows:

[0047] First, use an encoding model enc(·) to encode the medical entity u i and the entity set ε in the knowledge graph (consisting of entities e j ), generating embedding vectors enc(u i ), enc(e j ). In this embodiment, the encoding model specifically adopts the GTE encoder, which is currently the optimal retrieval model in the field of text vector embedding. The GTE encoder adopts a two-stage training process:

[0048] In the first stage, a large-scale dataset is used for weakly supervised training through text pairs;

[0049] In the second stage, contrastive learning fine-tuning is performed on high-quality manually annotated data to improve the model performance.

[0050] Then, calculate the inner product similarity between the embedding vectors enc(u i ), enc(e j ). The entity with the highest similarity and exceeding the preset threshold δ∈[0,1] is the matching entity, achieving vector alignment. This process is expressed as follows:

[0051]

[0052]

[0053] <enc(u i ), enc(e j )> represents the inner product between the extracted medical entity and the entity in the knowledge graph, which is used to achieve graph entity linking. Finally, the set of matching entities ε Q .

[0054] Finally, based on the matched entities, retrieve the inference chains within k hops and integrate this knowledge along with the descriptions of the head and tail entities. Considering multiple knowledge graph retrieval methods, the choice to utilize the inference chains between entities is mainly based on the following reasons: (1) Inference chains provide richer logical knowledge. Compared to only providing entities and their descriptions, inference chains can offer better knowledge digestion capabilities for large language models. (2) Inference chains help the LLM Reader understand the relationships between different entities, thus alleviating issues such as hallucination and errors. (3) As an efficient pruning mechanism, inference chains can filter out noise more effectively than subgraphs and save token resources. Therefore, three possible inference chains were considered from a medical perspective:

[0055] 1. Path (head - tail): Represents the triggering and causal relationships between diseases and symptoms, denoted as path ij 。

[0056] 2. Co - ancestor chain (tail - tail): Represents similar physiological or environmental factors for better analogical diagnosis, denoted as chainCA ij 。

[0057] 3. Co - occurrence chain (head - head): Used to better capture pathological characteristics and the evolution of diseases, denoted as chainCO ij 。

[0058] Overall, the set of filtered inference chains is denoted as RC.

[0059]

[0060] 3. Post - retrieval stage:

[0061] This stage is responsible for being executed by the noise knowledge filtering module. In the previous stage, a large number of inference chains can be collected through retrieval. However, due to the existence of a large amount of noise and the shortage of token resources, it is necessary to re-rank and filter the retrieved inference chains to eliminate irrelevant noise knowledge, retain relevant and diverse knowledge, and improve the utilization efficiency of token resources.

[0062] Due to the different knowledge densities between the user query and the inference chain, traditional query-based only re-ranking methods may filter out valuable knowledge obtained from the hypothetical output module, resulting in repetitive and monotonous results. Therefore, the present invention combines the user query and the hypothetical output, rather than relying only on the user query, and utilizes richer medical knowledge from the hypothetical output. Specifically, stop words are first removed from the natural language, and then it is segmented using the chunk method. This process is represented as follows:

[0063]

[0064] Among them, {C} represents the set of segmented segments, and carefully selected window sizes and overlap sizes are used during chunking; Q represents the user query, and HO represents the hypothetical output.

[0065] Then, a re-ranking model (Rerank), such as the BAAI / bge-reranker-base model, is used to screen the inference chains, trim and eliminate irrelevant noise knowledge. This model is trained through asymmetric instruction fine-tuning of a large-scale text pair, maps the text to a low-dimensional dense vector space to re-rank the topK documents. This re-ranking model, as a filter, will re-evaluate the importance of each inference chain, considering factors such as relevance, consistency, and informativeness. Specifically, this model selects the topK re-ranked inference chains from the inference chain set RC according to the set of segmented segments {C}. This process is represented as follows:

[0066] RC prune = Rerank(RC,{C}; topK)

[0067] Where, |RC prune | = topK.

[0068] 4. Retrieval and answer stage

[0069] This stage is responsible for being executed by the LLM Reader. The LLM Reader is an integrated module that can integrate the user query and the re-ranked inference chains by designing a prompt and input them into the large language model to generate the final answer. This module ensures the accuracy and interpretability of the generated answer, making the answer more valuable for reference.

[0070] An example of the prompt format of the LLM reader is as follows:

[0071]

[0072] Although the present invention has been disclosed above by way of examples, it is not intended to limit the present invention. Any appropriate modifications or equivalent replacements made by those of ordinary skill in the art to the technical solutions of the present invention shall be covered within the protection scope of the present invention. The protection scope of the present invention shall be subject to what is defined by the claims.

Claims

1. A knowledge graph retrieval enhancement method based on large model hypothesis knowledge enhancement, characterized in that: The following steps are involved: 1) Based on user queries, use a large language model to generate hypothetical outputs; 2) extracting medical entities from user queries and hypothetical outputs; 3) Search for reasoning chains in the knowledge graph based on the extracted medical entities; 4) Segment user queries and hypothetical outputs, and filter and re-rank the search reasoning chain according to the segmentation results; 5) Integrate the user query with the rearranged reasoning chain and input it into the large language model to generate the final answer.

2. The method according to claim 1, characterized in that In step 2), the W2NER model is used to identify medical entities. The W2NER model is pre-trained with a medical NER dataset.

3. The method according to claim 1, characterized in that Step 3) The steps of searching the reasoning chain from the knowledge graph using the dense vector retrieval method include: Use the encoding model to encode medical entities and entities in the knowledge graph to generate embedding vectors; Align the embedding vectors of medical entities and entities in the knowledge graph; Reasoning chains are formed by entities in the aligned knowledge graph.

4. The method according to claim 3, characterized in that The method for aligning the embedding vectors of medical entities and entities in the knowledge graph in step 3) is: Calculate the inner product similarity between the embedding vectors of medical entities and entities in the knowledge graph; Select the entity in the knowledge graph whose inner product similarity is the highest and exceeds the preset threshold.

5. The method according to claim 1, characterized in that In step 3), the encoding model uses the GTE encoder, which is pre-trained through weak supervision training of text pairs of large-scale datasets and comparative learning fine-tuning of manually annotated data.

6. The method according to claim 1, characterized in that The reasoning chains searched in the knowledge graph in step 3) include paths, common ancestor chains, and co-occurrence chains.

7. The method according to claim 1, characterized in that The step of segmenting the user query and the hypothetical output in step 4) includes: removing stop words from the user query and the hypothetical output, and then using a block method to segment them according to the selected window size and overlap size to obtain a set of segmented segments.

8. The method according to claim 1, characterized in that In step 4), a pre-fine-tuned trained re-ranking model is used to screen and re-rank the searched reasoning chains by importance. The steps include: mapping the segmentation results and the reasoning chain texts into a low-dimensional dense vector space, and then sorting all the searched reasoning chains according to importance, and selecting the top K re-ranked reasoning chains.

9. The method according to claim 8, characterized in that In step 4), the reranking model uses the BAAI / bge-reranker-base model.

10. A knowledge graph retrieval enhancement system based on large model hypothesis knowledge enhancement, implementing the method described in any one of claims 1 to 9, characterized in that: The following steps are involved: The hypothetical output module is responsible for generating hypothetical outputs based on user queries using a large language model; Named Entity Recognition module, responsible for extracting medical entities from user queries and hypothetical outputs; The knowledge graph retrieval module is responsible for searching the reasoning chain in the knowledge graph based on the extracted medical entities; The noise knowledge filtering module is responsible for segmenting user queries and hypothetical outputs, and screening and rearranging the importance of the search reasoning chain based on the segmentation results; The LLM reader is responsible for integrating the user query with the rearranged reasoning chain, inputting it into the large language model, and generating the final answer.

Images