An interpretable question and answer enhancement generation method and system based on a multi-layer knowledge graph and bidirectional retrieval

Abstract

The application discloses an explainable question and answer enhancement generation method and system based on a multi-layer knowledge graph and bidirectional retrieval, and belongs to the technical field of artificial intelligence. The application comprises: constructing a multi-layer knowledge graph, containing a private data entity layer, an industry knowledge entity layer and an authoritative term entity layer, and establishing "reference from" and "defined as" links among the three layers of entities through semantic similarity; generating a label summary for each private entity subgraph and performing hierarchical clustering to form a multi-layer label tree; in response to a user query, performing bidirectional retrieval, i.e., retrieving from a label tree root node to the most relevant subgraph from top to bottom, expanding the relevant entities in the subgraph and the industry knowledge and authoritative term entities linked with the relevant entities to generate a preliminary answer, and then refining the answer by layer-by-layer fusion of high-level label summaries from bottom to top. The application realizes traceability of the evidence of the answer and explainability of the professional terms, significantly improves the credibility and accuracy of the question and answer system, and can be widely applied to vertical fields such as law, medicine and finance.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a knowledge graph-based retrieval enhancement generation method and system, which is particularly suitable for question-answering scenarios requiring highly reliable and explainable answers, such as vertical fields like legal consultation, medical diagnosis, and financial analysis. Background Technology

[0002] With the rapid development of Large Language Models (LLMs), they have demonstrated powerful capabilities in natural language understanding and generation. However, in knowledge-intensive application scenarios, LLMs face two major bottlenecks: first, the domain knowledge system is vast and cannot be fully incorporated into the model's context window; second, practical applications have extremely high requirements for the accuracy, interpretability, and traceability of evidence in responses, and the model cannot generate fabricated content and must provide authoritative sources.

[0003] Retrieval-Augmented Generation (RAG) technology alleviates the knowledge capacity problem by introducing external knowledge bases. The classic RAG method vectorizes documents into chunks, retrieves the text chunk most similar to the question, and inputs it into an LLM (Local Level Management) to generate the answer. However, this method struggles to capture complex semantic relationships across documents and performs poorly on questions requiring the integration of information from multiple sources.

[0004] Graph Retrieval Enhanced Generation (GraphRAG) significantly improves complex reasoning capabilities by constructing knowledge graphs to enhance relational understanding. However, existing GraphRAG methods have the following drawbacks: (1) High graph construction cost: Its general design requires the construction of a large-scale graph community, which has a large computational overhead; (2) Lack of traceability of evidence: Entities are not explicitly linked to authoritative knowledge sources, and the generated answers cannot provide credible source citations and explanations of technical terms; (3) Coarse retrieval granularity: Retrieval based on graph community summaries is prone to losing detailed information, resulting in inaccurate answers; (4) Insufficient interpretability: It is impossible to show users the original evidence and terminology definitions on which the answer is based, making it difficult to meet compliance requirements.

[0005] Therefore, there is an urgent need for an enhanced generative technology that can efficiently integrate multi-source heterogeneous data and generate traceable and explainable answers. Summary of the Invention

[0006] I. Technical problems to be solved This invention aims to address the problem that existing RAG and GraphRAG technologies cannot provide traceable, interpretable, and highly reliable answers in interpretable question-answering scenarios, specifically including: (1) How to construct a multi-level knowledge graph that integrates private documents, public knowledge bases and authoritative terminology systems to achieve knowledge traceability; (2) How to achieve efficient retrieval that takes into account both global context and detailed evidence; (3) How to generate answers with clear source citations and explanations of technical terms.

[0007] II. Technical Solution This invention provides an interpretable question-answering enhancement generation method based on multi-layer knowledge graphs and bidirectional retrieval, comprising the following steps: Step 1: Construction of a multi-layered knowledge graph

[0008] 1.1 Semantic Blocking User-owned documents are segmented into independent data blocks based on semantic boundaries. A hybrid strategy is adopted: first, they are segmented by natural paragraphs, and then a large language model is used to determine whether adjacent paragraphs have the same topic. If they do, they are merged; otherwise, they are segmented. The final result is a set of semantic blocks, denoted as D, which contains multiple semantic blocks such as C1, C2, ..., Cm.

[0009] 1.2 Entity Recognition and Three-Layer Links First layer: Private entity layer

[0010] For each semantic block Ci, key entities are extracted using a large language model, denoted as E1. Each entity is represented as a quadruple: name, type, context, and block ID. The entity type is predefined based on the application domain.

[0011] Second layer: Industry knowledge entity layer Entities are extracted from an industry-wide public knowledge base and denoted as E2. A "reference from" relationship is established between E2 and the first-level entity based on semantic similarity. Specifically, the entity text is converted into a vector using an embedding model, and cosine similarity is calculated. If the similarity is greater than or equal to a preset threshold (typically 0.5), a directed edge is added, indicating that the first-level entity "references from" the second-level entity.

[0012] Third layer: Authoritative terminology entity layer Extract the entity from the industry standard terminology system and denote it as E3. Establish a "defined as" relationship with the second-level entity in the same way: if the similarity is greater than or equal to the preset threshold, add a directed edge to indicate that the second-level entity is "defined as" the third-level entity.

[0013] Through three layers of links, each private entity is ultimately associated with at least one authoritative source and a term definition, forming a triple structure: private entity, source entity, and definition entity.

[0014] 1.3 Relation Extraction Within the same semantic block, a large language model is used to identify relationships between private entities, generating relation descriptions and forming a directed graph called a meta-knowledge graph. Each meta-knowledge graph is represented as Gi, containing the set of private entities within the corresponding semantic block and the relationships between them. Example of a relation extraction prompt: "Given the following text: [text content], please identify the relationships between the entities (listed), with the output format (entity A, relation word, entity B)." Step 2: Tag-based hierarchical summary generation

[0015] 2.1 Tag System Definition A hierarchical tagging system is predefined, designed by domain experts, to cover core concepts. For example, in the legal field: first-level tags include "cause of action," "parties," and "legal clauses," while second-level tags, such as "cause of action," are further divided into "contract disputes," "tort disputes," etc.

[0016] 2.2 Single Image Tag Summary Generation For each meta-knowledge graph Gi, input its contained entities and relations into a large language model, and given a labeling system, require the large language model to output the corresponding label values ​​for the graph. The prompt template is: "Based on the following: [Graph Content Summary], fill in the value for each label according to the labeling system." This yields a structured label summary Ti for the graph.

[0017] 2.3 Hierarchical Clustering A cohesive hierarchical clustering method is used to cluster the meta-knowledge graph, generating a multi-level label tree. Initially, each graph forms its own cluster. The process is iterative: • Calculate the pairwise similarity between all classes. The similarity between classes is defined as the average similarity of the label vectors of all graphs within the class, where the label vectors are obtained by averaging the embedding vectors of the label names. • Merge the top p-th percentile class pairs with the highest similarity (p can be set to 20) to generate new classes, and use a large language model to merge the label summaries of these classes to generate high-level summaries; • Repeat until a preset number of layers (e.g., 5 layers) is reached or merging is no longer possible, eventually forming a label tree Tree_T from the bottom details to the top abstraction.

[0018] Step 3: Two-way search and answer generation 3.1 Query Tag Generation

[0019] For the user's question Q, the large language model is used to generate the corresponding tag set T_Q, with the prompt: "Please select the most relevant tag from the tag system and assign it a value based on the question: [Q]". 3.2 Top-down precise retrieval

[0020] Starting from the root node of the label tree, calculate the similarity between T_Q and the label summary of each node in the current layer (using the cosine similarity of the embedding vectors), select the most similar node to enter the next layer, until the leaf node (i.e. a certain meta-knowledge graph G_t) is reached, ensuring that the most relevant bottom layer graph is quickly located.

[0021] 3.3 Expanded Search of the Bottom Graph In the target graph G_t, a predetermined number of private entities (e.g., 50 to 100) most relevant to question Q are retrieved based on vector similarity, denoted as E_r. For each entity e belonging to E_r, its neighboring entities (including second- and third-level entities) are obtained by expanding outwards along the "refer to" and "defined as" relationships by a predetermined number of hops (e.g., 2 to 3 hops), forming an extended subgraph G_r containing E_r, its neighboring entities, and the relationships between them.

[0022] 3.4 Initial Response Generation Input the entity names and relationships from G_r into the large language model in text form, along with a question, and generate an initial answer A_0. The large language model is required to annotate the information sources and definitions of technical terms in the answer. The prompt is: "Based on the following knowledge graph: [graph content], answer the question: [Q]. Please annotate the information sources and terminology explanations in parentheses in your answer." 3.5 Bottom-up answers are concise

[0023] Starting from the current leaf node, backtrack to its parent node (the previous level of label summary), merge the parent node's summary information S_parent with the initial answer A_0, and input this into the large language model, requesting it to adjust the answer to incorporate a broader context. Repeat this process until the root node, obtaining the final answer A_final. The prompts at each step are: "Existing answer: [A_current]. Now, higher-level background information: [S_parent]. Please optimize the answer based on this information to make it more comprehensive and accurate, while preserving the original evidence." III. Beneficial Effects

[0024] Compared with the prior art, the present invention has the following significant advantages: (1) High credibility and traceability: Through the three-layer entity linking structure, the key information in each answer can be traced back to authoritative sources and professional definitions, which greatly improves the credibility of the answer; (2) Comprehensive context fusion: Bidirectional retrieval takes into account both low-level details and high-level summaries, avoiding the fragmentation problem of traditional RAG, and is more efficient than GraphRAG; (3) High interpretability: The generated answers come with clear source citations and terminology explanations, and users can verify the basis of the answers; (4) Wide industry adaptability: Tag-based summarization and hierarchical clustering can be flexibly adapted to knowledge systems in different fields. Deployment can be completed quickly simply by replacing the tag system and knowledge base. (5) Model-friendly: Significant improvements can be achieved even on lightweight models, reducing deployment costs. Attached Figure Description

[0025] Figure 1 Overall Flowchart of the Invention Figure 2 Schematic diagram of a multi-layered knowledge graph structure Figure 3 Schematic diagram of the label hierarchical clustering process Figure 4 Schematic diagram of bidirectional search process Detailed Implementation

[0026] Example 1: Legal Industry Application This embodiment uses the legal field as an example to describe in detail the implementation process of the present invention.

[0027] Hardware configuration: CPU Intel Xeon 32-core, GPU NVIDIA A100 40GB, RAM 128GB.

[0028] Software environment: Python 3.9, PyTorch 2.0, FAISS vector library, LLM using GPT-4 API.

[0029] Step 1: Data Preparation • Private Documents: 500 litigation case documents (PDF format) provided by a law firm.

[0030] • Industry knowledge base: China Laws and Regulations Database (including the Civil Code, etc.) and publicly available case precedents on the China Judgments Online website.

[0031] • Terminology system: Yuanzhao English-American Law Dictionary (electronic version).

[0032] Step 2: Three-layer graph construction • Semantic chunking: Each document is divided into paragraphs, and GPT-4 is used to determine the topic of adjacent paragraphs. After merging, an average of 3-5 semantic chunks are obtained per document.

[0033] • Entity Extraction: For each semantic block, the GPT-4 prompt "Please extract the legal entities from the following text, including parties, causes of action, legal clauses, courts, and judgments," resulting in a list of entities. For example, in the text "Company A filed a lawsuit against Company B for failing to deliver goods on time, and the court, based on Article 577 of the Civil Code, ordered Company B to pay liquidated damages," the extracted entities are: Company A (party), Company B (party), failure to deliver goods on time (cause of action), Article 577 of the Civil Code (legal clause), and liquidated damages (judgment).

[0034] • Three-layer linking: Match "Article 577 of the Civil Code" with the legal entity in the knowledge base, set the similarity threshold to 0.6, and link it to the legal entity; link the legal entity with the definition of "liability for breach of contract" in the dictionary.

[0035] • Relationship extraction: Identify relationships within the same block, such as "Company A - Files a lawsuit - Company B", "Court - Basis - Article 577 of the Civil Code", etc.

[0036] Step 3: Tag Summary • Tagging system: First-level tags: cause of action, type of party, relevant legal provisions; Second-level tags: cause of action includes contract disputes, tort disputes, etc.

[0037] • For each graph, generate a tag summary such as "Case type: Contract dispute; Party type: Enterprise; Relevant legal provisions: Article 577 of the Civil Code".

[0038] • Hierarchical clustering: Calculate the similarity of label vectors for all graph pairs, merge the top 20% of similarity pairs, and generate a high-level summary such as "Contract Breach Related Cases".

[0039] Step 4: Bidirectional Search A user asked: "If Company A sues Company B for failing to deliver goods on time, what law should apply?" • Query tag generation: GPT-4 output "Case type: Contract dispute; Problem type: Breach of contract".

[0040] • Top-down search: Match from the root node to the "Contract Dispute" category, and then match to the specific case image.

[0041] • Extended Search: Search the target graph for the 5 entities most relevant to "failure to deliver on time" (such as "delivery obligation" and "breach of contract"), extend by 2 hops to obtain legal entities and their definitions.

[0042] • Preliminary answer: "According to Article 577 of the Civil Code (source: China Laws and Regulations Database), if a party fails to perform its contractual obligations, it shall bear liability for breach of contract. Liability for breach of contract refers to the legal consequences of violating the contractual agreement (source: Yuanzhao English-American Law Dictionary)." • Bottom-up refinement: Going back to the parent node summary "Common Situations of Contract Breach", the answer is revised and supplemented with: "In similar cases, courts usually support the non-breaching party's claim for liquidated damages." The final answer contains more comprehensive information.

[0043] Performance Evaluation: On a self-built legal Q&A test set (200 questions), this method achieved an accuracy of 85.3%, a significant improvement compared to the standard RAG (68.2%) and GraphRAG (74.6%). In the human evaluation, 95% of the answers contained verifiable sources.

Claims

1. A method for enhancing the generation of interpretable question-answering based on multi-layer knowledge graphs and bidirectional retrieval, characterized in that, Includes the following steps: • Construct a multi-layered knowledge graph, which includes a private data entity layer, an industry knowledge entity layer, and an authoritative terminology entity layer. The private data entities are linked to the industry knowledge entities through semantic similarity, and the industry knowledge entities are linked to the authoritative terminology entities through semantic similarity. • Generate a label summary for each subgraph corresponding to a private data entity, and perform hierarchical clustering on all subgraphs to form a multi-layer label tree; • Responding to user queries, perform bidirectional retrieval: First, retrieve from the root node of the tag tree to the most relevant subgraph from top to bottom. Then, expand the retrieval of related entities and their linked industry knowledge entities and authoritative terminology entities in the most relevant subgraph to generate an initial answer. Finally, refine the answer by merging high-level tag summaries layer by layer from bottom to top.

2. The method according to claim 1, characterized in that, Constructing a multi-layered knowledge graph further includes: • Semantically segment private documents, with each semantic block serving as a subgraph building unit; • In each semantic block, private data entities are extracted using a large language model, and relationships between entities are identified to form a directed subgraph; • Extract industry knowledge entities from the industry knowledge base and authoritative terminology entities from the authoritative terminology database, and establish links through vector similarity calculation, with the similarity threshold preset to 0.4-0.

6.

3. The method according to claim 1, characterized in that, The generation of tag summaries further includes: using a predefined hierarchical tagging system and a large language model to generate structured tag values ​​for each subgraph.

4. The method according to claim 1, characterized in that, Hierarchical clustering employs agglomerative hierarchical clustering, specifically including: • Calculate inter-class similarity. Class similarity is defined as the average similarity of the label vectors of all subgraphs within a class. Merge the top 15%-25% of class pairs with the highest similarity; Generate a high-level tag summary for the merged new class.

5. The method according to claim 1, characterized in that, Top-down retrieval specifically includes: • Convert user queries into query tag sets; • Calculate the similarity between the query tag set and the tag summaries of each node in the current layer; • Select the most similar node to proceed to the next level, until the leaf node is reached.

6. The method according to claim 1, characterized in that, Extended search specifically includes: • In the target subgraph, retrieve the K most relevant private entities based on vector similarity, where K is between 50 and 100; • Extend outwards by L jumps along the "Refer to" and "Defined as" relationships, with L being 2-3, to obtain all relevant entities and relationships.

7. The method according to claim 1, characterized in that, Bottom-up refining specifically includes: • Backtrack from the current node to the parent node; • Merge the tag summary of the parent node with the current answer, and use a large language model to adjust and generate a new answer; Repeat until the root node.

8. An interpretable question-answering enhancement generation system based on multi-layer knowledge graph and bidirectional retrieval, characterized in that, include: • Memory, used to store executable instructions for a computer; • A processor for executing the computer-executable instructions to implement the method of any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1-7.

10. The system according to claim 8, characterized in that, Also includes: • A vector database used to store entity embedding vectors; • Large Language Model Interface, used to call the large language model to perform entity extraction, relation extraction, label generation, and answer generation tasks.

Images