Question and answer method and system based on LLM fused knowledge graph and retrieval enhancement generation
By integrating knowledge graphs and retrieval enhancement methods into a large language model, and through semantic retrieval and graph structure filtering, the accuracy of LLM in specialized scenarios is addressed, resulting in a high-precision and interpretable question-answering system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTH CHINA AGRICULTURAL UNIVERSITY
- Filing Date
- 2025-11-14
- Publication Date
- 2026-05-29
AI Technical Summary
In existing technologies, large language models (LLMs) struggle to provide accurate and reliable answers in highly specialized scenarios. The structured reasoning capabilities of knowledge graphs are difficult to integrate deeply with the dynamic knowledge acquisition capabilities generated by retrieval enhancement, resulting in insufficient accuracy of question-answering systems when dealing with complex relationship chains and multi-step logical reasoning.
By performing semantic retrieval on the question and document database, relevant documents are filtered and the local knowledge graph is expanded. Answers are generated by combining the large language model, integrating triples of sentence-level, entity-sentence, and entity-entity relationships, expanding the local knowledge graph using graph structure connection relationships, and pruning redundant edges through the graph structure filtering module to generate high-precision answers.
It enables the provision of interpretable, high-precision answers in complex question-and-answer scenarios, improving the system's response quality and credibility. By combining structured knowledge with dynamic external information, it solves the illusion problem of large language models and provides reliable knowledge services.
Smart Images
Figure CN121542378B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a question-answering method and system based on LLM-fused knowledge graph and retrieval enhancement generation. Background Technology
[0002] Large Language Models (LLMs) use deep learning techniques to understand and generate natural language, capturing subtle differences between languages and maintaining contextual coherence in complex dialogues. However, LLMs generally suffer from problems such as knowledge illusion, poor data real-time performance, and lack of domain-specific knowledge, making it difficult to provide accurate and reliable solutions in highly specialized scenarios.
[0003] Knowledge graphs, as a type of structured semantic network, effectively process, handle, and integrate complex document data, transforming it into clear triplets of "head entity-relationship-tail entity" to explicitly construct semantic relationships between entities, thereby enabling rapid knowledge response and reasoning. However, traditional question-answering methods based on knowledge graphs have limitations: on the one hand, they struggle to effectively integrate knowledge from outside the graph or from unstructured text, limiting their scalability; on the other hand, the reasoning mechanism of the graph itself has difficulty understanding multiple logical relationships.
[0004] Retrieval-Augmented Generation (RAG) is one of the advanced technical solutions in the field of artificial intelligence that has clear technical value in improving the factual accuracy and timeliness of generative models. It can utilize reliable external knowledge, combined with advanced natural language processing techniques and reinforcement learning algorithms, to achieve deep understanding and accurate answers in specific domains. However, standard RAG mainly relies on unstructured text retrieval fragments, making it difficult to fully utilize the deep relationships within knowledge fragments when solving problems involving complex relational chains and multi-step logical reasoning. Therefore, how to deeply integrate the structured reasoning capabilities of knowledge graphs with the dynamic knowledge acquisition capabilities of RAG to create a synergistic effect, and simultaneously design question-answering methods and systems based on large language models that integrate the above two technologies to overcome the shortcomings of single technologies and achieve more accurate, interpretable, and fact-consistent reasoning, has become a pressing technical challenge. Summary of the Invention
[0005] This invention overcomes the shortcomings of the prior art and provides a question-answering method and system based on LLM-fused knowledge graph and retrieval enhancement generation.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0007] The first aspect of this invention discloses a question-answering method based on LLM-fused knowledge graph and retrieval enhancement generation, comprising the following steps:
[0008] A preliminary semantic search is performed on the query and the document database. Then, based on similarity, the most relevant documents are selected from the entire document database to obtain the document set. This provides input for knowledge graph expansion;
[0009] The document set As input, the corresponding local knowledge graph is extracted from the large language model. , represented as ,in For a local entity set, It is a local relation set;
[0010] Traversing the global knowledge graph By using graph structure connections, entities that are connected to known entities but do not appear in the local knowledge graph are marked as related entities, and their relationships are added to the local knowledge graph, thereby expanding the local knowledge graph and forming a new local knowledge graph. ;
[0011] The new local knowledge graph Perform structured filtering to obtain semantic subgraphs and unique paths;
[0012] Weighted edge-wise within connected components Sort in descending order, and if the weights are the same, use the lexicographical order of the entity name as the second sorting key to ensure the determinacy and uniqueness of the sorting results; after sorting, add the text information corresponding to the nodes on the path in the order of sorting until the cumulative number of documents reaches the preset number K.
[0013] The resulting K documents are combined with the question query and input into a large language model to generate the final answer.
[0014] Furthermore, a preliminary semantic search is performed on the query and the document database, and the most relevant documents to the query are selected from the full set of documents based on similarity ranking. Specifically:
[0015] Vectorize the input question Query to obtain a vector. And through Ollam's pre-trained embedding model Encode text blocks to obtain vectors :
[0016] ,
[0017] in, The first word or text block in the query; The second word or text block in the query; The nth word or text block in the query;
[0018] Any document in the document library Similarly, vector encoding is performed to obtain... and vector With Documents Perform cosine similarity calculation to obtain vectors With Documents The semantic association strength between them is calculated using the following formula:
[0019] ,
[0020] vector With Documents The semantic association strength between documents is sorted in descending order of similarity, and the top K documents are selected as the set of relevant documents retrieved. :
[0021] ,
[0022] In the formula, This is the document ranked first after being sorted in descending order of similarity; This is the second-ranked document after being sorted in descending order of similarity. This refers to the document ranked Kth after sorting by similarity in descending order; This indicates the total number of documents in the document library.
[0023] The local knowledge graph integrates structured information at three different levels:
[0024] The first level is sentence-level relations, that is, relations between document sentences. Triples generated by extracting information ;in, The head entity represents the main body or starting point in the relationship; Representative relationship, describing the semantic relationship between the head entity and the tail entity; The tail entity represents the object or endpoint in the relationship;
[0025] The second layer is the entity-sentence relationship, that is, constructing entities. With sentences Triples of origination relationships ;in, A representative entity is a specific concept, object, or name extracted from a document; Representational relationship, specifically referring to the connection between entities and sentences; The representative sentence is a collection of documents. A specific sentence text;
[0026] The third layer is entity-entity relations, which are triples that establish relationships between different entities based on semantic reasoning. ;in, The head entity represents the main body or starting point in the relationship; Relationship, describing the semantic relationship between two entities; The tail entity represents the object or endpoint in the relationship.
[0027] Furthermore, traverse the global knowledge graph. By using graph structure connections, entities that are connected to known entities but do not appear in the local knowledge graph are marked as related entities, and their relationships are added to the local knowledge graph, thereby expanding the local knowledge graph and forming a new local knowledge graph. Specifically:
[0028] For local knowledge graphs Each entity in Find its direct neighbor entities in the global graph. :
[0029] ,
[0030] From adjacent entities Select new entities that do not exist in the local map. :
[0031] ,
[0032] Extract all relation triples connecting the old and new entities to obtain the new relation triples. :
[0033] ,
[0034] The new entity New Relationship Triad Merging into the local knowledge graph yields a new local knowledge graph. ;
[0035] In the formula, For the global entity collection; It is the set of global relations, that is, all known triples that connect global entities.
[0036] Among them, the ,in, It is the set of all new entities in the new local knowledge graph; It is the set of all new relation triples in the new local knowledge graph.
[0037] Furthermore, the new local knowledge graph Perform structured filtering to obtain semantic subgraphs and unique paths, specifically:
[0038] The new local knowledge graph In the process, an initial screening is performed based on the overlap between entities and relations and the n-grams in the query, retaining entities and relations with a similarity greater than 0.9:
[0039] ,
[0040] Entities and relationships with a similarity greater than 0.9 are used to construct a filter subgraph. ;
[0041] For the filter subgraph Structural analysis: Semantic communities are divided by connecting components, and a maximum spanning tree is built in each connected subgraph. Redundant edges are pruned to preserve core semantic associations. Finally, tightly coupled semantic subgraphs are extracted and the edge with the highest weight is selected to form a unique path.
[0042] Furthermore, if the number of filtered contexts is less than K, an alternative filtering mechanism is activated:
[0043] Prioritize selecting the top two nodes in the original search results and supplementing their context to ensure that core information is not lost;
[0044] If the final number still does not reach K, then supplement the context in descending order of the original search scores until the condition is met.
[0045] The second aspect of this invention discloses a question-answering system based on LLM-fused knowledge graph and retrieval enhancement generation, the question-answering system including a retrieval module, a knowledge graph expansion module, a graph structure filtering module, and an answer generation module;
[0046] The retrieval module performs preliminary semantic retrieval on the question query and the document database, and then selects the most relevant documents from the full document library according to similarity ranking to obtain a document set. This provides input for knowledge graph expansion;
[0047] The knowledge graph extension module will include the document set As input, the corresponding local knowledge graph is extracted from the large language model. , represented as ,in For a local entity set, It is a local relation set;
[0048] The knowledge graph extension module also traverses the global knowledge graph. By using graph structure connections, entities that are connected to known entities but do not appear in the local knowledge graph are marked as related entities, and their relationships are added to the local knowledge graph, thereby expanding the local knowledge graph and forming a new local knowledge graph. ;
[0049] The graph structure filtering module will filter the new local knowledge graph. Perform structured filtering to obtain semantic subgraphs and unique paths;
[0050] The answer generation module processes the connected components by edges and weights. Sort in descending order, and if the weights are the same, use the lexicographical order of the entity name as the second sorting key to ensure the determinacy and uniqueness of the sorting results; after sorting, add the text information corresponding to the nodes on the path in the order of sorting until the cumulative number of documents reaches the preset number K.
[0051] The answer generation module combines the final K documents with the question query and inputs them into the large language model to generate the final answer.
[0052] This invention addresses the technical deficiencies in the prior art and possesses the following beneficial effects: Through the collaborative operation of multiple modules including semantic retrieval, graph expansion, and graph structure filtering, this invention achieves deep semantic association mining and redundant information filtering, providing interpretable and high-precision answers for complex question-answering scenarios, improving the system's response quality and credibility, and effectively combining structured knowledge with dynamic external information retrieval to enhance the accuracy and interpretability of question answering, solving the illusion problem of large language models, and providing reliable knowledge service support for complex decision-making scenarios. Attached Figure Description
[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained from these drawings without creative effort.
[0054] Figure 1 This is a flowchart of a question-answering method based on LLM-integrated knowledge graph and retrieval enhancement.
[0055] Figure 2 This is a framework diagram of a question-answering system based on LLM-integrated knowledge graph and retrieval enhancement. Detailed Implementation
[0056] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0057] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.
[0058] like Figure 1 As shown, the first aspect of this invention discloses a question-answering method based on LLM-fused knowledge graph and retrieval enhancement generation, comprising the following steps:
[0059] A preliminary semantic search is performed on the query and the document database. Then, based on similarity, the most relevant documents are selected from the entire document database to obtain the document set. This provides input for knowledge graph expansion;
[0060] It should be noted that the retrieval module is responsible for performing preliminary semantic retrieval of the query and the document database. Its core task is to select the top K most relevant documents from the entire document library and sort them by similarity, providing input for subsequent knowledge graph expansion. Specifically, this module first uses the Ollama platform's pre-trained embedding model nomic-embed-text to encode the query, representing the query as a vector. Similarly, every document in the document library It is also obtained through vector encoding using the same model. Then, the problem of cosine similarity calculation was solved. With Documents The semantic association strength between documents is determined, and the documents are sorted in descending order of similarity. The top K documents are then selected as the results and sent to the knowledge graph extension module.
[0061] The document set As input, the corresponding local knowledge graph is extracted from the large language model. , represented as ,in For a local entity set, It is a local relation set;
[0062] It should be noted that the knowledge graph extension module uses the relevant document sets retrieved in the previous stage. As input, the corresponding local knowledge graph is extracted using a Large Language Model (LLM), and represented as follows: This local knowledge graph integrates structured information at three different levels: the first level is sentence-level relations, that is, relations from document sentences. Triples generated by extracting information The second layer is the entity-sentence relationship, that is, constructing entities. With sentences Triples of origination relationships The third layer is entity-entity relations, which are triples that establish relationships between different entities based on semantic reasoning. To further expand upon this, the module traverses the global knowledge graph. Identification of connection relationships based on graph structure New entities that are directly connected to existing entities but are not included in the current local graph are marked as related entities, and their corresponding relationships are added to the local knowledge graph, thus completing the graph expansion and forming an enhanced graph. .
[0063] Traversing the global knowledge graph By using graph structure connections, entities that are connected to known entities but do not appear in the local knowledge graph are marked as related entities, and their relationships are added to the local knowledge graph, thereby expanding the local knowledge graph and forming a new local knowledge graph. ;
[0064] The new local knowledge graph Perform structured filtering to obtain semantic subgraphs and unique paths;
[0065] It should be noted that the graph structure filtering module performs structured filtering on the expanded knowledge graph. First, it performs initial filtering based on the n-gram overlap between entities and relations and the query, retaining entities and relations with a similarity greater than 0.9 to form the filtering subgraph. The subgraph structure was then analyzed: semantic communities were divided by connected components, and a maximum spanning tree (MST) was built in each connected subgraph. Redundant edges were pruned to preserve core semantic associations, and finally, tightly coupled semantic subgraphs and unique paths were extracted.
[0066] Weighted edge-wise within connected components Sort in descending order, and when the weights are the same, use the lexicographical order of the entity name as the second sorting key to ensure the determinism and uniqueness of the sorting results; after sorting, add the text information corresponding to the nodes on the path in the order of sorting until the cumulative number of documents reaches the preset number K; where, a connected component refers to a maximum subset of nodes in the graph, in which there is at least one path connecting any two nodes, but no connection at all with nodes in other components.
[0067] The resulting K documents are combined with the question query and input into a large language model to generate the final answer.
[0068] It should be noted that the answer generation module, based on the MST results, performs edge-weighted processing within connected components. Sort in descending order; when weights are the same, use the lexicographical order of entity names as the second sorting key to ensure the uniqueness of the sorting results. Extract the text corresponding to the path nodes in this order until the number of documents reaches the preset value K. If the number of contexts after MST filtering is insufficient, an alternative filtering mechanism is activated: prioritize adding the contexts corresponding to the top 2 nodes in the original search results to retain core information; if the number still does not meet the requirement, add documents in descending order according to the original search scores. Finally, combine the integrated top K documents with the question and input them into the Large Language Model (LLM) to generate the final answer.
[0069] Furthermore, a preliminary semantic search is performed on the query and the document database, and the most relevant documents to the query are selected from the full set of documents based on similarity ranking. Specifically:
[0070] Vectorize the input question Query to obtain a vector. And through Ollam's pre-trained embedding model Encode text blocks to obtain vectors :
[0071] ,
[0072] in, The first word or text block in the query; The second word or text block in the query; The nth word or text block in the query;
[0073] Any document in the document library Similarly, vector encoding is performed to obtain... and vector With Documents Perform cosine similarity calculation to obtain vectors With Documents The semantic association strength between them is calculated using the following formula:
[0074] ,
[0075] vector With Documents The semantic association strength between documents is sorted in descending order of similarity, and the top K documents are selected as the set of relevant documents retrieved. :
[0076] ,
[0077] In the formula, This is the document ranked first after being sorted in descending order of similarity; This is the second-ranked document after being sorted in descending order of similarity. This refers to the document ranked Kth after sorting by similarity in descending order; This indicates the total number of documents in the document library.
[0078] The local knowledge graph integrates structured information at three different levels:
[0079] The first level is sentence-level relations, that is, relations between document sentences. Triples generated by extracting information ;in, The head entity represents the main body or starting point in the relationship; Representative relationship, describing the semantic relationship between the head entity and the tail entity; The tail entity represents the object or endpoint in the relationship;
[0080] The second layer is the entity-sentence relationship, that is, constructing entities. With sentences Triples of origination relationships ;in, A representative entity is a specific concept, object, or name extracted from a document; Representational relationship, specifically referring to the connection between entities and sentences; The representative sentence is a collection of documents. A specific sentence text;
[0081] The third layer is entity-entity relations, which are triples that establish relationships between different entities based on semantic reasoning. ;in, The head entity represents the main body or starting point in the relationship; Relationship, describing the semantic relationship between two entities; The tail entity represents the object or endpoint in the relationship.
[0082] Furthermore, traverse the global knowledge graph. By using graph structure connections, entities that are connected to known entities but do not appear in the local knowledge graph are marked as related entities, and their relationships are added to the local knowledge graph, thereby expanding the local knowledge graph and forming a new local knowledge graph. Specifically:
[0083] For local knowledge graphs Each entity in Find its direct neighbor entities in the global graph. :
[0084] ,
[0085] From adjacent entities Select new entities that do not exist in the local map. :
[0086] ,
[0087] Extract all relation triples connecting the old and new entities to obtain the new relation triples. :
[0088] ,
[0089] The new entity New Relationship Triad Merging into the local knowledge graph yields a new local knowledge graph. ;
[0090] In the formula, For the global entity collection; It is the set of global relations, that is, all known triples that connect global entities.
[0091] Among them, the ,in, It is the set of all new entities in the new local knowledge graph; It is the set of all new relation triples in the new local knowledge graph.
[0092] Furthermore, the new local knowledge graph Perform structured filtering to obtain semantic subgraphs and unique paths, specifically:
[0093] The new local knowledge graph In the process, an initial screening is performed based on the overlap between entities and relations and the n-grams in the query, retaining entities and relations with a similarity greater than 0.9:
[0094] ,
[0095] It should be noted that n-gram overlap refers to the proportion of n-grams (i.e., n consecutive words) that appear in both texts, and is used to measure the similarity between texts.
[0096] Entities and relationships with a similarity greater than 0.9 are used to construct a filter subgraph. ;
[0097] For the filter subgraph Structural analysis: Semantic communities are divided by connecting components, and a maximum spanning tree is built in each connected subgraph. Redundant edges are pruned to preserve core semantic associations. Finally, tightly coupled semantic subgraphs are extracted and the edge with the highest weight is selected to form a unique path.
[0098] It should be noted that in the subgraph structure analysis, the original graph is decomposed into multiple semantic communities using a connected component partitioning algorithm (such as depth-first search or disjoint-set data structure), with each community corresponding to a connected subgraph. Then, within each connected subgraph, a maximum spanning tree algorithm (such as Kruskal's algorithm or Prim's algorithm, but maximizing edge weights) is applied to construct a spanning tree. This involves prioritizing high-weight edges to connect all nodes, thus pruning low-weight or redundant edges and preserving core semantic connections. Next, based on the maximum spanning tree, tightly coupled semantic subgraphs (e.g., those with a weight ratio exceeding 80%) are extracted. Finally, by traversing each subtree and selecting the edge sequence with the highest weight (e.g., the path with the highest cumulative weight from the root node to the leaf node), a unique path is formed to highlight the main semantic flows and key connections within the semantic community.
[0099] Furthermore, if the number of filtered contexts is less than K, an alternative filtering mechanism is activated:
[0100] Prioritize selecting the top two nodes in the original search results and supplementing their context to ensure that core information is not lost;
[0101] If the final number still does not reach K, then supplement the context in descending order of the original search scores until the condition is met.
[0102] In this embodiment, the method further includes the following steps:
[0103] Obtain the semantic subgraph and its unique path obtained after structured filtering from the current question-and-answer session, and use them as the input source for the external memory module;
[0104] The graph attention network is used to encode the entities and relations to generate graph embedding vectors for each entity and relation, thereby capturing the topological and semantic features of the graph.
[0105] The graph embedding vector is input into a recurrent memory unit with forgetting gate. By calculating the correlation between the current input and the historical memory state, the forgetting gate value is adjusted to selectively retain or discard information, thereby updating the global memory state of the external memory module.
[0106] During the update process, the importance score of each entity in the memory state is calculated based on the graph attention mechanism. The focus is on the core entities according to the score, and attention-weighted memory output is generated.
[0107] The updated memory state is persistently stored and used as the initial memory input for the next round of question-and-answer sessions, enabling context awareness and knowledge state transfer across sessions.
[0108] Specifically, in the question-answering session, after semantic parsing and knowledge retrieval, the user's input question extracts relevant entities and relationships from the global knowledge graph, forming an initial local knowledge graph. Subsequently, a graph structure filtering module (such as n-gram overlap screening and maximum spanning tree pruning) simplifies the graph, removing redundant edges and low-relevance entities to obtain the core semantic subgraph and its unique path. This step ensures the relevance and simplicity of the input data and reduces noise. A graph attention network is used to encode the entities and relationships in the semantic subgraph, generating a graph embedding vector for each entity and relationship to capture the graph's topological structure and semantic features. The graph attention network aggregates neighbor information by calculating attention weights between nodes, thereby generating embedding vectors that reflect both local and global structures. In practice, for each entity and relationship, the network considers its connection strength with neighboring nodes, outputting a high-dimensional vector representation. These embedding vectors not only encode the semantics of the entity itself but also include its structural role in the graph, enabling the system to understand the relationships between knowledge points more precisely.
[0109] The graph embedding vector is then input into a recurrent memory unit with a forgetting gate. By calculating the correlation between the current input and the historical memory state, the forgetting gate value is adjusted to selectively retain or discard information, thereby updating the global memory state of the external memory module. This recurrent memory unit is similar to LSTM or GRU, but optimized for graph data: the forgetting gate value is calculated based on the dot product or cosine similarity between the current input embedding and the historical memory state, thus determining which old memories should be retained or forgotten.
[0110] During the update process, an importance score for each entity in the memory state is calculated based on a graph attention mechanism. The focus is then placed on core entities based on their scores, generating attention-weighted memory outputs. The importance score is derived by performing a secondary calculation on the entities in the memory state using a graph attention network. Entities with higher scores (such as those that appear frequently or are highly relevant to the current session) receive greater weight in the memory output.
[0111] Finally, the updated memory state is persistently stored as the initial memory input for the next round of question-answering, enabling context awareness and knowledge state transfer across sessions. Specifically, the updated memory state (including entity embeddings and relation weights) is saved to a database or distributed storage system. When a user initiates a new session, the system loads this memory state from storage as the initial input, thereby inheriting the knowledge context from previous sessions. This effectively solves the problems of missing cross-session memory and low efficiency in graph data processing in existing question-answering systems. By utilizing structured filtering, graph attention encoding, and dynamic memory updates, the system achieves efficient knowledge retention and transfer, improving the accuracy and coherence of question answering.
[0112] In this embodiment, the method further includes the following steps:
[0113] Extract all connected components and their corresponding edge weights from the local knowledge graph, and model the problem of ranking these edge weights as a combinatorial optimization problem with the objective of maximizing the total path weight.
[0114] The combinatorial optimization problem is transformed into a quadratic unconstrained binary optimization form that can be handled by a quantum annealer, where edge selection variables are encoded as binary variables and objective functions and constraints are constructed based on edge weights.
[0115] Configure the annealing parameters of the quantum annealer, including annealing time and coupling strength, and perform the quantum annealing process to obtain an approximate optimal solution in an exponential search space;
[0116] The state of binary variables is decoded from the near-optimal solution output by the quantum annealer and mapped back to the corresponding edge sequence to form candidate document paths;
[0117] Based on the node order in the candidate document path, extract the text information associated with the nodes sequentially until the cumulative number of documents reaches a preset number K, thereby completing the generation of the document sequence.
[0118] It should be noted that all connected components and their corresponding edge weights are extracted from the preprocessed and expanded local knowledge graph. Each connected component represents a semantically closely related subgraph, while the edge weights quantify the strength of the relationships between nodes. The edge weight ranking problem is constructed as a combinatorial optimization problem aiming to maximize the total weight of the paths. This combinatorial optimization problem is transformed into a quadratic unconstrained binary optimization form that can be directly processed by a quantum annealer. In this step, a binary decision variable is defined for each edge in the graph; a value of 1 indicates that the edge is selected, and a value of 0 indicates that it is not selected. An objective function is constructed based on the edge weights, whose mathematical expression is to minimize the negative sum of total weights. Simultaneously, constraints are introduced to ensure that the formed paths satisfy connectivity and continuity requirements; these constraints are integrated into the objective function in the form of penalty terms.
[0119] Additionally, it's important to note that after decoding the state of the binary variables from the near-optimal solution output by the quantum annealer, they are mapped back to the corresponding edge sequences. By analyzing the values of these binary variables, it's possible to determine which edges are selected, thus forming candidate document paths. Based on the node order in the candidate document paths, the original text information associated with each node is extracted sequentially. The system traverses the knowledge graph nodes according to the order of nodes in the path, extracting text content from the document fragments corresponding to each node, and forming a coherent document sequence through intelligent splicing technology. Extraction stops when the accumulated number of documents reaches a preset value K, thus generating the final document output. This achieves a smooth conversion from structured knowledge to natural language documents, ensuring the readability and completeness of the generated content.
[0120] like Figure 2 As shown, the second aspect of the present invention discloses a question-answering system based on LLM fusion of knowledge graph and retrieval enhancement generation, the question-answering system including a retrieval module, a knowledge graph expansion module, a graph structure filtering module and an answer generation module;
[0121] The retrieval module performs preliminary semantic retrieval on the question query and the document database, and then selects the most relevant documents from the full document library according to similarity ranking to obtain a document set. This provides input for knowledge graph expansion;
[0122] The knowledge graph extension module will include the document set As input, the corresponding local knowledge graph is extracted from the large language model. , represented as ,in For a local entity set, It is a local relation set;
[0123] The knowledge graph extension module also traverses the global knowledge graph. By using graph structure connections, entities that are connected to known entities but do not appear in the local knowledge graph are marked as related entities, and their relationships are added to the local knowledge graph, thereby expanding the local knowledge graph and forming a new local knowledge graph. ;
[0124] The graph structure filtering module will filter the new local knowledge graph. Perform structured filtering to obtain semantic subgraphs and unique paths;
[0125] The answer generation module processes the connected components by edges and weights. Sort in descending order, and if the weights are the same, use the lexicographical order of the entity name as the second sorting key to ensure the determinacy and uniqueness of the sorting results; after sorting, add the text information corresponding to the nodes on the path in the order of sorting until the cumulative number of documents reaches the preset number K.
[0126] The answer generation module combines the final K documents with the question query and inputs them into the large language model to generate the final answer.
[0127] It should be noted that this system includes a data preprocessing module, an algorithm module, and a user interaction module. The data preprocessing module performs structured parsing of the original documents, automatically extracting multi-level triples such as entity-entity, entity-sentence, and sentence-level relationships using a large language model, and persistently storing them in a knowledge base to provide high-quality knowledge support for downstream tasks. The algorithm module sequentially performs semantic retrieval, dynamic knowledge graph expansion, graph structure filtering, and answer generation. By integrating structured knowledge with retrieved information, it enhances the reasoning ability of questions and the interpretability of answers. The user interaction module serves as the system's entry and exit point, responsible for receiving user queries and clearly presenting structured answers to users, forming an efficient and intuitive interactive loop.
[0128] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.
[0129] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.
[0130] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0131] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0132] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.
[0133] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A question-answering method based on LLM-integrated knowledge graph and retrieval enhancement generation, characterized in that, Includes the following steps: A preliminary semantic search is performed on the query and the document database. Then, based on similarity, the most relevant documents are selected from the entire document database to obtain the document set. This provides input for knowledge graph expansion; The document set As input, the corresponding local knowledge graph is extracted from the large language model. , represented as ,in For a local entity set, It is a local relation set; Traversing the global knowledge graph By using graph structure connections, entities that are connected to known entities but do not appear in the local knowledge graph are marked as related entities, and their relationships are added to the local knowledge graph, thereby expanding the local knowledge graph and forming a new local knowledge graph. ; The new local knowledge graph Structured filtering is performed to obtain semantic subgraphs and unique paths: Initial screening is conducted based on the overlap between entities and relations and the n-grams in the query, retaining entities and relations with a similarity greater than 0.9 to form a filter subgraph. Further analysis of the subgraph structure: Semantic communities are divided by connected components, and a maximum spanning tree is built in each connected subgraph. Redundant edges are pruned to preserve core semantic associations, and finally, tightly coupled semantic subgraphs and unique paths are extracted. Weighted edge-wise within connected components Sort in descending order, and when the weights are the same, use the lexicographical order of the entity name as the second sorting key to ensure the determinacy and uniqueness of the sorting result; after sorting, add the text information corresponding to the nodes on the path in the order of sorting until the cumulative number of documents reaches the preset number K; the connected component refers to a maximum subset of nodes in the graph, in which there is at least one path connecting any two nodes, but no connection with nodes in other components. The resulting K documents are combined with the question query and input into a large language model to generate the final answer.
2. The question-answering method based on LLM-fused knowledge graph and retrieval enhancement generation according to claim 1, characterized in that, A preliminary semantic search is performed on the query and the document database. Then, based on similarity, the most relevant documents are selected from the entire document database to obtain the document set. Specifically: Vectorize the input question Query to obtain a vector. And through Ollam's pre-trained embedding model Encode text blocks to obtain vectors : ， in, The first word or text block in the query; The second word or text block in the query; The nth word or text block in the query; Any document in the document library Similarly, vector encoding is performed to obtain... and vector With Documents Perform cosine similarity calculation to obtain vectors With Documents The semantic association strength between them is calculated using the following formula: ， vector With Documents The semantic association strength between documents is sorted in descending order of similarity, and the top K documents are selected as the set of relevant documents retrieved. : ， In the formula, This is the document ranked first after being sorted in descending order of similarity; This is the second-ranked document after being sorted in descending order of similarity. This refers to the document ranked Kth after sorting by similarity in descending order; This indicates the total number of documents in the document library.
3. The question-answering method based on LLM-fused knowledge graph and retrieval enhancement generation according to claim 1, characterized in that, The local knowledge graph integrates structured information at three different levels: The first level is sentence-level relations, that is, relations between document sentences. Triples generated by extracting information ;in, The head entity represents the main body or starting point in the relationship; Representative relationship, describing the semantic relationship between the head entity and the tail entity; The tail entity represents the object or endpoint in the relationship; The second layer is the entity-sentence relationship, that is, constructing entities. With sentences Triples of origination relationships ;in, A representative entity is a specific concept, object, or name extracted from a document; Representational relationship, specifically referring to the connection between entities and sentences; The representative sentence is a collection of documents. A specific sentence text; The third layer is entity-entity relations, which are triples that establish relationships between different entities based on semantic reasoning. ;in, The head entity represents the main body or starting point in the relationship; Relationship, describing the semantic relationship between two entities; The tail entity represents the object or endpoint in the relationship.
4. The question-answering method based on LLM-fused knowledge graph and retrieval enhancement generation according to claim 3, characterized in that, Traversing the global knowledge graph By using graph structure connections, entities that are connected to known entities but do not appear in the local knowledge graph are marked as related entities, and their relationships are added to the local knowledge graph, thereby expanding the local knowledge graph and forming a new local knowledge graph. Specifically: For local knowledge graphs Each entity in Find its direct neighbor entities in the global graph. : ， From adjacent entities Select new entities that do not exist in the local map. : ， Extract all relation triples connecting the old and new entities to obtain the new relation triples. : ， The new entity New Relationship Triad Merging into the local knowledge graph yields a new local knowledge graph. ; In the formula, For the global entity collection; It is the set of global relations, that is, all known triples that connect global entities.
5. The question-answering method based on LLM-fused knowledge graph and retrieval enhancement generation according to claim 4, characterized in that, The ,in, It is the set of all new entities in the new local knowledge graph; It is the set of all new relation triples in the new local knowledge graph.
6. The question-answering method based on LLM-fused knowledge graph and retrieval enhancement generation according to claim 4, characterized in that, The new local knowledge graph Perform structured filtering to obtain semantic subgraphs and unique paths, specifically: The new local knowledge graph In the process, an initial screening is performed based on the overlap between entities and relations and the n-grams in the query, retaining entities and relations with a similarity greater than 0.9: ， Entities and relationships with a similarity greater than 0.9 are used to construct a filter subgraph. ; For the filter subgraph Structural analysis: Semantic communities are divided by connecting components, and a maximum spanning tree is built in each connected subgraph. Redundant edges are pruned to preserve core semantic associations. Finally, tightly coupled semantic subgraphs are extracted and the edge with the highest weight is selected to form a unique path.
7. The question-answering method based on LLM-fused knowledge graph and retrieval enhancement generation according to claim 1, characterized in that, If the number of filtered contexts is less than K, an alternative filtering mechanism will be activated: Prioritize selecting the top two nodes in the original search results and supplementing their context to ensure that core information is not lost; If the final number still does not reach K, then supplement the context in descending order of the original search scores until the condition is met.
8. A question-answering system based on LLM-integrated knowledge graph and retrieval enhancement generation, characterized in that, The question-answering system includes a retrieval module, a knowledge graph expansion module, a graph structure filtering module, and an answer generation module. The retrieval module performs preliminary semantic retrieval on the question query and the document database, and then selects the most relevant documents from the full document library according to similarity ranking to obtain a document set. This provides input for knowledge graph expansion; The knowledge graph extension module will include the document set As input, the corresponding local knowledge graph is extracted from the large language model. , represented as ,in For a local entity set, It is a local relation set; The knowledge graph extension module also traverses the global knowledge graph. By using graph structure connections, entities that are connected to known entities but do not appear in the local knowledge graph are marked as related entities, and their relationships are added to the local knowledge graph, thereby expanding the local knowledge graph and forming a new local knowledge graph. ; The graph structure filtering module will filter the new local knowledge graph. Structured filtering is performed to obtain semantic subgraphs and unique paths: Initial screening is conducted based on the overlap between entities and relations and the n-grams in the query, retaining entities and relations with a similarity greater than 0.9 to form a filter subgraph. Further analysis of the subgraph structure: Semantic communities are divided by connected components, and a maximum spanning tree is built in each connected subgraph. Redundant edges are pruned to preserve core semantic associations, and finally, tightly coupled semantic subgraphs and unique paths are extracted. The answer generation module processes the connected components by edges and weights. Sort in descending order, and when the weights are the same, use the lexicographical order of the entity name as the second sorting key to ensure the determinacy and uniqueness of the sorting result; after sorting, add the text information corresponding to the nodes on the path in the order of sorting until the cumulative number of documents reaches the preset number K; the connected component refers to a maximum subset of nodes in the graph, in which there is at least one path connecting any two nodes, but no connection with nodes in other components. The answer generation module combines the final K documents with the question query and inputs them into the large language model to generate the final answer.