Large model illusion suppression method fusing RAG and knowledge graph
By integrating RAG and knowledge graph methods, we constructed an enhanced context for confidence labeling, which solved the problem of generating false information in large language models and improved the accuracy and credibility of answers.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-03
AI Technical Summary
Large language models may generate false information, or hallucinations, when processing factual knowledge, and existing technologies are unable to fundamentally suppress these factual hallucinations.
The approach integrates RAG and knowledge graphs, retrieves evidence from external document libraries and pre-built knowledge graphs in parallel, performs cross-validation and evidence alignment, constructs enhanced contexts with confidence labels, dynamically arbitrates evidence support, contradictions and independence relationships, and cites evidence sources and confidence levels when generating answers.
It effectively suppresses the factual illusion of large language models, improves the accuracy and interpretability of answers, enhances the credibility of results, and forms a complete and reliable knowledge enhancement generation scheme.
Smart Images
Figure CN121787581A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for suppressing hallucinations in large models that integrates RAG and knowledge graphs. Background Technology
[0002] Large language models may generate false information, a phenomenon known as "illusion," when processing factual knowledge. Existing technologies mainly employ retrieval-enhanced generation methods, which use relevant text fragments retrieved from external knowledge bases as context to constrain the model's generation.
[0003] Traditional RAG methods rely on a single unstructured document library. The evidence retrieved may contain errors, contradictions, or lack structured verification, resulting in low-quality reference context provided to large language models and making it difficult to fundamentally suppress factual illusions in the model. Summary of the Invention
[0004] This invention provides a method for suppressing large model illusions by integrating RAG and knowledge graphs, in order to solve the technical problem that existing technologies are unable to fundamentally suppress factual illusions in models.
[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:
[0006] This invention provides a large-model hallucination suppression method that integrates RAG and knowledge graph, including:
[0007] Receive user queries and use a pre-trained large language model to parse the intent of the user queries, generating one or more search queries;
[0008] For each search query, based on an external document library and a pre-built knowledge graph, a first candidate evidence set and a second candidate evidence set corresponding to the search query are retrieved.
[0009] Cross-validation and evidence alignment are performed on the first candidate evidence set and the second candidate evidence set, and an enhanced context with confidence labels is constructed based on the results of the cross-validation and evidence alignment.
[0010] The user query and the enhanced context are input into the large language model so that the large language model generates an answer based on the enhanced context, wherein the answer references the evidence sources and confidence levels in the enhanced context.
[0011] In an optional embodiment, the step of retrieving a first candidate evidence set and a second candidate evidence set corresponding to each search query based on an external document library and a pre-built knowledge graph includes:
[0012] For each of the search queries, the following two retrieval branches are executed in parallel:
[0013] Branch 1: Retrieve the Top-K document fragments that are semantically related to the search query from an external document library, and use them as the first candidate evidence set;
[0014] Branch 2: From the pre-constructed domain knowledge graph, retrieve the entities, relationships, and attribute facts associated with the search query through graph query or semantic matching, as the second candidate evidence set.
[0015] In an optional embodiment, retrieving the Top-K document fragments from an external document library that are semantically related to the search query, as the first candidate evidence set, includes:
[0016] The search query is encoded into a query vector;
[0017] Calculate the similarity between the query vector and the vectors of all document fragments in the external document library;
[0018] Based on the similarity ranking, the top K document fragments are selected as the first candidate evidence set.
[0019] In an optional embodiment, retrieving entities, relationships, and attribute facts associated with the search query from a pre-built domain knowledge graph through graph querying or semantic matching, as the second candidate evidence set, includes:
[0020] The search query is input into a pre-trained entity linking model to identify the target entity in the domain knowledge graph corresponding to the search query.
[0021] Centered on the target entity, perform a one-hop or multi-hop graph traversal in the domain knowledge graph to obtain neighboring entities, connection relationships, and attribute values that are directly or indirectly connected to the target entity.
[0022] The second candidate evidence set is constructed based on the target entity, the neighboring entities, the connection relationships, the attribute values, and the graph structure between them.
[0023] In an optional embodiment, the cross-validation and evidence alignment of the first candidate evidence set and the second candidate evidence set includes:
[0024] Extract named entities and key relationships from document fragments in the first candidate evidence set;
[0025] Based on the associated entities, relationships, and attribute facts, a structured evidence subgraph of the second candidate evidence set is generated;
[0026] Align the named entities and key relationships with the nodes and edges in the structured evidence subgraph;
[0027] Based on the alignment results, information units that are mutually supportive, mutually contradictory, and / or exist independently in the first candidate evidence set and the second candidate evidence set are identified.
[0028] In an optional embodiment, aligning the named entities and key relationships with the nodes and edges in the structured evidence subgraph includes:
[0029] Calculate the name similarity and type matching degree between the named entity and the entity nodes in the structured evidence subgraph;
[0030] For entity pairs whose name similarity and type matching degree are both higher than a preset threshold, an alignment mapping is established;
[0031] For the entity pairs that establish the alignment mapping, determine the semantic consistency between the key relationship connecting them and the corresponding edge in the structured evidence subgraph;
[0032] The alignment result is obtained based on the semantic consistency judgment result.
[0033] In an optional embodiment, constructing an enhanced context with confidence labels based on the results of the cross-validation and evidence alignment includes:
[0034] For information units that support each other in the first and second candidate evidence sets, increase the confidence level of that information unit;
[0035] For contradictory information units in the first candidate evidence set and the second candidate evidence set, arbitration shall be conducted based on the authority and timeliness of the information unit in the first candidate evidence set and the second candidate evidence set.
[0036] For information units that exist independently in the first and second candidate evidence sets, the original confidence level of the information unit is retained.
[0037] In an optional embodiment, after inputting the user query and the enhanced context into the large language model to generate an answer based on the enhanced context, the method further includes:
[0038] From the answer, the source identifier of the cited evidence is extracted;
[0039] Based on the evidence source identifier, the corresponding original evidence content and corresponding confidence level are retrieved from the enhanced context;
[0040] The answer, the evidence source identifier, and the corresponding confidence level are combined to generate the final verifiable answer output.
[0041] In an optional embodiment, the method further includes:
[0042] The large language model is used to extract information from new domain documents to obtain candidate fact triples;
[0043] The candidate fact triples are checked for consistency with the existing knowledge in the domain knowledge graph.
[0044] The candidate fact triples that pass the verification are merged into the domain knowledge graph based on their association strength with other entities, and the external document library is updated.
[0045] This invention effectively suppresses the illusion of fact in large language models by constructing a technical path of dual-path retrieval, cross-validation, and confidence fusion. Specifically, the method retrieves heterogeneous evidence in parallel from unstructured document libraries and structured knowledge graphs. By performing fine-grained semantic alignment and cross-validation on both, it can identify the support, contradiction, and independence relationships between the evidence. Based on the validation results, it dynamically constructs enhanced contexts with quantified confidence labels, providing the model with high-quality references that have undergone fact-checking and conflict arbitration. Finally, when generating answers, the model must cite traceable evidence sources and confidence levels, which not only improves the factual accuracy of the answers but also enhances the interpretability and credibility of the results, forming a complete and reliable knowledge enhancement generation scheme. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0047] Figure 1 This is a flowchart illustrating the large-model hallucination suppression method that integrates RAG and knowledge graph provided in this embodiment of the invention. Detailed Implementation
[0048] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The following embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes, but the scope of protection of the present invention is not limited to the following embodiments.
[0049] In this invention, the terms "in one possible embodiment," "exemplary," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "in one possible embodiment," "exemplary," or "for example" in this invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "in one possible embodiment," "exemplary," or "for example" is intended to present the relevant concepts in a specific manner.
[0050] Figure 1 This is a flowchart illustrating the large-model hallucination suppression method integrating RAG and knowledge graph provided in an embodiment of the present invention. Figure 1 As shown, the large-model hallucination suppression method that integrates RAG and knowledge graph includes:
[0051] S101, Receive user query, and use a pre-trained large language model to perform intent parsing on the user query, and generate one or more search queries;
[0052] S102, for each search query, based on the external document library and the pre-built knowledge graph, retrieve the first candidate evidence set and the second candidate evidence set corresponding to the search query;
[0053] S103, perform cross-validation and evidence alignment on the first candidate evidence set and the second candidate evidence set, and construct an enhanced context with confidence labels based on the results of the cross-validation and evidence alignment.
[0054] S104, the user query and the enhanced context are input into the large language model so that the large language model generates an answer based on the enhanced context, wherein the answer references the evidence sources and confidence levels in the enhanced context.
[0055] For example, a user submits a query via API: "How effective is AA Company's BB vaccine against CC virus?" The system first calls a pre-trained large language model to parse the query intent, identifying the core entities "AA Company," "BB vaccine," "CC virus," and the relationship "effectiveness," and potentially reconstructing a more precise search query. Subsequently, the system searches for evidence in parallel in a medical literature vector database and a pharmaceutical knowledge graph. The retrieved document fragments and knowledge graph subgraphs are cross-validated. For instance, a document fragment states "the effectiveness is approximately 70%," while the knowledge graph records "a significant decrease in neutralizing antibody titers against CC virus." The system identifies this contradiction and arbitrates based on the document source and the number of times this fact is cited in the knowledge graph, ultimately constructing a context labeled with different pieces of evidence and their confidence levels. Finally, the original query and this context are input into the large language model, instructing it to generate an answer with citations.
[0056] In an optional embodiment, the step of retrieving a first candidate evidence set and a second candidate evidence set corresponding to each search query based on an external document library and a pre-built knowledge graph includes:
[0057] For each of the search queries, the following two retrieval branches are executed in parallel:
[0058] Branch 1: Retrieve the Top-K document fragments that are semantically related to the search query from an external document library, and use them as the first candidate evidence set;
[0059] Branch 2: From the pre-constructed domain knowledge graph, retrieve the entities, relationships, and attribute facts associated with the search query through graph query or semantic matching, as the second candidate evidence set.
[0060] For example, the system receives a search query: "cardiovascular benefits of the drug EE for DD disease". Branch 1: The system calls an encoder to convert the query into a vector and calculates its similarity with a summary fragment vector library, returning the 5 fragments with the highest similarity. Branch 2: The system calls a model fine-tuned on medical text as an entity linker to link "EE" to entities in the knowledge graph. Then, a two-hop traversal is performed in the graph to obtain its nodes and edges, forming an evidence subgraph. The two retrieval branches are executed asynchronously, and the results are synchronized upon completion.
[0061] In an optional embodiment, retrieving the Top-K document fragments from an external document library that are semantically related to the search query, as the first candidate evidence set, includes:
[0062] The search query is encoded into a query vector;
[0063] Calculate the similarity between the query vector and the vectors of all document fragments in the external document library;
[0064] Based on the similarity ranking, the top K document fragments are selected as the first candidate evidence set.
[0065] For example, the external document repository uses a vector database for storage, where each document fragment (such as a paper abstract) is encoded as a 768-dimensional vector using a model. Upon receiving the query vector, the system performs an approximate nearest neighbor search using cosine similarity. The system maintains a max-heap and updates the Top-K similarity results in real time. Finally, the top 5 fragments with a similarity greater than a threshold (e.g., 0.75) are selected, and each fragment is accompanied by its similarity score as an initial confidence level.
[0066] In an optional embodiment, retrieving entities, relationships, and attribute facts associated with the search query from a pre-built domain knowledge graph through graph querying or semantic matching, as the second candidate evidence set, includes:
[0067] The search query is input into a pre-trained entity linking model to identify the target entity in the domain knowledge graph corresponding to the search query.
[0068] Centered on the target entity, perform a one-hop or multi-hop graph traversal in the domain knowledge graph to obtain neighboring entities, connection relationships, and attribute values that are directly or indirectly connected to the target entity.
[0069] The second candidate evidence set is constructed based on the target entity, the neighboring entities, the connection relationships, the attribute values, and the graph structure between them.
[0070] For example, a user queries "possible causes of idling vibration in FF model cars". The entity linking model identifies that "FF model car" can be linked to the vehicle model entity in the knowledge graph. The system executes the query in the graph and returns a structured subgraph containing the fault node "idling vibration", possible cause component nodes such as "engine mounts" and "throttle body", and their respective system nodes "engine mounting system" and "intake system".
[0071] In an optional embodiment, the cross-validation and evidence alignment of the first candidate evidence set and the second candidate evidence set includes:
[0072] Extract named entities and key relationships from document fragments in the first candidate evidence set;
[0073] Based on the associated entities, relationships, and attribute facts, a structured evidence subgraph of the second candidate evidence set is generated;
[0074] Align the named entities and key relationships with the nodes and edges in the structured evidence subgraph;
[0075] Based on the alignment results, information units that are mutually supportive, mutually contradictory, and / or exist independently in the first candidate evidence set and the second candidate evidence set are identified.
[0076] For example, the entities "engine mounts", "carbon buildup", and the relation "caused by" are extracted from the first evidence set (fragments of repair forum posts). The second evidence set (a subgraph of the graph) contains the nodes "engine mounts", "throttle body carbon buildup", and the relation [caused_by]. The system performs alignment: "engine mounts" in the text matches "engine mounts" in the graph using an alias dictionary; "carbon buildup" matches "throttle body carbon buildup" using a hypernym rule ("carbon buildup" is the parent class of "throttle body carbon buildup"). The relation "caused by" is semantically consistent with [caused_by]. Therefore, the information about "engine mounts causing vibration" is determined to be mutually supportive.
[0077] In an optional embodiment, aligning the named entities and key relationships with the nodes and edges in the structured evidence subgraph includes:
[0078] Calculate the name similarity and type matching degree between the named entity and the entity nodes in the structured evidence subgraph;
[0079] For entity pairs whose name similarity and type matching degree are both higher than a preset threshold, an alignment mapping is established;
[0080] For the entity pairs that establish the alignment mapping, determine the semantic consistency between the key relationship connecting them and the corresponding edge in the structured evidence subgraph;
[0081] The alignment result is obtained based on the semantic consistency judgment result.
[0082] For example, a user query might involve "the construction time of the Acropolis in Athens, ancient Greece". The textual evidence mentions "the Parthenon", and the atlas shows "Parthenon (Athens Acropolis)".
[0083] Name similarity: Calculate the similarity between "Parthenon" and "Parthenon" after translation and transliteration.
[0084] Type matching: The word "temple" in the text is classified as ARCHITECTURE, which matches the ArchitecturalStructure type in the graph.
[0085] Semantic consistency assessment: The text mentions "proposed by Pericles," and the graph contains an edge (Parthenon) - [proposed_by] -> (Pericles). A relation matching model is used to determine that the semantic vectors of "proposed_by" are highly similar, thus confirming consistency.
[0086] In an optional embodiment, constructing an enhanced context with confidence labels based on the results of the cross-validation and evidence alignment includes:
[0087] For information units that support each other in the first and second candidate evidence sets, increase the confidence level of that information unit;
[0088] For contradictory information units in the first candidate evidence set and the second candidate evidence set, arbitration shall be conducted based on the authority and timeliness of the information unit in the first candidate evidence set and the second candidate evidence set.
[0089] For information units that exist independently in the first and second candidate evidence sets, the original confidence level of the information unit is retained.
[0090] For example, when searching for "possible causes of abnormal noises during the kneading stage of a household G-model fully automatic bread maker", contradictory evidence emerges:
[0091] Evidence A (a user's forum help post): claims that "the abnormal noise must be caused by bearing wear, because debris was seen after disassembly." It has low authority (non-professional user), but is direct observational evidence. The system marks its personal experience authority value as 0.3 and its timeliness as high (posted this month).
[0092] Evidence B (entry from an appliance repair encyclopedia website): states that "possible causes of abnormal noise in a Model G bread maker include: worn bearings, loose belts, and damaged motor gears." The website editor is certified, but the entry is a generic description. The system indicates its source authority is 0.7, and its timeliness is medium (updated last year).
[0093] Evidence C ("Product Failure Mode Library" in the structured knowledge graph): Records a fact triple (Product: Model G bread maker) - [Common Failure] → (Failure: Abnormal kneading noise), and (Failure: Abnormal kneading noise) - [Possible Cause] → (Component: Drive belt). This knowledge originates from the manufacturer's internal maintenance statistics summary. The system marks its knowledge authority value as 0.8.
[0094] Arbitration Process: The system identifies the point of contention as the "primary cause": Evidence A points to the "bearing," while Evidence C leans more towards the "belt." The arbitration function considers all factors: Evidence A is a specific case but has limitations, while Evidence C represents a statistical pattern but is not absolute. Ultimately, the system may adopt a weighted and supplementary explanation strategy: labeling "loose belt" as a high-confidence (0.75) "primary possible cause," labeling "bearing wear" as a medium-confidence (0.6) "one of the common causes," and adding a note to the enhanced context: "A case of bearing wear directly observed by the user suggests this possibility exists; further judgment based on the listening location is recommended."
[0095] In an optional embodiment, after inputting the user query and the enhanced context into the large language model to generate an answer based on the enhanced context, the method further includes:
[0096] From the answer, the source identifier of the cited evidence is extracted;
[0097] Based on the evidence source identifier, the corresponding original evidence content and corresponding confidence level are retrieved from the enhanced context;
[0098] The answer, the evidence source identifier, and the corresponding confidence level are combined to generate the final verifiable answer output.
[0099] For example, a user might query, "How do I prepare a safe homemade fertilizer for my indoor foliage plant, the Silver Star Pothos?"
[0100] The answer generated by the large language model based on enhanced context might be: "According to the gardening guide [Guide_Ref] and the plant database [DB_Ref], diluted fermented soybean cake water (confidence: high) or fermented rice water (confidence: medium) can be used for top dressing."
[0101] Post-processing analysis:
[0102] The system parses the reference tags [Guide_Ref] and [DB_Ref].
[0103] Location from enhanced context:
[0104] [Guide_Ref] corresponds to the excerpt on page 45 of the "Home Gardening Care Manual": "...fermented soybean cake water is a safe organic liquid fertilizer...", with a post-arbitration confidence level of 0.85.
[0105] [DB_Ref] corresponds to the fact in the "Indoor Plant Care Knowledge Graph": (Plant: Silver Star Pothos) - [Suitable Fertilizer] → (Fertilizer: Fermented Rice Water), and is accompanied by the attribute {Recommendation Frequency: Once a Month, Confidence Level: 0.65}. The confidence level of 0.65 is due to the fact that this suggestion comes from the community experience summary.
[0106] Formatted output:
[0107] Answer: You can use diluted fermented soybean cake water or rice water fermentation liquid as homemade fertilizer for Silver Star pothos.
[0108] Evidence and Confidence Level:
[0109] 1. [Professional Guide] The "Home Gardening Care Handbook" recommends using well-rotted soybean cake water as a safe organic liquid fertilizer. (Confidence level: 0.85)
[0110] 2. According to the [Community Knowledge Base] Plant Care Database, fermented rice water is one of the suitable fertilizers for Silver Star Pothos. (Confidence level: 0.65, Note: Based on common experience among plant enthusiasts)
[0111] In an optional embodiment, the method further includes:
[0112] The large language model is used to extract information from new domain documents to obtain candidate fact triples;
[0113] The candidate fact triples are checked for consistency with the existing knowledge in the domain knowledge graph.
[0114] The candidate fact triples that pass the verification are merged into the domain knowledge graph based on their association strength with other entities, and the external document library is updated.
[0115] Example, field: processing and properties of novel composite material Alpha.
[0116] Information Extraction: The system periodically crawls the latest research abstracts on "Alpha composites" from the Materials Science preprint server. It uses a large language model to extract candidate fact triples from the abstracts in an instruction-based manner. For example, from an abstract, it extracts:
[0117] (Material: Alpha composite material) - [Coefficient of thermal expansion] → (Value: 2.3e-6 / K)
[0118] (Material: Alpha composite material) - [Not applicable] → (Process: Electrical Discharge Machining)
[0119] Consistency check: The system checks the candidate triples against the existing "Materials Science Knowledge Graph".
[0120] For the "coefficient of thermal expansion": If a record with a value of 2.1e-6 / K already exists in the spectrum (from an earlier authoritative journal), the system will identify a numerical conflict. It will compare the authority and timeliness of the two sources and may temporarily store the new data in a "to be verified" state with a low confidence level (e.g., 0.5), while triggering a "manual review required" flag.
[0121] For "Not applicable to electrical discharge machining": If there is no relevant record in the spectrum and it is logically consistent with the known attribute "Poor conductivity of Alpha composite materials", then the verification is passed.
[0122] Knowledge merging and updating: The validated "not applicable to EDM" relationship is merged into the knowledge graph with a medium confidence level (0.7, derived from a relatively recent single study). Simultaneously, the full text of this preprint is sliced, encoded, and added to an external document vector library. For conflicting values, the system can create a "disputed" edge linking the two different values and their respective metadata for reference during subsequent searches.
[0123] Based on the above structure, this invention effectively suppresses the illusion of fact in large language models by constructing a technical path of dual-path retrieval, cross-validation, and confidence fusion. Specifically, the method retrieves heterogeneous evidence in parallel from unstructured document libraries and structured knowledge graphs. By performing fine-grained semantic alignment and cross-validation on both, it can identify the support, contradiction, and independence relationships between the evidence. Based on the validation results, an enhanced context with quantified confidence labels is dynamically constructed, providing the model with high-quality references that have undergone fact-checking and conflict arbitration. Finally, when generating answers, the model must cite traceable evidence sources and confidence levels, which not only improves the factual accuracy of the answers but also enhances the interpretability and credibility of the results, forming a complete and reliable knowledge enhancement generation scheme.
[0124] Furthermore, it should be noted that the present invention can be provided as a method, apparatus, or computer program product. Therefore, embodiments of the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.
[0125] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0126] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0127] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.
[0128] Finally, it should be noted that the above description represents a preferred embodiment of the present invention. It should be pointed out that although preferred embodiments have been described, those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles described herein. These improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.
Claims
1. A large-scale hallucination suppression method integrating RAG and knowledge graph, characterized in that, include: Receive user queries and use a pre-trained large language model to parse the intent of the user queries, generating one or more search queries; For each search query, based on an external document library and a pre-built knowledge graph, a first candidate evidence set and a second candidate evidence set corresponding to the search query are retrieved. Cross-validation and evidence alignment are performed on the first candidate evidence set and the second candidate evidence set, and an enhanced context with confidence labels is constructed based on the results of the cross-validation and evidence alignment. The user query and the enhanced context are input into the large language model so that the large language model generates an answer based on the enhanced context, wherein the answer references the evidence sources and confidence levels in the enhanced context.
2. The large-model hallucination suppression method integrating RAG and knowledge graph as described in claim 1, characterized in that, For each search query, based on an external document library and a pre-built knowledge graph, a first candidate evidence set and a second candidate evidence set corresponding to the search query are retrieved, including: For each of the search queries, the following two retrieval branches are executed in parallel: Branch 1: Retrieve the Top-K document fragments that are semantically related to the search query from an external document library, and use them as the first candidate evidence set; Branch 2: From the pre-constructed domain knowledge graph, retrieve the entities, relationships, and attribute facts associated with the search query through graph query or semantic matching, as the second candidate evidence set.
3. The large-model hallucination suppression method integrating RAG and knowledge graph as described in claim 2, characterized in that, The step of retrieving the Top-K document fragments that are semantically related to the search query from an external document library as the first candidate evidence set includes: The search query is encoded into a query vector; Calculate the similarity between the query vector and the vectors of all document fragments in the external document library; Based on the similarity ranking, the top K document fragments are selected as the first candidate evidence set.
4. The large-model hallucination suppression method integrating RAG and knowledge graph as described in claim 2, characterized in that, The step of retrieving entities, relationships, and attribute facts associated with the search query from a pre-constructed domain knowledge graph through graph query or semantic matching, as the second candidate evidence set, includes: The search query is input into a pre-trained entity linking model to identify the target entity in the domain knowledge graph corresponding to the search query. Centered on the target entity, perform a one-hop or multi-hop graph traversal in the domain knowledge graph to obtain neighboring entities, connection relationships, and attribute values that are directly or indirectly connected to the target entity. The second candidate evidence set is constructed based on the target entity, the neighboring entities, the connection relationships, the attribute values, and the graph structure between them.
5. The large-model hallucination suppression method integrating RAG and knowledge graph as described in claim 2, characterized in that, The cross-validation and evidence alignment of the first candidate evidence set and the second candidate evidence set includes: Extract named entities and key relationships from document fragments in the first candidate evidence set; Based on the associated entities, relationships, and attribute facts, a structured evidence subgraph of the second candidate evidence set is generated; Align the named entities and key relationships with the nodes and edges in the structured evidence subgraph; Based on the alignment results, information units that are mutually supportive, mutually contradictory, and / or exist independently in the first candidate evidence set and the second candidate evidence set are identified.
6. The large-model hallucination suppression method integrating RAG and knowledge graph as described in claim 5, characterized in that, Aligning the named entities and key relationships with the nodes and edges in the structured evidence subgraph includes: Calculate the name similarity and type matching degree between the named entity and the entity nodes in the structured evidence subgraph; For entity pairs whose name similarity and type matching degree are both higher than a preset threshold, an alignment mapping is established; For the entity pairs that establish the alignment mapping, determine the semantic consistency between the key relationship connecting them and the corresponding edge in the structured evidence subgraph; The alignment result is obtained based on the semantic consistency judgment result.
7. The large-model hallucination suppression method integrating RAG and knowledge graph as described in claim 1, characterized in that, The construction of an enhanced context with confidence labels based on the results of the cross-validation and evidence alignment includes: For information units that support each other in the first and second candidate evidence sets, increase the confidence level of that information unit; For contradictory information units in the first candidate evidence set and the second candidate evidence set, arbitration shall be conducted based on the authority and timeliness of the information unit in the first candidate evidence set and the second candidate evidence set. For information units that exist independently in the first and second candidate evidence sets, the original confidence level of the information unit is retained.
8. The large-model hallucination suppression method integrating RAG and knowledge graph as described in claim 1, characterized in that, After inputting the user query and the enhanced context into the large language model so that the large language model generates an answer based on the enhanced context, the method further includes: From the answer, the source identifier of the cited evidence is extracted; Based on the evidence source identifier, the corresponding original evidence content and corresponding confidence level are retrieved from the enhanced context; The answer, the evidence source identifier, and the corresponding confidence level are combined to generate the final verifiable answer output.
9. The large-model hallucination suppression method integrating RAG and knowledge graph as described in claim 1, characterized in that, The method further includes: The large language model is used to extract information from new domain documents to obtain candidate fact triples; The candidate fact triples are checked for consistency with the existing knowledge in the domain knowledge graph. The candidate fact triples that pass the verification are merged into the domain knowledge graph based on their association strength with other entities, and the external document library is updated.