A pluggable knowledge graph post-processing refining method and system

By implementing steps such as entity cleaning, type normalization, entity merging, and relationship deduplication, the problems of noisy entities, entity splitting, and redundant relationships in knowledge graphs are solved, improving graph quality and question-answering performance, and enabling low-cost pluggable deployment and effect propagation.

CN123117301APending Publication Date: 2026-11-17THE 54TH RESEARCH INSTITUTE OF CHINA ELECTRONICS TECHNOLOGY GROUP CORPORATION
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611257460.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-19
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies suffer from problems such as noisy entities, entity splitting, redundant relationships, and poor connectivity when constructing knowledge graphs, which leads to a decline in retrieval and generation quality. Furthermore, existing cleaning methods cannot trace decision-making processes, are difficult to deploy at low cost on existing systems, and fail to transfer improvements to the query stage.

Method used

A pluggable knowledge graph post-processing refinement method is adopted. Through entity hygiene cleaning, type normalization, entity merging, relation deduplication and connectivity enhancement, deterministic rules and large language models are used in collaboration to record the decisions at each level and put them into intermediate graph snapshots to ensure that the refinement results are transmitted to the query stage.

Benefits of technology

It significantly improves the structural quality of knowledge graphs and end-to-end question-answering performance, reduces the proportion of isolated nodes, compresses redundant descriptions, improves generation and retrieval quality, and enables low-cost pluggable deployment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123117301A_ABST
    Figure CN123117301A_ABST
Patent Text Reader

Abstract

This invention discloses a pluggable post-processing refinement method and system for knowledge graphs. Addressing the issues of noisy entities, multiple nomenclature splits, redundant relations, and poor connectivity in knowledge graphs constructed using large language models, this method sequentially performs entity hygiene cleaning, type normalization, entity merging, relation deduplication and rewriting, and connectivity enhancement on the original graph. Each stage is decided collaboratively by deterministic rules and the large language model, with security constraints such as failure closure, conflict voting interception, and cross-type secondary confirmation applied. Each decision is written to a traceable ledger and a snapshot of the intermediate graph is stored on disk, supporting sampling auditing and layer-by-layer ablation attribution. The alias metadata generated by merging is used to reconstruct the entity vector library and performs alias redirection during queries. This method only connects to the graph storage and vector library interface, without modifying the core logic of upstream extraction and downstream question answering, significantly improving question answering accuracy and retrieval recall, reducing illusion rate, and eliminating context overrun failures caused by relation redundancy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and knowledge graph technology, and in particular to a pluggable knowledge graph post-processing refinement method and system, which can be used to optimize the quality of knowledge graphs automatically constructed from large language models in applications such as knowledge graph-based retrieval enhancement generation (GraphRAG) and intelligent question answering. Background Technology

[0002] Retrieval augmentation generation techniques alleviate the knowledge obsolescence and illusion problems of large language models by retrieving relevant information from external knowledge sources before generating answers. Knowledge graph-based retrieval augmentation generation (GraphRAG) further organizes unstructured text into entity-relation graphs to support multi-hop reasoning and global question answering. Such systems typically segment documents into text chunks, extract entities and relations from each chunk using a large language model, and then merge the extraction results from each chunk into a knowledge graph.

[0003] However, the original knowledge graph obtained by automatically extracting and directly merging from large language models has systematic quality defects: (1) Noise entities: Non-entity text such as chapter titles, garbled characters, complete sentences, enumeration strings connected by commas or pauses are mistakenly extracted as entity nodes; the number of entity type labels is out of control and they overlap with each other, and the same type of object is labeled as different types in different blocks.

[0004] (2) Entity splitting: Different names of the same real object—aliases, courtesy names, titles, etc.—are extracted into independent nodes in different blocks. For example, in Chinese historical texts, "Xuande" and "Liu Bei", "Mengde" and "Cao Cao" refer to the same person, but they form multiple independent nodes in the graph; even if these nodes are occasionally connected by related edges, the facts surrounding the same object are still stored in each node in a scattered manner. Retrieving information by one of the names cannot aggregate the complete information of the object, and the multi-hop reasoning link breaks at the name switching point.

[0005] (3) Relation redundancy: When the same pair of entities is extracted into relations multiple times in different blocks, multiple relation descriptions are simply spliced ​​together and accumulated on the same edge, forming a large number of parallel descriptions with repeated content, which evolve over time and are even contradictory. In a measured original graph, edges containing multiple spliced ​​descriptions accounted for 19.7% of all edges.

[0006] (4) Poor connectivity: A large number of entity nodes have no associated edges (the measured proportion of isolated nodes in the original graph is 16.02%), the connected components are highly fragmented, and the largest connected component only covers 81.8% of the nodes, which makes it impossible for the retrieval of the dependent graph to reach the relevant evidence.

[0007] The aforementioned defects propagate along the chain of "graph quality → retrieval quality → generation quality" as actual technical faults and performance degradation: First, when asking questions by aliases, vector retrieval hits information-poor split nodes, and the correct subgraph cannot be recalled; Second, redundant parallel descriptions cause the context assembled in the retrieval stage to expand rapidly—especially when split entities are merged and multiple parallel descriptions are clustered at the same node. In actual testing with a 16k context window answering model, 15 out of 302 lines of question-answering evaluation failed due to excessively long prompts, constituting a direct systemic failure, indicating that entity merging must be performed in conjunction with relation deduplication; Third, the retrieval context is noisy and the evidence is verbose, increasing the illusion rate of generated answers.

[0008] To address the aforementioned problems, the existing technology has the following shortcomings: (1) The one-time map cleaning method directly modifies the map data on the spot, leaving no trace of the decision-making process, making it impossible to audit the accidental deletion or merging, unable to locate the source of quality improvement, and unable to roll back after an error. (2) The coverage of single methods is insufficient: pure rule methods cannot handle judgments that require semantic understanding (such as judging whether two titles refer to the same person), while pure large language model methods are uncontrollable in decision-making, easily affected by single output noise, and have high costs. (3) The cleaning process is usually deeply coupled with specific extraction or retrieval pipelines, making it difficult to deploy at low cost on existing systems; and the cleaning only applies to offline graph data. After merging entities, the old names cannot be found in the query stage, and the improvement in graph quality cannot be passed on to the final question-and-answer effect.

[0009] Therefore, there is a need for a knowledge graph post-processing refining method that allows for decision traceability, collaboration between rules and large language models, and can be deployed in a pluggable manner to pass the refining effect to the query stage. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of the prior art and provide a pluggable knowledge graph post-processing refinement method and system. Without changing the upstream entity relationship extraction and downstream graph retrieval question-answering processing logic, the original knowledge graph constructed by the large language model is sequentially subjected to entity hygiene cleaning, entity type normalization, entity merging, relationship deduplication and rewriting, and connectivity enhancement. Each layer of decision adopts deterministic rules and works in conjunction with the large language model and applies safe execution constraints. All decisions are written into a traceable ledger and stored as an intermediate graph snapshot. The refinement results are transmitted to the query stage through alias metadata and vector library reconstruction.

[0011] To achieve the above objectives, the technical solution adopted by this invention is: a pluggable knowledge graph post-processing refinement method, comprising the following steps: S1. Obtain the original knowledge graph: Obtain the original knowledge graph constructed by extracting entities and relations from text blocks using a large language model. The nodes in the original knowledge graph represent entities and carry entity name, entity type, description text and source block identifier, while the edges represent relations between entities and carry relation description text. S2. Physical Cleaning: According to a preset set of deterministic rules, the nodes in the original knowledge graph are classified into three categories: deletion candidates, review candidates, or retention candidates. Delete candidate nodes are deleted directly. For review candidate nodes, their names, types, descriptions, and associated edge information are submitted to the large language model for review. The large language model outputs the retention, deletion, or splitting decision and the corresponding confidence and evidence. The review decision is checked for structural legality and executability. Only the deletion or splitting decision that passes the check is executed. The decision that fails the check is closed and retained, resulting in the first intermediate graph. S3. Entity type unification: Identify nodes in the first intermediate graph whose entity type does not belong to the predefined type system, submit them to the large language model in batches for reclassification into the predefined type system, update the entity type attributes of the nodes, and obtain the second intermediate graph. S4. Entity Merging: Based on the vector representation of entity names and description texts, nearest neighbor segmentation is performed on the entities in the second intermediate graph to generate candidate entity pairs; the candidate entity pairs are selected for arbitration according to a preset rule set, and submitted to the large language model for arbitration to determine whether they refer to the same object and the standard name; the entity pairs that pass the arbitration are merged using a disjoint-set data structure to perform transitive merging. During merging, the surviving node inherits the name, alias, and historical merging record of the merged node as alias metadata, and the associated edges of the merged node are migrated to the surviving node. The above candidate generation, arbitration, and merging are iteratively executed until the preset convergence condition is met to obtain the third intermediate graph; S5. Relationship deduplication and rewriting: Scan the redundant relationship descriptions between the same pair of entities in the third intermediate graph, which are composed of multiple descriptions. Submit the multiple descriptions to the large language model for aggregation and rewriting into a single consistent description. Replace the original relationship description with the rewritten result to obtain the fourth intermediate graph. S6. Connectivity Enhancement: Identify low-degree nodes in the fourth intermediate graph whose degree is not higher than a preset threshold; for each low-degree node, retrieve candidate entities by vector similarity through the semantic channel, and retrieve candidate entities by source block co-occurrence and graph neighborhood expansion through the structural channel. After merging and deduplication, submit the low-degree nodes and candidate entities to the large language model to determine whether there is a relationship and the relationship description and confidence. Filter the determination results by confidence and add them as new edges; iterate until the preset convergence condition is met to obtain a refined knowledge graph. The method reads the original knowledge graph and writes it back to the refined knowledge graph via a graph storage interface, without changing the processing logic of the upstream entity relationship extraction process and the downstream graph retrieval question answering process, thereby achieving pluggable deployment.

[0012] Furthermore, the deterministic rule set mentioned in step S2 includes structural general rules, stylistic rules configured and enabled according to the corpus profile, and protective rules, wherein: The structural general rules are used to identify at least one of the following: empty names, garbled characters, chapter titles, sentence-like names, names enumerated with commas or semicolons, and excessively long names. The stylistic rules are used to identify name patterns in the stylistics of the adapted corpus that are easily misextracted as entities, including at least one of the following: extracted poetic lines or commentary statements, specific types of single-character names, and colloquial general terms. The matched nodes are determined as deletion candidates or review candidates. The protective rule excludes nodes that match the protection conditions from the deletion candidates. The protection conditions include at least one of the following: single-character names of people and titles of works enclosed in quotation marks. The executability verification in step S2 includes: verifying whether the deleted node meets the protection conditions for the deletion decision, and verifying whether the split components have source evidence to support the split decision; When splitting, edge routing is performed based on whether the text content of the associated edge mentions the split component: associated edges that mention a certain split component are directed to that component and retain their original edge weights; associated edges that do not mention any split component are migrated to all split components and their original edge weights are evenly distributed according to the number of split components.

[0013] Furthermore, the preset rule set mentioned in step S4 is a rule set consisting of multiple IF-THEN rules learned from training samples or manually configured. Each rule is conditional on at least one of the following: name editing similarity, name vector similarity, description vector similarity, name mutual inclusion, and entity type features of candidate entity pairs. If a candidate entity pair satisfies any one of the rules, it will enter the arbitration of the large language model; if it does not satisfy any of the rules, it will be automatically rejected. There is no automatic merging channel to bypass the arbitration of the large language model.

[0014] Further, the large language model arbitration in step S4 includes: routing the connected components consisting of candidate entity pairs to be arbitrated by scale, wherein: For components whose number of members does not exceed a preset size threshold, their candidate entity pairs are submitted to the large language model for arbitration one by one; Components exceeding the aforementioned scale threshold are arbitrated using a clustering approach: each focal entity within a component is processed sequentially, and each time the focal entity, along with its candidate entities, is submitted to the large language model. The large language model then determines whether each candidate entity refers to the same object as the focal entity and selects a canonical name.

[0015] Furthermore, the clustering arbitration method also includes historical expansion through merging, conflict vote interception, and cross-type merging secondary confirmation, wherein: The merging history expansion is as follows: before each submission of the large language model, the candidate entity set of the focus entity is expanded to the union of the original candidate entity and all members of the disjoint set cluster where each original candidate entity is currently located, and a preset upper limit is imposed on the number of expanded candidate entities; since each focus entity within a component is processed serially, the merging results of the previous focus entities can be seen through the expansion of the focus entities in the later processing, and the canonical name selected by the large language model must belong to the members of the current cluster, thereby preventing the canonical name drift in multiple rounds of merging; The conflict voting interception is as follows: for each large language model arbitration result, the number of votes that should be merged and the number of votes that should be separated for candidate entity pairs are accumulated. Before merging the entity pairs that have passed the arbitration into the disjoint set, the entity pairs consisting of the two cluster members to be merged are traversed. If the number of votes that should be separated for any entity pair is more than the number of votes that should be merged, then the current merger is intercepted. The cross-type merging secondary confirmation is as follows: when the entity types at both ends of the entity pair to be merged are different, an independent large language model confirmation is triggered. The confirmation request carries the member summary information of the clusters to which both ends belong. The entity pair that is confirmed to be rejected is written into the prohibited merging set, and merging will not be attempted in subsequent rounds.

[0016] Furthermore, in step S6, the semantic channel selects the top K non-neighbor entities with the highest cosine similarity to the joint vector of the name and description of the low-degree node; the structural channel selects the top K non-neighbor entities that share the most source blocks with the low-degree node, and optionally expands the one-hop neighbors of the shared entities on the graph as supplementary candidates. The confidence-based filtering in step S6 means retaining only the judgment results whose confidence is not lower than a preset threshold. The preset convergence condition is that no new edge is added in a single round or the maximum number of iteration rounds is reached.

[0017] Furthermore, the method also includes a query-side connection step: after obtaining the refined knowledge graph, an entity vector library is reconstructed based on the refined knowledge graph. During reconstruction, the node's name, alias metadata, and description text are used together as vectorized content. When receiving a query request, if the entity name hit by the vector retrieval does not exist in the refined knowledge graph but exists in the alias metadata of a surviving node, the hit is redirected to the surviving node before graph retrieval question answering is performed; otherwise, the hit is discarded. The alias metadata includes an alias list and a merge source list. In step S4, each time a merge occurs, the name of the node to be merged and its existing alias list, the merge source list and the surviving node's own alias list, and the merge source list are merged and deduplicated. After multiple rounds of merging, a complete alias chain is formed, so that any historical name appearing in the query can be redirected to the surviving node.

[0018] Furthermore, it also includes a ledger recording step: during the execution of steps S2 to S6, each decision of each layer, along with its input, judgment basis, confidence level, and state before and after execution, is appended to the corresponding traceable ledger file of that layer, and after the execution of each layer is completed, the intermediate graph snapshot of the output of that layer is persistently saved. The method also includes performing decision sampling audits based on the traceable ledger and the intermediate graph snapshots, and running downstream question-answering evaluations by replacing the refined knowledge graph with any layer of intermediate graph snapshots to obtain the ablation attribution results for that layer, wherein the ablation attribution does not require re-performing the large language model refinement calculation.

[0019] This invention also provides a pluggable knowledge graph post-processing and refining system, comprising: a graph acquisition module for acquiring the original knowledge graph constructed by extracting entities and relations from text blocks using a large language model; an entity sanitation module for classifying nodes into three categories—deletion candidates, review candidates, or retention candidates—according to a preset set of deterministic rules; deleting or splitting review candidate nodes after review by the large language model and executability verification; and retaining nodes that fail verification; a type normalization module for batch reclassifying nodes whose entity types do not belong to the predefined type system into the predefined type system using the large language model; and entity merging. The system comprises several modules: a module for generating candidate entity pairs based on vector nearest neighbor blocks, filtering them according to a preset rule set, submitting them to a large language model for arbitration, and performing transitive merging using a disjoint-set data structure, with surviving nodes inheriting alias metadata during merging; a relation deduplication module for aggregating and rewriting multiple redundant relation descriptions between the same pair of entities into a single consistent description using a large language model; and a connectivity enhancement module for retrieving candidate entities from low-degree nodes through semantic and structural channels, judging them using a large language model, filtering them by confidence, and iteratively supplementing new edges. The system is connected to the upstream entity relation extraction process and the downstream graph retrieval question-answering process via a graph storage interface without changing their processing logic.

[0020] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described pluggable knowledge graph post-processing refinement method.

[0021] Compared with the prior art, the beneficial effects of the present invention are: (1) The quality of the graph structure has been significantly improved: In the actual test of the full text of "Romance of the Three Kingdoms", the proportion of isolated nodes decreased from 16.02% to 0.19%, the number of connected components decreased from 1036 to 53, and the coverage of the maximum connected component increased from 81.8% to 97.5%; the proportion of edges with redundant parallel descriptions decreased from 19.7% to 0%, and the total number of words in relation descriptions was reduced by 58.7%; the nodes deleted in the entity hygiene cleanup during the review of the large language model accounted for only 0.39% of the total number of nodes in the original graph, which is conservative and controllable.

[0022] (2) Significantly improved end-to-end question answering performance: In the 302-line richly labeled question answering evaluation, the refined graph outperformed the original graph by 35 questions (net win rate of 11.6%, McNemar exact test p=2.9×10⁻). 5 95% confidence interval [+19, +51]); In the multi-jump bottleneck question type, the original graph only answered 28 questions correctly, while the refined graph answered 46 questions correctly; After changing to a stronger 32 billion parameter answering model, the net win was 36 questions (p=1.2×10⁻). 6 The returns are robust to the strength of the answer model.

[0023] (3) Improved generation quality and efficiency: The rate of answer illusion decreased from 27.4% to 20.3%; the length of evidence required to hit the same facts decreased from an average of 161.1 characters to 124.2 characters; the fault of excessively long prompt words caused by relation redundancy was eliminated - in the intermediate graph where only entity merging was done without relation deduplication, 15 out of 302 lines of question-and-answer evaluation failed due to excessively long prompt words, and after performing relation deduplication, there were 0 lines.

[0024] (4) Improved retrieval quality and closed transmission chain: The recall rate of multi-hop alias question type increased from 0.756 to 0.934, and the alias redirection hit rate on the query side increased from 0 to 0.982. The improved graph quality is effectively transmitted to retrieval and question answering through alias metadata.

[0025] (5) Fully auditable and attributable: Each layer of ledger supports decision sampling audit; based on ledger and intermediate graph snapshots, layer-by-layer ablation attribution can be completed without rerunning the large language model refinement. In actual testing, the entity merging layer has the largest marginal contribution among the four layers (net win +21 questions), and the contribution attribution of each layer is clear.

[0026] (6) Pluggable deployment: It can be deployed on existing GraphRAG systems at low cost by connecting only through the standard graph storage and vector library interface without changing the upstream extraction logic and the core logic of downstream retrieval and question answering. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the overall process of the method of the present invention; Figure 2 A schematic diagram of the decision-making process for physical sanitation cleaning (step S2); Figure 3 This is a schematic diagram of the entity merging process (step S4); Figure 4 A schematic diagram of the iterative repair process for connectivity enhancement (step S6); Figure 5 This diagram illustrates the ledger and intermediate graph data structures, as well as the redirection of aliases on the query side. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are only used to illustrate the technical solutions of the present invention and do not constitute a limitation on the scope of protection; although the embodiments use Chinese classical novel corpus, the present invention is not limited to corpus of any specific language or field. This embodiment uses the complete 120 chapters of the classic Chinese novel *Romance of the Three Kingdoms* as its corpus and implements the method on the open-source GraphRAG system. The corpus is divided into text blocks of approximately 1200 tokens. Entities and relations are extracted and merged block by block using a large language model (in this embodiment, Qwen2.5-32B-Instruct is used for graph construction and each refining step, the same below), resulting in the original knowledge graph G0: 6122 entity nodes and 10981 relation edges, including 981 isolated nodes (16.02%), 1036 connected components, with the largest connected component covering 81.8% of the nodes; 2162 edges containing multi-segment concatenation descriptions (19.7%). For example... Figure 1 As shown, the method includes the following steps.

[0029] S1. Obtain the original knowledge graph. Read G0 via the graph storage interface. Node attributes include entity name, entity type (entity_type), description text (description, multiple sources concatenated with a separator), and source block identifier (source_id); edge attributes include relation description, weight, and source block identifier.

[0030] S2, Physical cleaning (such as...) Figure 2(As shown). The deterministic rule set is divided into three categories: structural general rules (empty names, garbled characters, chapter titles such as "Chapter Thirty-Eight", sentence-like names containing pauses, names enumerated with commas such as "Guan Yu, Zhang Fei, Zhao Yun", and excessively long names); stylistic rules configured and enabled according to the corpus profile (this embodiment enables the classical vernacular profile: extracted poems / commentary statements are judged as deletion candidates, and single-character weapon and mount names are judged as review candidates; rules corresponding to the corresponding stylistics such as colloquial general terms can be configured under other corpus profiles); and protective rules (single-character names of the PERSON type and work names enclosed in paired quotation marks are not included in the deletion candidates). In this embodiment, the rule set marks a total of 215 candidate nodes, of which 41 deletion candidates are directly deleted, and 174 review candidates are submitted to the large language model for review. The review prompt carries the node name, type, description, and a summary of up to 8 related edges, requiring the output of the retention / deletion / splitting decision, confidence level, and textual evidence. Execution feasibility verification for decisions: Deletion decisions will not be executed if they remove protected single-character place names or regime names; splitting decisions will not be executed if they fragment complete event sentences, lack supporting input evidence for split components, or split single-character name fragments; decisions that fail verification are closed and retained. During splitting execution, the original node's associated edges are migrated according to the edge routing strategy: if the associated edge text mentions a certain split component, it is migrated to that component and the original edge weight is retained; if it does not mention any component, it is migrated to all split components and the edge weight is evenly distributed according to the number of components. In this embodiment, the review result is 130 retained, 24 deleted, 17 split, 3 retained and closed due to failure; the 24 deleted nodes account for only 0.39% of the total number of G0 nodes. Each rule hit and review decision is written to the first-level ledger.

[0031] S3. Entity Type Unification. A predefined type system is {PERSON, LOCATION, ORGANIZATION, EVENT, CONCEPT, OBJECT, TIME, OTHER}. Nodes in the graph whose entity type does not belong to this system are scanned (dozens of free type labels are often generated during the large language model extraction phase). 20 entities are submitted to the large language model for reclassification in batches. Built-in boundary rules are used for prompt words (e.g., official titles belong to CONCEPT, political power to ORGANIZATION, year names to TIME, books to OBJECT). Entities difficult to determine are classified as OTHER. The classification results are updated to the node attributes and written to the ledger, resulting in the second intermediate graph.

[0032] (a) Candidate generation: For each entity, a vector representation is calculated using the text "name + description" (this embodiment uses the BGE-M3 vector model). The top 20 nearest neighbors of each entity are selected according to cosine similarity to form a set of candidate entity pairs (nearest neighbor block) to avoid full pairwise comparisons. (b) Rule selection: For each candidate pair, calculate features such as name edit similarity, name vector similarity, description vector similarity, name mutual inclusion, and entity type similarity and difference, and substitute them into multiple IF-THEN rules learned from the training samples; candidate pairs that satisfy any rule enter the arbitration pool of the large language model, and those that do not satisfy any rule are automatically rejected; there is no automatic merging channel, and all merging must be confirmed by the large language model to prevent rule misjudgment from directly causing incorrect merging; (c) Large Language Model Arbitration: Connected components consisting of pairs to be arbitrated are routed by size—smaller components with no more than 3 members are submitted to the large language model in batches, outputting whether they are the same object, the canonical name, and the confidence level; larger components are arbitrated using clustering, with each focal entity within the component processed serially and a merge history expansion performed: before each submission, the candidate entity set of the focal entity is expanded to the union of the original candidate entity and all members of the disjoint-set cluster where each original candidate entity currently resides, and an upper limit is imposed on the number of expanded candidates (30 in this embodiment); the focal entity along with the expanded candidate entities are submitted to the large language model, determining whether each candidate refers to the same object as the focal entity and selecting the canonical name from the cluster members (the canonical name must belong to the current cluster member). Due to the serial processing, the merge results of the previous focal entity can be seen by the later focal entity through expansion, thus avoiding canonical name drift in multiple rounds of merging. For example, for the candidate pair ("Mengde", "Cao Cao"), the model determines that they are the same person based on evidence such as "Zi Mengde" and "Cao Mengde" in the description, and the canonical name is "Cao Cao". Two security constraints are imposed in the clustering arbitration: each arbitration result simultaneously accumulates the "should be merged / should be separated" votes for each entity pair; before merging the entity pairs that pass the arbitration into the ledger disjoint-set data structure, the entity pairs consisting of the members of the two clusters to be merged are traversed; if the "should be separated" votes of either pair are more numerous, the merger is blocked (conflict voting interception, resisting single-output noise of the large language model); when the entity types at both ends of the pair to be merged are different, an independent secondary confirmation by the large language model is triggered, and the confirmation request carries the summaries of the cluster members on both sides; the rejected pair is written into the permanent forbidden set (cross-type secondary confirmation); after the full component processing is completed, the merger proposal of each member to the final canonical name is derived from the final state of the ledger disjoint-set data structure. (d) Transitive merging: Summarize the merged pairs that have passed through both pairwise arbitration and cluster arbitration, and perform transitive merging in descending order of confidence using a disjoint-set data structure. Entity pairs whose two ends are already in the same cluster are automatically skipped. (e) Alias ​​metadata inheritance: During the merge execution, the description and source identifier of the merged node are incorporated into the surviving node, and its associated edges are migrated to the surviving node; the surviving node inherits the name of the merged node and its existing aliases / merged_from list after merging and deduplication, so that the alias chain formed by multiple rounds of merging such as "Xuande→Liu Bei" is completely preserved; (f) Iteration: Repeat (a)–(e) until no new merges are added in a single round or the maximum number of rounds is reached (3 rounds in this example).

[0033] This embodiment performed a total of 1214 merges, reducing the number of nodes from 6045 to 4831. For each merge, the names of both parties, the canonical name, the confidence level, the decision source, and the states before and after the merge were written into the second-level ledger. Using a self-built alias gold standard for comparison and evaluation, the recall rate for evaluable gold standard pairs in the ledger was 78.3%, and the merge precision on the gold standard coverage subset was 97.0%.

[0034] S5. Relationship Deduplication and Rewriting. Scan edges between the same pair of entities whose descriptions contain multiple segments concatenated by separators (2162 in G0, accounting for 19.7%), and submit them to the large language model for aggregation and rewriting in batches. Prompt words distinguish three cases: pure duplicate descriptions are deduplicated and merged; time-evolutionary descriptions are synthesized chronologically; and truly contradictory descriptions retain disagreements without arbitrary rulings, and adding information beyond the original description is strictly prohibited. In this embodiment, 2206 edge descriptions (including newly generated concatenations from S4) were rewritten, all successfully. The total number of characters in the descriptions was compressed from 183599 to 75734 (compression of 58.7%), and the number of edges containing multiple concatenated descriptions after rewriting was 0. The descriptions before and after each rewrite, the number of characters, and the compression ratio are written to the third-level ledger.

[0035] S6, enhanced connectivity (e.g.) Figure 4 (As shown). Iterative graph repair is employed: a set of low-degree nodes with a degree not exceeding a threshold k (k=0 in this embodiment, i.e., isolated nodes) is selected; for each low-degree node, the semantic channel selects the top 20 non-neighbor entities with the highest cosine similarity of the joint vector of name + description, and the structural channel selects the top 20 non-neighbor entities with the most shared source block times, and expands the graph one-hop neighbors of these shared entities as supplementary candidates; after merging and deduplicating the three candidate paths, they are submitted to the large language model in batches to determine whether there is a relationship between the low-degree node and each candidate entity, and to provide a relationship description and confidence level; only relationships with a confidence level not lower than the threshold τ (τ=0.6 in this embodiment) are retained, and each node is supplemented with at most 5 edges; after supplementing the edges, the set of low-degree nodes is recalculated, and the iteration continues until no new edges are added in a single round or a maximum of 3 rounds are reached. In this embodiment, the number of isolated nodes is reduced from 981 to 9 (16.02%→0.19%), the number of connected components is reduced from 1036 to 53, and the maximum connected component coverage is reduced from 81.8% to 97.5%. Each edge-filling determination is written into the fourth-level ledger.

[0036] Ledger records and intermediate graph snapshots. Steps S2–S6 use JSONL format for each layer's ledger, with one decision record per line. Fields include layer identifier, decision object, judgment criteria, confidence level, and state before and after execution. After each layer's execution, the current graph is persisted as an intermediate graph snapshot file (G1a / G1b / G2 / G3 / G4). The ledger and snapshots support three uses: decision sampling auditing (in this example, 61 manually reviewed entries are sampled from the second-layer ledger, and all deletion decisions from the first layer are fully reviewed); replacing the final graph with any intermediate graph snapshot to run downstream evaluations, achieving layer-by-layer ablation attribution without rerunning large language models; and version recovery and offline backtracking based on intermediate graph snapshots.

[0037] Query-side connections (such as...) Figure 5 (As shown). After refinement, the entity vector library is cleared and rebuilt according to the refined graph G4. Each node uses "name + alias metadata (alias list and merged source list after deduplication) + description" as vectorized content, so that old names can still be found by vector retrieval. During querying, if the entity name that is found no longer exists in G4, but exists in the aliases or merged_from list of a surviving node, the hit is redirected to that surviving node before subgraph retrieval and answer generation are performed; otherwise, the hit is discarded. In this embodiment, the redirection hit rate of alias multi-hop questions increases from 0 to 0.982.

[0038] The above refining steps S1–S6 and ledger records only read and write data through the graph storage interface and vector library interface. The core logic of upstream extraction logic and downstream retrieval and question answering does not need to be changed. Only alias redirection is attached at the retrieval result parsing, which reflects the pluggable characteristics.

[0039] like Figure 1 As shown, this embodiment provides a system corresponding to the method of Embodiment 1, including: The graph acquisition module is used to read the original knowledge graph constructed from the large language model via the graph storage interface; The physical hygiene cleaning module is used to perform deterministic rule three-classification, large language model review, executability verification and failure closure and retention; The type unification module is used to reclassify entities that exceed the predefined type system in batches; The entity merging module is used for generating nearest neighbor block candidates, filtering rules, arbitrating large language models, and transitive merging of disjoint-set data. It includes a conflict voting interception unit and a cross-type secondary confirmation unit, and performs alias metadata inheritance. The relation deduplication module is used for the aggregation and rewriting of multiple redundant relation descriptions. The connectivity enhancement module is used for dual-channel candidate retrieval of low-degree nodes, relation determination of large language models, confidence filtering, and iterative edge completion.

[0040] Furthermore, the system may also include: a ledger recording module for writing JSONL ledger data for decisions made by each module and for persisting intermediate graph snapshots to disk; and a query connection module for reconstructing the entity vector library according to the refined graph and performing alias redirection during queries. Both modules are enabled in this embodiment.

[0041] Each module can be deployed on the same computing device or in a distributed manner, and data can be transferred between modules via graph storage interface and vector library interface.

[0042] Under the corpus and evaluation configuration of Example 1 (302-line richly annotated question-answering evaluation, Qwen2.5-7B-Instruct answering model, 8k tokens for answering context budget, large language model judging the correctness of the answer and verifying it with conservative reordering), the present invention achieves the following results: (1) End-to-end question and answer: 243 rows of the refined graph G4 were answered correctly, and 208 rows of the original graph G0 were answered correctly, resulting in a net win of +35 (McNemar exact test p=2.9×10⁻). 5 Self-help method 95% confidence interval [+19,+51]); also known as multi-jump bottleneck question type (56 questions), G0 28 questions answered correctly, G4 46 questions answered correctly; after changing to a 32 billion parameter answering model, net win +36 (p=1.2×10⁻ 6 This indicates that the returns are not exclusive to weak models.

[0043] (2) Layer-by-layer elimination (based on the intermediate ledger graph, without rerunning the refinement): only the first layer has a net win of +3 (not significant); after adding the entity merging layer, the net win is +24 (p=0.0027), with a marginal contribution of +21, the highest among the four layers; after adding the relation deduplication layer, it is +28; with all four layers complete, it is +35. The contribution attribution of each layer is clear, verifying the effectiveness of the layered design and ledger mechanism.

[0044] (3) Generation quality: The sentence-by-sentence fidelity audit showed that the illusion rate (the percentage of sentences without supporting evidence) decreased from 27.4% to 20.3%; the average length of evidence required to hit the same facts decreased from 161.1 words to 124.2 words.

[0045] (4) Retrieval and fault elimination: The recall rate of multi-hop alias question type increased from 0.756 to 0.934, the entity recall rate increased from 0.911 to 0.976, and the query-side alias redirection hit rate increased from 0 to 0.982; the 3-hop reachability of the target entity in all 302 rows increased from 0.884 to 0.927, and there was no drop-off in the reachability of any question; 15 / 302 rows in the intermediate graph where only entity merging was performed without relation deduplication failed due to the prompt words exceeding the 16k context, and after performing relation deduplication, there were 0 rows.

[0046] (5) Controllability: The first-level large language model has a review deletion rate of only 0.39% (24 / 6122), and all review deletion and splitting decisions have been manually reviewed and no erroneous deletions were found; the entity merging accuracy on the gold standard coverage subset is 97.0%; all decisions can be traced back to the ledger.

[0047] Industrial applicability

[0048] This invention can be directly deployed in scenarios such as enterprise knowledge base question answering, intelligent analysis of documents and archives, intelligent customer service, and domain knowledge graph construction using the GraphRAG architecture. Due to its pluggable design, existing systems can achieve improvements in graph quality and question answering quality without modifying their extraction and retrieval question answering logic; the ledger mechanism meets the compliance requirements for auditable and traceable automated decision-making processes.

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A pluggable knowledge graph post-processing refinement method, characterized in that, Includes the following steps: S1. Obtain the original knowledge graph: Obtain the original knowledge graph constructed by extracting entities and relations from text blocks using a large language model. The nodes in the original knowledge graph represent entities and carry entity name, entity type, description text and source block identifier, while the edges represent relations between entities and carry relation description text. S2. Physical cleanup: Based on a preset set of deterministic rules, the nodes in the original knowledge graph are classified into three categories: deletion candidates, review candidates, or retention candidates. Delete the candidate node directly; For candidate nodes to be reviewed, their names, types, descriptions, and associated edge information are submitted to the large language model for review. The large language model outputs the decision to retain, delete, or split the node, along with the corresponding confidence level and evidence. The review decision is then checked for structural legality and executability. Only the deletion or split decision that passes the check is executed. Decisions that fail the check are closed and retained, resulting in the first intermediate graph. S3. Entity type unification: Identify nodes in the first intermediate graph whose entity type does not belong to the predefined type system, submit them to the large language model in batches, reclassify them to the predefined type system, update the entity type attributes of the nodes, and obtain the second intermediate graph. S4. Entity merging: Based on the vector representation of entity name and description text, perform nearest neighbor segmentation on the entities in the second intermediate graph to generate candidate entity pairs; Select the pairs to be arbitrated from the candidate entity pairs according to the preset rule set, and submit them to the large language model for arbitration to determine whether they refer to the same object and the specification name; For entity pairs that pass arbitration, transitive merging is performed using a disjoint-set data structure. During merging, surviving nodes inherit the name, alias, and historical merging records of the merged node as alias metadata. The associated edges of the merged node are migrated to the surviving nodes. The above candidate generation, arbitration, and merging are iteratively performed until the preset convergence condition is met, resulting in a third intermediate graph. S5. Relationship deduplication and rewriting: Scan the redundant relationship descriptions between the same pair of entities in the third intermediate graph, which are composed of multiple descriptions. Submit the multiple descriptions to the large language model for aggregation and rewriting into a single consistent description. Replace the original relationship description with the rewritten result to obtain the fourth intermediate graph. S6. Connectivity Enhancement: Identify low-degree nodes in the fourth intermediate graph whose degree is not higher than a preset threshold. For each low-degree node, candidate entities are retrieved by vector similarity through the semantic channel, and by source block co-occurrence and graph neighborhood expansion through the structural channel. After merging and deduplication, the low-degree nodes and candidate entities are submitted to the large language model to determine whether there is a relationship and the relationship description and confidence. The determination results are filtered by confidence and added as new edges. The process is iterated until the preset convergence condition is met to obtain a refined knowledge graph. The method reads the original knowledge graph and writes it back to the refined knowledge graph via a graph storage interface, without changing the processing logic of the upstream entity relationship extraction process and the downstream graph retrieval question answering process, thereby achieving pluggable deployment.

2. The method according to claim 1, characterized in that, The deterministic rule set mentioned in step S2 includes structural general rules, stylistic rules configured and enabled according to corpus profiles, and protective rules, wherein: The structural general rules are used to identify at least one of the following: empty names, garbled characters, chapter titles, sentence-like names, names enumerated with commas or semicolons, and excessively long names. The stylistic rules are used to identify name patterns in the stylistics of the adapted corpus that are easily misextracted as entities, including at least one of the following: extracted poetic lines or commentary statements, specific types of single-character names, and colloquial general terms. The matched nodes are determined as deletion candidates or review candidates. The protective rule excludes nodes that match the protection conditions from the deletion candidates. The protection conditions include at least one of the following: single-character names of people and titles of works enclosed in quotation marks. The executability verification in step S2 includes: verifying whether the deleted node meets the protection conditions for the deletion decision, and verifying whether the split components have source evidence to support the split decision; When splitting, edge routing is performed based on whether the text content of the associated edge mentions the split component: associated edges that mention a certain split component are directed to that component and retain their original edge weights; associated edges that do not mention any split component are migrated to all split components and their original edge weights are evenly distributed according to the number of split components.

3. The method according to claim 1, characterized in that, The preset rule set mentioned in step S4 is a set of rules consisting of multiple IF-THEN rules learned from training samples or manually configured. Each rule is conditional on at least one of the following: name editing similarity, name vector similarity, description vector similarity, name mutual inclusion, and entity type features of candidate entity pairs. If a candidate entity pair satisfies any one of the rules, it will enter the arbitration of the large language model. If it does not satisfy any of the rules, it will be automatically rejected. There is no automatic merging channel to bypass the arbitration of the large language model.

4. The method according to claim 1, characterized in that, The large language model arbitration in step S4 includes: routing the connected components consisting of candidate entity pairs to be arbitrated by scale, wherein: For components whose number of members does not exceed a preset size threshold, their candidate entity pairs are submitted to the large language model for arbitration one by one; Components exceeding the aforementioned scale threshold are arbitrated using a clustering approach: each focal entity within a component is processed sequentially, and each time the focal entity, along with its candidate entities, is submitted to the large language model. The large language model then determines whether each candidate entity refers to the same object as the focal entity and selects a canonical name.

5. The method according to claim 4, characterized in that, The clustering arbitration method also includes historical expansion, conflict voting interception, and cross-type merging secondary confirmation, wherein: The merging history expansion is as follows: before each submission of the large language model, the candidate entity set of the focus entity is expanded to the union of the original candidate entity and all members of the disjoint set cluster where each original candidate entity is currently located, and a preset upper limit is imposed on the number of expanded candidate entities; since each focus entity within a component is processed serially, the merging results of the previous focus entities can be seen through the expansion of the focus entities in the later processing, and the canonical name selected by the large language model must belong to the members of the current cluster, thereby preventing the canonical name drift in multiple rounds of merging; The conflict voting interception is as follows: for each large language model arbitration result, the number of votes that should be merged and the number of votes that should be separated for candidate entity pairs are accumulated. Before merging the entity pairs that have passed the arbitration into the disjoint set, the entity pairs consisting of the two cluster members to be merged are traversed. If the number of votes that should be separated for any entity pair is more than the number of votes that should be merged, then the current merger is intercepted. The cross-type merging secondary confirmation is as follows: when the entity types at both ends of the entity pair to be merged are different, an independent large language model confirmation is triggered. The confirmation request carries the member summary information of the clusters to which both ends belong. The entity pair that is confirmed to be rejected is written into the prohibited merging set, and merging will not be attempted in subsequent rounds.

6. The method according to claim 1, characterized in that, In step S6, the semantic channel selects the top K non-neighbor entities with the highest cosine similarity to the joint vector of the name and description of the low-degree node; the structural channel selects the top K non-neighbor entities that share the most source blocks with the low-degree node, and optionally expands the one-hop neighbors of the shared entities on the graph as supplementary candidates. The confidence-based filtering in step S6 means retaining only the judgment results whose confidence is not lower than a preset threshold. The preset convergence condition is that no new edge is added in a single round or the maximum number of iteration rounds is reached.

7. The method according to claim 1, characterized in that, The method further includes a query-side connection step: after obtaining the refined knowledge graph, an entity vector library is reconstructed based on the refined knowledge graph. During reconstruction, the node's name, alias metadata, and description text are used together as vectorized content. When receiving a query request, if the entity name hit by the vector retrieval does not exist in the refined knowledge graph but exists in the alias metadata of a surviving node, the hit is redirected to the surviving node before graph retrieval question answering is performed; otherwise, the hit is discarded. The alias metadata includes an alias list and a merge source list. In step S4, each time a merge occurs, the name of the node to be merged and its existing alias list, the merge source list and the surviving node's own alias list, and the merge source list are merged and deduplicated. After multiple rounds of merging, a complete alias chain is formed, so that any historical name appearing in the query can be redirected to the surviving node.

8. The method according to claim 1, characterized in that, It also includes the ledger recording step: during the execution of steps S2 to S6, each decision of each layer, along with its input, judgment basis, confidence level and state before and after execution, is appended to the corresponding traceable ledger file of that layer, and after the execution of each layer is completed, the intermediate graph snapshot of the output of that layer is persistently saved. The method also includes performing decision sampling audits based on the traceable ledger and the intermediate graph snapshots, and running downstream question-answering evaluations by replacing the refined knowledge graph with any layer of intermediate graph snapshots to obtain the ablation attribution results for that layer, wherein the ablation attribution does not require re-performing the large language model refinement calculation.

9. A pluggable knowledge graph post-processing and refining system, characterized in that, include: The knowledge graph acquisition module is used to acquire the original knowledge graph constructed by extracting entities and relations from text blocks using a large language model; The physical hygiene cleaning module is used to determine nodes into three categories: deletion candidates, review candidates, or retention candidates according to a preset set of deterministic rules. For review candidate nodes, after review by a large language model and execution verification, they are deleted or split. Decisions that fail the verification are closed and retained. The type normalization module is used to reclassify nodes whose entity types do not belong to the predefined type system into the predefined type system in batches using the large language model; The entity merging module is used to generate candidate entity pairs based on vector nearest neighbor blocks, filter them according to a preset rule set, submit them to the large language model for arbitration, and perform transitive merging using a disjoint-set data structure. During merging, surviving nodes inherit alias metadata. The relation deduplication module is used to aggregate and rewrite multiple redundant relation descriptions between the same pair of entities into a single consistent description through a large language model. The connectivity enhancement module is used to retrieve candidate entities for low-degree nodes through semantic and structural channels, and after being judged by the large language model and filtered according to confidence, new edges are iteratively added. The system connects with the upstream entity relationship extraction process and the downstream graph retrieval question-and-answer process via a graph storage interface without changing their processing logic.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the pluggable knowledge graph post-processing refinement method as described in any one of claims 1 to 8.