A knowledge graph pipeline multi-strategy entity disambiguation incremental alignment method and system
By employing multi-strategy similarity fusion and blocking bucketing techniques, the high complexity and repetitive scanning issues of entity disambiguation and alignment in knowledge graphs are resolved. This achieves efficient entity disambiguation and alignment, supports incremental updates and quality inspection linkage, and improves the processing efficiency and accuracy of the knowledge graph data production pipeline.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU BUSINESS ENTERPRISE HUITONG NETWORK TECHNOLOGY CO LTD
- Filing Date
- 2026-05-26
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies in knowledge graphs rely on a single feature for similarity judgment, resulting in high processing complexity. They also struggle to consider character surface differences, semantic synonyms, and structural context information. Furthermore, full alignment during historical knowledge base updates leads to duplicate scans, making it impossible to link with quality inspection results for targeted processing.
By employing multi-strategy similarity fusion and blocking bucketing techniques, and combining the name, alias, and relational context of candidate entities, blocking keys are generated for entity normalization. Through multi-strategy similarity calculation and clustering disambiguation, incremental alignment is performed only on changed entities, and the results of quality inspection are linked for targeted processing.
It reduces the size of candidate pairs, improves processing efficiency, supports incremental updates of the historical knowledge base, improves the accuracy of entity disambiguation and alignment and system throughput performance, and realizes linkage scheduling with quality inspection.
Smart Images
Figure CN122432352A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of knowledge graph fusion and data engineering technology, specifically to a method and system for entity disambiguation and alignment based on multi-strategy similarity, blocking index, and fingerprint increment in the fusion node of a knowledge graph data production pipeline. Background Technology
[0002] During the continuous construction of knowledge graphs, entities are typically extracted from multi-source text, business systems, and structured data, and deduplication, disambiguation, and alignment are performed at knowledge fusion nodes. Due to differences in naming conventions, abbreviation rules, alias expressions, and field completeness among different data sources, the same real-world object often corresponds to multiple candidate entities. Improper handling can easily lead to duplicate entities, conflicting entities, or incorrect merging.
[0003] Existing technologies typically have the following shortcomings: (1) Similarity judgment relies on a single feature, making it difficult to simultaneously consider character surface differences, semantic synonymy, and structural context information; (2) When the number of candidate entities is large, if a full pairwise comparison is directly used, the processing complexity is O(| |²), which is not conducive to real-time or near-real-time operation in production pipelines; (3) If the full alignment method is still used after the historical knowledge base is updated, it will lead to repeated scanning and affect the system throughput; (4) If the disambiguation and alignment process cannot be linked with the quality inspection results, it will be difficult to carry out targeted processing for abnormal repetition rate scenarios in a timely manner.
[0004] Therefore, there is an urgent need for an entity disambiguation and alignment scheme that can balance processing efficiency, result stability, and incremental update capability in knowledge graph production scenarios. Summary of the Invention
[0005] The purpose of this invention is to provide an entity disambiguation and incremental alignment scheme that can be embedded in the fusion node of the knowledge graph data production pipeline, which reduces the candidate comparison scale while ensuring the disambiguation accuracy, and supports incremental updates to the historical knowledge base and linkage scheduling with quality inspection.
[0006] The technical problems to be solved by this invention are: how to reduce the size of candidate pairs while ensuring the accuracy of disambiguation, how to align only the changed entities with the incremental historical knowledge base, and how to trigger calculations in conjunction with the quality inspection results of the data production pipeline.
[0007] To address the aforementioned technical problems, this invention provides a multi-strategy entity disambiguation incremental alignment method for knowledge graph pipelines. This method is applicable to entity objects deployed in the fusion nodes of knowledge graph data production pipelines and possessing names, aliases, codes, and relational contexts, as illustrated in the accompanying drawings. The method includes: Step S1, Entity Normalization: Based on the attribute definitions of the target ontology class, normalization processing is performed on the name, alias, and identifier attributes of candidate entities, and entity fingerprints are generated based on the identifier attributes (see...). Figure 1 ).
[0008] Step S2, Candidate Pair Generation: Generate a blocking key for each entity in the candidate entity set; the blocking key at least includes a prefix component of the normalized name and an industry code or regional code component, the length of the prefix component being a preset length; generate candidate entity pair sets only within the buckets with the same blocking key to reduce the size of the entity pairs to be compared (see...). Figure 1 , Figure 3 (corresponding to claim 3).
[0009] Step S3, Multi-strategy Similarity Fusion: For candidate entity pairs, calculate character similarity, semantic embedding similarity, and structural context similarity respectively. The structural context similarity is determined based on the set similarity between the neighbor entity sets of the candidate entity in the relation set. Then, weighted by a weight coefficient appropriate to the target ontology class, the similarity scores are fused into a comprehensive similarity score. , + (See Figure 1 Label 10 Figure 2 (corresponding to claims 4 and 5).
[0010] Step S4, Clustering Disambiguation: When the overall similarity score is not lower than the class relevance threshold corresponding to the target ontology class, an undirected similarity graph is constructed on the candidate entity set, connected components are obtained to get candidate entity clusters, and representative entities and cluster confidence are determined within each cluster (see...). Figure 1 , Figure 2 ).
[0011] Step S5, Incremental Alignment: Maintain the fingerprint index of the historical knowledge base and determine the change fingerprint set; the change fingerprint set includes entity fingerprints whose fingerprints are not in the fingerprint index, and entity fingerprints belonging to the conflict set, where the conflict set refers to the set of entity fingerprints that are pending conflict arbitration or require manual review; only the representative entities corresponding to the change fingerprint set are merged, created, or arbitrated with the fingerprint index, the incremental change set and conflict details are output, and the fingerprint index is updated (see...). Figure 1 Label 11 Figure 4 (corresponding to claim 6).
[0012] Preferably, the class relevance threshold can be dynamically adjusted based on the difference between the duplicate score provided by the upstream quality inspection module and the preset duplicate threshold; the similarity calculation module or the clustering disambiguation module performs this dynamic adjustment (see claims 7 and 8).
[0013] When the duplicate score indicates an increased risk of duplication, the linkage scheduling module only triggers incremental alignment for the entity subset corresponding to the changed fingerprint set (see Embodiment 2, Claim 8).
[0014] The present invention also provides a system for implementing the above method and a computer-readable storage medium. The system's normalization module, candidate pair generation module, similarity calculation module, clustering disambiguation module, incremental alignment module, and linkage scheduling module correspond one-to-one with the above steps and are deployed at the knowledge fusion node of the knowledge graph data production pipeline.
[0015] The beneficial effects of this invention are: Compared with the prior art, the present invention has at least the following beneficial effects: (1) Blocking bucketing reduces the candidate pair size from O(| |²) decreases to O(|P|), where | | represents the number of entities in the candidate entity set, and |P| represents the sum of the number of candidate entity pairs in each bucket after blocking key bucketing. |P| is usually much smaller than the sum of the number of candidate entity pairs in each bucket. | The number of pairs of entities in the middle | |(| |−1) / 2; (2) Multi-strategy integration improves the comprehensive judgment ability of abbreviations, aliases and structural associations; (3) Modify fingerprint incremental alignment to reduce the cost of full scanning of the historical database; (4) In conjunction with quality inspection, local entity subsets are processed in a targeted manner when the repetition rate is abnormal. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of the method flow of an embodiment of the present invention. S1 to S5 represent, in sequence, normalization, blocking candidate pairs, multi-strategy fusion, clustering disambiguation, and incremental alignment; S10 represents the multi-strategy fusion calculation module (mainly corresponding to step S3 and claims 4 and 5). Figure 2 ); Label 11 is the module for comparing the changed fingerprint set with the historical fingerprint index (corresponding to step S5 and claim 6, Figure 4 ); Label 12 is the output of incremental change sets and conflict details.
[0017] Figure 2This is a schematic diagram of multi-strategy similarity fusion in an embodiment of the present invention. Label 20 represents entity pairs; labels 21-25 represent the similarity across three paths: character, semantic embedding, and structural context, respectively; labels 22, 24, and 26 represent weighting coefficients; label 27 represents weighted fusion; and label 28 represents a comprehensive similarity score not lower than the class relevance threshold. Edge connection determination (time) The weights and thresholds in the figure are for class-related purposes and correspond to claims 1 and 7. The weights and thresholds in the figure are for illustrative purposes only and do not limit the scope of protection of the claims.
[0018] Figure 3 This is a schematic diagram of blocking key bucketing and candidate pair generation in an embodiment of the present invention. Reference numeral 30 is the candidate entity set; reference numeral 31 is the blocking key bucketing function, which includes a name prefix component and an industry / regional coding component (corresponding to claims 1 and 3, where L_p in the figure is a preset length); reference numerals 32 to 34 are the candidate entity pair sets within each bucket; the comparison scale within a bucket is smaller than the pairwise comparison scale of the entire candidate entity set.
[0019] Figure 4 This is a schematic diagram of incremental fingerprint alignment and local scanning in an embodiment of the present invention. Reference numeral 40 represents the representative entity fingerprint φ(e*); reference numeral 41 represents the historical fingerprint index I_K and version ver_K; reference numeral 42 represents the changed fingerprint set Δφ (including fingerprints in the conflict set, see claim 6 and embodiment 1); reference numerals 43 to 45 represent the three output types of merging, creating, and arbitrating, respectively. Detailed Implementation
[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments. Equivalent substitutions or obvious modifications made by those skilled in the art without departing from the concept of the present invention should fall within the protection scope of the present invention. The following parameters are used to illustrate feasibility and are not intended to limit the scope of protection of the claims.
[0021] Example 1 (e.g.) Figure 1-4 (As shown) This embodiment is applied to the knowledge fusion node in a knowledge graph data production pipeline. Input candidate entity set | |=120,000, the target ontology class is Company, and the identifier attribute key_attrs={name,industry_code}.
[0022] S1 ( Figure 1 ): Standardize the candidate entity "Shangqi Huitong Network Technology Co., Ltd." by removing the company name suffix to get "Shangqi Huitong Network Technology". Combine this with the industry code IC4510 to generate φ(e)=Hash("Shangqi Huitong Network Technology‖IC4510").
[0023] S2 ( Figure 1, Figure 3 ): Based on BK(e)=(prefix(name,L_p=3),industry_code), 4,200 buckets are obtained, with 87,326 candidate pairs |P| within each bucket (the theoretical total number of pairs is approximately 7.19 × 10⁻⁶). 9 ).
[0024] S3 ( Figure 1 Label 10 Figure 2 Example entity pair s_char=0.91, s_emb=0.88, s_ctx=0.40, overall similarity s=0.38×0.91+0.42×0.88+0.20×0.40=0.818.
[0025] S4 ( Figure 1 , Figure 2 Label 28): Class-related threshold =0.84, constructing a similarity graph yields 115,200 representative entities, with an average cluster confidence of conf=0.89.
[0026] S5 ( Figure 1 Label 11 Figure 4 ): Determine the change fingerprint set Δφ, which includes entity fingerprints that are not in I_K, and fingerprints belonging to the conflict set; in this embodiment, the conflict set is the set of entity fingerprints whose cluster confidence is lower than the preset arbitration threshold after S4 clustering and whose merging determination has not yet been completed. |Δφ|=3,200; after comparison with I_K, merge 2,800 fingerprints, create 400 new fingerprints, and arbitrate 47 fingerprints (cluster confidence is lower than the preset arbitration threshold of 0.78), and output the incremental change set ΔK and conflict details. Figure 4 Numbers 43 to 45 represent three types of outputs: merge, create, and arbitration. The quantities are shown in this example.
[0027] Example 2: Linkage with Quality Inspection The knowledge fusion node receives the duplication score output by the quality inspection module. The duplication score is a duplication control indicator output by the pipeline quality inspection module; a higher value indicates better duplication rate control. When the duplication score falls below a preset duplication threshold, indicating an increased duplication risk (corresponding to claims 7 and 8), the linkage scheduling module triggers an incremental fusion sub-pipeline, coordinating with the task scheduling module, to only initiate incremental fusion sub-pipelines for the subset of entities corresponding to the changed fingerprint set, without repeatedly scanning the fingerprint index for all historical entities. In a preferred embodiment, a duplication score of 0.962 and a preset duplication threshold of 0.965 trigger processing for approximately 3,247 entities.
[0028] Example 3: Adaptive Parameter Configuration The class relevance threshold, weight coefficient, name prefix length, and arbitration threshold can be configured according to the target ontology class and data distribution. For example, for the Company class, the class relevance threshold can be set to 0.84, the weight coefficient to (0.38, 0.42, 0.20), the name prefix length L_p to 3, and the arbitration threshold to 0.78. The class relevance threshold can also be adaptively adjusted within the upper and lower bounds based on the difference between the repeated quality inspection score and the preset threshold.
[0029] Example 4: System Deployment Method Each functional module is deployed on the knowledge fusion node according to claims 8 and 9, communicating with the task scheduling, quality inspection, and fingerprint indexing services. The similarity calculation module or clustering disambiguation module dynamically adjusts the class relevance threshold based on the difference between the duplication score output by the quality inspection module and the preset duplication threshold; the linkage scheduling module triggers incremental alignment only for the entity subset corresponding to the changed fingerprint set when the duplication risk increases. The fingerprint index can be stored in a database or an in-memory database, and the version number is updated after successful merging.
[0030] Example 5: Effect Description In | When |=120,000, the total theoretical logarithm is approximately 7.19 × 10⁻⁶. 9 In this invention, |P|=87,326; multi-strategy fusion increases the disambiguation F1 from 0.81 to 0.86; the alignment stage only processes records of the magnitude of |Δφ|, and the time consumption is proportional to the amount of change, decoupling it from the scale of the entire database.
[0031] Those skilled in the art may make equivalent substitutions for the hash algorithm, embedding model, blocking key field, and threshold without departing from the spirit of this invention, and all such substitutions should fall within the scope of protection.
Claims
1. A multi-strategy entity disambiguation incremental alignment method for knowledge graph pipelines, characterized in that, include: S1. Entity normalization step: Based on the attribute definition of the target ontology class, perform normalization processing on the name, alias and identifier attribute of the candidate entity to generate an entity fingerprint composed of the identifier attribute; S2. Candidate pair generation step: Generate a blocking key for each entity in the candidate entity set. The blocking key contains at least a prefix component of the normalized name and an industry code or regional code component. Generate candidate entity pair sets only in the buckets with the same blocking key. S3. Multi-strategy similarity fusion step: Calculate character similarity, semantic embedding similarity and structural context similarity for the candidate entity pairs respectively, and fuse them according to weight coefficients to obtain a comprehensive similarity score; S4. Clustering disambiguation step: When the comprehensive similarity score is not lower than the class correlation threshold, an undirected similarity graph is constructed on the candidate entity set, the connected components of the undirected similarity graph are obtained to obtain candidate entity clusters, and representative entities and cluster confidence are determined in each candidate entity cluster. S5. Incremental alignment step: Maintain the fingerprint index of the historical knowledge base, determine the change fingerprint set, and perform merging, creation, or arbitration operations only on the representative entity corresponding to the change fingerprint set and the fingerprint index, and output the incremental change set and conflict details.
2. The method according to claim 1, characterized in that, The physical fingerprint is obtained by hashing the normalized concatenated values of the identification attributes. The normalization includes removing the suffix of the enterprise name, unifying full-width and half-width characters, and unifying uppercase and lowercase.
3. The method according to claim 1, characterized in that, The length of the name prefix component of the blocking key is a preset length, which is determined based on the attribute characteristics of the target ontology class or the distribution of candidate entity data.
4. The method according to claim 1, characterized in that, The character similarity is Jaro-Winkler similarity or normalized similarity based on edit distance; the semantic embedding similarity is the vector cosine similarity of entity names and aliases after being encoded by a pre-trained language model; the structural context similarity is calculated based on the set similarity of the candidate entity's neighbor entity set in the relation set.
5. The method according to claim 1, characterized in that, The overall similarity score is calculated as the sum of the products of each similarity score and its corresponding weight coefficient, and the sum of the weight coefficients is 1. The weight coefficients are determined based on the verification sample set or historical disambiguation results corresponding to the target ontology class.
6. The method according to claim 1, characterized in that, The changed fingerprint set includes entity fingerprints whose fingerprints are not in the fingerprint index and entity fingerprints that belong to the conflict set; when the cluster confidence is lower than the preset arbitration threshold, conflict arbitration is performed; When the overall similarity between the representative entity and an existing entity in the fingerprint index is not lower than the class relevance threshold, the fingerprint index is merged and updated.
7. The method according to claim 1, characterized in that, The class-related threshold is dynamically adjusted based on the difference between the duplicate score provided by the upstream quality inspection module and the preset duplicate threshold.
8. A knowledge graph pipeline multi-strategy entity disambiguation incremental alignment system, characterized in that, include: (1) A normalization module, used to perform step S1 as described in claim 1; (2) Candidate pair generation module, used to execute step S2; (3) Similarity calculation module, used to execute step S3; (4) Clustering disambiguation module, used to execute step S4; (5) Incremental alignment module, used to perform step S5 and maintain the fingerprint index; (6) Linkage scheduling module, used to trigger the incremental alignment module to process only the entity subset corresponding to the change fingerprint set when the duplicate score output by the quality inspection module is lower than the preset duplicate threshold; The similarity calculation module or clustering disambiguation module is used to dynamically adjust the class correlation threshold based on the difference between the duplication score output by the quality inspection module and the preset duplication threshold. The system embeds a knowledge fusion node into the knowledge graph data production pipeline and connects it with the quality inspection module and the task scheduling module.
9. The system according to claim 8, characterized in that, The fingerprint index is stored in the database and a version number is maintained. The version number is updated after the merge operation is successful.
10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1 to 7.