Knowledge graph entity deduplication and cross-document fusion method and apparatus

Abstract

The application provides a knowledge graph entity deduplication and cross-document fusion method and device. Raw entities are extracted from unstructured documents and hashed to generate a pre-merge entity list, and a global distance matrix is generated based on the pre-merge entity list; the global distance matrix is subjected to agglomerative hierarchical clustering to construct a first cluster set with limited cluster diameter, the corresponding sub-matrix array of the first cluster set in the global distance matrix is called to construct a full-link hierarchical clustering pedigree graph and a second cluster set is constructed accordingly; the second cluster set is subjected to semantic merge judgment processing to obtain a semantic merge decision result for entity reference relationship filtering to generate a cascadingly updated intra-document deduplicated entity relationship feature; the intra-document deduplicated entity relationship feature is subjected to cross-document entity alignment processing to generate a cross-document fusion entity result, and entity nodes and relationship edges are constructed based on the cross-document fusion entity result through a graph database to generate a unified knowledge graph.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and knowledge graph technology, and more specifically, to a method and apparatus for knowledge graph entity deduplication and cross-document fusion. Background Technology

[0002] In the automated construction of knowledge graphs, the system needs to extract named entities from multiple unstructured documents and integrate them into a logically unified knowledge system. Since the extraction process is often carried out independently, the differences in the representation of the same entity in different contexts and the entity alignment issues in cross-document scenarios make entity deduplication and fusion key steps in achieving graph quality.

[0003] Existing entity fusion schemes typically employ a processing path based on single-link clustering and semantic adjudication. This scheme first uses an embedding model to map entity names into high-dimensional vectors. By calculating the similarity between vectors, a single-link hierarchical clustering algorithm is used to initially divide similar entities into different clusters. Subsequently, the list of entities within each cluster is used as input to perform the final merging decision.

[0004] However, this traditional clustering strategy has significant technical flaws. Single-link clustering algorithms are prone to a "chain reaction" effect when handling similarity propagation. This means that as long as adjacent entities meet a similarity threshold, they will be grouped into the same cluster even if the semantics of the first and last entities are inconsistent. This uncontrolled expansion results in an extremely high cardinality of entities in a single cluster, often far exceeding the capacity of subsequent processing stages. This directly leads to processing failures or a significant drop in accuracy, failing to meet the technical requirements for controllable cluster size in large-scale knowledge graph construction. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a method and apparatus for knowledge graph entity deduplication and cross-document fusion, thereby at least alleviating the aforementioned technical problems.

[0006] A method for entity deduplication and cross-document fusion in knowledge graphs, comprising: Step 1: Extract the original entities from the unstructured document and hash them to generate a pre-merged entity list, and generate a global distance matrix based on the pre-merged entity list; Step 2: Perform agglomerative hierarchical clustering on the global distance matrix to construct a first cluster set with a limited cluster diameter. Retrieve the sub-matrix array corresponding to the first cluster set in the global distance matrix to construct a fully linked hierarchical clustering genealogy and construct a second cluster set accordingly. Step 3: Perform semantic merging judgment processing on the second cluster set to obtain the semantic merging decision result, and perform entity reference relationship filtering based on the semantic merging decision result to generate the deduplicated entity relationship features within the document after cascading update; Step 4: Perform cross-document entity alignment processing on the deduplicated entity relationship features within the document to generate cross-document fused entity results, and construct entity nodes and relationship edges based on the cross-document fused entity results using a graph database to generate a unified knowledge graph after cross-document fusion.

[0007] A knowledge graph entity deduplication and cross-document fusion device, comprising: The first program unit is used to extract the original entities from the unstructured document and hash them to generate a pre-merged entity list, and generate a global distance matrix based on the pre-merged entity list. The second program unit is used to perform agglomerative hierarchical clustering on the global distance matrix to construct a first cluster set with a limited cluster diameter, retrieve the sub-matrix array corresponding to the first cluster set in the global distance matrix to construct a fully linked hierarchical clustering genealogy diagram, and construct a second cluster set accordingly. The third program unit is used to perform semantic merging judgment processing on the second cluster set to obtain semantic merging decision results, and to perform entity reference relationship filtering based on the semantic merging decision results to generate cascaded updated document-deduplicated entity relationship features. The fourth program unit is used to perform cross-document entity alignment processing on the deduplicated entity relationship features within the document to generate cross-document fused entity results, and to construct entity nodes and relationship edges based on the cross-document fused entity results through a graph database to generate a unified knowledge graph after cross-document fusion.

[0008] Technical advantages of the technical solution provided in this application This application provides a knowledge graph entity deduplication and cross-document fusion method that addresses the technical shortcomings of traditional solutions, such as uncontrollable cluster size leading to processing failures. It solves the problem of disordered cluster size expansion caused by similarity propagation by constructing a first cluster set with a limited diameter through agglomerated hierarchical clustering of the global distance matrix. Compared to traditional methods, the limited diameter design of the first cluster set provides a basic semantic compactness at the algorithm level, effectively preventing the generation of ultra-large-scale clusters. Based on this, a second cluster set is constructed by retrieving its corresponding submatrices to build a fully linked hierarchical clustering genealogy graph, achieving accurate control over the cluster size. This provides appropriately sized data for subsequent semantic merging and judgment processing, ensuring the feasibility of the deduplication process.

[0009] To address the technical flaw of potential gaps in relational references after entity merging, this application filters entity reference relationships based on semantic merging decision results to generate cascadingly updated deduplicated entity relation features within the document. This resolves the issue of inconsistencies between entity identifier changes and relational references during the deduplication process. Compared to static merging methods, cascading updates ensure that the reference relationships between entities within the document are synchronized in real time with deduplication decisions, achieving logical rigor in the reconstruction process of deduplicated entity relation features within the document.

[0010] To address the technical shortcomings of difficult cross-document entity alignment and incomplete global topology construction, this application performs cross-document entity alignment processing on deduplicated data to generate a fusion result, and constructs entity nodes and relationship edges using a graph database, thus solving the problem of entity logical silos in cross-document scenarios. Compared with traditional solutions, constructing a unified knowledge graph using a graph database can accurately restore the relational topology between entities, resulting in a knowledge graph with higher semantic consistency and topological completeness. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a knowledge graph entity deduplication and cross-document fusion method according to an embodiment of this application.

[0012] Figure 2 This is a structural block diagram of a knowledge graph entity deduplication and cross-document fusion device according to an embodiment of this application.

[0013] Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0014] like Figure 1 As shown, this embodiment of the present application provides a method for deduplication of knowledge graph entities and cross-document fusion, including: Step 1: Extract the original entities from the unstructured document and hash them to generate a pre-merged entity list, and generate a global distance matrix based on the pre-merged entity list; Step 2: Perform agglomerative hierarchical clustering on the global distance matrix to construct a first cluster set with a limited cluster diameter. Retrieve the sub-matrix array corresponding to the first cluster set in the global distance matrix to construct a fully linked hierarchical clustering genealogy and construct a second cluster set accordingly. Step 3: Perform semantic merging judgment processing on the second cluster set to obtain the semantic merging decision result, and perform entity reference relationship filtering based on the semantic merging decision result to generate the deduplicated entity relationship features within the document after cascading update; Step 4: Perform cross-document entity alignment processing on the deduplicated entity relationship features within the document to generate cross-document fused entity results, and construct entity nodes and relationship edges based on the cross-document fused entity results using a graph database to generate a unified knowledge graph after cross-document fusion.

[0015] Optionally, step 1 specifically includes: The entity names and entity types of different original entities are determined to form corresponding binary features, and the binary features are hashed to obtain a set of hash groups and the pre-merged entity list is constructed accordingly. The different entity names in the pre-merged entity list are batch-embedded and vectorized to generate entity vectors, thereby obtaining an entity vector set. Based on this, the global distance matrix that satisfies the value range constraint and represents the semantic differences between entities is generated.

[0016] Preferably, in the specific implementation of step 1, the basic attributes of the original entities extracted from the unstructured document are first parsed to extract the entity name and entity type corresponding to each original entity, and a binary feature corresponding to each original entity is constructed. Here, the entity name is the text content extracted from the text block of the unstructured document, used to refer to the corresponding object in the real world; the entity type is a predefined identifier used to distinguish the semantic category to which the entity belongs. The binary feature is composed of the one-to-one corresponding entity name and entity type. Within the same binary feature, the entity name and entity type have a fixed binding relationship, thus enabling each binary feature to represent the basic identity attributes of the corresponding original entity to a high degree, providing a unified and standardized feature input for subsequent hash grouping processing.

[0017] Preferably, in the specific implementation of step 1, hash grouping processing is performed on all constructed binary features to obtain a hash group set. The specific process of hash grouping is as follows: the complete entity name and complete entity type contained in each binary feature are used as input, and a hash value is calculated. The original entities corresponding to binary features with identical hash values ​​are grouped into the same group, thus forming multiple independent groups. All the groups that have been divided together constitute the hash group set. In this process, only when two original entities have completely identical entity names and entity types will the corresponding binary features generate the same hash value. This ensures that each group in the hash group set contains only original entities with identical entity names and entity types, avoiding erroneous grouping problems caused by entities with the same name but different semantic categories.

[0018] Preferably, in one scenario, step 1 is specifically implemented by performing entity cardinality reduction processing based on the obtained hash group set to construct a pre-merged entity list. For each group within the hash group set, all original entities contained in the group are traversed first, the entity description field corresponding to each original entity is extracted, the text length corresponding to each entity description field is counted, and the original entity corresponding to the original entity description field with the longest text length is selected and determined as the reference entity for the group. Subsequently, the text block identifiers corresponding to the remaining original entities in the group other than the reference entity are all merged into the reference entity, completing the pre-merging processing of the group. After completing the pre-merging processing for all groups in the hash group set, the reference entities corresponding to all groups are summarized in a preset order to form a pre-merged entity list. This process can reduce the total number of original entities with extremely low computational complexity, reducing the computational overhead for subsequent vectorization processing, while completely preserving all source information of the original entities and avoiding the loss of effective data.

[0019] Preferably, in the specific implementation of step 1, batch embedding vectorization processing is performed on the pre-merged entity list to generate a set of entity vectors that correspond one-to-one with the pre-merged entity list. First, the entity name corresponding to each reference entity in the pre-merged entity list is extracted. All entity names are then aggregated into a batch text sequence according to the order of the pre-merged entity list. This sequence is fed into a pre-trained semantic embedding model for vectorization mapping processing, generating a corresponding high-dimensional numerical array for each entity name. This high-dimensional numerical array is the entity vector corresponding to the entity name. The entity vectors corresponding to all entity names together constitute the entity vector set. The semantic embedding model maps text content to a high-dimensional semantic space, making entity vectors corresponding to semantically similar text content closer together in the high-dimensional semantic space, and entity vectors corresponding to text content with significant semantic differences farther apart in the high-dimensional semantic space. This allows the generated entity vectors to represent the semantic features of the corresponding entity names to a higher degree.

[0020] Preferably, in the specific implementation of step 1, normalization processing is performed on the generated entity vector set to generate a normalized vector set for subsequent similarity calculation. For each entity vector in the entity vector set, the L2 norm corresponding to the entity vector is first calculated, and then the value of each dimension of the entity vector is divided by the L2 norm corresponding to the entity vector to obtain the normalized vector corresponding to the entity vector. For entity vectors with a L2 norm of zero, the L2 norm value is forcibly set to 1 before the above normalization processing is performed to avoid division by zero errors during the calculation process. The normalized vectors corresponding to all entity vectors together constitute the normalized vector set. The L2 norm of each normalized vector after normalization processing is 1, thereby eliminating the interference of the difference in the magnitude of entity vectors on the subsequent similarity calculation results, so that the similarity results obtained in the subsequent calculation are only related to the semantic direction of the entity vectors.

[0021] Preferably, in one scenario, step 1 is specifically implemented by performing a dot product operation on the generated set of normalized vectors to generate an embedded feature semantic association matrix. The embedded feature semantic association matrix is ​​a two-dimensional matrix structure. The rows and columns of the matrix correspond to reference entities in the pre-merged entity list. The row and column indices of the matrix correspond one-to-one with the arrangement indices of the reference entities in the pre-merged entity list. The element at a specific row and column position is the result of performing a dot product operation between the normalized vector corresponding to that row and column indices. The value of this dot product operation ranges from -1 to 1. The closer the value is to 1, the higher the semantic consistency between the two corresponding normalized vectors; the closer the value is to -1, the greater the semantic difference between the two corresponding normalized vectors. This ensures that each element in the embedded feature semantic association matrix can represent the semantic similarity between the corresponding two reference entities to a high degree.

[0022] Preferably, in the specific implementation of step 1, a distance mapping transformation process is performed on the generated embedded feature semantic association matrix to generate a global distance matrix that satisfies the value range constraint and represents the semantic differences between entities. First, for each element in the embedded feature semantic association matrix, a numerical transformation process is performed, subtracting the original value of the element from 1 to obtain the initial distance value corresponding to that element. Then, a non-negativity constraint process is performed on each initial distance value, comparing the initial distance value with 0 and taking the larger value as the final distance value corresponding to that element. This eliminates negative values ​​that may be generated by floating-point operation errors, ensuring the non-negativity of the final distance value. For diagonal elements in the matrix that are at the same row and column positions, their corresponding final distance values ​​are forcibly set to 0, thus representing that the semantic difference between the same entity and itself is 0. The final distance values ​​corresponding to all elements together constitute the global distance matrix. The value of each element in the global distance matrix is ​​between 0 and 2. The smaller the value, the smaller the semantic difference between the two corresponding reference entities. The larger the value, the greater the semantic difference between the two corresponding reference entities. This enables the global distance matrix to provide unified and standardized input data for subsequent clustering processing.

[0023] Optionally, constructing the pre-merged entity list specifically includes: The entity with the longest entity description field is identified from the original entities and used as a reference entity to perform entity cardinality reduction processing on the hash group set, thereby generating the pre-merged entity list.

[0024] Preferably, in the specific implementation of constructing the pre-merged entity list, a full traversal is first performed on the divided hash group set, extracting each group contained within the hash group set in turn. For each group, all original entities contained within that group are traversed, and the entity description field corresponding to each original entity is extracted. The entity description field is text content extracted from the corresponding text block of the unstructured document, used to supplement the semantic meaning and attribute information of the corresponding original entity. Each original entity corresponds to a unique entity description field, and the text content of the entity description field is directly related to the content of the text block to which the original entity belongs, thus providing a unified comparison basis for subsequent reference entity selection.

[0025] Preferably, in the specific technical implementation of constructing the pre-merged entity list, for the entity description fields corresponding to all original entities within each group, text length statistical processing is performed. The text length corresponding to each entity description field is calculated sequentially, generating a text length value that corresponds one-to-one with each entity description field. The statistical scope of the text length statistical processing includes all characters contained in the complete text content of the entity description field. This ensures that the generated text length value accurately reflects the richness of the content of the corresponding entity description field, providing a quantifiable comparison indicator for subsequent reference entity selection.

[0026] Preferably, in a specific implementation of constructing the pre-merged entity list in a scenario, for the text length values ​​corresponding to all entity description fields within each group, numerical comparison and extreme value filtering are performed. The original entity corresponding to the entity description field with the largest text length value is determined as the reference entity for that group. If multiple entity description fields within the same group have the same text length value, and all are the maximum value, then the original entity that appears first in the unstructured document is selected as the reference entity for that group. This ensures that each group has a unique reference entity, enabling the reference entity to carry the richest entity attribute information within that group.

[0027] Preferably, in the specific implementation of constructing the pre-merged entity list, for each group, the text block identifiers corresponding to all original entities within that group (excluding the reference entity) are extracted, and all extracted text block identifiers are merged into the reference entity corresponding to that group. Here, the text block identifier is a unique identifier used to mark the position of a text block in the unstructured document to which the original entity belongs. Each original entity corresponds to at least one text block identifier. The merged reference entity contains all text block identifiers corresponding to all original entities within that group, thereby completely preserving the source information of all original entities within that group and avoiding the loss of valid data during the pre-merging process.

[0028] Preferably, in the specific technical implementation of constructing the pre-merged entity list, after determining the reference entity and merging the text block identifiers for all groups within the hash grouping set, entity cardinality reduction processing is performed based on the reference entities corresponding to all groups. The specific process of entity cardinality reduction processing is as follows: Each group's corresponding reference entity is treated as a unique valid entity, and all other original entities within that group are removed, thereby transforming groups that originally contained multiple duplicate entities into valid data units containing only one reference entity, significantly reducing the total number of entities to be processed. This process differs from the traditional method of comparing and merging entities one by one. Relying on the pre-processing results of hash grouping, it completes entity cardinality reduction with extremely low computational complexity, reducing computational overhead for subsequent batch embedding vectorization processing, and avoiding interference from duplicate entities on subsequent clustering results.

[0029] Preferably, in a specific implementation of constructing the pre-merged entity list in a scenario, all reference entities corresponding to groups within the hash grouping set are aggregated according to a preset arrangement rule to generate the pre-merged entity list. The preset arrangement rule can be based on the order of the first appearance of the reference entities in the unstructured document, or it can be based on the classification order of the entity types corresponding to the reference entities. Each reference entity in the pre-merged entity list contains a corresponding entity name, entity type, entity description field, and a full text block identifier after merging, thereby forming a standardized entity data set, providing standardized and unified input data for subsequent batch embedding vectorization processing, global distance matrix generation, and other steps.

[0030] Preferably, in the specific implementation of constructing the pre-merged entity list, after the pre-merged entity list is constructed, data consistency verification is performed on all reference entities within the pre-merged entity list. The specific process of data consistency verification is as follows: The entity name and entity type corresponding to each reference entity are verified sequentially to ensure that there are no duplicate reference entities in the pre-merged entity list with identical entity names and entity types. Simultaneously, the integrity of the merged text block identifiers within each reference entity is verified to ensure that no text block identifiers are omitted. Through data consistency verification, the data quality of the pre-merged entity list can be further improved, providing a reliable data foundation for subsequent full-process entity deduplication and fusion processing.

[0031] Optionally, generating the global distance matrix that satisfies the value range constraint and represents the semantic differences between entities specifically includes: Normalize each entity vector in the entity vector set to obtain a corresponding normalized vector, and perform a dot product operation on every two normalized vectors to generate an embedded feature semantic association matrix. The embedded feature semantic association matrix is ​​subjected to distance mapping transformation to generate the global distance matrix that satisfies the value range constraint and represents the semantic differences between entities.

[0032] Preferably, in the specific implementation of generating the global distance matrix, a preprocessing verification is first performed on the entity vector set obtained after batch embedding and vectorization of the pre-merged entity list. Each entity vector contained in the entity vector set is extracted sequentially, and the dimensionality consistency and numerical validity of each entity vector are verified. Here, the entity vector is a high-dimensional semantic representation of the corresponding reference entity name in the pre-merged entity list, and each reference entity name corresponds to a unique entity vector. The dimensionality consistency verification ensures that all entity vectors in the entity vector set have the same number of dimensions, thus meeting the basic requirements for subsequent vector operations. The numerical validity verification identifies all-zero numerical vectors in the entity vector set, providing compliant and valid input data for subsequent normalization processing.

[0033] Preferably, in the specific technical implementation of generating the global distance matrix, for each entity vector in the preprocessed and verified entity vector set, a L2 norm calculation is performed to generate a vector magnitude value corresponding to each entity vector. The specific process of the L2 norm calculation is as follows: the values ​​of all dimensions within the entity vector are squared, summed, and then the square root of the sum is taken. The final value is the vector magnitude value corresponding to that entity vector. The vector magnitude value can characterize the length of the corresponding entity vector in the high-dimensional semantic space, providing a standardized scaling benchmark for subsequent normalization processing, thereby laying a data foundation for eliminating the interference of entity vector magnitude differences on semantic similarity calculation.

[0034] Preferably, in a scenario, when generating the global distance matrix, a normalization process is performed on each entity vector and its corresponding vector magnitude to generate a normalized vector corresponding to each entity vector. The specific normalization process involves dividing the original value of each dimension of the entity vector by the corresponding vector magnitude to obtain the normalized value for that dimension. The normalized values ​​for all dimensions of the entity vector together constitute the normalized vector for that entity vector. For entity vectors with a magnitude of zero, their magnitude is forcibly set to 1 before the normalization process is performed to avoid division by zero errors during calculation. The normalized vectors corresponding to all entity vectors in the entity vector set together constitute the normalized vector set. The magnitude of each normalized vector in the normalized vector set is 1, thereby eliminating the interference of differences in entity vector magnitudes on subsequent semantic similarity calculations, ensuring that the similarity results obtained later only reflect the semantic association between the reference entity names.

[0035] Preferably, in the specific implementation of generating the global distance matrix, pairwise dot product operations are performed on the generated set of normalized vectors to generate an embedded feature semantic association matrix. The embedded feature semantic association matrix is ​​a two-dimensional matrix structure. The rows and columns of the matrix correspond one-to-one with the reference entities in the pre-merged entity list. The row and column indices of the matrix are completely consistent with the arrangement indices of the reference entities in the pre-merged entity list. The element at the intersection of a row and a column is the semantic similarity value obtained by performing a dot product operation between the normalized vector corresponding to that row index and the normalized vector corresponding to that column index. The semantic similarity value obtained by performing a dot product operation on two normalized vectors with a magnitude of 1 ranges from -1 to 1. The closer the value is to 1, the higher the semantic consistency of the reference entity names corresponding to the two normalized vectors; the closer the value is to -1, the greater the semantic difference between the two reference entity names. This ensures that each element in the embedded feature semantic association matrix directly represents the semantic similarity between the corresponding two reference entities. The diagonal elements in the matrix that are at the same row and column positions correspond to the dot product of the normalized vectors of the same reference entity. The semantic similarity value is 1, which indicates that the semantic similarity between the reference entity and itself is the highest value.

[0036] Preferably, in the specific technical implementation of generating the global distance matrix, an initial distance mapping transformation process is performed on the generated embedded feature semantic association matrix to generate an initial distance matrix. The specific process of the initial distance mapping transformation is as follows: for each semantic similarity value in the embedded feature semantic association matrix, the semantic similarity value is subtracted by a fixed value of 1 to obtain the corresponding initial distance value. The initial distance values ​​corresponding to all semantic similarity values ​​in the embedded feature semantic association matrix together constitute the initial distance matrix. After this mapping transformation, the reference entity pair with higher original semantic similarity values ​​has a smaller initial distance value, representing a smaller semantic difference between the two reference entities. This converts the original positive similarity value representing the degree of similarity into a negative distance value representing the degree of difference, adapting to the format requirements of the distance input data in subsequent agglomerative hierarchical clustering processing. After the transformation, the initial distance value corresponding to the semantic similarity value of 1 at the diagonal position in the matrix is ​​0, thus representing that the semantic difference between the same reference entity and itself is 0, which conforms to the basic logic of entity semantic representation.

[0037] Preferably, in a specific implementation of generating the global distance matrix in a given scenario, non-negativity constraint and numerical correction processing are performed on the generated initial distance matrix to generate a global distance matrix that satisfies the value range constraints and represents the semantic differences between entities. The specific process of non-negativity constraint and numerical correction processing is as follows: First, for each initial distance value in the initial distance matrix, it is compared with the value 0, and the larger value is taken as the corrected distance value. This eliminates negative values ​​that may be caused by floating-point operation errors, ensuring that all distance values ​​are non-negative. Then, for the diagonal elements in the matrix at the same row and column positions, their corrected distance values ​​are forcibly set to 0, further ensuring that the distance values ​​between the reference entity and itself conform to semantic logic. All distance values ​​after the above correction processing constitute the global distance matrix. The value range of each element in the global distance matrix is ​​between 0 and 2. The smaller the value, the smaller the semantic difference between the corresponding two reference entities; the larger the value, the greater the semantic difference between the corresponding two reference entities. This provides unified, standardized, and directly usable distance input data for subsequent agglomerative hierarchical clustering processing.

[0038] Preferably, in the specific implementation of generating the global distance matrix, after the global distance matrix is ​​generated, a symmetry verification process is performed on the global distance matrix to verify whether the distance value at a certain row and column position in the matrix is ​​completely consistent with the distance value at the corresponding row and column position. The semantic distance between two reference entities has natural symmetry, that is, the semantic distance from the first reference entity to the second reference entity is exactly the same as the semantic distance from the second reference entity to the first reference entity. Through the symmetry verification process, small numerical errors that may occur during vector operations can be identified and corrected, further improving the data quality of the global distance matrix, ensuring the reliability of the subsequent agglomerative hierarchical clustering results, and providing stable basic data support for the entire entity deduplication and fusion process.

[0039] Optionally, the submatrix array corresponding to the first cluster set in the global distance matrix is ​​retrieved to construct a fully linked hierarchical clustering genealogy graph, and a second cluster set is constructed accordingly, including: Based on the first cluster set, extract the corresponding sub-matrix array from the global distance matrix; Based on the sub-matrix array, the hierarchical merging path between entities is determined, and a fully linked hierarchical clustering genealogy is constructed accordingly. Based on the fully linked hierarchical clustering genealogy diagram, the second cluster set is constructed.

[0040] Preferably, in the specific implementation of step 2, a cluster size verification process is first performed on the constructed first cluster set. Each cluster within the first cluster set is traversed sequentially, and the number of entities within each cluster is counted. Clusters with an entity count exceeding a preset size limit are marked as clusters to be split, while other clusters with an entity count within the preset size limit are directly marked as qualified clusters. The preset size limit is set based on the contextual carrying capacity of the subsequent semantic merging judgment process. The first cluster set is the clustering result obtained after performing agglomerative hierarchical clustering with a full-link strategy based on the global distance matrix. Each cluster within the first cluster set satisfies the constraint of a limited cluster diameter, meaning that the semantic distance between any two entities within a cluster does not exceed a preset distance threshold, thereby ensuring basic semantic consistency among entities within the cluster.

[0041] Preferably, in the specific implementation of step 2, for each cluster to be split after labeling, entity index mapping processing is performed to extract the permutation numbers of all entities within the cluster to be split in the pre-merged entity list, generating an entity index sequence that corresponds one-to-one with the cluster to be split. The rows and columns of the global distance matrix correspond one-to-one with the reference entities in the pre-merged entity list, and the row and column numbers perfectly match the permutation numbers of the reference entities in the pre-merged entity list. Elements at any position in the global distance matrix correspond to the semantic distance values ​​between the two reference entities represented by the row and column numbers. Based on the generated entity index sequence, all semantic distance values ​​contained in the corresponding rows and columns are extracted from the global distance matrix to construct a sub-distance matrix that corresponds one-to-one with the cluster to be split. All sub-distance matrices corresponding to the clusters to be split together constitute a sub-matrix array.

[0042] Preferably, in one scenario, when implementing step 2, for each sub-distance matrix within the sub-matrix array, a compression format conversion is performed to extract all semantic distance values ​​within the upper triangular region of the sub-distance matrix, generating a compressed distance vector corresponding one-to-one with that sub-distance matrix. The sub-distance matrix is ​​a symmetric matrix, with diagonal elements representing the semantic distance values ​​between the reference entity and itself. The values ​​in the upper and lower triangular regions are completely symmetrical. Extracting only the values ​​in the upper triangular region is sufficient to fully cover the unique semantic distance information of all pairwise reference entity pairs within the cluster, reducing the computational overhead of subsequent iterations without losing valid data. Based on the generated compressed distance vector, agglomerative hierarchical clustering iteration is performed using a fully linked strategy. The merging order between each pair of reference entities within the cluster is determined sequentially, generating a hierarchical merging path corresponding to the cluster to be split. The hierarchical merging path fully records the two reference entities or sub-clusters corresponding to each merging operation during the iteration process, as well as the semantic distance values ​​corresponding to the merging operation, completely reconstructing the entire process of reference entities gradually aggregating from individual entities into clusters.

[0043] Preferably, in the specific implementation of step 2, for each hierarchical merging path corresponding to the cluster to be split, a tree-structure mapping process is performed to construct a fully linked hierarchical clustering genealogy diagram corresponding one-to-one with the cluster to be split. The fully linked hierarchical clustering genealogy diagram is presented in a tree structure. The leaf nodes of the fully linked hierarchical clustering genealogy diagram correspond to each reference entity within the cluster to be split, and the non-leaf nodes correspond to the sub-clusters generated by each merging operation in the hierarchical merging path. The height of the non-leaf nodes perfectly matches the semantic distance value corresponding to the merging operation. The total height of the path from two leaf nodes to the common parent node corresponds to the semantic distance between the reference entities represented by the two leaf nodes. The fully linked hierarchical clustering genealogy diagram completely preserves the semantic hierarchy of all reference entities within the cluster to be split, which can intuitively reflect the semantic affinity between different reference entities. At the same time, it fully follows the merging rules of the fully linked strategy, ensuring that each cluster branch in the fully linked hierarchical clustering genealogy diagram satisfies the constraint that the cluster diameter does not exceed the corresponding merging distance, providing a reliable tree-structure foundation for subsequent adaptive cutting processing.

[0044] Preferably, in the specific implementation of step 2, for each fully connected hierarchical clustering phylogenetic tree that has been constructed, an adaptive threshold search process is performed to determine the optimal cutting threshold corresponding to that fully connected hierarchical clustering phylogenetic tree. The specific process of the adaptive threshold search process is as follows: First, a candidate threshold set is constructed, starting from the distance threshold used in global clustering, and gradually decreasing according to a fixed search step size until a preset minimum threshold is reached. Then, for each candidate threshold in the candidate threshold set, a horizontal cut is performed at the height position corresponding to that candidate threshold on the fully connected hierarchical clustering phylogenetic tree to obtain multiple sub-clusters after the cut, and the maximum number of entities in all sub-clusters is counted. Finally, from the candidate threshold set, all candidate thresholds that ensure the maximum number of entities in the largest sub-cluster does not exceed a preset size limit are selected, and the candidate threshold with the largest value is chosen as the optimal cutting threshold. Selecting the candidate threshold with the largest value as the optimal cutting threshold can, under the premise of satisfying the sub-cluster size constraint, preserve the semantic aggregation between reference entities to the greatest extent, avoiding over-cutting that causes semantically similar reference entities to be split into different sub-clusters.

[0045] Preferably, in one scenario, step 2 is specifically implemented by performing a spectrum cutting process on each fully connected hierarchical clustering spectrum and its corresponding optimal cutting threshold, generating multiple compliant sub-clusters corresponding to that fully connected hierarchical clustering spectrum. The specific process of spectrum cutting is as follows: at the height position of the fully connected hierarchical clustering spectrum corresponding to the optimal cutting threshold, all branch connections exceeding the threshold are cut, splitting the complete tree structure into multiple independent subtrees. All reference entities corresponding to the leaf nodes within each subtree constitute a compliant sub-cluster. A splitting process is performed on all generated compliant sub-clusters. Compliant sub-clusters containing only a single reference entity are directly retained, without needing to enter the subsequent semantic merging judgment process; compliant sub-clusters containing multiple reference entities are sorted in descending order of the number of entities, providing ordered input data for the subsequent semantic merging judgment process.

[0046] Preferably, in the specific implementation of step 2, all qualified clusters marked in the first clustering set and all compliant sub-clusters generated by cutting the fully linked hierarchical clustering phylogenetic graph are combined to construct a second clustering set. Each cluster in the second clustering set simultaneously satisfies two constraints: the first is that the semantic distance between any two reference entities within the cluster does not exceed a preset distance threshold, ensuring high semantic consistency among reference entities within the cluster; the second is that the number of reference entities contained in the cluster does not exceed a preset size limit, adapting to the carrying capacity of subsequent semantic merging judgment processing. The second clustering set completely covers all reference entities in the pre-merged entity list, without any omissions or duplicate attributions of reference entities. It provides standardized input for subsequent semantic merging judgment processing that satisfies both semantic compactness and scalability control, unlike the chain expansion problem that easily occurs in traditional single-linked clustering, thus improving the stability and usability of clustering results from an algorithmic perspective.

[0047] Optionally, entity reference relationship filtering is performed based on the semantic merging decision result to generate cascaded updated in-document deduplicated entity relationship features, including: Perform a semantic merge judgment on the second cluster set to obtain a semantic merge decision result that includes merge index grouping and merge description; Based on the semantic merging decision results, a name mapping table is constructed to represent the entity renaming and merging mapping status for filtering entity reference relationships, so as to generate cascaded updated deduplicated entity relationship features within the document.

[0048] Preferably, in the specific implementation of step 3, cluster splitting and priority sorting are first performed on the second cluster set to provide standardized input data for subsequent semantic-level merging judgment. The second cluster set is the clustering result obtained after two levels of processing: fully linked hierarchical clustering and adaptive segmentation of the phylogenetic graph. Each cluster in the set simultaneously satisfies the semantic consistency constraint of limited cluster diameter and the scale controllability constraint of the number of entities not exceeding the scale limit. During the splitting process, clusters containing only a single reference entity are marked as clusters that do not need to be merged, and all the original data of the reference entities in the cluster are directly and completely retained, including entity index, entity name, entity type, entity description field and corresponding text block identifier list, without entering the subsequent semantic-level merging judgment stage; clusters containing multiple reference entities are marked as clusters to be merged, and are sorted in descending order according to the number of entities contained in the cluster, forming a sequence of clusters to be merged. This sorting process enables subsequent semantic judgment processing to prioritize clusters with a large number of entities, rationally allocate computing resources, and ensure that the number of entities in each cluster to be merged is suitable for the context window capacity of the subsequent large language model, thus avoiding processing failures caused by an excessive number of entities at the input level.

[0049] Preferably, in the specific technical implementation of step 3, for each cluster to be merged in the sorted sequence of clusters to be merged, structured input data encapsulation and compliance verification processing are performed to adapt to the processing requirements of semantic-level merging judgment. During the structured encapsulation process, each reference entity data in the cluster is standardized and organized according to the fixed format of "entity index - entity name - entity type - entity description field - text block identifier list" to form a semantic judgment input unit; if the text length of the entity description field exceeds the preset threshold, semantic-based summary compression processing is adopted to retain the core attribute information and semantic keywords of the reference entity, remove redundant decorative text, and ensure that the total length of the encapsulated single cluster input data adapts to the context window limit of the large language model. The compliance verification process consists of three parts: First, format consistency verification, which verifies that all reference entity data within a cluster conforms to the preset encapsulation format, with no missing fields or data type errors. Second, index uniqueness verification, which verifies that entity indexes within a cluster are unique and contain no invalid values, ensuring that each reference entity has a unique identifier. Third, semantic integrity verification, which verifies that the entity name, entity type, and entity description fields match without logical conflicts. Clusters that pass verification proceed to subsequent processing stages; those that fail verification undergo data completion and format correction until they pass. This provides high-quality, standardized input data for semantic-level merging judgment, reducing the probability of judgment bias in the large language model.

[0050] Preferably, in the application scenario of enterprise knowledge graph construction, in the specific implementation of step 3, the clustered data to be merged after encapsulation and verification is sent to the large language model in parallel through an asynchronous concurrency framework to perform semantic merging judgment processing to generate preliminary semantic judgment results. The asynchronous concurrency framework has a built-in dynamic load balancing mechanism and an exponential backoff retry mechanism. The dynamic load balancing mechanism can adjust the number of concurrent processing in real time according to the hardware resource usage and the call limit of the large language model interface, maximizing processing efficiency within the hardware carrying capacity. The exponential backoff retry mechanism automatically initiates retries at increasing time intervals for single call failures. The number of retries can be configured according to business needs. If the call still fails after exceeding the number of retries, a degradation strategy is triggered. After receiving the clustered data to be merged, the large language model performs deep semantic analysis based on entity name, entity type, entity description fields, and text block source information. It completes two core judgment tasks: the first is to identify which reference entities within the cluster belong to the same entity with the same name but different meanings and need to be merged; the second is to identify which reference entities within the cluster belong to different entities with the same name but different meanings and need to be kept independent and supplemented with distinguishing identifiers. The final preliminary semantic judgment result output by the large language model includes the merge index grouping and merge description corresponding to the clusters to be merged. The merge index grouping groups the reference entity indexes with consistent semantics within the cluster into the same group, ensuring that each entity index appears in only one group. The merge description includes the unified entity name corresponding to each group, the integrated entity description field, and the semantic judgment basis for merging or splitting.

[0051] Preferably, in the specific implementation of step 3, a two-level verification process is performed on the preliminary semantic judgment result output by the large language model to generate a formal semantic merging decision result. The first level of verification is a completeness verification, which uses the index full set comparison method to perform a difference operation on all entity indexes contained in the merged index group and the corresponding entity index full set of the cluster to be merged. If the difference set is empty, it is determined that the merged index group completely covers all entities in the cluster, with no omissions, no duplications, and no invalid indexes; if the difference set is not empty, it is determined that the verification fails, triggering the large language model to be called again to supplement the missing entity index group. The second level of verification is a logical consistency verification. On the one hand, it verifies whether the merged description and the merged index group correspond one-to-one, and that each merged index group has a corresponding unified name, integrated description, and judgment basis, with no cases of mismatch between group and description; on the other hand, it verifies whether the judgment basis in the merged description has no logical conflict with the original description information of the reference entities in the corresponding group, ensuring that the merging or splitting decision has reasonable semantic support. The preliminary semantic judgment result that passes both levels of verification is determined as the formal semantic merging decision result; for clusters that still fail verification after being called again, a downgrade process is performed, marking all reference entities within the cluster as independent entities, retaining the original data and not performing merging, to ensure that the overall processing flow is not interrupted.

[0052] Preferably, in the specific technical implementation of step 3, based on the semantic merging decision results corresponding to all clusters to be merged, multi-dimensional mapping relationship extraction and structured name mapping table construction are performed to provide core mapping basis for subsequent entity reference relationship filtering and cascading updates. From the semantic merging decision results, three types of core mapping relationships are extracted: The first type is the merging mapping relationship, where for reference entities belonging to the same group in the merged index group, each original entity identifier and original entity name are mapped to the target entity identifier and unified entity name corresponding to that group, respectively, completely recording the identifier correspondence before and after entity merging; the second type is the renaming mapping relationship, where for cases where entities are judged as independent entities due to homonyms, the original name of the entity is mapped to a new name supplemented with a distinguishing description, thereby resolving the ambiguity problem of the same name corresponding to different semantic entities; the third type is the independent mapping relationship, where for reference entities within clusters that do not need to be merged, and independent entities retained during the downgrade process, the original index and original name of the entity are mapped to itself, while the merging status is marked as independent, ensuring that the name mapping table covers all entities without omission. The completed name mapping table adopts a structured multi-field design, including eight core fields: original entity identifier, original entity name, target entity identifier, target entity name, merge status identifier, merge confidence, source index chain, and full text block identifier list, which completely records the entire chain information of entity identifier changes. At the same time, the name mapping table adopts a hybrid storage architecture combining Bloom filters and hash tables. Bloom filters are used to quickly determine whether there is a corresponding mapping record for the original entity identifier, and hash tables are used to store complete mapping details, optimizing the query time complexity of a single mapping record to constant level, supporting efficient batch updates of massive entity reference relationships in the future.

[0053] Preferably, in the specific implementation of step 3, a full entity relation extraction process is performed on the original input unstructured document to generate an entity relation feature set covering all entity associations within the document, providing basic data for subsequent entity reference relationship filtering. Entity relation extraction adopts a hybrid strategy of deep integration of rules and models, divided into two branches: explicit relation extraction and implicit relation extraction. Explicit relation extraction is implemented based on a domain-adapted rule base, which contains three core rules: subject-verb-object sentence template rules, relation trigger word matching rules, and entity type association rules. The rule base can automatically mine and supplement new rules based on newly added labeled data to adapt to the knowledge graph construction needs of different domains. Implicit relation extraction is implemented based on a pre-trained relation extraction model that has been fine-tuned by incremental adjustments to domain data. It infers potential associations between entities that are not expressed through explicit sentence structures by using the contextual semantic information of the unstructured document. The extraction results from the two branches are subjected to a confidence-weighted fusion process. Explicit relationship extraction results are assigned initial confidence scores according to rule priorities, while implicit relationship extraction results directly use the model's output confidence scores. The final confidence score for each relationship record is calculated using a preset weighting ratio. Relationship records with final confidence scores higher than a preset base threshold are selected to form an entity relationship feature set. Each relationship record in the entity relationship feature set contains five core pieces of information: source entity field, target entity field, relationship type, final confidence score, and text block identifier. The source entity field corresponds to the original index and name of the entity initiating the relationship, and the target entity field corresponds to the original index and name of the entity receiving the relationship. This fully preserves the original reference relationships between entities, providing comprehensive and accurate basic data for subsequent updates to reference relationships.

[0054] Preferably, in one scenario, when implementing step 3, based on the constructed name mapping table, entity reference relationship replacement, conflict resolution, and invalid relationship filtering are performed on each relationship record in the entity relationship feature set to generate initially updated entity relationship features. The first step performs full reference relationship replacement processing, traversing each relationship record in the entity relationship feature set. The original indexes and names of the source entity fields and the target entity fields within the relationship record are uniformly replaced with the corresponding target entity identifiers and names according to the mapping relationship in the name mapping table. This ensures that after entity merging or renaming, the entity identifiers in the relationship records can be updated synchronously, fundamentally avoiding inconsistencies between entity identifiers and relationship references. The second step performs multi-source conflict resolution processing. When multiple relationship records are replaced and the source entity identifiers and target entity identifiers are completely identical, but the relationship types differ, conflict fusion logic is triggered. Following the rule of prioritizing explicit extraction results over implicit extraction results, the relationship type with the highest confidence and most reliable source is retained. Simultaneously, the conflict content and fusion trajectory are recorded to ensure the processing process is traceable. The third step involves filtering invalid relationships, which has three dimensions: First, filtering self-referencing relationships, removing relationship records where the source entity identifier and target entity identifier are completely identical after replacement, thus avoiding logical contradictions arising from entity self-associations after entity merging; second, filtering low-confidence relationships, adaptively setting confidence thresholds for different relationship types, removing relationship records with a final confidence level lower than the corresponding threshold, thus improving the overall quality of relationship data; and third, filtering invalid entity references, removing relationship records where the source or target entity does not have a corresponding mapping record in the name mapping table, thus avoiding dangling invalid references.

[0055] Preferably, in the specific technical implementation of step 3, the entity-relationship bidirectional consistency verification and cascading integrity completion processing are performed on the filtered and updated entity relationship features to generate the final cascading updated document-within-deduplication entity relationship features. The entity-relationship bidirectional consistency verification is divided into two dimensions: the first dimension is entity-side verification, which verifies that all target entities in the name mapping table have corresponding reference records in the updated entity relationship features. For isolated entities without any association, an unrelated identifier is added to ensure that no entity information is lost. The second dimension is relationship-side verification, which verifies that the source entity identifier and target entity identifier in all updated relationship records exist within the target entity range of the name mapping table, with no invalid entity references outside the range, ensuring that each relationship record has logical validity. The cascading integrity completion process consists of two parts: The first is entity metadata completion. For the merged target entity, the text block identifier list and attribute description information corresponding to all original entities within the merged index group are fully merged and deduplicated, and added to the target entity's metadata to ensure no loss of the entity's source and attribute information. The second is relationship tracing information completion. For each updated relationship record, the original entity identifier and original text block identifier before merging are added to ensure that each relationship can be traced back to its original source in the unstructured document. The final generated cascading updated document-within-a-document deduplicated entity relationship features consist of two core parts: The first part is the document-within-a-document deduplicated entity list. Each entity entry in the list contains a unique target entity identifier, a unified entity name, a merged entity description field, a full text block identifier list, and merged status metadata. The second part is the document-within-a-document deduplicated entity relationship list. Each relationship entry in the list contains a unified source entity index, target entity identifier, relationship type, final confidence level, and full-link tracing information. This data not only achieves semantic-level deduplication of duplicate entities within documents, but also ensures the cascading synchronous update and logical integrity of reference relationships between entities, providing high-quality foundational data for subsequent cross-document entity alignment and unified knowledge graph construction.

[0056] Preferably, in the specific implementation of step 3, a full-link degradation processing and exception handling mechanism is designed for abnormal scenarios that occur during the semantic-level merging judgment and entity reference relationship update process, so as to ensure the continuity and stability of the entity deduplication and relationship update process within the document. For scenarios where semantic-level merging cannot be completed due to continuous failures in large language model interface calls or network anomalies, the degradation strategy adopted is to mark all reference entities within the corresponding cluster to be merged as independent entities, fully preserving the original entity data and relational references. Merging can be re-executed once the interface recovers, without blocking the overall process. For scenarios where semantic merging decisions still fail integrity and consistency checks after multiple retries, the degradation strategy adopted is to split the original cluster to be merged into the smallest unit of pairs of entities, re-execute semantic-level merging, reducing the complexity of a single judgment and improving the compliance of the judgment results. For scenarios involving circular mappings and one-to-many mapping conflicts during name mapping table construction, the repair strategy is to locate the conflicting entity nodes, reset the conflicting nodes to an independent mapping state, synchronously update the mapping table, and record the anomaly repair log. For scenarios where the confidence of entity relationship extraction results is generally low and the number of effective relationships is insufficient, the optimization strategy adopted is to increase the weight of explicit rule extraction, supplement domain-specific extraction rules, re-execute entity relationship extraction, and supplement effective relationship data. This end-to-end exception handling mechanism can cover various exception scenarios during the processing, ensuring the stability of the process and the integrity of the data. At the same time, the generated cascading updated document-within-deduplication entity relationship features can be directly used as the core input for subsequent cross-document entity alignment processing and the construction of the global entity alignment knowledge base. The entity name, entity type, and key attribute fields can also be directly used to generate entity attribute fingerprint features, ensuring the logical coherence and data consistency of the entire technical chain.

[0057] Optionally, based on the semantic merging decision result, a name mapping table is constructed to characterize the entity renaming and merging mapping status for entity reference relationship filtering, so as to generate cascaded updated in-document deduplicated entity relationship features, including: Based on the semantic merging decision results, a name mapping table is constructed to represent the entity renaming and merging mapping status; The unstructured document is subjected to entity relationship extraction processing to extract entity relationship features, which include the reference relationship between the source entity field and the target entity field. Based on the name mapping table, the reference relationship between the source entity field and the target entity field is filtered to generate the deduplicated entity relationship feature within the document after cascading updates.

[0058] Preferably, in the specific implementation of step 3, based on the semantic merging decision result after verification, multi-dimensional mapping relationship extraction and structured name mapping table construction are performed to provide the core mapping basis for subsequent entity reference relationship filtering and cascading updates. The semantic merging decision result includes two core contents: merged index grouping and merged description. Merged index grouping groups semantically consistent entity indexes within the second cluster set into the same group, ensuring that each entity index belongs to only one group, with no duplication or omission. The merged description includes the unified entity name corresponding to each group, the integrated entity description field, and the semantic judgment basis for merging or splitting. During the mapping relationship extraction process, three types of mapping relationships with clear business meanings are extracted from the semantic merging decision results: The first type is the merging mapping relationship, which maps each original entity identifier and original entity name in the same group to the target entity identifier and unified entity name corresponding to the group, respectively, for entities with different names and similarities in the merged index group, thus completely recording the unique identifier correspondence before and after entity merging; The second type is the renaming mapping relationship, which maps the original name of the entity to the new entity name with added semantic differentiation description for entities that are determined to be independent entities due to homonyms and need to be supplemented with distinguishing identifiers, thus resolving the ambiguity problem of the same name corresponding to different semantic entities in the same text; The third type is the independent mapping relationship, which maps the original index and original name of the entity to itself for single entity clusters that do not need to be merged, as well as independent entities retained in the downgrade process, while marking the merged status as independent, ensuring that the name mapping table covers all entities in the second cluster set without any entity omission. The completed name mapping table adopts a multi-field structured design, including eight core fields: original entity identifier, original entity name, target entity identifier, target entity name, merge status identifier, merge confidence, traceability index chain, and full text block identifier list. It fully records the entire link information of entity identifier changes, clearly represents the renaming and merge mapping status of each entity, and provides an accurate and traceable benchmark for subsequent batch updates of entity reference relationships.

[0059] Preferably, in the specific technical implementation of step 3, a three-level compliance verification process is performed on the initially constructed name mapping table to correct logical anomalies in the mapping relationship, ensure the rigor and usability of the name mapping table, and provide reliable basic data for subsequent entity reference filtering processing. The first level of verification is a completeness verification, which uses the entity index full set comparison method to perform a difference operation on all original entity identifiers recorded in the name mapping table and all entity indexes contained in the second cluster set. If the difference result is empty, it is determined that the name mapping table completely covers all entities and there is no index omission; if the difference result is not empty, the missing entity index is located, and the mapping relationship of the corresponding entity is supplemented from the semantic merging decision result until all entity indexes are covered. The second level of verification is a uniqueness verification, which verifies that each original entity identifier in the name mapping table corresponds to only one target entity identifier and target entity name, and prevents one-to-many logical conflicts where one original entity maps to multiple target entities; for one-to-many conflict records found in the verification, the corresponding merged index group in the semantic merging decision result is located, the entity's belonging group is reconfirmed, and the mapping relationship is corrected until the uniqueness requirement is met. The third level of verification is a loop-free check. It iterates through all mapping records in the name mapping table to verify that there are no logical anomalies in circular mapping, i.e., there is no closed-loop structure where the original entity A maps to entity B, and entity B maps back to entity A. For circular mapping records found during verification, the entities involved are reset to an independent mapping state, the semantic merging decision result and the name mapping table are updated synchronously, and an anomaly repair log is recorded. After all three levels of verification pass, the name mapping table is stored using a hybrid storage architecture combining Bloom filters and hash tables. The Bloom filter is used to quickly determine whether the original entity identifier has a corresponding mapping record, and the hash table is used to store complete mapping details. This optimizes the query time complexity of a single mapping record to constant level, enabling efficient batch updates and queries of massive entity reference relationships.

[0060] Preferably, in the application scenario of automated enterprise knowledge graph construction, step 3 specifically involves performing full-volume, batch entity relation extraction processing on the original input unstructured document to extract entity relation features covering all entity associations within the document, providing complete basic data for subsequent entity reference relation filtering. The unstructured document is split into text blocks as the smallest processing unit, with each text block corresponding to a unique text block identifier, ensuring complete consistency with the text block identifiers in the entity extraction stage, thus ensuring the traceability of the source information of entities and relations. Entity relation extraction employs a hybrid extraction strategy that deeply integrates rules and pre-trained models, divided into two parallel processing branches: explicit relation extraction and implicit relation extraction. Explicit relation extraction is implemented based on a domain-adapted rule base, which contains three core rule categories: subject-verb-object sentence template rules, relation trigger word matching rules, and entity type association rules. The rule base can automatically mine and supplement new rule entries based on newly added labeled data, adapting to the knowledge graph construction needs of different industry domains. Implicit relation extraction is implemented based on a pre-trained relation extraction model that has been incrementally fine-tuned with domain-specific labeled data. It infers potential relationships between entities that are not explicitly expressed through explicit sentences by using the contextual semantic information of unstructured documents, thus compensating for the insufficient coverage of implicit relationships by explicit rule extraction. The extraction results from both branches are subjected to a confidence-weighted fusion process. For explicit relationships, the initial confidence score is assigned according to rule priority, with subject-verb-object sentence template rules having the highest priority, followed by relationship trigger word matching rules, and entity type association rules having the lowest priority. For implicit relationships, the model output confidence score is directly used, and the final confidence score for each relationship record is calculated using a preset weighting ratio. Relationship records with a final confidence score higher than a preset base threshold are selected to form a document-level entity relationship feature set. Each entity relationship feature in the document-level entity relationship feature set contains five core pieces of information: source entity field, target entity field, relationship type, final confidence score, and text block identifier. The source entity field corresponds to the original index and name of the entity initiating the relationship, and the target entity field corresponds to the original index and name of the entity receiving the relationship, fully preserving the original reference relationship between entities and forming an accurate correspondence with the original entity identifiers in the name mapping table.

[0061] Preferably, in the specific implementation of step 3, preprocessing and compliance verification are performed on the extracted entity relationship feature set within the document to remove invalid and low-quality relationship records, correct format anomalies, and ensure the compatibility between entity relationship features and the name mapping table, providing high-quality input data for subsequent reference relationship filtering. The preprocessing and verification process consists of four core steps: The first step is format consistency verification, verifying that each relationship record in the entity relationship feature set fully contains the five required fields: source entity field, target entity field, relationship type, final confidence level, and text block identifier, with no missing fields or data type mismatches. For records with format anomalies, entity relationship extraction of the corresponding text block is re-executed to supplement missing fields and correct data types until the format requirements are met. The second step is entity index matching verification, verifying that the original entity identifiers contained in the source entity field and target entity field of each relationship record are within the range of original entity identifiers in the name mapping table, with no invalid entity indexes exceeding the range. Relationship records containing invalid entity indexes are marked as invalid reference relationships and stored separately, not entering the subsequent filtering and update stage, to avoid mapping matching failures caused by invalid indexes. The third step is duplicate relationship deduplication. For multiple relationship records where the source entity field, target entity field, and relationship type are completely identical, the record with the highest final confidence score is retained, and the remaining duplicate records are removed to avoid redundant relationship data after subsequent updates. The fourth step is confidence score grading. Based on the business importance of the relationship type, differentiated confidence score thresholds are set for different types of relationships. Higher confidence score thresholds are set for core business relationship types, and lower confidence score thresholds are set for auxiliary and explanatory relationship types. Low-confidence relationship records below the corresponding type threshold are removed to further improve the overall data quality of the entity relationship feature set.

[0062] Preferably, in the specific technical implementation of step 3, based on the verified name mapping table, a full entity reference relationship replacement process is performed on the preprocessed entity relationship feature set to achieve cascading synchronous updates of entity identifiers and solve the technical problem of inconsistency between relationship references and entity identifiers after entity merging and renaming. The reference relationship replacement process takes a single entity relationship feature as the smallest processing unit, traverses all relationship records within the entity relationship feature set, and performs two synchronous replacement operations: The first operation is the mapping replacement of the source entity field, which matches the original entity identifier and original entity name contained in the source entity field within the relationship record with the corresponding mapping record in the name mapping table and replaces them with the target entity identifier and target entity name in the mapping record; the second operation is the mapping replacement of the target entity field, which matches the original entity identifier and original entity name contained in the target entity field within the relationship record with the corresponding mapping record in the name mapping table and replaces them with the target entity identifier and target entity name in the mapping record. During the replacement process, to address multi-source mapping conflicts—that is, situations where multiple relationship records, after replacement, have completely identical source entity indexes and target entity identifiers but differ in relationship types—a conflict fusion processing logic is triggered. Following the rule of prioritizing explicit extraction results over implicit extraction results, the relationship type with the highest confidence and most reliable source is retained, based on the final relationship confidence level from highest to lowest. Simultaneously, the conflict content, fusion basis, and processing trajectory are recorded to ensure the entire replacement process is traceable and verifiable. Through full-scale reference relationship replacement processing, the entity identifiers in all entity relationship features within the document are synchronously updated with the semantic merging decision results. This fundamentally avoids the logical discontinuity problem caused by relationship references remaining at the original entity identifiers after entity merging or renaming, providing a unified and standardized entity identifier foundation for subsequent invalid relationship filtering.

[0063] Preferably, in the specific implementation of step 3, multi-dimensional filtering is performed on the entity relationship feature set after the replacement of reference relationships to remove logically invalid or substandard relationship records and generate initially updated entity relationship features. The filtering process consists of three core dimensions, each with clearly defined filtering rules and processing logic: The first dimension is self-referencing relationship filtering. It iterates through all replaced relationship records, and for those where the source entity index and target entity identifier are completely identical after replacement, they are determined to be self-referencing relationships generated after entity merging and are directly removed from the valid data. This filtering operation can resolve the logical contradiction of relationship records that originally pointed to two entities becoming the same entity after merging synonymous entities, ensuring the logical rigor of entity relationship features. The second dimension is low-confidence relationship filtering. For each replaced relationship record, its business importance and final confidence value are compared with the corresponding confidence threshold, removing relationship records with a final confidence value lower than the corresponding threshold. This filtering operation can further filter out noisy data generated during the extraction process, improving the reliability of the final entity relationship features. The third dimension is invalid entity reference filtering. For the replaced relationship records, it is verified again that both the source entity index and the target entity identifier exist within the target entity identifier range of the name mapping table. Relationship records containing invalid target entity identifiers are removed. This filtering operation can eliminate dangling reference relationships caused by mapping anomalies during the replacement process, ensuring that the source entity and target entity of each relationship record correspond to real, deduplicated entities without any invalid references. After completing the filtering process in these three dimensions, the remaining relationship records form the preliminary updated entity relationship characteristics. All relationship records have unified entity identifiers, rigorous logical relationships, and high confidence levels.

[0064] Preferably, in the specific technical implementation of step 3, entity-relationship bidirectional consistency verification and full information completion processing are performed on the initially updated entity relationship features to generate the final cascaded updated document-within-deduplication entity relationship features. The entity-relationship bidirectional consistency verification is divided into two directions: the first direction is entity-side verification, which verifies that all target entities in the name mapping table have corresponding reference records in the updated entity relationship features; for isolated entities without any association, the unrelated identifier and corresponding entity metadata are supplemented to ensure that no entity information is lost during the deduplication and update process. The second direction is relationship-side verification, which verifies that the source entity index and target entity identifier in all updated relationship records exist within the target entity range of the name mapping table, with no invalid entity references outside the range, ensuring that each relationship record has complete logical validity. The full information completion process consists of two core operations: The first is entity metadata completion. For the merged target entity, the text block identifier list and attribute description information corresponding to all original entities within the merged index group are fully merged and deduplicated, and then added to the target entity's metadata to ensure that no source information or attribute information of the entity is lost. The second is relation tracing information completion. For each updated relation record, the original entity identifier, original text block identifier, extracted branch type, and other tracing information before the merge are added to ensure that each relation can be traced back to its original source in the unstructured document, meeting the data tracing requirements in the knowledge graph construction process. The final generated cascading updated deduplicated entity relationship features within a document consist of two core parts: a list of deduplicated entities and a list of deduplicated entity relationships. Each entity entry in the deduplicated entity list includes a unique target entity identifier, a unified entity name, a merged entity description, a full list of text block identifiers, and merged status metadata. Each relationship entry in the deduplicated entity relationship list includes a unified source entity index, a target entity identifier, a relationship type, a final confidence score, and full-link tracing information. This data achieves semantic-level deduplication of duplicate entities within the document while ensuring the cascading synchronous update and logical integrity of inter-entity reference relationships, providing high-quality foundational data for subsequent cross-document entity alignment processing, the construction of a global entity alignment knowledge base, and the generation of a unified knowledge graph.

[0065] Preferably, in the specific implementation of step 3, a full-link degradation handling and exception handling mechanism is designed to address various abnormal scenarios that may occur throughout the entire process of name mapping table construction, entity relationship extraction, and reference relationship replacement and filtering, ensuring the continuity and stability of the entire processing flow. For scenarios where the semantic merging decision result is abnormal and the name mapping table cannot be constructed, the degradation strategy adopted is to mark all entities within the second cluster set as independent entities, construct a full set of independent mapping relationship tables, and completely retain the original entity data and relationship references. After the anomaly is fixed, the merging judgment and mapping update can be re-executed without blocking the overall process. For scenarios where the effectiveness of entity relationship extraction is lower than a preset threshold and cannot cover core entity associations, the optimization strategy adopted is to increase the weight ratio of explicit rule extraction, supplement domain-specific extraction rules and sentence templates, re-execute entity relationship extraction for the corresponding unstructured document, and supplement effective relationship data. For scenarios involving widespread mapping matching failures during reference relationship replacement, the remediation strategy involves pausing the batch replacement operation, re-executing the three-level compliance check of the name mapping table, locating and correcting anomalies in the mapping relationships, and then re-executing the reference relationship replacement process. For scenarios where the number of valid relationship records is insufficient after filtering, the optimization strategy involves dynamically adjusting the confidence thresholds for different relationship types to retain more business-valuable relationship records while maintaining data quality. This end-to-end anomaly handling mechanism covers various anomaly scenarios during processing, ensuring process stability and data integrity. Furthermore, the generated cascading updated deduplicated entity relationship features within the document can be directly used as core input for subsequent cross-document entity alignment processing. The entity name, entity type, and key attribute fields can also be directly used for generating entity attribute fingerprint features and constructing a global entity alignment knowledge base, ensuring logical coherence and data consistency across the entire technical chain.

[0066] Optionally, step 4 specifically includes: Based on the entity fusion scheduling rules, the entity state parsing process is performed on the deduplicated entity relationship features within the document to mark the entities to be fused; In the entity global alignment knowledge base, a similarity search is performed on the entities to be merged based on their organization and entity type to generate a list of similarity search results; Determine the scene description features matched by the similarity search results list, and perform cross-document entity alignment processing on the entities to be fused based on the scene description features to generate cross-document fused entity results; The cross-document fusion entity results are written into the graph database, triggering atomic operations in the graph database to construct entity nodes and relation edges, thereby generating a unified knowledge graph after cross-document fusion.

[0067] Preferably, in the specific implementation of step 4, the entity fusion scheduling rule parsing and entity status classification processing are first performed on the deduplicated entity relationship features within the document to mark the entities to be merged. The deduplicated entity relationship features within the document are the data after completing the entity deduplication and relationship cascading update within a single document. They include two core parts: a list of deduplicated entities within the document and a list of deduplicated entity relationships within the document. Each entity entry in the list of deduplicated entities within the document contains a unique target entity identifier, a unified entity name, an integrated entity description field, a full list of text block identifiers, merged status metadata, and entity type information. Each relationship entry in the list of deduplicated entity relationships within the document contains a unified source entity index, target entity identifier, relationship type, final confidence level, and full-link traceability information. The entity fusion scheduling rules are designed around the business characteristics of enterprise knowledge graph construction and include three core modules: entity priority ranking rules, fusion batch division rules, and filtering and exclusion rules. The entity priority ranking rules calculate the fusion priority score for each entity based on three dimensions: the business importance of the entity type, the number of entity relationships, and the business weight of the source document. Entities with higher scores are given priority for fusion processing. The fusion batch division rules divide all entities into multiple parallel processing batches based on the entity priority score and the maximum number of entities to be processed in a single batch. The number of entities within a batch is adapted to the underlying hardware resources and interface call limitations, and batches are executed serially from high to low priority. The filtering and exclusion rules set filtering conditions for entities that have already completed cross-document fusion, are marked as not needing repeated fusion, and non-core entities extracted from temporary test documents and auxiliary instruction documents, directly excluding them from the fusion processing scope. Based on the entity fusion scheduling rules, all entities in the deduplication entity list within the document are classified into three categories: the first category is already fused entities, which are entities that already exist in the global entity alignment knowledge base and do not need to be fused again; the second category is excluded entities, which are entities that meet the filtering exclusion rules and are not included in the scope of this fusion; and the third category is entities to be fused, which are entities that have not been fused in the global entity alignment knowledge base, have business priority, and need to undergo cross-document entity alignment processing. Entities in this category are marked as to be fused and have a corresponding batch number, thus completing the standardized marking process for entities to be fused.

[0068] Preferably, in the specific technical implementation of step 4, based on the marked entities to be fused, a similarity search process involving two-dimensional pre-filtering and multi-feature fusion is performed in the entity global alignment knowledge base to generate a standardized list of similarity search results. The entity global alignment knowledge base is a pre-built knowledge base covering all fused entities. Each entity record in the base contains a globally unique entity identifier, entity name, entity type, entity attribute fingerprint features, full set of relational information, a list of source document identifiers, and graph topology relational data. Among them, the entity attribute fingerprint features are feature data that can characterize the uniqueness of entity identity, generated based on entity name, entity type, and key attribute fields through position-sensitive hashing, providing the core basis for similarity matching. The first step in similarity search processing is a two-dimensional pre-filter. The first dimension is organization-level filtering, which limits the search to entity data that belongs to the same organization as the entity to be merged, thus avoiding invalid entity matching across organizations. The second dimension is entity type-level filtering, which limits the search to entity data that is exactly the same as the entity to be merged, thus avoiding incorrect matching of different types of entities from the source. For example, enterprise type entities will not be matched with product type entities, which greatly reduces the search scope and improves matching efficiency and accuracy. The second step is a similarity search involving multi-feature fusion. For the entity range after pre-filtering, two layers of similarity matching calculation are performed: The first layer is vector semantic similarity calculation. The entity name of the entity to be fused is embedded into a vector, generating a query vector with the same dimension as the entity vector in the knowledge base globally aligned with the entity. Similarity retrieval is performed in the vector index of the knowledge base, returning candidate entities with semantic similarity higher than a preset basic threshold. The second layer is entity attribute fingerprint feature matching calculation. For the candidate entities returned by semantic retrieval, the attribute fingerprint feature matching degree between the entity to be fused and the candidate entities is calculated. Combining the semantic similarity score and the attribute fingerprint matching degree, the final comprehensive matching score is calculated through a preset weight ratio. The candidate entities are sorted from high to low according to the comprehensive matching score, and the top-ranked candidate entities are selected to form the initial search results. The third step is to standardize the similarity search results. This involves supplementing each candidate entity in the initial search results with a globally unique identifier, entity name, entity type, semantic similarity score, attribute fingerprint matching degree, comprehensive matching score, and association topology matching information, forming a standardized list of similarity search results. Each record in the list fully records the core information and matching degree data of the candidate entity, providing a complete basis for judgment in subsequent scene classification and entity alignment processing.

[0069] Preferably, in the application scenario of automated construction of enterprise knowledge graphs, step 4 specifically involves performing scene feature matching and classification processing on the generated similarity search result list to determine the scene description features matched in the list. Based on the corresponding scene description features, differentiated cross-document entity alignment processing is performed on the entities to be merged to generate cross-document fusion entity results. The scene feature matching processing is based on the core features of the similarity search result list and is divided into three mutually exclusive processing scenarios. Each scenario corresponds to a unique scene description feature and a dedicated entity alignment processing logic: The first scenario is a scenario with no candidates, where the corresponding scene description feature is that the similarity search result list is empty and there are no candidate entities that meet the pre-filtering conditions and matching thresholds. The alignment processing logic in this scenario is to determine that the entity to be merged is a completely new entity, assign it a globally unique entity identifier, integrate the full metadata, attribute information, association relationships, and source document identifier of the entity to be merged, generate the fusion result of the new entity, and directly include it in the cross-document fusion entity results. The second scenario is the exact matching scenario, characterized by the existence of a candidate entity in the similarity search results list that has a name completely identical to the entity to be fused, and the comprehensive matching score of the candidate entity is higher than the preset exact matching threshold. The alignment processing logic in this scenario is as follows: The full metadata, attribute descriptions, source document identifiers, and relationship information of the entity to be fused and the exact matching candidate entities are fed into the large language model. The large language model determines whether the two are semantically consistent entities. If the model determines they are the same entity, entity merging is performed, merging the attribute information, source document identifiers, and relationship information of the entity to be fused into the candidate entity, updating the entity's metadata and fusion record, and generating the entity merging fusion result. If the model determines they are different entities with the same name but different meanings, the large language model generates a new entity name for the entity to be fused with a semantically distinguishable description, assigns it a globally unique entity identifier, integrates all information to generate a completely new entity fusion result, and avoids ambiguity and confusion between entities with the same name. The third scenario is the similarity matching scenario. The corresponding scenario description feature is that there are no candidate entities with exact name matches in the similarity search results list, but there are similar candidate entities with a comprehensive matching score higher than the basic threshold. The alignment processing logic in this scenario is as follows: the complete information of the entity to be fused and all similar candidate entities in the list is sent to the large language model. The model determines whether the entity to be fused is the same entity with a certain candidate entity in semantics, or a completely new entity that does not match any candidate entities. If the model determines that there is a matching candidate entity, the corresponding entity merging process is performed to generate the entity merging fusion result. If the model determines that there is no matching candidate entity, a globally unique identifier is assigned to the entity to be fused to generate the fusion result of a completely new entity.For abnormal scenarios such as failure of large language model calls and output results not conforming to the verification rules, a degradation processing logic was designed. The entity to be merged is temporarily set as a brand new entity to generate the fusion result, and an exception flag is added. After the interface returns to normal, the alignment verification can be re-executed to ensure that the entire fusion process is not interrupted.

[0070] Preferably, the large language model is the Tongyi Qianwen Qwen-7B-Chat large language model. This model pre-configures prompts based on the entity merging judgment task and performs adaptation processing using domain entity samples. Its input data includes the entity name, entity type, entity description field, text block source information, entity index, and clustering source information of the two entities to be judged in the second clustering set. During the judgment process, the entity type of the two entities is first used as the admission verification criterion. When the entity types of the two entities are inconsistent and there is no preset type compatibility relationship, the judgment result of "not the same entity" is output. When the entity types of the two entities are consistent or there is a preset type compatibility relationship, the entity name, entity description field, and text block source information of the two entities are read. Semantic comparison is performed based on the relationship between full and abbreviation names, alias relationships, hierarchical organizational relationships, time position, spatial position, business scenario, and attribute description consistency to determine whether the two entities both point to the same real-world object. The Tongyi Qianwen Qwen-7B-Chat large language model outputs a common entity identifier, a judgment basis field, and a confidence level field according to a preset structure. The common entity identifier is used to indicate whether the two are semantically consistent entities. The judgment basis field is used to record the entity name, entity type, entity description field, and text block source information that trigger the same entity judgment or non-same entity judgment. The confidence level field is used to characterize the credibility of the semantic judgment result. Subsequently, a back-pointing check is performed on the judgment basis field to confirm that the judgment basis field can point back to the original description information of the corresponding entity. If the common entity identifier is yes and the back-pointing check passes, the two are determined to be semantically consistent entities. Otherwise, the two remain as independent entities.

[0071] Preferably, in the specific implementation of step 4, a full-link three-level compliance verification and standardized integration process is performed on the initially generated cross-document fusion entity results to generate a final standardized cross-document fusion entity result that can be written to storage. The three-level compliance verification consists of three stages: integrity verification, logical consistency verification, and conflict verification. Each stage has clear verification rules and exception handling logic: The first level is integrity verification, which verifies that each entity record in the cross-document fusion entity result contains seven mandatory fields: globally unique entity identifier, entity name, entity type, full attribute description, source document identifier list, relationship information, and fusion type marker. There are no missing fields or null values. For records with missing fields found during verification, the alignment process of the corresponding entity to be merged is re-executed to fill in the missing field information until the integrity verification is passed. The second level is logical consistency verification, which verifies three core aspects: First, in the entity merging results, the source and target entities are of completely identical entity types, with no logical anomalies in cross-type merging; second, in the new entity class results, the globally unique identifier of the entity does not exist in the global entity alignment knowledge base, with no identifier duplication conflicts; third, in the entity association relationship, both the source and target entities exist in the merged entity results or the global entity alignment knowledge base, with no dangling invalid relationship references. For logical anomaly records found during verification, scenario matching and alignment processing are re-executed to correct the logical anomaly until the logical consistency verification is passed. The third level is conflict verification, which focuses on the entity merging results, verifying that the merged entity attribute information has no logical conflicts. For example, there should not be two contradictory attribute values ​​for the same key attribute field. If a conflict exists, conflict resolution logic is triggered. The final retained attribute value is determined according to three dimensions: the business weight of the source document, the generation time of the attribute information, and the confidence level of the attribute description. At the same time, the conflict content and resolution trajectory are recorded to ensure that the processing process is traceable. After completing the three-level verification, the cross-document fusion entity results undergo standardized integration processing, which is divided into two parts: entity data integration and relationship data integration. During entity data integration, four metadata items are added to each entity record: a fusion timestamp, a fusion batch number, a fusion scenario marker, and an operation traceability log, forming standardized entity node data. During relationship data integration, a unified identifier replacement is performed on all relationship records associated with the entity, replacing the entity index in the relationship with the corresponding globally unique entity identifier. Simultaneously, relationship confidence, relationship source information, and a fusion timestamp are added, forming standardized relationship edge data. After standardized integration, the final cross-document fusion entity results are formed, providing compliant, complete, and consistent foundational data for subsequent multi-database synchronous writing and knowledge graph construction.

[0072] Preferably, in the specific technical implementation of step 4, the standardized cross-document fusion entity results are subjected to multi-database transactional synchronous write processing, triggering atomic operations in the graph database to complete the construction of entity nodes and relation edges, thereby generating a unified knowledge graph after cross-document fusion. Multi-database transactional synchronous write covers three types of storage media: relational databases, graph databases, and vector databases. A distributed transaction mechanism ensures that write operations on all three types of databases either succeed completely or are rolled back completely, avoiding data inconsistency issues between multiple databases. The write process for graph databases is a core component in building a unified knowledge graph. The MERGE atomic operation of graph databases is used to achieve idempotent writing of entity nodes and relation edges, avoiding data redundancy caused by duplicate node or edge creation. For entity node writing, the globally unique identifier of the entity is used as the unique matching key. The MERGE atomic operation is executed. When an existing node is matched, the ON MATCH clause is triggered, appending and updating the node attributes, adding the source document identifier, new attribute information, and relationships to the node without overwriting existing valid data. When no existing node is matched, the ON CREATE clause is triggered, creating a new entity node based on standardized entity node data, and fully writing all attribute information and metadata of the entity. For writing relation edges, the global identifier of the source entity, the global identifier of the target entity, and the relation type are used as a joint unique matching key. A MERGE atomic operation is performed. When an existing relation edge is matched, the ON MATCH clause is triggered, updating the relation's confidence, source information, and fusion timestamp. When no existing relation edge is matched, the ON CREATE clause is triggered, creating a new relation edge based on the standardized relation edge data and fully writing all attribute information of the relation. The relational database write process stores structured metadata of entities and relations, fusion operation logs, and version management information, supporting business queries and data traceability. The vector database write process adds or updates the embedded vectors and index information corresponding to entity names, providing the latest vector data for subsequent similarity searches and ensuring the continuous iteration of the entity global alignment knowledge base. After all the write operations of the three types of databases are completed, a complete unified knowledge graph after cross-document fusion is constructed based on the entity nodes and relation edges already written in the graph database. Each entity node in the graph has a globally unique identity and complete attribute information. Semantic associations are established between nodes through relation edges, which fully restores the inherent relationship between entities in unstructured documents and realizes the unified fusion and knowledge accumulation of multi-document entities.

[0073] Preferably, in the specific implementation of step 4, a post-write bidirectional consistency verification and full-link anomaly degradation implementation mechanism is designed for the entire construction process of the unified knowledge graph to ensure the data integrity, logical consistency, and process continuity of the unified knowledge graph. Post-write bidirectional consistency verification is divided into entity-side verification and relation-side verification, which are automatically triggered after multi-database synchronous writing is completed: Entity-side verification compares the number of entity nodes and the list of globally unique entity identifiers written this time with the entity records in the standardized cross-document fusion entity results one by one to verify that all entities have been successfully written to the graph database, with no entity omissions or node creation failures. It also verifies that the attribute information of the entity nodes is completely consistent with the fusion results, with no attribute loss or writing errors. Relation-side verification compares the number of relation edges and the list of joint unique keys written this time with the relation records in the standardized cross-document fusion entity results one by one to verify that all relations have been successfully written to the graph database, with no relation omissions or edge creation failures. It also verifies that the source and target nodes of the relation edges exist in the graph database, with no dangling edges or logical anomalies such as mismatched nodes and edges. To address issues such as incomplete writes and data inconsistencies detected during verification, an automatic data rollback and retry write mechanism is triggered. First, all data written in this instance is rolled back through distributed transactions in the three databases. Then, the synchronous write operation of the standardized fusion result is re-executed. If the number of retries exceeds the preset limit and still fails, the write operation is terminated, a complete exception log is retained, and the database is restored to its state before the write to avoid data pollution. The end-to-end anomaly degradation mechanism covers the entire process of cross-document entity alignment and writing. Corresponding degradation strategies are set for different anomaly scenarios: For scenarios where access to the global entity alignment knowledge base fails or similarity search cannot be executed, the degradation strategy is to mark all entities to be merged as new entities, generate the corresponding fusion results and write them to a temporary storage area. Alignment processing is re-executed after the knowledge base access is restored, without blocking the overall process. For scenarios where large language model calls fail continuously or scene alignment judgment cannot be completed, the degradation strategy is to perform hard rule judgment based on the comprehensive matching score, merging only entities with comprehensive matching scores higher than the accurate matching threshold, while other entities are temporarily marked as new entities and marked for review. Alignment verification is re-executed after the model recovers. For scenarios where graph database writing fails or node and edge construction cannot be completed, the degradation strategy is to write the standardized cross-document fusion entity results to a message queue for persistent storage, continuously retrying the write operation, and retaining complete operation logs to ensure no data loss and process recovery.

[0074] Preferably, in the specific technical implementation of step 4, for the scenario of batch processing of large-scale unstructured documents, the entity fusion scheduling rules are designed with multi-dimensional dynamic adaptation to achieve a balance between efficiency and accuracy of cross-document entity fusion. In the scenario of enterprise-level knowledge graph construction, it is often necessary to process hundreds or thousands of unstructured documents simultaneously, and the number of entities extracted can reach tens of thousands to hundreds of thousands. The traditional single-entity serial fusion method cannot meet the processing efficiency requirements of large-scale data. Therefore, the entity fusion scheduling rules have added three dynamic adaptation capabilities on the basis of the basic rules: First, the ability to dynamically adjust the batch parallelism. Based on the current CPU, memory, and network bandwidth resource usage of the server, as well as the concurrent carrying capacity of the entity global alignment knowledge base and the large language model interface, the number of entities in each fusion batch and the number of batches processed in parallel are adjusted in real time to maximize the parallel processing efficiency within the hardware and interface carrying capacity and avoid interface call failures and processing timeouts caused by excessive concurrency; Second, the ability to dynamically switch between incremental fusion and full fusion. Based on the relationship between the document being processed and the documents that have already been processed, the rules automatically determine whether to perform incremental fusion or full fusion: if the document being processed is a newly added independent document... For documents with no business relevance to previously processed documents, incremental fusion is performed, only aligning the entities to be merged extracted from the current document without a full scan of the knowledge base. If the document being processed is a revised version of a previously processed document, or has a strong business relevance to a previously processed document, full fusion is performed, performing full alignment verification on all entities in the current document and updating the relationships and attribute information of the merged entities to ensure the timeliness of the knowledge graph. Thirdly, there is the ability to dynamically adapt matching thresholds. Based on the semantic characteristics of different entity types, the matching threshold for similarity search is dynamically adjusted. For entity types with standardized names and low ambiguity, the matching threshold is appropriately lowered to expand the search range of candidate entities and improve the recall rate of synonymous entities. For entity types with short names and high ambiguity, the matching threshold is appropriately increased to narrow the search range of candidate entities and reduce the probability of incorrect matching of synonymous entities, achieving a balance between matching accuracy and recall rate for different types of entities. Through multi-dimensional dynamic adaptation design, this solution can adapt to all scenarios from incremental processing of a small number of documents to batch processing of large-scale documents, and significantly improve the processing efficiency of large-scale data while achieving fusion accuracy.

[0075] Preferably, in the specific implementation of step 4, based on the final generated cross-document fusion entity results and unified knowledge graph, incremental update processing of the entity global alignment knowledge base is performed to achieve continuous iteration of the knowledge base and continuous improvement of data accuracy. The entity global alignment knowledge base is the core foundation of cross-document entity alignment, and the completeness and accuracy of its data directly determine the effect of similarity search and entity alignment. Therefore, an incremental update mechanism linked to the fusion process is designed to ensure that the knowledge base can accumulate the latest entity fusion results in real time. Incremental update processing consists of three core steps: The first step is incremental update of entity data. For new entity class records in the cross-document fusion entity results, the globally unique identifier, entity name, entity type, entity attribute information, embedding vector, entity attribute fingerprint features, and graph topology association data of the entity are completely written into the global alignment knowledge base of the entity, and the corresponding entity record is added. At the same time, the vector index and attribute fingerprint index of the fully linked hierarchical clustering genealogy are updated. For entity merged class records in the cross-document fusion entity results, the latest attribute information of the merged entity, the list of source document identifiers, association data, updated embedding vectors and attribute fingerprint features are synchronously updated to the corresponding entity record in the fully linked hierarchical clustering genealogy, ensuring that the entity data in the fully linked hierarchical clustering genealogy is completely consistent with the entity node data in the unified knowledge graph. The second step is knowledge base quality verification and optimization. For the newly added and updated entity records, data quality verification is performed to check the completeness of entity attributes, the validity of vector indexes, and the uniqueness of attribute fingerprint features. For low-quality data discovered during verification, optimization is triggered to supplement missing attribute information and rebuild invalid vector indexes, ensuring that the data quality within the fully linked hierarchical clustering phylogenetic graph continues to meet matching requirements. Simultaneously, based on the results of this entity alignment, the weighting and threshold settings for similarity matching are optimized to improve the accuracy of subsequent similarity searches. The third step is knowledge base version management. A corresponding knowledge base version number is generated for this incremental update, recording the version update time, the number of entities updated, the list of added and modified entity identifiers, and the update operation log. Snapshot data of historical versions is also retained, supporting knowledge base version rollback and historical data traceability. This ensures that in the event of data update anomalies, a rapid recovery to a stable historical version can be achieved, enabling the continuous and stable operation of the cross-document entity fusion process.

[0076] Optionally, entity attribute fingerprint features are obtained by performing the following steps, and the global alignment knowledge base of the entity is constructed accordingly, including: Retrieve the entity name and entity type from the pre-merged entity list to extract the corresponding key attribute fields; The key attribute fields are subjected to position-sensitive hashing-based numerical mapping to generate the entity attribute fingerprint feature that represents the uniqueness of the entity identity; The entity attribute fingerprint features and the deduplicated entity relationship features within the document are subjected to graph topology association modeling processing to construct and generate the global alignment knowledge base of the entity that covers the entity attribute constraint relationships.

[0077] Preferably, the first step in generating entity attribute fingerprint features and constructing a globally aligned entity knowledge base involves performing full data retrieval and structured parsing on the pre-merged entity list to extract the key attribute fields of the corresponding entities. The pre-merged entity list is a standardized entity list generated in step 1 through hash grouping of entity name and entity type tuples and entity cardinality reduction. Each entity entry in the list contains basic information such as a one-to-one corresponding entity name, entity type, entity description field, text block identifier list, and entity extraction confidence level, and is the core processing object for extracting key attribute fields. In the structured parsing process, all entities in the pre-merged entity list are first categorized and grouped according to entity type. Corresponding attribute extraction rules are preset for different entity types. For example, enterprise entities correspond to attributes such as legal representative, registered address, establishment time, and industry; product entities correspond to attributes such as product function, parent entity, technical parameters, and application scenarios; and person entities correspond to attributes such as job title, affiliated organization, professional field, and responsibilities. Based on the preset attribute extraction rules, and combined with the entity description fields and the context of the source unstructured document, named entity recognition and attribute chain extraction technologies are used to extract core attribute information strongly related to the entity's identity from the full set of associated information, forming key attribute fields that are one-to-one bound to the entity. These key attribute fields are not simply composed of the entity name, but include multiple fields such as entity name, entity type, core identity attribute, auxiliary description attribute, source text block, and extraction confidence level. This allows subsequent position-sensitive hashing processing to simultaneously utilize the entity's name semantics, category constraints, attribute information, and source context information. The generated key attribute fields fully retain the core identity information of the entity, eliminate redundant text unrelated to the entity's identity, provide standardized input data for subsequent numerical mapping processing, and effectively avoid the problem of insufficient entity distinguishability caused by single entity name information, providing multi-dimensional semantic support for the unique representation of the entity.

[0078] Preferably, the extracted key attribute fields undergo standardization preprocessing, field labeling, and block encoding to provide compliant and unified input data for subsequent position-sensitive hash-based numerical mapping. Standardization preprocessing consists of two core steps: text cleaning and synonym normalization. The text cleaning step removes meaningless special symbols, redundant spaces, common stop words, and noise generated during extraction from the original text in the key attribute fields, retaining only valid text content directly related to the entity attributes. The synonym normalization step standardizes different textual representations of the same attribute value. For example, address information in different formats, full and abbreviated entity names, time information in different writing formats, and attribute descriptions with different synonyms are all uniformly converted into standardized textual representations in a fixed format. This ensures consistency in textual representations of the same attribute meaning and avoids deviations in entity uniqueness representation caused by differences in textual representation. In the field tagging process, a field tag is configured for each key attribute field. This tag indicates that the corresponding text attribute originates from the entity name, entity type, registered address, industry, product function, application scenario, job title, organization, source text block, or other preset attribute dimensions, thus preserving the field source of the text attribute in the entity identity representation. In the block encoding process, based on the contribution weight of the attribute field to the entity identity, the key attribute fields are divided into core identity attribute blocks and auxiliary descriptive attribute blocks. The core identity attribute blocks contain attribute fields that play a decisive role in the entity identity, while the auxiliary descriptive attribute blocks contain secondary attribute fields that supplement the entity's semantic features. Then, text segmentation and sequence encoding are performed on the core identity attribute blocks and the auxiliary descriptive attribute blocks respectively, converting the text-based attribute fields into fixed-length attribute text sequences. This ensures that the key attribute fields of different entities can be converted into processing units with a uniform format and consistent dimensions. After preprocessing, field labeling, and block encoding, the key attribute fields can clearly distinguish the source fields, semantic content, and identity contribution weights of different text attributes. This avoids subsequent hash mapping from only blurring the unstructured text as a whole, thereby reducing the calculation error of position-sensitive hashing and improving the discriminability and stability of the generated entity attribute fingerprint features.

[0079] Preferably, the intermediate conversion process from text attribute mapping to hash encoding includes the numerical encoding of text attributes. For each text attribute after standardization preprocessing and field tagging, the text attribute is first split into multiple consecutive tokens, which can be Chinese words, character fragments, number fragments, or alphanumeric combination fragments. Then, based on a pre-configured token number table, each token is converted into a corresponding token number, and based on a pre-configured field number table, the field tag corresponding to the text attribute is converted into a corresponding field number, while recording the position number of the token in its respective text attribute. Thus, each token in each text attribute forms an attribute numerical fragment containing a token number, a field number, and a position number. This attribute numerical fragment is used to simultaneously represent the text content, field source, and occurrence position of the token at the numerical level. For new tokens not included in the token number table, character-level segmentation is first performed on the new token to form multiple character fragments. Then, temporary token numbers are configured for each of the multiple character fragments, and the temporary token numbers, along with the corresponding field numbers and position numbers, form an attribute numerical fragment to avoid the problem of new tokens being unable to participate in hash encoding. After sorting all attribute value fragments within the same text attribute according to their position numbers, a sequence of attribute value fragments corresponding to that text attribute is formed. Similarly, after concatenating the sequence of attribute value fragments corresponding to all text attributes of the same entity according to their field numbers, a sequence of entity attribute value sequences is formed. This entity attribute value sequence serves as an intermediate numerical representation connecting text attributes and hash encoding. It converts the original text attributes into numerical inputs that can be directly received by position-sensitive hashing, thus explicitly realizing the technical conversion from "text attributes" to "numerical attribute inputs."

[0080] Preferably, after generating the entity attribute numerical sequence, a local fragment construction process is performed on the entity attribute numerical sequence to form a candidate numerical fragment set for position-sensitive hash mapping. During the local fragment construction process, for each attribute numerical fragment, the term ID, field ID, and position ID are read, and the preceding and following attribute numerical fragments at its adjacent positions are also read. Subsequently, the current attribute numerical fragment is combined with adjacent attribute numerical fragments to form a combined numerical fragment that can represent the local text context. The combined numerical fragment includes at least the current term ID, current field ID, current position ID, preceding term ID, following term ID, and relative position identifier within the field, so that not only the numerical information of individual terms is preserved during the hash mapping process, but also the adjacency relationship and field source relationship of terms in the text attributes. For core identity attributes such as entity name, unified social credit code, registered address, legal representative, and product model, the corresponding combined numerical fragments are classified into the core identity attribute fragment set; for auxiliary attributes such as industry description, application scenario, responsibility description, and function description, the corresponding combined numerical fragments are classified into the auxiliary description attribute fragment set. Through the above local fragment construction process, the originally continuous text attributes are converted into multiple candidate numerical fragments with field source, lexical content and positional relationship. This makes position-sensitive hashing no longer directly process the original text, but process the set of numerical fragments converted from text attributes, thus clearly defining the specific data form of the text attributes before entering hash encoding.

[0081] Preferably, the constructed candidate numerical fragment set is subjected to position-sensitive hashing-based numerical mapping processing to generate entity attribute fingerprint features that represent the uniqueness of entity identity. The core technical logic of position-sensitive hashing is to map the candidate numerical fragment set obtained by converting text attributes into low-dimensional hash codes. Candidate numerical fragment sets with similar semantics, similar field sources, and similar local fragments will have high similarity hash codes after mapping; candidate numerical fragment sets with significant semantic differences, inconsistent field sources, or large differences in local fragments will have low similarity hash codes after mapping. This characteristic is highly compatible with the business requirements of entity identity matching and entity uniqueness representation. In the specific execution process of numerical mapping, multiple rounds of position-sensitive hash mapping calculations are first performed on the core identity attribute fragment set and the auxiliary description attribute fragment set respectively; in each round of hash mapping, the term number, field number, position number, and adjacent term number in the candidate numerical fragment are read, and the corresponding intermediate hash value is generated according to the preset hash seed parameter; then, multiple intermediate hash values ​​belonging to the same attribute block in the same round are aggregated to obtain the hash sub-code corresponding to that round. A higher hash mapping round and weighting ratio are set for the core identity attribute fragment set to enhance the contribution of core identity information to the uniqueness representation of the entity; a relatively lower hash mapping round is set for the auxiliary descriptive attribute fragment set to supplement the semantic features of the entity while avoiding interference from secondary information in the core identity determination. After completing the block hash mapping, the multiple sets of hash sub-encoders output from the two attribute blocks are concatenated, weighted fusion, and truncated according to the preset weight ratio to generate a fixed-length binary hash sequence. This binary hash sequence is the entity attribute fingerprint feature representing the uniqueness of the entity identity. Through the above processing, text attributes are first converted into attribute numerical fragments and candidate numerical fragment sets, and then mapped to fixed-length hash codes by position-sensitive hashing, thus fully defining the executable technical path from text attributes to hash codes.

[0082] Preferably, to facilitate stable execution of the position-sensitive hash mapping process, the generation process of the hash sub-encode can be carried out as follows: For the core identity attribute fragment set of the same entity, each combined numerical fragment is read sequentially, and the term number, field number, position number, and adjacent term number in the combined numerical fragment are concatenated into a numerical key in a fixed order; then, the numerical key is input into multiple preset hash processing channels to generate multiple channel hash values ​​respectively; then, according to the field weight corresponding to the combined numerical fragment, the multiple channel hash values ​​are weighted and accumulated, and the corresponding binary code is generated according to the positive or negative state or high or low interval of the weighted accumulation result, thereby forming the core hash sub-encode corresponding to the core identity attribute block. For the auxiliary descriptive attribute fragment set, the same numerical key generation method is used to obtain the auxiliary hash sub-encode corresponding to the auxiliary descriptive attribute block, but the weight of the auxiliary hash sub-encode in the subsequent fusion is lower than that of the core hash sub-encode. Subsequently, the hash codes are concatenated in a fixed order: core hash code first, auxiliary hash code second, and the concatenated result is length-checked. When the length of the concatenated result exceeds the preset fingerprint length, the code bits corresponding to the core identity attribute are retained according to preset truncation rules, and the remaining bits are truncated from the code bits corresponding to the auxiliary description attribute. When the length of the concatenated result is less than the preset fingerprint length, it is padded with preset padding code bits, thus obtaining a fixed-length entity attribute fingerprint feature. The above processing ensures that each part of the hash code can be traced back to the corresponding field source and text attribute numerical fragment, avoiding the misunderstanding of "how text attributes are converted into hash codes" during review.

[0083] Preferably, the generated entity attribute fingerprint features possess two core characteristics: first, discriminability—entities with different semantics can generate significantly different hash sequences, which can be used to accurately distinguish different entities with the same name but different meanings; second, stability—different representations of the same entity, after text cleaning, synonym normalization, lexical number conversion, field number conversion, and position-sensitive hash mapping, can generate highly similar hash sequences, which can be used to match the same entity with different names but the same meaning. Furthermore, the entity attribute fingerprint features maintain a traceable binding relationship with the entity name, entity type, and key attribute fields. The system simultaneously retains a fingerprint generation source record when generating entity attribute fingerprint features. This fingerprint generation source record includes field labels involved in fingerprint generation, standardized text attributes, lexical number sequences, field numbers, position numbers, a set of candidate numerical fragments, hash mapping round identifiers, and hash sub-encoding source identifiers. The fingerprint generation source record is used to trace the source attribute of each fingerprint encoding bit during subsequent verification or conflict resolution, ensuring that the entity attribute fingerprint features are not uninterpretable random codes, but rather entity identity representation results obtained from key attribute fields through traceable numerical processing and position-sensitive hash mapping. Therefore, the entity attribute fingerprint feature can not only accurately represent the uniqueness of the entity's identity, but also support fast similarity matching in the subsequent cross-document entity alignment process, greatly reducing the computational complexity of entity matching.

[0084] Preferably, the generated entity attribute fingerprint features undergo uniqueness verification and conflict resolution to ensure a stable correspondence between the entity attribute fingerprint features and the entities, providing reliable basic feature data for the subsequent construction of the global entity alignment knowledge base. Uniqueness verification consists of two stages: full-scale conflict detection and consistency detection. The full-scale conflict detection stage performs a full pairwise comparison of the entity attribute fingerprint features corresponding to all entities to detect hash conflicts where different entities generate completely identical hash sequences. The consistency detection stage performs similarity calculations on the entity attribute fingerprint features corresponding to different representations of the same entity to verify whether the similarity of the hash sequences meets the preset threshold requirements. For any anomalies discovered during the verification process, corresponding conflict resolution processing is performed. In response to detected hash collisions—that is, different entities generating the same entity attribute fingerprint features—an attribute field supplementation and hash recalculation process is triggered. First, the differential attribute fields between the conflicting entities are extracted, such as different registration addresses, different affiliated entities, different product models, different job scopes, different source text block identifiers, or different scene description features. Then, these differential attribute fields are added to the corresponding entity's key attribute fields. Field labeling, lexical number conversion, candidate numerical fragment construction, and position-sensitive hash mapping are then re-executed to form new entity attribute fingerprint features until the hash collision is eliminated. For cases where the hash sequence similarity of different representations of the same entity does not meet the standard, synonym normalization processing of the key attribute fields is re-executed to supplement the entity's core association attribute information, optimize the weight ratio of core identity attribute blocks and auxiliary descriptive attribute blocks, and regenerate entity attribute fingerprint features with high similarity to ensure the consistency of feature representation for the same entity. After verification and conflict resolution, the entity attribute fingerprint features form a unique and stable binding relationship with the entities in the pre-merged entity list. Each entity corresponds to a unique entity attribute fingerprint feature that can stably represent its identity, which solves the problem of hash collision and feature representation inconsistency, and provides a stable entity identity index for subsequent graph topology association modeling.

[0085] Preferably, the verified entity attribute fingerprint features are combined with the deduplicated entity relationship features within the document to perform graph topology association modeling processing, thereby constructing the basic topology data layer of the entity global alignment knowledge base. The deduplicated entity relationship features within the document are the standardized processing results generated in step 3, containing two core parts: a list of deduplicated entities within the document and a list of deduplicated entity relationships within the document. The list of deduplicated entities within the document contains full metadata of the entities after semantic merging and deduplication processing, while the list of deduplicated entity relationships within the document contains complete data such as the association relationships between entities, relationship types, relationship confidence, and source information. In the specific execution of graph topology association modeling, each deduplicated entity in the deduplicated entity list within the document is first treated as an independent node unit. Each node unit is bound with full information including corresponding entity attribute fingerprint features, entity name, entity type, key attribute fields, source document identifier, and merge status metadata. Simultaneously, the entity attribute fingerprint feature is set as the unique identity index key for that node. Next, each association in the deduplicated entity relationship list within the document is treated as an independent edge unit. Each edge unit is bound with corresponding attributes such as relationship type, relationship confidence, text block identifier, and extracted branch information. The two ends of each edge unit are accurately associated with the corresponding source entity node and target entity node, ensuring that the association between edges and nodes completely corresponds to the entity associations in the original document. Through the standardized construction and association of nodes and edges, an entity relationship topology graph with entity attribute fingerprint features as the core index is formed. This fully restores the inherent semantic relationships and topological structure between entities, merging the originally scattered entity attribute information and relationship data, providing core topological data support for the global entity alignment knowledge base. Unlike traditional knowledge base architectures that only store basic entity information, this graph topology fully preserves the relationships between entities, providing semantic context and topological associations to aid in subsequent cross-document entity alignment, effectively improving the accuracy of entity alignment determination.

[0086] Preferably, based on the entity relationship topology graph generated by graph topology association modeling, attribute constraint rules are constructed and a multi-dimensional indexing system is built to generate a final globally aligned entity knowledge base covering entity attribute constraint relationships. In the attribute constraint rule construction stage, hierarchical attribute constraint rules are constructed for different types of entities. These rules specifically include five categories: correspondence constraints between entity types and attribute fields, value range and format constraints, type matching constraints of relationships between entities, uniqueness constraints of entity attribute fingerprint features, and compliance constraints of entity data source. These constraint rules are embedded in the underlying storage architecture of the knowledge base to ensure that when new or updated entity data is added to the knowledge base, the compliance of the data is automatically verified, and invalid and conflicting data is automatically blocked, thus achieving consistency and accuracy of entity data within the knowledge base from the bottom layer. In the multi-dimensional indexing system construction phase, a high-performance hash index is built using entity attribute fingerprint features as the core primary key to support fast and accurate entity matching and uniqueness verification. A categorized clustering index is built using entity type and organization as secondary classification dimensions to support pre-filtering in cross-document entity alignment, quickly narrowing the entity matching range. A full-text search index is built using the textual features of entity names as a tertiary dimension to support fuzzy and exact matching of entity names. A graph structure index is built using the topological relationships between entities as a quaternary dimension to support deep semantic matching of entities based on relationships. Integrating the entity relationship topology graph, hierarchical attribute constraint rules, and the multi-dimensional indexing system generates a complete global entity alignment knowledge base. Each entity data entry in this knowledge base possesses a unique entity attribute fingerprint feature, complete attribute information, clear topological relationships, and strict attribute constraints. This provides comprehensive data support for similarity search, scene matching, and entity alignment determination in cross-document entity alignment. It also supports incremental updates and dynamic iterations, continuously supplementing new entity data and relationships as new documents are processed, ensuring the timeliness and completeness of the global entity alignment knowledge base.

[0087] Preferably, an incremental update mechanism and a full-link data quality verification mechanism are designed for the entire lifecycle of the entity global alignment knowledge base to achieve continuous availability and data quality stability. The execution logic of the incremental update mechanism is as follows: after completing the entity deduplication and fusion processing of each batch of unstructured documents, the incremental update process of the knowledge base is automatically triggered. First, the entity attribute fingerprint features of the newly added entity are used to perform a full-scale matching search in the entity global alignment knowledge base to determine whether the entity already exists in the entity global alignment knowledge base. If it is determined to be an existing entity, then based on the newly added entity attribute information, association relationship, and source document identifier, the corresponding entity node in the entity global alignment knowledge base is appended with attributes and the relationship edge is updated. Only the newly added valid information is supplemented, without overwriting the original valid data. If it is determined to be a brand new entity, a new entity node and corresponding relationship edge are created in the entity global alignment knowledge base, and the multi-dimensional index system and attribute constraint rules are updated simultaneously to ensure that the entity global alignment knowledge base can accumulate the latest entity fusion results in real time without performing a full reconstruction, which greatly reduces the computational overhead and time cost of knowledge base updates. The execution logic of the end-to-end data quality verification mechanism is as follows: after each incremental update, a full data quality verification of the entity global alignment knowledge base is automatically triggered. This verification includes four main categories: uniqueness verification of entity attribute fingerprint features, consistency verification of the association between entity nodes and relation edges, compliance verification of attribute constraint rules, and validity verification of the multi-dimensional indexing system. For abnormal data found during the verification, the corresponding correction process is automatically triggered. At the same time, complete operation logs and version information are recorded, and a corresponding knowledge base version number and historical snapshot are generated for each update. This supports version rollback and historical data traceability of the entity global alignment knowledge base, ensuring that when data updates are abnormal, the system can quickly recover to a stable historical version and achieve continuous and stable operation of the cross-document entity alignment process.

[0088] like Figure 2 As shown, this is an embodiment of a knowledge graph entity deduplication and cross-document fusion device, which includes: The first program unit is used to extract the original entities from the unstructured document and hash them to generate a pre-merged entity list, and generate a global distance matrix based on the pre-merged entity list. The second program unit is used to perform agglomerative hierarchical clustering on the global distance matrix to construct a first cluster set with a limited cluster diameter, retrieve the sub-matrix array corresponding to the first cluster set in the global distance matrix to construct a fully linked hierarchical clustering genealogy diagram, and construct a second cluster set accordingly. The third program unit is used to perform semantic merging judgment processing on the second cluster set to obtain semantic merging decision results, and to perform entity reference relationship filtering based on the semantic merging decision results to generate cascaded updated document-deduplicated entity relationship features. The fourth program unit is used to perform cross-document entity alignment processing on the deduplicated entity relationship features within the document to generate cross-document fused entity results, and to construct entity nodes and relationship edges based on the cross-document fused entity results through a graph database to generate a unified knowledge graph after cross-document fusion.

[0089] like Figure 3 The image shows an electronic device that includes a processor and a memory. The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the steps of the aforementioned knowledge graph entity deduplication and cross-document fusion method, or implements the functions of each module of the aforementioned knowledge graph entity deduplication and cross-document fusion device.

[0090] Figures 2-3 For an exemplary description, please refer to the above. Figure 1 This will not be elaborated upon here.

Claims

1. A method for entity deduplication and cross-document fusion in knowledge graphs, characterized in that, include: Step 1: Extract the original entities from the unstructured document and hash them to generate a pre-merged entity list, and generate a global distance matrix based on the pre-merged entity list; Step 2: Perform agglomerative hierarchical clustering on the global distance matrix to construct a first cluster set with a limited cluster diameter. Retrieve the sub-matrix array corresponding to the first cluster set in the global distance matrix to construct a fully linked hierarchical clustering genealogy and construct a second cluster set accordingly. Step 3: Perform semantic merging judgment processing on the second cluster set to obtain the semantic merging decision result, and perform entity reference relationship filtering based on the semantic merging decision result to generate the deduplicated entity relationship features within the document after cascading update; Step 4: Perform cross-document entity alignment processing on the deduplicated entity relationship features within the document to generate cross-document fused entity results, and construct entity nodes and relationship edges based on the cross-document fused entity results using a graph database to generate a unified knowledge graph after cross-document fusion.

2. The knowledge graph entity deduplication and cross-document fusion method according to claim 1, characterized in that, Step 1 specifically includes: The entity names and entity types of different original entities are determined to form corresponding binary features, and the binary features are hashed to obtain a set of hash groups and the pre-merged entity list is constructed accordingly. The different entity names in the pre-merged entity list are batch-embedded and vectorized to generate entity vectors, thereby obtaining an entity vector set. Based on this, the global distance matrix that satisfies the value range constraint and represents the semantic differences between entities is generated.

3. The knowledge graph entity deduplication and cross-document fusion method according to claim 2, characterized in that, Constructing the pre-merged entity list specifically includes: The entity with the longest entity description field is identified from the original entities and used as a reference entity to perform entity cardinality reduction processing on the hash group set, thereby generating the pre-merged entity list.

4. The knowledge graph entity deduplication and cross-document fusion method according to claim 2, characterized in that, Generating the global distance matrix that satisfies the value range constraint and represents the semantic differences between entities specifically includes: The entity vectors in the entity vector set are normalized to obtain the corresponding normalized vectors, and the dot product operation is performed on every two normalized vectors to generate the embedded feature semantic association matrix. The embedded feature semantic association matrix is ​​subjected to distance mapping transformation to generate the global distance matrix that satisfies the value range constraint and represents the semantic differences between entities.

5. The knowledge graph entity deduplication and cross-document fusion method according to claim 1, characterized in that, Retrieve the submatrix array corresponding to the first cluster set in the global distance matrix to construct a fully linked hierarchical clustering genealogy graph, and construct the second cluster set accordingly, including: Based on the first cluster set, extract the corresponding sub-matrix array from the global distance matrix; Based on the sub-matrix array, the hierarchical merging path between entities is determined, and a fully linked hierarchical clustering genealogy is constructed accordingly. Based on the fully linked hierarchical clustering genealogy diagram, the second cluster set is constructed.

6. The knowledge graph entity deduplication and cross-document fusion method according to claim 1, characterized in that, Based on the semantic merging decision results, entity reference relationships are filtered to generate cascaded updated deduplicated entity relationship features within the document, including: Perform a semantic merge judgment on the second cluster set to obtain a semantic merge decision result that includes merge index grouping and merge description; Based on the semantic merging decision results, a name mapping table is constructed to represent the entity renaming and merging mapping status for filtering entity reference relationships, so as to generate cascaded updated deduplicated entity relationship features within the document.

7. The knowledge graph entity deduplication and cross-document fusion method according to claim 6, characterized in that, Based on the semantic merging decision results, a name mapping table is constructed to represent the entity renaming and merging mapping status for entity reference relationship filtering, in order to generate cascaded updated in-document deduplicated entity relationship features, including: Based on the semantic merging decision results, a name mapping table is constructed to represent the entity renaming and merging mapping status; The unstructured document is subjected to entity relationship extraction processing to extract entity relationship features, which include the reference relationship between the source entity field and the target entity field. Based on the name mapping table, the reference relationship between the source entity field and the target entity field is filtered to generate the deduplicated entity relationship feature within the document after cascading updates.

8. The knowledge graph entity deduplication and cross-document fusion method according to claim 1, characterized in that, Step 4 specifically includes: Based on the entity fusion scheduling rules, the entity state parsing process is performed on the deduplicated entity relationship features within the document to mark the entities to be fused; In the entity global alignment knowledge base, a similarity search is performed on the entities to be merged based on their organization and entity type to generate a list of similarity search results; Determine the scene description features matched by the similarity search results list, and perform cross-document entity alignment processing on the entities to be fused based on the scene description features to generate cross-document fused entity results; The cross-document fusion entity results are written into the graph database, triggering atomic operations in the graph database to construct entity nodes and relation edges, thereby generating a unified knowledge graph after cross-document fusion.

9. A knowledge graph entity deduplication and cross-document fusion method according to claim 8, characterized in that, The entity attribute fingerprint features are obtained by performing the following steps, and the global alignment knowledge base of the entity is constructed accordingly, including: Retrieve the entity name and entity type from the pre-merged entity list to extract the corresponding key attribute fields; The key attribute fields are subjected to position-sensitive hashing-based numerical mapping to generate the entity attribute fingerprint feature that represents the uniqueness of the entity identity; The entity attribute fingerprint features and the deduplicated entity relationship features within the document are subjected to graph topology association modeling processing to construct and generate the global alignment knowledge base of the entity that covers the entity attribute constraint relationships.

10. A knowledge graph entity deduplication and cross-document fusion device, characterized in that, include: The first program unit is used to extract the original entities from the unstructured document and hash them to generate a pre-merged entity list, and generate a global distance matrix based on the pre-merged entity list. The second program unit is used to perform agglomerative hierarchical clustering on the global distance matrix to construct a first cluster set with a limited cluster diameter, retrieve the sub-matrix array corresponding to the first cluster set in the global distance matrix to construct a fully linked hierarchical clustering genealogy diagram, and construct a second cluster set accordingly. The third program unit is used to perform semantic merging judgment processing on the second cluster set to obtain semantic merging decision results, and to perform entity reference relationship filtering based on the semantic merging decision results to generate cascaded updated document-deduplicated entity relationship features. The fourth program unit is used to perform cross-document entity alignment processing on the deduplicated entity relationship features within the document to generate cross-document fused entity results, and to construct entity nodes and relationship edges based on the cross-document fused entity results through a graph database to generate a unified knowledge graph after cross-document fusion.

Images