Methods for cross-partition edge consistency verification in distributed graph databases

Abstract

This invention discloses a method for cross-partition edge consistency verification in a distributed graph database, comprising the following steps: identifying cross-partition edges and generating cross-partition edge identifiers; establishing bidirectional verification digests for cross-partition edges, generating endpoint digests, attribute digests, and version digests; constructing digest buckets by partition pairs; periodically scheduling verification tasks, sequentially performing partition pair-level digest comparison, bucket-level digest comparison, edge-level digest comparison, and field-level difference location for cross-partition edges; classifying inconsistencies into structural inconsistencies, semantic inconsistencies, version inconsistencies, and additional attribute inconsistencies based on the difference type, and generating corresponding risk levels; generating repair tasks and performing consistency repair. This invention can achieve accurate discovery, step-by-step location, and hierarchical repair of cross-partition edge inconsistencies, effectively improving the reliability of distributed graph databases while reducing the cost of periodic verification, and reducing the probability of errors in path queries, neighbor queries, and graph traversal queries.

Description

Technical Field

[0001] This invention relates to the technical field of edge verification in distributed graph databases, and specifically to a method for cross-partition edge consistency verification in distributed graph databases. Background Technology

[0002] In distributed graph databases, to improve storage capacity and query throughput, graph data is typically divided into multiple partitions or storage nodes according to rules such as vertex ID, business entity type, tenant, relationship type, and graph sharding hash value. When the starting vertex and ending vertex of an edge are located in different partitions, the edge is called a cross-partition edge.

[0003] To support efficient queries, existing distributed graph databases typically store cross-partition edges in the following ways: the starting partition stores forward edge records; the ending partition stores reverse indexes, mirrored edges, or edge endpoint references; when there are many edge attributes or frequent queries, some or redundant attributes may be stored in the two partitions respectively; when data is updated, the records on both sides are maintained through distributed transactions, asynchronous replication, log replay, or background repair mechanisms.

[0004] However, in large-scale distributed graph storage scenarios, the above mechanism still has shortcomings: When network jitter, storage node failure, write timeout, log replay delay, asynchronous replication failure, or partition migration interruption occur, the starting and ending partitions may have edges on one side but not on the other, or the edge attributes, version numbers, and endpoint information on both sides may be inconsistent.

[0005] Traditional verification typically uses methods such as partition-level data volume, log position, file checksum, and full snapshot hash to detect macro-level differences, but it is difficult to pinpoint the specific cross-partition edge, endpoint, attribute field, or version where inconsistency occurs.

[0006] Consistency checks in ordinary key-value databases or relational databases often revolve around primary keys, record rows, and log sequence numbers. However, edges in graph databases possess graph semantics, including start and end points, directions, relation types, attributes, versions, and visibility. If only ordinary record checks are performed, it is difficult to simultaneously verify whether "forward edge records" and "reverse edge records" are semantically equivalent in the graph.

[0007] Directly scanning all edges and comparing them across partitions one by one would result in significant network access and random read overhead, impacting online query performance. This is especially problematic in high-edge-count scenarios such as social graphs, device graphs, and transaction graphs, where full comparison is prohibitively expensive.

[0008] Different types of inconsistencies have different impacts on business operations. For example, endpoint inconsistencies may lead to incorrect path queries, attribute inconsistencies may lead to incorrect filtering conditions, and version inconsistencies may only result in asynchronous replication delays. Existing methods typically lack classification, prioritization, and difference remediation strategies for inconsistency types. Summary of the Invention

[0009] This invention provides a method for cross-partition edge consistency verification in distributed graph databases to solve the aforementioned technical problems, specifically employing the following technical solution: A method for cross-partition edge consistency verification in a distributed graph database includes the following steps: S1: When making an edge write request, obtain the starting vertex identifier V. s Endpoint vertex identifier V t The starting point belongs to the partition P. s The destination partition P t Edge type R, edge direction D, edge attribute set A, edge version number Ver, and edge state State, when P s ≠P t When this happens, mark the edge as a cross-partition edge and generate a globally unique cross-partition edge identifier EdgeKey; S2: In the starting partition P s Establish a positive verification digest F At the endpoint partition P t Establish a reverse verification digest R Digest F and Digest R Each includes the cross-partition edge identifier EdgeKey, partition identifier, endpoint digest EndDigest, attribute digest AttrDigest, version digest VerDigest, state digest StateDigest, and digest generation time TimeStamp; S3: Build summary buckets by partition, divide cross-partition edges into buckets, and maintain a bucket-level aggregated summary (BucketDigest) in each bucket. S4: Periodically schedule verification tasks, perform hierarchical comparisons of cross-partition edges according to the verification period T, and sequentially perform partition-level summary comparison, bucket-level summary comparison, edge-level summary comparison and field-level difference location to locate the inconsistency range of the edges level by level; S5: Based on the type of difference, inconsistencies are classified into structural inconsistencies, semantic inconsistencies, version inconsistencies, and additional attribute inconsistencies, and corresponding risk levels are generated; S6: Generate a repair task for inconsistent edges. The repair task includes EdgeKey, starting partition, ending partition, difference type, risk level, base version and repair strategy, and performs consistency repair.

[0010] Furthermore, the method for generating EndDigest is as follows: For forward edge records, generate the forward endpoint canonical string:

[0011] And calculate:

[0012] For reverse records, generate the reverse endpoint canonical string:

[0013] And calculate:

[0014] like If the endpoints, directions, edge types, graph spaces, or tenant identifiers are inconsistent, it is determined that there are inconsistencies.

[0015] Furthermore, the AttrDigest generation method is as follows: The edge attributes are divided into core attribute group A. c Filter attribute group A f and additional attribute group A e After generating summaries for each attribute group, the following calculations are performed:

[0016] The core attribute group consists of attributes that affect path queries, permission judgments, and relational semantics; the filtering attribute group consists of attributes used for filtering queries based on timestamps, tags, regions, and device types; and the supplementary attribute group consists of extended attributes used for display, notes, or low-frequency queries.

[0017] Furthermore, the method for generating VerDigest is as follows: Maintain logical version number:

[0018] Where WriteEpoch is the write cycle or global logical clock, Seq is the write sequence number within the partition, WriterId is the write node identifier, and OpType is the operation type. And calculate: .

[0019] Furthermore, the method for comparing partition-level summaries is as follows: Calculate the partition-level summary:

[0020] Wherein, PairKey represents a directed partition pair (P i→P j TotalCount represents the number of cross-partition edges for this partition; XorBucketDigest represents the XOR aggregation of all bucket-level digests for this partition; MaxPairVer and MinPairVer represent the maximum and minimum versions within this partition, respectively; StateStatAll represents the number of states such as valid, deleted, and migrated; DigestEpoch represents the digest generation batch. The starting partition stores the digest of the forward partition pairs. The reverse partition pairs are recovered from the endpoint partition based on the reverse digest set. The same PairKey, the same bucket partitioning rules, and the same normalization order are used for calculation; if the two are equal, the partition pair is considered to be consistent; otherwise, bucket-level comparison is performed.

[0021] Furthermore, the method for bucket-level digest comparison is as follows: The BucketDigest of the starting and ending partitions is compared bucket by bucket. If the digests of a bucket match, the bucket is skipped; if the digests of a bucket do not match, the bucket is added to the set of differing buckets. DiffBuckets={B_k|BucketDigestF(B_k)≠BucketDigestR(B_k)}; The method for edge-level summary comparison is as follows: edge-level summaries in the difference bucket are compared by EdgeKey, and edge-level difference comparison is performed in cases where the forward summary exists but the reverse summary does not exist, the reverse summary exists but the forward summary does not exist, the endpoint summaries are inconsistent, the attribute summaries are inconsistent, the version summaries are inconsistent, or the status summaries are inconsistent.

[0022] Furthermore, for edges with inconsistent endpoint summaries, inconsistent attribute summaries, and inconsistent version summaries, field-level difference localization is performed separately, including the following methods: For edges with inconsistent attribute summaries, the differences are located by attribute group summaries, including: comparing differences in core attribute groups, differences in filtered attribute groups, and differences in additional attribute groups. For edges with inconsistent endpoint summaries, compare whether the start point, end point, edge direction, edge type, graph space, or tenant are consistent. For edges with inconsistent version summaries, compare whether the write cycle, write sequence number, write node, and operation type are consistent.

[0023] Furthermore, the risk levels in step S5 include: Level 1 risks of structural inconsistency include missing forward edges, missing reverse edges, inconsistent start or end points, inconsistent edge directions, or inconsistent edge types. Secondary risks of semantic inconsistency include inconsistencies in core attribute groups, filter attribute groups, edge states, or deletion states. Level 3 risks of version inconsistency include inconsistent version numbers but consistent attributes, inconsistent write nodes but consistent logical content, or version differences within the allowed replication delay window. Level 4 risk of inconsistent additional attributes includes inconsistent display attributes, inconsistent remarks attributes, or inconsistent low-frequency extended attributes.

[0024] Furthermore, the method for performing the repair task is as follows: The repair work will be carried out step by step in the order of Level 1 risk, Level 2 risk, Level 3 risk, and Level 4 risk. For the same edge, first add a short-term repair lock or an idempotent repair flag to avoid multiple repair tasks concurrently covering the same edge, and repair tasks between different EdgeKeys are executed in parallel; After the repair is completed, the bidirectional verification digest of the edge is regenerated and the edge-level verification is triggered: if the edge-level verification passes, the corresponding BucketDigest and PairDigest are updated; if the edge-level verification fails, the next stage repair task is generated again according to the new difference type; if the number of consecutive retries exceeds the threshold or there are still conflicts between the endpoint, tenant, and edge type, the repair is automatically upgraded to manual confirmation and an alarm is issued.

[0025] Further, the remediation strategies include: log replay remediation, high version overwrite remediation, main-side record remediation, deletion consistency remediation, manual confirmation remediation, and delay observation remediation. When a summary is missing on one side but the change log exists, log replay can be used to repair it. When there are differences in core attribute groups, filter attribute groups, or states, and there is a clearly higher version, the higher version will be used to overwrite and fix the issue. When a summary is missing on one side and the log is unavailable, but the primary side record is reliable, the primary side record is used for repair. When one side is in a deleted state and the other side is in a valid state, first verify the deletion log and tombstone record, and then use deletion consistency repair. When endpoints, directions, edge types, or tenant identifiers conflict and the actual record cannot be determined from the logs, manual confirmation and repair are initiated. When VerDiges are different and within the replication delay window, delayed observation repair is used.

[0026] The advantage of this invention lies in the cross-partition edge consistency verification method provided for distributed graph databases. By establishing a bidirectional verification digest containing graph semantics for cross-partition edges and performing hierarchical comparison of the constructed digest buckets according to partitions, it achieves accurate discovery, step-by-step location, and hierarchical repair of inconsistencies in cross-partition edges. While reducing the cost of periodic verification, it effectively improves the reliability of distributed graph databases and reduces the probability of errors in path queries, neighbor queries, and graph traversal queries. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of a cross-partition edge consistency verification method in a distributed graph database according to the present invention. Detailed Implementation

[0029] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0030] The present invention provides a method for cross-partition edge consistency verification in a distributed graph database, which is applied to a cross-partition edge consistency verification system, such as... Figure 1 As shown, it specifically includes: A method for cross-partition edge consistency verification in a distributed graph database includes the following steps: S1: When making an edge write request, obtain the starting vertex identifier V. s Endpoint vertex identifier V t The starting point belongs to the partition P. s The destination partition P t Edge type R, edge direction D, edge attribute set A, edge version number Ver, and edge state State, when P s ≠P t When the edge is marked as a cross-partition edge, a globally unique cross-partition edge identifier EdgeKey is generated.

[0031] The above set of edge attributes .

[0032] Specifically when P s ≠P tWhen this happens, mark the edge as a cross-partition edge and generate a globally unique cross-partition edge identifier:

[0033] in, Here, is the hash function, Tenant is the tenant identifier, and Graphld is the graph space identifier. This method enables the generation of distinguishable cross-partition edge identifiers even when multiple relation types or edges with different directions exist between the same pair of vertices.

[0034] In other words, in a distributed graph database, when the starting vertex and the ending vertex of an edge are located in different partitions, the edge is considered a cross-partition edge. Because existing technologies lack a bidirectional verification mechanism that can simultaneously represent forward and reverse records, the records on both sides cannot be accurately located and compared. Step S1 obtains the starting vertex identifier, ending vertex identifier, the partition to which the starting vertex belongs, the partition to which the ending vertex belongs, the edge type, the edge direction, the set of edge attributes, the edge version number, and the edge status. When the starting partition and the ending partition are different, the edge is marked as a cross-partition edge, and a globally unique cross-partition edge identifier, EdgeKey, is generated. This identifier is calculated by hashing the starting vertex identifier, ending vertex identifier, edge type, edge direction, tenant identifier, and graph space identifier. This allows for the generation of a distinguishable cross-partition edge identifier even when there are multiple relationship types or multiple edges with different directions between the same pair of vertices, thus providing a unified association basis for the subsequent establishment of bidirectional verification digests. Step S1 solves the problem of existing technologies lacking a globally unique identifier for cross-partition edges, making bidirectional association verification difficult.

[0035] S2: In the starting partition P s Establish a positive verification digest F At the endpoint partition P t Establish a reverse verification digest R Digest F and Digest R Each includes the cross-partition edge identifier (EdgeKey), partition identifier, endpoint digest (EndDigest), attribute digest (AttrDigest), version digest (VerDigest), state digest (StateDigest), and digest generation time (TimeStamp). Specifically: Forward Validation Digest F Stored in the starting partition, used to describe the origin. point to Edge records. Includes:

[0036] Reverse verification digest RStored in the endpoint partition, this describes the reverse reference, mirror edge, or reverse index record corresponding to the forward edge. It includes:

[0037] Endpoint Digest is used to verify whether the start point, end point, direction, and edge type are consistent; Attr Digest is used to verify whether the edge attributes are consistent; Ver Digest is used to verify whether the edge version is consistent; State Digest is used to verify whether the edge lifecycle state is consistent; and Digest Generation TimeStamp is used to determine whether it is within a reasonable replication delay window.

[0038] In this scheme, the forward checksum is stored in the starting partition, describing the edge record from the starting vertex to the ending vertex; the reverse checksum is stored in the ending partition, describing the reverse reference, mirror edge, or reverse index record corresponding to the forward edge. Both use the same cross-partition edge identifier and unified logical direction, without changing the unified logical direction of the edge. This constraint clearly defines the storage location and functional positioning of the forward and reverse summaries, ensuring that even if the reverse record uses the ending vertex as its local vertex in physical storage, its summary is restored to the same logical direction as the forward record during generation, thus guaranteeing that the summaries on both sides can be equivalently compared in the same normalized check space. This scheme ensures that forward and reverse edge records can be uniformly compared at the graph semantic level even if their physical storage structures are different, improving the accuracy of bidirectional checksums.

[0039] In other words, consistency checks in ordinary key-value databases or relational databases mainly revolve around primary keys, record rows, and log sequence numbers. However, edges in graph databases possess graph semantics such as start point, end point, direction, relation type, attribute, version, and visibility. Verifying only ordinary records makes it difficult to simultaneously verify whether forward and reverse edge records are semantically equivalent. Step S2 establishes a forward verification digest in the start-point partition and a reverse verification digest in the end-point partition. Both digests include cross-partition edge identifiers, partition identifiers, endpoint digests, attribute digests, version digests, status digests, and digest generation times. Thus, by establishing bidirectional verification digests containing graph semantic information in both partitions for each cross-partition edge, forward and reverse edge records, even with different physical storage formats, can be mapped to the same normalized verification space for consistency judgment. This solves the problem of existing technologies lacking a graph-semantic-oriented bidirectional verification mechanism for cross-partition edges.

[0040] S3: Construct summary buckets by partition, dividing cross-partition edges into buckets, and each bucket maintains a bucket-level aggregated summary (BucketDigest). Specifically: For partition pair P i ,P j, , all from P i Point to P j Cross-partition edges are bucketed as follows:

[0041] Where N is the number of buckets, which can be dynamically adjusted according to the number of cross-partition edges.

[0042] Each bucket maintains a bucket-level aggregate summary:

[0043] Where Count is the number of cross-partition edges within a bucket; XorDigest is the XOR aggregation value of the edge-level summary within a bucket; MaxVer is the maximum version within a bucket; MinVer is the minimum version within a bucket; and StateStat is the count of different edge states.

[0044] In other words, directly scanning all edges and comparing them one by one across partitions would result in significant network access and random read overhead, impacting online query performance. This is especially problematic in high-edge-count scenarios such as social graphs, device graphs, and transaction graphs, where full comparisons are prohibitively expensive. Step S3 constructs summary buckets based on partition pairs, dividing cross-partition edges into buckets, with each bucket maintaining a bucket-level aggregated summary. By organizing a large number of cross-partition edges into summary buckets according to partition pairs and replacing edge-by-edge comparisons with bucket-level aggregated summaries, the system can quickly determine which range within a partition pair exhibits a difference without immediately reading all edge-level summaries. This significantly reduces the network communication and storage scanning overhead of periodic verification, solving the problem of high periodic verification costs in existing technologies.

[0045] S4: Periodically schedule verification tasks, perform hierarchical comparisons of cross-partition edges according to the verification period T, and sequentially perform partition-level summary comparison, bucket-level summary comparison, edge-level summary comparison and field-level difference location to locate the inconsistency range of the edges level by level.

[0046] Specifically, the system sets up a consistency check scheduler to periodically check cross-partition edges according to the check period T. The check period can be dynamically adjusted according to business risks: a shorter period is used for hot partition pairs; a shorter period is used for partition pairs with frequent writes; a longer period is used for partition pairs with historically cold data; the check frequency is temporarily increased for partition pairs that have recently experienced failures, migrations, or replication delays; and the check frequency is reduced for low-risk tenants or low-frequency relationship types.

[0047] The scheduler generates verification tasks:

[0048] Among them, CheckLevel includes partition-level, bucket-level, edge-level, and field-level; BucketRange represents the range of summary buckets covered by this verification task, which can be a continuous bucket range or a discrete bucket set; Priority represents the priority of the verification task, which is used to determine the scheduling order and resource quota.

[0049] Specifically, as one specific implementation method, the above-mentioned method for comparing partition-level summaries is as follows: Calculate the partition-level summary:

[0050] Wherein, PairKey represents a directed partition pair (P i →P j TotalCount represents the number of cross-partition edges for this partition; XorBucketDigest represents the XOR aggregation of all bucket-level digests for this partition; MaxPairVer and MinPairVer represent the maximum and minimum versions within this partition, respectively; StateStatAll represents the number of states such as valid, deleted, and migrated; DigestEpoch represents the digest generation batch. During the comparison, the starting partition P i Saved forward partition summary The reverse partition pairs are recovered from the endpoint partition based on the reverse digest set. The same PairKey, the same bucket partitioning rules, and the same normalization order are used for calculation; if the two are equal, the partition pair is considered to be consistent; otherwise, bucket-level comparison is performed.

[0051] This scheme calculates partition pair-level summaries that include partition pair identifiers, total number of cross-partition edges, XOR aggregation of all bucket-level summaries, maximum version, minimum version, state count, and summary generation batch. The forward partition pair summaries saved in the starting partition and the reverse partition pair summaries recovered from the reverse summary set of the ending partition are calculated using the same partition pair identifier, the same bucket partitioning rules, and the same normalization order. If they are equal, the partition pair is considered consistent; otherwise, bucket-level comparison is performed. This scheme, through rapid comparison of partition pair-level summaries, can quickly filter out consistent partition pairs when the inconsistency range is small, avoiding unnecessary drill-down comparisons, significantly reducing the computational overhead of full verification, and improving the efficiency of periodic verification.

[0052] Furthermore, the method for bucket-level summary comparison is as follows: The BucketDigest of the starting and ending partitions is compared bucket by bucket. If the digests of a bucket match, the bucket is skipped; if the digests of a bucket do not match, the bucket is added to the set of differing buckets. DiffBuckets={B_k|BucketDigestF(B_k)≠BucketDigestR(B_k)}.

[0053] Based on the above set of difference buckets, edge-level summaries in the difference buckets are compared using EdgeKey, including: It can identify cases where a forward digest exists but a reverse digest does not exist, a reverse digest exists but a forward digest does not exist, endpoint digests are inconsistent (both forward and reverse digests exist), attribute digests are inconsistent (both forward and reverse digests exist), version digests are inconsistent (both forward and reverse digests exist), or state digests are inconsistent (both forward and reverse digests exist).

[0054] Furthermore, for edges with inconsistent endpoint summaries, inconsistent attribute summaries, and inconsistent version summaries, field-level difference localization is performed, including: For edges with inconsistent attribute summaries, locate the differences by attribute group summaries, including comparing differences in core attribute groups, filtering differences in attribute groups, and comparing differences in additional attribute groups. For edges with inconsistent endpoint summaries, compare whether the start point, end point, edge direction, edge type, graph space, or tenant are consistent. For edges with inconsistent version summaries, compare whether the write cycle, write sequence number, write node, and operation type are consistent.

[0055] This scheme compares the bucket-level aggregated summaries of the starting and ending partitions bucket by bucket. If the summaries of a bucket match, the bucket is skipped; otherwise, it is added to the difference bucket set. The edge-level summary comparison method compares the edge-level summaries in the difference buckets according to the cross-partition edge identifier, identifying cases where the forward summary exists but the reverse summary does not, the reverse summary exists but the forward summary does not, endpoint summaries are inconsistent, attribute summaries are inconsistent, version summaries are inconsistent, or state summaries are inconsistent. Then, for edges with inconsistent endpoint summaries, attribute summaries, and version summaries, field-level difference localization is performed. This step-by-step comparison scheme quickly locates the difference range through bucket-level summaries, performing edge-level comparison only on edges within the difference buckets. Beneficial effects: Based on the inconsistencies found through partition-level summary comparison, it further narrows down the range of edges that need to be compared line by line, achieving rapid convergence of the inconsistency range and reducing network transmission and storage read overhead.

[0056] In summary, existing consistency detection technologies have a coarse-grained approach. Traditional verification typically uses methods such as partition-level data volume, log position, file checksum, and full snapshot hashing to detect macroscopic differences, but struggles to pinpoint the specific cross-partition edge, endpoint, attribute field, or version where inconsistency occurs. Step S4 periodically schedules verification tasks, performing layered comparisons of cross-partition edges according to the verification cycle, sequentially executing partition-level summary comparison, bucket-level summary comparison, edge-level summary comparison, and field-level difference location. By quickly determining the existence of differences through partition-level summaries, locating the range of differences through bucket-level summaries, identifying specific inconsistent edges through edge-level summaries, and determining the type of difference through field-level difference location, a multi-level progressive location from macro to micro is achieved, solving the problem of coarse-grained consistency detection and the inability to accurately locate specific inconsistent edges and difference types in existing technologies.

[0057] S5: Based on the type of difference, edge inconsistencies are categorized into structural inconsistencies, semantic inconsistencies, version inconsistencies, and additional attribute inconsistencies, and corresponding risk levels are generated. Specifically: Risk levels include: The first-level risk of structural inconsistency includes missing forward edges (with missing forward edges in reverse edges), missing reverse edges (with missing reverse edges in forward edges), inconsistent start or end points, inconsistent edge directions, or inconsistent edge types. Such inconsistencies may directly lead to path query errors, missing neighbor queries, or incorrect associations, and should be repaired first. Secondary risks of semantic inconsistency include inconsistencies in core attribute groups, filter attribute groups, edge states, or deletion states. Such inconsistencies may lead to incorrect business query results, incorrect permission judgments, or incorrect condition filtering, and should be fixed with higher priority. Level 3 risk of version inconsistency includes version numbers that are different but attributes that are the same, write nodes that are different but logical content that is the same, or version differences that are within the allowed replication delay window. This type of inconsistency can be observed first, and upgraded to Level 2 risk if the delay threshold is exceeded. Level 4 risks of inconsistent additional attributes include inconsistent display attributes, inconsistent remarks attributes, or inconsistent low-frequency extended attributes. These types of inconsistencies can be fixed with low priority to avoid affecting the main business process.

[0058] Inconsistencies are determined according to the following procedure: (1) First determine whether DigestF and DigestR exist simultaneously. If only one side of the digest exists, it means that the corresponding record of the cross-partition edge is missing in the starting partition or the ending partition, and it is directly judged as a structural inconsistency of level one risk.

[0059] (2) If both DigestF and DigestR exist, then compare EndDigest. If EndDigest is inconsistent, continue to compare the start point, end point, edge direction, edge type, graph space and tenant identifier item by item; if any field is inconsistent, it is determined to be structurally inconsistent.

[0060] (3) If EndDigest is the same, then compare AttrDigest and StateDigest. If the core attribute group summary H(A) is the same, then compare AttrDigest and StateDigest. c ) or filter attribute group summary H(A) f If there is an inconsistency, or if the StateDigest shows a conflict between the deletion status, migration status, and valid status, it is judged as a semantic inconsistency of level two risk; if only the attribute group summary H(A) is attached... e If they are inconsistent, it is determined that the additional attributes of the level four risk are inconsistent.

[0061] (4) If EndDigest, AttrDigest, and StateDigest are all consistent, but VerDigest is inconsistent, then a version difference determination is performed. The system compares WriteEpoch, Seq, WriterId, OpType, and TimeStamp on both sides; if it is within the allowed replication delay window, it is marked as a delay observation and the overwrite repair is not performed for the time being; if it has not converged after exceeding the replication delay window and the logical content on both sides is consistent, it is determined to be a version inconsistency of level three risk; if the version difference is accompanied by core attributes, filter attributes, or state differences, it is treated as semantic inconsistency; if the version difference is accompanied by endpoint, direction, edge type, graph space, or tenant identifier differences, it is treated as structural inconsistency.

[0062] By following the above sequence, the system first eliminates high-risk structural issues such as missing records and incorrect endpoint directions, then judges differences in business semantic attributes and states, and finally processes observable version differences and low-frequency additional attribute differences. This avoids misjudging replication delays as real data errors and also avoids mistreating endpoint errors as ordinary attribute differences.

[0063] In existing technologies, the remediation process lacks risk levels. Different types of inconsistencies have varying impacts on business operations, and existing methods typically lack classification, prioritization, and differential remediation strategies for inconsistency types. Step S5 categorizes inconsistencies into structural inconsistencies, semantic inconsistencies, versioning inconsistencies, and additional attribute inconsistencies based on their type, generating corresponding risk levels. This clarifies the specific components of the four risk levels, enabling the system to prioritize inconsistencies based on the severity of their impact on business operations. This provides a clear hierarchical basis for implementing differentiated remediation strategies, ensuring that high-risk inconsistencies are handled first, while low-risk inconsistencies are handled later, improving the security and efficiency of the remediation process. Thus, by classifying inconsistencies according to their severity of impact on business operations, this solution allows the system to prioritize structural inconsistencies that may lead to path query errors or missing neighbor queries, while delaying the handling of additional attribute inconsistencies that only affect display or notes. This avoids the security issues associated with a one-size-fits-all approach to remediation.

[0064] S6: Generate a repair task for inconsistent edges. The repair task includes EdgeKey and starting partition P. s Endpoint partition P t The calculations are as follows: DiffType (difference type), RiskLevel (risk level), BaseVersion (base version), and RepairPolicy (repair policy), followed by consistency repair.

[0065] The remediation strategies include: Log replay repair refers to replaying missing operations based on the written logs or edge change logs; High version overwrite repair refers to the process where, when there is a conflict between versions on both sides, the higher version (log index or transaction ID) is used as the record. Main-side record repair refers to repairing mirrored records based on preset main partition records; Deletion consistency repair refers to the unified deletion after confirmation based on the deletion log when one side is in a deletion state and the other side is still in a valid state; Manual confirmation and repair refers to the process where endpoint or edge type conflicts occur and cannot be automatically determined, and the conflict is entered into the manual review queue. (6) Delayed observation repair means that when the difference is within the copy delay window, it is not repaired temporarily, and only the observation task is recorded.

[0066] In the above scheme, the correspondence between inconsistency types and specific remediation strategies includes: When a summary is missing on one side but the change log exists, log replay can be used to repair it. When there are differences in core attribute groups, filter attribute groups, or states, and there is a clearly higher version, the higher version will be used to overwrite and fix the issue. When a summary is missing on one side and the log is unavailable, but the primary side record is reliable, the primary side record is used for repair. When one side is in a deleted state and the other side is in a valid state, first verify the deletion log and tombstone record, and then use deletion consistency repair. When endpoints, directions, edge types, or tenant identifiers conflict and the actual record cannot be determined from the logs, manual confirmation and repair are initiated. When version digests (VerDiges) differ and are within the copy delay window, delayed observation repair is used.

[0067] Inconsistencies can easily arise between records on opposite sides of a cross-partition edge. When network jitter, storage node failures, write timeouts, log replay delays, asynchronous replication failures, or partition migration interruptions occur, the starting and ending partitions may have an edge on one side but not the other, or inconsistencies in edge attributes, version numbers, and endpoint information. Existing repair methods lack targeted strategies. Step S6 generates a repair task for the inconsistent edge, including the cross-partition edge identifier, starting partition, ending partition, difference type, risk level, base version, and repair strategy, and executes consistency repair. This achieves differentiated and precise repair by matching corresponding repair strategies based on the inconsistency type and risk level, including log replay repair, high-version overwrite repair, master-side record repair, deletion consistency repair, manual confirmation repair, and delay observation repair, avoiding blind overwrite or erroneous repair. The above solution matches different repair strategies based on the specific type of inconsistency and available information, avoiding blind overwrite and achieving precise matching between repair strategies and inconsistency types. While ensuring repair accuracy, it introduces a manual confirmation mechanism for complex conflicts that cannot be automatically identified, and adopts an observation strategy for differences within the normal replication delay range, balancing the degree of automation and security in the repair process.

[0068] The above-mentioned repair task is executed as follows: The repair process is carried out in the order of Level 1 risk, Level 2 risk, Level 3 risk and Level 4 risk, that is, first deal with Level 1 structural inconsistencies, then deal with Level 2 semantic inconsistencies, then deal with Level 3 version inconsistencies that exceed the delay threshold, and finally deal with Level 4 additional attribute inconsistencies. For the same edge, first add a short-term repair lock or an idempotent repair flag to avoid multiple repair tasks concurrently covering the same edge, and repair tasks between different EdgeKeys are executed in parallel; After the repair is completed, the bidirectional verification digest of the edge is regenerated and the edge-level verification is triggered. If the edge-level verification passes, the corresponding BucketDigest and PairDigest are updated. If the edge-level verification fails, the next stage repair task is generated again according to the new difference type. If the number of consecutive retries exceeds the threshold or there are still conflicts between the endpoint, tenant, and edge type, the repair is automatically upgraded to manual confirmation and an alarm is issued.

[0069] After the overall repair is completed, the system regenerates the bidirectional verification digest for that edge and adds it to the next round of verification tasks to confirm the repair result. This scheme controls the repair priority through risk level sorting, avoids concurrent conflicts through repair locks, ensures the correctness of the repair result through a re-verification mechanism, and prevents infinite loops through retry thresholds and escalation mechanisms. This ensures that the repair process is orderly, safe, and traceable, avoids secondary inconsistencies caused by concurrent repairs, and verifies the repair effect through a closed-loop re-verification mechanism. For persistent inconsistencies, manual handling is promptly escalated, improving the reliability of the repair and the stability of the system.

[0070] In summary, this invention achieves accurate discovery, step-by-step location, and hierarchical repair of inconsistencies across partition edges by establishing bidirectional verification digests containing graph semantics for cross-partition edges and performing hierarchical comparison of the constructed digest buckets according to partitions. While reducing the cost of periodic verification, it effectively improves the reliability of distributed graph databases and reduces the probability of errors in path queries, neighbor queries, and graph traversal queries.

[0071] As a specific implementation scheme, to avoid the inability to compare forward and reverse records due to different storage directions, this invention standardizes and encodes endpoint information to generate an endpoint digest (EndDigest). The method is as follows: For a forward edge record, generate a forward endpoint canonical string parallelization summary containing graph space identifier, tenant identifier, starting vertex identifier, ending vertex identifier, edge type, and edge direction:

[0072] calculate:

[0073] For the reverse record, generate a reverse endpoint canonical string containing the same content and compute the reverse endpoint digest (although its physical storage may be based on Vt as the local vertex, it is still restored to the same logical direction when generating the digest):

[0074] calculate:

[0075] like If the endpoints, directions, edge types, graph spaces, or tenant identifiers are inconsistent, it is determined that there are inconsistencies.

[0076] This scheme avoids the problem of incomparability between forward and reverse records due to different storage directions by standardizing the encoding of endpoint information. It realizes the standardized expression of cross-partition edge endpoint information, enabling direct comparison of endpoint summaries on both sides and accurately locating inconsistencies at the endpoint, direction, edge type, graph space, or tenant identifier level.

[0077] As a specific implementation method, the AttrDigest attribute summary is generated as follows: The edge attributes are divided into core attribute group A. c Filter attribute group A f and additional attribute group A e After generating summaries for each attribute group, the following calculations are performed:

[0078] in: Core Attribute Group A c These are attributes that affect path queries, permission judgments, and relational semantics, such as relation strength, status, weight, and business type. Filtering attribute group Attributes used for conditional query filtering, such as timestamp, tags, region, and device type; Additional attribute group Extended attributes for display, notes, or low-frequency queries.

[0079] In this scheme, each attribute group undergoes normalization processing before generating a summary, including: sorting attribute names in lexicographical order; unifying the precision of numeric attributes; using fixed placeholders for null values; unifying the encoding of strings; sorting array or set attributes according to preset rules; and excluding temporary attributes that do not participate in consistency checks from summary calculation.

[0080] In other words, by grouping edge attributes according to business importance, the problem of excessively high one-time validation costs when there are too many edge attribute fields is avoided. In this way, when inconsistencies are found in attributes, it is possible to further pinpoint which category of core attributes, filtering attributes, or additional attributes is causing the difference, achieving fine-grained location of attribute differences and reducing validation costs.

[0081] As a specific implementation method, the method for generating the version digest (VerDigest) is as follows: For each cross-partition edge, maintain a logical version number:

[0082] Where: WriteEpoch is the write cycle or global logical clock; Seq is the write sequence number within the partition; WriterId is the write node identifier; OpType is the operation type, including insert, update, delete, migration, etc.

[0083] Version summary calculation is as follows: .

[0084] This scheme incorporates version information into digest calculation, enabling the system to identify differences caused by asynchronous replication latency, duplicate writes, or different version replay orders. This provides a comparable basis for version consistency across partition edges. Combined with time windows, it can distinguish between normal replication latency and genuine version inconsistencies, reducing false positives. Specifically: when EndDigest and AttrDigest are consistent, but VerDigest is inconsistent, the system determines that the difference may be caused by asynchronous replication latency, duplicate writes, or different version replay orders, requiring further evaluation using time windows.

[0085] When making specific judgments based on time windows, the system first determines the higher version based on the write cycle, write sequence number, log index, or transaction ID corresponding to the version summaries on both sides. The system then calculates the time difference between the generation of the summaries on both sides: Δt = |TimeStampF - TimeStampR|, and read the most recent change log positions on both sides.

[0086] If Δt is less than or equal to the replication delay threshold ΔT, and the log position on the lower version side has not yet caught up with the higher version side, it is determined to be a normal replication delay, a delay observation task is generated, and it is re-verified in the next verification cycle.

[0087] If Δt is greater than ΔT, or if the log position on the lower version side has been caught up but VerDigest is still inconsistent, then the replication delay cannot explain the difference. In this case, if EndDigest, AttrDigest, and StateDigest are all consistent, and only the write cycle, write sequence number, write node, or operation type corresponding to the version digest differs, then it is classified as version inconsistency, and the higher version is used as the base version BaseVersion for this edge repair task.

[0088] If VerDigest is inconsistent, and the core attribute group, filter attribute group, or StateDigest is also inconsistent, it is upgraded to semantic inconsistency; if there are conflicts in the start point, end point, direction, edge type, graph space, or tenant identifier, it is upgraded to structural inconsistency and enters the manual confirmation or log replay repair stage.

[0089] As a specific solution, the verification results of step S6 are recorded and visualized. That is, the system records the results of each verification, including: verification time, partition pairs, verification bucket range, number of inconsistent edges, distribution of inconsistent types, distribution of risk levels, number of repair tasks, repair success rate, and reasons for failure to repair.

[0090] The system's management interface displays the consistency status of cross-partition edges in the form of a partition topology graph: normal partition pairs are displayed in green, partition pairs with low-risk differences are displayed in yellow, partition pairs with structural inconsistencies are displayed in red, partition pairs under repair are displayed in blue or flashing, and users can click on any partition pair to view the difference bucket, difference edge, difference field, and repair status.

[0091] As an embodiment of this solution, the method of this solution is applied to a specific implementation plan. Taking a device relationship graph in a distributed graph database as an example, the invention will be described in detail below: Suppose a distributed graph database has three partitions: The hash of the storage device ID for partition P1 falls at the vertex of range 1; The hash of the storage device ID in partition P2 falls at the vertex of range 2; There exists a device connection edge:

[0092] DeviceA is located in partition P1, DeviceB is located in partition P2, the edge type is CONNECTS, the edge attributes include linkType=wifi, weight=0.82, status=active, lastSeen=2026-05-13 10:00:00, and the edge version number is: WriteEpoch=1024, Seq=336, WriterId=Node-5, OpType=UPDATE.

[0093] Since P1≠P2, this edge is a cross-partition edge.

[0094] Step 1: System generated: Step 2: The forward digest is stored in P1: Save the reverse digest in P2:

[0095] Both forward and reverse digests use the same logical direction DeviceA→DeviceB. Therefore, even if a reverse reference is physically stored with DeviceB as the local vertex in P2, it will be restored to the same logical edge when the digest is generated.

[0096] Step 3: Let the number of buckets N = 1024, then This edge is assigned to a specific digest bucket under the P1→P2 partition pair.

[0097] Step 4: The check scheduler detected a large number of device connection edges between P1 and P2, and a recent node failure recovery event, therefore, it increased the check priority of this partition pair. The system first compares the partition pair-level digest of P1→P2 with the corresponding reverse digest set in P2. If a discrepancy is found between the partition pair-level digests, the system continues to compare the bucket-level digests. Assuming that the bucket-level digests of a certain bucket are inconsistent, the system further reads the edge-level digests within that bucket and finds:

[0098] The system continued to locate the field at the field level and found that the status field was active in P1 but inactive in P2, and that the version number of P1 was higher.

[0099] Step 5: Since status belongs to the core attribute group and the version number shows P1 as a higher version, the system classifies this difference as a level two risk: semantic inconsistency.

[0100] Step 6: The system generates a repair task:

[0101] The system's repair module synchronizes the higher-version edge attributes in P1 to the reverse digest and reverse edge records in P2 according to the higher-version overwrite strategy. After the repair is completed, the digest is regenerated and confirmed in the next round of verification: DigestF = DigestR. This completes one closed loop of cross-partition edge consistency repair.

[0102] In summary: The cross-partition edge consistency verification method provided by this invention in a distributed graph database does not only detect the amount of data at the partition level or the log position, but also establishes a multi-dimensional summary of endpoints, attributes, versions, and states for cross-partition edges, which can accurately locate specific edges and specific difference types, and accurately detect inconsistencies in cross-partition edges. The cross-partition edge consistency verification method provided by this invention in a distributed graph database avoids full cross-partition reading by using a hierarchical comparison mechanism of partition-level digest, bucket-level digest, edge-level digest, and field-level digest, significantly reducing network communication and storage scanning overhead and reducing periodic verification costs. The cross-partition edge consistency verification method provided by this invention incorporates graph semantics such as the start point, end point, direction, edge type, attribute, and version of cross-partition edges into the digest calculation, so that forward edge records and reverse edge records can be compared uniformly even if they have different physical storage forms. The cross-partition edge consistency verification method in distributed graph databases provided by this invention handles structural inconsistencies, semantic inconsistencies, version inconsistencies, and additional attribute inconsistencies separately, supports risk-level repair, avoids a one-size-fits-all approach to repairing all differences, and improves repair security. The cross-partition edge consistency verification method provided by this invention can promptly detect cross-partition edge anomalies caused by network failures, node downtime, asynchronous replication delays, and partition migration anomalies, reducing the probability of errors in path queries, neighbor queries, and graph traversal queries, and enhancing the reliability of distributed graph databases.

[0103] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A method for cross-partition edge consistency verification in a distributed graph database, characterized in that, Includes the following steps: S1: When making an edge write request, obtain the starting vertex identifier V. s Endpoint vertex identifier V t The starting point belongs to the partition P. s The destination partition P t Edge type R, edge direction D, edge attribute set A, edge version number Ver, and edge state State, when P s ≠P t When this happens, mark the edge as a cross-partition edge and generate a globally unique cross-partition edge identifier EdgeKey; S2: In the starting partition P s Establish a positive verification digest F At the endpoint partition P t Establish a reverse verification digest R The Digest F and Digest R Each includes the cross-partition edge identifier EdgeKey, partition identifier, endpoint digest EndDigest, attribute digest AttrDigest, version digest VerDigest, state digest StateDigest, and digest generation time TimeStamp; S3: Build summary buckets by partition, divide cross-partition edges into buckets, and maintain a bucket-level aggregated summary (BucketDigest) in each bucket. S4: Periodically schedule verification tasks, perform hierarchical comparisons of cross-partition edges according to the verification period T, and sequentially perform partition-level summary comparison, bucket-level summary comparison, edge-level summary comparison and field-level difference location to locate the inconsistency range of the edges level by level; S5: Based on the type of difference, classify the inconsistencies into structural inconsistencies, semantic inconsistencies, version inconsistencies, and additional attribute inconsistencies, and generate corresponding risk levels; S6: Generate a repair task for inconsistent edges. The repair task includes EdgeKey, starting partition, ending partition, difference type, risk level, base version and repair strategy, and perform consistency repair.

2. The method for cross-partition edge consistency verification in a distributed graph database according to claim 1, characterized in that, The method for generating the EndDigest is as follows: For forward edge records, generate the forward endpoint canonical string: And calculate: For reverse records, generate the reverse endpoint canonical string: And calculate: like If the endpoints, directions, edge types, graph spaces, or tenant identifiers are inconsistent, it is determined that there are inconsistencies.

3. The method for cross-partition edge consistency verification in a distributed graph database according to claim 1, characterized in that, The method for generating AttrDigest is as follows: The edge attributes are divided into core attribute group A. c Filter attribute group A f and additional attribute group A e After generating summaries for each attribute group, the following calculations are performed: The core attribute group consists of attributes that affect path queries, permission judgments, and relational semantics; the filtering attribute group consists of attributes used for filtering queries based on timestamps, tags, regions, and device types; and the supplementary attribute group consists of extended attributes used for display, notes, or low-frequency queries.

4. The method for cross-partition edge consistency verification in a distributed graph database according to claim 1, characterized in that, The method for generating the VerDigest is as follows: Maintain logical version number: Where WriteEpoch is the write cycle or global logical clock, Seq is the write sequence number within the partition, WriterId is the write node identifier, and OpType is the operation type. And calculate: 。 5. The method for cross-partition edge consistency verification in a distributed graph database according to claim 1, characterized in that, The method for comparing the hierarchical summaries of the partitions is as follows: Calculate the partition-level summary: Wherein, PairKey represents a directed partition pair (P i →P j TotalCount represents the number of cross-partition edges for this partition; XorBucketDigest represents the XOR aggregation of all bucket-level digests for this partition; MaxPairVer and MinPairVer represent the maximum and minimum versions within this partition, respectively; StateStatAll represents the number of states such as valid, deleted, and migrated; DigestEpoch represents the digest generation batch. The starting partition stores the digest of the forward partition pairs. The reverse partition pairs are recovered from the endpoint partition based on the reverse digest set. The same PairKey, the same bucket partitioning rules, and the same normalization order are used for calculation; if the two are equal, the partition pair is considered to be consistent; otherwise, bucket-level comparison is performed.

6. The method for cross-partition edge consistency verification in a distributed graph database according to claim 5, characterized in that, The method for bucket-level digest comparison is as follows: The BucketDigest of the starting and ending partitions is compared bucket by bucket. If the digests of a bucket match, the bucket is skipped; if the digests of a bucket do not match, the bucket is added to the set of differing buckets. DiffBuckets={B_k|BucketDigestF(B_k)≠BucketDigestR(B_k)}; The method for edge-level summary comparison is as follows: edge-level summaries in the difference bucket are compared by EdgeKey, and edge-level difference comparison is performed in cases where the forward summary exists but the reverse summary does not exist, the reverse summary exists but the forward summary does not exist, the endpoint summaries are inconsistent, the attribute summaries are inconsistent, the version summaries are inconsistent, or the state summaries are inconsistent.

7. The method for cross-partition edge consistency verification in a distributed graph database according to claim 6, characterized in that, For edges with inconsistent endpoint summaries, inconsistent attribute summaries, and inconsistent version summaries, field-level difference location is performed separately. Methods include: For edges with inconsistent attribute summaries, compare the differences in the core attribute group, the filter attribute group, and the additional attribute group. For edges with inconsistent endpoint summaries, compare whether the start point, end point, edge direction, edge type, graph space, or tenant are consistent. For edges with inconsistent version summaries, compare whether the write cycle, write sequence number, write node, and operation type are consistent.

8. The method for cross-partition edge consistency verification in a distributed graph database according to claim 1, characterized in that, The risk levels include: Level 1 risks of structural inconsistency include missing forward edges, missing reverse edges, inconsistent start or end points, inconsistent edge directions, or inconsistent edge types. Secondary risks of semantic inconsistency include inconsistencies in core attribute groups, filter attribute groups, edge states, or deletion states. Level 3 risks of version inconsistency include inconsistent version numbers but consistent attributes, inconsistent write nodes but consistent logical content, or version differences within the allowed replication delay window. Level 4 risk of inconsistent additional attributes includes inconsistent display attributes, inconsistent remarks attributes, or inconsistent low-frequency extended attributes.

9. The method for cross-partition edge consistency verification in a distributed graph database according to claim 8, characterized in that, The method for executing the repair task is as follows: The repair work will be carried out step by step in the order of Level 1 risk, Level 2 risk, Level 3 risk, and Level 4 risk. For the same edge, first add a short-term repair lock or an idempotent repair flag to avoid multiple repair tasks concurrently covering the same edge, and repair tasks between different EdgeKeys are executed in parallel; After the repair is completed, the bidirectional verification digest of the edge is regenerated and the edge-level verification is triggered: if the edge-level verification passes, the corresponding BucketDigest and PairDigest are updated; if the edge-level verification fails, the next stage repair task is generated again according to the new difference type; if the number of consecutive retries exceeds the threshold or there are still conflicts between the endpoint, tenant, and edge type, the repair is automatically upgraded to manual confirmation and an alarm is issued.

10. The method for cross-partition edge consistency verification in a distributed graph database according to claim 8, characterized in that, The repair strategies include: log replay repair, high version overwrite repair, main side record repair, deletion consistency repair, manual confirmation repair, and delayed observation repair. When a summary is missing on one side but the change log exists, log replay can be used to repair it. When there are differences in core attribute groups, filter attribute groups, or states, and there is a clearly higher version, the higher version will be used to overwrite and fix the issue. When a summary is missing on one side and the log is unavailable, but the primary side record is reliable, the primary side record is used for repair. When one side is in a deleted state and the other side is in a valid state, first verify the deletion log and tombstone record, and then use deletion consistency repair. When endpoints, directions, edge types, or tenant identifiers conflict and the actual record cannot be determined from the logs, manual confirmation and repair are initiated. When VerDiges are different and within the replication delay window, delayed observation repair is used.

Images