Dual-active arbitration cross-domain data synchronization method, system and device and medium

Abstract

The invention relates to an active-active arbitration cross-domain data synchronization method, system and device and a medium. The method comprises the following steps: acquiring a write-in request, and generating an operation log record of a source site according to the write-in request; fragmenting the operation log record according to the entity identifier through a duplicator, and pushing the fragmented operation log record to a remote site to obtain an operation log record copy of the remote site; performing conflict candidate detection on the compression causal abstract in the operation log record copy and the local concurrent pending operation log record corresponding to the timestamp to obtain a conflict candidate set; determining an arbitration strategy of each conflict pair according to the operation semantic tag in the conflict candidate set, and performing merging operation on each conflict pair according to the arbitration strategy to obtain an arbitration certificate; and writing the final state of the write request indicated by the arbitration certificate into local materialized views of the source site and the remote site, and carrying out persistence processing to obtain a data visible consistent result. By adopting the method, self-arbitration synchronization under the dynamic conflict condition can be realized.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and in particular relates to a cross-domain data synchronization method, system, device and medium with dual-active arbitration. Background Technology

[0002] With the development of distributed storage and WAN multi-active system technologies, dual-active data synchronization and arbitration technologies have emerged. Each site can perform writes nearby to reduce local latency, support multi-site parallel services to improve availability, and rely on conflict detection and decision-making mechanisms to avoid split-brain and restore global consistency.

[0003] In traditional technologies, one type aims for strong consistency or distributed transactions, using distributed consensus (Paxos / Raft) or two-phase commit (2PC) to ensure the atomicity and order of cross-domain writes; the other type prioritizes availability and low latency, using an eventual consistency method to commit locally and merge in the background.

[0004] However, the aforementioned methods, based on strong consistency distributed transactions or global consensus, suffer from high latency, limited throughput, and are prone to prolonged blocking in wide area networks, severely impacting write latency and availability in active-active scenarios. Furthermore, the vector clock / version vector used for causal tracing exhibits O(N) metadata bloat as the number of nodes increases, leading to a significant increase in storage, transmission, and computational overhead. Summary of the Invention

[0005] Therefore, it is necessary to provide a cross-domain data synchronization method, system, device, and medium with active-active arbitration that can maintain low-latency local writes while improving cross-domain consistency decisions, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a cross-domain data synchronization method for active-active arbitration, including:

[0007] Obtain write requests and generate operation log records for the source site based on the write requests; the operation log records include entity identifier, operation identifier, source site, operation type, payload data, timestamp, compressed causal summary, operation hash value, and operation semantic tag;

[0008] The operation log records are fragmented by entity identifier and pushed to the remote site through the replicator to obtain a copy of the operation log records at the remote site;

[0009] In response to the discovery that there are local concurrent pending operation log records in the local pre-write logs of the remote site that correspond to the entity identifier of the operation log record copy, the compressed causal summary in the operation log record copy is used to perform conflict candidate detection with the local concurrent pending operation log records with the corresponding timestamps to obtain a conflict candidate set; the conflict candidate set includes each conflict pair and its operation semantic label.

[0010] The arbitration strategy for each conflict pair is determined based on the operational semantic labels in the conflict candidate set, and the conflict pairs are merged according to the arbitration strategy to obtain the arbitration certificate.

[0011] The final state of the write request indicated by the arbitration certificate is written to the local materialized views of the source and remote sites and persisted to obtain a consistent data visibility result.

[0012] In one embodiment, before obtaining the write request and generating the operation log record of the source site based on the write request, the method further includes:

[0013] The entities to be managed are classified according to business consistency requirements and semantic characteristics to obtain a consistency grouping mapping table; the consistency grouping mapping table includes entity identifiers and their corresponding operation semantic tags: operation semantic tags include strong consistency class, mergeable class and eventual consistency class.

[0014] In one embodiment, generating an operation log record for the source site based on the write request includes:

[0015] Generate an operation identifier for the write request;

[0016] A compressed causal summary is generated based on the local set of most recently interacted sites from the source site. The compressed causal summary includes short vector summaries of hot sites and compressed summaries of long-tail sites, along with their corresponding timestamps. The local set of most recently interacted nodes is obtained from a list of site activity within a preset time window. The list of site activity includes hot sites and long-tail sites.

[0017] Calculate the operation hash value for the operation log record, and write the operation hash value, operation identifier, and compressed causal summary into the operation log record;

[0018] The operation log is recorded and written to the local pre-write log of the source site, and a local commit confirmation is received.

[0019] In one embodiment, determining the arbitration strategy for each conflict pair based on the operational semantic tags in the conflict candidate set includes:

[0020] When the operation semantic label is a mergeable class, the arbitration strategy is the fast merge mode. The fast merge mode corresponds to the arbitrator performing server-side merging of the operation type and payload data of the conflicting pair based on the mergeable algorithm to produce a merge result and obtain an arbitration certificate.

[0021] When the operation semantic label is eventually consistent, the arbitration strategy is the deterministic semantic arbitration mode: the deterministic semantic arbitration mode corresponds to the arbitrator executing a deterministic semantic merging function on the operation type and payload data of the conflicting pair under the vote of the source site, remote site and witness node to produce a deterministic output and obtain the arbitration certificate;

[0022] When the semantic label of the operation is a strong consistency class, the arbitration strategy is a short-term local consensus mode. The short-term local consensus mode corresponds to a local-scale consensus group composed of the source site, the remote site and the witness node. Based on the arbitration vote of the local-scale consensus group, the final data state is determined according to the operation type and payload data of the conflicting pair, and an arbitration certificate is obtained.

[0023] In one embodiment, generating the source site's operation log record based on the write request further includes:

[0024] When the operation semantic label is a mergeable class, the corresponding operation log records in the local pre-write log are merged based on the idempotent convergent algorithm to obtain the updated operation log records.

[0025] When the operation semantic label is eventually consistent or strongly consistent, the corresponding operation log record in the local pre-write log is marked as temporary; the temporary status is used to indicate that the source site locks the entity identifier to prevent local concurrent write requests.

[0026] In one embodiment, the method further includes:

[0027] The anti-entropy synchronization algorithm based on entity-level Merkle trees is used for difference detection. The Merkle trees of each site are compared. When a difference is detected, the corresponding operation log records are replayed in the order of the certificates according to the decision order recorded in the arbitration certificate, and the Merkle trees of each site are updated.

[0028] In one embodiment, the method further includes:

[0029] When the operation log is in a temporary state, the arbitrator and the witness node will vote on whether the operation is ready to be committed, and obtain the voting result.

[0030] When the voting result is a majority of responses in agreement, the arbitrator generates an arbitration certificate containing the participating sites' decisions on whether to submit and be ready. The arbitration certificate is used to instruct each site to write the temporary state recorded in the operation log to the final submission based on the arbitration certificate.

[0031] When the voting result indicates that the submission conditions have not been met, the arbitrator generates an arbitration certificate containing compensation instructions; the compensation instructions are used to instruct each site to perform compensation actions on the operation log records.

[0032] Secondly, this application also provides a cross-domain data synchronization system for dual-active arbitration, including:

[0033] The data modification request module is used to obtain write requests and generate operation log records for the source site based on the write requests. The operation log records include entity identifier, operation identifier, source site, operation type, payload data, timestamp, compressed causal summary, operation hash value, and operation semantic tag.

[0034] The cross-domain push module is used to fragment operation log records by entity identifier and push them to remote sites through a replicator, thereby obtaining a copy of the operation log records on the remote site.

[0035] The conflict request module is used to respond to the occurrence of local concurrent unresolved issues corresponding to the entity identifier of the operation log record copy in the operation log record of the remote site. It performs conflict candidate detection on the compressed causal summary in the operation log record copy and the local concurrent unresolved issues with the corresponding timestamp to obtain a conflict candidate set. The conflict candidate set includes each conflict pair and its operation semantic tag.

[0036] The arbitration module is used to determine the arbitration strategy for each conflict pair based on the operation semantic tags in the conflict candidate set, and to perform a merging operation on each conflict pair according to the arbitration strategy to obtain an arbitration certificate.

[0037] The data synchronization module is used to write the arbitration certificate to the local materialized views of the source and remote sites and perform persistent processing to obtain a consistent data visibility result.

[0038] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above-described dual-active arbitration cross-domain data synchronization methods.

[0039] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above-described dual-active arbitration cross-domain data synchronization methods.

[0040] The aforementioned cross-domain data synchronization method, system, device, and medium for active-active arbitration achieves parallel writing and eventual consistency in an active-active environment by synchronously writing operation logs between the source and remote sites and making dynamic decisions during conflict detection and arbitration phases. This ensures that multiple sites have simultaneous writing capabilities in an active-active deployment, while the arbitration certificate mechanism prevents data inconsistencies caused by write conflicts. By using compressed causal summaries and operation semantic tags included in the operation logs, a traceable conflict detection mechanism based on logical time and operation type can be established, ensuring that conflict judgments are based on genuine causal dependencies and improving the accuracy and interpretability of arbitration decisions. By generating an arbitration certificate for each arbitration, the conflict pair, arbitration strategy, and final decision result are stored in a structured record. By writing the decision result indicated by the arbitration certificate to the local materialized view of each site and performing persistence, it is ensured that all nodes in the active-active environment have the same physically visible results, achieving a globally consistent final state. Since the arbitration mechanism is based on operation logs and causal summaries independently and does not rely on centralized locks or synchronization clocks, even under cross-domain communication delays or short-term network partitions, state convergence can still be achieved through subsequent conflict detection and arbitration certificate repair mechanisms, which significantly improves the robustness and availability of cross-domain data synchronization. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the cross-domain data synchronization method of dual-active arbitration according to the present invention.

[0043] Figure 2 This is a step-by-step flowchart of step S101;

[0044] Figure 3 This is a flowchart illustrating the steps of step S104.

[0045] Figure 4 This is a structural diagram of the cross-domain data synchronization system with dual-active arbitration according to the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0047] In one embodiment, such as Figure 1 As shown, a cross-domain data synchronization method with active-active arbitration is provided. This embodiment illustrates the method applied to a terminal, but it is understood that the method can also be applied to a server, and to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0048] S101. Obtain the write request and generate the operation log record of the source site according to the write request; the operation log record includes entity identifier, operation identifier, source site, operation type, payload data, timestamp, compressed causal summary, operation hash value and operation semantic tag.

[0049] This example illustrates receiving write requests from the origin server client. A write request refers to a modification request initiated by the business or application side to a specific business entity, such as modifying an account balance, updating a document, or adding a record. The write request is converted into an operation log record (OpRecord) to structure a logical operation into a persistent, transmissible, and verifiable unit for subsequent cross-domain replication, conflict detection, and arbitration processing. Specifically, a complete operation log is generated for this write request. The entity identifier (entity_id) is used to uniquely identify the business object being operated on; the operation identifier (op_id) is used to uniquely identify the operation to prevent duplication or replay; the origin site (origin_site) identifies the site where the operation initially occurred; the operation type (op_type) includes creation, addition, reduction, update, deletion, etc.; the payload data is the actual data content to be written or changed; the timestamp (hlc_timestamp, HLC) is used for approximate order determination, and a hybrid logical clock mechanism can be used to record the specific write request time; the compressed causal summary (causal_summary) is used for lightweight causal relationship screening; the operation hash (op_hash) is used to verify the operation content summary to prevent tampering; and the operation semantic tag (op_semantic_tag) indicates the consistency category to which the operation belongs, such as strong consistency, mergeable, or eventual consistency. Furthermore, after generating the operation log record, it is written to the source site's local write-alive log (WAL) for persistent storage, and a local commit confirmation is returned. This persistent operation log record serves as input for subsequent cross-domain transfers and local concurrency checks. For example, when two clients initiate modifications to the same account on different sites almost simultaneously, each site will generate and persist its own operation log record, providing necessary timing and causal information for subsequent conflict detection.

[0050] S102. The operation log records are fragmented by entity identifier and pushed to the remote site through the replicator to obtain a copy of the operation log records at the remote site.

[0051] For example, the operation log records corresponding to the write request from the source site are transmitted to other sites to achieve multi-site data flow and visibility. Specifically, the logs are grouped or hashed based on the entity identifiers within the operation log records to route operations of the same entity to the same target partition or processing queue. This ensures a balance between order and parallelism during cross-site replay, narrowing the granularity of cross-domain transmission and replay to the business entity level, thereby reducing the scope of conflict detection and improving parallel throughput. For example, the source site entrusts the persistent operation log records to a replicator for reliable transmission. The replicator is responsible for packaging, fragmenting, batching, and pushing the operation log records to one or more remote sites using an acknowledgment and retry strategy based on the entity identifier. After receiving the operation log records, the remote sites write them to their local pre-write logs, forming a replica of the operation log records at the remote sites. The remote sites form a peer-to-peer deployment with the source site in a dual-active or multi-active manner, and also maintain local data structures such as WAL and materialized views. The generation of operation log record replicas ensures that each site can see the same operation event sequence fragments. For example, if a document's entity_id is mapped to shard A, all operations on that document will be routed by the replicator to the processing queue of shard A and delivered to the corresponding remote site, thus facilitating subsequent conflict resolution and arbitration at the document level.

[0052] S103. In response to obtaining the existence of local concurrent unresolved operation log records in the local pre-write log of the remote site that correspond to the entity identifier of the operation log record copy, perform conflict candidate detection on the compressed causal summary in the operation log record copy and the local concurrent unresolved operation log records with the corresponding timestamp to obtain a conflict candidate set; the conflict candidate set includes each conflict pair and its operation semantic label.

[0053] Indicatively, when a remote site receives a copy of an operation log record and finds a locally concurrent pending operation log record in its local pre-write log corresponding to the entity identifier of that copy, it needs to determine whether a concurrent conflict has occurred and which operations need to be arbitrated. Here, locally concurrent pending operation log records refer to other operations on the same entity that already exist in the local pre-write log but have not yet been determined as final commits or arbitrated. Specifically, to identify a potential conflict candidate set with low overhead in a wide area network environment, compressed causal summaries and timestamps carried in the operation log records are used as the first screening method. The compressed causal summary includes short vector summaries for hot sites and compressed summaries for long-tail sites. When the causal summaries of the remote copy and the locally pending operations intersect, or when the timestamps indicate possible concurrency, the operation identifier pair is identified as a conflict candidate. Further, the conflict candidates are compared more precisely based on the order of the timestamps or partial vector information of the operation log records to obtain the final conflict candidate set. Optionally, this conflict candidate set is listed in the form of conflict pairs, with each pair accompanied by its own operation semantic label, serving as input for the next step of selecting the arbitration strategy. Hierarchical screening can reduce bandwidth and computational overhead while maintaining feasibility under large-scale node counts. For example, when both the compressed causal summary of the remote replica and the summary of the local pending operation identify a recent write to the same hot site, they are identified as conflict candidates and added to the set to be arbitrated.

[0054] S104. Determine the arbitration strategy for each conflict pair based on the operation semantic labels in the conflict candidate set, and perform a merging operation on each conflict pair according to the arbitration strategy to obtain the arbitration certificate.

[0055] Furthermore, after obtaining the conflict candidate set, an appropriate arbitration strategy is selected for each conflict pair, and the corresponding merging operation is performed to generate a final decision that can be applied at each site. The selection of the arbitration strategy is based on the classification of the entity by the operational semantic tags contained in the conflict pair, ensuring the correctness of business semantics while achieving a provable final state at the lowest cost. Specifically, a deterministic processing path is selected according to the strategy to process the conflict pair and generate an illustrative and verifiable arbitration certificate. The arbitration certificate, as the carrier of the arbitration result, records the set of operational identifiers involved in the conflict, the arbitration decision, such as the merging result, the selected order or rollback instruction, as well as the timestamp of the arbitration decision and summary information for post-verification, etc., and is generated or signed by the arbitrator in conjunction with witness nodes when necessary to improve auditability. Specifically, the arbitration process reads the conflict candidate set and its operational semantic tags to determine the type of strategy to be used, performs merging or sorting operations on the conflict pair under the strategy, and produces a final operational result description. This result is encapsulated in the arbitration certificate, which is then issued to the relevant sites as a trusted decision credential after generation.

[0056] S105. Write the final state of the write request indicated by the arbitration certificate to the local materialized views of the source and remote sites and persist it to obtain a consistent data visibility result.

[0057] In a schematic manner, each site writes the final state indicated by the arbitration certificate into a local materialized view, thereby enabling the business side to observe a consistent result. The local materialized view refers to a copy or cache layer of entity state maintained locally at each site for fast read / query purposes; it is a materialization of the replay or merged results of operation log records. Specifically, it verifies the integrity and signature of the arbitration certificate, applies the final decision order and merge result recorded in the arbitration certificate to the corresponding operation log records in the local pre-write log, such as overwriting, merging, or revoking tentative states, and persists the final state to local storage to ensure visibility after fault recovery.

[0058] Optionally, the persistence process should also be written to the audit log to maintain an immutable link between the arbitration certificate and application actions, facilitating retrospective review and remediation. For example, if the arbitration certificate specifies that the merged result of two concurrent updates for an account is X, then each site will update the value of that account in its local materialized view to X based on the certificate, and persist and record this change in the audit log, thereby ensuring that the client receives a consistent value when reading it.

[0059] In the aforementioned dual-active arbitration cross-domain data synchronization method, by integrating semantic and causal information into the operation log, multi-dimensional operation uniqueness identification and conflict determinability are achieved. This enables correct identification and differentiation of causal relationships in remote concurrent scenarios, reducing the probability of false conflict judgments. Data sharding based on entity identifiers and pushing to remote sites achieves parallelization and granular control of cross-domain data synchronization, reducing the resource consumption of full replication and improving data transmission efficiency and the target site's receiving throughput. A conflict candidate detection mechanism based on compressed causal summaries and timestamps can detect potential conflicts through compact causal summaries without requiring the full operation history, thus balancing detection accuracy and low computational cost, achieving real-time conflict identification. Arbitration strategies are determined and merged based on operation semantic tags. By distinguishing the semantics of different types of operations, targeted arbitration decisions are achieved, flexibly adapting to different business consistency requirements, thus balancing performance and consistency goals. Arbitration certificates record the arbitration results in a structured manner, giving each arbitration an independent credential. This allows for verification of decision correctness during data recovery, anti-entropy synchronization, and auditing, improving fault tolerance and traceability. The updating and persistence of local materialized views ensures visibility consistency in a multi-site environment by applying arbitration decisions at the materialized view layer, reducing read / write conflicts and improving eventual consistency of data synchronization. The end-to-end data visibility consistency output under the dual-active architecture achieves closed-loop control of data from writing, replication, conflict detection, arbitration to persistence, ensuring that the final state of cross-domain data is predictable, verifiable, and consistent across the entire network.

[0060] In one embodiment, before obtaining the write request and generating the operation log record of the source site based on the write request, the method further includes:

[0061] S11. The entities to be managed are classified according to business consistency requirements and semantic features to obtain a consistency grouping mapping table; the consistency grouping mapping table includes entity identifiers and their corresponding operation semantic labels: operation semantic labels include strong consistency class, mergeable class and eventual consistency class.

[0062] Different business objects exhibit significant differences in consistency requirements, conflict merging capabilities, transaction coupling, and compliance. For example, some objects, such as accounting objects, must guarantee strong ordering and atomicity; others, such as counters or sets, can be automatically converged semantically using merging rules; and still others, such as statistical or cached data, allow eventual consistency in exchange for higher availability. A consistency grouping mapping table associates each entity to be managed with a suitable arbitration / synchronization semantic for that entity, enabling rapid determination of the processing path and arbitration strategy for subsequent operations. This approach both sinks global strategies down to the entity level, avoiding the performance loss caused by static coarse-grained arbitration, and quickly selects the appropriate processing path when conflicts occur, thereby reducing cross-domain latency and computational overhead.

[0063] Specifically, constructing a consistent grouping mapping table involves four closely interconnected stages: data collection, feature extraction, hierarchical rule application, and mapping table generation and maintenance. For example, in the data collection stage, static and dynamic information of candidate entities is gathered from existing business descriptions, data models, database schemas, access logs, and historical operation logs. This includes, but is not limited to, the entity's business semantics, such as accounts, orders, and configurations, as well as cross-entity transaction correlations, such as whether it frequently participates in atomic transactions with other entities, read / write ratios and update frequencies, operational load sizes, historical conflict rates and conflict types, and whether there are compliance / audit requirements. In the feature extraction stage, a set of feature vectors is constructed based on the collected data for judgment, such as high / low transaction coupling, high / low semantic merging capability, high / low conflict frequency, and the need for irreversible auditing, to express the entity's key consistency attributes. In the hierarchical rule application stage, entities are mapped to several predefined categories according to a predefined set of rules, including strongly consistent, mergeable, and eventually consistent categories. The rules can be combined according to priority. For example, entities with legal compliance or financial atomicity requirements are prioritized as strongly consistent. If the business semantics of an entity naturally support mergeable operations, such as idempotent counters, set operations, or idempotent configuration updates, it is marked as mergeable. Other non-critical entities that allow delayed convergence are marked as eventually consistent. Optionally, the mapping table can be periodically analyzed and re-evaluated based on operational metrics during runtime. If necessary, a change approval process can be initiated to write the adjustments to the mapping table and distribute them to the entire network. Optionally, the consistency group mapping table uses the entity identifier as the index. The record fields include at least the entity identifier, the consistency group identifier, the operation semantic tag (i.e., strongly consistent / mergeable / eventically consistent), a summary of the mapping activation strategy or reason, the mapping version, and the last update timestamp. This consistency group mapping table can be efficiently queried by the write paths of each site and components such as arbitrators and replicators, so that the op_semantic_tag can be automatically included when generating OpRecord.

[0064] In one embodiment, such as Figure 2 As shown, the operation log records of the source site are generated based on the write request, including:

[0065] S201. Generate an operation identifier for the write request.

[0066] A unique and traceable operation identifier (op_id) is generated for each write request, ensuring that each logical operation can be globally and uniquely identified in the distributed environment, thereby supporting idempotency detection, duplicate request suppression, and post-event tracing. For example, an identifier based on a UUID or a composite identifier composed of a site identifier and a local monotonic sequence is generated and written to the operation log record to be created, serving as a key index item for subsequent replication, conflict comparison, and arbitration.

[0067] S202. Generate a compressed causal summary based on the local set of most recently interacted sites from the source site; the compressed causal summary includes short vector summaries of hot sites and compressed summaries of long-tail sites and their corresponding timestamps; the local set of most recently interacted nodes is obtained from the list of site activity within a preset time window; the list of site activity includes hot sites and long-tail sites.

[0068] Furthermore, a time base is assigned to this operation record, i.e., a timestamp is generated. Optionally, a hybrid logical clock (HLC) time series is used as the timestamp to balance physical time order and event causality. The timestamp setting can provide an efficient approximate basis for subsequent concurrency determination and preliminary sorting, especially in cross-domain environments, reducing the need for strict global synchronization. Simultaneously with timestamp generation, the set of most recently interacted sites within a preset time window is collected from the source site, and a compressed causal summary is calculated based on this set to avoid O(N) metadata bloat as the node scale expands. The local set of most recently interacted sites is obtained by statistically analyzing the site activity within a certain time window. The activity list is used to determine which sites should be considered hot sites and enter the vector summary; the remaining sites enter the compressed representation, thus achieving a hot-cold tiered causal summary construction. Specifically, short vectors or small-range counts are maintained for hot sites (sites with interaction records within the preset time window), while Bloom filters or sparse signatures are used for compressed summary representation of long-tail sites (sites without interaction records within the preset time window), thereby significantly reducing transmission and storage overhead while retaining sufficient causal determination capabilities.

[0069] S203. Calculate the operation hash value for the operation log record, and write the operation hash value, operation identifier, and compressed causal summary into the operation log record.

[0070] Furthermore, after obtaining op_id, timestamp, and compressed causal digest, the operation hash value (op_hash) is calculated for key fields such as payload data, operation type, and entity identifier in the operation log record. This hash value can be digested using a cryptographic hash algorithm to summarize the deterministic serialized representation of the operation, so as to be used for integrity verification, anti-tampering verification, and rapid comparison of operation content during arbitration and auditing.

[0071] S204. Write the operation log to the local pre-write log of the source site and receive the local submission confirmation.

[0072] As an illustration, after filling in the above fields, the operation log record, which constitutes a complete field set, is atomically written to the local pre-write log of the source site. The writing process must ensure persistence and sequential consistency; that is, the write either succeeds and is immediately persisted to the local storage medium, or it fails and is clearly reported for retry. The purpose of persisting the operation log record to WAL is to ensure that the operation is not lost and can be replayed in the event of a failure or partition. At the same time, the sequential semantics of WAL also provide a reliable basis for subsequent local concurrency judgment and arbitration. After the write is completed, a local commit confirmation is returned to the caller. Optionally, for mergeable operations, since the local application is already on the mergeable path and can converge quickly, an immediate final confirmation can be returned. For eventually consistent or strongly consistent operations, a local commit confirmation or a temporary confirmation can be returned to indicate that the operation has been persisted, but whether it becomes a final visible state depends on the subsequent arbitration result.

[0073] In one embodiment, such as Figure 3 As shown, the arbitration strategy for each conflict pair is determined based on the operational semantic labels in the conflict candidate set, including:

[0074] S301. When the operation semantic label is a mergeable class, the arbitration strategy is the fast merge mode. The fast merge mode corresponds to the arbitrator performing server-side merging of the operation type and payload data of the conflicting pair based on the mergeable algorithm to generate a merge result and obtain an arbitration certificate.

[0075] Indicatively, the fast merge mode is an arbitration strategy for handling conflicting pairs whose operational semantic labels are mergeable (Class-C). It generates a converged result directly on the server by using an idempotent, convergent merge algorithm, thereby achieving eventual consensus with minimal latency in most conflict scenarios.

[0076] Mergeable data inherently supports merging semantically. Delegating the merging process to the server and directly generating the final state minimizes cross-domain arbitration overhead and transaction wait time, while ensuring idempotency and convergence, making it suitable for high-frequency writes and weak-conflict scenarios. Specifically, when the arbitrator receives a set of conflict candidates containing mergeable conflict pairs, it reads the operation record units involved in the conflict, including their payload data, operation type, operation hash value, and operation identifier. Based on the `op_semantic_tag` of the conflict pair, it determines the appropriate mergeable algorithm category, such as G-Counter, PN-Counter, OR-Set, sequence CRDT, or equivalent OT algorithms implemented on the server. Furthermore, the arbitrator performs deterministic merging operations on the server side. The merging process guarantees idempotency (i.e., the result remains unchanged when the same operation is applied multiple times) and convergence (i.e., different sites obtain the same result by using the same merging logic). After the merge is complete, the arbitrator generates an arbitration certificate. The certificate contains at least the set of participating operation identifiers, the operation hash value of each participating operation, the final merged payload data or its digest, the arbitration decision timestamp (decision_ts), and a digest for verification, i.e., the verification hash (verifier_hash). The verifier_hash is calculated by the arbitrator in a deterministic manner, such as by combining the decision identifiers and the participating op_hash list in a fixed order with the decision_ts to calculate the hash, ensuring the verifiability and immutability of the certificate. For example, for two concurrent counter increment operations, the arbitrator can sum the two increments on the server to obtain the final count and generate an arbitration certificate containing the two op_ids and the merged count, thus enabling sites to quickly reach consensus without initiating expensive voting.

[0077] S302. When the operation semantic label is eventually consistent, the arbitration strategy is the deterministic semantic arbitration mode: the deterministic semantic arbitration mode corresponds to the arbitrator executing a deterministic semantic merging function on the operation type and payload data of the conflicting pair under the vote of the source site, remote site and witness node to produce a deterministic output and obtain the arbitration certificate.

[0078] The illustrative deterministic semantic arbitration mode is used to handle complex objects marked as eventually consistent (Class-E) or requiring semantic-level merging. It enhances the verifiability and auditability of decisions by using a deterministic semantic merging function provided by the application and executed by the arbitrator, and by introducing voting or verification from witness nodes or relevant sites before / after the arbitration decision is generated. Complex business objects often cannot be automatically converged using general CRDTs; their merging relies on business semantics, such as priority field selection, field-level merging rules, and three-way merging strategies in case of conflicts. By executing the application-defined deterministic merging function on the arbitrator side, combined with the participation of witness nodes, a verifiable arbitration certificate can be formed while ensuring semantic correctness, meeting the requirements of business interpretability and compliance auditing. Specifically, when a conflict pair in the conflict candidate set is identified as eventually consistent and triggers this mode, the arbitrator reads the entity's merging strategy metadata from the consistency grouping map table to determine the semantic merging function to be executed. This function can be registered by the application and run on the arbitrator in a secure and controlled manner, or executed by the arbitrator calling an external secure execution environment. The arbitrator aggregates all payload data, operation identifiers, and operation hashes of the conflicting pairs, and requests supplementary audit information or signatures from participating source or remote sites when necessary. Further, the arbitrator executes the semantic merge function to produce a deterministic output under a voting or verification process involving witness nodes. The witness nodes can verify the arbitrator's inputs, such as the participating op_hash list, and return signatures or votes to demonstrate the transparency of the arbitration process. After merging, the arbitrator generates an arbitration certificate, which records the participating op_id list, each op_hash, the merged deterministic output, the decision timestamp, the witness node's voting results and / or signatures, and the verifier_hash. The certificate is then signed and persisted. This certificate is subsequently distributed to the relevant sites, which apply the merged output to their local materialized views in S105 based on the certificate.

[0079] In this context, a witness node is a neutral node deployed in a dual-active architecture, responsible for read-only access and arbitration assistance. It provides operation verification, arbitration certificate witnessing, and conflict decision confirmation between the source and remote sites. Specifically, the witness node does not directly participate in business write operations; it only listens to operation log summaries from the source and remote sites and generates or verifies arbitration certificates when necessary. When there are concurrent writes or overlapping operations between two sites, the witness node compares the operation hashes and semantic tags submitted by both parties to verify the legitimacy of the conflict candidate set and issues a verification hash for the arbitration result. This hash signature is written into the arbitration certificate, ensuring that the decision is verifiable and non-repudiable. For the arbitration merging results of multiple conflict pairs, the witness node can participate as a consensus confirmation point in the final write state decision confirmation. Its output signature or timestamp (decision_ts) can serve as the trigger for the source and remote sites to write the final materialized view, ensuring the consistency of the results visible across the entire network.

[0080] For example, for concurrently modified user configuration objects, the semantic merge function provided by the application may specify that field A takes the maximum version, field B is merged into a union, and field C is rolled back according to business priority. The arbitrator executes the function under the authentication of the witness node and issues the result in the form of an arbitration certificate, thereby ensuring the business correctness and verifiability of the merge rule.

[0081] S303. When the operation semantic label is a strong consistency class, the arbitration strategy is a short-term local consensus mode. The short-term local consensus mode corresponds to a local-scale consensus group composed of the source site, the remote site and the witness node. Based on the arbitration vote of the local-scale consensus group, the final data state is determined according to the operation type and payload data of the conflicting pair, and an arbitration certificate is obtained.

[0082] Indicatively, the short-term local consensus model is used to handle conflict pairs whose operation semantic labels are of the strongly consistent class (Class-S), or for scenarios requiring atomic cross-entity commits. It involves temporarily forming a small consensus group among the source site, remote site, and several witness nodes involved in the conflict, and using short-term consensus or arbitration voting to determine the final commit or rollback. Strongly consistent operations typically have atomicity or legal / financial consistency requirements, which simple CRDT merging cannot meet. Global 2PC is affected by latency and availability in wide area network scenarios. Using a locally granularized short-term consensus group allows for rapid security decisions and the generation of auditable arbitration certificates within a smaller membership scope. Specifically, when the conflict candidate set is identified as strongly consistent, the arbitrator initiates the short-term local consensus process. The arbitrator determines the list of members participating in the consensus based on the conflict pair. This list typically includes the source site, remote site, and several pre-deployed witness nodes. Subsequently, the arbitrator sends a consensus proposal to each member according to a predefined negotiation protocol. The proposal includes the participating op_id to be confirmed, each op_hash, and the proposed decision, such as prioritizing or rolling back a certain operation. Upon receiving a proposal, each member determines its readiness status based on its local WAL and business rules, and returns a vote (agree or reject), with an optional local signature verification. The arbitrator collects member votes and decides whether to submit based on majority rules or configured thresholds. If the submission conditions are met, the arbitrator issues a submission resolution and generates an arbitration certificate, which includes the participating op_id, vote summary, final decision, decision_ts, and verifier_hash. If the submission conditions are not met, the arbitrator issues an arbitration certificate with rollback / compensation instructions to trigger the compensation process. This arbitration certificate is persisted by the arbitrator and, if necessary, signed by witness nodes to enhance trust. Subsequently, the relevant sites use the certificate to write the temporary state to a final submission or execute compensation actions.

[0083] Optionally, to mitigate the risk of prolonged blocking, local consensus typically employs timeout and rollback strategies. If a majority consensus is not reached within a preset timeout, the arbitrator generates a compensatory arbitration certificate or enters a controlled degradation mode, retaining sufficient audit information for subsequent manual or automatic repair. For example, in cross-site transfer transactions, if concurrent borrowing and lending operations conflict and are classified as strongly consistent, the arbitrator initiates a short-term consensus among the borrowing and lending parties and the witness node, collects ready votes, and issues an arbitration certificate containing the submitted decision when a majority agrees, thereby ensuring the consistency and auditability of the total account balance.

[0084] Furthermore, regardless of the mode adopted, the arbitrator should ensure the determinism and verifiability of the arbitration certificate when generating it. The certificate must explicitly list the participating operation identifiers and operation hash values, the arbitration decision (i.e., the merge result or commit / rollback decision), the arbitration decision timestamp, and the verifier_hash digest field used for verification. The verifier_hash can be obtained by hashing the semantic concatenation of the decision, so that any party can recalculate and verify the integrity of the certificate afterward. The arbitrator and witness nodes can sign the arbitration certificate and persist it to an immutable or auditable log, facilitating final application, the order of anti-entropy repair, and post-audit. Once the arbitration certificate is generated and persisted, it serves as the sole authoritative basis for subsequent state transitions, and relevant sites apply changes according to the certificate to achieve data visibility and consistency.

[0085] In one embodiment, generating the source site's operation log record based on the write request further includes:

[0086] S21. When the operation semantic label is a mergeable class, perform a merge operation on the corresponding operation log records in the local pre-write log based on the idempotent convergent algorithm to obtain the updated operation log records.

[0087] Mergeable operations semantically allow for independent merging across sites using deterministic, idempotent, and convergent algorithms to achieve a consistent final state. After identifying multiple operation log records belonging to the same entity in the local pre-write log, a server-side merge operation is executed locally to merge several operations awaiting synchronization into a semantically equivalent and more compact record as early as possible. This reduces cross-domain transmission volume, lowers arbitration frequency, and improves local readability.

[0088] Specifically, the merging process must satisfy determinism, meaning that the same input set should produce the same output when processed by the same merging algorithm at any site or at any time; idempotency, meaning that repeatedly applying the same operation or certificate will not change the final result; and convergence, meaning that merging at all participating sites using the same algorithm will eventually result in a consistent state. For example, the arbitrator or local merging module selects a specific merging operator based on the operation semantic label, such as a CRDT-based counter / set operator or a normalized OT operator. Using the operation identifier and operation hash as input, it first filters out already applied or repeated operations locally by comparing op_id and op_hash, and then performs merging calculations on the payload data of the remaining operations in a fixed deterministic order. The merging result is encapsulated into a new or updated operation log record, i.e., the updated operation log record. Its fields, such as payload data summary and merged version / verification hash, are written back to the local pre-write log and persisted. Simultaneously, the local materialized view can be selectively updated to improve local read consistency and latency performance. By performing this merging step at the source site, the content subsequently sent by the replicator to the remote site is a more compact merged record, thereby reducing network bandwidth and the number of collisions and improving overall efficiency. For example, if the same counter is incremented by 1 three times consecutively locally, the merging algorithm will accumulate the three increments locally and write them into a single update log entry with an increment of 3, and then replicate the merge result externally.

[0089] S22. When the operation semantic label is eventually consistent or strongly consistent, the corresponding operation log record in the local pre-write log is marked as temporary. The temporary status is used to indicate that the source site locks the entity identifier to prevent local concurrent write requests.

[0090] As an illustration, for operations marked as eventually consistent or strongly consistent, simple local merging or optimistic immediate application may violate their semantic requirements. Therefore, the corresponding operation log record will not be immediately applied as the final visible state, but will be marked as tentative. This tentative state indicates that the operation has been persisted to the local pre-write log, but its final visibility depends on subsequent arbitration or consensus decisions, and triggers the source site to locally lock or sequence control the corresponding entity identifier to prevent local concurrent writes from causing semantic conflicts or inconsistencies before arbitration is completed.

[0091] Specifically, necessary metadata, such as operation identifier, timestamp, operation hash value, op_semantic_tag, and local sequence position at the time of writing, is retained in the operation log records marked as temporary. A lightweight lock or sequence token structure based on entity identifier is maintained locally. This lock can be a short-term mutex in memory, a queue-based write token, or write sequence number control. The granularity of the lock is usually based on entity identifier to avoid excessive serialization. Before arbitration or consensus is determined, concurrent commits of other local writes to the same entity are prevented. This ensures that when the arbitration certificate arrives and determines the final state, the local system only needs to convert the temporary write to a final commit based on the certificate or revoke it according to the compensation strategy. At the same time, to prevent deadlock or long-term blocking, the lock implementation should combine a timeout strategy and a rollback mechanism. That is, if no arbitration result is received within the preset timeout period, a controlled rollback can be triggered according to the strategy or the write can be retained for manual intervention, and necessary audit information should be recorded.

[0092] Optionally, in the specific pre-write process, after marking the operation as temporary, the source site will still copy the record to the remote site to participate in cross-domain arbitration. However, in the local materialized view, the operation will not directly become the final visible value until the corresponding arbitration certificate is issued and instructs the local site to convert the temporary record into a final commit or execute a compensation action. This ensures strong consistency and the correctness of transaction semantics while allowing high availability and audit trail capabilities to be maintained during arbitration. For example, in a cross-site fund transfer scenario, write operations on the debit and credit sides will be marked as temporary locally and the corresponding account entities will be locked. Only after the arbitrator issues a commit arbitration certificate will these temporary writes be converted into a final commit; if the arbitration fails or times out, a rollback or compensation action will be executed according to the arbitration certificate or compensation certificate.

[0093] In one embodiment, the method further includes:

[0094] S31. Based on the entity-level Merkle tree anti-entropy synchronization algorithm, perform difference detection, compare the Merkle trees of each site, and when a difference is detected, replay the corresponding operation log records in the order of the certificates according to the decision order recorded in the arbitration certificate, and update the Merkle trees of each site.

[0095] Based on entity-level Merkle trees, anti-entropy difference detection is performed. When a difference is found, the corresponding operation log records are replayed in the order of the arbitration certificate according to the decision order recorded in the certificate to repair the difference. Finally, the Merkle trees of each site are updated to reflect the repair results. Illustratively, the Merkle trees are constructed with entities as the smallest granularity. During the periodic detection phase, each site maintains a Merkle tree organized by entity identifier. Each leaf represents a state summary of the entity or a chunked operation sequence summary, and is periodically or by scheduling to trigger root hash exchanges or tree difference queries with other sites. For example, the construction of the Merkle tree can adopt a chunking strategy. For large objects or frequently updated entities, the entity state is divided into several chunks, each chunk is hashed and serves as a leaf node, so that when a root hash inconsistency is found, only the mismatched subtrees or chunks are further compared, thus avoiding full transmission. The leaf hash here is based at least on a summary of the persistent operation log records, such as a deterministic combination of the load hashes aggregated by time window or by arbitration certificate paragraphs, i.e., the op_hash list, to ensure the traceability of the replay order.

[0096] When a discrepancy is found after comparing the Merkle root hashes of two sites, the mismatched subtrees are expanded sequentially to pinpoint the precise chunk or entity range. This triggers a difference repair request per entity or chunk. The input difference unit typically includes the entity identifier, the relevant chunk number, and the corresponding local and remote hashes. To avoid large amounts of redundant data transmission, the repair process prioritizes incremental retrieval. The requester only requests chunks that differ from the local hash or their corresponding operation digest list, rather than the complete payload. The responder returns the operation log index within that chunk or directly returns a digest of the OpRecord list grouped by arbitration certificate, along with the necessary payload data.

[0097] Furthermore, when the operational history of the entity corresponding to the discrepancy involves a conflict decision segment for which an arbitration certificate has been generated, the remediation end replays or reapplies the corresponding OpRecords in sequence according to the decision sequence listed in the arbitration certificate. The replay process uses the order of the arbitration certificates as the sequence number. First, it verifies whether the op_hash of each OpRecord matches the participating op_hash recorded in the arbitration certificate to verify the integrity and correctness of the data source. If the verification passes, it then replays the operation in the local pre-write log with the original operational semantics or the updated merge record. If necessary, it converts the replayed temporary items or merge results into a final commit or retains them as temporarys. Specific actions follow the decision instructions within the arbitration certificate. This certificate-sequence-based replay ensures that the remediation actions do not semantically conflict with previous arbitration results, and all remediation steps are based on traceable arbitration certificates, facilitating auditing.

[0098] Optionally, to avoid conflicts with ongoing local writes, replay operations should be performed under appropriate lock granularity or sequence control. This could involve applying short-term repair locks to the affected entities or using write sequence numbers to ensure the atomicity of the replay. If new writes occur simultaneously during the repair period, a strategy should be adopted to temporarily queue the new writes, such as prioritizing arbitral certificate repair or allowing concurrency and re-checking differences after repair. Alternatively, during repair, short-term write token control can be applied to the affected entities and released after repair to avoid complex conflicts. Simultaneously, the replay process should be linked to arbitral certificate verification. That is, before each application of OpRecord, the op_hash should be recalculated and compared. If it matches the op_hash in the certificate, the application is confirmed; otherwise, an anomaly is recorded, and a manual or automatic rollback process is triggered to prevent the application of potentially tampered data.

[0099] Once the repair is complete, the repairer needs to update its local Merkle tree, which involves recalculating the leaf hashes of the replaced or added blocks and updating them up the tree to the root node so that the next difference check can reflect the latest state.

[0100] In one embodiment, the method further includes:

[0101] S41. When the operation log is in a temporary state, the arbitrator and the witness node will vote to determine the submission readiness and obtain the voting result.

[0102] As an illustration, when an operation log record is in a temporary state, the arbitrator, based on the consistency group to which the record belongs and a pre-determined list of participating members, sends a commit readiness request to the relevant participating sites and independent witness nodes. This request includes key information for verification, such as the operation identifier, operation hash value, local sequence number of the temporary write, and necessary contextual metadata, such as the associated transaction identifier `tentative_tx_id` (a set of temporary state identifiers). Specifically, the voting collection process requires participants to determine their readiness based on their local pre-write logs and business rules. If there are no impactful conflicts locally, the local verification has passed, and the necessary resources can be provided for submission, then agreement is returned; otherwise, rejection or a specific reason for incompatibility is returned. Voting interactions can use synchronous or asynchronous messaging mechanisms, and the timestamp and signature of each response are recorded for post-event verification. After collecting the votes, the arbitrator uses them as an important input for generating the arbitration certificate.

[0103] S42. When the voting result is a majority of responses agreeing, the arbitrator generates an arbitration certificate containing the participating sites and the decision that the submission is ready. The arbitration certificate is used to instruct each site to write the temporary state recorded in the operation log to the final submission based on the arbitration certificate.

[0104] Furthermore, when the arbitrator determines that the collected votes meet the preset submission conditions, such as a majority of responses agreeing or reaching a configured threshold, the arbitrator will generate an arbitration certificate based on the input. This certificate explicitly records the list of participating sites, the summary of agreeing and disagreeing votes, the list of participating operation identifiers being decided, the final decision, the arbitration decision timestamp (decision_ts), and a digest field for verification. It may also include the signatures or authentication credentials of the witness nodes or participating sites. The distribution process typically involves writing the certificate to the arbitrator's persistent audit log and distributing it to all relevant sites via a replicator. After verifying the certificate's integrity and signature, the receiving site applies the decision indicated by the certificate to its local pre-write log. This means converting the previously marked temporary operation log records into final commits and making them effective in the local materialized view, while simultaneously generating corresponding local persistent updates and audit entries. To reduce synchronization latency, certificate distribution can employ a layered confirmation strategy. This involves first writing the certificate to the logs of the arbitrator and witness nodes and broadcasting the digest. Then, each site verifies the certificate and submits locally. Finally, an anti-entropy mechanism ensures consistency across the entire network.

[0105] S43. When the voting result indicates that the submission conditions have not been met, the arbitrator generates an arbitration certificate containing compensation instructions; the compensation instructions are used to instruct each site to perform compensation actions on the operation log records.

[0106] Furthermore, when the arbitrator fails to collect enough votes to meet the submission criteria, it generates an arbitration certificate containing compensation instructions under a configurable strategy. This certificate instructs participating sites on how to revoke or compensate for previous temporary writes. The content of the compensation instructions depends on the business semantics and may include revoking a temporary change, executing a compensation transaction (such as reversing operations or adjusting records), or triggering upper-level manual / automatic exception handling processes. Regardless of the compensation action taken, the arbitration certificate should clearly define the compensation steps, participating entities, and compensation order, and include necessary verification fields to ensure each site executes compensation securely and in one go. Specifically, after determining that the conditions are not met, the arbitrator constructs and persists a compensation-type arbitration certificate according to the above rules. After the certificate is issued, each site verifies and executes the compensation instructions within the certificate, while simultaneously updating the pre-write log and materialized view locally to reflect the compensation result. In addition, the compensation execution process should generate logs and evidence, such as the op_hash and timestamp of the compensation operation, and send them back to the arbitrator to complete the closed-loop recording.

[0107] Optionally, to avoid prolonged blocking and uncertainty, a reasonable timeout strategy should be set for the voting and waiting processes. Upon timeout, the arbitrator can choose to automatically enter the compensation path, initiate a broader voting process, or trigger manual intervention. To improve the credibility of the decision, voting responses and arbitration certificates should, as far as possible, employ signature or digest verification mechanisms to prevent tampering, and the certificate and vote summary should be written to an immutable or audit log to preserve the chain of evidence. To avoid concurrent conflicts between certificate issuance and local applications, each site should use local short-term locks or sequence tokens when making application certificate decisions to ensure the atomicity of the conversion operation. All arbitration certificates serve as the authoritative basis for anti-entropy repair and final materialized view updates; the persistent recording of arbitration certificates ensures the traceability and verifiability of the entire process.

[0108] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0109] Based on the same inventive concept, this application also provides a cross-domain data synchronization system for dual-active arbitration to implement the cross-domain data synchronization method for dual-active arbitration described above. The solution provided by this system is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more embodiments of the cross-domain data synchronization system for dual-active arbitration provided below can be found in the limitations of the cross-domain data synchronization method for dual-active arbitration described above, and will not be repeated here.

[0110] In one exemplary embodiment, such as Figure 4 As shown, a cross-domain data synchronization system with dual-active arbitration is provided, including:

[0111] The data modification request module 401 is used to obtain write requests and generate operation log records for the source site based on the write requests. The operation log records include entity identifier, operation identifier, source site, operation type, payload data, timestamp, compressed causal summary, operation hash value, and operation semantic tag.

[0112] The cross-domain push module 402 is used to fragment the operation log records according to the entity identifier and push them to the remote site through the replicator, so as to obtain a copy of the operation log records of the remote site.

[0113] The request conflict module 403 is used to respond to the discovery that there is a local concurrent unresolved issue in the operation log record of the remote site that corresponds to the entity identifier of the operation log record copy. The compressed causal summary in the operation log record copy and the local concurrent unresolved issue with the corresponding timestamp are used to perform conflict candidate detection to obtain a conflict candidate set. The conflict candidate set includes each conflict pair and its operation semantic tag.

[0114] Arbitration module 404 is used to determine the arbitration strategy for each conflict pair based on the operation semantic tags in the conflict candidate set, and to perform a merging operation on each conflict pair according to the arbitration strategy to obtain an arbitration certificate.

[0115] The data synchronization module 405 is used to write the arbitration certificate into the local materialized views of the source site and the remote site and perform persistent processing to obtain a consistent data visibility result.

[0116] In one embodiment, a business operation definition module is also included, for:

[0117] The entities to be managed are classified according to business consistency requirements and semantic characteristics to obtain a consistency grouping mapping table; the consistency grouping mapping table includes entity identifiers and their corresponding operation semantic tags: operation semantic tags include strong consistency class, mergeable class and eventual consistency class.

[0118] In one embodiment, the data modification request module 401 is further configured to:

[0119] Generate an operation identifier for the write request;

[0120] A compressed causal summary is generated based on the local set of most recently interacted sites from the source site. The compressed causal summary includes short vector summaries of hot sites and compressed summaries of long-tail sites, along with their corresponding timestamps. The local set of most recently interacted nodes is obtained from a list of site activity within a preset time window. The list of site activity includes hot sites and long-tail sites.

[0121] Calculate the operation hash value for the operation log record, and write the operation hash value, operation identifier, and compressed causal summary into the operation log record;

[0122] The operation log is recorded and written to the local pre-write log of the source site, and a local commit confirmation is received.

[0123] In one embodiment, the arbitration module 404 is further configured to:

[0124] When the operation semantic label is a mergeable class, the arbitration strategy is the fast merge mode. The fast merge mode corresponds to the arbitrator performing server-side merging of the operation type and payload data of the conflicting pair based on the mergeable algorithm to produce a merge result and obtain an arbitration certificate.

[0125] When the operation semantic label is eventually consistent, the arbitration strategy is the deterministic semantic arbitration mode: the deterministic semantic arbitration mode corresponds to the arbitrator executing a deterministic semantic merging function on the operation type and payload data of the conflicting pair under the vote of the source site, remote site and witness node to produce a deterministic output and obtain the arbitration certificate;

[0126] When the semantic label of the operation is a strong consistency class, the arbitration strategy is a short-term local consensus mode. The short-term local consensus mode corresponds to a local-scale consensus group composed of the source site, the remote site and the witness node. Based on the arbitration vote of the local-scale consensus group, the final data state is determined according to the operation type and payload data of the conflicting pair, and an arbitration certificate is obtained.

[0127] In one embodiment, a fast path status marking module is also included, for:

[0128] When the operation semantic label is a mergeable class, the corresponding operation log records in the local pre-write log are merged based on the idempotent convergent algorithm to obtain the updated operation log records.

[0129] When the operation semantic label is eventually consistent or strongly consistent, the corresponding operation log record in the local pre-write log is marked as temporary; the temporary status is used to indicate that the source site locks the entity identifier to prevent local concurrent write requests.

[0130] In one embodiment, an anti-entropy repair module is also included, for:

[0131] The anti-entropy synchronization algorithm based on entity-level Merkle trees is used for difference detection. The Merkle trees of each site are compared. When a difference is detected, the corresponding operation log records are replayed in the order of the certificates according to the decision order recorded in the arbitration certificate, and the Merkle trees of each site are updated.

[0132] In one embodiment, a state transition module is also included, for:

[0133] When the operation log is in a temporary state, the arbitrator and the witness node will vote on whether the operation is ready to be committed, and obtain the voting result.

[0134] When the voting result is a majority of responses in agreement, the arbitrator generates an arbitration certificate containing the participating sites' decisions on whether to submit and be ready. The arbitration certificate is used to instruct each site to write the temporary state recorded in the operation log to the final submission based on the arbitration certificate.

[0135] When the voting result indicates that the submission conditions have not been met, the arbitrator generates an arbitration certificate containing compensation instructions; the compensation instructions are used to instruct each site to perform compensation actions on the operation log records.

[0136] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps in the above method embodiments.

[0137] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0138] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0139] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A cross-domain data synchronization method with dual-active arbitration, characterized in that, The method includes: Obtain a write request and generate an operation log record for the source site based on the write request; the operation log record includes entity identifier, operation identifier, source site, operation type, payload data, timestamp, compressed causal summary, operation hash value, and operation semantic tag; The operation log records are fragmented according to the entity identifier and pushed to the remote site using a replicator to obtain a copy of the operation log records at the remote site. In response to the discovery that there is a locally concurrent pending operation log record in the local pre-write log of the remote site that corresponds to the entity identifier of the operation log record copy, the compressed causal summary in the operation log record copy and the locally concurrent pending operation log record with the corresponding timestamp are used to perform conflict candidate detection to obtain a conflict candidate set; the conflict candidate set includes each conflict pair and its operation semantic tag; The arbitration strategy for each conflict pair is determined based on the operation semantic tags in the conflict candidate set, and the conflict pairs are merged according to the arbitration strategy to obtain an arbitration certificate. The final state of the write request indicated by the arbitration certificate is written to the local materialized views of the source site and the remote site and persisted to obtain a consistent data visibility result.

2. The method according to claim 1, characterized in that, Before obtaining the write request and generating the source site's operation log record based on the write request, the method further includes: The entities to be managed are classified according to business consistency requirements and semantic features to obtain a consistency grouping mapping table; the consistency grouping mapping table includes the entity identifier and its corresponding operation semantic label: the operation semantic label includes strong consistency class, mergeable class and eventual consistency class.

3. The method according to claim 2, characterized in that, The step of generating the source site's operation log record based on the write request includes: Generate an operation identifier for the write request; A compressed causal summary is generated based on the local set of most recently interacted sites of the source site; the compressed causal summary includes a short vector summary of hot sites and a compressed summary of long-tail sites and their corresponding timestamps; the local set of most recently interacted nodes is obtained from a list of site activity within a preset time window; the list of site activity includes the hot sites and the long-tail sites. Calculate the operation hash value for the operation log record, and write the operation hash value, the operation identifier, and the compressed causal summary into the operation log record; The operation log is recorded and written to the local pre-write log of the source site, and a local submission confirmation is received.

4. The method according to claim 2, characterized in that, The step of determining the arbitration strategy for each conflict pair based on the operational semantic labels in the conflict candidate set includes: When the operation semantic label is the mergeable class, the arbitration strategy is the fast merge mode; the fast merge mode corresponds to the arbitrator performing server-side merging of the operation type and payload data of the conflict pair based on the mergeable algorithm to generate a merge result and obtain the arbitration certificate; When the operation semantic label is the eventual consistency class, the arbitration strategy is a deterministic semantic arbitration mode: the deterministic semantic arbitration mode corresponds to the arbitrator executing a deterministic semantic merging function on the operation type and payload data of the conflict pair under the votes of the source site, remote site and witness node to generate a deterministic output and obtain the arbitration certificate; When the operation semantic label is the strong consistency class, the arbitration strategy is a short-term local consensus mode; the short-term local consensus mode corresponds to a local-scale consensus group composed of the source site, the remote site and the witness node, and the arbitration certificate is obtained based on the arbitration vote of the local-scale consensus group, the final data state is determined according to the operation type of the conflicting pair and the payload data.

5. The method according to claim 3, characterized in that, The step of generating the source site's operation log record based on the write request further includes: When the operation semantic label is the mergeable class, the corresponding operation log records in the local pre-write log are merged based on the idempotent convergent algorithm to obtain the updated operation log records. When the operation semantic label is the eventually consistent class or the strongly consistent class, the corresponding operation log record in the local pre-write log is marked as temporary; the temporary state is used to instruct the source site to lock the entity identifier to prevent local concurrent write requests.

6. The method according to claim 1, characterized in that, The method further includes: The anti-entropy synchronization algorithm based on entity-level Merkle trees is used to detect differences. The Merkle trees of each site are compared. When a difference is detected, the corresponding operation log record is replayed in the order of the certificate according to the decision order recorded in the arbitration certificate, and the Merkle trees of each site are updated.

7. The method according to claim 5, characterized in that, The method further includes: When the operation log is recorded as the temporary state, the arbitrator and the witness node conduct a vote to determine the submission readiness and obtain the voting result. When the voting result is a majority of responses in agreement, the arbitrator generates an arbitration certificate containing the participating sites and the decision that the submission is ready; the arbitration certificate is used to instruct each site to write the temporary state recorded in the operation log to the final submission based on the arbitration certificate; When the voting result indicates that the submission conditions have not been met, the arbitrator generates an arbitration certificate containing a compensation instruction; the compensation instruction is used to instruct each site to perform a compensation action on the operation log record.

8. A cross-domain data synchronization system with dual-active arbitration, characterized in that, The system includes: The data modification request module is used to obtain write requests and generate operation log records for the source site based on the write requests; the operation log records include entity identifier, operation identifier, source site, operation type, payload data, timestamp, compressed causal summary, operation hash value, and operation semantic tag; The cross-domain push module is used to fragment the operation log records according to the entity identifier and push them to the remote site through a replicator, thereby obtaining a copy of the operation log records of the remote site. The conflict request module is configured to, in response to the discovery that there is a local concurrent unresolved issue in the operation log record of the remote site that corresponds to the entity identifier of the operation log record copy, perform conflict candidate detection on the compressed causal summary in the operation log record copy and the local concurrent unresolved issue with the corresponding timestamp to obtain a conflict candidate set; the conflict candidate set includes each conflict pair and its operation semantic tag; The arbitration module is used to determine the arbitration strategy for each conflict pair based on the operation semantic tags in the conflict candidate set, and to perform a merging operation on each conflict pair according to the arbitration strategy to obtain an arbitration certificate; The data synchronization module is used to write the arbitration certificate into the local materialized views of the source site and the remote site and perform persistent processing to obtain a consistent data visibility result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images