Data comparison method and apparatus, storage medium, and electronic device
By using Merkle trees for data comparison and synchronization in a cross-cluster environment, the problem of data consistency between different server clusters is solved, achieving low-bandwidth, low-traffic, and high-timeliness data synchronization.
Patent Information
- Application Number
- CN202310916222.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-24
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2043-07-24
AI Technical Summary
In the era of big data, how can we ensure the consistency of replica data on each cluster in a distributed computing platform that spans multiple regions and clusters, especially when synchronizing data between different server clusters, and how can we achieve data comparison and synchronization in a low-bandwidth, low-traffic, and high-time-efficiency manner?
By obtaining the Merkle tree corresponding to the original data in the first server cluster, allocating target bandwidth resources to the second server cluster based on bandwidth resources, replicating the Merkle tree for data comparison, and using the Merkle tree for data synchronization, the amount of data copied across clusters is reduced.
It enables low-bandwidth, low-traffic, and high-time-efficiency data comparison and synchronization, effectively manages bandwidth resources, reduces cross-cluster data replication, and ensures data consistency.
Smart Images

Figure CN116962434B_ABST
Abstract
Description
Technical Field
[0001] This specification relates to the field of computer technology, and in particular to a data comparison method, apparatus, storage medium, and electronic device. Background Technology
[0002] The advent of the big data era has enhanced the value and expanded the application scope of data across various industries. Different tasks need to access the same data and perform downstream scheduling, but this data is often stored in data centers in different countries or regions. Distributed computing platforms that support cross-regional and multi-cluster operations can effectively solve this problem. The common approach is to collect the data into multiple replicas across different clusters, with tasks in different regions accessing different replicas. This means that the same data will be stored on multiple clusters, and the data on each cluster may correspond to different versions. Therefore, ensuring the consistency of data replicas across different clusters is crucial. Summary of the Invention
[0003] This specification provides a data comparison method, apparatus, storage medium, and electronic device, which can achieve data comparison and synchronization between two server clusters in a low-bandwidth, low-traffic, and high-time-efficiency manner. The technical solution is as follows:
[0004] Firstly, embodiments of this specification provide a data comparison method, the method comprising:
[0005] Obtain the Merkle tree corresponding to the original data in the first server cluster;
[0006] Based on the bandwidth resources of the first server cluster, allocate the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster;
[0007] The Merkle tree is replicated to the second server cluster according to the target bandwidth resources, so that the second server cluster can perform data comparison based on the Merkle tree.
[0008] Secondly, embodiments of this specification provide a data comparison device, the device comprising:
[0009] The data acquisition module is used to acquire the Merkle tree corresponding to the original data in the first server cluster;
[0010] The resource allocation module is used to allocate the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster based on the bandwidth resources of the first server cluster.
[0011] The data replication module is used to replicate the Merkle tree to the second server cluster according to the target bandwidth resources, so that the second server cluster can perform data comparison based on the Merkle tree.
[0012] Thirdly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the above-described method steps.
[0013] Fourthly, embodiments of this specification provide a computer program product that stores multiple instructions adapted for loading by a processor and executing the above-described method steps.
[0014] Fifthly, embodiments of this specification provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.
[0015] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:
[0016] In the embodiments of this specification, the target server of the first server cluster calculates and determines the Merkle tree corresponding to the original data based on the original data in the first server cluster. Furthermore, it allocates the target bandwidth resources required to execute the replication task based on the bandwidth resources of the first server cluster. The replication task is to copy the Merkle tree corresponding to the original data to the second server cluster, so that the second server cluster can perform a data comparison task based on the Merkle tree, thereby achieving data synchronization between the first and second server clusters. Since cross-cluster replication tasks consume the bandwidth resources of the server clusters, this embodiment of the specification achieves bandwidth management by reasonably configuring the bandwidth resources consumed by the replication task. Furthermore, by copying the Merkle tree of the original data instead of the original data, the amount of data copied across clusters is significantly reduced, thereby achieving data comparison and synchronization between the two server clusters in a low-bandwidth, low-traffic, and high-time-efficiency manner. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a schematic diagram illustrating a data comparison method provided in the embodiments of this specification;
[0019] Figure 2 This is a flowchart illustrating a data comparison method provided in the embodiments of this specification;
[0020] Figure 3 This is a schematic diagram of the structure of a data comparison method provided in the embodiments of this specification;
[0021] Figure 4 This is a flowchart illustrating a data comparison method provided in the embodiments of this specification;
[0022] Figure 5 This is a schematic diagram of a Merkle tree structure provided in the embodiments of this specification;
[0023] Figure 6 This is an interactive schematic diagram of a data comparison method provided in the embodiments of this specification;
[0024] Figure 7 This is a schematic diagram of the structure of a data comparison device provided in the embodiments of this specification;
[0025] Figure 8 This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification. Detailed Implementation
[0026] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.
[0027] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.
[0028] The present specification will now be described in detail with reference to specific embodiments.
[0029] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the object characteristics, interactive behavior characteristics, and user information involved in this specification were all obtained with full authorization.
[0030] The advent of the big data era has enhanced the value and expanded the application scope of data across various industries. Different tasks need to access the same data and perform downstream scheduling, but this data is often stored in data centers in different countries or regions. Distributed computing platforms that support cross-regional and multi-cluster operations can effectively solve this problem. The common approach is to collect the data into multiple replicas across different clusters, with tasks in different regions accessing different replicas. This means that the same data will be stored on multiple clusters, and the data on each cluster may correspond to different versions. Therefore, ensuring the consistency of data replicas across different clusters is crucial.
[0031] In this specification, data synchronization can be understood as the operation of eliminating differences between data stored on different servers and different server clusters. After the data on each server in a server cluster is identical, the specific content of the data stored separately on different server clusters may differ due to limitations in data update methods, etc. The process of unifying the data stored separately on different server clusters into the same content is the data synchronization process described in the embodiments of this specification.
[0032] like Figure 1 As shown, Figure 1 This is a schematic diagram illustrating a data comparison method provided in an embodiment of this specification. The schematic diagram includes a first server cluster 101 and a second server cluster 102. It is understood that... Figure 1 The number of server clusters shown is for illustrative purposes only. The data comparison method provided in this specification is also applicable to scenarios where data comparison is performed between three or more server clusters.
[0033] The first server cluster 101 and the second server cluster 102 each include at least one server. The aforementioned servers can be multiple physical servers, which are hardware-independent, or they can be multiple virtual servers deployed in the same hardware resource pool. The deployment methods of virtual servers include, but are not limited to, VMware, VirtualBox, and Virtual PC.
[0034] When the first server cluster 101 includes multiple servers, these servers communicate with each other through their respective configured interfaces, and send data or provide services to other servers in the cluster based on the specific content of the communication. Specifically, the multiple servers in the first server cluster 101 can communicate through communication links established based on communication protocols, such as the gRPC protocol. gRPC is a high-performance, general-purpose open-source Remote Procedure Call (RPC) framework, primarily designed for mobile application development and based on the HTTP / 2 protocol standard. It is developed based on the Protocol Buffers (PB) serialization protocol and supports numerous programming languages. The communication link can be a wireless communication link or a wired communication link. For example, wired communication links include fiber optic cables, twisted-pair cables, or coaxial cables, while wireless communication links include Bluetooth communication links, Wi-Fi communication links, or microwave communication links, etc.
[0035] The second server cluster 102 is the same as the first server cluster 101, and will not be described again here.
[0036] The first server cluster 101 and the second server cluster 102 are deployed in different regions, and the two server clusters communicate with each other through the servers included in each server cluster. For example, the first server cluster 101 includes server 1011, and the second server cluster 102 includes server 1021. Servers 1011 and 1021 can communicate with each other through multiple interfaces they have set up, and can send data or provide services to servers in the other cluster based on the specific content of the communication, thereby realizing data exchange between the two server clusters.
[0037] This specification provides a data comparison method to address the aforementioned issues regarding data comparison and synchronization between different server clusters. In one embodiment, as follows... Figure 2 The illustration shows a data comparison method proposed in an embodiment of this specification. This method can be implemented using a computer program and can run on a data comparison device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application.
[0038] Specifically, the data comparison method includes:
[0039] S102. Obtain the Merkle tree corresponding to the original data in the first server cluster.
[0040] The data comparison method described in this specification is implemented by the target server in the first server cluster. The first server cluster can be understood as a cluster of servers synchronizing data with other server clusters. The target server in the first server cluster can be understood as a server that includes the original data from the first server cluster and has the capability to exchange data with servers in other server clusters to achieve data synchronization. For example, ... Figure 1 As shown, the executing entity is the target server 1011 in the first server cluster 101. In this embodiment, other server clusters include a second server cluster. For example, the data in the second server cluster is a copy of the original data in the first server cluster. By executing a data comparison method on the target server of the first server cluster, data synchronization between the first server cluster and the second server cluster is achieved.
[0041] The first server cluster can contain massive amounts of data. The original data can be understood as the data that needs to be synchronized from the massive amount of data included in the first server cluster. In one embodiment, the original data is stored in the first server cluster using a distributed storage (key-value) method, where the key can be the keyword (or data identifier) of the original data, and the value can be the data value of the original data.
[0042] A Merkle Tree can be understood as a data structure implemented based on blockchain. In this embodiment, the original data undergoes multiple hash calculations to obtain a Merkle Tree implemented by a large aggregation of hash values. Specifically, a Merkle Tree relies on splitting the original data into smaller data units, each bucket containing only a few data "blocks," and then hashing the data from each bucket unit again, repeating the same process until the remaining total hash value becomes only 1. In this embodiment, the Merkle Tree can be a binary tree or a multi-way tree; this specification does not impose any limitations.
[0043] S104. Based on the bandwidth resources of the first server cluster, allocate the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster.
[0044] The bandwidth resources of a server cluster are a limited, fixed value. Servers in the cluster consume bandwidth resources when performing tasks such as replication and read / write operations. Therefore, it is necessary to allocate target bandwidth resources reasonably for the target servers in the first server cluster to perform replication tasks, thereby ensuring the execution of tasks in the first server cluster. The content of the aforementioned replication task is to replicate the Merkle tree to the second server cluster.
[0045] It is understood that, in this specification, the task of copying the Merkle tree to the second server cluster can specifically be either the target server in the first server cluster copying data to each server cluster in the second server cluster, or the target server in the first server cluster copying data to the target server cluster in the second server cluster.
[0046] In one embodiment, the target bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster are allocated based on the bandwidth resource usage of the first server cluster. The bandwidth resource usage of the first server cluster includes the total bandwidth resources of the first server cluster, the currently remaining bandwidth resources, and the historical usage of the bandwidth resources.
[0047] For example, if the remaining bandwidth resources of the first server cluster are aGB, then the target bandwidth resources allocated for the replication task of replicating the Merkle tree to the second server cluster will not exceed aGB. As another example, if the remaining bandwidth resources of the first server cluster are aGB, and based on the historical bandwidth usage of the first server cluster, it is predicted that the bandwidth resources required for the first server cluster to perform other tasks in the future period will be bGB, then the target bandwidth resources allocated for the replication task will not exceed (ab)GB, and the duration of this future period can be determined based on the time required to execute the replication task.
[0048] In another embodiment, such as Figure 3 As shown, Figure 3 This is a schematic diagram of a data comparison method provided in an embodiment of this specification. A data replication task to other cluster servers includes at least one of the following information: server cluster address, table, partition, version, physical file address, physical file meta, etc.
[0049] Figure 3 The processor 10111 shown is the processor of the target server 1011 of the first server cluster 101. The processor 10111 can perform the replication task of copying the Merkle tree to the second server cluster 102 based on Open Table Service (OST). The data management of the first server cluster 101 relies on Open Table Service (OST) and is organized and represented according to the hierarchy of server cluster project, table, and partition.
[0050] A partition consists of at least two server clusters. Figure 3 In the illustrated embodiment, the partition includes at least a first server cluster 101 and a second server cluster 102. Each server cluster project includes a basic physical file, Basic meta, which is the original data. In this embodiment, the Basic physical file Basic meta in each server cluster is characterized by a version. Different versions mean that the specific content of the Basic physical file Basic meta is different, and each version corresponds to a version number.
[0051] For example, the metadata table information corresponding to this partition includes {"Latest Version": V1, "ClusterStatus": {"ClusterA": "V1", "ClusterB": "V0"}}. This information indicates that the partition includes server clusters ClusterA and ClusterB, and the effective version number of the original data in this partition is V1. The version number corresponding to the original data on server clusterA is V1, and the version number corresponding to the original data on server clusterB is V0. Here, "Latest Version" is an abstract description representing the version number corresponding to the effective version of the original data in this partition. V1 and V0 are strings, typically not used for semantic judgment, and there is no temporal relationship between V1 and V0. "Cluster Status" indicates the server clusters included in this partition. Since the version number of the original data in this partition is V1, the version number of server cluster A is V1, and the version number of server cluster B is V0, then server cluster A is a valid cluster, namely the first server cluster 101. Server cluster A copies the Merkle tree of the basic physical file Basic Meta of server cluster A to server cluster B, which is the second server cluster 102.
[0052] Specifically, when the processor 10111 of the first server cluster 101 detects that the version of the original data included in the first server cluster 101 has changed, it generates a replication task to copy the Merkle tree corresponding to the original data to the second server cluster 102.
[0053] The method for detecting a change in the version of the raw data of the first server cluster 101 may be one or more of the following methods: detecting that the versions corresponding to the first server cluster 101 and the second server cluster 102 are different when periodically scanning the metadata table information of the partition; receiving the instruction information event generated when the raw data of the first server cluster 101 occurs through the Message Service; receiving the target instruction sent by the user through the user terminal 201ConTroL to start the replication task. This specification embodiment also includes other possible ways for the first server cluster to start executing the replication task; the above are merely examples.
[0054] Furthermore, the replication tasks of processor 10111 are waiting to be executed by processor 1011 in the waiting queue (Waiting List) where all tasks are executed in the first server cluster 101. In one embodiment, the replication tasks of processor 10111 are arranged in the waiting queue (Waiting List) in descending order of priority.
[0055] Furthermore, the processor 10111 organizes the Merkle tree included in the replication task into a job. The Merkle tree corresponding to the basic physical file Basic meta of the first server cluster 101 is copied to the second server cluster 102 through this job. When organizing the job, the number of process instances required by the job is calculated and determined according to the upper limit of each process instance. The calculated number of process instances required by the replication task represents the computing bandwidth resources required by the replication task.
[0056] In this embodiment, the first server cluster 101 manages its bandwidth resources through a responder worker. The responder worker can be set on the target server 1011 of the first server cluster 101, or on other servers within the first server cluster 101. The responder worker is the interface through which the replication process (task) corresponding to a replication task interacts with the task processes corresponding to other tasks. One of the functions of the responder worker is to implement bandwidth management.
[0057] The method for obtaining the target bandwidth resources required for the replication task from the responder worker is as follows: Based on information such as the bandwidth resource usage of the first server cluster 101, the computational bandwidth resources of the replication task, and the priority of the replication task, the target bandwidth resources for the replication task are obtained. Specifically, the bandwidth resources of the first server cluster 101 are a finite, fixed value. Assuming that the read / write throughput of each process instance is x MB / s, and the current bandwidth resources of the first server cluster 101 are y Gb, then the target bandwidth resources for the replication task include z process instances, and z satisfies: z ≤ y Gb × 1024 / (8 × x).
[0058] Since replication tasks consume bandwidth resources of the first server cluster 101, it is necessary to configure the task concurrency of the first server cluster 101 appropriately. For example, the task concurrency of the first server cluster 101 can be set through the global configuration of the multi-cluster system. The global configuration specifically includes the starting concurrency number Available Bandwidth and the synchronization rate tcspeed.
[0059] S106. Copy the Merkle tree to the second server cluster according to the target bandwidth resources, so that the second server cluster can perform data comparison based on the Merkle tree.
[0060] The target servers in the first server cluster perform a replication task based on the target bandwidth resources of the first server cluster, that is, replicating the Merkle tree to the second server cluster. The second server cluster compares the Merkle tree corresponding to the original data sent by the first server cluster. Specifically, after receiving the Merkle tree sent by the first server cluster, at least one target server in the second server cluster compares its own Merkle tree corresponding to the original data with the Merkle tree of the first server cluster until it finds the data difference between the target servers in the second server cluster and the first server cluster, thereby completing the data synchronization.
[0061] In one embodiment, such as Figure 3 As shown, the processor 10111 retrieves a copy task from the waiting queue (Waiting List) based on the target bandwidth resources and executes it. After the copy task is completed, the processor records the execution record in the running queue (Running List) and modifies the metadata table information corresponding to the partition.
[0062] Specifically, the replication job, which copies the Merkle tree to the second server cluster, consists of three subtasks: a planning subtask, a replication subtask, and a statics subtask. The planning subtask distributes the Merkle tree to be copied evenly among the instances of the replication subtask according to its size. The replication subtask copies the Merkle tree from the first server cluster 101 to the second server cluster 102. The statics subtask records whether the replication was successful for each instance based on the replication results of the replication subtask.
[0063] It is understood that the above-described method of replicating the Merkle tree from the first server cluster to the second server cluster is only one possible embodiment provided by the embodiments of this specification. This specification also includes other technical means for performing replication tasks.
[0064] In the embodiments of this specification, the target server of the first server cluster calculates and determines the Merkle tree corresponding to the original data based on the original data in the first server cluster. Furthermore, it allocates the target bandwidth resources required to execute the replication task based on the bandwidth resources of the first server cluster. The replication task is to copy the Merkle tree corresponding to the original data to the second server cluster, so that the second server cluster can perform a data comparison task based on the Merkle tree, thereby achieving data synchronization between the first and second server clusters. Since cross-cluster replication tasks consume the bandwidth resources of the server clusters, this embodiment of the specification achieves bandwidth management by reasonably configuring the bandwidth resources consumed by the replication task. Furthermore, by copying the Merkle tree of the original data instead of the original data, the amount of data copied across clusters is significantly reduced, thereby achieving data comparison and synchronization between the two server clusters in a low-bandwidth, low-traffic, and high-time-efficiency manner.
[0065] In one embodiment, such as Figure 4 The illustration shows a data comparison method proposed in an embodiment of this specification. This method can be implemented using a computer program and can run on a data comparison device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application.
[0066] Specifically, the data comparison method includes:
[0067] S202. When the preset conditions are met, obtain the Merkle tree corresponding to the original data in the first server cluster.
[0068] The preset conditions include at least one of the following: the metadata table containing information about the first server cluster and the second server cluster changes; a periodic instruction to execute a replication task is received; or a target instruction to execute a replication task is received from a user.
[0069] For example, after a server in the first server cluster executes a task that modifies data content, the original data of the first server cluster changes, including changes to the metadata tables containing information about both the first and second server clusters. Another example is that the first server cluster has a pre-defined instruction to periodically execute replication tasks; when this instruction is triggered, the replication task is executed. Yet another example is that the target server in the first server cluster receives a target instruction from a user via a user terminal to execute a replication task. This target instruction instructs the target server in the first server cluster to send a Merkle tree to the second server cluster to achieve data synchronization between the first and second server clusters.
[0070] In one embodiment, the method for obtaining the Merkle tree corresponding to the original data is as follows: the original data is sliced to obtain at least one slice data corresponding to the original data, and the slice number corresponding to the at least one slice data respectively; based on the at least one slice data, the Merkle tree corresponding to the at least one slice data is determined, wherein the Merkle tree includes the correspondence between the Merkle tree and the at least one slice number.
[0071] Specifically, such as Figure 5 As shown, Figure 5 This is a schematic diagram of a Merkle tree structure provided in the embodiments of this specification. The original data of the first server cluster is sliced to obtain at least one slice. The granularity of the data slice can be determined according to the amount of original data. For example, when the amount of original data is large, it can be sliced according to the number of rows in the original data, thus obtaining N slices.
[0072] Furthermore, after obtaining at least one slice of data corresponding to the original data, a slice number is assigned to each slice, resulting in a mapping table between the slice data and the slice number. For example... Figure 5 As shown, the original data of the first server cluster is sliced to obtain slice data 1, slice data 2, slice data 3, slice data 4, ..., slice data N, as well as the slice number 1 corresponding to slice data 1, the slice number 2 corresponding to slice data 2, the slice number 3 corresponding to slice data 3, the slice number 4 corresponding to slice data 4, ..., the slice number N corresponding to slice data N.
[0073] Furthermore, a hash calculation is performed on at least one slice of data to obtain a hash value corresponding to each slice. The hash calculation can be implemented based on various hash algorithms, such as SHA-2 or MD5. After hashing each slice of data, a hash value for each slice is obtained. These hash values have a consistent length and are used as the root node value of a Merkle tree. The hash values at the root node are set according to a fixed data identifier order. Figure 5 As shown, hash calculations are performed on slice data 1, slice data 2, slice data 3, slice data 4, ..., slice data N respectively to obtain hash value 1 corresponding to slice data 1, hash value 2 corresponding to slice data 2, hash value 3 corresponding to slice data 3, hash value 4 corresponding to slice data 4, ..., hash value N corresponding to slice data N. Hash value 1, hash value 2, hash value 3, hash value 4, ..., hash value N are used as leaf nodes of the Merkle tree corresponding to the original data.
[0074] Furthermore, a cyclic hash calculation is performed on the hash values of the leaf nodes to obtain the complete Merkle tree. Specifically, multiple leaf nodes are merged according to the preset Merkle tree structure to obtain upper-level leaf nodes; for example, the preset Merkle tree structure is a binary tree structure. Then, the leaf nodes are hashed again to merge them, obtaining new upper-level nodes, until the number of upper-level nodes is 1, which is the root node of the Merkle tree. Figure 5 As shown, hash calculations are performed on hash value 1 and hash value 2 to merge them to obtain hash value 12, hash calculations are performed on hash value 3 and hash value 4 to merge them to obtain hash value 34, hash calculations are performed on hash value N-1 and hash value N to merge them to obtain hash value (N-1)N, then hash calculations are performed on hash value 12 and hash value 34 to merge them to obtain hash value 1234, until a Merkle tree is obtained, and the Merkle tree includes the correspondence between the Merkle tree and at least one slice number.
[0075] S204. Based on the Merkle tree, determine the preset bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster.
[0076] Specifically, based on the amount of data in the Merkle tree, the preset bandwidth resources required for the replication task of copying the Merkle tree to the second server cluster are determined. Since the Merkle tree is obtained by slicing and hashing the original data, there is a correlation between the amount of data in the Merkle tree and the amount of data in the original data. The preset bandwidth resources can be understood as the bandwidth resources required for the replication task, calculated accordingly. In this embodiment, the preset bandwidth resources required for the replication task are determined based on the amount of data in the Merkle tree to ensure the smooth execution of the replication task.
[0077] In one embodiment, a method for determining the preset bandwidth resources required for a replication task includes: determining the preset bandwidth resources required for a replication task of replicating a Merkle tree to a second server cluster based on the data volume of the Merkle tree and the data throughput of a single process. The preset bandwidth resources for the Merkle tree include the number of processes required for the Merkle tree replication task. Specifically, the target server in the first server cluster organizes the Merkle tree included in the replication task into a job. Based on the preset data throughput of each process instance in the first server cluster and the data volume of the Merkle tree, the number of process instances required for the replication task is calculated and determined. The number of process instances represents the computational bandwidth resources required for the replication task.
[0078] S206. Based on the bandwidth resources of the first server cluster and the preset bandwidth resources, allocate target bandwidth resources for the replication task.
[0079] Since the bandwidth resources of a server cluster are a limited, fixed value, each server in the cluster will consume bandwidth resources when performing tasks such as replication and read / write operations. Therefore, it is necessary to allocate target bandwidth resources reasonably for the target servers in the first server cluster to perform replication tasks, thereby ensuring the execution of tasks on the first server cluster.
[0080] In one embodiment, target bandwidth resources are allocated to the replication task based on the bandwidth resources of the first server cluster, the preset bandwidth resources, and the priority of the replication task. In this embodiment, the target bandwidth resources allocated to the replication task are determined based on the bandwidth resources of the first server cluster and the preset bandwidth resources required by the replication task. Specifically, the bandwidth resources of the first server cluster are a finite fixed value. Assuming that the read / write throughput of each process instance is cMB / s and the data volume of the Merkle tree is m, the preset bandwidth resources required by the replication task are determined to be uGb. Since the current bandwidth resources of the first server cluster are kGb, the target bandwidth resources allocated to the replication task are tGB, and t satisfies u≤t≤k.
[0081] Since replication tasks consume bandwidth resources of the first server cluster, it is necessary to configure the task concurrency of the first server cluster appropriately. For example, the task concurrency of the first server cluster can be set through global configuration of multiple clusters. The global configuration specifically includes the available bandwidth and the synchronization rate tcspeed.
[0082] S208. Copy the Merkle tree to the second server cluster according to the target bandwidth resources, so that the second server cluster can perform data comparison based on the Merkle tree.
[0083] See S106 above, which will not be repeated here.
[0084] In the embodiments of this specification, the target server of the first server cluster calculates and determines the Merkle tree corresponding to the original data based on the original data in the first server cluster. Furthermore, it allocates the target bandwidth resources required to execute the replication task based on the bandwidth resources of the first server cluster. The replication task is to copy the Merkle tree corresponding to the original data to the second server cluster, so that the second server cluster can perform a data comparison task based on the Merkle tree, thereby achieving data synchronization between the first and second server clusters. Since cross-cluster replication tasks consume the bandwidth resources of the server clusters, this embodiment of the specification achieves bandwidth management by reasonably configuring the bandwidth resources consumed by the replication task. Furthermore, by copying the Merkle tree of the original data instead of the original data, the amount of data copied across clusters is significantly reduced, thereby achieving data comparison and synchronization between the two server clusters in a low-bandwidth, low-traffic, and high-time-efficiency manner.
[0085] In one embodiment, such as Figure 6 The illustration shows a data comparison method proposed in an embodiment of this specification. This method can be implemented using a computer program and can run on a data comparison device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application.
[0086] Specifically, the data comparison method includes:
[0087] S302, The first target server of the first server cluster obtains the Merkle tree corresponding to the original data in the first server cluster.
[0088] See S102 above, which will not be repeated here.
[0089] S304. The first target server of the first server cluster allocates the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster based on the bandwidth resources of the first server cluster.
[0090] See S104 above, which will not be repeated here.
[0091] S306. The first target server of the first server cluster replicates the Merkle tree to the second server cluster.
[0092] See S106 above, which will not be repeated here.
[0093] S308, The second target server of the second server cluster receives the Merkle tree of the first server cluster.
[0094] The second target server 3021 of the second server cluster 302 receives a Merkle tree sent by the first target server 3011 of the first server cluster 301, which is obtained based on the original data of the first target server 3011.
[0095] S310, the second target server of the second server cluster compares the Merkle tree of the second server cluster with the Merkle tree of the first server cluster to obtain the data comparison result.
[0096] The second target server 3021 slices the original data of the second target server 3021 according to the slicing granularity of the first server cluster, and performs hash calculation to obtain the Merkle tree corresponding to the original data of the second target server 3021.
[0097] After receiving the Merkle tree sent by the first target server 3011, the second target server 3021 obtains the root nodes corresponding to the Merkle trees of the first target server 3011 and the second target server 3021 respectively, and compares the values of the two root nodes.
[0098] If the values of the two root nodes are the same, a data comparison result is obtained, which indicates that the data between the first server cluster 301 and the second target server 302 in the second server cluster 302 is the same.
[0099] If the values of the two root nodes are different, it indicates that there are differing leaf nodes between the Merkle tree of the first server cluster 301 and the Merkle tree of the second target server 302. In other words, there is data difference between the first server cluster 301 and the second target server 302. Further, the traversal continues along the leaf nodes of the Merkle trees of the first server cluster 301 and the second target server 302 until a target leaf node with a different hash value is identified. Then, based on the association between the Merkle tree and the slice number, the slice number corresponding to the target leaf node is determined. During this process, for leaf nodes with the same hash value, it can be determined that the data corresponding to that leaf node is the same, and there is no need to focus on leaf nodes with the same hash value, thus finding the differing target leaf node more quickly. The data comparison result includes at least one differing target leaf node between the Merkle tree of the first server cluster 301 and the Merkle tree of the second target server 302, and the slice number corresponding to the target leaf node.
[0100] As the volume of raw data on the server clusters grows increasingly large, identifying discrepancies between two server clusters containing vast amounts of data requires reading almost all the data from both clusters. Synchronizing data between the two server clusters in this way would consume significant computing resources and time, resulting in extremely low efficiency. In this embodiment, by comparing the original data using a Merkle tree, discrepancies between the two server clusters can be identified, avoiding the massive data reading required to determine these discrepancies and thus effectively improving data synchronization efficiency.
[0101] S312, The second target server of the second server cluster sends the data comparison results to the first server cluster.
[0102] The second target server 3021 of the second server cluster 302 sends a data comparison result to the first server cluster 301. The data comparison result is that the data between the first server cluster 301 and the second target server 302 in the second server cluster 302 is the same, or that there is at least one difference between the Merkle tree of the first server cluster 301 and the Merkle tree of the second target server 302, and the slice number corresponding to the target leaf node.
[0103] S314. The first target server of the first server cluster receives the data comparison results of the second server cluster.
[0104] The first target server 3011 receives the data comparison results from the second server cluster 302.
[0105] S316. The first target server of the first server cluster sends at least one target slice data to the second server cluster based on the data comparison results.
[0106] If the first target server 3011 determines, based on the data comparison results, that the data between the first server cluster 301 and the second target server 302 in the second server cluster 302 is the same, then it will not send data to the second target server 3021.
[0107] If the data comparison result shows at least one difference between the target leaf node and the Merkle tree of the first server cluster 301 and the Merkle tree of the second target server 302, and the slice number corresponding to the target leaf node, then the first target server 3011 determines the target slice data corresponding to at least one slice number corresponding to the target leaf node according to the mapping table of slice data and slice number, and the slice number corresponding to the target leaf node, and sends the target slice data to the second target server 3021.
[0108] S318. The second target server of the second server cluster receives at least one target slice data from the first server cluster and completes data synchronization.
[0109] The second target server 3021 receives at least one target slice data and updates the original data included in the second target server 3021 based on the at least one target slice data, thereby completing data synchronization.
[0110] In the embodiments of this specification, the target server of the first server cluster calculates and determines the Merkle tree corresponding to the original data based on the original data in the first server cluster. Furthermore, it allocates the target bandwidth resources required to execute the replication task based on the bandwidth resources of the first server cluster. The replication task is to copy the Merkle tree corresponding to the original data to the second server cluster, so that the second server cluster can perform a data comparison task based on the Merkle tree, thereby achieving data synchronization between the first and second server clusters. Since cross-cluster replication tasks consume the bandwidth resources of the server clusters, this embodiment of the specification achieves bandwidth management by reasonably configuring the bandwidth resources consumed by the replication task. Furthermore, by copying the Merkle tree of the original data instead of the original data, the amount of data copied across clusters is significantly reduced, thereby achieving data comparison and synchronization between the two server clusters in a low-bandwidth, low-traffic, and high-time-efficiency manner.
[0111] The following are embodiments of the apparatus described in this specification, which can be used to execute the embodiments of the methods described in this specification. For details not disclosed in the apparatus embodiments of this specification, please refer to the embodiments of the methods described in this specification.
[0112] Please see Figure 7 This diagram illustrates the structure of a data comparison device provided in an exemplary embodiment of this specification. The data comparison device can be implemented as all or part of a device through software, hardware, or a combination of both. The data comparison device includes a data acquisition module 701, a resource allocation module 702, and a data copying module 703.
[0113] Data acquisition module 701 is used to acquire the Merkle tree corresponding to the original data in the first server cluster;
[0114] The resource allocation module 702 is used to allocate the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster based on the bandwidth resources of the first server cluster.
[0115] The data replication module 703 is used to replicate the Merkle tree to the second server cluster according to the target bandwidth resources, so that the second server cluster can perform data comparison based on the Merkle tree.
[0116] In one embodiment, the resource allocation module 702 includes:
[0117] A preset resource unit is used to determine the preset bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster, based on the Merkle tree.
[0118] The resource allocation unit is used to allocate target bandwidth resources to the replication task based on the bandwidth resources of the first server cluster and the preset bandwidth resources.
[0119] In one embodiment, the preset resource unit includes:
[0120] The preset subunit is used to determine the preset bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster based on the amount of data in the Merkle tree.
[0121] In one embodiment, the preset subunit is specifically used for:
[0122] Based on the data volume of the Merkle tree and the data throughput of a single process, a preset bandwidth resource is determined for the replication task of copying the Merkle tree to the second server cluster. The preset bandwidth resource includes the number of processes required for the replication task.
[0123] In one embodiment, allocating resource units includes:
[0124] The allocation subunit is used to allocate target bandwidth resources to the replication task based on the bandwidth resources of the first server cluster, the preset bandwidth resources, and the priority of the replication task.
[0125] In one embodiment, the resource allocation module 702 includes:
[0126] An allocation unit is used to allocate the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster, based on the bandwidth resource usage of the first server cluster.
[0127] In one embodiment, the data acquisition module 701 includes:
[0128] A data slicing unit is used to slice the original data to obtain at least one slice of data corresponding to the original data, and a slice number corresponding to each of the at least one slice of data.
[0129] A hash calculation unit is used to determine the Merkle tree corresponding to the at least one slice data based on the at least one slice data, wherein the Merkle tree includes the correspondence between the Merkle tree and at least one slice number.
[0130] In one embodiment, the data acquisition module 701 includes:
[0131] The preset acquisition unit is used to acquire the Merkle tree corresponding to the original data in the first server cluster when a preset condition is detected. The preset condition includes at least one of the following: the metadata table including the information of the first server cluster and the information of the second server cluster has changed; a periodic instruction to execute the replication task has been received; or a target instruction sent by the user to execute the replication task has been received.
[0132] In the embodiments of this specification, the target server of the first server cluster calculates and determines the Merkle tree corresponding to the original data based on the original data in the first server cluster. Furthermore, it allocates the target bandwidth resources required to execute the replication task based on the bandwidth resources of the first server cluster. The replication task is to copy the Merkle tree corresponding to the original data to the second server cluster, so that the second server cluster can perform a data comparison task based on the Merkle tree, thereby achieving data synchronization between the first and second server clusters. Since cross-cluster replication tasks consume the bandwidth resources of the server clusters, this embodiment of the specification achieves bandwidth management by reasonably configuring the bandwidth resources consumed by the replication task. Furthermore, by copying the Merkle tree of the original data instead of the original data, the amount of data copied across clusters is significantly reduced, thereby achieving data comparison and synchronization between the two server clusters in a low-bandwidth, low-traffic, and high-time-efficiency manner.
[0133] It should be noted that the data comparison device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the data comparison method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the data comparison device and the data comparison method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.
[0134] The example numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the examples.
[0135] This specification also provides a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figure 1 - Figure 6 The data comparison method described in the illustrated embodiment can be found in the following document for a detailed execution process: Figure 1 - Figure 6 The specific details of the illustrated embodiments will not be elaborated here.
[0136] This specification also provides a computer program product that stores at least one instruction, said at least one instruction being loaded and executed by the processor as described above. Figure 1 - Figure 6 The data comparison method described in the illustrated embodiment can be found in the following document for a detailed execution process: Figure 1 - Figure 6 The specific details of the illustrated embodiments will not be elaborated here.
[0137] Please see Figure 8 This document provides a schematic diagram of the structure of an electronic device as an embodiment of the present specification. Figure 8 As shown, the electronic device 800 may include: at least one processor 801, at least one network interface 804, a user interface 803, a memory 805, and at least one communication bus 802.
[0138] The communication bus 802 is used to enable communication between these components.
[0139] The user interface 803 may include a display screen and a camera. Optionally, the user interface 803 may also include a standard wired interface and a wireless interface.
[0140] The network interface 804 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).
[0141] The processor 801 may include one or more processing cores. The processor 801 connects to various parts of the server 800 via various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 805, and by calling data stored in the memory 805. Optionally, the processor 801 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 801 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 801.
[0142] The memory 805 may include random access memory (RAM) or read-only memory. Optionally, the memory 805 may include a non-transitory computer-readable storage medium. The memory 805 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 805 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 805 may also be at least one storage device located remotely from the aforementioned processor 801. Figure 8 As shown, the memory 805, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a data comparison application.
[0143] exist Figure 8In the illustrated electronic device 800, the user interface 803 is mainly used to provide an input interface for the user and to acquire user input data; while the processor 801 can be used to call the data comparison application stored in the memory 805 and specifically perform the following operations:
[0144] Obtain the Merkle tree corresponding to the original data in the first server cluster;
[0145] Based on the bandwidth resources of the first server cluster, allocate the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster;
[0146] The Merkle tree is replicated to the second server cluster according to the target bandwidth resources, so that the second server cluster can perform data comparison based on the Merkle tree.
[0147] In one embodiment, processor 801 executes the process of allocating the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster based on the bandwidth resources of the first server cluster, specifically by:
[0148] Based on the Merkle tree, determine the preset bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster;
[0149] Based on the bandwidth resources of the first server cluster and the preset bandwidth resources, the target bandwidth resources are allocated to the replication task.
[0150] In one embodiment, processor 801 executes the preset bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster based on the Merkle tree, specifically by:
[0151] Based on the amount of data in the Merkle tree, determine the preset bandwidth resources required for the replication task of copying the Merkle tree to the second server cluster.
[0152] In one embodiment, processor 801 executes the step of determining the preset bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster based on the amount of data in the Merkle tree, specifically by:
[0153] Based on the data volume of the Merkle tree and the data throughput of a single process, a preset bandwidth resource is determined for the replication task of copying the Merkle tree to the second server cluster. The preset bandwidth resource includes the number of processes required for the replication task.
[0154] In one embodiment, the processor 801 performs the step of allocating target bandwidth resources to the replication task based on the bandwidth resources of the first server cluster and the preset bandwidth resources, specifically by:
[0155] Based on the bandwidth resources of the first server cluster, the preset bandwidth resources, and the priority of the replication task, target bandwidth resources are allocated to the replication task.
[0156] In one embodiment, processor 801 executes the process of allocating the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster based on the bandwidth resources of the first server cluster, specifically by:
[0157] Based on the bandwidth resource usage of the first server cluster, the required target bandwidth resources are allocated for the replication task of replicating the Merkle tree to the second server cluster.
[0158] In one embodiment, processor 801 executes the step of obtaining the Merkle tree corresponding to the original data in the first server cluster, specifically by:
[0159] The original data is sliced to obtain at least one slice of data corresponding to the original data, and the slice number corresponding to each of the at least one slice of data;
[0160] Based on the at least one slice data, a Merkle tree corresponding to the at least one slice data is determined, wherein the Merkle tree includes the correspondence between the Merkle tree and at least one slice number.
[0161] In one embodiment, processor 801 executes the step of obtaining the Merkle tree corresponding to the original data in the first server cluster, specifically by:
[0162] When a preset condition is detected, the Merkle tree corresponding to the original data in the first server cluster is obtained. The preset condition includes at least one of the following: the metadata table containing information of the first server cluster and information of the second server cluster changes; a periodic instruction to execute the replication task is received; or a target instruction to execute the replication task is received from the user.
[0163] In the embodiments of this specification, the target server of the first server cluster calculates and determines the Merkle tree corresponding to the original data based on the original data in the first server cluster. Furthermore, it allocates the target bandwidth resources required to execute the replication task based on the bandwidth resources of the first server cluster. The replication task is to copy the Merkle tree corresponding to the original data to the second server cluster, so that the second server cluster can perform a data comparison task based on the Merkle tree, thereby achieving data synchronization between the first and second server clusters. Since cross-cluster replication tasks consume the bandwidth resources of the server clusters, this embodiment of the specification achieves bandwidth management by reasonably configuring the bandwidth resources consumed by the replication task. Furthermore, by copying the Merkle tree of the original data instead of the original data, the amount of data copied across clusters is significantly reduced, thereby achieving data comparison and synchronization between the two server clusters in a low-bandwidth, low-traffic, and high-time-efficiency manner.
[0164] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.
[0165] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.
Claims
1. A data comparison method, applicable to a target server in a first server cluster, the method comprising: Obtain the Merkle tree corresponding to the original data in the first server cluster; Based on the bandwidth resources of the first server cluster, the required target bandwidth resources are allocated for the replication task of replicating the Merkle tree to the second server cluster; wherein, the replication task includes a planned subtask, a replication subtask, and a static subtask, the planned subtask is used to evenly distribute the Merkle tree among the processes of the replication subtask according to the data size, the replication subtask is used to replicate the Merkle tree from the first server cluster to the second server cluster, the static subtask is used to count whether the process has been successfully replicated based on the replication result of the process, and the number of processes represents the preset bandwidth resources required by the replication task; The Merkle tree is copied to the second server cluster according to the target bandwidth resources, so that the second server cluster can perform data comparison based on the Merkle tree; The step of allocating the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster based on the bandwidth resources of the first server cluster includes: Based on the Merkle tree, determine the preset bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster; Based on the bandwidth resources of the first server cluster and the preset bandwidth resources, the target bandwidth resources are allocated to the replication task.
2. The data comparison method according to claim 1, wherein determining the preset bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster based on the Merkle tree includes: Based on the amount of data in the Merkle tree, determine the preset bandwidth resources required for the replication task of copying the Merkle tree to the second server cluster.
3. The data comparison method according to claim 2, wherein determining the preset bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster based on the data volume of the Merkle tree includes: Based on the data volume of the Merkle tree and the data throughput of a single process, a preset bandwidth resource is determined for the replication task of copying the Merkle tree to the second server cluster. The preset bandwidth resource includes the number of processes required for the replication task.
4. The data comparison method according to claim 1, wherein allocating target bandwidth resources to the replication task based on the bandwidth resources of the first server cluster and the preset bandwidth resources includes: Based on the bandwidth resources of the first server cluster, the preset bandwidth resources, and the priority of the replication task, the target bandwidth resources are allocated to the replication task.
5. The data comparison method according to claim 1, wherein allocating the required target bandwidth resources for the replication task of replicating the Merkle tree to the second server cluster based on the bandwidth resources of the first server cluster includes: Based on the bandwidth resource usage of the first server cluster, the required target bandwidth resources are allocated for the replication task of replicating the Merkle tree to the second server cluster.
6. The data comparison method according to claim 1, wherein obtaining the Merkle tree corresponding to the original data in the first server cluster includes: The original data is sliced to obtain at least one slice of data corresponding to the original data, and the slice number corresponding to each of the at least one slice of data; Based on the at least one slice data, a Merkle tree corresponding to the at least one slice data is determined, wherein the Merkle tree includes the correspondence between the Merkle tree and at least one slice number.
7. The data comparison method according to claim 1, wherein obtaining the Merkle tree corresponding to the original data in the first server cluster includes: When a preset condition is detected, the Merkle tree corresponding to the original data in the first server cluster is obtained. The preset condition includes at least one of the following: the metadata table containing information of the first server cluster and information of the second server cluster changes; a periodic instruction to execute the replication task is received; or a target instruction to execute the replication task is received from the user.
8. A data comparison device, the device comprising: The data acquisition module is used to acquire the Merkle tree corresponding to the raw data in the first server cluster; The resource allocation module is used to allocate the required target bandwidth resources for the replication task of copying the Merkle tree to the second server cluster based on the bandwidth resources of the first server cluster. The replication task includes a planned subtask, a replication subtask, and a static subtask. The planned subtask distributes the Merkle tree among the processes of the replication subtask according to its data size. The replication subtask copies the Merkle tree from the first server cluster to the second server cluster. The static subtask counts whether the replication process was successful based on the replication results. The number of processes represents the preset bandwidth resources required by the replication task. The data replication module is used to replicate the Merkle tree to the second server cluster according to the target bandwidth resources, so that the second server cluster can perform data comparison based on the Merkle tree; The resource allocation module includes: A preset resource unit is used to determine the preset bandwidth resources required for the replication task of replicating the Merkle tree to the second server cluster, based on the Merkle tree. The resource allocation unit is used to allocate target bandwidth resources to the replication task based on the bandwidth resources of the first server cluster and the preset bandwidth resources.
9. A computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method steps of any one of claims 1 to 7.
10. A computer program product storing a plurality of instructions adapted for loading by a processor and executing the method steps of any one of claims 1 to 7.
11. An electronic device, comprising: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed the method steps as claimed in any one of claims 1 to 7.
Citation Information
Patent Citations
Data synchronization method and device, storage medium and computer equipment
CN111917834A
Digital twin information synchronization method and system, readable storage medium and equipment
CN112711631A
Resource allocation method and device, electronic equipment and storage medium
CN115017030A
