Data synchronization method, device, storage medium and computer equipment

Abstract

The embodiment of the present application discloses a data synchronization method, device, storage medium and computer equipment; the embodiment of the present application is related to file storage and database synchronization of cloud storage, and the embodiment of the present application can obtain a first hash tree of a data set on a first server, and obtain a second hash tree of a target data set on a second server to be synchronized; start traversal comparison based on the root nodes of the first hash tree and the second hash tree, determine the target identifier corresponding to the target leaf node with differences; determine the first data and the second data corresponding to the target identifier in the data set and the target data set; synchronize data according to the difference between the version information of the first data and the second data. The present application can effectively improve the efficiency of data synchronization.

Description

Technical Field

[0001] The present application relates to the field of data storage, and in particular to a data synchronization method, device, storage medium and computer equipment. Background Art

[0002] In the process of distributed data storage, multiple copies can be set for a piece of data in the data set, and different copies can be stored in different servers to ensure the reliability and availability of the data. Since the modification and reading and writing of data are uninterrupted, but some servers are not always available, the copy value of the data in some servers is not the latest data value of this data.

[0003] In the process of research and practice of the prior art, the inventor of the present application found that a large amount of data will be read for comparison in the process of synchronizing copies in multiple servers. Due to the huge amount of data in the data set, the synchronization time is often very slow, resulting in low data synchronization efficiency. Summary of the invention

[0004] The embodiments of the present application provide a data synchronization method, apparatus, storage medium and computer equipment, which can improve data synchronization efficiency.

[0005] The present application provides a data synchronization method, including:

[0006] Obtain a first hash tree of a data set on a first server, and obtain a second hash tree of a target data set on a second server to be synchronized;

[0007] Starting traversal and comparison based on the root nodes of the first hash tree and the second hash tree, determining target identifiers corresponding to target leaf nodes with differences;

[0008] Determine first data and second data corresponding to the target identifier in the data set and the target data set;

[0009] Data synchronization is performed according to the difference between the version information of the first data and the second data.

[0010] Accordingly, an embodiment of the present application provides a data synchronization device, including:

[0011] An acquisition module, used to acquire a first hash tree of a data set on a first server, and to acquire a second hash tree of a target data set on a second server to be synchronized;

[0012] A comparison module, configured to start traversal and comparison based on the root nodes of the first hash tree and the second hash tree, and determine a target identifier corresponding to a target leaf node with a difference;

[0013] A determination module, configured to determine first data and second data corresponding to the target identifier in the data set and the target data set;

[0014] A synchronization module is used to synchronize data according to the difference between the version information of the first data and the second data.

[0015] In some embodiments of the present application, the acquisition module includes a data submodule, a setting submodule and a loop submodule, wherein:

[0016] A data submodule, used for performing hash calculation on each data in the data set on the first server to obtain a hash value of each data;

[0017] Setting a submodule, used to set the hash value of each data on the first server as the value of the leaf node, and performing hash calculation on the sum of the values ​​of the leaf nodes to obtain the value of the upper node;

[0018] The loop submodule is used to loop through the hash calculation of the sum of the values ​​of the upper nodes to obtain the value of a new upper node until the upper node becomes the root node, so as to obtain the first hash tree of the data set.

[0019] In some embodiments of the present application, the data set on the first server includes multiple data, each data includes an identifier, a data value and version information, and the data submodule is specifically used to:

[0020] A hash calculation is performed on each data including an identifier, a data value and a version information in the data set on the first server to obtain a hash value of each data.

[0021] In some embodiments of the present application, the comparison module includes a setting submodule, a circulation submodule and an acquisition submodule, wherein:

[0022] A submodule is set, which is used to set subnodes with different hash values ​​among the subnodes of the root nodes on the first hash tree and the second hash tree as candidate nodes;

[0023] A loop submodule, used for looping and replacing the child nodes with different hash values ​​among the child nodes of the candidate nodes on the first hash tree and the second hash tree with new candidate nodes until the candidate node becomes a leaf node;

[0024] The acquisition submodule is used to determine the candidate node at the end of the loop as the target leaf node and obtain the target identifier corresponding to the target leaf node.

[0025] In some embodiments of the present application, the data synchronization device further includes:

[0026] An identification module, configured to start traversal and comparison based on the root nodes of the first hash tree and the second hash tree, and determine a to-be-added identification corresponding to a target leaf node;

[0027] A data module, used to determine the target data corresponding to the to-be-added identifier in the data set;

[0028] The saving module is used to save the target data in the target data set to perform data synchronization, and update the second hash tree based on the synchronized target data set.

[0029] In some embodiments of the present application, the synchronization module is specifically used to:

[0030] When the first version information of the first data is greater than the second version information of the second data, the second data in the target data set is replaced with the first data to perform data synchronization.

[0031] In some embodiments of the present application, the synchronization module includes a comparison submodule and a replacement submodule, wherein:

[0032] a comparison submodule, configured to compare whether a hash value of the first data value is greater than a hash value of the second data value when the first version information of the first data is equal to the second version information of the second data and the first data value of the first data and the second data value of the second data are different;

[0033] A replacement submodule is used to replace the second data in the target data set with the first data to perform data synchronization when the hash value of the first data value is greater than the hash value of the second data value.

[0034] In some embodiments of the present application, the replacement submodule is specifically used for:

[0035] When the hash value of the first data value is greater than the hash value of the second data value, obtaining a storage address corresponding to the target identifier in the target data set;

[0036] The first data is written into a storage address corresponding to the target identifier in the target data set to replace the second data in the target data set.

[0037] In some embodiments of the present application, the data synchronization device further includes:

[0038] a target module, configured to determine a target server adjacent to the second server based on the sequence information;

[0039] An updating module, configured to update the second server to a new first server, and update the adjacent server to a new second server;

[0040] The re-module is used to re-execute the process of obtaining the first hash tree of the data set on the first server and obtaining the second hash tree of the target data set on the second server to be synchronized.

[0041] Correspondingly, an embodiment of the present application further provides a storage medium, which stores a computer program, and the computer program is suitable for loading by a processor to execute any data synchronization method provided in the embodiment of the present application.

[0042] Correspondingly, an embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the data synchronization methods provided in the embodiment of the present application is implemented.

[0043] The embodiment of the present application obtains the first hash tree of the data set on the first server, and obtains the second hash tree of the target data set on the second server to be synchronized; starts traversal and comparison based on the root nodes of the first hash tree and the second hash tree, determines the target identifier corresponding to the target leaf node with differences; determines the first data and second data corresponding to the target identifier in the data set and the target data set; and performs data synchronization based on the difference between the version information of the first data and the second data. By comparing the first hash tree and the second hash tree corresponding to the data set and the target data set, the target identifier with differences in the data set and the target data set can be quickly determined, and then based on this target identifier, the corresponding target data set is synchronized, which greatly improves the efficiency of data synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 is a scenario diagram of a data synchronization system provided in an embodiment of the present application;

[0046] Figure 2 is another scenario schematic diagram of the data synchronization system provided in an embodiment of the present application;

[0047] Figure 3 is another scenario schematic diagram of the data synchronization system provided in an embodiment of the present application;

[0048] Figure 4 It is a flowchart of a data synchronization method provided in an embodiment of the present application;

[0049] Figure 5is an example diagram of a first hash tree generated by the data synchronization method provided in an embodiment of the present application;

[0050] Figure 6 is another flowchart of the data synchronization method provided in an embodiment of the present application;

[0051] Figure 7 This is an example diagram of a server to be synchronized in the data synchronization method provided in an embodiment of the present application;

[0052] Figure 8 This is a data synchronization example diagram of the data synchronization method provided in an embodiment of the present application;

[0053] Fig. 9 is another data synchronization example diagram of the data synchronization method provided in an embodiment of the present application;

[0054] Fig.10 is another data synchronization example diagram of the data synchronization method provided in an embodiment of the present application;

[0055] Fig.11 is an example diagram of a hash tree in a data synchronization method provided in an embodiment of the present application;

[0056] Fig.12 This is an example diagram of multi-server data synchronization of the data synchronization method provided in an embodiment of the present application;

[0057] Fig.13 is a structural diagram of a data synchronization device provided in an embodiment of the present application;

[0058] Fig.14 is another structural schematic diagram of the data synchronization device provided in an embodiment of the present application;

[0059] Fig.15 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described in this application are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0061] Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0062] At present, the storage method of the storage system is: create a logical volume, and when creating a logical volume, allocate physical storage space for each logical volume. The physical storage space may be composed of disks of a storage device or several storage devices. The client stores data on a logical volume, that is, stores the data on the file system. The file system divides the data into many parts, each of which is an object. The object contains not only data but also additional information such as data identification (ID, ID entity). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.

[0063] The process of the storage system allocating physical storage space to a logical volume is as follows: based on the estimated capacity of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of independent redundant disk arrays (RAID, Redundant Array of Independent Disks), the physical storage space is pre-divided into stripes. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0064] In short, a database can be seen as an electronic filing cabinet - a place where electronic files are stored, and users can add, query, update, delete, and other operations on the data in the files. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application.

[0065] A database management system (DBMS) is a computer software system designed for managing databases. It generally has basic functions such as storage, retrieval, security, and backup. Database management systems can be classified according to the database model they support, such as relational, XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters, mobile phones; or according to the query language used, such as SQL (Structured Query Language), XQuery; or according to performance focus, such as maximum scale, maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMS can cross categories, for example, supporting multiple query languages ​​at the same time.

[0066] The embodiments of the present application provide a data synchronization method, apparatus, storage medium and computer equipment. Specifically, the embodiments of the present application can be integrated into a data synchronization system.

[0067] The data synchronization system can be integrated in a computer device, which can include a terminal or a server, etc. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to these. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0068] like Figure 1 As shown, in an embodiment of the present application, a data synchronization system may include at least a data synchronization server, a first server, and a second server. The data synchronization server may synchronize the data on the first server and the second server. Specifically, the data synchronization system may obtain a first hash tree of the data set on the first server, and obtain a second hash tree of the target data set on the second server to be synchronized, and then start traversal and comparison based on the root nodes of the first hash tree and the second hash tree, determine the target identifier corresponding to the target leaf node with a difference, and then determine the first data and the second data corresponding to the target identifier in the data set and the target data set, and finally perform data synchronization according to the difference between the version information of the first data and the second data.

[0069] In order to perform data synchronization more flexibly, the data synchronization system can also be integrated on the data synchronization server and the data synchronization terminal. Relevant personnel can set key information related to data synchronization (such as the server to be synchronized, etc.) through the terminal and send it to the data synchronization server, so that the data synchronization server can perform data synchronization based on the key information.

[0070] In actual operation, there may be multiple servers to be synchronized. In this case, the data synchronization system needs to synchronize the data of the servers to be synchronized one by one. For example, Figure 2 As shown, when there are 5 servers to be synchronized, data synchronization can be performed between server 1 and server 2, server 2 and server 3, server 3 and server 4, server 4 and server 5, and server 5 and server 1 in turn, and the next data synchronization is performed after the previous data synchronization is completed (for example, after the data synchronization between server 2 and server 3 is completed, the next data synchronization will continue with the synchronized server 3 and the unsynchronized server 4) until all 5 servers complete a round of data synchronization.

[0071] See also Figure 3 In some embodiments, the servers to be synchronized may include server 1, server 2, server 3, server 4 and server 5, and the data in server 1 may include<k1,v:”v1”,seq:5> and<k2,v:”v2”,seq:3> , the data in server 2 may include<k1,v:”v2”,seq:4> and<k2,v:”v2”,seq:3> , the data in server 3 may include<k1,v:”v3”,seq:6> and<k2,v:”v2”,seq:3> , the data in server 4 may include<k1,v:”v1”,seq:5> , the data in server 5 may include<k1,v:”v1”,seq:5> When performing data synchronization, you can first synchronize server 2 through server 1, then synchronize server 3 through server 2, then synchronize server 4 through server 3, then synchronize server 5 through server 4, and finally synchronize server 1 through server 5, thus completing the data synchronization between the five servers.

[0072] It should be noted that Figure 1 , Figure 2 as well as Figure 3The scenario diagram of the data synchronization system shown is merely an example. The data synchronization system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can know that with the evolution of data synchronization devices and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0073] In this embodiment, a data synchronization method is described in detail. The data synchronization method can be integrated in a data synchronization server, such as Figure 4 As shown, Figure 4 : is a flow chart of a data synchronization method provided in an embodiment of the present application. The data synchronization method may include:

[0074] 101. Obtain a first hash tree of a data set on a first server, and obtain a second hash tree of a target data set on a second server to be synchronized.

[0075] In the present application, data synchronization may include operations that can remove differences between data stored in different servers. For data in different servers, due to limitations on data update methods and other reasons, the data values ​​in the data sets in different servers may not be consistent. The process of unifying the data values ​​of data in different data sets into the same data values ​​is the process of data synchronization.

[0076] The first server may include a server that provides reference data for the server to be synchronized, and the second server may be the server to be synchronized.

[0077] The first server and the second server may be computer devices including data storage and processing capabilities. For example, the first server may store a data set, and the second server may store a target data set. During data synchronization, the second server may use the data set in the first server as a reference to synchronize data with the target data set in the second server. The first server and the second server may be physical servers or cloud servers, etc.

[0078] The data set and the target data set can contain massive amounts of data, and one of them can be a copy of the other. Therefore, the data identifiers in the data set and the target data set are roughly the same. Since the copy data may not be updated in time when the data is updated, the data values ​​of the same data identifier in the data set and the target data set may be different. The technical effect achieved by the present application is to eliminate the difference in data between the data set and the target data set. When storing data, the data set and the target data set can be implemented based on distributed storage (key-value), where the key can be the keyword (or data identifier) ​​of the data and the value can be the data value of the data.

[0079] The first hash tree and the second hash tree can be a tree structure for storing hash values. The hash can include a function that maps data of any length into data of a fixed length. When the data changes, the hash value of the data obtained by the hash function will also change accordingly. Therefore, it is possible to quickly determine whether the data has changed by comparing the hash values. Each node in the hash tree can store a corresponding hash value and the hash value of each node can be determined based on the hash value of its child node. The hash value of the leaf node of the hash tree can be obtained by hashing the data in the data set. The first hash tree can be obtained based on the data in the data set, and the second hash tree can be obtained based on the data in the target data set. The hash tree is usually an ordered tree, so the order of the leaf nodes in the first hash tree and the second hash tree is fixed. In one embodiment, the leaf nodes can be sorted by key.

[0080] The first hash tree and the second hash tree may be stored on the first server and the second server, respectively. When data synchronization is required between the data set on the first server and the target data set on the second server, a request may be sent to the first server and the second server, and the returned first hash tree and the second hash tree may be received. The first hash tree and the second hash tree may be generated by a data synchronization server. When data synchronization is performed through the data synchronization server, the data synchronization server may also update the hash tree in a timely manner, thereby always ensuring the consistency between the hash tree and its corresponding data set, and so on.

[0081] For example, a first hash tree a2 of a data set a1 on a first server A is obtained, and a second hash tree b2 of a target data set b1 on a second server B to be synchronized is obtained.

[0082] In some embodiments of the present application, the step of “obtaining a first hash tree of a data set on a first server” may include:

[0083] (1) performing a hash calculation on each data in the data set on the first server to obtain a hash value of each data;

[0084] (2) setting the hash value of each data on the first server to the value of the leaf node, and performing hash calculation on the sum of the values ​​of the leaf nodes to obtain the value of the upper node;

[0085] (3) Perform hash calculation on the sum of the values ​​of the upper nodes in a loop to obtain the value of a new upper node, until the upper node becomes the root node, thereby obtaining the first hash tree of the data set.

[0086] Among them, hash calculation can be implemented based on a variety of hash algorithms, such as SHA-2 (a hash algorithm) or MD5 (a hash algorithm). After hash calculation is performed on each data, the hash value of each data can be obtained. The hash value length of these data is consistent, and these hash values ​​with consistent lengths are used as the values ​​(hash values) of the leaf nodes of the hash tree. The hash values ​​on the leaf nodes are set according to a fixed data identification order. See Figure 5 , the leaf nodes on the first hash tree store the hash values ​​of data p1, p2, p3, p4, p5, and p6 from left to right.

[0087] Next, a cyclic hash calculation is performed through the hash values ​​of the leaf nodes to obtain a complete hash tree. Specifically, the tree structure of the hash tree can be actually set, such as presetting the tree structure of the first hash tree as a binary tree, and merging the leaf nodes to obtain the upper nodes. The hash value of the upper node can be obtained by performing a hash calculation on the sum of the hash values ​​of the leaf nodes. For example, the leaf nodes corresponding to data p1 and p2, the leaf nodes corresponding to data p3 and p4, and the leaf nodes corresponding to data p5 and p6 are merged respectively to obtain upper nodes p12, p34, and p56. The value of p12 is obtained by performing a hash calculation on the sum of the hash values ​​of p1 and p2, the value of p34 is obtained by performing a hash calculation on the sum of the hash values ​​of p3 and p4, and the value of p56 is obtained by performing a hash calculation on the sum of the hash values ​​of p5 and p6.

[0088] Then the loop continues to merge the upper nodes according to the tree structure of the hash tree to obtain a new upper node, until the number of upper nodes is 1, that is, the upper node is the root node, and a complete hash tree is generated at this time. For example, the tree structure preset by the first hash tree is a binary tree, then the upper nodes p34 and p56 are merged to obtain a new upper node p1234, and the value of p1234 is obtained by hashing the sum of the hash values ​​of p12 and p34. Then, p1234 and p56 are merged to obtain a new upper node p0. Since p0 is the only upper node at this time, that is, p0 is the root node, such as Figure 5 As shown, a complete first hash tree a2 is generated at this time.

[0089] In some embodiments of the present application, the step of “performing a hash calculation on each data in the data set on the first server to obtain a hash value of each data” may include:

[0090] (1.1) Perform a hash calculation on each data including an identifier, a data value and a version information in the data set on the first server to obtain a hash value of each data.

[0091] The data in the data set may include an identifier, a data value, and version information. The identifier may include information that serves as a unique identifier. The data value may be the content corresponding to the identifier, such as a character, a picture, etc. The version information may be the version information of the data value in this piece of data. The data value of the data identifier may change (such as external modification of the data value, or automatic update of the data value, etc.), and each change in the data value may be marked by the version information. When performing a hash calculation on the data in the data set, the identifier, data value, and version information of the data may be integrated together as a data block, and a hash calculation may be performed on the data block to obtain the hash value of the data.

[0092] In addition, since the first hash tree and the second hash tree need to be traversed and compared, the preset tree structure of the first hash tree and the second hash tree, the hash algorithm used in the hash calculation, and the data identification order of the leaf nodes need to be consistent. The specific generation process of the second hash tree is consistent with that of the first hash tree, and the generation process of the first hash tree can be referred to, which will not be repeated here.

[0093] 102. Start traversal and comparison based on the root nodes of the first hash tree and the second hash tree to determine the target identifiers corresponding to the target leaf nodes with differences.

[0094] Among them, the target leaf node can be a leaf node with a hash value difference between the first hash tree and the second hash tree. Since the first hash tree and the second hash tree are constructed according to the same preset tree structure, the same hash algorithm and the same leaf node data identification order, the nodes on the first hash tree and the second hash tree can be compared accordingly, and the leaf nodes with differences can be determined through the corresponding comparison.

[0095] In one embodiment, after obtaining the first hash tree and the second hash tree, the root node value (root node hash value) of the first hash tree and the root node value of the second hash tree can be obtained respectively, and the two can be compared. If the two are different, it means that there are leaf nodes with different hash values ​​in the first hash tree and the second hash tree, and the traversal continues along the child nodes of the first hash tree and the second hash tree until the target leaf node with a hash value difference in the first hash tree and the second hash tree is determined, and then the target identifier corresponding to the target leaf node is determined. If the two are the same, it means that the data set is completely consistent with the target data set, that is, there is no need to synchronize the data of the target data set. In this process, for nodes with the same hash value, it can be determined that the data corresponding to their leaf nodes are the same, and there is no need to pay attention to the child nodes of the nodes with the same hash value, so as to find the target leaf nodes with differences more quickly.

[0096] Furthermore, with the development of technology, the amount of data in the data set on the server is getting larger and larger. To determine the data value corresponding to an identifier from a huge data set (to find different data from two huge data sets), it is necessary to read almost all the data in this data set. Since data synchronization needs to first determine the data with differences, that is, it is necessary to determine the data values ​​corresponding to all identifiers in the data set, then only determining the data value corresponding to an identifier from the first server and the second server requires reading almost all the data of the two data sets respectively. If data synchronization is performed in this way, the data synchronization process will consume a lot of computer resources and time, and the data synchronization efficiency is extremely low. The present application can determine the identifiers of data with differences in the data set by comparing hash trees, avoiding the reading of massive data when determining the difference data, thereby effectively improving the efficiency of data synchronization.

[0097] For example, the traversal and comparison are started based on the root nodes of the first hash tree a2 and the second hash tree b2 to determine the target identifier pp corresponding to the target leaf node with differences.

[0098] In some embodiments of the present application, the step of "starting traversal and comparison based on the root nodes of the first hash tree and the second hash tree to determine the target identifier corresponding to the target leaf node with differences" may include:

[0099] The child nodes with different hash values ​​among the child nodes of the root nodes on the first hash tree and the second hash tree are set as candidate nodes; the child nodes with different hash values ​​among the child nodes of the candidate nodes on the first hash tree and the second hash tree are replaced with new candidate nodes in a loop until the candidate node is a leaf node; the candidate node at the end of the loop is determined as a target leaf node, and a target identifier corresponding to the target leaf node is obtained.

[0100] When the values ​​(hash values) of the root nodes of the first hash tree and the second hash tree are different, each child node of the root nodes of the first hash tree and the second hash tree is compared respectively. For example, the child nodes of the root node of the first hash tree are ma and mb, and the child nodes of the root node of the second hash tree are na and nb. The hash values ​​of ma and na, and mb and nb can be compared respectively. If the hash values ​​of ma and na are the same, it can be indicated that the leaf nodes under this child node a in the first hash tree and the second hash tree are the same. When there are child nodes with different hash values, the leaf nodes with different hash values ​​are used as candidate nodes. For example, if the hash values ​​of mb and nb are different, the child node b is determined to be a new candidate node.

[0101] The loop replaces the child nodes with different hash values ​​in the child nodes of the candidate nodes on the first hash tree and the second hash tree with new candidate nodes, and determines whether the candidate node is a leaf node (that is, the candidate node has no child nodes). When the candidate node is a leaf node, the loop ends.

[0102] For example, the child nodes of candidate nodes mb and nb are obtained from the first hash tree and the second hash tree respectively, the child nodes of candidate node mb are mb1, mb2 and mb3, and the child nodes of candidate node nb are nb1, nb2 and nb3. The hash values ​​of mb1 and nb1, the hash values ​​of mb2 and nb2, and the hash values ​​of mb3 and nb3 are respectively determined to have different child nodes, that is, the candidate node is b2. After judgment, candidate node b2 also includes child node b21, then the child nodes of candidate nodes mb2 and nb2 are obtained from the first hash tree and the second hash tree respectively, the child node of candidate node mb2 is mb21, and the child node of candidate node nb2 is nb21. When the hash values ​​of mb21 and nb21 are different and mb21 and nb21 are both leaf nodes, the loop ends.

[0103] The candidate node that ends the loop is determined as the target leaf node, and the target identifier (target data identifier) ​​corresponding to the target leaf node is obtained. Here, the identifier of the leaf node can be consistent with the data identifier, and the identifier of the target leaf node is the target identifier; the identifier of the leaf node can also be inconsistent with the data identifier, and there can be a corresponding mapping relationship table between the two. The target identifier corresponding to the target leaf node can be obtained by searching the data identifier corresponding to the target leaf node identifier in the mapping relationship table. For example, the leaf node b21 that ends the loop can be determined as the target leaf node, and the target data identifier Q corresponding to b21 can be determined from the mapping relationship table, that is, the corresponding data of the first server and the second server on the target data identifier Q are different.

[0104] 103. Determine first data and second data corresponding to the target identifier in the data set and the target data set.

[0105] Among them, the first data may include data corresponding to the target identifier in the data set on the first server, and the second data may include data corresponding to the target identifier in the target data set on the second server. Specifically, data requests can be sent to the first server and the second server respectively to request the data corresponding to the target identifier, and then the first data and the second data returned by the first server and the second server can be received.

[0106] For example, first data q1 corresponding to the target identifier Q returned by the first server is received, and second data q2 corresponding to the target identifier Q returned by the second server is received.

[0107] 104. Perform data synchronization according to the difference between version information of the first data and the second data.

[0108] Among them, since the data in the data set may include an identifier, a data value and version information, the version information may record the number of times the data value is updated. The higher the version, the newer the data value.

[0109] The updated and unupdated data in the first data and the second data can be determined according to the version information, and then it can be determined whether the second data needs to be synchronized. If necessary, the second data in the target data set is replaced with the first data to achieve data synchronization.

[0110] In addition, the version information of the first data and the second data can be used to determine whether the first data needs to be synchronized. If necessary, the first data in the data set can be replaced by the second data to achieve data synchronization of the data set through the target data set, that is, to achieve two-way data synchronization between the data set and the target data set.

[0111] In some cases (such as when the first server has strong data processing capabilities and can support the smooth implementation of the data synchronization process), the method of this embodiment can also be performed on the server that stores the data set. At this time, the acquisition of relevant data (such as hash trees, data corresponding to target identifiers), the execution of data synchronization operations, etc. can be adjusted according to actual conditions, and these adjustments should still fall within the scope of the scheme of this embodiment.

[0112] In some embodiments, the step of “synchronizing data according to the difference between version information of the first data and the second data” may include:

[0113] (1) When the first version information of the first data is greater than the second version information of the second data, the second data in the target data set is replaced with the first data to perform data synchronization.

[0114] Among them, the version information can be in the form of characters, etc. As the data value is updated, the version information of the data value can become larger and larger. Therefore, among the two data values ​​of a data, the data value with a larger version information is the updated data value. Then, when the first version information of the first data is greater than the second version information of the second data, it can be indicated that the first data is the updated data, and the second data in the target data set needs to be synchronized. Specifically, the second data in the target data set can be replaced with the first data.

[0115] When the first version information of the first data is smaller than the second version information of the second data, it indicates that the second data is updated data, and there is no need to perform a data synchronization operation on the second data in the target data set.

[0116] For example, the first version information of the first data q1 is g5, and the second version information of the second data q2 is g4. By comparison, it can be determined that the first version information g5 is greater than the second version information g4. Then the second data in the target data set can be replaced with the first data q1 to complete data synchronization for the target data set.

[0117] In some embodiments of the present application, the step of “synchronizing data according to the difference between version information of the first data and the second data” may include:

[0118] (1.1) when the first version information of the first data is equal to the second version information of the second data, and the first data value of the first data and the second data value of the second data are different, comparing whether the hash value of the first data value is greater than the hash value of the second data value;

[0119] (1.2) When the hash value of the first data value is greater than the hash value of the second data value, the second data in the target data set is replaced with the first data to perform data synchronization.

[0120] Among them, due to the huge amount of data in the data set, in certain situations (such as network delays, code errors, or the generation mechanism of version information), the data values ​​may be different but the version information may be different (whether they are the same). When this happens, you can use preset rules to select one from different data values ​​as the updated data, and perform data synchronization based on the updated data. The preset rules can include multiple rules and can be flexibly set according to actual needs. For example, the data value that occupies a larger memory can be used as the updated data value. When the first data value occupies a larger memory, the first data value can be used as the updated data value, and the second data in the target data set can be replaced with the first data for data synchronization.

[0121] For another example, you can calculate the hash value of the data value and take the larger hash value as the updated data value. You can calculate the hash values ​​of the first data value and the second data value respectively. When the hash value of the first data value is greater than the hash value of the second data value, you can determine that the first data value is the updated data value. Then you can replace the second data in the target data set with the first data to perform data synchronization. Similarly, you can take the smaller hash value as the updated data value, and so on.

[0122] For example, the first version information of the first data q1 is g3, and the first data value is "ED", the second version information of the second data q2 is g3, and the second data value is "AFG", that is, the first version information g3 of the first data q1 is equal to the second version information g3 of the second data q2, and the first data value "ED" of the first data is different from the second data value "AFG" of the second data, then the hash values ​​of the first data value and the second data can be calculated respectively, which are H1 and H2. By comparison, it is determined that H1 is greater than H2, that is, the first data q1 is the updated data, and the second data in the target data set can be replaced by the first data q1.

[0123] In some embodiments of the present application, the step of “replacing the second data in the target data set with the first data” may include:

[0124] (2.1) Obtain the storage address corresponding to the target identifier in the target data set;

[0125] (2.2) Writing the first data into the storage address corresponding to the target identifier in the target data set to replace the second data in the target data set.

[0126] Among them, when replacing data, it is necessary to obtain the storage address of the second data in the target data set. When the second server stores the data in the target data set, it allocates storage space to each data according to the characteristics of the data. For example, storage space A is allocated to the second data. The storage address of storage space A in the second server can be 0x00000001. After determining the storage of the second data, the first data can be written to the storage address 0x00000001 of the second data, thereby realizing the replacement of the second data in the target data set.

[0127] In some embodiments of the present application, the data synchronization method may further include:

[0128] (3.1) Starting traversal and comparison based on the root nodes of the first hash tree and the second hash tree, determining the identifier to be added corresponding to the target leaf node;

[0129] (3.2) Determine the target data corresponding to the identifier to be added in the data set; save the target data in the target data set for data synchronization, and update the second hash tree based on the synchronized target data set.

[0130] Among them, when performing data synchronization, there may be some new data in the data set on the first server, that is, data that exists in the data set but does not exist in the target data set. In the first hash tree of the data set, the leaf node corresponding to the hash value of the new data is located at the rightmost side of the hash tree and will not affect the existing node structure. Similarly, traversal and comparison can be started based on the first hash data and the root node of the second hash tree. When a target leaf node is found that exists in the first hash but does not exist in the second hash tree, the data identifier corresponding to the target leaf node is determined, that is, the identifier to be added.

[0131] The target data corresponding to the identifier to be added is obtained from the data set, and then the obtained target data is saved in the target data set, so that data synchronization for the target data set can be achieved.

[0132] After completing data synchronization of the target data set, the second hash tree can be updated based on the synchronized target data set to match the data storage content of the second hash tree and the updated target data set, which helps to smoothly carry out subsequent data synchronization.

[0133] For example, start traversal and comparison based on the root nodes of the first hash tree a2 and the second hash tree b2, determine the identifier kk to be added corresponding to the target leaf node, determine the target data K corresponding to the identifier kk to be added in the data set, save the target data K in the target data set for data synchronization, and update the second hash tree b2 based on the synchronized target data set.

[0134] In addition, when comparing the data set and the target data set, there may be data to be deleted, that is, data that exists in the target data set but does not exist in the data set. In order to ensure that the first hash tree and the second hash tree can perform node comparison, the hash tree can be specially processed. For example, the data that has been deleted in the data set / target data set still retains the leaf node in the first / second hash tree, and the value of the leaf node is saved as a recognizable deletion keyword. This deletion keyword indicates that the data identifier corresponding to this leaf node does not exist in the data set. Then, when it is identified that the value of the leaf node in the first hash tree is the deletion keyword, and the value of this leaf node in the second hash tree is the hash value, this leaf node is determined to be the target leaf node, and the data identifier corresponding to the target leaf node is determined as the identifier to be deleted. Then, the data corresponding to the identifier to be deleted in the target data set can be deleted to complete data synchronization.

[0135] In some embodiments of the present application, the data synchronization method may further include:

[0136] (4.1) determining a target server adjacent to the second server based on the sequence information;

[0137] (4.2) The second server is updated to be a new first server, and the adjacent server is updated to be a new second server; and the first hash tree of the data set on the first server is obtained again, and the second hash tree of the target data set on the second server to be synchronized is obtained.

[0138] When data synchronization is performed, there may be multiple servers to be synchronized, and there may be preset sequence information between these servers. In order to achieve data synchronization between multiple servers, the following can be done: Figure 2 As shown, the data sets on two adjacent servers are synchronized in turn until a round of data synchronization is completed, that is, all servers serve as the first server and the second server, that is, the servers to be synchronized are connected to each other to form a closed loop.

[0139] Furthermore, after completing the data synchronization between the two servers, the target server adjacent to the second server can be determined through the sequence information of the servers to be synchronized. When multiple servers to be synchronized are connected end to end in a ring, the target server is the server adjacent to the second server except the first server. Then, the second server is updated to the new first server, that is, the updated server, and the target server is used as the new second server, that is, the server to be updated. Then, the process of obtaining the first hash tree and the second hash tree is re-executed to perform a new data synchronization until all servers have been synchronized as second servers to be synchronized.

[0140] In an embodiment of the present application, the target identifiers that differ between the data set and the target data set can be quickly and conveniently determined only by comparing the first hash tree and the second hash tree corresponding to the data set and the target data set, and then the target data set can be synchronized based on the target identifier, so that the data synchronization method described in this embodiment can significantly improve the data synchronization efficiency.

[0141] The method described in the above embodiment is further described in detail below with examples.

[0142] The embodiment of the present application takes multiple servers to be synchronized as an example, each server contains a copy of a master data set, that is, each copy is a data set, and the data volume of the data set is huge. The present application will realize data synchronization of the copies on each server through a data synchronization method, so that the data in each copy is the latest version of the data, wherein any data in the data set includes data identification, data value and version information. The present embodiment can be performed on a server, and the server can be a physical server or a cloud server, to introduce the data synchronization method.

[0143] The data in this embodiment can be stored in a distributed manner, and each data includes a data identifier (i.e., key), a data value (i.e., value), and version information of the data value. The version information can be the time point when the data value is changed, etc. The data synchronization method of this embodiment can be used in a distributed storage system, or a metadata storage system of an object storage, etc.

[0144] The overall flow chart of this embodiment can be found in Figure 6 ,in:

[0145] 201. The server determines a first server and a second server to be synchronized from a plurality of servers to be synchronized that contain sequence information.

[0146] For example, see Figure 7 , the server can determine the first server and the second server from server 1, server 2, server 3, server 4 and server 5, which are server 1 and server 2 respectively.

[0147] 202. The server obtains a first hash tree of a data set on a first server and a second hash tree of a target data set on a second server to be synchronized.

[0148] For example, the server obtains a first hash tree A1 of a data set on server 1 and a second hash tree A2 of a target data set on server 2 .

[0149] 203. The server sets subnodes with different hash values ​​among the subnodes of the root nodes on the first hash tree and the second hash tree as candidate nodes.

[0150] 204. The server cyclically replaces the child nodes with different hash values ​​among the child nodes of the candidate nodes on the first hash tree and the second hash tree with new candidate nodes until the candidate node becomes a leaf node.

[0151] 205. The server determines the candidate node at the end of the loop as the target leaf node, and obtains the target identifier corresponding to the target leaf node.

[0152] For example, determine that the target identifier corresponding to the target leaf node is k1.

[0153] Since the amount of data in each data set to be synchronized can reach tens of millions or even hundreds of millions, the resources required to read and compare each data in the data set separately are too high. In this embodiment, by comparing the hash trees of the data sets, the data with differences in the data sets can be quickly determined, thereby greatly improving the data synchronization efficiency.

[0154] 206. The server obtains first data and second data corresponding to the target identifier from the data set and the target data set respectively.

[0155] For example, the first data corresponding to the target identifier k1 can be determined from the data set on the server 1.<k1,v:”v1”,seq:5> , determine the second data corresponding to the target identifier k1 from the target data set on server 2<k1,v:”v2”,seq:4> .

[0156] 207. When the version information of the first data and the version information of the second data meet a preset requirement, the server replaces the second data in the target data set with the first data to perform data synchronization.

[0157] For example, when the first data is<k1,v:”v1”,seq:5> , the second data is<k1,v:”v2”,seq:4> , we can know that the version information seq:5 of the first data is greater than the version information seq:4 of the second data. Figure 8 As shown, the server device replaces the second data in server 2 with the first data<k1,v:”v1”,seq:5> .

[0158] For example, when the first data is<k1,v:”v3”,seq:6> , the second data is<k1,v:”v1”,seq:6> , the server can determine that the hash value of the first data value v:"v3" is greater than the hash value of the second data value v:"v1", such as Fig. 9 As shown, the server replaces the second data in server 4 (i.e., the target data set) with the first data<k1,v:”v3”,seq:6> .

[0159] For example, see Fig. 9 , the data to be added corresponding to the to-be-added identifier that exists in server 3 but does not exist in server 4<k2,v:”v2”,seq:3> , then the new data will be<k2,v:”v2”,seq:3> The data is saved in server 4.

[0160] 208. When data synchronization for the second server is completed, determine the adjacent server of the second server according to the sequence information, and update the second server to a new first server and the adjacent server to a new second server.

[0161] After completing step 208, the server may re-execute step 202 until each server to be synchronized has completed data synchronization as a second server, that is, data synchronization for multiple servers to be synchronized is completed.

[0162] For example, a new round of data synchronization can be performed based on server 2 and server 3. After completing the data synchronization for server 3, server 3 can be updated as the new first server, server 4 can be determined as the new second server, and a round of data synchronization can be performed based on server 3 and server 4. This process is performed in sequence until server 1 to server 5 have all completed data synchronization as the second server. Fig.10 As shown, after completing the data synchronization of server 2, server 3, server 4 and server 5 in turn, server 5 is used as the first server and server 1 is used as the second server to synchronize the data set of server 1, that is, to complete the data synchronization of multiple servers to be synchronized.

[0163] This embodiment can quickly and conveniently determine the target identifier that is different between the data set and the target data set simply by comparing the first hash tree and the second hash tree corresponding to the data set and the target data set, and then synchronize the target data set based on the target identifier, so that the data synchronization method of the present application can significantly improve the data synchronization efficiency.

[0164] The method described in the above embodiment is further described in detail below with examples.

[0165] See also Fig.11 , Fig.11 This is an example diagram of a hash tree required by the data synchronization method of the present application when determining target identifiers with differences. The hash values ​​of the leaf nodes of the hash tree are obtained by hashing the data in the data blocks, such as Fig.11 The hash value of the leaf node Hash 0-0 is the hash calculation of data b1 (hash(b1)), and then the nodes are merged in sequence until they are merged to the root node. When merging, the hash value of the merged parent node is the hash calculation sum of the hash values ​​of the merged child nodes. For example, the hash value of the parent node hash 0 is the hash calculation sum of the hash value of hash0-0 and the hash value of hash 0-1 (hash(hash 0-0+hash 0-1).

[0166] When performing data synchronization, after obtaining the first hash tree and the second hash tree, you can start from the root node and compare the hash values ​​of the corresponding nodes on the first hash tree and the second hash tree until you determine the leaf nodes with different hash values, and then determine the target identifier of the data corresponding to the leaf node. For example, the target identifier can be data b1, and then perform data synchronization based on the target identifier.

[0167] In the right Figure 3 When multiple servers to be synchronized are synchronized, the specific process can be seen in Fig.12First, synchronize the data of server 2 through server 1. After determining the target identifier k1 that is different between server 1 and server 2, compare the version information and find that the version information seq:5 of k1 in server 1 is greater than the version information seq:4 of k1 in server 2. Then, the data in server 1 is synchronized.<k1,v:”v1”,seq:5> Update to Server 2.

[0168] Then, server 2 synchronizes data with server 3. After determining that there is a difference in target identifier k1 between server 2 and server 3, the version information is compared and it is found that the version information seq:5 of k1 in server 2 is less than the version information seq:6 of k1 in server 3. In this case, no operation is performed on k1 in server 3.

[0169] Then, the data of server 4 is synchronized through server 3. After determining the target identifier k1 and the identifier k2 to be added that are different between server 3 and server 4, the version information is compared and it is found that the version information seq:6 of k1 in server 3 is equal to the version information seq:6 of k1 in server 4. Then, the hash value of data value v1 is smaller than the hash value of data value v3, and the data in server 3 is synchronized.<k1,v:”v3”,seq:6> Update to server 4, and the target data corresponding to the new identifier in server 3<k2,v:”v2”,seq:3> Updated to Server 4.

[0170] Then, the data of server 5 is synchronized through server 4. After determining the target identifier k1 and the identifier k2 to be added that are different between server 4 and server 5, the version information is compared and it is found that the version information seq:5 of k1 in server 4 is greater than the version information seq:5 of k1 in server 5. Then the data in server 4 is synchronized.<k1,v:”v3”,seq:6> Update to server 5, and the target data corresponding to the new identifier in server 4<k2,v:”v2”,seq:3> Updated to Server 5.

[0171] Finally, the data of server 1 is synchronized through server 5. After determining the target identifier k1 that is different between server 5 and server 1, the version information is compared and it is found that the version information seq:6 of k1 in server 5 is greater than the version information seq:5 of k1 in server 1. Then the data in server 5 is synchronized.<k1,v:”v3”,seq:6> Update to Server 1.

[0172] Through the above process, data synchronization can be performed efficiently and quickly on multiple servers to be synchronized.

[0173] In order to better implement the data synchronization method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the above data synchronization method. The meanings of the terms are the same as those in the above data synchronization method, and the specific implementation details can refer to the description in the method embodiment.

[0174] like Fig.13 As shown, Fig.13 A schematic diagram of the structure of a data synchronization device provided in an embodiment of the present application, wherein the data synchronization device may include an acquisition module 301, a comparison module 302, a determination module 303 and a synchronization module 304, wherein:

[0175] An acquisition module 301 is used to acquire a first hash tree of a data set on a first server, and to acquire a second hash tree of a target data set on a second server to be synchronized;

[0176] A comparison module 302 is used to start traversal comparison based on the root nodes of the first hash tree and the second hash tree to determine the target identifier corresponding to the target leaf node with differences;

[0177] A determination module 303, used to determine the first data and the second data corresponding to the target identifier in the data set and the target data set;

[0178] The synchronization module 304 is used to synchronize data according to the difference between the version information of the first data and the second data.

[0179] In some embodiments of the present application, Fig.14 As shown, the acquisition module 301 includes a data submodule 3011, a setting submodule 3012 and a loop submodule 3013, wherein:

[0180] The data submodule 3011 is used to perform hash calculation on each data in the data set on the first server to obtain a hash value of each data;

[0181] A setting submodule 3012 is used to set the hash value of each data on the first server to the value of the leaf node, and perform a hash calculation on the sum of the values ​​of the leaf nodes to obtain the value of the upper node;

[0182] The loop submodule 3013 is used to loop and perform hash calculation on the sum of the values ​​of the upper nodes to obtain the value of a new upper node until the upper node is the root node, so as to obtain the first hash tree of the data set.

[0183] In some embodiments of the present application, the data set on the first server includes multiple data, each data includes an identifier, a data value and version information, and the data submodule is specifically used to:

[0184] A hash calculation is performed on each data including an identifier, a data value and a version information in the data set on the first server to obtain a hash value of each data.

[0185] In some embodiments of the present application, the comparison module includes a setting submodule, a circulation submodule and an acquisition submodule, wherein:

[0186] A submodule is set, which is used to set subnodes with different hash values ​​among the subnodes of the root nodes on the first hash tree and the second hash tree as candidate nodes;

[0187] A loop submodule, used for looping and replacing the child nodes with different hash values ​​among the child nodes of the candidate nodes on the first hash tree and the second hash tree with new candidate nodes, until the candidate node becomes a leaf node;

[0188] The acquisition submodule is used to determine the candidate node at the end of the loop as the target leaf node and obtain the target identifier corresponding to the target leaf node.

[0189] In some embodiments of the present application, the data synchronization device further includes:

[0190] An identification module, configured to start traversal and comparison based on the root nodes of the first hash tree and the second hash tree, and determine a to-be-added identification corresponding to a target leaf node;

[0191] A data module is used to determine the target data corresponding to the identifier to be added in the data set;

[0192] The saving module is used to save the target data in the target data set for data synchronization, and update the second hash tree based on the synchronized target data set.

[0193] In some embodiments of the present application, the synchronization module is specifically used to:

[0194] When the first version information of the first data is greater than the second version information of the second data, the second data in the target data set is replaced with the first data to perform data synchronization.

[0195] In some embodiments of the present application, the synchronization module includes a comparison submodule and a replacement submodule, wherein:

[0196] a comparison submodule, configured to compare whether a hash value of the first data value is greater than a hash value of the second data value when the first version information of the first data is equal to the second version information of the second data and the first data value of the first data and the second data value of the second data are different;

[0197] The replacement submodule is used to replace the second data in the target data set with the first data to perform data synchronization when the hash value of the first data value is greater than the hash value of the second data value.

[0198] In some embodiments of the present application, the replacement submodule is specifically used for:

[0199] When the hash value of the first data value is greater than the hash value of the second data value, obtaining a storage address corresponding to the target identifier in the target data set;

[0200] The first data is written into a storage address corresponding to the target identifier in the target data set to replace the second data in the target data set.

[0201] In some embodiments of the present application, the data synchronization device further includes:

[0202] a target module, configured to determine a target server adjacent to the second server based on the sequence information;

[0203] An updating module, used to update the second server to a new first server, and update the adjacent server to a new second server;

[0204] The re-execution module is used to re-execute the process of obtaining the first hash tree of the data set on the first server and obtaining the second hash tree of the target data set on the second server to be synchronized.

[0205] In the present application, the acquisition module 301 acquires the first hash tree of the data set on the first server, and acquires the second hash tree of the target data set on the second server to be synchronized. The comparison module 302 starts traversal and comparison based on the root nodes of the first hash tree and the second hash tree, and determines the target identifier corresponding to the target leaf node with differences. The determination module 303 determines the first data and the second data corresponding to the target identifier in the data set and the target data set. Finally, the synchronization module 304 performs data synchronization according to the difference between the version information of the first data and the second data.

[0206] In the present application, only by comparing the first hash tree and the second hash tree corresponding to the data set and the target data set, the target identifier that is different between the data set and the target data set can be quickly and conveniently determined, and then based on this target identifier, the target data set is synchronized, so that the data synchronization method of the present application can significantly improve the data synchronization efficiency.

[0207] In addition, the embodiment of the present application also provides a computer device, which can be a terminal or a server. Fig.15 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0208] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Fig.15 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0209] The processor 401 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 402 and calling data stored in the memory 402, it executes various functions of the computer device and processes data, thereby performing overall detection of the computer device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0210] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0211] The computer device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0212] The computer device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0213] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, as follows:

[0214] Obtain a first hash tree of a data set on a first server, and obtain a second hash tree of a target data set on a second server to be synchronized; start traversal and comparison based on the root nodes of the first hash tree and the second hash tree, and determine the target identifier corresponding to the target leaf node with differences; determine the first data and the second data corresponding to the target identifier in the data set and the target data set; and synchronize data according to the difference between the version information of the first data and the second data.

[0215] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0216] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0217] To this end, an embodiment of the present application further provides a storage medium in which a computer program is stored, and the computer program can be loaded by a processor to execute the steps in any data synchronization method provided in the embodiment of the present application. For example, the computer program can execute the following steps:

[0218] Obtain a first hash tree of a data set on a first server, and obtain a second hash tree of a target data set on a second server to be synchronized; start traversal and comparison based on the root nodes of the first hash tree and the second hash tree, and determine the target identifier corresponding to the target leaf node with differences; determine the first data and the second data corresponding to the target identifier in the data set and the target data set; and synchronize data according to the difference between the version information of the first data and the second data.

[0219] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0220] Since the computer program stored in the storage medium can execute the steps in any data synchronization method provided in the embodiments of the present application, the beneficial effects that can be achieved by any data synchronization method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0221] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in various optional implementations in the above-mentioned embodiments.

[0222] The above is a detailed introduction to a data synchronization method, device, storage medium and computer equipment provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data synchronization method, characterized in that: include: Obtain a first hash tree of a data set on a first server, and obtain a second hash tree of a target data set on a second server to be synchronized; Starting traversal and comparison based on the root nodes of the first hash tree and the second hash tree, determining the target identifier corresponding to the target leaf node with differences, including: when it is identified that the value of the leaf node in the first hash tree is a deletion keyword, and the value of the corresponding leaf node in the second hash tree is a hash value, determining the corresponding leaf node as the target leaf node, and determining the data identifier corresponding to the target leaf node as the identifier to be deleted; wherein the deletion keyword is the value of the leaf node retained in the first hash tree of the deleted data; starting traversal and comparison based on the root nodes of the first hash tree and the second hash tree, when it is found that the target leaf node exists in the first hash tree but does not exist in the second hash tree, determining the data identifier corresponding to the target leaf node as the identifier to be added, wherein the leaf node corresponding to the hash value of the newly added data is located at the side of the first hash tree; Determine first data and second data corresponding to the target identifier in the data set and the target data set; Data synchronization is performed according to the difference between version information of the first data and the second data; wherein the version information is represented by a hash value.

2. The method according to claim 1, characterized in that The step of obtaining a first hash tree of a data set on a first server includes: Perform hash calculation on each data in the data set on the first server to obtain a hash value of each data; The hash value of each data on the first server is set as the value of the leaf node, and the sum of the values ​​of the leaf nodes is hashed to obtain the value of the upper node; The sum of the values ​​of the upper nodes is hashed in a loop to obtain the value of a new upper node, until the upper node becomes the root node, thereby obtaining the first hash tree of the data set.

3. The method according to claim 2, characterized in that The data set on the first server includes multiple data, each data includes an identifier, a data value and version information, and performing a hash calculation on each data in the data set on the first server to obtain a hash value of each data includes: A hash calculation is performed on each data including an identifier, a data value and a version information in the data set on the first server to obtain a hash value of each data.

4. The method according to claim 1, characterized in that: The traversing and comparing based on the root nodes of the first hash tree and the second hash tree to determine the target identifier corresponding to the target leaf node with a difference includes: Setting the child nodes with different hash values ​​among the child nodes of the root nodes on the first hash tree and the second hash tree as candidate nodes; Loopingly replace the child nodes with different hash values ​​among the child nodes of the candidate nodes on the first hash tree and the second hash tree with new candidate nodes until the candidate node becomes a leaf node; The candidate node at the end of the loop is determined as the target leaf node, and the target identifier corresponding to the target leaf node is obtained.

5. The method according to claim 1, characterized in that The method further comprises: Starting traversal and comparison based on the root nodes of the first hash tree and the second hash tree, determining the to-be-added identifier corresponding to the target leaf node; Determine the target data corresponding to the to-be-added identifier in the data set; The target data is stored in the target data set to perform data synchronization, and the second hash tree is updated based on the synchronized target data set.

6. The method according to claim 1, characterized in that The synchronizing the data according to the difference between the version information of the first data and the second data includes: When the first version information of the first data is greater than the second version information of the second data, the second data in the target data set is replaced with the first data to perform data synchronization.

7. The method according to claim 1, characterized in that The synchronizing the data according to the difference between the version information of the first data and the second data includes: When the first version information of the first data is equal to the second version information of the second data, and the first data value of the first data and the second data value of the second data are different, comparing whether the hash value of the first data value is greater than the hash value of the second data value; When the hash value of the first data value is greater than the hash value of the second data value, the second data in the target data set is replaced with the first data to perform data synchronization.

8. The method according to claim 7, characterized in that The replacing the second data in the target data set with the first data comprises: Obtaining a storage address corresponding to the target identifier in the target data set; The first data is written into a storage address corresponding to the target identifier in the target data set to replace the second data in the target data set.

9. The method according to claim 1, characterized in that: After synchronizing the data according to the difference between the version information of the first data and the second data, the method further includes: determining a target server adjacent to the second server based on the sequence information; Updating the second server to a new first server, and updating the adjacent server to a new second server; Re-execute the process of obtaining a first hash tree of the data set on the first server, and obtaining a second hash tree of the target data set on the second server to be synchronized.

10. A data synchronization device, characterized in that: include: An acquisition module, used to acquire a first hash tree of a data set on a first server, and to acquire a second hash tree of a target data set on a second server to be synchronized; A comparison module, used to start traversal comparison based on the root nodes of the first hash tree and the second hash tree, and determine the target identifier corresponding to the target leaf node with differences, including: when it is identified that the value of the leaf node in the first hash tree is a deletion keyword, and the value of the corresponding leaf node in the second hash tree is a hash value, determine the corresponding leaf node as the target leaf node, and determine the data identifier corresponding to the target leaf node as the identifier to be deleted; wherein the deletion keyword is the value of the leaf node retained by the deleted data in the first hash tree; start traversal comparison based on the root nodes of the first hash tree and the second hash tree, when it is found that the target leaf node exists in the first hash tree but does not exist in the second hash tree, determine the data identifier corresponding to the target leaf node as the identifier to be added, wherein the leaf node corresponding to the hash value of the newly added data is located at the side of the first hash tree; A determination module, configured to determine first data and second data corresponding to the target identifier in the data set and the target data set; A synchronization module is used to synchronize data according to the difference between version information of the first data and the second data; wherein the version information is represented by a hash value.

11. A storage medium, characterized in that: The storage medium stores a plurality of computer programs, and the computer programs are suitable for being loaded by a processor to execute the steps in any one of the methods according to claims 1 to 9.

12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the method according to any one of claims 1 to 9.

Images