Data synchronization method, system, electronic device and storage medium

Abstract

The present application discloses a data synchronization method, a system, an electronic device and a storage medium. The method comprises: receiving a data synchronization command sent by a source node; determining a target slot for the data synchronization command; on the basis of the target slot, target sequence information carried by the data synchronization command, and sequence information corresponding to each slot recorded for a synchronization task of the source node in a destination node, determining to write or discard the data synchronization command. According to the present application, the sequence information for data synchronization is accurate to a minimum data partition (i.e., a slot), that is, the sequence information is maintained on a node according to slot dimension, and the sequence information corresponding to the slots is independent of each other, so that even if cluster configuration adjustment occurs, sequence information of a single slot does not change.

Description

Technical Field of Data Synchronization Method, System, Electronic Device and Storage Medium

[0001] This application relates to the field of computer technology, and particularly to a data synchronization method, system, electronic device and storage medium. Background Art

[0002] In the field of distributed database clusters, in order to ensure that data can be updated orderly and uniquely, each node maintains a sequence information, which is automatically incremented according to certain rules. That is, for each data update command, a sequence number is assigned to the data update command and recorded in the form of a log together with the data update command.

[0003] To ensure data consistency between nodes, data synchronization between two nodes is involved. That is, when the source node synchronizes the data update command to the destination node, the destination node compares the sequence number carried by the data update command with the latest sequence number maintained on the destination node. If it is greater than the maintained latest sequence number, the data update command is written to the destination node. If it is less than the maintained latest sequence number, it is considered a redundant command and the data update command is discarded.

[0004] However, if a cluster reconfiguration occurs, the sequence numbers maintained on the nodes will change, resulting in problems such as command loss and duplicate command writing during the data synchronization process. Summary of the Invention

[0005] The purpose of this application is to propose a data synchronization method, system, electronic device and storage medium for the above-mentioned deficiencies in the prior art, and this purpose is achieved through the following technical solutions.

[0006] The first aspect of this application proposes a data synchronization method applied to a destination node. The method includes: receiving a data synchronization command sent by a source node; determining a target slot for the data synchronization command; and determining whether to write or discard the data synchronization command based on the target slot, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task of the destination node for the source node.

[0007] Based on the data synchronization method described in the above first aspect, it has at least the following beneficial effects or advantages.

[0008] When the destination node receives a data synchronization command from the source node, by determining the target of the data synchronization command Slot positions, and determine whether to write or discard the data synchronization command according to the target slot position, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot position recorded for the source node in the destination node. Since the sequence information for data synchronization in this application is accurate to the smallest data partition (i.e., the slot position), that is, the sequence information is maintained according to the slot dimension on the node, rather than maintaining a single sequence information for one node, the sequence information corresponding to each slot position is independent of each other. In this way, even if a cluster reconfiguration occurs, the sequence information of a single slot position will not change. Therefore, during the data synchronization process, the problem of command loss or duplicate writing can be reduced, and the data consistency between nodes can be ensured.

[0009] A second aspect of this application proposes a data synchronization method, which is applied to the source node. The method further includes: generating a data synchronization command according to the local data update command log; sending the data synchronization command to the destination node, so that the destination node determines whether to write or discard the data synchronization command according to the target slot position targeted by the data synchronization command, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot position recorded for the synchronization task of the source node in the destination node.

[0010] Based on the data synchronization method described in the above second aspect, it has at least the following beneficial effects or advantages.

[0011] On the source node, a data synchronization command is generated through the local data update command log, and the data synchronization command is sent to the destination node. Since on the destination node, the sequence information used for data synchronization is accurate to the smallest data partition (i.e., the slot position), that is, the sequence information is maintained according to the slot dimension, rather than maintaining a single sequence information for one node. Therefore, even if a cluster reconfiguration occurs during the data synchronization process, the sequence information of a single slot position will not change. On the destination node, it is still possible to determine whether to write or discard the data synchronization command according to the target slot position targeted by the data synchronization command, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot position recorded for the synchronization task of the source node in the destination node, so as to complete the data synchronization and ensure the data consistency between nodes.

[0012] A third aspect of this application proposes a data synchronization system, which includes a source node and a destination node; the destination node is used to execute the method described in the above first aspect; the source node is used to execute the method described in the above second aspect.

[0013] A fourth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the method described in the first aspect or the second aspect above.

[0014] A fifth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, and the program is executed by a processor to implement the method described in the first aspect or the second aspect above.

[0015] The above description is only an overview of the technical solution of the present application. In order to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically exemplified below. Brief Description of the Drawings

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings.

[0017] FIG. 1 is a schematic diagram of command loss caused by cluster scale-down;

[0018] FIG. 2 is a schematic diagram of repeated command writing caused by cluster scale-up;

[0019] FIG. 3 is a flowchart of a data synchronization method for a destination node example shown according to an exemplary embodiment;

[0020] FIG. 4 is a schematic diagram of a data synchronization task record on a node shown according to an exemplary embodiment;

[0021] FIG. 5 is a flowchart of a data synchronization method on the source node side shown according to an exemplary embodiment;

[0022] FIG. 6 is an overall schematic diagram of implementing data synchronization shown according to an exemplary embodiment;

[0023] FIG. 7 is a schematic diagram of the hardware structure of an electronic device shown according to an exemplary embodiment;

[0024] FIG. 8 is a schematic diagram of the structure of a storage medium shown according to an exemplary embodiment. Detailed Description of the Embodiments

[0025] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of systems and methods consistent with some aspects of the present application as detailed in the appended claims.

[0026] The terms used in this application are for the purpose of describing particular embodiments only and are not intended to limit the present application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0027] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when... ... time" or "when... is... " or "in response to determining".

[0028] As mentioned above, in the prior art, a scheme of maintaining a sequence information by one node is adopted. In this way, if the cluster is reconfigured, that is, the cluster is scaled down or scaled up, it is very easy to cause the sequence information maintained by the node to change, resulting in a sequence information conflict problem during the data synchronization process, and further leading to problems such as command loss and command duplicate writing.

[0029] As shown in Figure 1, the problem of command loss caused by cluster scaling down. Assume that node B1 in cluster B needs to synchronize data to node A1 in cluster A. Node B1 sequentially sends data update commands with serial numbers 100 - 104 to node A1. When node A1 receives the data update command with serial number 100, the serial number maintained by node A1 is 100. At this time, cluster A needs to stop node A2 for scaling down, and migrate the slot slot2 on node A2 to node A1 through slot migration. Since the cluster scaling down uses incremental slot migration, the serial number 106 maintained by node A2 will also be carried to node A1 along with slot slot2. As a result, the serial number maintained by node A1 changes from 100 to 106. Then, when node A1 subsequently receives data update commands with serial numbers 101 - 104, because the serial numbers 101 - 104 are all less than the serial number 106 maintained by node A1, node A1 will skip the data update commands with serial numbers 101 - 104, resulting in the loss of commands that have not been synchronized yet.

[0030] As shown in Figure 2, the problem of command duplicate writing caused by cluster scaling up. Assume that node B1 in cluster B needs to synchronize data to node A2 in cluster A. Node B1 sends data update commands with serial numbers 80 - 106 to node A2, and these data update commands are all written to slot slot2 on node A2. Therefore, the latest serial number maintained by node A2 is 106. At this time, a new empty node A1 is added to cluster A for scaling up. The initial serial number maintained by node A1 is 0, and the slot slot2 on node A2 is migrated to node A1 through slot migration. Since the cluster scaling up uses full - volume slot migration, the serial number maintained by node A2 will not be carried to node A1 along with slot slot2. At this time, the serial number maintained by node A1 is still 0. Assume that node B1 sends a data update command with serial number 107 to node A2 again, and this data update command is for slot slot2. Since slot slot2 no longer exists on node A2, node A2 returns the slot allocation information of cluster A to node B1. Node B1 rolls back the data synchronization to serial number 80 according to the serial number 80 recorded for data synchronization by itself, that is, node B1 sends data update commands with serial numbers 80 - 106 to node A1 in sequence. Because the serial number maintained by node A1 is 0, these data update commands will all be written to slot slot2 on node A1 repeatedly, resulting in the problem of command duplicate writing.

[0031] To solve the above technical problems, the present application proposes a data synchronization method. By making the sequence information used for data synchronization accurate to the smallest data partition (i.e., slot), that is, maintaining the sequence information at the node according to the slot dimension, rather than maintaining a sequence information for each node, and the sequence information corresponding to each slot is independent of each other. When the destination node receives the data synchronization command sent by the source node, by determining the target slot targeted by the data synchronization command, and based on the target slot, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node, it decides whether to write or discard the data synchronization command. In this way, even if a cluster reconfiguration occurs, since the sequence information of a single slot does not change, the problem of command loss or duplicate writing can be reduced during the data synchronization process, and the data consistency between nodes can be ensured. Slot corresponding sequence information, decides whether to write or discard the data synchronization command. In this way, even if a cluster reconfiguration occurs, since the sequence information of a single slot does not change, the problem of command loss or duplicate writing can be reduced during the data synchronization process, and the data consistency between nodes can be ensured.

[0032] The following uses specific embodiments to elaborate in detail on the technical solution of the present application and how the technical solution of the present application solves the foregoing technical problems. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following will describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0033] Embodiment 1.

[0034] FIG. 3 is a flowchart of an embodiment of a data synchronization method shown according to an exemplary embodiment. The execution subject of this embodiment is on the side of the destination node, and the destination node can be considered as the destination end of data synchronization, including the following steps 301 to 303.

[0035] Step 301: Receive the data synchronization command sent by the source node.

[0036] Step 302: Determine the target slot targeted by the data synchronization command.

[0037] Step 303: According to the target slot, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node, determine whether to write or discard the data synchronization command.

[0038] In the embodiment of the present application, the source node and the destination node can be data nodes in two different clusters respectively. The two data nodes need to complete data synchronization to maintain data consistency.

[0039] The data synchronization command is an instruction used to implement the same data operations on the destination node as on the source node. That is to say, the data operations on the source node and the destination node should be kept in sync to achieve data consistency between the destination node and the source node. This data synchronization command consists of two parts: command header information and data operation statement information. The command header information includes the synchronization task ID between the source node and the destination node and the sequence information assigned by the source node for this command, that is, the target sequence information. The data operation statement information includes the operation instruction type, the key (representing the index of the data in the database) to be operated on, and the corresponding key value value. For example, for the data synchronization command ars seq(task ID, 100) set k v, where "seq(task ID, 100)" represents the command header information and "set k v" represents the data operation statement, indicating that the key value value corresponding to the key key = k is set to v.

[0040] The target slot for the data synchronization command refers to the slot to which this data synchronization command is to be written. Here, it needs to be explained that in a distributed database cluster, a slot is a logical container for storing data. A slot can be regarded as a data shard or data partition, which is used to divide and store data on different nodes according to certain rules. The number of slots is usually fixed, and each slot is assigned to a node. When data needs to be stored or queried the distributed database cluster will route the command to the corresponding node for operation according to the slot rules.

[0041] The sequence information corresponding to each slot in the synchronization task record of the destination node for the source node refers to the sequence information maintained for each slot managed by the destination node for the synchronization task of the source node. The sequence information corresponding to each slot is independent of each other. For example, the sequence information seq_num = 101 maintained for slot slotl indicates that the sequence information of the latest data synchronization command written by the source node to slot slotl is 101.

[0042] For example, as shown in Figure 4, the destination node belongs to cluster A, the source node belongs to cluster B, the task ID of the synchronization task between the destination node in cluster A and the source node in cluster B is abcd987. There are five slot positions 101 - 105 on the destination node. For the task ID: abcd987, the sequence information maintained corresponding to these five slot positions is: 50, 60, 70, 80, 90. At the same time, for the task ID: abcd987, the timestamp information carried by the previous data synchronization command from this source node in cluster B is also recorded.

[0043] Furthermore, as can be seen from Figure 4, on the destination node in cluster A, the sequence information and timestamp information maintained corresponding to each slot of the synchronization task with nodes in other clusters are also recorded, and the task ID is abcd990.

[0044] So far, through the data synchronization process shown in Figure 3 above, data consistency between the two nodes has been achieved. By making the sequence information used for data synchronization accurate to the smallest data partition, that is, maintaining sequence information according to the slot dimension on the node, rather than a single node maintaining a single sequence information, the sequence information corresponding to each slot is independent of each other. When the destination node receives a data synchronization command sent by the source node, by determining the target slot for which the data synchronization command is targeted, and based on the target slot, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot in the node, it decides whether to write or discard the data synchronization command. In this way, even if there is a cluster reconfiguration, since the sequence information of a single slot does not change, the problem of command loss or duplicate writing can be reduced during the data synchronization process.

[0045] In some embodiments of the present application, for the process of determining the target slot for which the data synchronization command is targeted, the key to be operated in the data synchronization command can be obtained, and the slot index can be calculated using this key, so as to determine the slot indicated by the slot index as the target slot for which the data synchronization command is targeted.

[0046] Among them, the key to be operated represents the index of the data in the database. In a specific implementation, the slot index to which the key to be operated belongs can be calculated by the CRC16 (Cyclic Redundancy Check) operation method. For example, the cluster consists of three nodes, and there are 16,384 slots corresponding to the entire cluster, and the corresponding slot indexes are 0 - 16383. Suppose the slot indexes assigned to Node 1 are 0 - 4999, the slot indexes assigned to Node 2 are 5000 - 9999, and the slot indexes assigned to Node 3 are 10000 - 16383. For the data synchronization command to execute the set key1 value1 command, the key to be operated is key1. Through CRC16(key1)&16383 = 8876, it is considered that the slot index of the target slot for the data synchronization command is 8876.

[0047] As a feasible embodiment, before determining the target slot for the data synchronization command, that is, before executing step 302, the first timestamp information carried by the data synchronization command can be obtained, and the second timestamp information carried by the previous data synchronization command from the source node can be obtained. By comparing the first timestamp information and the second timestamp information, when the first timestamp information is greater than the second timestamp information, it indicates that the data synchronization command is a new command, and then step 302 is executed. When the first timestamp information is less than or equal to the second timestamp information, it indicates that the data synchronization command is an expired command, and an error notification of command expiration is returned to the source node.

[0048] Among them, the first timestamp information refers to the time when the source node sends the command. The second timestamp information is the time when the source node sends the previous command. This embodiment can exclude the expired commands between the destination node and the source node by using the timestamp information of the command for judgment and comparison.

[0049] In some embodiments of the present application, for the process of determining whether to write or discard the data synchronization command according to the target slot, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task of the destination node for the source node, the following two situations can be divided.

[0050] The first is when the target slot belongs to the management of the destination node. The sequence information corresponding to the target slot is obtained from the sequence information corresponding to each slot recorded in the synchronization task for the source node, and then according to the sequence information corresponding to the target slot and the target sequence information, it is determined whether to write or discard the data synchronization command.

[0051] The second case is when the target slot is not managed by the destination node. In this case, discard the data synchronization command and return the slot allocation information of the cluster where the destination node is located to the source node, so that the source node can send the data synchronization command to the target node to which the target slot belongs according to the slot allocation information.

[0052] Among them, the first case is the synchronization situation before the cluster reconfiguration. The target slot targeted by the data synchronization command is still on the destination node. The second case is the synchronization situation after the cluster reconfiguration. The target slot targeted by the data synchronization command has migrated to other nodes and is not managed by the destination node. Therefore, the slot allocation information of the cluster has changed. At this time, the destination node cannot process the data synchronization command and needs to return the new slot allocation information to the source node so that the source node can send the data synchronization command to the target node to which the target slot belongs for processing.

[0053] It should be noted here that the cluster reconfiguration referred to in the second case can be cluster expansion or cluster contraction, and this application does not make specific limitations on this. However, it should be noted that when the target slot migrates to other nodes, since the sequence information corresponding to each slot is independent of each other, the sequence information corresponding to the target slot maintained by the destination node for different synchronization tasks will also be sent to other nodes along with the slot migration, but will not be changed due to the cluster reconfiguration.

[0054] In a specific implementation, for the process of determining whether to write or discard the data synchronization command according to the sequence information corresponding to the target slot and the target sequence information, by comparing the magnitude relationship between the sequence information corresponding to the target slot and the target sequence information, if the sequence information corresponding to the target slot is less than the target sequence information, then write the data synchronization command; if the sequence information corresponding to the target slot is greater than or equal to the target sequence information, it is considered that the data synchronization command is a duplicate command, and then discard the data synchronization command.

[0055] Among them, the sequence information corresponding to the target slot represents the sequence information of the latest data synchronization command written by the source node on the target slot. If the sequence information corresponding to the target slot is greater than or equal to the target sequence information carried by the current data synchronization command, it means that the current data synchronization command is a duplicate command during the task failure retry process.

[0056] In this implementation, writing the data synchronization command means operating on the corresponding keys and key values on the target slot according to the data operation statements in the data synchronization command, and at the same time recording the data synchronization command in the form of a log.

[0057] It should be noted that after writing the data synchronization command, the sequence information corresponding to the target slot of the synchronization task record for the source node in the destination node can be updated by using the target sequence information carried by the data synchronization command.

[0058] Embodiment 2.

[0059] Based on the above-described embodiment shown in FIG. 3, FIG. 5 is a flowchart of another data synchronization method shown according to an exemplary embodiment. The execution subject of this embodiment is on the source node side. The source node can be considered as the source end of data synchronization, and includes the following steps 501 to 502.

[0060] Step 501: Generate a data synchronization command according to the local data update command log.

[0061] Step 502: Send the data synchronization command to the destination node, so that the destination node determines whether to write or discard the data synchronization command according to the target slot targeted by the data synchronization command, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot of the synchronization task record for the source node in the destination node.

[0062] In the embodiments of the present application, all executed write operations are usually recorded in the form of a log on the node. To ensure data consistency between the source node and the destination node, a data update command log is used to generate a data synchronization command during the data synchronization process, such as a binlog log (a binary log).

[0063] Based on the above Embodiment 2, on the source node, a data synchronization command is generated through the local data update command log and sent to the destination node. Since the sequence information used for data synchronization on the destination node is accurate to the smallest data partition (i.e., the slot), that is, the sequence information is maintained according to the slot dimension, rather than maintaining a sequence information for each node, even if a cluster configuration change occurs during the data synchronization process, the sequence information of a single slot will not change. On the destination node, it is still possible to determine whether to write or discard the data synchronization command according to the target slot targeted by the data synchronization command, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot of the synchronization task record for the source node in the destination node, so as to complete the data synchronization and ensure data consistency between nodes.

[64] In some embodiments of the present application, the process of generating a data synchronization command according to the local data update command log can be divided into the following two synchronization forms.

[65] The first is the full synchronization form: By rewriting the local data update command log, a new data update command log is obtained, and thus data synchronization commands are generated using the data update commands included in the new data update command log.

[66] Among them, log rewriting (rewrit) means that multiple commands in the old log become one command in the new log obtained after rewriting. For example, in the data update command log, before log rewriting, for a key-key value pair, 6 data update commands are recorded in chronological order: set keyl vl; set keyl v2; set key 1 v3; set keyl v4; set keyl v5; set keyl v6; that is to say, the change process of the value value in this key-key value pair is vl I v2 ->■ v31 v4 ->■ v5 ->• v6. After log rewriting, in the new data update command log obtained, for this key-key value pair, only 1 data update command is recorded: set keyl v6 (that is, the last data update command among the 6 data update commands). Thus, it can be seen that by performing log rewriting on the data update command log, the data synchronization commands transmitted during the data synchronization process can be reduced.

[67] The above full synchronization form can be the initial synchronization process between the source node and the destination node, and it is necessary to synchronize all the data on the source node to the destination node.

[68] The second is the incremental synchronization form: Data synchronization commands are generated using the data update commands recorded most recently in the local data update command log.

[69] Among them, the data update commands recorded most recently are the records of the data update operations currently executed on the source node. That is to say, every time a data update command is executed on the source node, it will be synchronized to the destination node once.

[70] The above incremental synchronization form can be the synchronization process after the full synchronization of the initial synchronization process between the source node and the destination node is completed.

[71] It should be noted that the above two synchronization forms are only for the source node. For the destination node, whether it is the full form or the incremental form, what is received are data synchronization commands.

[72] Based on the embodiments shown in FIGS. 3 and 5 above, FIG. 6 is an overall schematic diagram of realizing data synchronization shown according to an exemplary embodiment. Node B1 belonging to cluster B is used as the source node, and node A1 belonging to cluster A is used as the destination node. The task ID of the synchronization task between cluster B and cluster A is B1. The entire data synchronization is divided into a full synchronization process and an incremental synchronization process.

[73] In the full synchronization process, node B1 performs log rewriting on the local data update command log to obtain a new data update command log, and generates a data synchronization command using the data update commands included in the new data update command log. And send it to node A1. For example, assume that the data update command recorded in the new data update command log is: set k v, and the corresponding sequence information is 80. A data synchronization command is generated using this data update command, the corresponding sequence information, and the task ID: ars seq(B1, 80) set k v. Further, when node A1 receives this data synchronization command, it determines the target slot slot1 and determines to write this data synchronization command, so as to update the sequence information corresponding to slot1 recorded for the synchronization task: B1 on node A1: seq_num = 80.

[0074] In the incremental synchronization process, node B1 generates a data synchronization command using the data update command recorded most recently in the data update command log and sends it to node A1. For example, assume that the data update command currently executed by node B1 is: set k1 v1, and the corresponding assigned sequence information is 100. A data synchronization command is generated using this data update command, the corresponding sequence information, and the task ID: ars seq(B1, 100) set k1 v1. Further, when node A1 receives this data synchronization command, it determines the target slot slot1 and determines to write this data synchronization command, so as to update the sequence information corresponding to slot1 recorded for the synchronization task: B1 on node A1: seq_num = 100.

[0075] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0076] Corresponding to the embodiments of the foregoing data synchronization method, the present application further provides a data synchronization system, which includes a source node and a destination node. Among them, the destination node is used to execute the data synchronization method provided in the embodiment shown in FIG. 3 above, and the source node is used to execute the data synchronization method provided in the embodiment shown in FIG. 5 above.

[0077] The embodiment of the present application also provides an electronic device corresponding to the data synchronization method provided in the foregoing embodiment to execute the above data synchronization method.

[0078] FIG. 7 is a hardware structure diagram of an electronic device shown according to an exemplary embodiment. The electronic device includes: a communication interface 601, a processor 602, a memory 603, and a bus 604; among them, the communication interface 601, the processor 602, and the memory 603 complete mutual communication through the bus 604. The processor 602 can execute the data synchronization method described above by reading and executing machine-executable instructions corresponding to the control logic of the data synchronization method in the memory 603. For the specific content of this method, see the above embodiments and will not be repeated here.

[0079] The memory 603 mentioned in the present application can be any electronic, magnetic, optical or other physical storage system, and can contain stored information such as executable instructions, data, etc. Specifically, the memory 603 can be RAM (Random Access Memory), flash memory, a storage drive (such as a hard disk drive), any type of storage disk (such as an optical disk, DVD, etc.), or a similar storage medium, or a combination thereof. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 601 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.

[0080] The bus 604 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Among them, the memory 603 is used to store a program, and the processor 602 executes the program after receiving an execution instruction.

[0081] The processor 602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 602 or the instructions in the form of software. The above-mentioned processor 602 may be a general-purpose processor, including a network processor (NP for short), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed by the hardware decoding processor, or executed by a combination of the hardware and software modules in the decoding processor.

[0082] The electronic device provided in the embodiments of the present application and the data synchronization method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run, or implemented by it.

[0083] The embodiments of the present application also provide a computer-readable storage medium corresponding to the data synchronization method provided in the foregoing embodiments. Referring to FIG. 8, the computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the data synchronization method provided in any of the foregoing embodiments.

[0084] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other optical and magnetic storage media, which will not be elaborated here one by one.

[0085] The computer-readable storage medium provided in the above embodiments of the present application and the data synchronization method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run, or implemented by the application program stored in it.

[0086] Those skilled in the art will readily think of other implementation schemes of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application. Alternatively, the adaptive changes follow the general principles of this application and include well-known general knowledge or common technical means in the technical field not disclosed in this application. The description and examples are only regarded as exemplary, and the true scope and spirit of this application are pointed out by the following claims.

[0087] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising a....." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the said element.

[0088] The above are only the preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

Claims 1. A data synchronization method, wherein, Applied to a destination node, the method includes: receiving a data synchronization command sent by a source node; determining a target slot for the data synchronization command; determining whether to write or discard the data synchronization command based on the target slot, target sequence information carried in the data synchronization command, and sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node.

2. The method according to claim 1, wherein The step of determining whether to write or discard the data synchronization command based on the target slot, target sequence information carried in the data synchronization command, and sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node includes: when the target slot is under the management of the destination node, obtaining the sequence information corresponding to the target slot from the sequence information corresponding to each slot recorded in the synchronization task for the source node; determining whether to write or discard the data synchronization command based on the sequence information corresponding to the target slot and the target sequence information; when the target slot is not under the management of the destination node, discarding the data synchronization command and returning the slot allocation information of the cluster where the destination node is located to the source node, so that the source node can send the data synchronization command to the target node to which the target slot belongs according to the slot allocation information.

3. The method according to claim 2, wherein The step of determining whether to write or discard the data synchronization command based on the sequence information corresponding to the target slot and the target sequence information includes: writing the data synchronization command when the sequence information corresponding to the target slot is less than the target sequence information; discarding the data synchronization command when the sequence information corresponding to the target slot is greater than or equal to the target sequence information.

4. The method according to claim 1, wherein Before determining the target slot for the data synchronization command, the method further includes: obtaining first timestamp information carried in the data synchronization command, where the first timestamp information indicates the time when the source node sends the command; obtaining second timestamp information carried in the previous data synchronization command from the source node; when the first timestamp information is greater than the second timestamp information, performing the step of determining the target slot for the data synchronization command; when the first timestamp information is less than or equal to the second timestamp information, returning an error notification indicating that the command has expired to the source node.

5. The method according to claim 1, wherein The step of determining the target slot for the data synchronization command includes: obtaining the key to be operated on in the data synchronization command, where the key represents the index of the data in the database; calculating the slot index using the key; and determining the slot indicated by the slot index as the target slot for the data synchronization command.

6. The method according to any one of claims 1-5, wherein After writing the data synchronization command, the method further includes: updating the sequence information corresponding to the target slot recorded in the synchronization task for the source node in the destination node using the target sequence information carried in the data synchronization command.

7. A data synchronization method, wherein Applied to the source node, the method further includes: generating a data synchronization command according to a local data update command log; sending the data synchronization command to the destination node, so that the destination node determines whether to write or discard the data synchronization command according to the target slot targeted by the data synchronization command, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node.

8. The method according to claim 7, wherein The generating a data synchronization command according to the local data update command log includes: rewriting the data update command log to obtain a new data update command log, and generating a data synchronization command by using the data update commands included in the new data update command log; or generating a data synchronization command by using the data update command recorded latest in the data update command log.

9. A data synchronization system, wherein The system includes a source node and a destination node; the destination node is configured to execute the method according to any one of claims 1-6 above; the source node is configured to execute the method according to any one of claims 7-8 above.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor executes the program to implement the method according to any one of claims 1-8 above.

11. A computer-readable storage medium having a computer program stored thereon, wherein, The program is executed by the processor to implement the method according to any one of claims 1-8 above.

Images