Data synchronization method and system, electronic equipment and storage medium

Abstract

The invention discloses a data synchronization method and system, electronic equipment and a storage medium. The method comprises the following steps: receiving a data synchronization command sent by a source node; determining a target slot position which the data synchronization command aims at; and determining to write or discard the data synchronization command according to the target slot position, target sequence information carried by the data synchronization command and sequence information corresponding to each slot position recorded for the synchronization task of the source node in the destination node. As the sequence information for data synchronization is accurate to the minimum data partition (namely slot position), the sequence information is maintained on the node according to the slot position dimension instead of maintaining one piece of sequence information for one node, and the sequence information corresponding to each slot position is mutually independent, even if cluster transformation and distribution occur, the sequence information can be accurately synchronized. And the sequence information of a single slot does not change, so that the problem of command loss or repeated writing can be reduced in the data synchronization process, and the consistency of data between nodes can be ensured.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a data synchronization method, system, electronic device and storage medium. Background Art

[0002] In the field of distributed database clusters, in order to ensure that data can be updated orderly and uniquely, each node maintains a sequence information and automatically increments it according to certain rules, that is, for each data update command, a sequence number is assigned to the data update command and recorded together with the data update command in the form of a log.

[0003] To ensure data consistency between nodes, data synchronization between two nodes is involved. That is, when the source node synchronizes the data update command to the destination node, the destination node compares the sequence number carried by the data update command with the latest sequence number maintained on the destination node. If it is greater than the maintained latest sequence number, the data update command is written to the destination node. If it is less than the maintained latest sequence number, it is considered a redundant command and the data update command is discarded.

[0004] However, if a cluster reconfiguration occurs, the sequence numbers maintained on the nodes will change, resulting in problems such as command loss and duplicate command writing during the data synchronization process. Summary of the Invention

[0005] The purpose of this application is to propose a data synchronization method, system, electronic device and storage medium for the deficiencies of the above-mentioned prior art, and this purpose is achieved through the following technical solutions.

[0006] The first aspect of this application proposes a data synchronization method, which is applied to a destination node, and the method includes:

[0007] Receiving a data synchronization command sent by a source node;

[0008] Determining a target slot for the data synchronization command;

[0009] Determining whether to write or discard the data synchronization command according to the target slot, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task of the destination node for the source node.

[0010] Based on the data synchronization method described in the above first aspect, it has at least the following beneficial effects or advantages:

[0011] When the destination node receives a data synchronization command from the source node, it determines the target slot targeted by the data synchronization command, and decides whether to write or discard the data synchronization command based on the target slot, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot recorded for the source node in the destination node. Since the sequence information used for data synchronization in this application is accurate to the smallest data partition (i.e., the slot), that is, the sequence information is maintained according to the slot dimension on the node, rather than maintaining one sequence information for one node, and the sequence information corresponding to each slot is independent of each other. In this way, even if there is a cluster reconfiguration, the sequence information of a single slot will not change. Therefore, during the data synchronization process, the problem of command loss or duplicate writing can be reduced, and the data consistency between nodes can be ensured.

[0012] The second aspect of this application proposes a data synchronization method, which is applied to the source node. The method further includes:

[0013] Generate a data synchronization command according to the local data update command log;

[0014] Send the data synchronization command to the destination node, so that the destination node determines whether to write or discard the data synchronization command based on the target slot targeted by the data synchronization command, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot recorded for the source node in the destination node's synchronization task record for the source node.

[0015] Based on the data synchronization method described in the above second aspect, it has at least the following beneficial effects or advantages:

[0016] On the source node, a data synchronization command is generated according to the local data update command log and sent to the destination node. Since on the destination node, the sequence information used for data synchronization is accurate to the smallest data partition (i.e., the slot), that is, the sequence information is maintained according to the slot dimension, rather than maintaining one sequence information for one node. Therefore, even if there is a cluster reconfiguration during the data synchronization process, the sequence information of a single slot will not change. On the destination node, it can still determine whether to write or discard the data synchronization command based on the target slot targeted by the data synchronization command, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot recorded for the source node in the destination node's synchronization task record for the source node, so as to complete the data synchronization and ensure the data consistency between nodes.

[0017] The third aspect of this application proposes a data synchronization system, which includes a source node and a destination node;

[0018] The destination node is used to execute the method described in the above first aspect;

[0019] The source node is used to execute the method described in the above second aspect.

[0020] A fourth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor executes the program to implement the method described in the first aspect or the second aspect above.

[0021] A fifth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. The program is executed by a processor to implement the method described in the first aspect or the second aspect above.

[0022] The above description is only an overview of the technical solution of the present application. In order to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the following specific embodiments of the present application are specifically exemplified. Description of the Drawings

[0023] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0024] Figure 1 It is a schematic diagram of command loss caused by cluster scaling down;

[0025] Figure 2 It is a schematic diagram of repeated command writing caused by cluster scaling up;

[0026] Figure 3 It is a flowchart of a data synchronization method on the destination node side shown according to an exemplary embodiment;

[0027] Figure 4 It is a schematic diagram of a data synchronization task record on a node shown according to an exemplary embodiment;

[0028] Figure 5 It is a flowchart of a data synchronization method on the source node side shown according to an exemplary embodiment;

[0029] Figure 6 It is a schematic diagram of the overall implementation of data synchronization shown according to an exemplary embodiment;

[0030] Figure 7 It is a schematic diagram of the hardware structure of an electronic device shown according to an exemplary embodiment;

[0031] Figure 8 It is a schematic diagram of the structure of a storage medium shown according to an exemplary embodiment. Detailed Embodiments

[0032] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of systems and methods consistent with some aspects of the present application as detailed in the appended claims.

[0033] The terms used in this application are for the purpose of describing particular embodiments only and are not intended to limit the present application. The singular forms "a", "said", and "the" used in this application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0034] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "upon" or "in response to determining".

[0035] As described above, in the prior art, a scheme of maintaining a sequence information by one node is adopted. In this way, if the cluster undergoes configuration changes, that is, the cluster shrinks or expands, it is very easy to cause the sequence information maintained by the node to change, resulting in sequence information conflict problems during the data synchronization process, and further leading to problems such as command loss and command duplicate writing.

[0036] As Figure 1The cluster scaling-down as shown causes the problem of command loss. Assume that node B1 in cluster B needs to synchronize data to node A1 in cluster A. Node B1 sequentially sends data update commands with serial numbers 100 - 104 to node A1. When node A1 receives the data update command with serial number 100, the serial number maintained by node A1 is 100. At this time, cluster A needs to stop node A2 for scaling-down, and migrates the slot slot2 on node A2 to node A1 through slot migration. Since the cluster scaling-down adopts incremental slot migration, the serial number 106 maintained by node A2 will also be carried to node A1 along with slot slot2. As a result, the serial number maintained by node A1 changes from 100 to 106. Then, when node A1 subsequently receives data update commands with serial numbers 101 - 104, because the serial numbers 101 - 104 are all smaller than the serial number 106 maintained by node A1, node A1 will skip the data update commands with serial numbers 101 - 104, resulting in the loss of commands that have not been synchronized yet.

[0037] As Figure 2 The cluster scaling-up as shown causes the problem of duplicate command writing. Assume that node B1 in cluster B needs to synchronize data to node A2 in cluster A. Node B1 sends data update commands with serial numbers 80 - 106 to node A2, and these data update commands are all written to slot slot2 on node A2. Therefore, the latest serial number maintained by node A2 is 106. At this time, a new empty node A1 is added to cluster A for scaling-up. The initial serial number maintained by node A1 is 0, and the slot slot2 on node A2 is migrated to node A1 through slot migration. Since the cluster scaling-up adopts full-scale slot migration, the serial number maintained by node A2 will not be carried to node A1 along with slot slot2. At this time, the serial number maintained by node A1 is still 0. Assume that node B1 sends a data update command with serial number 107 to node A2 again, and this data update command is for slot slot2. Since slot slot2 no longer exists on node A2, node A2 returns the slot allocation information of cluster A to node B1. Node B1 reverts the data synchronization back to serial number 80 according to the serial number 80 recorded for data synchronization by itself, that is, node B1 sequentially sends data update commands with serial numbers 80 - 106 to node A1 again. Because the serial number maintained by node A1 is 0, these data update commands will all be written to slot slot2 on node A1 repeatedly, causing the problem of duplicate command writing.

[0038] To solve the above technical problems, the present application proposes a data synchronization method. By precise positioning of the sequence information used for data synchronization to the smallest data partition (i.e., slot), that is, maintaining the sequence information at the node according to the slot dimension, rather than maintaining a sequence information for each node, and the sequence information corresponding to each slot is independent of each other. When the destination node receives a data synchronization command sent by the source node, it determines the target slot for the data synchronization command, and decides whether to write or discard the data synchronization command based on the target slot, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node. In this way, even if a cluster reconfiguration occurs, since the sequence information of a single slot does not change, the problem of command loss or duplicate writing can be reduced during the data synchronization process, and the data consistency between nodes can be ensured.

[0039] The following uses specific embodiments to detail the technical solutions of the present application and how the technical solutions of the present application solve the foregoing technical problems. The several specific embodiments listed may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following will describe the embodiments of the present application in detail with reference to the drawings.

[0040] Embodiment 1:

[0041] Figure 3 FIG. is a flowchart of an embodiment of a data synchronization method shown according to an exemplary embodiment. The execution subject of this embodiment is on the destination node side. The destination node can be considered as the destination end of data synchronization, and includes the following steps:

[0042] Step 301: Receive a data synchronization command sent by the source node.

[0043] Step 302: Determine the target slot for the data synchronization command.

[0044] Step 303: Determine whether to write or discard the data synchronization command based on the target slot, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node.

[0045] In the embodiment of the present application, the source node and the destination node may be data nodes in two different clusters respectively. The two data nodes need to complete data synchronization to maintain data consistency.

[0046] The data synchronization command is an instruction used to implement the same data operations on the destination node as on the source node. That is to say, the data operations on the source node and the destination node should be kept synchronized to achieve the purpose of data consistency between the destination node and the source node. This data synchronization command includes two parts: command header information and data operation statement information. The command header information contains the synchronization task ID between the source node and the destination node and the sequence information assigned by the source node for this command, that is, the target sequence information. The data operation statement information includes the operation instruction type, the key (representing the index of the data in the database) to be operated on, and the corresponding key value. For example, the data synchronization command "ars seq(task ID, 100) set k v", where "seq(task ID, 100)" represents the command header information and "set k v" represents the data operation statement, indicating that the key value corresponding to the key key = k is set to v.

[0047] The target slot for the data synchronization command refers to the slot where this data synchronization command is to be written. Here, it needs to be explained that in a distributed database cluster, a slot is a logical container for storing data. A slot can be regarded as a data shard or data partition, used to divide and store data on different nodes according to certain rules. The number of slots is usually fixed, and each slot is assigned to a node. When data needs to be stored or queried, the distributed database cluster will route the command to the corresponding node for operation according to the slot rules.

[0048] The sequence information corresponding to each slot of the synchronization task record for the source node in the destination node refers to the sequence information maintained for each slot managed by the destination node for the synchronization task of the source node. The sequence information corresponding to each slot is independent of each other. For example, the sequence information seq_num = 101 maintained for slot slot1 indicates that the sequence information of the latest data synchronization command written by the source node on slot slot1 is 101.

[0049] For example, as Figure 4 shown, the destination node belongs to cluster A, the source node belongs to cluster B, the task ID of the synchronization task between the destination node in cluster A and the source node in cluster B is abcd987. There are five slot slots 101 - 105 on the destination node. For the task ID: abcd987, the sequence information maintained for these five slot slots is respectively: 50, 60, 70, 80, 90. At the same time, for the task ID: abcd987, the timestamp information carried by the previous data synchronization command from this source node in cluster B is also recorded.

[0050] Furthermore, by Figure 4It can be seen that on the destination node in Cluster A, sequence information and timestamp information corresponding to and maintained for each slot of the synchronization task with nodes in other clusters are also recorded, and the task ID is abcd990.

[0051] Thus far, through the Figure 3 data synchronization process shown above, data consistency between two nodes is achieved. By making the sequence information used for data synchronization accurate to the smallest data partition, that is, maintaining sequence information at the slot dimension on the node instead of maintaining one sequence information for one node, the sequence information corresponding to each slot is independent of each other. When the destination node receives a data synchronization command sent by the source node, it determines the target slot for the data synchronization command and decides whether to write or discard the data synchronization command based on the target slot, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot in the node. In this way, even if a cluster reconfiguration occurs, since the sequence information of a single slot does not change, the problem of command loss or duplicate writing can be reduced during the data synchronization process.

[0052] In some embodiments of the present application, for the process of determining the target slot for the data synchronization command, the key to be operated in the data synchronization command can be obtained, and the slot index can be calculated using this key, so as to determine the slot indicated by the slot index as the target slot for the data synchronization command.

[0053] Among them, the key to be operated represents the index of the data in the database. In specific implementation, the slot index to which the key to be operated belongs can be calculated through the CRC16 (Cyclic Redundancy Check) operation method. For example, the cluster consists of three nodes, the total number of slots corresponding to the entire cluster is 16384, and the corresponding slot indices are 0 - 16383. Suppose the slot indices assigned to Node 1 are 0 - 4999, the slot indices assigned to Node 2 are 5000 - 9999, and the slot indices assigned to Node 3 are 10000 - 16383. The data synchronization command is to execute the set key1 value1 command, and the key to be operated is key1. Through CRC16(key1)&16383 = 8876, it is considered that the slot index of the target slot for the data synchronization command is 8876.

[0054] As a feasible embodiment, before determining the target slot for which the data synchronization command is targeted, that is, before executing step 302, the first timestamp information carried in the data synchronization command can be obtained, and the second timestamp information carried in the previous data synchronization command from the source node can be obtained. By comparing the first timestamp information and the second timestamp information, if the first timestamp information is greater than the second timestamp information, it indicates that the data synchronization command is a new command, and then step 302 is executed. If the first timestamp information is less than or equal to the second timestamp information, it indicates that the data synchronization command is an expired command, and an error notification of command expiration is returned to the source node.

[0055] Among them, the first timestamp information refers to the time when the source node sends the command. The second timestamp information is the time when the source node sends the previous command. By using the timestamp information of the command for judgment and comparison, this embodiment can exclude the expired commands between the destination node and the source node.

[0056] In some embodiments of the present application, for the process of determining whether to write or discard the data synchronization command according to the target slot, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node, the following two cases can be divided:

[0057] The first case is when the target slot belongs to the management of the destination node. The sequence information corresponding to the target slot is obtained from the sequence information corresponding to each slot recorded in the synchronization task for the source node, and then according to the sequence information corresponding to the target slot and the target sequence information, it is determined whether to write or discard the data synchronization command.

[0058] The second case is when the target slot does not belong to the management of the destination node. The data synchronization command is discarded, and the slot allocation information of the cluster where the destination node is located is returned to the source node, so that the source node can send the data synchronization command to the target node to which the target slot belongs according to the slot allocation information.

[0059] Among them, the first case is the synchronization situation before the cluster reconfiguration. The target slot for which the data synchronization command is targeted is still on the destination node. The second case is the synchronization situation after the cluster reconfiguration. The target slot for which the data synchronization command is targeted has migrated to other nodes, and the target slot does not belong to the management of the destination node. Therefore, the slot allocation information of the cluster has changed. At this time, the destination node cannot process the data synchronization command, and the new slot allocation information needs to be returned to the source node so that the source node can send the data synchronization command to the target node to which the target slot belongs for processing.

[0060] It should be noted here that the cluster reconfiguration referred to in the second case can be cluster expansion or cluster contraction, and the present application does not make specific limitations on this. However, it should be noted that while the target slot is migrated to other nodes, since the sequence information corresponding to each slot is independent of each other, the sequence information corresponding to the target slot maintained for different synchronization tasks on the destination node will also be sent to other nodes along with the slot migration, but will not be changed due to cluster reconfiguration.

[0061] In a specific embodiment, for the process of determining whether to write or discard the data synchronization command according to the sequence information corresponding to the target slot and the target sequence information, by comparing the magnitude relationship between the sequence information corresponding to the target slot and the target sequence information, if the sequence information corresponding to the target slot is less than the target sequence information, the data synchronization command is written; if the sequence information corresponding to the target slot is greater than or equal to the target sequence information, it is considered that the data synchronization command is a duplicate command, and the data synchronization command is discarded.

[0062] Among them, the sequence information corresponding to the target slot represents the sequence information of the latest data synchronization command written on the target slot by the source node. If the sequence information corresponding to the target slot is greater than or equal to the target sequence information carried by the current data synchronization command, it indicates that the current data synchronization command is a duplicate command during the task failure retry process.

[0063] In this embodiment, writing the data synchronization command means operating on the corresponding keys and key values on the target slot according to the data operation statements in the data synchronization command, and at the same time recording the data synchronization command in the form of a log.

[0064] It should be noted that after writing the data synchronization command, the sequence information corresponding to the target slot recorded in the synchronization task for the source node in the destination node can be updated using the target sequence information carried by the data synchronization command.

[0065] Embodiment 2:

[0066] Based on the above Figure 3 illustrated embodiment, Figure 5 As shown in the flowchart of another data synchronization method according to an exemplary embodiment, the execution subject of this embodiment is on the source node side. The source node can be considered as the source end of data synchronization, and includes the following steps:

[0067] Step 501: Generate a data synchronization command according to the local data update command log.

[0068] Step 502: Send the data synchronization command to the destination node, so that the destination node determines whether to write or discard the data synchronization command according to the target slot targeted by the data synchronization command, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node.

[0069] In the embodiments of the present application, all executed write operations are usually recorded in the form of logs on the node. To ensure data consistency between the source node and the destination node, a data synchronization command is generated using the data update command log during the data synchronization process, such as the binlog log (a binary log).

[0070] Based on the above-mentioned Embodiment 2, on the source node, a data synchronization command is generated through the local data update command log and sent to the destination node. Since the sequence information used for data synchronization on the destination node is accurate to the smallest data partition (i.e., the slot), that is, the sequence information is maintained according to the slot dimension, rather than a single sequence information for a single node, even if a cluster reconfiguration occurs during the data synchronization process, the sequence information of a single slot will not change. On the destination node, it is still possible to determine whether to write or discard the data synchronization command according to the target slot targeted by the data synchronization command, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node, so as to complete the data synchronization and ensure data consistency between nodes.

[0071] In some embodiments of the present application, the process of generating a data synchronization command according to the local data update command log can be divided into the following two synchronization forms:

[0072] The first is the full - volume synchronization form: By rewriting the local data update command log, a new data update command log is obtained, and then a data synchronization command is generated using the data update commands included in the new data update command log.

[0073] Among them, log rewriting means that multiple commands in the old log become one command in the new log obtained after rewriting. For example, in the data update command log, before log rewriting, for a key-key value pair, 6 data update commands are recorded in chronological order: set key1 v1; set key1 v2; set key1 v3; set key1 v4; set key1 v5; set key1 v6; that is to say, the change process of the value value in the key-value pair is v1→v2→v3→v4→v5→v6. After log rewriting, in the new data update command log obtained, for this key-value pair, only 1 data update command is recorded: set key1 v6 (that is, the last data update command among the 6 data update commands). Thus, it can be seen that by performing log rewriting on the data update command log, the data synchronization commands transmitted during the data synchronization process can be reduced.

[0074] The above full synchronization form can be the initial synchronization process between the source node and the destination node, and it is necessary to synchronize all the data on the source node to the destination node.

[0075] The second is the incremental synchronization form: generating data synchronization commands by using the data update commands recorded in the latest record of the local data update command log.

[0076] Among them, the data update commands recorded in the latest record are the records of the data update operations currently executed on the source node. That is to say, when a data update command is executed on the source node, it will be synchronized to the destination node once.

[0077] The above incremental synchronization form can be the synchronization process after the full synchronization of the initial synchronization process between the source node and the destination node.

[0078] It should be noted that the above two synchronization forms are only for the source node. For the destination node, whether it is the full form or the incremental form, the received are all data synchronization commands.

[0079] Based on the above Figure 3 and Figure 5 shown in the embodiments, Figure 6 FIG. is an overall schematic diagram of implementing data synchronization shown according to an exemplary embodiment. Node B1 belonging to cluster B is used as the source node, and node A1 belonging to cluster A is used as the destination node. The task ID of the synchronization task between cluster B and cluster A is B1. The entire data synchronization is divided into a full synchronization process and an incremental synchronization process:

[0080] During the full synchronization process, node B1 performs log rewriting on the local data update command log to obtain a new data update command log, generates a data synchronization command using the data update commands included in the new data update command log, and sends it to node A1. For example, assume that the data update command recorded in the new data update command log is: set k v, and the corresponding sequence information is 80. A data synchronization command: arsseq(B1, 80)set k v is generated using this data update command, the corresponding sequence information, and the task ID. Further, when node A1 receives this data synchronization command, it determines the target slot slot1 and decides to write this data synchronization command, thereby updating on node A1 the sequence information corresponding to slot1 recorded for the synchronization task: B1: seq_num = 80.

[0081] During the incremental synchronization process, node B1 generates a data synchronization command using the data update command recorded most recently in the data update command log and sends it to node A1. For example, assume that the data update command currently executed by node B1 is: set k1 va, and the corresponding assigned sequence information is 100. A data synchronization command: ars seq(B1, 100)set k1 va is generated using this data update command, the corresponding sequence information, and the task ID. Further, when node A1 receives this data synchronization command, it determines the target slot slot1 and decides to write this data synchronization command, thereby updating on node A1 the sequence information corresponding to slot1 recorded for the synchronization task: B1: seq_num = 100.

[0082] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0083] Corresponding to the embodiment of the foregoing data synchronization method, this application also provides a data synchronization system, which includes a source node and a destination node. Among them, the destination node is used to execute the data synchronization method provided by the embodiment shown above Figure 3 and the source node is used to execute the data synchronization method provided by the embodiment shown above Figure 5 shown.

[0084] The embodiment of this application also provides an electronic device corresponding to the data synchronization method provided by the foregoing embodiment to execute the above data synchronization method.

[0085] Figure 7The figure is a hardware structure diagram of an electronic device according to an exemplary embodiment. The electronic device includes a communication interface 601, a processor 602, a memory 603, and a bus 604. Among them, the communication interface 601, the processor 602, and the memory 603 complete communication with each other through the bus 604. By reading and executing the machine-executable instructions corresponding to the control logic of the data synchronization method in the memory 603, the processor 602 can execute the data synchronization method described above. For the specific content of this method, refer to the above embodiments and will not be repeated here.

[0086] The memory 603 mentioned in this application can be any electronic, magnetic, optical, or other physical storage system, and can store information such as executable instructions, data, etc. Specifically, the memory 603 can be a RAM (Random Access Memory), a flash memory, a storage drive (such as a hard disk drive), any type of storage disk (such as an optical disk, a DVD, etc.), or a similar storage medium, or a combination thereof. Through at least one communication interface 601 (which can be wired or wireless), a communication connection is realized between this system network element and at least one other network element, and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.

[0087] The bus 604 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Among them, the memory 603 is used to store a program, and the processor 602 executes the program after receiving an execution instruction.

[0088] The processor 602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 602 or by instructions in software form. The above-mentioned processor 602 can be a general-purpose processor, including a network processor (abbreviated as NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of this application can be directly embodied as being executed by a hardware decoding processor or by a combination of hardware and software modules in the decoding processor.

[0089] The electronic device provided in the embodiments of this application and the data synchronization method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run, or implemented by them.

[0090] An embodiment of the present application also provides a computer-readable storage medium corresponding to the data synchronization method provided in the foregoing embodiment. Please refer to Figure 8 As shown, the computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the data synchronization method provided in any of the foregoing embodiments.

[0091] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other optical and magnetic storage media, which will not be elaborated here one by one.

[0092] The computer-readable storage medium provided in the above embodiments of the present application and the data synchronization method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run, or implemented by the application programs stored therein.

[0093] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include common general knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and embodiments are only to be regarded as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0094] It should also be noted that the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity, or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity, or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, commodity, or device comprising the element.

[0095] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A data synchronization method, characterized in that, Applied to the destination node, the method includes: Receiving a data synchronization command sent by the source node; Determining the target slot for which the data synchronization command is targeted; Determining whether to write or discard the data synchronization command according to the target slot, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node.

2. The method according to claim 1, wherein The determining whether to write or discard the data synchronization command according to the target slot, the target sequence information carried in the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node includes: When the target slot is under the management of the destination node, obtaining the sequence information corresponding to the target slot from the sequence information corresponding to each slot recorded in the synchronization task for the source node; determining whether to write or discard the data synchronization command according to the sequence information corresponding to the target slot and the target sequence information; When the target slot is not under the management of the destination node, discarding the data synchronization command and returning the slot allocation information of the cluster where the destination node is located to the source node, so that the source node sends the data synchronization command to the target node to which the target slot belongs according to the slot allocation information.

3. The method according to claim 2, wherein The determining whether to write or discard the data synchronization command according to the sequence information corresponding to the target slot and the target sequence information includes: Writing the data synchronization command when the sequence information corresponding to the target slot is less than the target sequence information; Discarding the data synchronization command when the sequence information corresponding to the target slot is greater than or equal to the target sequence information.

4. The method according to claim 1, wherein Before determining the target slot for which the data synchronization command is targeted, the method further includes: Obtaining the first timestamp information carried in the data synchronization command, where the first timestamp information indicates the time when the source node sends the command; Obtaining the second timestamp information carried in the previous data synchronization command from the source node; When the first timestamp information is greater than the second timestamp information, performing the step of determining the target slot for which the data synchronization command is targeted; When the first timestamp information is less than or equal to the second timestamp information, returning an error notification indicating that the command has expired to the source node.

5. The method according to claim 1, characterized in that The determining the target slot for which the data synchronization command is targeted includes: Obtaining the key to be operated on in the data synchronization command, where the key represents the index of the data in the database; Calculating the slot index using the key; Determining the slot indicated by the slot index as the target slot for which the data synchronization command is targeted.

6. The method according to any one of claims 1-5, characterized in that, After writing the data synchronization command, the method further includes: Updating the sequence information corresponding to the target slot recorded in the synchronization task for the source node in the destination node using the target sequence information carried in the data synchronization command.

7. A data synchronization method, characterized in that, Applied to the source node, the method further includes: Generating a data synchronization command according to the local data update command log; Send the data synchronization command to the destination node, so that the destination node determines whether to write or discard the data synchronization command according to the target slot targeted by the data synchronization command, the target sequence information carried by the data synchronization command, and the sequence information corresponding to each slot recorded in the synchronization task for the source node in the destination node.

8. The method according to claim 7, characterized in that Generating a data synchronization command according to the local data update command log includes: Rewriting the data update command log to obtain a new data update command log, and generating a data synchronization command using the data update commands included in the new data update command log; or, Generating a data synchronization command using the data update command recorded most recently in the data update command log.

9. A data synchronization system, characterized in that, The system includes a source node and a destination node; The destination node is configured to execute the method according to any one of claims 1-6 above; The source node is configured to execute the method according to any one of claims 7-8 above.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method according to any one of claims 1-8 above.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method according to any one of claims 1-8 above.

Images