Data synchronization method and device, medium and product

Abstract

The invention discloses a data synchronization method and device, a medium and a product. The method comprises the following steps: sending a data change log to a first type area standby node through a main node of a first area; sending the data change log to at least one second type area standby node in the same area through the first type area standby node in each area on the basis of a distributed information propagation protocol, so that the second type area standby node receiving the data change log sends the data change log to the at least one second type area standby node in the same area; and sending the data change logs to the second type area standby nodes which do not receive the data change logs in the same area to complete data synchronization. According to the technical scheme, the problem that the calculation pressure of the main node is large in current data synchronization is solved, data synchronization can be carried out only through the main node and the unique area standby node of each area, data synchronization is carried out among the internal nodes of each area through a distributed information propagation protocol, the calculation pressure of the main node is relieved, and the data synchronization efficiency is improved. The use of cross-regional bandwidth resources is reduced, and the expandability of a database system is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of distributed database, and particularly relate to a data synchronization method, device, medium and product. BACKGROUND

[0002] At present, under the background of global service demand, a distributed database system usually adopts a deployment mode across multiple geographic regions to achieve proximity service and scalability, and achieves read-write separation and high availability through a master-slave, master-slave replication architecture.

[0003] However, the master node needs to establish an independent replication connection with each standby node in each region. When the number of standby nodes is large, the master node needs to maintain a large number of network connections, serialize logs and send them, and the CPU, memory and network export bandwidth of the master node will become the performance bottleneck of the system to provide external services.

[0004] The same data update log needs to be sent by the master node to each standby node in each region through the network. This means that for one data update, the same data packet will be repeatedly transmitted multiple times on the cross-region network link, causing a great waste of expensive cross-region bandwidth resources.

[0005] With each new region or new standby node, the master node needs to add a new replication link, which brings greater pressure to the master node, and the scalability of the system is severely restricted. SUMMARY

[0006] Embodiments of the present application provide a data synchronization method, device, medium and product to solve the problem of large computing pressure of the master node in the current data synchronization, which can synchronize data through the master node only with the unique regional standby node in each region, and synchronize data through the distributed information propagation protocol between the nodes in each region, thereby reducing the computing pressure of the master node, reducing the use of cross-region bandwidth resources, and improving the scalability of the database system.

[0007] In a first aspect, embodiments of the present application provide a data synchronization method applied to a distributed database system, the method comprising:

[0008] sending, by a master node in a first region, a data change log to a first type regional standby node in the first region and at least one second region, wherein the first region and each second region each include a first type regional standby node and at least one second type regional standby node;

[0009] In the first region and the second region, the first type region backup node sends the data change log to at least one second type region backup node in the same region based on the first distributed information propagation protocol, so that the second type region backup node receiving the data change log sends the data change log to the second type region backup node in the same region which does not receive the data change log, and data synchronization is completed.

[0010] In a second aspect, an embodiment of the present application further provides an electronic device, which comprises:

[0011] at least one processor; and

[0012] a memory connected with the at least one processor in communication; wherein,

[0013] the memory stores a computer program which can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the data synchronization method in any of the embodiments of the present application.

[0014] In a third aspect, an embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to enable a processor to implement the data synchronization method in any of the embodiments of the present application when the processor executes the computer instructions.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer program product, which comprises a computer program, and the computer program is used to implement the data synchronization method in any of the embodiments of the present application when the computer program is executed by a processor.

[0016] In the embodiment of the present application, the master node of the first region sends the data change log to the first type region backup node in the first region and at least one second region, wherein the first region and each second region comprise one first type region backup node and at least one second type region backup node; in the first region and the second region, the first type region backup node sends the data change log to at least one second type region backup node in the same region based on the first distributed information propagation protocol, so that the second type region backup node receiving the data change log sends the data change log to the second type region backup node in the same region which does not receive the data change log, and data synchronization is completed. The technical scheme of the embodiment of the present application solves the problem that the master node has large computing pressure in the current data synchronization, and can reduce the computing pressure of the master node, reduce the use of cross-region bandwidth resources, and improve the scalability of the database system by enabling the master node to only synchronize data with the unique region backup node of each region and enabling the nodes in each region to synchronize data through the distributed information propagation protocol. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0018] Figure 1 A flow chart of a data synchronization method provided by the embodiment of the present application;

[0019] Figure 2 A node architecture schematic diagram of improved data synchronization provided by the embodiment of the present application;

[0020] Figure 3 A flow chart of a data synchronization method provided by the embodiment of the present application;

[0021] Figure 4 A node architecture schematic diagram of improved data synchronization provided by the embodiment of the present application;

[0022] Figure 5 A fault recovery flow schematic diagram provided by the embodiment of the present application;

[0023] Figure 6 A structure schematic diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0024] The present application will be further described in detail below in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and not to limit the present application. In addition, it should be noted that, in order to facilitate the description, only the parts related to the present application are shown in the drawings, and not all the structures.

[0025] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. The acquisition, storage, use, processing, etc. of data in the technical solutions of the present application all comply with the relevant provisions of the national laws and regulations.

[0026] It should be noted that, in the embodiments of the present application, some software, components, models, etc. of the prior art may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the technical solutions of the present application, but it does not mean that the applicant has or will necessarily use the related content of the scheme.

[0027] Figure 1 This is a flowchart illustrating a data synchronization method provided in an embodiment of the present invention. This embodiment is applicable to data synchronization scenarios. The method can be applied to distributed database systems.

[0028] like Figure 1 As shown, the data synchronization method includes the following steps:

[0029] S110. Send the data change log to the first region and at least one first-type region standby node in the second region through the master node of the first region.

[0030] This embodiment can be applied to a distributed database system spanning multiple geographical regions. It includes a first region providing read-write services and at least one second region providing only read-only services. Each of the first and second regions includes one first-type region backup node and at least one second-type region backup node.

[0031] like Figure 2 As shown, in the existing master-slave architecture, the master node needs to establish independent replication connections with every slave node in all regions. Even for the same data update log, the master node needs to send it to every slave node in every region via the network. Furthermore, each time a new region or a new slave node is added, the master node needs to add a new replication link.

[0032] In this embodiment, to reduce the data replication pressure on the master node and reduce the use of cross-regional network bandwidth resources, a unique first-type regional backup node is set up in each region to establish a replication link with the master node. The master node only needs to send data change logs to the first-type regional backup node, and then the first-type regional backup node sends data change logs within the region until all nodes complete the synchronization of data change logs.

[0033] S120. In the first and second regions, the data change log is sent from the first type of regional backup node to at least one second type of regional backup node in the same region based on the first distributed information propagation protocol, so that the second type of regional backup node that receives the data change log can send the data change log to the second type of regional backup node in the same region that has not received the data change log, thereby completing data synchronization.

[0034] Distributed information propagation protocols can be broadcast protocols such as Gossip, consensus protocols such as Paxos and Raft, or routing protocols such as SPF and BGP. This embodiment does not limit these.

[0035] The first type of area backup node in the first area and the second area receives the data change log sent by the master node, and sends the data change log to the second type of area backup node in the same area based on the first distributed information propagation protocol. The second type of area backup node that receives the data change log repeats the data sending process based on the first distributed information propagation protocol, and sends the data change log to the second type of area backup node in the same area that has not received the data change log.

[0036] The second type of area backup node that receives the data change log repeats the data sending process, and completes data synchronization in the area. After all areas complete data synchronization, the distributed data system completes data synchronization.

[0037] The master node only sends the data change log to the area backup machine that is unique in each area and establishes a replication connection with the master node. The master node no longer needs to maintain replication connections for all backup nodes, but only needs to maintain replication connections equal to the number of areas, reducing the CPU, memory and network connection overhead of the master node, and improving the processing capacity and stability of the master node.

[0038] The technical scheme of the embodiment sends the data change log to the first type of area backup node in the first area and at least one second area through the master node of the first area; the first area and each second area include a first type of area backup node and at least one second type of area backup node; in the first area and the second area, the first type of area backup node sends the data change log to at least one second type of area backup node in the same area based on the first distributed information propagation protocol, so that the second type of area backup node that receives the data change log sends the data change log to the second type of area backup node in the same area that has not received the data change log, and completes data synchronization. The technical scheme of the embodiment solves the problem of large computing pressure of the master node in the current data synchronization, can perform data synchronization between the master node and the unique area backup node in each area, and can perform data synchronization between the nodes in each area through the distributed information propagation protocol, thereby reducing the computing pressure of the master node, reducing the use of cross-area bandwidth resources, and improving the scalability of the database system.

[0039] Figure 2 A flowchart of a data synchronization method provided by the embodiment of the application is provided. The data synchronization method in the embodiment and the data synchronization method in the above embodiment belong to the same inventive concept, and further describes the process of handling node failure. The method can be executed by a data synchronization device, which can be implemented in software and / or hardware, integrated in an electronic device with application development function, and applied to a distributed database system.

[0040] As shown in Figure 3 the data synchronization method includes the following steps:

[0041] S210, sending the data change log by the master node of the first region to the first type region standby node in the first region and at least one second region.

[0042] As shown in the figure, the master node A-0 of the first region sends the data change log generated by the read-write service to the first type standby node A-1 in the region, i.e. the first region, and the first type standby node B-1, N-1 in the other region, i.e. the second region, respectively, forming a hierarchical "tree" topology and breaking the original "star" topology with one master and multiple slaves. Figure 4

[0043] S220, in the first region and the second region, sending the data change log by the first type region standby node based on the first distributed information propagation protocol to at least one second type region standby node in the same region, so that the second type region standby node receiving the data change log sends the data change log to the second type region standby node in the same region that has not received the data change log, to complete the data synchronization.

[0044] The first distributed information propagation protocol can be a gossip communication protocol. The second type region standby node of the region is not directly connected with the master node, but is connected with the first type region standby node, other standby nodes and a gossip communication module to form a gossip communication ring in the region. Each standby node is provided with a gossip communication module. The first type region standby node can send the data change log to the second type region standby node in the same region in a one-to-many or one-to-one manner; the second type region standby node also continues to send data to the node that has not received the data change log based on the one-to-many or one-to-one manner.

[0045] ​In an alternative embodiment, the first type of regional backup node sends the data change log to at least one second type of regional backup node in the same region based on the first distributed information propagation protocol, so that the second type of regional backup node receiving the data change log sends the data change log to the second type of regional backup node in the same region that has not received the data change log, and completes data synchronization. The first type of regional backup node can select one node as a target node in the second type of regional backup node in the same region based on the first distributed information propagation protocol, and send the data change log to the target node, so that the target node repeats the data synchronization operation of the first type of regional backup node, sends the data change log to the second type of regional backup node in the same region that has not received the data change log, and completes data synchronization. Alternatively, the first type of regional backup node can randomly select at least two nodes as target nodes in the second type of regional backup node in the same region based on the first distributed information propagation protocol, and send the data change log to the target nodes, so that the at least two target nodes repeat the data synchronization operation of the first type of regional backup node, send the data change log to the second type of regional backup node in the same region that has not received the data change log, and complete data synchronization.

[0046] As shown in Figure 4 Taking the read-write service region A as an example, when the primary node A-0 generates a data change log, the log is first synchronized to the first type of backup node A-1 in the same region. At this time, the backup node A-1 executes the “one-to-one” strategy based on the Gossip communication protocol: selects one target node in the second type of backup node in the same region, such as A-2, A-n, etc., and sends the change log to the node A-2, which is determined as the target node in the figure.

[0047] Subsequently, the backup node A-2 will repeat this process, taking itself as a new propagation node, and select another target node in the backup node in the region A that has not received the log, such as A-3 in the figure, to forward the log, and so on. Through the Gossip diffusion mode of node-by-node unicast, the data synchronization of all backup nodes in the region A is gradually completed.

[0048] Taking the read-only service region B as an example, when the first type of backup node B-1 in the region B receives the change log from the primary node A-0 in the region A, it executes the “one-to-many” strategy based on the Gossip communication protocol: randomly selects at least two target nodes such as B-2 and B-3 in the second type of backup node in the same region, such as B-2, B-3, B-n, etc., and sends the data change log to the two nodes.

[0049] The two target nodes B-2 and B-3 will immediately reuse the same strategy, each in the standby nodes in the region B that do not receive the log, and randomly select multiple nodes to forward the log, through the multi-node parallel diffusion characteristics of the Gossip protocol, quickly realize the change log synchronization of all standby nodes in the region B.

[0050] S230, in response to the failure of the node in the first region or the second region, determining a replacement node for replacing the failed node in the nodes in the same region as the failed node.

[0051] The communication failure of the node is monitored, for example, in each region, the standby node failure in the region can be sensed in time through the Gossip communication module of the node, and the node will periodically send a state heartbeat packet to other standby nodes in the same region through the Gossip protocol, and receive the heartbeat feedback of the adjacent node. If a standby node such as the standby node B-2 in the region B fails, its Gossip module will stop sending the heartbeat, and other nodes in the same region such as B-1 and B-3 can sense the failure state of the standby node through the Gossip communication module after not receiving the heartbeat packet of the node within a preset time, thereby triggering the subsequent failure node replacement process in time.

[0052] In this embodiment, the failure node replacement can be adopted when the master node or the first type of region standby node fails, so as to restore the normal communication of the region.

[0053] In the failure node replacement process, the replacement node for replacing the failed node is determined in the nodes in the same region as the failed node, because the cross-region switching will bring greater switching cost.

[0054] It can be understood that the level of the master node is the highest, the first type of region standby node is the second, and the second type of region standby node is the lowest. Different levels of node failure can adopt different replacement strategies, for example, the replacement node is determined in the adjacent low-level nodes in the same region. In an optional embodiment, the failed node is the first type of region standby node, and the replacement node for replacing the failed node is determined in the nodes in the same region as the failed node, which can be that the first replacement node for replacing the failed node is determined in the second type of region standby node in the region where the failed node is located.

[0055] When the failed node is the first type of region standby node in a region, such as the standby node B-1 in the region B, one node in the second type of region standby node B-2, B-3... B-N in the region is selected as the first replacement node, such as the standby node B-2 in the region B.

[0056] After selection, the first replacement node will inherit the first type of backup node responsibilities of the original failed node, on the one hand, receive the data change log of the master node A-0, and on the other hand, based on the Gossip protocol, execute the data synchronization strategy, continue to complete the data synchronization of other backup nodes in the region B, and guarantee the service continuity in the region.

[0057] In an optional implementation, in the second type of regional backup nodes in the region where the failed node is located, determining the first replacement node for replacing the failed node can be, according to at least one of the preset node priority, the node data version and the load status of the second type of regional backup nodes, determining the replaceability of the second type of regional backup nodes for the failed node; and the second type of regional backup node with the highest replaceability is determined as the first replacement node for replacing the failed node.

[0058] The key information of each second type of backup node is collected, including the preset node priority, such as the inherent priority corresponding to the deployment level of the hardware configuration, the node data version, which can reflect the synchronization integrity with the master node data, the current load status, such as CPU occupancy, network bandwidth occupancy, etc., and the replaceability of each node for the failed node is calculated based on at least one information, such as through a preset algorithm to quantitatively weight at least one information, to obtain the replaceability score of each second type of backup node for the failed node, and finally the node with the highest score is determined as the first replacement node, which will immediately inherit all the responsibilities of the original first type of backup node, including receiving the latest data change log issued by the master node, executing data synchronization based on the Gossip protocol, and ensuring the data consistency and service continuity of all backup nodes in the region B are not affected by the failure.

[0059] The preset node priority directly improves the basic replaceability as an inherent attribute. The higher the priority, the more the node is originally adapted to the high-concurrency synchronization requirements of the first type of backup node, and the basic score usually has the highest proportion; the node data version determines the data validity, and the closer the synchronization progress with the master node data, the less data needs to be supplemented after replacement, the responsibility can be quickly taken over, significantly reducing the switching cost, which is the core scoring item of the replaceability; the load status reflects the running stability, and the lower the current CPU occupancy, memory usage, network bandwidth occupancy and task queue length, the more sufficient the remaining resources of the node, which can avoid synchronization delay or secondary failure after replacement due to high load, and is a key indicator to ensure service continuity after replacement. When the load exceeds the standard, the replaceability will be directly reduced or even lose the candidate qualification.

[0060] After the three are quantitatively weighted, the final replaceability score of each node is formed, and the higher the score, the more the node meets the requirements of the responsibilities of the first type of backup node.

[0061] Of course, the alternative node can also be determined by some indicators, such as selecting the node with the highest preset node priority or the node with the most recent data synchronization progress with the master node as the alternative node.

[0062] In an optional embodiment, the failed node is the master node of the first region, and among the nodes in the region of the failed node, the alternative node for replacing the failed node can be determined as follows: the first type regional backup node of the first region is determined as a second alternative node for replacing the master node; and a third alternative node is determined from the second type regional backup nodes of the first region for replacing the first type regional backup node of the first region.

[0063] When the failed node is the master node of the first region, such as the master node A-0 of the region A, the system will start a double-layer alternative node selection process to quickly restore the region service.

[0064] First, the first type regional backup node A-1 in the first region is directly determined as the second alternative node, which will immediately inherit all the core responsibilities of the original master node A-0, including generating data change logs, synchronizing data with cross-region nodes, and receiving state feedback from each backup node.

[0065] Meanwhile, a third alternative node is selected from the second type regional backup nodes of the first region, such as A-2, A-3,..., and A-n. In the selection, the preset priority, data version, consistency degree with the master node, current load idle rate, and other key indicators of each node are comprehensively considered to determine the node with the highest replaceability, such as A-3, as the third alternative node, so that it takes over the responsibilities of the original first type regional backup node (A-1), receives the change logs issued by the second alternative node (A-1), and completes the data synchronization of the remaining second type backup nodes in the region based on the Gossip protocol. Through the double-layer mechanism of master node replacement and first type backup node repositioning, the data consistency and service continuity of the first region are ensured.

[0066] When the data version of the first type regional backup node A-1 in the first region differs from that of the master node by more than a preset difference threshold, considering the difficulty of data completion, a new master node can be determined in at least one second region according to the preset cross-region switchable setting or the manually input cross-region switch instruction, and the original master node region is downgraded to a read-only service region.

[0067] In an optional embodiment, the failed node is the master node and the first type regional backup node of the first region, and among the nodes in the region of the failed node, the alternative node for replacing the failed node can be determined as follows: a fourth alternative node is determined from the second type regional backup nodes of the first region for replacing the master node; and a fifth alternative node is determined from the second type regional backup nodes of the first region except the fourth alternative node for replacing the first type regional backup node.

[0068] In an alternative embodiment, when the failed node is both the master node A-0 of the first region and the first-type regional backup node A-1, the system will start an emergency double-layer replacement process to quickly restore regional services.

[0069] First, a fourth replacement node for replacing the master node is selected from all second-type regional backup nodes of the first region, such as A-2, A-3, and A-n. The selection can also be based on preset priorities, data versions, current load idle rates, and other key indicators.

[0070] For example, the second-type regional backup node A-3 is determined as the fourth replacement node, which immediately inherits the core responsibilities of the original master node, including generating data change logs, synchronizing data to cross-regional nodes, and receiving state feedback from each backup node. Then, among the remaining second-type regional backup nodes of the first region, i.e., excluding the selected fourth replacement node A-3, the remaining backup nodes are evaluated based on the same evaluation dimensions, and the node with the highest score, such as A-4, is selected as the fifth replacement node, which takes over the responsibilities of the original first-type regional backup node A-1, receives the change logs issued by the fourth replacement node A-3, and completes data synchronization among the remaining second-type backup nodes within the region based on the Gossip protocol. Through the dual replacement mechanism of master node replacement and first-type backup node replacement, the impact of double failures on data consistency and service continuity in the first region is minimized.

[0071] S240, replacing the failed node by the replacement node.

[0072] Replacing the failed node by the replacement node requires determining the data differences between the replacement node and the failed node, and making up for these data differences by the replacement node. The replacement node needs to establish connections with other nodes according to its connection with the failed node and other nodes.

[0073] For example, the second-type regional backup node A-3 replaces the original master node A-0, and A-3 needs to establish data replication links with the first-type regional backup nodes B-1...N-1 of the second region to send data change logs to the first-type regional backup nodes through the newly established links.

[0074] S250, in response to the failure of all nodes in the first region, determining a third region for replacing the first region in at least one second region.

[0075] When all nodes in the first region fail and cannot autonomously recover services, the system will start an emergency replacement mechanism across regions. From at least one normally operating second region, such as region B, region C, etc., read-only service region, the overall load capacity of each region, the network link quality with the original first region, data synchronization integrity, and network distance, etc. Predefined policies such as the coverage of the original first region master node change log received by each second region standby node and the hardware resource redundancy are evaluated to select the most suitable region as the third region, such as region B, to fully replace the service functions of the original first region.

[0076] S260, in the third region, determine a new master node and a new first type of regional standby node.

[0077] Subsequently, the node role reconstruction process is started in the third region. Based on the preset priority of each standby node, data version integrity, real-time load status and other key indicators, the highest replaceable node is selected as the new master node through preset rules such as quantitative scoring calculation, and another optimal scoring node is selected from the remaining nodes as the new first type of regional standby node. The new master node will take over all read and write responsibilities of the original first region master node, and the new first type of regional standby node will be responsible for receiving the change log of the new master node and completing data synchronization within the third region based on the Gossip protocol, quickly restoring cross-region data interaction and service availability.

[0078] The data change log of the master node in this embodiment only needs to be transmitted once to each region, eliminating the waste of repeated transmission of the same data change log on cross-region links under the original architecture, fundamentally optimizing bandwidth utilization, and greatly reducing operating costs; within the region, the peer-to-peer and multi-path propagation characteristics of the Gossip protocol are used to quickly improve data synchronization efficiency and robustness; even if a certain internal standby node cannot connect to the regional standby node temporarily, it can still obtain the data change log from other standby nodes within the region, enhancing the fault tolerance of internal synchronization; provides a method for election and data synchronization recovery in the case of single node failure, multiple node failure within the region, and overall failure of the region, ensuring the high availability of the distributed database system; on the basis of ensuring strong consistency of data between the master node and each regional standby node, the Gossip protocol ensures the eventual consistency of data among all standby nodes, meeting the data consistency requirements of the distributed database system.

[0079] The technical scheme of the embodiment sends the data change log to the first type regional backup node in the first region and at least one second region through the master node in the first region; wherein the first region and each second region comprise one first type regional backup node and at least one second type regional backup node; in the first region and the second region, the data change log is sent to at least one second type regional backup node in the same region based on the first distributed information propagation protocol through the first type regional backup node, so that the second type regional backup node receiving the data change log sends the data change log to the second type regional backup node in the same region which does not receive the data change log, to complete data synchronization; in response to the failure of the node in the first region or the second region, the replacement node for replacing the failed node is determined among the nodes in the same region of the failed node; the failed node is replaced by the replacement node; in response to the failure of all nodes in the first region, the third region for replacing the first region is determined in at least one second region; in the third region, the new master node and the new first type regional backup node are determined. The technical scheme of the embodiment solves the problem of large calculation pressure of the master node in the current data synchronization, can reduce the calculation pressure of the master node by data synchronization between the master node and the unique regional backup node in each region, and guarantees the stability and robustness of the system through the replacement node election mechanism in the case of node failure.

[0080] In a preferred data synchronization instance, the data synchronization process is as shown in Figure 5

[0081] In the embodiment, the regional backup node refers to the first type regional backup node, and the internal backup node refers to the second type regional backup node. The state of all nodes is monitored to trigger the global failure switching process. In the case of single master node failure, the regional backup node of the master region is preferentially elected as the new master node, and the data consistency of the regional backup node of the master region and the original master node is checked, and further supplement or cross-region switching is decided according to the pre-defined strategy such as whether the cross-region can be switched.

[0082] In the case of failure of both the master node and the regional backup node of the master region, the internal backup node of the master region which is consistent with the original master data is elected as the new master node.

[0083] Otherwise, in the case of overall failure at the regional level, a backup node is elected as the new master node according to the pre-defined strategy such as network distance and the rule of preferential regional backup node and then internal backup node based on region.

[0084] ​After the new master node is successfully elected, the fault election and data synchronization recovery process of the regional backup nodes mentioned above is reused. For each region where the regional backup node has failed or where there are no longer any regional backup nodes after the regional backup node has become the master node, an internal backup node is elected as the new regional backup node. All regional backup nodes establish replication connections with the new master node, compare data differences, and catch up with incremental data changes. Within the region, stable data internal broadcasting and synchronization continue through the Gossip communication module. At this point, the fault switchover is completed, and the data synchronization of the entire system is restored to normal.

[0085] In the event of a regional backup node failure, or the original regional backup node being promoted to the primary node, a replacement node is determined from among the internal backup nodes within the region. The Gossip communication module within the region quickly detects the regional backup node failure and, based on predefined strategies such as specified node priority, data version, or load conditions, automatically elects an internal backup node within the region as the new regional backup node. This node then re-establishes a replication connection with the primary node, compares data differences, and catches up on incremental data changes.

[0086] Figure 6 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0087] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0088] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0089] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the data synchronization method.

[0090] In some embodiments, the data synchronization method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the data synchronization method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the data synchronization method by any other appropriate means, such as by means of firmware.

[0091] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0092] Computer programs for implementing the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program, when executed, enables the functions / acts specified in the flowcharts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package and partially on a remote machine or entirely on a remote machine or server.

[0093] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0094] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0095] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0096] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, and solves the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0097] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in sequence, or executed in different orders, as long as the desired results of the technical solutions of the present disclosure can be achieved, and the present disclosure is not limited herein.

[0098] The embodiments of the present application further provide a computer program product, comprising a computer program which, when executed by a processor, implements the data synchronization method provided by any of the embodiments of the present application.

[0099] The computer program code can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce the computer implemented process such that the

[0100] It should be understood that various forms of flow shown above can be used, re-ordered, added to, or deleted from without departing from the spirit of the present application. For example, the steps recited in the present application can be performed in parallel, in series, or in different orders as long as the desired results of the present application are achieved, and are not limited herein.

[0101] The specific embodiments have been shown and described for the purposes of illustrating the physiological principles of and the practical application of the present application. It is therefore contemplated that various modifications, combinations, sub-combinations and alternatives can occur to one skilled in the art without departing from the spirit and scope of the present application. It is intended that such modifications, combinations, sub-combinations and alternatives be included within the scope of the present application as set forth in the following claims.

Claims

1. A data synchronization method, characterized in that, Applied to distributed database systems, including: The data change log is sent from the master node of the first region to the first region and at least one backup node of the first type region in the second region, wherein the first region and each of the second regions includes one backup node of the first type region and at least one backup node of the second type region. Within the first and second regions, the data change log is sent from the first type of regional backup node to at least one second type of regional backup node in the same region based on the first distributed information propagation protocol. This enables the second type of regional backup node that receives the data change log to send the data change log to other second type of regional backup nodes in the same region that have not received the data change log, thereby completing data synchronization.

2. The method according to claim 1, characterized in that, The step of sending the data change log to at least one second-type regional backup node in the same region via the first type of regional backup node based on the first distributed information propagation protocol, so that the second-type regional backup node that receives the data change log sends the data change log to the second-type regional backup node in the same region that has not received the data change log, thereby completing data synchronization, includes: Based on a first distributed information propagation protocol, the first type of regional backup node selects a node as a target node from the second type of regional backup nodes in the same region, and sends the data change log to the target node. This causes the target node to repeat the data synchronization operation of the first type of regional backup node, sending the data change log to second type of regional backup nodes in the same region that have not received the data change log, thus completing data synchronization; or... According to the first distributed information propagation protocol, the first type of regional backup node randomly selects at least two nodes as target nodes from the second type of regional backup nodes in the same region, and sends the data change log to the target nodes, so that at least two target nodes repeat the data synchronization operation of the first type of regional backup node and send the data change log to the second type of regional backup nodes in the same region that have not received the data change log, thereby completing the data synchronization.

3. The method according to claim 1, characterized in that, The method further includes: In response to a node failure in the first region or the second region, a replacement node is determined from among the nodes in the same region as the failed node to replace the failed node. The faulty node is replaced by the alternative node.

4. The method according to claim 3, characterized in that, The faulty node is a first-type regional backup node. The step of determining a replacement node for the faulty node among nodes in the same region as the faulty node includes: Among the second type of backup nodes in the region where the faulty node is located, a first replacement node is determined to replace the faulty node.

5. The method according to claim 4, characterized in that, Among the second type of backup nodes in the region where the faulty node is located, determining a first replacement node to replace the faulty node includes: The substitutability of the second type of regional backup node for the faulty node is determined based on at least one of the following: preset node priority, node data version, and load status. The second type of backup node with the highest substitutability is used as the first replacement node to replace the faulty node.

6. The method according to claim 3, characterized in that, The faulty node is the master node in the first region. Determining a replacement node from among the nodes in the same region as the faulty node includes: The first type of region backup node in the first region is determined as the second alternative node to replace the main node; A third replacement node is determined from the second type of area backup nodes in the first region to replace the first type of area backup node in the first region.

7. The method according to claim 3, characterized in that, The faulty node is the primary node of the first region and the backup node of the first type of region. The step of determining a replacement node for the faulty node among nodes in the same region as the faulty node includes: In the second type of backup nodes in the first region, a fourth replacement node is determined to replace the master node; Among the second type of backup nodes in the first region, excluding the fourth replacement node, a fifth replacement node is determined to replace the first type of backup node.

8. The method according to claim 1, characterized in that, The method further includes: In response to a failure of all nodes in the first region, a third region is determined in at least one of the second regions to replace the first region; In the third region, a new master node and a new first-type region backup node are determined.

9. An electronic device, characterized in that, The system includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data synchronization method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data synchronization method of any one of claims 1-8.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the data synchronization method as described in any one of claims 1-8.

Images