Distributed database system disaster recovery method and system based on cross-cluster data synchronization

Abstract

The invention relates to the technical field of data synchronization, in particular to a distributed database system disaster recovery method and system based on cross-cluster data synchronization. The method comprises the steps that a target database cluster pulls source cluster metadata from a source database cluster based on a synchronization period, the target database cluster is any slave database cluster, and the source cluster metadata comprises total data information and total node information of the source database cluster; the target database cluster compares the target cluster metadata of the target database cluster with the source cluster metadata to determine difference data; the target database cluster determines a plurality of transactions based on the difference data and executes a data version synchronization task corresponding to each transaction, and the data version synchronization tasks are used for synchronizing data from the source database cluster. According to the invention, efficient and flexible cross-cluster data synchronization can be realized.

Description

Technical Field

[0001] This application relates to the technical field of data synchronization, and in particular, to a disaster recovery method and system for a distributed database system based on cross-cluster data synchronization. Background Art

[0002] In modern distributed database systems, cross-regional data synchronization is a key technology for achieving high availability, disaster recovery, and load balancing. With the expansion of business scale and the growth of data volume, the need for cross-regional data synchronization has become increasingly common. For example, global enterprises need to synchronize data between database clusters in different regions to ensure that users can access consistent data regardless of their location. In addition, cross-regional data synchronization can also improve the disaster recovery ability of the system. When a cluster in one region fails, clusters in other regions can take over the service to ensure business continuity.

[0003] Data synchronization methods can be divided from different perspectives, including: push and pull, synchronous replication and asynchronous replication, homogeneous replication and heterogeneous replication, logical replication and physical replication, master-slave replication, multi-master replication and leaderless replication, point-to-point replication and broker replication, replication based on logs or not, and dynamic membership change.

[0004] Currently, the industry widely uses consensus algorithms such as Paxos / Raft to achieve cross-regional data synchronization. These algorithms build replication groups, partition data, and allocate it to different Paxos / Raft instances. Each instance is responsible for the state machine replication of a data partition. Each node in a Paxos / Raft instance is located in the distributed database systems of different regions. Each Paxos / Raft instance elects a Leader node. Write requests are processed by the Leader node and must be replicated to a majority of nodes before returning to ensure data consistency and reliability. In addition, Paxos / Raft also supports dynamic membership change, allowing cluster members to be added or removed at runtime to adapt to changes in business requirements.

[0005] However, the cross-region data synchronization technology based on Paxos / Raft has some significant deficiencies. First, the engineering implementation is complex. Especially in analytical distributed databases, due to the lack of mechanisms similar to redolog and binlog, the transformation and implementation are difficult. Second, this semi-synchronous replication method has a greater impact on the write latency of the source cluster. Especially in cross-region scenarios, network latency will significantly increase the write response time, making it difficult to meet the requirements of high-throughput writes. In addition, Paxos / Raft requires generating logs and snapshots, resulting in redundant data writes, increasing additional storage and computing overhead. At the same time, since the Leader node may not be on the source cluster, write requests need to be forwarded across regions, further increasing latency and overhead. Another problem is that replication based on Paxos / Raft cannot guarantee snapshot consistency. Since each replication group replicates independently, the data version on the target cluster may not exactly correspond to a certain historical version of the source cluster, which may cause problems in scenarios with high consistency requirements. In addition, Paxos / Raft requires the number of clusters participating in replication to be odd, and a majority of clusters need to be alive to work properly, which results in a minimum deployment specification of three clusters, increasing the deployment cost and complexity. In some scenarios, two clusters may be sufficient.

[0006] In summary, consensus algorithms such as Paxos / Raft have obvious deficiencies in aspects such as engineering implementation complexity, write latency, data redundancy, snapshot consistency, and deployment flexibility in cross-region data synchronization, and it is difficult to meet the actual needs of cross-region data synchronization. Summary of the Invention

[0007] In order to solve the problem that the existing technology is difficult to meet the actual needs of cross-region data synchronization, this application provides a disaster recovery method and system for a distributed database system based on cross-cluster data synchronization.

[0008] In the first aspect, this application provides a disaster recovery method for a distributed database system based on cross-cluster data synchronization, adopting the following technical solutions: A disaster recovery method for a distributed database system based on cross-cluster data synchronization, applied to a distributed database system, includes: The target database cluster pulls source cluster metadata from the source database cluster based on a synchronization period. The target database cluster is any slave database cluster, and the source cluster metadata includes the full amount of data information and full amount of node information of the source database cluster; The target database cluster compares the target cluster metadata of the target database cluster with the source cluster metadata to determine the differential data; The target database cluster determines a number of transactions based on the differential data and executes the data version synchronization task corresponding to each transaction, where the data version synchronization task is used to synchronize data from the source database cluster.

[0009] By adopting the above technical solution, the target database cluster regularly pulls the full metadata, can comprehensively grasp the status of the source database cluster, and ensure the timeliness and accuracy of data synchronization; by comparing the metadata to determine the differential data, it avoids the resource waste caused by full synchronization and significantly improves the synchronization efficiency; based on the differential data, transactions are divided and the data version synchronization task is executed, ensuring the atomicity and consistency of data synchronization, and at the same time supporting concurrent execution, further improving the synchronization performance.

[0010] In a preferred example of the present application, it can be further configured that: the target database cluster determines a number of transactions based on the differential data, including: The target database cluster determines whether it has received a consistency range input by the user, where the consistency range includes any one of table level, database level, and cluster level; If the target database cluster has received the consistency range input by the user, it determines a number of transactions based on the consistency range and the differential data; If the target database cluster has not received the consistency range input by the user, it determines a number of transactions based on the default consistency range and the differential data.

[0011] By adopting the above technical solution, it supports the user to customize the consistency range (such as table level, database level, cluster level), can flexibly adjust the synchronization granularity according to different business requirements, not only meets the scenarios with high consistency requirements, but also avoids the performance overhead caused by excessive synchronization; when the user does not specify the consistency range, the system automatically adopts the default consistency range, ensuring the simplicity of operation and the stability of the system; based on the consistency range, transactions are divided, so that the synchronization task can be executed targeted, which not only improves the synchronization efficiency, but also reduces the resource waste, optimizes the division and execution of the synchronization task, significantly improves the performance and reliability of data synchronization, and provides an efficient and flexible cross-cluster data synchronization capability for the distributed database system.

[0012] In a preferred example of the present application, it can be further configured that: the differential data includes: tables or partitions in the target cluster metadata where there are data version missing or lagging compared to the source cluster metadata; When the target database cluster has received the consistency range input by the user, the determining a number of transactions based on the consistency range and the differential data includes: Take the consistency range of the user input as the target level, and divide the difference data belonging to the same target level in the difference data into the same transaction to obtain several transactions formed by the difference data.

[0013] By adopting the above technical solution, dividing transactions according to the consistency range enables the system to accurately match the synchronization task granularity with the business requirements, meeting both scenarios with high consistency requirements and avoiding unnecessary synchronization overhead. Dividing the difference data of the same target level into the same transaction ensures the logical consistency and execution efficiency of the synchronization task, while supporting concurrent execution, further improving the synchronization performance.

[0014] In a preferred example of the present application, it can be further configured that: the execution of the data version synchronization task corresponding to each transaction includes: Based on the data version synchronization task corresponding to each transaction, the target database cluster sends a data synchronization request to the source database cluster; After receiving the data synchronization request, the source database cluster pauses reclaiming the old data version; After completing the data version synchronization task corresponding to each transaction, the target database cluster submits a transaction completion notification to the source database cluster; After receiving the transaction completion notification, the source database cluster starts reclaiming the old data version.

[0015] By adopting the above technical solution, pausing the reclaiming of the old data version allows the source database cluster to retain the data versions required by the target database cluster, avoiding synchronization failures caused by premature reclamation of data versions, thereby improving the reliability of data synchronization. The target database cluster submits a transaction completion notification to the source database cluster after completing the synchronization task, ensuring the atomicity and consistency of the synchronization process and avoiding problems of partial synchronization success and partial failure. The source database cluster resumes reclaiming the old data version after receiving the transaction completion notification, optimizing the utilization rate of storage resources.

[0016] In a preferred example of the present application, it can be further configured that: the method further includes: When the source database cluster is unavailable, select an available slave database cluster as the new source database cluster to complete the failover; The new source database cluster sends an assumption message to each slave database cluster; After receiving the assumption message, each slave database cluster synchronizes data with the new source database cluster based on the synchronization period.

[0017] By adopting the above technical solution, quickly selecting the new source database cluster and sending the takeover message, the system can quickly resume services when the primary cluster fails, minimizing the service interruption time and improving the high availability of the system; the new source database cluster and other slave database clusters continue to synchronize data based on the synchronization period, ensuring data consistency and integrity and avoiding data loss or inconsistency issues; the system can quickly resume normal operation after failover, and at the same time, by dynamically adjusting the synchronization policy, it optimizes the efficiency and reliability of data synchronization.

[0018] In a preferred example of the present application, it can be further configured that: the target database cluster synchronizes data with the new source database cluster, including: The target database cluster pulls the new source cluster metadata from the new source database based on the synchronization period, and compares the new source cluster metadata with the target cluster metadata to obtain new differential data; When the data version of the target cluster metadata in the new differential data is higher than that of the new source cluster metadata, delete the data in the target cluster metadata whose data version is higher than that of the new source cluster metadata, and resynchronize the deleted data from the new source cluster metadata.

[0019] By adopting the above technical solution, comparing the target cluster metadata with the new source cluster metadata can accurately identify the differences in data versions, ensuring the accuracy and efficiency of synchronization; when the data version of the target cluster metadata is higher than that of the new source cluster metadata, deleting these high-version data and resynchronizing from the new source database cluster avoids data conflicts or errors caused by inconsistent data versions, thus ensuring data consistency.

[0020] In a preferred example of the present application, it can be further configured that: after completing the failover, the method further includes: The new source database cluster queries the transaction records synchronized from the source database cluster, and determines the latest transaction record from the transaction records; Apply to rewrite the lost data during the failover process to the source database cluster based on the latest transaction record.

[0021] By adopting the above technical solution, querying the transaction records and determining the latest transaction record can accurately locate the scope of lost data, ensuring the accuracy and efficiency of data recovery; applying to rewrite the lost data based on the latest transaction record minimizes the impact of data loss, while ensuring data consistency and integrity; significantly improves the data recovery ability and business continuity of the distributed database system after failover.

[0022] In a preferred example, the present application can be further configured such that: the synchronization period is not greater than the recovery point target, and the synchronization period is not greater than the duration for which the application retains data.

[0023] By adopting the above technical solution, since the synchronization period is not greater than the recovery point target, data synchronization can be completed within the maximum allowable data loss range, minimizing the risk of data loss; and since the synchronization period is not greater than the duration for which the application retains data, it ensures that the application can supplement the lost part based on the locally retained data after failover, further enhancing the data recovery ability.

[0024] In a second aspect, the present application provides a distributed database system, adopting the following technical solution: A distributed database system, the system includes: a source database cluster, multiple slave database clusters, and an application; any one of the multiple slave database clusters serves as a target database cluster to execute the disaster recovery method for a distributed database system based on cross-cluster data synchronization as described in any item of the first aspect; The application is used to access the source database cluster; The source database cluster is used to provide corresponding services based on the access of the application; Each slave database is used to synchronize the data of the source database cluster based on the synchronization period.

[0025] In summary, the present application includes the following beneficial technical effects: In the present application, the target database cluster regularly pulls full metadata, enabling a comprehensive understanding of the status of the source database cluster, ensuring the timeliness and accuracy of data synchronization; by comparing the metadata to determine the differential data, it avoids the resource waste caused by full synchronization, significantly improving the synchronization efficiency; dividing transactions based on the differential data and executing the data version synchronization task ensures the atomicity and consistency of data synchronization, and at the same time supports concurrent execution, further enhancing the synchronization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of a disaster recovery method for a distributed database system based on cross-cluster data synchronization provided by an embodiment of the present application; Figure 2 is a flowchart of executing a data synchronization provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The following further elaborates on the present application in conjunction with the attached Figure 1 - attached Figure 2 to further illustrate the present application in detail.

[0028] This specific embodiment is only an interpretation of the present application and does not limit the present application. After reading this specification, those skilled in the art can make modifications to this embodiment without creative contributions as needed, but as long as it is within the scope of the claims of the present application, it is protected by the patent law.

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without making creative efforts fall within the scope of protection of the present application.

[0030] In addition, the term "and / or" in this document is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, unless otherwise specified.

[0031] It should be noted that in the optional embodiments of the present application, for relevant data such as object information, when the embodiments in the present application are applied to specific products or technologies, object permission or consent needs to be obtained, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. That is to say, if the embodiments of the present application involve data related to objects, it needs to be obtained under the authorization and consent of the objects, the authorization and consent of relevant departments, and compliance with the relevant laws, regulations, and standards of relevant countries and regions. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained, and the embodiments also need to be implemented under the authorization and consent of the objects.

[0032] In a distributed database system, cross-region disaster tolerance is a key requirement to ensure high data reliability and business continuity. Especially for users with high requirements for data reliability, even in the case of a complete cluster outage in a certain region due to extreme natural disasters, the system still needs to ensure that data is not lost and can quickly resume services. To achieve this goal, it is usually necessary to deploy multiple sets of the same distributed database clusters in different regions. One region's cluster is used as the primary cluster. Under normal circumstances, the application only accesses the primary cluster, and the primary cluster provides corresponding services. The clusters in other regions are used as secondary clusters for backing up data and providing read-only services.

[0033] The data in the primary cluster is synchronized to all secondary clusters near real-time. This cross-region data synchronization mechanism ensures that even if a cluster in a certain region completely fails due to natural disasters, the data can still be restored from clusters in other regions. When a failure occurs in the primary cluster, the system needs to perform a failover, selecting one of the secondary clusters as the new primary cluster to continue providing services. The speed of failover is crucial and the time of service unavailability needs to be minimized as much as possible. In addition, a part of the most recent data may be lost after failover, and the system needs to provide the starting position of the data loss so that the application can rewrite the lost data, thereby minimizing the impact of data loss.

[0034] However, cross-region data synchronization faces some challenges. The network latency between different regions is large and the bandwidth may also be limited. Therefore, the data synchronization process needs to minimize the impact on the primary cluster and avoid affecting the normal use of the application. To achieve this, the system usually adopts asynchronous replication or semi-synchronous replication methods to minimize the impact on write performance while ensuring data reliability. In addition, the data replenishment mechanism after failover also needs to be efficient and flexible to ensure that the application can quickly restore data consistency.

[0035] To enable the distributed database to have the ability of cross-region disaster tolerance, an embodiment of the present application provides a distributed database system, including: a source database cluster, multiple secondary database clusters, and an application. Among them, the application is used to access the source database cluster; the source database cluster is used to provide corresponding services based on the access of the application; each secondary database is used to synchronize the data of the source database cluster based on a synchronization period.

[0036] An embodiment of the present application provides a disaster tolerance method for a distributed database system based on cross-cluster data synchronization, as Figure 1 shown, applied to the above-mentioned distributed database system. The method includes steps S101 - S103, where: S101. The target database cluster pulls the source cluster metadata from the source database cluster based on a synchronization period. The target database cluster is any secondary database cluster, and the source cluster metadata includes the full amount of data information and full amount of node information of the source database cluster.

[0037] Specifically, both the source database cluster and the secondary database clusters are distributed database clusters. The source database cluster serves as the primary database cluster, and cross-cluster data synchronization adopts asynchronous synchronization. Each secondary database cluster periodically pulls data from the source database cluster, and synchronizing the data enables the secondary database clusters to catch up with the metadata cluster to the current latest data. In this embodiment, taking any secondary database cluster (target database cluster) as an example, the process of synchronizing data from the source database cluster by a single secondary database cluster is described.

[0038] The synchronization period can be selected according to actual needs under the premise of constraints, such as every minute, every ten minutes, every hour, every day, etc. The cron expression can also be used to precisely specify the moment to initiate synchronization. For example, initiate synchronization during the low-traffic period at 2 am every day. The constraints of the synchronization period include: the synchronization period is not greater than the Recovery Point Object (RPO), and the synchronization period is not greater than the duration of the application's retained data.

[0039] See Figure 2 , which shows the schematic flow diagram of performing a data synchronization provided in this embodiment. The target database cluster synchronizes data from the source database cluster, including: the target database cluster pulls the source cluster metadata from the source database cluster. The source cluster metadata includes the current full data information and full node information of the source database cluster, such as the tables included in the cluster, the partitions included in each table, the current data version of the partitions, and the corresponding nodes in the cluster where the partitions are stored. The total amount of data in the source cluster metadata is very small, and synchronizing the source cluster metadata synchronizes a snapshot of the full metadata.

[0040] S102. The target database cluster compares the target cluster metadata of the target database cluster with the source cluster metadata to determine the differential data.

[0041] See Figure 2 , after the target database cluster obtains the source cluster metadata of the source database cluster, it compares the source cluster metadata with its own target cluster metadata to determine the differential data. The differential data includes the tables or partitions in the target cluster metadata whose data versions are missing or lagging behind compared to the source cluster metadata. For the tables or partitions with lagging or missing data versions, data synchronization is required.

[0042] S103. The target database cluster determines a number of transactions based on the differential data and executes the data version synchronization tasks corresponding to each transaction. The data version synchronization tasks are used to synchronize data from the source database cluster.

[0043] Specifically, for the determined differential data, a number of transactions corresponding to these differential data are created. Each transaction is responsible for a part of the tables or partitions with missing or lagging data versions. Different transactions are responsible for different tables or partitions, and there is no intersection between different transactions, so they can be executed concurrently. The target database cluster executes the number of transactions started in the previous step. Each transaction represents a data version synchronization task. When all transactions are completed, the target database cluster has completed synchronizing the missing data from the source database cluster, that is, the data synchronization within this synchronization period is completed.

[0044] The target database cluster periodically pulls the full amount of metadata, enabling it to comprehensively understand the status of the source database cluster, ensuring the timeliness and accuracy of data synchronization; by comparing the metadata to determine the differential data, it avoids the resource waste caused by full synchronization and significantly improves the synchronization efficiency; based on the differential data, transactions are divided and data version synchronization tasks are executed, ensuring the atomicity and consistency of data synchronization, and at the same time supporting concurrent execution, further enhancing the synchronization performance.

[0045] In this embodiment, asynchronous pulling is used to achieve cross-cluster data synchronization, which can flexibly control the timing of triggering synchronization. The coupling degree between the clusters participating in the synchronization is low, and it does not affect the original writing process on the source cluster, having little impact on the source cluster. Compared with the single-cluster deployment without cross-cluster synchronization, the access experience of the source cluster is basically the same. The target database cluster first synchronizes the snapshot of the full amount of metadata from the source database cluster, then compares the synchronized metadata with its own source data, and based on the comparison result, starts corresponding transactions to synchronize the incremental data. When the transaction is committed, the data synchronized this time is atomically visible externally, which makes the data on the target cluster a snapshot of a certain historical version on the source cluster after each synchronization is completed, ensuring the snapshot consistency of the synchronization. Using the multi-version data management technology commonly adopted by database systems, synchronizing according to the data version, realizing the synchronization of incremental or full data versions, without the need for redo log or binlog to achieve synchronization, with little transformation of the distributed database and simple implementation. At the same time, using the version of the data itself to ensure data consistency makes the data alignment after failover simple and natural.

[0046] A possible implementation manner of the embodiment of this application is that the target database cluster determines a number of transactions based on the differential data, including: The target database cluster determines whether it has received a user-input consistency range, and the consistency range includes any one of table level, database level, and cluster level; If the target database cluster has received the user-input consistency range, it determines a number of transactions based on the consistency range and the differential data; If the target database cluster has not received the user-input consistency range, it determines a number of transactions based on the default consistency range and the differential data.

[0047] In this embodiment, the user input permission can be opened to advanced users, and the consistency range options can be set. When the user wants to set the consistency range by themselves, they can input it through the options. When the advanced user chooses to skip the options, the default option, that is, the default consistency range, is automatically selected. Among them, the default consistency range can be selected as the table level, and ordinary users can directly set the default consistency range without setting the consistency range options.

[0048] This embodiment supports users to customize the consistency scope (such as table level, database level, cluster level), and can flexibly adjust the synchronization granularity according to different business requirements, which not only meets the scenarios with high consistency requirements, but also avoids the performance overhead caused by excessive synchronization. When the user does not specify the consistency scope, the system automatically adopts the default consistency scope, ensuring the simplicity of operations and the stability of the system. By dividing transactions based on the consistency scope, the synchronization tasks can be executed in a targeted manner, which not only improves the synchronization efficiency, but also reduces resource waste, optimizes the division and execution of synchronization tasks, significantly improves the performance and reliability of data synchronization, and provides an efficient and flexible cross-cluster data synchronization capability for the distributed database system.

[0049] In a possible implementation manner of the embodiment of the present application, the differential data includes: tables or partitions in the target cluster metadata where there are missing or backward data versions compared with the source cluster metadata. When the target database cluster receives the consistency scope input by the user, several transactions are determined based on the consistency scope and the differential data, including: Taking the consistency scope input by the user as the target level, and dividing the differential data belonging to the same target level in the differential data into the same transaction, to obtain several transactions formed by the differential data.

[0050] In this embodiment, several transactions are determined based on the differential data, and the number of the determined several transactions is related to the consistency scope.

[0051] In a possible case, to ensure the consistency at the table level based on business requirements, refer to Figure 2 , then a transaction is started for each table to be synchronized in the differential data. In another possible case, assume that it is necessary to ensure the consistency at the cluster level based on business requirements. When determining the transaction, all the tables to be synchronized in the differential data should be divided into the same transaction. In another possible case, assume that it is necessary to ensure the consistency at the database level based on business requirements. When determining the transaction, the tables belonging to the same database in the differential data should be divided into the same transaction.

[0052] Among them, the larger the granularity of the divided transaction, the larger the consistency scope, but the greater the cost of failure retry; the smaller the transaction granularity, the smaller the consistency scope, and the smaller the cost of failure retry.

[0053] This embodiment divides transactions according to the consistency scope. The system can accurately match the synchronization task granularity with business requirements, which not only meets the scenarios with high consistency requirements, but also avoids unnecessary synchronization overhead. Dividing the differential data of the same target level into the same transaction ensures the logical consistency and execution efficiency of the synchronization task, and at the same time supports concurrent execution, further improving the synchronization performance.

[0054] A possible implementation of the embodiment of the present application is to execute a data version synchronization task corresponding to each transaction, including: The target database cluster sends a data synchronization request to the source database cluster based on the data version synchronization task corresponding to each transaction; After receiving the data synchronization request, the source database cluster suspends recycling of old data versions; After completing the data version synchronization task corresponding to each transaction, the target database cluster submits a transaction completion notification to the source database cluster; After the source database cluster receives the transaction completion notification, it starts recycling the old data version.

[0055] In this embodiment, the target database cluster synchronizes the missing or outdated data versions from the source database cluster, and the data organization of the distributed database is multi-version. Each synchronization only incrementally synchronizes the missing or outdated data versions from the last synchronization to the current moment.

[0056] In order to successfully synchronize missing or outdated data versions, the recovery of old data versions in the source database cluster is delayed for a period of time, and can only be recovered in the source database cluster after the synchronization of the target database cluster is completed. Specifically, when the target database cluster synchronizes metadata from the source database cluster, the recovery of old data versions in the source database cluster is suspended until the target database cluster completes synchronization of the missing or outdated data versions from the source database cluster, and the source database cluster resumes the recovery of old data versions. After the missing or outdated data versions of all tables or partitions involved in a transaction are synchronized, the transaction is committed. After all transactions are committed, the synchronization within the current synchronization cycle is completed. Transaction commit makes the atomicity of the synchronized data visible to the outside world, ensuring the consistency of the synchronized snapshot.

[0057] This embodiment suspends the recycling of old data versions, so that the source database cluster can retain the data version required by the target database cluster, avoiding synchronization failure caused by premature recycling of data versions, thereby improving the reliability of data synchronization; after completing the synchronization task, the target database cluster submits a transaction completion notification to the source database cluster, ensuring the atomicity and consistency of the synchronization process, and avoiding the problem of partial synchronization success and partial failure; the source database cluster resumes recycling the old data version after receiving the transaction completion notification, optimizing the utilization of storage resources.

[0058] In a possible implementation manner of the embodiment of the present application, the method further includes: When the source database cluster is unavailable, select an available slave database cluster as the new source database cluster to complete the failover; The new source database cluster sends a message to each slave database cluster; After receiving the inauguration message, each slave database cluster synchronizes data with the new source database cluster based on the synchronization cycle.

[0059] In this embodiment, when the source database cluster participating in data synchronization fails and becomes unavailable, a failover is required, and the user or the slave database cluster votes to select a cluster from the remaining surviving slave database clusters to be promoted to the new master cluster (i.e., the new source database cluster). The basis for selecting the new source database cluster can be the geographical location where the application is deployed, and the slave database cluster in the region closest to the application is selected to reduce the network delay of the application accessing the master database cluster.

[0060] After the new source database cluster is selected, it sends a message to other slave database clusters to inform them of its presence. After receiving the message from the new source database cluster, other slave database clusters can synchronize data with the new source database cluster.

[0061] After the failover, the application needs to switch to access the new source database. The application can use DNS access so that after the primary database cluster is changed, the DNS points to the address of the new primary cluster, keeping the domain name accessed by the application unchanged.

[0062] This embodiment quickly selects a new source database cluster and sends an appointment message, so that the system can quickly restore service when the main cluster fails, minimize business interruption time, and improve the high availability of the system; the new source database cluster and other slave database clusters continue to synchronize data based on the synchronization cycle, ensuring data consistency and integrity and avoiding data loss or inconsistency problems; the system can quickly resume normal operation after failover, and at the same time optimize the efficiency and reliability of data synchronization by dynamically adjusting the synchronization strategy.

[0063] In a possible implementation of the embodiment of the present application, the target database cluster synchronizes data with the new source database cluster, including: The target database cluster pulls the new source cluster metadata from the new source database based on the synchronization cycle, and compares the new source cluster metadata with the target cluster metadata to obtain new difference data; When the data version of the target cluster metadata in the new differential data is higher than the new source cluster metadata, the data in the target cluster metadata with a data version higher than the new source cluster metadata is deleted, and the deleted data is synchronized from the new source cluster metadata again.

[0064] In this embodiment, the data on the new source database cluster may lag behind other slave database clusters after failover, and other slave database clusters need to align the data to the new source database cluster. The alignment method is that during the data synchronization process, when it is found that the data version of a table or partition on the target database cluster is larger than that of the new source database cluster, the table or partition is deleted in the target database cluster and synchronized again from the new source database cluster, so that the target database cluster remains consistent with the new source database cluster in terms of data.

[0065] By comparing the metadata of the target cluster with the metadata of the new source cluster in this embodiment, the differences in data versions can be accurately identified, ensuring the accuracy and efficiency of synchronization; when the data version of the target cluster metadata is higher than that of the new source cluster metadata, these high-version data are deleted and synchronized again from the new source database cluster, avoiding data conflicts or errors caused by inconsistent data versions, thus ensuring data consistency.

[0066] A possible implementation manner of the embodiment of this application is that after the failover is completed, the method further includes: The new source database cluster queries the transaction records synchronized from the source database cluster and determines the latest transaction record from the transaction records; The application rewrites the lost data during the failover process to the source database cluster based on the latest transaction record.

[0067] In this embodiment, since some data may be lost during the failover, some users want the application to rewrite the lost data after the failover, and then the starting position of the lost data needs to be provided to facilitate the application to fill in the missing data. Each data write is a transaction, and retaining the transaction records can clarify the current write transaction.

[0068] Specifically, during the metadata synchronization, the transaction records on the source database cluster are also synchronized. The synchronization of the transaction records and the synchronization of the corresponding data are in one transaction to ensure the consistency of the synchronization of the transaction records and the corresponding data. Querying the latest transaction records synchronized from the source database cluster on the target database cluster can know which transaction was written at the moment of the failure. The data written in subsequent transactions is the data that needs to be filled in to the new source database cluster and needs to be rewritten after the failover. The application records which data were written in some recent transactions so that these data can be rewritten when filling in the data.

[0069] The following table gives an example of a transaction record: Transaction ID Transaction Label Start Time End Time 10001 customer_a_20250102 2025-01-0210:00:00:000 2025-01-0210:00:00:100 Among them, the transaction ID is a globally unique transaction number allocated internally by the system. The transaction label is an optional description information provided by the application for the transaction. The application can record some transaction information in the transaction label. The transaction record also includes a time label, which includes a start time and an end time. The start time is the time when the transaction starts, and the end time is the time when the transaction ends. After failover, the transaction record is used to provide the starting position of the lost data, facilitating the application to fill in the lost data.

[0070] Exemplarily, after failover, the latest transaction records synchronized from the source database cluster and saved in the new source database cluster are as shown in the above table, indicating that the data written for transactions with a transaction ID greater than 10001 needs to be rewritten. It can also be seen that the data written after the transaction start time has been lost and needs to be filled in.

[0071] In this embodiment, by querying the transaction record and determining the latest transaction record, the scope of the lost data can be accurately located, ensuring the accuracy and efficiency of data recovery; the application rewrites the lost data based on the latest transaction record, minimizing the impact of data loss, while ensuring data consistency and integrity; significantly improving the data recovery ability and business continuity of the distributed database system after failover.

[0072] In a possible implementation manner of the embodiment of the present application, the synchronization period is not greater than the recovery point objective, and the synchronization period is not greater than the data retention duration of the application.

[0073] Among them, the smaller the synchronization period, the more resources of the source database cluster are occupied. After failover, data in the recent period may be lost. If the application needs to fill in the lost data, the synchronization period is not greater than the data retention duration of the application.

[0074] The synchronization period not being greater than the recovery point objective can complete data synchronization within the maximum allowable data loss range, minimizing the risk of data loss; the synchronization period not being greater than the data retention duration of the application ensures that the application can fill in the lost part based on the locally retained data after failover, further enhancing the data recovery ability.

[0075] A disaster tolerance method for a distributed database system based on cross-cluster data synchronization provided by the embodiment of the present application has the following technical effects: 1. The impact on the source database cluster during synchronization is small. The cross-cluster synchronization adopts the method that the target database cluster asynchronously pulls data from the source database cluster, which does not affect the original writing process on the source database cluster and has little impact on the source database cluster.

[0076] 2. The coupling degree between clusters participating in synchronization is low. Different target database clusters independently initiate the pull synchronization from the source cluster, and the clusters are loosely coupled. The downtime or exit of any cluster outside the main cluster does not affect cross-cluster synchronization.

[0077] 3. The isolation between clusters participating in synchronization is good. Only cross-cluster synchronization generates cross-cluster interactions. Normal writes and queries are only processed locally within the cluster and do not generate cross-cluster interactions, so the clusters have good isolation.

[0078] 4. Provide snapshot consistency. At any given moment, the data on the target database cluster is a snapshot of a certain historical moment on the source database cluster. Querying on the target database cluster ensures that the queried data is in a consistent state.

[0079] 5. Can flexibly control the timing of triggering synchronization. Synchronization can be triggered periodically, and the synchronization period can be flexibly selected, or synchronization can be triggered at a specified moment.

[0080] 6. Do not require redo log or binlog. It makes full use of the multi-version data management technology commonly adopted by database systems, synchronizes according to data versions, and realizes incremental and full-volume data version synchronization without requiring redo log or binlog.

[0081] 7. Unified full-volume and incremental synchronization. Synchronize according to data versions, and the synchronization process for full-volume or incremental data versions is the same, realizing the unification of full-volume and incremental synchronization.

[0082] 8. Simple engineering implementation. Compared with methods based on consensus protocols such as Paxos / Raft, the present invention is simpler in engineering implementation and more conducive to engineering realization.

[0083] 9. Flexible deployment. It can be flexibly deployed according to actual needs using any number of clusters, not limited to only odd numbers of clusters. As long as one cluster survives, the system can work properly, without requiring a majority of clusters to survive.

[0084] 10. Simple member change. The addition or reduction of clusters participating in synchronization is very simple to handle, without the need for complex member change methods such as the joint consensus proposed by raft.

[0085] 11. Can well limit cross-cluster network traffic. Normal writes and queries are only processed locally within the cluster and do not generate cross-cluster network traffic. Only cross-cluster synchronization generates cross-cluster network traffic, and cross-cluster network traffic can be conveniently limited.

[0086] In addition, in this embodiment, cross-cluster synchronization is achieved by the target database cluster periodically pulling data from the source database cluster actively, or it can also be achieved by the source database cluster periodically pushing data to the target database cluster actively.

Claims

1. A distributed database system disaster recovery method based on cross-cluster data synchronization, characterized in that: Applied to a distributed database system, the method comprises: The target database cluster pulls source cluster metadata from the source database cluster based on the synchronization cycle, the target database cluster is any slave database cluster, and the source cluster metadata includes full data information and full node information of the source database cluster; The target database cluster compares the target cluster metadata of the target database cluster with the source cluster metadata to determine difference data; The target database cluster determines a number of transactions based on the difference data, and executes a data version synchronization task corresponding to each transaction, where the data version synchronization task is used to synchronize data from the source database cluster.

2. The method for disaster recovery of a distributed database system based on cross-cluster data synchronization according to claim 1, characterized in that: The target database cluster determines a number of transactions based on the difference data, including: The target database cluster determines whether a consistency range input by a user is received, where the consistency range includes: any one of a table level, a database level, and a cluster level; If the target database cluster receives the consistency range input by the user, determining a number of transactions based on the consistency range and the difference data; If the target database cluster does not receive the consistency range input by the user, a number of transactions are determined based on a default consistency range and the difference data.

3. The method for disaster recovery of a distributed database system based on cross-cluster data synchronization according to claim 2, characterized in that: The difference data includes: tables or partitions in which the target cluster metadata has missing or outdated data versions compared with the source cluster metadata; When the target database cluster receives the consistency range input by the user, determining a number of transactions based on the consistency range and the difference data includes: The consistency range input by the user is used as the target level, and the difference data belonging to the same target level in the difference data are divided into the same transaction, so as to obtain a plurality of transactions formed by the difference data.

4. The method for disaster recovery of a distributed database system based on cross-cluster data synchronization according to claim 1, characterized in that: The execution of the data version synchronization task corresponding to each transaction includes: The target database cluster sends a data synchronization request to the source database cluster based on the data version synchronization task corresponding to each transaction; After receiving the data synchronization request, the source database cluster suspends recycling of old data versions; After completing the data version synchronization task corresponding to each transaction, the target database cluster submits a transaction completion notification to the source database cluster; After receiving the transaction completion notification, the source database cluster starts to recycle the old data version.

5. The method for disaster recovery of a distributed database system based on cross-cluster data synchronization according to claim 1, characterized in that: The method further comprises: When the source database cluster is unavailable, selecting an available slave database cluster as a new source database cluster to complete the failover; The new source database cluster sends a take-up message to each slave database cluster; After receiving the taking office message, each slave database cluster synchronizes data with the new source database cluster based on the synchronization cycle.

6. The method for disaster recovery of a distributed database system based on cross-cluster data synchronization according to claim 5, characterized in that: The target database cluster synchronizes data with the new source database cluster, including: The target database cluster pulls new source cluster metadata from the new source database based on the synchronization period, and compares the new source cluster metadata with the target cluster metadata to obtain new difference data; When the data version of the target cluster metadata in the new differential data is higher than that of the new source cluster metadata, the data in the target cluster metadata with a data version higher than that of the new source cluster metadata is deleted, and the deleted data is synchronized again from the new source cluster metadata.

7. The method for disaster recovery of a distributed database system based on cross-cluster data synchronization according to claim 5, characterized in that: After completing the failover, the method further includes: The new source database cluster queries the transaction records synchronized from the source database cluster, and determines the latest transaction record from the transaction records; The application rewrites the lost data during the failover process to the source database cluster based on the latest transaction record.

8. The method for disaster recovery of a distributed database system based on cross-cluster data synchronization according to claim 1, characterized in that: The synchronization period is no longer than the recovery point objective, and the synchronization period is no longer than the duration for which the application retains data.

9. A distributed database system, characterized in that: The system comprises: a source database cluster, multiple slave database clusters and an application; any slave database cluster among the multiple slave database clusters is used as a target database cluster to execute the distributed database system disaster recovery method based on cross-cluster data synchronization according to any one of claims 1 to 8; The application is used to access the source database cluster; The source database cluster is used to provide corresponding services based on the access of the application; Each slave database is used to synchronize the data of the source database cluster based on a synchronization period.

Images