Cluster backup and recovery system based on multiple data centers

By introducing global synchronization optimization module, distributed consistent guarantee module and off-site parallel recovery module in the multi-data center cluster backup and recovery system, the problems of low backup efficiency, poor data consistency and slow recovery speed are solved, and an efficient and reliable backup and recovery process is achieved.

CN119377013BActive Publication Date: 2025-03-21SICHUAN RURAL COMMERCIAL UNITED BANK CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411964742.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-30
Publication Date
2025-03-21
Estimated Expiration
2044-12-30

AI Technical Summary

Technical Problem

The existing multi-data center cluster backup and recovery technology has problems such as low backup efficiency, poor data consistency and slow recovery speed. Especially when processing large-scale data, the backup time is significantly extended, and there may be incomplete or conflicts during off-site backup and recovery.

Method used

A cluster backup and recovery system based on multi-data centers is adopted, including data centers with geographically distributed locations, global synchronization optimization modules, distributed consistent guarantee modules and off-site parallel recovery modules. The global synchronization optimization module can be used to process transmission between multiple data centers in parallel, reducing the impact of network latency; the distributed consistent guarantee module ensures that the data is consistent in the order between all nodes; the off-site parallel recovery module can achieve the simultaneous participation of multiple data nodes in recovery, improving recovery speed.

Benefits of technology

It significantly shortens backup time, ensures data consistency and backup content integrity, greatly improves the system's recovery speed, and is suitable for business scenarios that require high availability and rapid recovery.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119377013B_ABST
    Figure CN119377013B_ABST
Patent Text Reader

Abstract

The present invention discloses a cluster backup and recovery system based on multiple data centers, which belongs to the technical field of cluster backup and recovery. The system includes: a number of geographically dispersed data centers, which are used to simultaneously deploy and run the same business applications and database services; a global synchronization optimization module, which is used to dynamically select the optimal data transmission channel according to the system's scheduling instructions and the network conditions between the data centers to transmit data fragments in parallel to the target data center; a distributed consistency guarantee module, which is used to enable each data center to write correct data in parallel backup according to the Multi‑Paxos consistency protocol, log synchronization and conflict detection; and a remote parallel recovery module, which is used to extract the decompressed target backup data from each data center in parallel according to the task number, and transmit it to the data center to be restored for data recovery. The present invention solves the problems of low efficiency in identifying recovery tasks in the backup system and insufficient optimization of the scheduling mechanism.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of cluster backup and recovery, and in particular relates to a cluster backup and recovery system based on multiple data centers. Background Art

[0002] With the rapid development of cloud computing and big data technology, more and more companies choose to deploy high-availability clusters through data center architecture to ensure the continuity of business systems. However, as the scale of data increases, ensuring data security and high availability of systems, especially in terms of backup and recovery, has become a key challenge.

[0003] Existing multi-data center cluster backup and recovery technologies are usually: (1) off-site backup: backing up data at multiple physical locations; (2) incremental backup: backing up only the data that has changed since the last full backup; (3) snapshot backup: mirroring the system state at a certain moment; (4) multi-point recovery: providing multiple versions of backup at multiple time points. Although most existing central cluster backup and recovery technologies can cope with general system failures and data loss, there is still the problem of multi-data center network latency. In particular, when processing large-scale data, the backup time is significantly extended. In addition, when performing off-site backup and recovery, since multiple data centers write data in parallel, the backup content may be incomplete or conflicting. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a cluster backup and recovery system based on multiple data centers, which solves the problems of low backup efficiency, poor data consistency and slow recovery speed.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0006] The present invention provides a cluster backup and recovery system based on multiple data centers, comprising:

[0007] Several geographically dispersed data centers are used to deploy and run the same business applications and database services simultaneously;

[0008] The global synchronization optimization module is used to dynamically select the optimal data transmission channel and transmit data fragments to the target data center in parallel according to the system scheduling instructions and the network conditions between data centers;

[0009] The distributed consistency guarantee module is used to enable each data center to write correct data in parallel based on the Multi-Paxos consistency protocol, log synchronization, and conflict detection.

[0010] The off-site parallel recovery module is used to extract the decompressed target backup data from each data center in parallel according to the task number, and transmit it to the data center to be restored for data recovery.

[0011] The beneficial effects of the present invention are as follows: a cluster backup and recovery system based on multiple data centers provided by the present invention builds a cluster backup and recovery system through several geographically dispersed data centers, thereby providing a guarantee for the recovery of some data centers after being damaged by disasters; through a global synchronization optimization module, parallel processing of transmission between multiple data centers is realized, the influence of network delays during the backup process is reduced, the backup time is greatly shortened, and stable and high-speed data transmission can be achieved even in the case of high network delays or limited bandwidth; through a distributed consistency guarantee module, under the Multi-Paxos consistency protocol, each data center can ensure that the order of data between all nodes is consistent when writing, and the consistency of backup data can be guaranteed even in the case of network fluctuations or node failures; through a remote parallel recovery module, multiple data nodes are enabled to participate in the recovery at the same time, which greatly improves the recovery speed of the system and ensures the continuity and stability of the business; the present invention overcomes the limitations of slow backup speed, poor data consistency, and long recovery time in traditional multi-data center backup and recovery technologies, comprehensively improves the performance and reliability of the backup and recovery system, and is suitable for business scenarios requiring high availability and rapid recovery, such as finance, medical care, Internet platforms, and other large-scale distributed systems that need to ensure high data reliability.

[0012] Furthermore, the global synchronization optimization module includes:

[0013] The transmission channel submodule is used to establish a number of independent data transmission channels between data centers;

[0014] The data sharding submodule is used to divide and compress data files or data streams according to the redundancy mechanism of sharding before data backup or data transmission begins, so as to obtain data shards that can be transmitted independently;

[0015] The path selection submodule is used to monitor the network bandwidth, delay and current load information of each data transmission channel in real time according to the routing algorithm, and dynamically select the optimal data transmission channel to transmit the data fragment;

[0016] The parallel control submodule is used to control the backup clients of each data center in parallel to transmit the shard data to the target data center through the real-time optimal data transmission channel according to the system scheduling instructions.

[0017] The beneficial effect of adopting the above further scheme is: by constructing independent parallel data transmission channels and using data sharding and routing algorithms to optimize the transmission path, the efficiency of data transmission is improved and the amount of data to be transmitted is reduced.

[0018] Furthermore, the routing algorithm in the path selection submodule is an equivalent multi-path selection algorithm after reinforcement learning.

[0019] The beneficial effect of adopting the above further scheme is: the equivalent multi-path selection algorithm based on reinforcement learning can dynamically and automatically select the optimal path according to the real-time network conditions, such as bandwidth, delay, packet loss rate, etc., to minimize transmission delay and maximize throughput.

[0020] Furthermore, the distribution consistency guarantee module includes:

[0021] The consensus protocol submodule is used to elect a leader node from each data center according to the Multi-Paxos consensus protocol, so that each time data is backed up, the leader node coordinates and writes data to each data center in the same order concurrently;

[0022] The log synchronization submodule is used to periodically synchronize logs based on the local logs of each data center according to a preset time interval, so that the status of all data centers after the write operation is consistent;

[0023] The conflict detection submodule is used to automatically detect data conflicts between different data centers and determine the correct data based on the priority or timestamp of the data center to adjust and repair the data in other data centers that conflict with the correct data.

[0024] The beneficial effect of adopting the above further scheme is: when multiple data centers perform backup write operations simultaneously, by having each write operation coordinated by the leader node, the sequential consistency of all write data among the data centers is guaranteed, thereby providing effective protection for ensuring the integrity and consistency of the backup data.

[0025] Furthermore, the remote parallel recovery module includes:

[0026] The task allocation and scheduling submodule is used to register the recovery task from the global task manager and obtain the corresponding task number, read the backup set corresponding to the task globally according to the task number, send metadata query requests to each data center according to the records in the backup set to obtain the backup data information fed back by each data center, and allocate target backup data, start the task flow to transfer the target backup data from each data center to the data center to be restored, wherein the target backup data is the backup data extracted from the backup data pool of each data center according to the records in the backup set corresponding to the task;

[0027] The data parallel extraction submodule is used to query each data center whether the target backup data included in the backup set is stored according to the task number, and if so, feedback the backup data information corresponding to the metadata, and use the task scheduler to send a first recovery signal for transmitting the target backup data to each data center, prepare the target backup data in the backup data pool of each data center according to the first recovery signal, and transmit the target backup data to the data center to be restored through the task flow;

[0028] The data decompression and transmission submodule is used to start each data center according to the first recovery signal, extract a compressed target backup data from its backup data pool into the memory, decompress it to restore the data format of the target backup data, and send the decompressed target backup data into the data stream to transmit it to the data center to be restored. After the target backup data is written into the recovery target of the data center to be restored, each data center is started to start the next round of target backup data decompression, data stream transmission and writing into the recovery target of the data center to be restored until the entire recovery task is completed;

[0029] The data transmission optimization submodule is used to determine the transmission protocol used by each data center according to the transmission task information in the global task manager. When starting to transmit the target backup data according to the transmission protocol, the global task manager is used to perform duplicate data verification on the recovery target of the data center to be restored. If a certain data center has already transmitted a certain segment of target backup data, the target backup data transmitted by other data centers will be ignored.

[0030] The recovery conflict detection submodule is used to obtain the version number of the target backup data stored in each data center, determine the target version number, and call the data center of the target backup data with the target version number to transmit the target backup data to the data center to be restored for recovery;

[0031] The consistency synchronization submodule is used to send a recovery completion signal to each data center according to the task number in the global task manager after confirming that the data center to be restored has received all the target backup data, and check whether the target backup data transmitted by each data center is consistent. If so, the data transmission channel between each data center and the data center to be restored is closed according to the recovery task;

[0032] The integrity check submodule is used to check whether the complete target backup data transmitted by each data transmission center is complete and valid by comparing the CRC check information and the Hash check information. If so, the global task manager completes the recovery task corresponding to the task number, otherwise rechecks whether the target backup data transmitted by each data center is consistent.

[0033] The beneficial effect of adopting the above further scheme is: by introducing a multi-point parallel recovery mechanism, some data centers in the system can read and recover data from multiple data centers at the same time during disaster recovery, and through the global scheduling mechanism, the system can extract backup data in parallel in each data center and restore it to the data center and cluster to be restored.

[0034] Other advantages of the present invention will be analyzed in more detail in subsequent embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 This is a block diagram of a cluster backup and recovery system based on multiple data centers in Example 1 of the present invention. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0038] like Figure 1 As shown, in one embodiment of the present invention, the present invention provides a cluster backup and recovery system based on multiple data centers, including:

[0039] Several geographically dispersed data centers are used to deploy and run the same business applications and database services simultaneously;

[0040] The global synchronization optimization module is used to dynamically select the optimal data transmission channel to transmit data fragments in parallel to the target data center based on the system's scheduling instructions and the network conditions between data centers. In this solution, the global synchronization module builds a multi-channel parallel transmission mechanism across data centers to facilitate simultaneous transmission of data on multiple transmission paths.

[0041] The global synchronization optimization module includes:

[0042] The transmission channel submodule is used to establish a number of independent data transmission channels between data centers. In this embodiment, each data transmission channel operates independently without interfering with each other, and can simultaneously undertake the task of data transmission. Each data transmission channel can be implemented through different network protocols and physical links such as TCP and UDP to ensure a certain redundancy in the transmission path, thereby improving the reliability and speed of the overall data.

[0043] The data sharding submodule is used to divide and compress data files or data streams according to the redundancy mechanism of sharding before data backup or data transmission begins, so as to obtain data shards that can be transmitted independently. In this solution, the redundancy mechanism of sharding is to copy key data or metadata to multiple data shards during the data division process, so that even if individual data shards are lost, the missing data can still be compensated. Data sharding enables each data transmission channel to simultaneously undertake the transmission tasks of different data shards.

[0044] In this embodiment, when implementing data sharding, the data is divided into several data blocks of fixed size, which are marked as different backup batches, and the integrity of the data is ensured by hash verification during the transmission process; each data block will be compressed on the backup server and stored in the backup nodes of multiple data centers, thereby ensuring the redundancy and security of the data between different nodes.

[0045] The path selection submodule is used to monitor the network bandwidth, delay and current load information of each data transmission channel in real time according to the routing algorithm, and dynamically select the optimal data transmission channel to transmit the data fragments; in this solution, the network situation includes the network bandwidth, delay and current load information of each data transmission channel, and the data transmission efficiency can be effectively improved by dynamically selecting the optimal data transmission channel to transmit the data fragments;

[0046] The routing algorithm in the path selection submodule is an equivalent multi-path selection algorithm after reinforcement learning. In this solution, the equivalent multi-path selection method after reinforcement learning can automatically evaluate each data transmission path for performing data backup tasks and evenly distribute the transmission load, thereby avoiding overload of a data center network or transmission bottleneck.

[0047] The parallel control submodule is used to control the backup clients of each data center in parallel to transmit the shard data to the target data center through the real-time optimal data transmission channel according to the system scheduling instructions. By controlling the backup clients of multiple data centers to work together in parallel, it can ensure that the data shards are transmitted on multiple data transmission channels, thereby improving the data transmission efficiency.

[0048] The distributed consistency guarantee module is used to enable each data center to write correct data in parallel based on the Multi-Paxos consistency protocol, log synchronization, and conflict detection.

[0049] The distribution consistency guarantee module includes:

[0050] The consensus protocol submodule is used to elect a leader node from each data center according to the Multi-Paxos consensus protocol, so that each time data is backed up, the leader node coordinates and writes data to each data center in the same order concurrently;

[0051] The log synchronization submodule is used to synchronize logs according to the local logs of each data center regularly according to the preset time interval, so that the status of all data centers after the write operation is consistent; in this solution, since when data is written between data centers, all operations will first be written to the local log of the data center. Through regular log synchronization updates, the order of logs can be ensured to be consistent, thus providing a solid foundation for backup and recovery;

[0052] In this embodiment, whenever there is a data write request, it is first recorded in the log system of the data center corresponding to the leader node, and the other data centers are notified to synchronize the logs through the Paxos protocol. The log synchronization of each data center will be verified twice: one is to verify the order of data when writing, and the other is to perform a second verification after the log synchronization is completed to ensure data consistency.

[0053] The conflict detection submodule is used to automatically detect data conflicts between different data centers, and determine the correct data according to the priority or timestamp of the data center, so as to adjust and repair the data in other data centers that conflict with the correct data. In this solution, when the data center receives a data write request, it extracts the data identifier, version number and timestamp from the data write request; while executing the data write to the data center, the data version number and timestamp are written to the local log, and the written data is transmitted to other data centers in parallel according to the Multi-Paxos consistency protocol; the conflict detection submodule can confirm whether a data conflict occurs by comparing the version number and timestamp in the data write request of each data center; if there is no data conflict, the data synchronization write is directly executed to complete the cluster backup of multiple data centers; if there is a data conflict, the correct data is selected according to the preset data center priority or timestamp, and the conflicting data is repaired based on the correct data, and the correct data is synchronized to other data centers; all data centers record local logs, verify the repair results and generate automatic repair reports for conflict detection.

[0054] The off-site parallel recovery module is used to extract the decompressed target backup data from each data center in parallel according to the task number, and transmit it to the data center to be restored for data recovery.

[0055] The remote parallel recovery module includes:

[0056] The task allocation and scheduling submodule is used to register the recovery task from the global task manager and obtain the corresponding task number, read the backup set corresponding to the task globally according to the task number, send metadata query requests to each data center according to the records in the backup set to obtain the backup data information fed back by each data center, and allocate target backup data, start the task flow to transfer the target backup data from each data center to the data center to be restored, wherein the target backup data is the backup data extracted from the backup data pool of each data center according to the records in the backup set corresponding to the task;

[0057] The data parallel extraction submodule is used to query each data center whether the target backup data included in the backup set is stored according to the task number, and if so, feedback the backup data information corresponding to the metadata, and use the task scheduler to send a first recovery signal for transmitting the target backup data to each data center, prepare the target backup data in the backup data pool of each data center according to the first recovery signal, and transmit the target backup data to the data center to be restored through the task flow;

[0058] The data decompression and transmission submodule is used to start each data center according to the first recovery signal, extract a compressed target backup data from its backup data pool into the memory, decompress it to restore the data format of the target backup data, and send the decompressed target backup data into the data stream to transmit it to the data center to be restored. After the target backup data is written into the recovery target of the data center to be restored, each data center is started to start the next round of target backup data decompression, data stream transmission and writing into the recovery target of the data center to be restored until the entire recovery task is completed;

[0059] The data transmission optimization submodule is used to determine the transmission protocol adopted by each data center according to the transmission task information in the global task manager. When starting to transmit the target backup data according to the transmission protocol, the global task manager is used to perform duplicate data verification on the recovery target of the data center to be restored. If a certain data center has already transmitted a certain segment of target backup data, the target backup data transmitted by other data centers will be ignored. This solution reduces the amount of network data transmission and improves the transmission efficiency of network data by determining the transmission protocol and performing duplicate data verification on the recovery target.

[0060] When obtaining and transmitting the target backup data of each data center, there is a chance that conflicts may occur when data is transmitted from different data centers to the data center to be restored, resulting in data anomalies.

[0061] The recovery conflict detection submodule is used to obtain the version number of the target backup data stored in each data center, determine the target version number, and call the data center with the target backup data with the target version number to transmit the target backup data to the data center to be restored for recovery; when each data center starts to prepare and transmit the target backup data, it verifies according to the CRC information during storage compression to avoid data conflicts.

[0062] The consistency synchronization submodule is used to send a recovery completion signal to each data center according to the task number in the global task manager after confirming that the data center to be restored has received all the target backup data, and check whether the target backup data transmitted by each data center is consistent. If so, the data transmission channel between each data center and the data center to be restored is closed according to the recovery task;

[0063] The integrity check submodule is used to check whether the complete target backup data transmitted by each data transmission center is complete and valid by comparing the CRC check information and the Hash check information. If so, the global task manager completes the recovery task corresponding to the task number, otherwise rechecks whether the target backup data transmitted by each data center is consistent.

[0064] Embodiment 2:

[0065] On the basis of Example 1, the cluster backup and recovery system based on multiple data centers provided by the present invention also includes a data preloading module, which is used to determine the backup data information corresponding to the priority recovery service on the business side by reading the index directory of the backup set according to the task number. By determining the priority recovery service such as hot data or the latest backup data, it is possible to achieve priority recovery of important services, thereby ensuring that important services are quickly launched.

[0066] Embodiment 3:

[0067] Based on Example 1 or a combination of Example 1 and Example 2, the cluster backup and recovery system based on multiple data centers provided by the present invention also includes a data stratification module, which is used to divide the data stored in the data center into core data and secondary data according to the priority of the business side data, and perform full backup and recovery of the core data and incremental backup of the secondary data. Through data stratification, when a disaster occurs, the core data can be quickly restored, so that the key parts of the business can be quickly put online, and the secondary data can be gradually loaded after the recovery is completed.

[0068] In this embodiment, when a disaster or failure occurs in a data center, the system determines the data range that needs to be restored by analyzing the latest backup log. The global scheduler assigns the recovery nodes of multiple data centers to download data blocks in parallel. The recovery module will first restore the core data, that is, the key business data, and continue to restore the secondary data in the background.

[0069] Embodiment 4:

[0070] Based on the combination of Example 1 with Example 2 and / or Example 3, the cluster backup and recovery system based on multiple data centers provided by the present invention also includes an intelligent scheduling module for marking the data centers to which the tenants belong, so as to automatically restore the required tenants in each data center according to the markings. By marking the data centers to which each tenant belongs, in the scenario of distributed database recovery, the required tenants can be automatically restored in each data center according to the markings, thereby reducing the amount of restored data and reducing bandwidth usage.

[0071] In this embodiment, in a distributed database multi-active multi-data center scenario, there are many tenants under the same distributed database cluster, and tenants are distinguished by primary and backup roles. When backing up the distributed database, each tenant is marked and the data center to which it belongs is recorded. When performing cluster-level recovery, the intelligent scheduling module restores the primary tenant in each data center according to the mark, so as to achieve the purpose of restoring the production environment.

[0072] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A cluster backup and recovery system based on multiple data centers, characterized in that: include: Several geographically dispersed data centers are used to deploy and run the same business applications and database services simultaneously; The global synchronization optimization module is used to dynamically select the optimal data transmission channel and transmit data fragments to the target data center in parallel according to the system scheduling instructions and the network conditions between data centers; The global synchronization optimization module includes: The transmission channel submodule is used to establish a number of independent data transmission channels between data centers; The data sharding submodule is used to divide and compress data files or data streams according to the redundancy mechanism of sharding before data backup or data transmission begins, so as to obtain data shards that can be transmitted independently; The redundancy mechanism of sharding is: copy key data or metadata to multiple data shards during data partitioning, so that even if individual data shards are lost, the missing data can still be compensated. Data sharding enables each data transmission channel to simultaneously undertake the transmission tasks of different data shards; divide the data into several fixed-size data blocks, which are marked as different backup batches, and use hash verification to ensure data integrity during transmission; The distributed consistency guarantee module is used to enable each data center to write correct data in parallel based on the Multi-Paxos consistency protocol, log synchronization, and conflict detection. The off-site parallel recovery module is used to extract the decompressed target backup data from each data center in parallel according to the task number, and transmit it to the data center to be restored for data recovery; The remote parallel recovery module includes: The recovery conflict detection submodule is used to obtain the version number of the target backup data stored in each data center, determine the target version number, and call the data center of the target backup data with the target version number to transmit the target backup data to the data center to be restored for recovery; The consistency synchronization submodule is used to send a recovery completion signal to each data center according to the task number in the global task manager after confirming that the data center to be restored has received all the target backup data, and check whether the target backup data transmitted by each data center is consistent. If so, the data transmission channel between each data center and the data center to be restored is closed according to the recovery task; The integrity check submodule is used to check whether the complete target backup data transmitted by each data transmission center is complete and valid by comparing the CRC check information and the Hash check information. If so, the global task manager completes the recovery task corresponding to the task number, otherwise rechecks whether the target backup data transmitted by each data center is consistent.

2. The cluster backup and recovery system based on multiple data centers according to claim 1 is characterized in that: The global synchronization optimization module also includes: The path selection submodule is used to monitor the network bandwidth, delay and current load information of each data transmission channel in real time according to the routing algorithm, and dynamically select the optimal data transmission channel to transmit the data fragment; The parallel control submodule is used to control the backup clients of each data center in parallel to transmit the shard data to the target data center through the real-time optimal data transmission channel according to the system scheduling instructions.

3. The cluster backup and recovery system based on multiple data centers according to claim 2 is characterized in that: The routing algorithm in the path selection submodule is an equivalent multi-path selection algorithm after reinforcement learning.

4. The cluster backup and recovery system based on multiple data centers according to claim 3 is characterized in that: The distribution consistency guarantee module includes: The consensus protocol submodule is used to elect a leader node from each data center according to the Multi-Paxos consensus protocol, so that each time data is backed up, the leader node coordinates and writes data to each data center in the same order concurrently; The log synchronization submodule is used to periodically synchronize logs based on the local logs of each data center according to a preset time interval, so that the status of all data centers after the write operation is consistent; The conflict detection submodule is used to automatically detect data conflicts between different data centers and determine the correct data based on the priority or timestamp of the data center to adjust and repair the data in other data centers that conflict with the correct data.

5. The cluster backup and recovery system based on multiple data centers according to claim 4 is characterized in that: The remote parallel recovery module also includes: The task allocation and scheduling submodule is used to register the recovery task from the global task manager and obtain the corresponding task number, read the backup set corresponding to the task globally according to the task number, send metadata query requests to each data center according to the records in the backup set to obtain the backup data information fed back by each data center, and allocate target backup data, start the task flow to transfer the target backup data from each data center to the data center to be restored, wherein the target backup data is the backup data extracted from the backup data pool of each data center according to the records in the backup set corresponding to the task; The data parallel extraction submodule is used to query each data center whether the target backup data included in the backup set is stored according to the task number, and if so, feedback the backup data information corresponding to the metadata, and use the task scheduler to send a first recovery signal for transmitting the target backup data to each data center, prepare the target backup data in the backup data pool of each data center according to the first recovery signal, and transmit the target backup data to the data center to be restored through the task flow; The data decompression and transmission submodule is used to start each data center according to the first recovery signal, extract a compressed target backup data from its backup data pool into the memory, decompress it to restore the data format of the target backup data, and send the decompressed target backup data into the data stream to transmit it to the data center to be restored. After the target backup data is written into the recovery target of the data center to be restored, each data center is started to start the next round of target backup data decompression, data stream transmission and writing into the recovery target of the data center to be restored until the entire recovery task is completed; The data transmission optimization submodule is used to determine the transmission protocol adopted by each data center based on the transmission task information in the global task manager. When starting to transmit the target backup data according to the transmission protocol, the global task manager is used to perform duplicate data verification on the recovery target of the data center to be restored. If a certain data center has already transmitted a certain segment of target backup data, the target backup data transmitted by other data centers will be ignored.

6. The cluster backup and recovery system based on multiple data centers according to claim 5 is characterized in that: It also includes a data preloading module, which is used to determine the backup data information corresponding to the priority recovery business on the business side by reading the index directory of the backup set according to the task number.

7. The cluster backup and recovery system based on multiple data centers according to claim 6 is characterized in that: It also includes a data stratification module, which is used to divide the data stored in the data center into core data and secondary data according to the priority of business-side data, and to perform full backup and recovery of core data and incremental backup of secondary data.

8. The cluster backup and recovery system based on multiple data centers according to claim 7 is characterized in that: It also includes an intelligent scheduling module for marking the data centers to which the tenants belong, so as to automatically restore the required tenants in each data center according to the markings.

Citation Information

Patent Citations

  • Oracle multichannel parallel backup and recovery method

    CN105045681A

  • A distributed data task processing method and device

    CN109445921A