Distributed database backup method and system based on LVM snapshot

Abstract

The invention belongs to the technical field of databases, and particularly discloses a distributed database backup method and system based on LVM snapshots.The method comprises the steps that a barrier creating request sent by a coordination node is received, and a barrier point is created based on the barrier creating request; wherein the generation processes of the barrier points are not carried out at the same time, so that the uniqueness and consistency of the barrier points are realized; submission of a target distributed transaction of the distributed system is suspended, it is ensured that only distributed transactions with a created barrier exist in the distributed system to be submitted, and the target distributed transaction is any distributed transaction except the created barrier; barrier unlocking is automatically carried out under the condition that all the distributed nodes complete barrier creation, the LVM snapshots are created, and the LVM snapshots are independently created by all the distributed nodes; and the LVM snapshot is dumped into a backup storage server, and the LVM snapshot is deleted or reserved. According to the invention, the integrity and consistency of the backup data can be realized.

Description

Technical Field

[0001] The present application belongs to the field of database technology, and more specifically, to a method and system for distributed database backup based on LVM snapshot. Background Art

[0002] The current database backup and recovery methods include: full and incremental backup based on WAL and CBM, and full backup based on LVM snapshots. The first method has slow backup and recovery speeds and only supports single-machine backup, not distributed consistent backup. The second method has fast backup and recovery speeds, but only supports single-machine backup, not distributed consistent backup.

[0003] However, in a distributed database system, data is distributed on multiple physical nodes, and backup must ensure that the data is in the same snapshot on all physical nodes. LVM runs independently on each physical node, so a distributed database system cannot complete consistent backups relying solely on LVM.

[0004] Therefore, how to solve the problem of inconsistent backup data in distributed database systems is a technical problem that urgently needs to be solved. Summary of the invention

[0005] In view of the defects of the prior art, the purpose of the present application is to provide a method and system for distributed database backup based on LVM snapshots, aiming to solve the problem of inconsistent backup data in distributed database systems.

[0006] To achieve the above objectives, in a first aspect, the present application provides a method for backing up a distributed database based on an LVM snapshot, which is applied to a distributed node of a database, and the method comprises: Receiving a barrier creation request sent by a coordination node, and creating a barrier point based on the barrier creation request; wherein the generation process of each barrier point is not performed simultaneously to achieve uniqueness and consistency of the barrier point; Suspending the submission of a target distributed transaction of the distributed system, ensuring that only a distributed transaction that creates a barrier exists in the distributed system for submission, and the target distributed transaction is any distributed transaction other than the one that creates the barrier; When all distributed nodes have completed barrier creation, the barrier is automatically unlocked and an LVM snapshot is created. Each distributed node creates an LVM snapshot independently. The LVM snapshot is dumped to a backup storage server, and the LVM snapshot is deleted or retained.

[0007] Optionally, it also includes: Generate a WAL log with a special mark based on the barrier creation request, so as to perform log playback based on the WAL log; The special mark is used to indicate the position of the barrier point, and the special mark corresponding to each distributed node is consistent.

[0008] Optionally, the log playback process includes: Determine the creation time of the barrier point of each distributed node, and use each creation time as a cutoff point, wherein the cutoff point is used to make the database state after recovery consistent with the database state at the time of backup; During the recovery process, the generation time of the data blocks of the database is compared with the cutoff point, and the data blocks after the cutoff point are excluded; Log playback is performed based on the WAL log to restore the data block to the state at the cutoff point.

[0009] Optionally, performing log playback based on the WAL log to restore the data block to the state at the cutoff point includes: Receive the backup medium sent by the coordinating node, where the backup medium includes the data file and WAL log corresponding to the LVM snapshot; Based on the original file blocks of the WAL log in the database, log playback is performed from the last time point of the data file; Replay the WAL log to the cutoff point and stop replaying the log; When the log playback of all distributed nodes is completed, it is determined that the database recovery is complete.

[0010] Optionally, delete or keep LVM snapshots, including: Deleting the LVM snapshot from the original storage medium to reduce the occupied snapshot space; Or the LVM snapshot storage is retained in the original storage medium for timely recovery in case of emergency, until the LVM snapshot is deleted when the snapshot space memory of the original storage medium reaches the storage threshold.

[0011] The present application also provides a method for backing up a distributed database based on an LVM snapshot, which is applied to a coordination node of a database, and the method includes: Send a barrier creation request to the distributed node, where the barrier creation request is used to create a barrier point and generate a WAL log with a special mark for log playback; Poll each distributed node to determine whether each distributed node has successfully created a barrier point to start creating an LVM snapshot.

[0012] Optionally, it also includes: Obtaining a backup medium of each distributed node from the dump file of the LVM snapshot or the backup storage server; the backup medium includes a data file and a WAL log; The backup medium is distributed to each distributed node so that the distributed node can perform log playback.

[0013] In a second aspect, the present application also provides a system for distributed database backup based on LVM snapshots, including: A creation module, configured to receive a barrier creation request sent by a coordination node, and to create a barrier point based on the barrier creation request; wherein the generation process of each barrier point is not performed simultaneously to achieve uniqueness and consistency of the barrier point; A suspension module, used for suspending the submission of a target distributed transaction of a distributed system, ensuring that only a distributed transaction that creates a barrier exists in the distributed system for submission, and the target distributed transaction is any distributed transaction except the one that creates the barrier; A generation module is used to automatically unlock the barrier and create an LVM snapshot when all distributed nodes have completed the barrier creation. Each distributed node creates an LVM snapshot independently. The dump module is used to dump the LVM snapshot to a backup storage server, and delete or retain the LVM snapshot.

[0014] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0016] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0017] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0018] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art: (1) The present application receives a barrier creation request sent by a coordination node and generates a barrier point based on the request. The generation process of each barrier point is not performed simultaneously, and the process of creating barrier points on different distributed nodes is coordinated and synchronized, thereby achieving consistency of backup operations; since the generation of barrier points is not performed simultaneously, it can ensure that a synchronous and consistent state can be achieved on all nodes, thereby achieving integrity and consistency of backup data; and by waiting for all distributed nodes to complete barrier creation before creating an LVM snapshot, the creation of the LVM snapshot is performed at a consistent time for all nodes, thereby avoiding the problem of data inconsistency, and each distributed node independently creates an LVM snapshot, so that the backup state of each node is independent and consistent, thereby improving the consistency of data backup.

[0019] (2) The process of creating snapshots in this application ensures that it is possible to quickly restore to a specific point in time, which can significantly reduce the recovery time; the use of snapshot technology for dumping can collect data without affecting operations, thereby achieving incremental backup of data and improving data availability and security.

[0020] (3) When creating a barrier, this application suspends the submission of the target distributed transaction, which can avoid modifying the data during the snapshot creation period, thereby reducing the interference with system performance; by quickly creating barriers and LVM snapshots, the time for data consistency processing can be shortened and the long-term impact on the overall system performance can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is one of the flow charts of the method for distributed database backup based on LVM snapshot provided in an embodiment of the present application; Figure 2 This is the second flow chart of the method for distributed database backup based on LVM snapshot provided in an embodiment of the present application; Figure 3 This is the third flow chart of the method for distributed database backup based on LVM snapshot provided in an embodiment of the present application; Figure 4 This is a fourth flow chart of a method for distributed database backup based on LVM snapshots provided in an embodiment of the present application; Figure 5 It is a schematic diagram of the principle of a distributed database backup method based on LVM snapshot provided in an embodiment of the present application; Figure 6 It is a structural schematic diagram of a device for distributed database backup based on LVM snapshot provided in an embodiment of the present application; Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0023] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0024] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0025] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0026] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0027] First, the relevant terms of this application are explained: Datanode: A data node in a distributed database. Each data node independently manages a data shard. In a distributed database system, there are usually multiple data nodes. The collection of data managed by these data nodes constitutes the data set managed by the database system.

[0028] Coordinator: The coordinating node of a distributed database. The responsibilities of a coordinating node are: accepting user input (usually SQL commands), parsing, optimizing, and controlling the execution of user commands. Note that the roles of Coordinator and Datanode can be assumed by two different instances independently, or by one instance (i.e., one instance has both the roles of Coordinator and Datanode).

[0029] LVM: It is the abbreviation of Logical Volume Manager, which is a mechanism for managing disk partitions in Linux environment. Copy-on-write (COW) is mainly used to reduce latency and delay memory allocation, increase execution efficiency, and allocate physical resources only during actual write operations. It can also protect data from loss in the event of a system crash. For example, when we modify a file, the file system will first move the data to be modified to another location, and then perform the operation.

[0030] Snapshot: A disk snapshot is a quick file system backup of the entire disk volume. The main difference from other backup methods is speed. When taking a disk snapshot, no file copying is involved. Even if the amount of data is large, generally speaking, the backup operation can usually be completed within one second.

[0031] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0032] Reference Figure 1 The present application provides a method for backing up a distributed database based on LVM snapshots, which is applied to distributed nodes of a database. The method includes: S101. Receive a barrier creation request sent by a coordination node, and create a barrier point based on the barrier creation request; wherein the generation process of each barrier point is not performed simultaneously to achieve uniqueness and consistency of the barrier point; S102. Suspend the submission of the target distributed transaction of the distributed system, and ensure that only the distributed transaction that creates the barrier is submitted in the distributed system, and the target distributed transaction is any distributed transaction except the one that creates the barrier; S103. When all distributed nodes have completed the barrier creation, the barrier is automatically unlocked and an LVM snapshot is created. Each distributed node independently creates an LVM snapshot. S104: Dump the LVM snapshot to a backup storage server, and delete or retain the LVM snapshot.

[0033] First, in the above S101, the coordination node in the distributed system is responsible for sending a barrier creation request to notify all participating distributed nodes to prepare to create a barrier point, ensuring that all nodes can create barriers in the same time period, thereby providing a basis for subsequent snapshots and data consistency.

[0034] The distributed nodes include the coordination node CN, the data node DN, and the global transaction management node GTM. After receiving the barrier creation request, each node starts to generate a barrier point. It should be noted that in order to achieve the uniqueness of the barrier point, the barrier point generation process is not performed simultaneously between the nodes. This ensures that each barrier point is unique in time and avoids state conflicts caused by concurrent generation.

[0035] In the above S102, during the barrier point creation, the distributed system will suspend the submission of all target distributed transactions, and only the distributed transaction corresponding to the barrier creation is allowed to be submitted, including all transactions except the barrier creation, to ensure that no other transactions modify the data during the barrier creation process.

[0036] By suspending the submission of other transactions, the distributed system can maintain a consistent data state when creating a snapshot, avoiding data inconsistency that occurs during snapshot creation.

[0037] In the above S103, after all distributed nodes complete the creation of barrier points, the distributed system automatically unlocks the barrier. After the barrier is unlocked, each distributed node independently creates an LVM snapshot. The LVM snapshot can capture the current state of the data and provide a basis for subsequent data recovery.

[0038] Finally, through S104, the created LVM snapshot will be dumped to the backup storage server, ensuring the security and persistence of the data and facilitating subsequent recovery operations. After the snapshot is dumped to the backup storage, the original data can be prevented from being lost or damaged. You can choose to delete or retain the LVM snapshot as needed. This flexibility makes data management more efficient and can be adjusted according to current storage needs.

[0039] The present application receives a barrier creation request sent by a coordination node and generates a barrier point based on the request. The generation process of each barrier point is not performed simultaneously, and the process of creating barrier points on different distributed nodes is coordinated and synchronized, thereby achieving consistency in backup operations. Since the generation of barrier points is not performed simultaneously, it is possible to ensure that a synchronized and consistent state can be achieved on all nodes, thereby ensuring the integrity and consistency of the data. And by waiting for all distributed nodes to complete the barrier creation before creating an LVM snapshot, the LVM snapshot is created at a consistent time on all nodes, thereby avoiding the problem of data inconsistency, and each distributed node independently creates an LVM snapshot, thereby achieving that the backup state of each node is independent and consistent.

[0040] Optionally, it also includes: Generate a WAL log with a special mark based on the barrier creation request, so as to perform log playback based on the WAL log; The special mark is used to indicate the position of the barrier point, and the special mark corresponding to each distributed node is consistent.

[0041] Furthermore, in the process of creating a barrier request, the distributed system generates a WAL log with a special mark, which records all transaction operations and provides a basis for subsequent playback.

[0042] It should be noted that special markers are used to indicate the location of barrier points to ensure that all distributed nodes can be accurately restored to a consistent state when recovery is required.

[0043] After the backup is performed, the distributed system can be replayed based on the WAL log and restored to the state of the barrier point, ensuring the consistency and integrity of the data.

[0044] Furthermore, the log playback process includes: Determine the creation time of the barrier point of each distributed node, and use each creation time as a cutoff point, wherein the cutoff point is used to make the database state after recovery consistent with the database state at the time of backup; During the recovery process, the generation time of the data blocks of the database is compared with the cutoff point, and the data blocks after the cutoff point are excluded; Log playback is performed based on the WAL log to restore the data block to the state at the cutoff point.

[0045] Further, performing log playback based on the WAL log to restore the data block to the state at the cutoff point includes: Receive the backup medium sent by the coordinating node, where the backup medium includes the data file and WAL log corresponding to the LVM snapshot; Based on the original file blocks of the WAL log in the database, log playback is performed from the last time point of the data file; Replay the WAL log to the cutoff point and stop replaying the log; When the log playback of all distributed nodes is completed, it is determined that the database recovery is complete.

[0046] Specifically, when each distributed node in the embodiment of the present application creates a barrier point, it will record the creation timestamp of the barrier point, and the timestamp is used to define the cutoff point in the recovery process.

[0047] During the recovery process, the distributed system traverses all data blocks in the database and compares the generation time of each data block with the cutoff point. If the generation time of the data block is later than the cutoff point, these data blocks will be excluded, ensuring that the recovery process will not introduce any inconsistent data after the barrier point.

[0048] The specific process of performing log playback based on WAL logs is as follows: First, the backup medium sent by the coordination node is received. The backup medium includes the data files and WAL logs corresponding to the LVM snapshot. The backup medium provides the basic data and operation records required for recovery, ensuring that all important data and operations are fully recorded during the backup process to facilitate subsequent recovery.

[0049] Secondly, according to the cutoff point time, the database state at that time point is determined to ensure that the restored database state is consistent with the database state at the time of backup, and the corresponding original file blocks in the snapshot are identified according to the cutoff point.

[0050] Furthermore, based on the WAL log, log playback is performed from the last time point of the data file based on the original file blocks to ensure that all valid operations are reapplied. According to the records in the WAL log, operations are performed one by one to restore to the state at the cutoff point.

[0051] Finally, during the playback process, the current state is monitored compared with the cutoff point. Once the log is restored to the cutoff point, the distributed system will stop log playback. In this way, it is ensured that no changes after the cutoff point are introduced during the recovery process, ensuring data consistency. After all distributed nodes have completed log playback, the status is confirmed to ensure that the data status of each node is consistent and meets the requirements of the cutoff point. Once the log playback of all nodes is completed and the status is consistent, the database recovery will be confirmed to be complete.

[0052] The embodiment of the present application sets a clear time limit for the recovery process by recording the creation time of the barrier point. By comparing the timestamps, it is ensured that no inconsistent data blocks are introduced during the recovery process, and the integrity of the data is maintained; using the records in the WAL log, the system can accurately restore to the state at the cutoff point, ensuring data consistency.

[0053] Optionally, delete or keep LVM snapshots, including: Deleting the LVM snapshot from the original storage medium to reduce the occupied snapshot space; Or the LVM snapshot storage is retained in the original storage medium for timely recovery in case of emergency, until the LVM snapshot is deleted when the snapshot space memory of the original storage medium reaches the storage threshold.

[0054] Specifically, in this embodiment, as time goes by, unmanaged snapshots may occupy a large amount of storage space, causing a shortage of storage resources. Therefore, it is necessary to effectively delete or retain the snapshots.

[0055] After you determine that some LVM snapshots are no longer needed, you can delete them from the original storage media. The deletion process should ensure that the deleted snapshots do not affect the current state of the system or available backups.

[0056] However, in some cases, you may want to keep the LVM snapshot on the original storage medium for quick recovery in an emergency.

[0057] Furthermore, to avoid taking up too much storage space, a storage threshold can be set. When the storage space approaches this threshold, the system will prompt to delete the snapshot to prevent impact on normal operations. The system needs to have the ability to monitor storage space usage, track the storage space occupied by snapshots, and issue warnings in a timely manner. The threshold can be set based on the total storage capacity and the usage of snapshots. For example, it can be set to trigger a warning when 80% of the total space is reached.

[0058] When the snapshot space memory of the original storage medium reaches the set storage threshold, the system should automatically delete the LVM snapshots that are no longer needed, thereby ensuring efficient use of storage resources and avoiding performance degradation or storage shortage problems caused by too many snapshots.

[0059] Reference Figure 2 The present application also provides a method for backing up a distributed database based on an LVM snapshot, which is applied to a coordination node of a database, and the method includes: S202. Send a barrier creation request to the distributed node, wherein the barrier creation request is used to create a barrier point and generate a WAL log with a special mark for log playback; S203. Poll each distributed node to determine that each of the distributed nodes has successfully created a barrier point to start creating an LVM snapshot.

[0060] Optionally, it also includes: Obtaining a backup medium of each distributed node from the dump file of the LVM snapshot or the backup storage server; the backup medium includes a data file and a WAL log; The backup medium is distributed to each distributed node so that the distributed node can perform log playback.

[0061] The embodiment of the present application is an embodiment of a coordination node. The specific implementation process and beneficial effects can be referred to the embodiment of a distributed node, and will not be elaborated here.

[0062] Reference Figure 3 , Figure 3 It is a flowchart of the backup process of the distributed database backup method based on LVM snapshot of the present application; 1. Start backup.

[0063] 2. Send Create Barrier to all nodes (CN, DN, GTM distributed transaction nodes). All components involved in the distributed database system need to create a Barrier.

[0064] Create Barrier is a distributed transaction that generates a WAL log with a special tag. All distributed components have the same tag.

[0065] There are two important points at this time: 1. All other distributed transactions need to be suspended and committed. 2. The system does not allow two barriers to be generated at the same time to ensure the uniqueness and consistency of the barriers.

[0066] 3. All distributed transactions are suspended from committing. The distributed system can continue to create and start new distributed transactions, but suspends committing. The opening and committing of local transactions are not affected, which is equivalent to the entire system only allowing the Create Barrier distributed transaction to commit. 4. Wait for all nodes to complete Create Barrier. The system polls each node to see if the Barrier point has been successfully created. When all participating nodes have completed the process, unlock the Barrier, and the core consistency guarantee process can be completed.

[0067] 5. All nodes create LVM snapshots. At this point, the LVM snapshot process can be started. There is no strict requirement for the order in which nodes create snapshots. Because the database recovery process can ensure that new blocks generated after the cutoff point will not be restored, WAL playback can ensure that blocks will only be restored to the state of the cutoff point.

[0068] 6. Dump and delete the snapshot. You have two options. You can choose to dump the snapshot file to the backup storage server and then delete the snapshot. You can also choose to keep the snapshot for immediate recovery. Note: If you keep a snapshot for a long time, the snapshot space will expand as the original file is constantly modified.

[0069] 7. Backup completed.

[0070] Reference Figure 4 , Figure 4 The figure is a flowchart of the recovery process of the distributed database backup method based on LVM snapshot of the present application, which includes the following steps: 1. Start recovery.

[0071] 2. Copy the backup media from the snapshot or dump file. 3. Replay WAL logs. All distributed nodes replay their own WAL logs based on the original file blocks.

[0072] 4. Wait for all nodes to replay WAL to the Barrier point. When the node replays WAL to the Barrier point, stop replaying. Poll each node to see if they have replayed to the Barrier point.

[0073] 5. Recovery is complete. The distributed database system can be started.

[0074] Reference Figure 5 , Figure 5 It is a schematic diagram of the principle of an embodiment of the present application.

[0075] In the figure, the original file is divided into multiple blocks, which shows that parallel processing can be used to improve computing efficiency. The barrier in the figure shows that different blocks need to be synchronized in certain computing stages.

[0076] The following is a description of a specific embodiment of the present application: 1. User A opens the following two transactions: Transaction-1: single-node transaction; Transaction-2: distributed transaction; 2. User B starts a transaction on the primary backup node (any node in the distributed system) and executes Create Barrier.

[0077] 3. User A starts the following distributed transaction: Transaction-3: distributed transaction; 4. All distributed nodes start to execute Create Barrier; 5. All nodes lock the Barrier, prohibiting the creation of the Barrier and the submission of distributed transactions; prohibiting the submission of all current distributed transactions (such as Transaction-2 and Transaction-3); however, new distributed transactions are allowed to be created until the Barrier is unlocked. Even if these new transactions exist, their submission will still be blocked to ensure the consistency of the state; 6. Transaction-1 is submitted successfully, indicating that the operation of this node has been completed; 7. Since the Barrier is locked, Transaction-2 and Transaction-3 are blocked from submission and the background waits (the waiting time is affected by the cluster size and is generally completed within <2s); 8. Create Barrier for all nodes is completed, ensuring the state consistency of each node; 9. The primary and backup nodes send the unlock command, and all barriers are unlocked; 10. The submission process of Transaction-2 and Transaction-3 continues automatically; 11. User B enables LVM snapshots on all nodes. This operation is usually completed in less than 1 second and the business is not affected. 12. All node snapshots are completed, and we now have a complete backup medium; 13. The user dumps the backup media from the snapshot directory of all nodes to other storage media; 14. Start recovery; 15. Copy the backup media of each node from the backup storage (snapshot directory or backup dump storage); 16. Distribute to each node of the distributed cluster (including data files and WAL logs); 17. Each node in the distributed cluster starts to replay the WAL log until it reaches the Barrier mark point; 18. Poll all nodes to see if they have been replayed to the Barrier mark point; 19. Recovery completed.

[0078] Reference Figure 6 The present application also provides a distributed database backup system based on LVM snapshots, including: A creation module 610, configured to receive a barrier creation request sent by a coordination node, and to create a barrier point based on the barrier creation request; wherein the generation process of each barrier point is not performed simultaneously to achieve uniqueness and consistency of the barrier point; A suspension module 620 is used to suspend the submission of a target distributed transaction of the distributed system, ensuring that only a distributed transaction that creates a barrier is submitted in the distributed system, and the target distributed transaction is any distributed transaction except the one that creates the barrier; A generation module 630 is used to automatically unlock the barrier and create an LVM snapshot when all distributed nodes have completed the barrier creation. Each distributed node independently creates an LVM snapshot. The dump module 640 is used to dump the LVM snapshot to a backup storage server, and delete or retain the LVM snapshot.

[0079] Optionally, a log generation module is also included, for: Generate a WAL log with a special mark based on the barrier creation request, so as to perform log playback based on the WAL log; The special mark is used to indicate the position of the barrier point, and the special mark corresponding to each distributed node is consistent.

[0080] Optionally, the log playback process includes: Determine the creation time of the barrier point of each distributed node, and use each creation time as a cutoff point, wherein the cutoff point is used to make the database state after recovery consistent with the database state at the time of backup; During the recovery process, the generation time of the data blocks of the database is compared with the cutoff point, and the data blocks after the cutoff point are excluded; Log playback is performed based on the WAL log to restore the data block to the state at the cutoff point.

[0081] Optionally, performing log playback based on the WAL log to restore the data block to the state at the cutoff point includes: Receive the backup medium sent by the coordinating node, where the backup medium includes the data file and WAL log corresponding to the LVM snapshot; Based on the WAL log and on the basis of the original file blocks, log playback is performed starting from the cutoff point; Replay the WAL log to the barrier point and stop log playback; When the log playback of all distributed nodes is completed, it is determined that the database recovery is complete.

[0082] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0083] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0084] Reference Figure 7 Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, which may include: a processor (Processor) 710, a communication interface (Communications Interface) 720, a memory (Memory) 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call the logic instructions in the memory 730 to execute the method in the above embodiment.

[0085] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0086] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0087] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0088] It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0089] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0090] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0091] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0092] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for backing up a distributed database based on LVM snapshots, characterized in that: Applied to a distributed node of a database, the method comprises: Receiving a barrier creation request sent by a coordination node, and creating a barrier point based on the barrier creation request; wherein the generation process of each barrier point is not performed simultaneously to achieve uniqueness and consistency of the barrier point; Suspending the submission of a target distributed transaction of the distributed system, ensuring that only a distributed transaction that creates a barrier exists in the distributed system for submission, and the target distributed transaction is any distributed transaction other than the one that creates the barrier; When all distributed nodes have completed barrier creation, the barrier is automatically unlocked and an LVM snapshot is created. Each distributed node creates an LVM snapshot independently. The LVM snapshot is dumped to a backup storage server, and the LVM snapshot is deleted or retained.

2. The method for distributed database backup based on LVM snapshot according to claim 1, characterized in that: Also includes: Generate a WAL log with a special mark based on the barrier creation request, so as to perform log playback based on the WAL log; The special mark is used to indicate the position of the barrier point, and the special mark corresponding to each distributed node is consistent.

3. The method for distributed database backup based on LVM snapshot according to claim 2, characterized in that: The log playback process includes: Determine the creation time of the barrier point of each distributed node, and use each creation time as a cutoff point, wherein the cutoff point is used to make the database state after recovery consistent with the database state at the time of backup; During the recovery process, the generation time of the data blocks of the database is compared with the cutoff point, and the data blocks after the cutoff point are excluded; Log playback is performed based on the WAL log to restore the data block to the state at the cutoff point.

4. The method for distributed database backup based on LVM snapshot according to claim 1, characterized in that: Log playback is performed based on the WAL log to restore the data block to the state at the cutoff point, including: Receive the backup medium sent by the coordinating node, where the backup medium includes the data file and WAL log corresponding to the LVM snapshot; Based on the original file blocks of the WAL log in the database, log playback is performed from the last time point of the data file; Replay the WAL log to the cutoff point and stop replaying the log; When the log playback of all distributed nodes is completed, it is determined that the database recovery is complete.

5. The method for distributed database backup based on LVM snapshot according to claim 1, characterized in that: Deleting or retaining LVM snapshots, including: Deleting the LVM snapshot from the original storage medium to reduce the occupied snapshot space; Or the LVM snapshot storage is retained in the original storage medium for timely recovery in case of emergency, until the LVM snapshot is deleted when the snapshot space memory of the original storage medium reaches the storage threshold.

6. A method for backing up a distributed database based on LVM snapshots, characterized in that: Applied to a coordination node of a database, the method comprises: Send a barrier creation request to the distributed node, where the barrier creation request is used to create a barrier point and generate a WAL log with a special mark for log playback; Poll each distributed node to determine whether each distributed node has successfully created a barrier point to start creating an LVM snapshot.

7. The method for distributed database backup based on LVM snapshot according to claim 6, characterized in that: Also includes: Obtaining a backup medium of each distributed node from the dump file of the LVM snapshot or the backup storage server; the backup medium includes a data file and a WAL log; The backup medium is distributed to each distributed node so that the distributed node can perform log playback.

8. A distributed database backup system based on LVM snapshot, characterized in that: include: A creation module, configured to receive a barrier creation request sent by a coordination node, and to create a barrier point based on the barrier creation request; wherein the generation process of each barrier point is not performed simultaneously to achieve uniqueness and consistency of the barrier point; A suspension module, used for suspending the submission of a target distributed transaction of a distributed system, ensuring that only a distributed transaction that creates a barrier exists in the distributed system for submission, and the target distributed transaction is any distributed transaction except the one that creates the barrier; A generation module is used to automatically unlock the barrier and create an LVM snapshot when all distributed nodes have completed the barrier creation. Each distributed node creates an LVM snapshot independently. The dump module is used to dump the LVM snapshot to a backup storage server, and delete or retain the LVM snapshot.

9. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Images