Data backup and recovery method, server, distributed system, and storage medium

Abstract

This invention discloses a data backup and recovery method, a server, a distributed system, and a storage medium. The data backup method includes: responding to a backup command, backing up the full data of each shard in the distributed system to a local node using a backup and recovery management tool, and reading the consistency point of each shard, wherein the full data of each shard ends at the consistency point of the corresponding shard; triggering an incremental backup operation based on the consistency point of each shard, the incremental backup operation including, through an incremental backup recovery service, performing real-time backup of the incremental data of each shard according to the consistency point of each shard. Using the above method, unified management of full and incremental backup data can be achieved, and incremental data can be synchronized in real time through the incremental backup recovery service.

Description

Technical Field

[0001] This invention relates to the field of database backup and recovery technology, and in particular to a data backup and recovery method, server, distributed system and storage medium. Background Technology

[0002] Currently, database backups can generally be categorized into three types based on the backup method: full backup, incremental backup, and differential backup.

[0003] However, existing solutions separate full backup and recovery functions from incremental backup and recovery functions, failing to create a seamless integration and requiring manual confirmation, which is inconvenient for management. Furthermore, incremental backup and recovery can only restore to the latest time, cannot specify arbitrary points in time, and cannot guarantee global consistency after recovery. Summary of the Invention

[0004] This invention provides a data backup and recovery method, server, distributed system, and storage medium to achieve unified management of full backup and incremental backup data, and to perform incremental recovery based on any point in time on the basis of full recovery, ensuring that the recovered data is globally consistent.

[0005] In a first aspect, embodiments of the present invention provide a data backup method, comprising:

[0006] In response to the backup command, the backup and recovery management tool backs up the full data of each shard in the distributed system to the local node, and reads the consistency point of each shard, wherein the full data of each shard ends at the consistency point of the corresponding shard.

[0007] Incremental backup operations are triggered based on the consistency sites of each of the aforementioned shards. The incremental backup operations include real-time backup of incremental data of each of the aforementioned shards through an incremental backup recovery service, based on the consistency sites of each of the aforementioned shards.

[0008] Optionally, a backup and recovery management tool can be used to back up the full data of each shard in the distributed system to the local node, including:

[0009] The local backup directory and the connection information of each of the fragments are initialized using the backup and recovery management tool.

[0010] The backup and recovery management tool remotely calls the physical backup and recovery tool of each shard, so that the physical backup and recovery tool copies the full data of the corresponding shard to the node where the backup and recovery management tool is located and records the consistency point of the corresponding shard.

[0011] Optionally, the incremental backup operation includes:

[0012] The incremental backup and recovery service monitors the incremental data of each of the aforementioned shards, starting from the consistency point of each shard.

[0013] The incremental backup and recovery service copies the incremental data of each shard to the backup and recovery management tool in real time.

[0014] Secondly, embodiments of the present invention also provide a data recovery method, comprising:

[0015] In response to a recovery command, a backup recovery management tool performs local recovery based on the full data of each shard in the distributed system backed up by the local node, wherein the recovery command includes the recovery time.

[0016] The backup and recovery management tool is used to copy the fully restored data from the local storage to each of the partitions.

[0017] An incremental recovery operation is triggered based on the recovery time, and the incremental recovery operation includes restoring the incremental data backed up by each shard on the local node according to the recovery time.

[0018] Optionally, local recovery can be performed using backup and recovery management tools based on the full data of each shard in the distributed system backed up by the local node, including:

[0019] The local backup directory and the connection information of each of the fragments are initialized using the backup and recovery management tool.

[0020] The backup and recovery management tool calls the physical backup and recovery tool of the local node, so that the physical backup and recovery tool restores the full data of each of the shards backed up by the local node to the local node.

[0021] Optionally, before performing local recovery based on the full data of each shard in the distributed system backed up by the local node using the backup and recovery management tool, the following steps are also included:

[0022] The backup and recovery management tool sends a recovery time verification request to the incremental backup and recovery service so that the incremental backup and recovery service can verify the recovery time.

[0023] Optionally, the incremental recovery operation includes:

[0024] The incremental backup and recovery service is used to trim the incremental data of each shard backed up on the local node. The starting point of the trimming is the consistency point at the end of the full data backup of the corresponding shard, and the ending point of the trimming is the recovery time.

[0025] The backup and recovery management tool is used to copy the trimmed incremental data to each of the aforementioned fragments.

[0026] Thirdly, embodiments of the present invention also provide a server, comprising:

[0027] At least one processor; and

[0028] A memory communicatively connected to the at least one processor; wherein,

[0029] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a data backup method or a data recovery method as described in any of the embodiments of the present invention.

[0030] Fourthly, embodiments of the present invention also provide a distributed system, including: multiple database nodes and a server as described in the third aspect, wherein the multiple database nodes are divided into at least one shard.

[0031] Fifthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a data backup method or a data recovery method as described in any of the embodiments of the present invention.

[0032] This invention provides a data backup and recovery method, server, distributed system, and storage medium. A backup and recovery management tool backs up the full data of each shard in the distributed system to a local node and reads the consistency points of each shard. Then, based on the consistency points of each shard, an incremental backup operation is triggered. This enables unified management of full and incremental backup data and real-time synchronization of incremental data. The backup and recovery management tool performs local recovery based on the full data of each shard in the distributed system backed up by the local node, copies the locally recovered full data to each shard, and then triggers an incremental recovery operation based on the recovery time. This allows for incremental recovery based on any point in time, building upon full recovery, and ensures that the recovered data is globally consistent.

[0033] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of a data backup method provided according to Embodiment 1 of the present invention;

[0036] Figure 2 This is a schematic diagram illustrating the operation process of a data backup method according to Embodiment 1 of the present invention;

[0037] Figure 3 This is a flowchart of a data recovery method provided according to Embodiment 2 of the present invention;

[0038] Figure 4 This is a schematic diagram of the operation process of a data recovery method according to Embodiment 2 of the present invention;

[0039] Figure 5 This is a schematic diagram of time points for a fragmented backup and recovery method according to Embodiment 2 of the present invention;

[0040] Figure 6 This is a schematic diagram of the structure of a server according to Embodiment 3 of the present invention;

[0041] Figure 7 This is a schematic diagram of the structure of a distributed system provided in Embodiment 4 of the present invention. Detailed Implementation

[0042] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0043] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0044] Example 1

[0045] Figure 1 This is a flowchart illustrating a data backup method provided in Embodiment 1 of the present invention. This embodiment is applicable to data backup and recovery in a distributed database scenario. The method can be executed by a server, which can be implemented in hardware and / or software. Figure 1 As shown, the method includes:

[0046] S110. In response to the backup command, the full data of each shard in the distributed system is backed up to the local node through the backup and recovery management tool, and the consistency point of each shard is read.

[0047] Distributed database systems (i.e., distributed systems) typically use smaller computer systems. Each computer can be located in a separate location and has its own local database. Many computers in different locations are interconnected through a network to form a complete, globally centralized, logically centralized, but physically distributed large database. When distributed data needs to be backed up, it can be done in response to user-initiated backup commands. Sharding can be considered as partitioning a distributed database; a distributed database system often has multiple shards. A consistency point can be considered as the endpoint of a full backup of each shard in the distributed database, achieved through backup and recovery management tools; that is, the full data of each shard ends at the corresponding consistency point.

[0048] For example, in a distributed database, two shards record the position where a full backup of each shard ends. Shard 1 is backed up to position 100, and shard 2 is backed up to position 99. During incremental backups, shard 1 starts from position 100, and shard 2 starts from position 99. To calculate the actual position, a global timestamp can be introduced. For example, the timestamp at position 100 of shard 1 is converted to time 14:00:00, and for shard 2 it is 14:00:03. That is, the consistency position of shard 1 is 14:00:00, and the consistency position of shard 2 is 14:00:03.

[0049] A full backup can be considered a complete copy of the data in a distributed database at a specific point in time. Correspondingly, the full data can be considered all the data in the distributed database at that point in time. An incremental backup, on the other hand, can be considered as each subsequent backup only backing up the data that has been added to or modified since the first full or incremental backup. This means that the first incremental backup covers the data added or modified after the full backup; the second incremental backup covers the data added or modified after the first incremental backup, and so on.

[0050] Backup and recovery management tools can be considered as database tools within a server. Using database tools to back up distributed databases ensures that online services running on the distributed database are not affected, and the load on data nodes is not increased. Specifically, backup and recovery management tools provide an entry point for distributed database backup and recovery, maintain backup and recovery files, and control the backup and recovery process.

[0051] S120. Incremental backup operation is triggered based on the consistency sites of each of the fragments. The incremental backup operation includes real-time backup of incremental data of each fragment according to the consistency sites of each fragment through incremental backup recovery service.

[0052] The incremental backup and recovery service can be considered as a database tool on the same server as the backup and recovery management tool. It can be used to obtain synchronous incremental logs from each shard in real time during backup and to trim the incremental logs during recovery.

[0053] Incremental backups are performed on top of full backups. In other words, an incremental backup can be performed after a full backup, backing up data that has been added to or modified compared to the full backup. Specifically, the consistency point of each shard is the end point of a full backup for that shard. Incremental backups are triggered based on the consistency points of each shard, meaning they are performed after a full backup. The incremental backup recovery service uses a backup function to monitor changes on each shard and then performs real-time incremental data backups based on changes to each shard after the consistency point.

[0054] It should be noted that the full backup and recovery functions of related technologies are separate from the incremental backup and recovery functions, which cannot form a seamless connection. They require manual confirmation and are executed step by step, which is inconvenient for management. However, the embodiments of the present invention record the end position of the full backup of each segment, i.e., the consistency point, after performing a full backup. Then, based on the consistency point of each segment, the incremental backup operation is triggered, which can realize unified management of full backup and incremental backup data at a certain node.

[0055] Optionally, a backup and recovery management tool can be used to back up the full data of each shard in the distributed system to the local node, including:

[0056] Initialize the local backup directory and connection information for each partition using the backup and recovery management tool;

[0057] The backup and recovery management tool remotely calls the physical backup and recovery tool of each shard, so that the physical backup and recovery tool copies the full data of the corresponding shard to the backup and recovery management tool and records the consistency point of the corresponding shard.

[0058] The backup directory can be considered as a backup folder, and the connection information of each segment can represent the specific connection relationship between different segments.

[0059] Physical backup and recovery tools can also be database tools on the same server as backup and recovery management tools. They can perform consistent full physical hot backup and recovery of distributed databases (hot backup is a backup when the system is in normal operating condition), and record consistency points after the backup is completed, which are used to provide the starting synchronization position for incremental backup and recovery servers.

[0060] Specifically, the backup and recovery management tool first receives the backup command, initializes the local backup directory and the connection information for each partition, then remotely calls the physical backup and recovery tools on each partition to begin performing a full physical backup. Next, the physical backup and recovery tools copy the full data from each partition to the node where the backup and recovery management tool resides, records the consistency points for each partition upon completion, and finally, after completing the full backup, the physical backup and recovery tools respond with the backup results to the backup and recovery management tool. The backup and recovery management tool reads the consistency points from the backup information of each partition.

[0061] Optional incremental backup operations include:

[0062] The incremental backup and recovery service monitors the incremental data of each shard starting from the consistency point of each shard; the incremental data of each shard is copied to the backup and recovery management tool in real time through the incremental backup and recovery service.

[0063] Specifically, related technologies use distributed database services to periodically call log collection tools for incremental backups. In contrast, this invention uses an incremental backup and recovery service to register synchronization tasks with each shard starting from the consistency point of each shard, monitors the incremental data of each shard, and automatically pushes new incremental data to the incremental backup and recovery service in real time when there is new incremental data in each shard. The incremental backup and recovery service reads and parses the incremental data and writes it to the local incremental backup file, thus enabling real-time copying of the incremental data of each shard to the node where the backup and recovery management tool is located.

[0064] Figure 2 This is a schematic diagram illustrating the operation process of a data backup method provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the overall process of performing data backup operations on a distributed database is as follows:

[0065] Step 1: The backup and recovery management tool receives the backup command (i.e., the backup instruction) and initializes the local backup directory and the connection information of each partition.

[0066] Step 2: The backup and recovery management tool sends a backup request to the physical backup and recovery tool based on the received backup command.

[0067] Specifically, a backup request can be considered as a request to instruct the physical backup and recovery tool to perform a full physical backup. After sending a backup request to the physical backup and recovery tool, the backup and recovery management tool remotely calls the physical backup and recovery tools on each node to start executing a full physical backup.

[0068] Step 3: The physical backup and recovery tool copies the physical data files of each shard to the node where the backup and recovery management tool is located, and records the consistency point after completion.

[0069] Step 4: After the physical backup and recovery tool completes the full backup, it sends the backup results to the backup and recovery management tool.

[0070] Step 5: The backup and recovery management tool reads the consistency points in the backup information of each shard.

[0071] Step 6: The backup and recovery management tool sends an incremental synchronization request to the incremental backup and recovery server.

[0072] Step 7: The incremental backup recovery server starts performing real-time incremental backups of the logs of each shard based on the consistency point.

[0073] Step 8: The incremental backup and recovery server responds to the backup and recovery management tool with the result of the synchronization backup task.

[0074] This invention provides a data backup method that uses a backup and recovery management tool to back up the full data of each shard in a distributed system to a local node, reads the consistency points of each shard, and then triggers incremental backup operations based on the consistency points of each shard. This enables unified management of full and incremental backup data on a single node and real-time synchronization of incremental data.

[0075] Example 2

[0076] Figure 3 The flowchart is for a data recovery method provided in Embodiment 2 of the present invention. Technical details not described in detail in this embodiment can be found in any of the above embodiments.

[0077] like Figure 3 As shown, the method includes:

[0078] S210. In response to the recovery command, perform local recovery using the backup and recovery management tool based on the full data of each shard in the distributed system backed up by the local node.

[0079] The recovery command includes a recovery time. The backup and recovery management tool provides a query portal for customers, allowing them to query the time range within which global consistency can be restored in real time. The user then specifies a recovery time, such as 14:25:00. The backup and recovery management tool needs to convert the user-provided recovery time into a snapshot time stamp (STS) format that the distributed database can recognize.

[0080] Local recovery can be considered an initialization process when the database starts up. It simulates a full database restart operation locally, that is, it restores the full data of each shard locally. The full data restored locally corresponds to the full data of the full backup operation.

[0081] S220. The backup and recovery management tool is used to copy the full data that was restored locally to each of the partitions.

[0082] S230. An incremental recovery operation is triggered based on the recovery time, wherein the incremental recovery operation includes restoring the incremental data backed up by each of the shards on the local node according to the recovery time.

[0083] The incremental recovery operation starts from the consistency point of each shard and ends at the recovery time specified by the user to restore the incremental data backed up on the local node for each shard.

[0084] The backup and recovery management tool performs local recovery based on the full data of each shard in the distributed system backed up by the local node, copies the locally recovered full data to each shard, and then performs incremental recovery operations starting from the consistency point of each shard and ending at the recovery time specified by the user. It can perform incremental recovery based on any time point within the recovery time interval on the basis of full recovery, and ensures that the recovered data is globally consistent.

[0085] Optionally, local recovery can be performed using backup and recovery management tools based on the full data of each shard in the distributed system backed up by the local node, including:

[0086] Initialize the local backup directory and connection information for each partition using the backup and recovery management tool;

[0087] The backup and recovery management tool calls the physical backup and recovery tool of the local node, so that the physical backup and recovery tool can restore the full data of each shard backed up by the local node to the local node.

[0088] Specifically, when the backup and recovery management tool receives a user's recovery command, it initializes the local backup directory and the connection information of each shard, and calls the physical backup and recovery tool of the local node to restore the full data of each shard backed up on the local node to the position of the consistency point of each shard.

[0089] Optionally, before performing local recovery based on the full data of each shard in the distributed system backed up by the local node using the backup and recovery management tool, the following steps are also included:

[0090] Send a recovery time verification request to the incremental backup and recovery service through the backup and recovery management tool so that the incremental backup and recovery service can verify the recovery time.

[0091] Specifically, verifying the recovery time through incremental backup recovery service is to check whether the converted STS format of the recovery time is incorrect and whether it falls within the time range of global consistency recovery.

[0092] Optional incremental recovery operations include:

[0093] The incremental backup and recovery service is used to trim the incremental data backed up on the local node for each shard. The starting point of the trimming is the consistency point at the end of the full data backup of the corresponding shard, and the ending point of the trimming is the recovery time.

[0094] Using backup and recovery management tools, the incremental data obtained from the trimming is copied to each partition.

[0095] The consistency points of each shard are likely to be different. Pruning can be considered as removing the data between the recovery time and the latest incremental data of each shard. After pruning, the data of each shard in the database is at the recovery time specified by the user, which can maintain global consistency.

[0096] For example, if the user-specified recovery time is 14:25:00, the consistency point at the end of the full backup of shard 1 in the distributed database is 14:00:00, and the consistency point at the end of the full backup of shard 2 is 14:00:03. Therefore, the pruning starts at the consistency point of each shard and ends at the recovery time of 14:25:00. That is, the data retained after pruning is: data from 14:00:00 to 14:25:00 for shard 1; and data from 14:00:03 to 14:25:00 for shard 2.

[0097] Furthermore, offline operations can be selectively performed during backups. When performing a full or incremental backup, the distributed database management node is notified. The management node then goes offline (i.e., data nodes go offline) when no global transactions are running, and re-comes online after the backup operation is complete. This approach ensures global consistency with each backup, eliminating the need for pruning operations during recovery. However, it is only suitable for scenarios where the distributed database has idle business periods; otherwise, it will impact online services. An operation performed from the main entry point of each shard involves modifications to multiple shards and is called a global transaction.

[0098] Figure 4 This is a schematic diagram illustrating the operation process of a data recovery method provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the overall process of data recovery operations for a distributed database is as follows:

[0099] Step 1: The backup and recovery management tool receives the recovery command (i.e., the recovery instruction), which includes the specified recovery time, initializes the local backup directory, and sets the connection information for each partition.

[0100] Step 2: The backup and recovery management tool converts the recovery time provided by the user into STS format and sends a verification request for the recovery time to the incremental backup and recovery server.

[0101] Step 3: The incremental backup and recovery server responds to the backup and recovery management tool with the verification result of the recovery time.

[0102] Step 4: The backup and recovery management tool calls the physical backup and recovery tool on this node to begin performing a full physical recovery locally.

[0103] Step 5: After the physical backup and recovery tool completes the full recovery, it sends a response to the backup and recovery management tool with the recovery results.

[0104] Step 6: The backup and recovery management tool synchronizes the locally restored data to the data directories of each shard.

[0105] Step 7: The backup and recovery management tool sends an incremental recovery request to the incremental backup and recovery server to begin incremental consistency recovery.

[0106] Step 8: The incremental backup and recovery server trims the incremental log files of each shard locally based on the recovery time (STS) provided by the backup and recovery management tool.

[0107] Step 9: The incremental backup and recovery server sends the trimmed log file results to the backup and recovery management tool.

[0108] Step 10: The recovery management tool copies the locally trimmed log files to each shard.

[0109] Figure 5 This is a schematic diagram of time points for a sharded backup and recovery provided in an embodiment of the present invention, as shown below. Figure 5 As shown, full data backups for both shards 1 and 2 started at 13:55:00. The full backup of shard 1 ended at 14:00:00, meaning the consistency point for shard 1 was 14:00:00. The full backup of shard 2 ended at 14:00:03, meaning the consistency point for shard 2 was 14:00:03. The latest incremental data for shard 1 corresponds to the time 14:27:00, and the latest incremental data for shard 2 corresponds to the time 14:27:05. The current time is 14:28:00. 14:00:03–14:27:00 is the user-selectable recovery time range.

[0110] Specifically, the full data consists of the full data for each shard, and after the full backup is completed, the consistency point at the end of the full backup is recorded (the corresponding time is the full backup end time, but this time cannot be used as the basis for starting incremental backups; it can only be used as a reference for the recovery time interval. Incremental backups still start based on the consistency point at the end of each shard's full backup). Incremental data is the data between the consistency point at the end of each shard's full backup and the time point at which the current shard has been incrementally backed up.

[0111] The selectable recovery time range is 14:00:03 to 14:27:00, which is the minimum incremental backup time from the largest end time of the full backup of each shard to the smallest incremental backup time corresponding to the latest incremental data of each shard. For example, if the recovery time is specified as 14:25:00, the full recovery data is the data from the full backup of each shard, and the incremental recovery data is the data retained by trimming the incremental backup of each shard. Specifically, the data retained by shard 1 is the incremental data from the consistency point at the end of the full backup of shard 1 to 14:25:00; the data retained by shard 2 is the incremental data from the consistency point at the end of the full backup of shard 2 to 14:25:00. The data discarded by shard 1 is the incremental data of shard 1 from 14:25:00 to 14:27:00; the data discarded by shard 2 is the incremental data of shard 2 from 14:25:00 to 14:27:05.

[0112] This invention provides a data recovery method that uses a backup recovery management tool to perform local recovery based on the full data of each shard in the distributed system backed up by the local node, and copies the locally recovered full data to each shard. Then, incremental recovery operations are triggered based on the recovery time. This method can perform incremental recovery based on any point in time on the basis of full recovery, and ensures that the recovered data is globally consistent.

[0113] Example 3

[0114] Figure 6 A schematic diagram of a server 10, which can be used to implement embodiments of the present invention, is shown. The server is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0115] like Figure 6As shown, server 10 includes at least one processor 11 and memory, such as read-only memory (ROM) 12 and random access memory (RAM) 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer program stored in ROM 12 or loaded into RAM 13 from storage unit 18. RAM 13 can also store various programs and data required for the operation of server 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. Input / output (I / O) interface 15 is also connected to bus 14.

[0116] Multiple components in server 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows server 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0117] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as method XXX.

[0118] In some embodiments, Method Embodiments 1 and 2 may be implemented as computer programs tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute Method Embodiments 1 and 2 by any other suitable means (e.g., by means of firmware).

[0119] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0121] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0122] To provide interaction with the user, the systems and techniques described herein can be implemented on a server having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the server. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0123] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0124] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0125] Example 4

[0126] Figure 7 This is a schematic diagram of the structure of a distributed system provided in Embodiment 4 of the present invention. Figure 7 As shown, the distributed system includes: multiple database nodes 62 and a server 10 provided in this embodiment of the invention, wherein the multiple database nodes 62 are divided into at least one shard.

[0127] Server 10 is equipped with a variety of database tools, including backup and recovery management tools, incremental backup and recovery services, and physical backup and recovery tools.

[0128] This invention utilizes a backup and recovery management tool to back up the full data of each shard in a distributed system to a local node, reads the consistency points of each shard, and then triggers incremental backup operations based on the consistency points of each shard. This enables unified management of full and incremental backup data, and periodic synchronization of incremental data. The backup and recovery management tool performs local recovery based on the full data of each shard in the distributed system backed up by the local node, copies the locally recovered full data to each shard, and then triggers incremental recovery operations based on the recovery time. This allows for incremental recovery based on any point in time, on top of full recovery, and ensures that the recovered data is globally consistent.

[0129] The distributed system provided in this embodiment four can be used to implement the data backup or data recovery methods provided in any of the above embodiments, and has the corresponding functions and beneficial effects.

[0130] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0131] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data backup method, characterized in that, include: In response to the backup command, the backup and recovery management tool backs up the full data of each shard in the distributed system to the local node, and reads the consistency point of each shard, wherein the full data of each shard ends at the consistency point of the corresponding shard. Incremental backup operations are triggered based on the consistency points of each shard. The incremental backup operations include real-time backup of incremental data of each shard according to the consistency points of each shard through an incremental backup recovery service. The incremental backup recovery service is a database tool on the same server as the backup recovery management tool, which is used to obtain synchronous incremental logs from each shard in real time during backup and to prune the incremental logs during recovery. The incremental backup operation further includes: monitoring the incremental data of each of the fragments starting from the consistency point of each fragment through the incremental backup recovery service; and copying the incremental data of each fragment to the backup recovery management tool in real time through the incremental backup recovery service.

2. The method according to claim 1, characterized in that, The backup and recovery management tool backs up the full data of each shard in the distributed system to the local node, including: The local backup directory and the connection information of each of the fragments are initialized using the backup and recovery management tool. The backup and recovery management tool remotely calls the physical backup and recovery tool of each shard, so that the physical backup and recovery tool copies the full data of the corresponding shard to the node where the backup and recovery management tool is located and records the consistency point of the corresponding shard.

3. A data recovery method, characterized in that, include: The backup and recovery management tool sends a recovery time verification request to the incremental backup and recovery service so that the incremental backup and recovery service can verify the recovery time. The incremental backup and recovery service is a database tool on the same server as the backup and recovery management tool. It is used to obtain synchronous incremental logs from each shard in real time during backup and to prune the incremental logs during recovery. In response to a recovery command, the backup and recovery management tool performs local recovery based on the full data of each shard in the distributed system backed up by the local node; wherein, the recovery command includes the recovery time. The backup and recovery management tool is used to copy the fully restored data from the local storage to each of the partitions. An incremental recovery operation is triggered based on the recovery time, and the incremental recovery operation includes restoring the incremental data backed up by each shard on the local node according to the recovery time.

4. The method according to claim 3, characterized in that, The backup and recovery management tool performs local recovery based on the full data of each shard in the distributed system backed up by the local node, including: The local backup directory and the connection information of each of the fragments are initialized using the backup and recovery management tool. The backup and recovery management tool calls the physical backup and recovery tool of the local node, so that the physical backup and recovery tool restores the full data of each of the shards backed up by the local node to the local node.

5. The method according to claim 3, characterized in that, The incremental recovery operation includes: The incremental backup and recovery service is used to trim the incremental data backed up on the local node for each of the aforementioned shards; wherein the starting point of the trimming is the consistency point at the end of the full data backup of the corresponding shard, and the ending point of the trimming is the recovery time. The backup and recovery management tool is used to copy the trimmed incremental data to each of the aforementioned fragments.

6. A server, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data backup method as described in any one of claims 1-2 or the data recovery method as described in any one of claims 3-5.

7. A distributed system, characterized in that, include: Multiple database nodes and the server as described in claim 6, wherein the multiple database nodes are divided into at least one shard.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the data backup method as described in any one of claims 1-2 or the data recovery method as described in any one of claims 3-5.

Images