Data backup method and device and data backup recovery method and device for distributed database

Abstract

The embodiment of the invention provides a data backup recovery method for a distributed database, which is executed through a target node in the distributed database and comprises the following steps: determining to recover interrupted backup operation for target data, the target data comprising a plurality of partitions, and the plurality of partitions are divided into a plurality of partition groups; and searching a first record file in a target storage medium for data backup, wherein the first record file is used for recording a partition list in the partition group which completes backup. And determining a plurality of candidate partitions to be processed according to the searched first record file. And for each candidate partition, a second record file corresponding to the candidate partition is searched from a target storage medium, whether the candidate partition is backed up or not is determined according to a search result, and the second record file is used for marking the corresponding partition as a backed-up partition.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of databases, and in particular, to a data backup, data backup and recovery method and device for a distributed database. Background Art

[0002] In a distributed system environment, especially in large-scale data storage scenarios, data backup is a critical task. However, in actual operations, due to various unforeseen factors such as network fluctuations and hardware failures, the ongoing data backup may be interrupted.

[0003] In the conventional technology, the method for restoring the interrupted data backup is to restart the entire backup process, which is not only time-consuming but also inefficient. Therefore, a more efficient data backup and recovery solution is needed. Summary of the invention

[0004] One or more embodiments of this specification describe a data backup and recovery method for a distributed database, which can significantly improve the overall efficiency of data backup.

[0005] In a first aspect, a method for backing up and restoring data in a distributed database is provided, which is executed by a target node in the distributed database and includes: Determine to resume the interrupted backup operation for target data, wherein the target data includes a plurality of partitions, and the plurality of partitions are divided into a plurality of partition groups; Searching for a first record file in a target storage medium for data backup, wherein the first record file is used to record a list of partitions in a partition group that has completed backup; Determine a plurality of candidate partitions to be processed according to the searched first record file; For each candidate partition, a second record file corresponding to the candidate partition is searched from the target storage medium, and whether to back up the candidate partition is determined according to the search result, wherein the second record file is used to mark the corresponding partition as a backed up partition.

[0006] In a second aspect, a method for backing up data in a distributed database is provided, which is executed by any first node in the distributed database, and includes: Determine a plurality of target partition groups to be backed up by the first node; the plurality of target partition groups are at least some of the partition groups among the plurality of partition groups obtained by grouping a plurality of partitions included in the target data to be backed up; Backing up the target partition groups and performing a first record and a second record; The first record includes, after any target partition group completes the backup, creating a corresponding first record file in the target storage medium for data backup, the first record file being used to record the partition list in the target partition group; The second record includes, after the backup of any target partition in any target partition group is completed, creating a corresponding second record file in the target storage medium, which is used to mark the target partition as a backed up partition; wherein the first and second record files are used to restore the interrupted backup operation for the target data.

[0007] In a third aspect, a data backup and recovery device of a distributed database is provided, which is arranged in a target node of the distributed database, and comprises: a determining unit, configured to determine to restore the interrupted backup operation for target data, wherein the target data includes a plurality of partitions, and the plurality of partitions are divided into a plurality of partition groups; A search unit, used for searching a first record file in a target storage medium for data backup, wherein the first record file is used for recording a list of partitions in a partition group that has completed backup; The determining unit is further used to determine a plurality of candidate partitions to be processed according to the searched first record file; The search unit is used to search for a second record file corresponding to each candidate partition from the target storage medium, and determine whether to back up the candidate partition according to the search result, wherein the second record file is used to mark the corresponding partition as a backed up partition.

[0008] In a fourth aspect, a data backup device of a distributed database is provided, which is arranged at any first node of the distributed database, and includes: a determining unit, configured to determine a plurality of target partition groups to be backed up by the first node; the plurality of target partition groups being at least some of the plurality of partition groups obtained by grouping a plurality of partitions included in the target data to be backed up; A backup unit for backing up the plurality of target partition groups and performing a first record and a second record; The backup unit comprises: A creation submodule, used for creating a corresponding first record file in a target storage medium for data backup after the backup of any target partition group is completed, wherein the first record file is used for recording a partition list in the target partition group; After any target partition in any target partition group is backed up, a corresponding second record file is created in the target storage medium, which is used to mark the target partition as a backed up partition; wherein the first and second record files are used to restore the interrupted backup operation for the target data.

[0009] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method of the first or second aspect.

[0010] In a sixth aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of the first or second aspect is implemented.

[0011] The data backup and recovery method of a distributed database provided in one or more embodiments of the present specification can quickly determine the partitions that need to restore the data backup based on the first record file corresponding to the backed-up partition group and the second record file corresponding to the backed-up partition stored in the target storage medium for data backup when the data backup is interrupted, and perform data backup for the determined partitions. In other words, this solution can quickly locate the location where the backup failed after the data backup is interrupted, and continue to execute the remaining backup process from there, thereby significantly improving the efficiency of data backup as a whole, and thus improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 A schematic diagram of a distributed database in an example of this specification is shown; Figure 2 A schematic diagram of an implementation scenario of an embodiment disclosed in this specification; Figure 3 A flow chart of a distributed database data backup method according to an embodiment of the present specification is shown; Figure 4 A schematic diagram showing first and second recording files in one example of the present specification; Figure 5 A flow chart of a data backup and recovery method for a distributed database according to an embodiment of the present specification is shown; Figure 6 A schematic diagram of a data backup and recovery method in an example of this specification is shown; Figure 7 A schematic diagram of a data backup and recovery device for a distributed database according to an embodiment of the present specification is shown; Figure 8A schematic diagram of a data backup device for a distributed database according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0014] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0015] As mentioned above, traditional data backup and recovery solutions are time-consuming and inefficient. To solve this problem, there are currently several improvement solutions: First, breakpoint resume (i.e., restore data backup) is achieved through snapshots and list files, that is, first record the database snapshot information and back up the data file list to the corresponding preset list file. After that, the file status of the data file in the preset list file is updated after the data transmission is successful, and the file status is used to achieve breakpoint resume of the backup data during breakpoint resume. However, this method does not take into account the complexity of the distributed environment, especially how to accurately determine the partitions that have not been backed up when multiple database nodes perform data backup in parallel, and this method does not propose an effective solution.

[0016] Second, by using a distributed backup system for backup processing, that is, splitting the backup tasks and resuming the data after a breakpoint, this method can reduce the time it takes to restore the backup in the event of a backup failure, but it is inefficient when processing a large number of partitions and lacks fine management of the specific partition status.

[0017] Third, breakpoint resume is achieved based on cloud storage technology. Although this method improves storage space utilization, it does not provide sufficient support for partition-level breakpoint resume in a distributed environment.

[0018] Fourth, under the NDMP protocol, according to the context information of the file to be backed up at the transmission interruption and the first part of the content stored before the transmission interruption, the second part of the content that was not successfully transmitted at the transmission interruption is determined, and the second part of the content is continued to be sent. However, this method is mainly for data backup under the NDMP protocol and is not suitable for data backup scenarios in distributed databases.

[0019] In summary, the above improvement schemes have the following disadvantages: a. Support for large-scale concurrent execution of tasks is weak; b. Lack of fine-grained management capabilities at the individual partition level; c. Poor performance in error detection and recovery in complex environments.

[0020] To this end, in an embodiment of the present specification, an improved method for data backup and recovery is proposed. First, during the data backup process, each node in the distributed database stores a first record file corresponding to the backed-up partition group and a second record file corresponding to the backed-up partition in the target storage medium used for data backup. Afterwards, after the data backup is interrupted, the target node used to restore the data backup reads the first record file and the second record file from the target storage medium, and based on them, quickly determines the partitions that need to restore the data backup, and restores the data backup for the determined partitions. In other words, this solution can quickly locate the location where the backup failed after the data backup is interrupted, and continue to execute the remaining backup process from there, thereby significantly improving the efficiency of data backup as a whole, and thus improving the user experience.

[0021] Figure 1 A schematic diagram of a distributed database in one example of the present specification is shown. Figure 1 In a distributed database, data is distributedly stored in each node (node ​​A-node C). Specifically, a data table can be divided into multiple data partitions (hereinafter referred to as partitions) according to the partition key. The partitions are stored in each node respectively. Each node can store one or more partitions, and the partitions stored between nodes can overlap with each other.

[0022] For example, assume that there is a data table t1 in the distributed database, and the data table t1 can be divided into 6 partitions, represented as: p1-p6, then node A can store partitions p1, p2 and p3, node B can store partitions p3, p4 and p5, and node C can store partitions p4, p5 and p6.

[0023] It should be understood that Figure 1 This is only an exemplary description. In practice, a distributed database may include a larger number of nodes, and each node may also store one or more partitions divided from other data tables.

[0024] Figure 2 A schematic diagram of an implementation scenario of an embodiment disclosed in this specification. Figure 2 In the distributed database, when you need to back up the target data table, you can first divide the multiple partitions of the target data table into several partition groups, and assign each partition group to each node for backup. For example, assign partition group A (including partitions p1-p2) to node X, and assign partition group B (including partitions p3-p5) to node Y.

[0025] Afterwards, each node may back up the assigned target partition group to a storage medium shared by all nodes, and create a first record file corresponding to the backed up partition group and a second record file corresponding to the backed up partition in the shared storage medium. Figure 2 In the example, it is assumed that each partition in the partition group A has been backed up, so that the storage medium stores the first record file corresponding to the partition group A and the second record files corresponding to the partition p1 and the partition p2. In addition, it is assumed that the partition p5 in the partition group B has not been backed up, so the storage medium only stores the second record files corresponding to the partition p3 and the partition p4, but does not store the first record file corresponding to the partition group B.

[0026] It should be understood that for the interrupted backup operation, the target node for restoring the data backup reads the first record file and the second record file from the shared storage medium, and based on them determines the partition where the data backup needs to be restored and backs it up (the specific determination method will be described later).

[0027] Figure 3 The flowchart of the data backup method of the distributed database according to one embodiment of the present specification is shown. The method can be executed by any node (hereinafter referred to as the first node) in the distributed database. Figure 3 As shown, the method may include the following steps: Step S302: determine a number of target partition groups to be backed up by the first node.

[0028] The aforementioned target partition groups may be at least part of the partition groups among the partition groups obtained by grouping multiple partitions included in the target data (including one or more data tables) to be backed up.

[0029] In an example, the multiple partitions may be grouped based on the log stream, that is, each partition in the same partition group belongs to the same log stream.

[0030] Specifically, the root service (RS) node in the distributed database may first receive the SQL statement input by the user, which indicates the target data to be backed up. Afterwards, the root service node allocates corresponding partition groups to each node according to the pre-recorded partition information of the target data (i.e., the identifier of the node where each partition of the target data is located). Of course, in practice, corresponding partition groups may also be allocated to each node in combination with load information (described in detail later).

[0031] Afterwards, the root service node can send data backup instructions to each node respectively, wherein the data backup instruction sent to any node can include the identifiers of several target partition groups to be backed up by the node. In other words, the first node can determine several target partition groups to be backed up by it according to the data backup instruction.

[0032] by Figure 1 For example, the target data to be backed up may be data table t1, and it is assumed that the following four partition groups are obtained for the six partitions included therein: partition group 1: p1, p2, partition group 2: p3, partition group 3: p4, p5, partition group 4: p6. Assuming that the load information of node B is large, node A may back up partition group 1 and partition group 2, node B may partition partition group 3, and node C may back up partition group 4.

[0033] It should be understood that this is just an example. In practice, a partition group may usually include 1,000 or even more partitions, and each node used for data backup may be a part of the nodes in the distributed database, which is not limited in this specification.

[0034] Of course, in practice, the user can also pre-configure several target partition groups for each node to back up, and then each node periodically backs up the corresponding target partition groups. In other words, the backup operation for the target data is not triggered by the user through the root service node, but is executed periodically by each node.

[0035] Step S304: back up a plurality of target partition groups, and perform a first record and a second record.

[0036] The first node may start multiple threads, and use the multiple threads to back up several target partition groups in parallel, thereby improving the backup efficiency for the several target partition groups.

[0037] In practice, before the first node backs up several target partition groups, the root service node can first perform a partition list backup operation, that is, back up the partition list containing the identifiers of all partitions (also called the initial partition list) to the target storage medium shared by each node for data backup. Of course, each partition in the initial partition list is organized according to the partition group. For example, the initial partition list will record the corresponding relationship between the identifier of the partition group and the identifier of the partition.

[0038] Specifically, the first node can read the identifiers of each target partition contained in each target partition group from the target storage medium according to the identifiers of the target partition groups. Afterwards, the first node reads the data of each target partition based on the identifier of each target partition, stores it in the target storage medium, and performs the first record and the second record.

[0039] The first record includes, after any target partition group completes the backup, creating a corresponding first record file (also called a partition group checkpoint file) in the target storage medium, the first record file is used to record the partition list in the target partition group. In one example, the partition list is formed by the identifiers of each target partition in the target partition group.

[0040] The second record includes creating a corresponding second record file (also called partition checkpoint file) in the target storage medium after any target partition in any target partition group is backed up, and the second record file is used to mark the target partition as a backed up partition.

[0041] In one embodiment, the second record file is a placeholder file whose content is empty, thereby saving storage space.

[0042] More specifically, the second record file may be stored in a partition directory to which the corresponding partition that has been backed up in the target storage medium belongs.

[0043] Figure 4 A schematic diagram showing first and second recording files in one example of the present specification, Figure 4 In the example, the first record file may be created when all partitions in any partition group are backed up, or when any partition group is backed up, so it can also be expressed that the first record file corresponds to the backed up partition group. The second record file may be created when any partition is backed up, so it can also be expressed that the second record file corresponds to the backed up partition.

[0044] It can be seen that in this solution, during the data backup process, each node in the distributed database stores a first record file corresponding to the backed up partition group and a second record file corresponding to the backed up partition in a storage medium shared by multiple nodes and used for data backup.

[0045] In addition, from the above description, it can be seen that this solution uses partition groups as scheduling units for data backup, and each partition in a partition group belongs to a log stream, so the data backup method of this solution can improve the recovery efficiency of the backed up data.

[0046] Finally, as mentioned above, in actual operation, due to the influence of various unforeseen factors such as network fluctuations and hardware failures, the ongoing data backup may be interrupted. The following describes the recovery process after the above data backup is interrupted.

[0047] Figure 5 A flow chart of a data backup and recovery method for a distributed database according to an embodiment of the present specification is shown. The method can be executed by a target node in the distributed database.

[0048] In one example, the target node can be selected from each node by the root service node based on the load information of each node. For example, a node with smaller load information is selected as the target node. Of course, in practice, the target node can also be selected in combination with the partition information. For example, the root service RS node can analyze the log stream that failed to be backed up, and then select the node in which each partition belonging to the log stream is stored as the above-mentioned target node.

[0049] by Figure 1 For example, assuming that the backup of partitions p4 and p5 fails, node B or node C can be selected as the target node.

[0050] In another example, the target node may also be pre-specified by the user.

[0051] like Figure 5 As shown, the method may include the following steps: Step S502: Determine to resume the interrupted backup operation for the target data.

[0052] Among them, when the target node is dynamically selected by the root service node in combination with partition information and load information, the target node can receive the data backup and recovery instruction sent by the root service node, and then determine the need for data backup and recovery based on the data backup and recovery instruction.

[0053] When the target node is pre-specified by the user, it can detect the backup operation of each node for the target data, and after detecting that the backup operation of any node is interrupted, it is determined that data backup and recovery is required.

[0054] Step S504: searching the target storage medium for data backup for a first record file, where the first record file is used to record a list of partitions in the partition group that has completed the backup.

[0055] In one example, the partition list is formed by partition identifiers of each partition in the partition group that has completed the backup.

[0056] As mentioned above, the first record file corresponds to the backed-up partition group, or in other words, it is created when a partition group has completed the backup, so that the partition that has completed the backup can be determined based on it.

[0057] In addition, it should be understood that when multiple partition groups have completed backup, multiple first record files can be searched.

[0058] Step S506: determining a plurality of candidate partitions to be processed according to the searched first record file.

[0059] The multiple candidate partitions to be processed here can also be understood as multiple candidate partitions to be restored and backed up.

[0060] Specifically, the initial partition list (including the identifiers of the multiple partitions divided for the target data) can be read from the target storage medium, and then the multiple candidate partitions to be processed can be determined by comparing the initial partition list with the partition list in the searched first record file. For example, the partitions in the partition list included in each first record file can be removed from the initial partition list, and the remaining partitions can be determined as the multiple candidate partitions to be processed. Here, the partitions in the partition list included in the first record file are removed because these partitions are all partitions that have completed the backup.

[0061] Of course, in practice, multiple candidate partitions to be processed may also be determined by other methods, which is not limited in this specification.

[0062] Step S508, for each candidate partition, searching the target storage medium for a second record file corresponding to the candidate partition, and determining whether to back up the candidate partition based on the search result, wherein the second record file is used to mark the corresponding partition as a backed up partition.

[0063] As mentioned above, the second record file corresponds to the backed up partition and is usually stored in the partition directory to which the corresponding partition belongs. Thus, for each candidate partition, the second record file corresponding to the candidate partition can be found from the partition directory corresponding to the candidate partition.

[0064] In one embodiment, the second record file is a placeholder file whose content is empty, so that for any candidate partition, if the corresponding second record file is found, the candidate partition is skipped; if the corresponding second record file is not found, the candidate partition is backed up.

[0065] In another embodiment, the second record file records the partition identifier of the corresponding partition, so that each candidate partition can be screened according to the partition identifier recorded in each found second record file, and the screened candidate partitions are backed up.

[0066] Figure 6 A schematic diagram of a data backup and recovery method in an example of this specification is shown. Figure 6 In the example, it is assumed that the target data to be backed up includes partitions p1 to p6, and partitions p1, p3 and p5 belong to partition group A, and partitions p2, p4 and p6 belong to partition group B. After backing up the target data, it is assumed that the target storage medium for data backup stores a first record file corresponding to partition group A (wherein the identifiers of partitions p1, p3 and p5 are recorded), a second record file corresponding to partition p2, and a second record file corresponding to partition p4. Based on the first record file, partitions p1, p3 and p5 can be excluded, and based on the two second record files, partitions p2 and p4 can be excluded. Finally, it can be determined that the partition that needs to be restored is partition p6.

[0067] In summary, in this solution, the initial partitions can be screened based on the first record file corresponding to the backed-up partition group, and since the screening process is performed in units of partition groups, the efficiency of data backup and recovery can be ensured. Based on the second record file corresponding to the backed-up partition, the remaining partitions can be screened again, thereby avoiding backing up the partitions that have been backed up again, thereby saving time and resources.

[0068] It should be noted that this solution is based on the first record file and the second record file. In fact, it adopts a multi-level checkpoint mechanism to determine the partition that needs to restore the data backup. By adopting this multi-level checkpoint mechanism, the state changes of each operation can be tracked more accurately, and even in extreme cases, it can ensure that important information will not be lost. In addition, this solution also takes into account the needs of cross-node collaboration, so that good consistency and reliability can be maintained even in a distributed deployment mode. The ultimate effect is to greatly shorten the time period from discovering a fault to restoring normal operation, and improve the stability and maintainability of the system.

[0069] Compared with the existing technology, this solution can bring the following technical effects: 1. Accurate breakpoint resumption: A multi-level checkpoint mechanism is used to accurately determine which partitions have been backed up and which need to be backed up again.

[0070] 2. Efficient state management: Based on the first and second record files, the amount of state to be verified one by one is reduced, and the efficiency of data backup and recovery is improved.

[0071] 3. Support for parallel processing: Even in a parallel processing environment, it can accurately identify the partitions that have not been fully backed up and continue to back up.

[0072] 4. Reduce duplication of work: Avoid unnecessary repeated backups, saving time and resources.

[0073] 5. Improve backup efficiency: Adopt a multi-level checkpoint mechanism to significantly shorten the recovery time after backup interruption.

[0074] 6. Enhanced system stability: In a distributed environment, the reliability and stability of backup operations are improved.

[0075] The following is an explanation of the innovative features of this solution: 1. Multi-level checkpoint mechanism: The combination of the first record file corresponding to the backed-up partition group and the second record file corresponding to the backed-up partition can ensure that the backup can be accurately continued from the breakpoint after the data backup is interrupted, and even if the backup of some partitions in the partition group fails, it can accurately determine which partitions have not been backed up.

[0076] 2. Flexible task scheduling mechanism: allows the root service node to dynamically adjust the workload of each node according to actual conditions, thereby optimizing resource utilization.

[0077] 3. Parallel processing support: In a distributed environment, it can effectively handle parallel backup operations to ensure that the status of each partition is accurately recorded and managed.

[0078] 4. Efficient recovery process: The multi-level checkpoint mechanism is adopted to reduce the time required for data backup failure recovery and improve the overall efficiency of data backup.

[0079] Corresponding to the data backup and recovery method of the distributed database, an embodiment of the present specification also provides a data backup and recovery device for the distributed database, which is set in a target node of the distributed database, such as Figure 7 As shown, the device may include: The determination unit 702 is configured to determine to restore the interrupted backup operation for the target data, wherein the target data includes a plurality of partitions, and the plurality of partitions are divided into a plurality of partition groups.

[0080] The search unit 704 is used to search the target storage medium for data backup for a first record file, where the first record file is used to record a list of partitions in the partition group that has completed the backup.

[0081] The determining unit 702 is further configured to determine a plurality of candidate partitions to be processed according to the searched first record file.

[0082] The search unit 706 is used to search for a second record file corresponding to each candidate partition from the target storage medium, and determine whether to back up the candidate partition according to the search result, wherein the second record file is used to mark the corresponding partition as a backed up partition.

[0083] In one embodiment, the determining unit 702 includes: The reading submodule 7022 is used to read an initial partition list from a target storage medium, including multiple partitions; The comparison submodule 7024 is used to determine a plurality of candidate partitions to be processed by comparing the initial partition list with the partition list in the searched first record file.

[0084] In one embodiment, the searching unit 706 is specifically configured to: A second record file corresponding to the candidate partition is searched from the partition directory corresponding to the candidate partition in the target storage medium.

[0085] In one embodiment, the second record file is a placeholder file, and its content is empty.

[0086] In one embodiment, each partition in a single partition group belongs to the same log stream.

[0087] In one embodiment, the target node is selected from other nodes by a root service RS node in a distributed database at least according to load information of other nodes except the root service RS node.

[0088] The functions of the functional modules of the device in the above-mentioned embodiment of this specification can be implemented through the steps of the above-mentioned method embodiment. Therefore, the specific working process of the device provided by one embodiment of this specification will not be repeated here.

[0089] A data recovery device for a distributed database provided by an embodiment of the present specification can significantly improve the overall efficiency of data backup.

[0090] Corresponding to the data backup method of the distributed database, an embodiment of the present specification also provides a data backup device for the distributed database, which is arranged at any first node in the distributed database, such as Figure 8 As shown, the device may include: The determining unit 802 is configured to determine a plurality of target partition groups to be backed up by the first node, wherein the plurality of target partition groups are at least part of the plurality of partition groups obtained by grouping a plurality of partitions included in the target data to be backed up.

[0091] The backup unit 804 is used to back up a plurality of target partition groups and perform a first record and a second record.

[0092] The backup unit 804 includes: A creation submodule 8042 is used to create a corresponding first record file in a target storage medium for data backup after the backup of any target partition group is completed, and the first record file is used to record the partition list in the target partition group; After any target partition in any target partition group is backed up, a corresponding second record file is created in the target storage medium, which is used to mark the target partition as a backed up partition; wherein the first and second record files are used to restore the interrupted backup operation for the target data.

[0093] In one embodiment, the backup unit 804 further includes: The start submodule 8044 is used to start multiple threads and back up multiple target groups in parallel through the multiple threads.

[0094] The functions of the functional modules of the device in the above-mentioned embodiment of this specification can be implemented through the steps of the above-mentioned method embodiment. Therefore, the specific working process of the device provided by one embodiment of this specification will not be repeated here.

[0095] A data backup device for a distributed database provided by an embodiment of the present specification creates a first and a second record file during the data backup process. The two record files are helpful to quickly determine the partition that needs to restore the data backup.

[0096] According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 3 or Figure 5 The method described.

[0097] According to another embodiment of the present invention, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the Figure 3 or Figure 5 The method described.

[0098] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the medium or device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0099] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0100] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of this specification. It should be understood that the above description is only the specific implementation method of this specification and is not intended to limit the protection scope of this specification. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of this specification should be included in the protection scope of this specification.

Claims

1. A data backup and recovery method for a distributed database, executed by a target node in the distributed database, comprising: Determine to resume the interrupted backup operation for target data, wherein the target data includes a plurality of partitions, and the plurality of partitions are divided into a plurality of partition groups; Searching for a first record file in a target storage medium for data backup, wherein the first record file is used to record a list of partitions in a partition group that has completed backup; Determine a plurality of candidate partitions to be processed according to the searched first record file; For each candidate partition, a second record file corresponding to the candidate partition is searched from the target storage medium, and whether to back up the candidate partition is determined according to the search result, wherein the second record file is used to mark the corresponding partition as a backed up partition.

2. The method according to claim 1, wherein: The determining of a plurality of candidate partitions to be processed includes: Reading an initial partition list from the target storage medium, including the plurality of partitions; By comparing the initial partition list with the partition list in the searched first record file, a plurality of candidate partitions to be processed are determined.

3. The method according to claim 1, wherein: The step of searching the target storage medium for a second record file corresponding to the candidate partition includes: A second record file corresponding to the candidate partition is searched from the partition directory corresponding to the candidate partition in the target storage medium.

4. The method according to claim 1, wherein: The second record file is a placeholder file, and its content is empty.

5. The method according to claim 1, wherein: The partitions in a single partition group belong to the same log stream.

6. The method according to claim 1, wherein: The target node is a root service RS node in the distributed database, and is selected from other nodes at least according to load information of other nodes except the root service RS node.

7. A method for backing up data in a distributed database, executed by any first node in the distributed database, comprising: Determining a number of target partition groups to be backed up by the first node; The plurality of target partition groups are at least some of the partition groups among the plurality of partition groups obtained by grouping the plurality of partitions included in the target data to be backed up; Backing up the target partition groups and performing a first record and a second record; The first record includes, after any target partition group completes the backup, creating a corresponding first record file in the target storage medium for data backup, the first record file being used to record the partition list in the target partition group; The second record includes, after the backup of any target partition in any target partition group is completed, creating a corresponding second record file in the target storage medium, which is used to mark the target partition as a backed up partition; wherein the first and second record files are used to restore the interrupted backup operation for the target data.

8. The method according to claim 7, wherein: The backing up of the plurality of target groups comprises: A plurality of threads are started, and the plurality of target groups are backed up in parallel through the plurality of threads.

9. A data backup and recovery device for a distributed database, arranged in a target node of the distributed database, comprising: a determining unit, configured to determine to restore the interrupted backup operation for target data, wherein the target data includes a plurality of partitions, and the plurality of partitions are divided into a plurality of partition groups; A search unit, used for searching a first record file in a target storage medium for data backup, wherein the first record file is used for recording a list of partitions in a partition group that has completed backup; The determining unit is further used to determine a plurality of candidate partitions to be processed according to the searched first record file; The search unit is used to search for a second record file corresponding to each candidate partition from the target storage medium, and determine whether to back up the candidate partition according to the search result, wherein the second record file is used to mark the corresponding partition as a backed up partition.

10. The device according to claim 9, wherein: The determining unit comprises: A reading submodule, configured to read an initial partition list from the target storage medium, including the plurality of partitions; The comparison submodule is used to determine a plurality of candidate partitions to be processed by comparing the initial partition list with the partition list in the searched first record file.

11. The device according to claim 9, wherein: The searching unit is specifically used for: A second record file corresponding to the candidate partition is searched from the partition directory corresponding to the candidate partition in the target storage medium.

12. The device according to claim 9, wherein: The second record file is a placeholder file, and its content is empty.

13. The device according to claim 9, wherein: The partitions in a single partition group belong to the same log stream.

14. The device according to claim 9, wherein: The target node is a root service RS node in the distributed database, and is selected from other nodes at least according to load information of other nodes except the root service RS node.

15. A data backup device for a distributed database, arranged at any first node in the distributed database, comprising: a determination unit, configured to determine a number of target partition groups to be backed up by the first node; The plurality of target partition groups are at least some of the partition groups among the plurality of partition groups obtained by grouping the plurality of partitions included in the target data to be backed up; A backup unit for backing up the plurality of target partition groups and performing a first record and a second record; The backup unit comprises: A creation submodule, used for creating a corresponding first record file in a target storage medium for data backup after the backup of any target partition group is completed, wherein the first record file is used for recording a partition list in the target partition group; After any target partition in any target partition group is backed up, a corresponding second record file is created in the target storage medium, which is used to mark the target partition as a backed up partition; wherein the first and second record files are used to restore the interrupted backup operation for the target data.

16. The device according to claim 15, wherein: The backup unit also includes: The startup submodule is used to start multiple threads and back up the multiple target groups in parallel through the multiple threads.

17. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

18. A computing device comprising a memory and a processor, wherein: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 8 is implemented.

Images