A data backup method, a data recovery method and a device

Abstract

The embodiment of the present application provides a data backup method, device, electronic device and computer-readable storage medium, which relates to the field of distributed database technology. The method includes: for each sub-backup data obtained by segmenting the data to be backed up, performing the following backup operations: receiving a backup request for the sub-backup data; in response to the backup request, obtaining a data snapshot of the database node corresponding to the sub-backup data; based on the data range of the sub-backup data, filtering the file to be backed up from the data snapshot; uploading the file to be backed up to the backup storage node. The embodiment of the present application completes data backup by obtaining a data snapshot of the database node and filtering the file to be backed up from the data snapshot, thereby solving the resource conflict problem between data backup and online business.

Description

Technical Field

[0001] The present application relates to the technical field of distributed databases. Specifically, the present application relates to a data backup method, a data recovery method and a device. Background Art

[0002] Distributed databases are the core of modern data processing. Distributed database systems usually use smaller computer systems. Each computer can be placed in a separate place. Each computer may have a complete copy or a partial copy of the DBMS (Database Management System) and its own local database. Many computers located in different locations are interconnected through the network to form a complete, global, logically centralized, and physically distributed large database. These databases can distribute data among multiple computers to achieve high availability, scalability, and fault tolerance.

[0003] Distributed databases usually shard data and distribute it to multiple database nodes. When backing up data in a distributed database, existing solutions generally use a logical scan solution: that is, obtaining a data snapshot based on the data shard (i.e., the primary backup data), and scanning the data snapshot to obtain the data to be backed up, so as to back up the data determined by the scan. Since scanning data snapshots consumes a large amount of IO (Input / Output) and CPU (Central Processing Unit) resources, it will cause the request latency of online services to increase and the throughput during reading and writing to decrease, and there is a resource conflict problem between data backup and online services. Summary of the invention

[0004] The embodiment of the present application provides a data backup method, a data recovery method and a device, which can solve the resource conflict problem between data backup and online business. The technical solution is as follows:

[0005] According to one aspect of an embodiment of the present application, a data backup method is provided, which is applied to multiple database nodes in a distributed database, and the method includes:

[0006] For each sub-backup data obtained by splitting the data to be backed up, perform the following backup operations:

[0007] receiving a backup request for the sub-backup data;

[0008] In response to a backup request, a data snapshot of a database node corresponding to the sub-backup data is obtained; wherein each sub-backup data is distributedly stored in a plurality of database nodes, and each sub-backup data corresponds to a database node;

[0009] Based on the data range of the sub-backup data, the files to be backed up are obtained by filtering from the data snapshot;

[0010] Upload the files to be backed up to the backup storage node.

[0011] In a possible implementation, after uploading the to-be-backed-up files to the backup storage node, the method further includes:

[0012] The timestamp, data range and file name of the data snapshot are sent to the backup client, so that the backup client saves the timestamp, data range and file name into the meta information file, and uploads the meta information file to the backup storage node; wherein, uploading the meta information file to the backup storage node occurs after the backup operation is completed for all sub-backup data.

[0013] In a possible implementation, before obtaining the data snapshot of the database node, the following steps are included:

[0014] The cached data in the memory of the distributed database is written to the disk; wherein the cached data includes database read and write data to be processed that is initiated before the timestamp of the data snapshot; and the data snapshot includes the cached data.

[0015] According to another aspect of an embodiment of the present application, a data recovery method is provided, which is applied to multiple database nodes in a distributed database, and the method includes:

[0016] Get data recovery request;

[0017] In response to a data recovery request, a data snapshot of the corresponding database node is downloaded from the backup storage node, and a backup file is obtained by screening the data snapshot; wherein the backup file is stored in the backup storage node by the data backup method provided by the first aspect above.

[0018] In another possible implementation, the method further includes:

[0019] Write the backup file to disk as a target file;

[0020] Import the target file from the disk into the storage engine of the database node through the preset interface.

[0021] In another possible implementation, the above-mentioned screening from the data snapshot to obtain the backup file includes:

[0022] Filter candidate data from the data snapshot based on the data range and target file name;

[0023] The backup file is selected from the candidate data according to the target timestamp of the data to be restored.

[0024] In another possible implementation, the above-mentioned screening and determining of the backup file from the candidate data according to the target timestamp of the data to be restored includes:

[0025] Determine a candidate timestamp corresponding to each candidate data;

[0026] When the first candidate data is screened out from the candidate data, the first candidate data is used as a backup file; wherein the candidate timestamp of the first candidate data is equal to the target timestamp;

[0027] When the first candidate data cannot be screened out from the candidate data, the second candidate data is screened out from the candidate data and the second candidate data is used as a backup file; wherein the candidate timestamp of the second candidate data is the maximum value among the candidate timestamps that are not later than the target timestamp.

[0028] According to another aspect of an embodiment of the present application, a data backup device is provided, the device comprising:

[0029] The backup module is used to perform the following backup operations on each sub-backup data obtained by splitting the data to be backed up:

[0030] receiving a backup request for the sub-backup data;

[0031] In response to a backup request, a data snapshot of a database node corresponding to the sub-backup data is obtained; wherein each sub-backup data is distributedly stored in a plurality of database nodes, and each sub-backup data corresponds to a database node;

[0032] Based on the data range of the sub-backup data, the files to be backed up are obtained by filtering from the data snapshot;

[0033] Upload the files to be backed up to the backup storage node.

[0034] According to another aspect of an embodiment of the present application, a data recovery device is provided, the device comprising:

[0035] An acquisition module, used to acquire a data recovery request;

[0036] The recovery module is used to download the data snapshot of the corresponding database node from the backup storage node in response to the data recovery request, and filter the backup file from the data snapshot; wherein the backup file is stored in the backup storage node through the data backup device provided by the third aspect above.

[0037] According to another aspect of an embodiment of the present application, an electronic device is provided, which includes: a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps of the method shown in the first aspect of the embodiment of the present application.

[0038] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method shown in the first aspect of the embodiments of the present application are implemented.

[0039] According to one aspect of an embodiment of the present application, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, the steps of the method shown in the first aspect of the embodiment of the present application are implemented.

[0040] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0041] The embodiment of the present application receives the backup request sent by the backup database through the database node corresponding to the sub-backup data. The database node responds to the backup request, obtains the data snapshot of the database node, and based on the data range of the sub-backup data, filters the files to be backed up from the data snapshot to upload the files to be backed up to the backup storage node. The embodiment of the present application directly obtains the data snapshot of the database node, and filters the files to be backed up from the data snapshot to complete the files to be backed up related to the sub-backup data in the backup data snapshot; that is, in the embodiment of the present application, the physical files involved in the data snapshot are directly used as the files to be backed up to complete the data backup, which is different from the prior art of performing logical scanning and analysis on the data snapshot to back up the scanned data; the present application avoids the large amount of IO and CPU resource consumption caused by the logical scanning operation, effectively avoids the increase in the request delay of the online business and the decrease in the throughput during reading and writing, and solves the resource conflict problem between data backup and online business.

[0042] On the other hand, the embodiment of the present application obtains the data recovery request sent by the recovery client through the database node, responds to the data recovery request, downloads the data snapshot of the corresponding database node from the backup storage node, and screens the backup file from the data snapshot to realize the recovery of the backup file. In the embodiment of the present application, multiple versions of data snapshots (i.e., each version corresponds to a timestamp) can be stored in the backup storage node. Different from the prior art that can only perform data backup and recovery for one version of the data snapshot, the embodiment of the present application realizes data recovery for backup files of different versions, and effectively improves the adaptability of data recovery while maintaining the consistency of file data in the same version of the data snapshot. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0044] Figure 1 A schematic diagram of an application scenario of a data backup method provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of a data backup method provided in an embodiment of the present application;

[0046] Figure 3 A flowchart of a data recovery method provided in an embodiment of the present application;

[0047] Figure 4 A timing diagram of an exemplary data backup method provided in an embodiment of the present application;

[0048] Figure 5 A timing diagram of an exemplary data recovery method provided in an embodiment of the present application;

[0049] Figure 6 A schematic diagram of the structure of a data backup device provided in an embodiment of the present application;

[0050] Figure 7 A schematic diagram of the structure of a data recovery device provided in an embodiment of the present application;

[0051] Figure 8 A schematic diagram of the structure of a data backup electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0053] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B".

[0054] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0055] In a MVCC (Multiversion Concurrency Control) distributed database that is based on a consensus protocol and supports transactions, data backup and data recovery are essential functions.

[0056] Distributed databases usually shard data and distribute the data shards to multiple database nodes. Each data shard (i.e., sub-backup data) has multiple copies (sub-backup data), and the consistency of external reading and writing between multiple copies can be guaranteed by the Paxos (distributed consistency algorithm) or Raft (distributed consistency protocol) consensus protocol. In general, the consensus protocol cannot guarantee that the data on each shard will remain the same at any time. Therefore, existing data backup solutions require a data snapshot that meets consistency, and scan and back up logical data on the data snapshot.

[0057] The inventors have found that the existing data backup and data recovery methods have the following problems due to the need to scan the logical data in the data snapshot:

[0058] Backing up data has a significant impact on online business, causing increased latency in business requests and decreased data read and write throughput. At the same time, since only one version of the data snapshot can be backed up, when restoring data, the data can only be restored to the data version recorded at the time of backup.

[0059] The data backup method, data recovery method and device provided in this application are intended to solve the above technical problems in the prior art.

[0060] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0061] like Figure 1 As shown, the data backup method of the present application can be applied to Figure 1In the scenario shown, specifically, the data backup method can be applied to multiple database nodes in a distributed database; the data to be backed up can be divided into multiple sub-backup data in advance, and multiple secondary backup data are created for each sub-backup data, and the above-mentioned sub-backup data and secondary backup data are distributed and stored in each database node; the backup client 101 sends a backup request for the sub-backup data to the database node 102 corresponding to the sub-backup data, and the database node 102 responds to the backup request and obtains a data snapshot of this database node; then, the database node 102 filters the files to be backed up from the data snapshot based on the data range of the sub-backup data, and uploads the files to be backed up to the backup storage node 103.

[0062] Figure 1 In the scenario shown, the above data backup method can be performed in a server, and in other scenarios, it can also be performed in a terminal.

[0063] Technicians in this technical field can understand that the "terminal" used here can be a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a MID (Mobile Internet Device), etc.; the "server" can be implemented by an independent server or a server cluster consisting of multiple servers.

[0064] The present application provides a data backup method, such as Figure 2 As shown, it can be applied to multiple database nodes in a distributed database, and the method includes:

[0065] For each sub-backup data obtained by splitting the data to be backed up, perform the following backup operations:

[0066] S201, receiving a backup request for sub-backup data.

[0067] The backup request may be sent from the backup client to the database node.

[0068] Specifically, the backup client may determine a backup request for the sub-backup data based on the user's input operation, and send the backup request to the database node corresponding to the sub-backup data based on the wireless network.

[0069] S202: In response to a backup request, obtain a data snapshot of a database node corresponding to the sub-backup data.

[0070] Among them, each sub-backup data is distributed and stored in multiple database nodes, and each sub-backup data corresponds to a database node; each database node in the distributed database stores at least one of the sub-backup data or the secondary backup data; each sub-backup data has multiple secondary backup data. Taking RocksDB (an embeddable key-value storage system that supports persistence) as an example, the data snapshot of the database node can be obtained based on Snapshot (snapshot technology). A Snapshot captures a consistent view of the database at the creation timestamp.

[0071] In some embodiments, when the data range in the distributed database is [-∞, +∞], the data can be divided into a limited number of sub-backup data, each sub-backup data corresponding to a data range; for example, the data range corresponding to each sub-backup data may include [-∞, bbb), [bbb, ddd), [ddd, +∞]. The above sub-backup data and secondary backup data are distributed and stored in each database node.

[0072] S203: Based on the data range of the sub-backup data, filter the files to be backed up from the data snapshot, and upload the files to be backed up to the backup storage node.

[0073] In the embodiment of the present application, since a data snapshot involves the data content of the entire storage engine on the corresponding database node, it may include multiple files, and the multiple files may contain data that exceeds the data range of the sub-backup data, it is necessary to filter from the data snapshot based on the data range. For example, the data range of the current sub-backup data is [bbb, ddd), and the data snapshot includes the following three files:

[0074] File 01, the data range in the file is [aaa, ccc];

[0075] File 02, the data range in the file is [bbb, ccc];

[0076] File 03, the data range in the file is [ddd, eee].

[0077] The above data range includes part of the data in file 01, and at the same time, the data range includes all the data in file 02, while the data in file 03 does not belong to the above data range; therefore, file 01 and file 02 can be screened as files to be backed up.

[0078] The embodiment of the present application receives the backup request sent by the backup database through the database node corresponding to the sub-backup data. The database node responds to the backup request, obtains the data snapshot of the database node, and based on the data range of the sub-backup data, filters the files to be backed up from the data snapshot to upload the files to be backed up to the backup storage node. The embodiment of the present application directly obtains the data snapshot of the database node, and filters the files to be backed up from the data snapshot to complete the files to be backed up related to the sub-backup data in the backup data snapshot; that is, in the embodiment of the present application, the physical files involved in the data snapshot are directly used as the files to be backed up to complete the data backup, which is different from the prior art of performing logical scanning and analysis on the data snapshot to back up the scanned data; the present application avoids the large amount of IO and CPU resource consumption caused by the logical scanning operation, effectively avoids the increase in the request delay of the online business and the decrease in the throughput during reading and writing, and solves the resource conflict problem between data backup and online business.

[0079] A possible implementation method is provided in an embodiment of the present application. After uploading the files to be backed up to the backup storage node, the method further includes:

[0080] The timestamp, data range and file name of the data snapshot are sent to the backup client, so that the backup client saves the timestamp, data range and file name into a meta information file, and uploads the meta information file to the backup storage node.

[0081] The uploading of the meta information file to the backup storage node occurs after the backup operation is completed for all the sub-backup data.

[0082] The embodiment of the present application stores the timestamp of the data snapshot, the data range of the sub-backup data, and the file name of the file to be backed up, and uploads them to the backup storage node, thereby realizing the joint backup of the file to be backed up and the corresponding attribute data, laying a good foundation for the subsequent accurate data recovery of the backup data.

[0083] A possible implementation method is provided in an embodiment of the present application. Before obtaining the data snapshot of the database node, the method includes:

[0084] Writes the cached data in the distributed database's memory to disk.

[0085] The cached data includes database read and write data initiated before the timestamp of the data snapshot and to be processed; the data snapshot includes the cached data.

[0086] Before obtaining a data snapshot, the embodiment of the present application needs to write cached data in the memory of the distributed database that has not been written to the disk to the disk. Since the cached data is the database read and write data to be processed initiated before the timestamp of the data snapshot, the data consistency between the data snapshot and the disk can be guaranteed at the current timestamp, thereby effectively ensuring the accuracy of the backup data.

[0087] The present application provides a data recovery method, such as Figure 3 As shown, it can be applied to multiple database nodes in a distributed database, and the method includes:

[0088] S301, obtaining a data recovery request.

[0089] The data recovery request may be sent by the recovery client to the database node.

[0090] Specifically, the recovery client can determine the target file name of the data to be recovered and the target timestamp corresponding to the data to be recovered based on the user's input operation; then, the recovery client downloads and reads the meta information file from the backup storage node, and queries the meta information file based on the target file name to obtain the data range corresponding to the target file name and the database node corresponding to the data range. Then, the recovery client uses the target file name, target timestamp and data range as a data recovery request, and sends the data recovery request to the corresponding database node, so that the database node obtains the data recovery request.

[0091] S302 , in response to a data recovery request, downloading a data snapshot of a corresponding database node from a backup storage node, and filtering the data snapshot to obtain a backup file.

[0092] The backup file is stored in the backup storage node through the data backup method provided in the first aspect above.

[0093] Specifically, the database node downloads the corresponding data snapshot from the backup storage node in response to the data recovery request, and can filter the backup file from the data snapshot based on the target file name and data range in the data recovery request.

[0094] In some implementations, the database node can download and restore the corresponding data snapshot according to the storage engine's requirements for file organization. Taking RocksDB as an example, the steps for obtaining the data snapshot are as follows:

[0095] Start a temporary RocksDB instance, and then import the file data into RocksDB through the IngestExternalFile interface according to the layer number of the file data at the time of backup, from Level 6 to Level 0; after all the file data is imported, the data snapshot at the time of backup can be obtained based on the RocksDB snapshot.

[0096] The detailed steps for filtering backup files will be introduced in detail below.

[0097] The embodiment of the present application obtains the data recovery request sent by the recovery client through the database node, responds to the data recovery request, downloads the data snapshot of the corresponding database node from the backup storage node, and screens the backup file from the data snapshot to realize the recovery of the backup file. In the embodiment of the present application, multiple versions of data snapshots (i.e., each version corresponds to a timestamp) can be stored in the backup storage node. Different from the prior art that can only perform data backup and recovery for one version of the data snapshot, the embodiment of the present application realizes data recovery for backup files of different versions, and effectively improves the adaptability of data recovery under the premise of maintaining the consistency of file data in the same version of the data snapshot.

[0098] A possible implementation method is provided in an embodiment of the present application, and the method further includes:

[0099] S401, writing the backup file into the disk as the target file;

[0100] S402, importing the target file from the disk into the storage engine of the database node through a preset interface.

[0101] Specifically, taking RocksDB as an example, the target file can be imported from the disk into the storage engine of the database node through the IngestExternalFile interface.

[0102] In an embodiment of the present application, a possible implementation method is provided, wherein the above-mentioned screening of the backup files from the data snapshot includes:

[0103] Filter candidate data from the data snapshot based on the data range and target file name;

[0104] The backup file is selected from the candidate data according to the target timestamp of the data to be restored.

[0105] In the embodiment of the present application, the data snapshot can be filtered according to the data range and the target file name, and the candidate data of the data snapshot within the data range is retained, and then the backup file is filtered from the candidate data based on the specified target timestamp to achieve the recovery of the backup data. Specifically, the process of data filtering according to the target timestamp will be described in detail below.

[0106] In an embodiment of the present application, a possible implementation method is provided, wherein the above-mentioned screening and determining of the backup file from the candidate data according to the target timestamp of the data to be restored includes:

[0107] Determine a candidate timestamp corresponding to each candidate data;

[0108] When the first candidate data is screened out from the candidate data, the first candidate data is used as a backup file; wherein the candidate timestamp of the first candidate data is equal to the target timestamp;

[0109] When the first candidate data cannot be screened out from the candidate data, the second candidate data is screened out from the candidate data and the second candidate data is used as a backup file; wherein the candidate timestamp of the second candidate data is the maximum value among the candidate timestamps that are not later than the target timestamp.

[0110] In some implementations, there are multiple records in a table in the database, and each record has multiple versions. Each version has two timestamps, namely, the start timestamp and the commit timestamp of the two-stage transaction submission. In the embodiment of the present application, the commit timestamp is used as the candidate timestamp of the candidate data. For each candidate data, data screening is performed based on the commit timestamp of the candidate data, and the data of the largest version with a commit timestamp less than or equal to the target timestamp can be retained as a backup file.

[0111] In order to better understand the above data backup method and data recovery method, Figure 4 and Figure 5 An example of a data backup method and a data recovery method of the present application is described in detail, and the method comprises the following steps:

[0112] S501, the backup client obtains the latest version number of the data snapshot in the distributed database, that is, the latest timestamp, and checks each sub-backup data stored in the distributed database based on the latest timestamp, so that all transactions with a starttimestamp earlier than the latest timestamp are committed or rolled back.

[0113] Each sub-backup data is obtained by segmenting the data to be backed up, and the multiple sub-backup data are distributedly stored in each database node.

[0114] S502, the backup client determines the database node corresponding to each sub-backup data, and sends a backup request to each database node, so that each database node performs the following backup operation:

[0115] Receive a backup request for the sub-backup data sent by the backup client, and in response to the backup request, obtain a data snapshot of the database node. Based on the data range of the sub-backup data, filter the files to be backed up from the data snapshot, and upload the files to be backed up to the backup storage node.

[0116] After each database node completes uploading the backup file, the backup client obtains the timestamp, data range and file name of the to-be-backed-up file of the data snapshot of the database node, and saves the timestamp, data range and file name into the meta-information file.

[0117] S503: After completing the backup operation for all sub-backup data, the backup client uploads the meta information file to the backup storage node.

[0118] S504, the recovery client downloads and reads the meta information file from the backup storage node; and parses the target file name and target timestamp to be restored from the meta information file.

[0119] S505: The recovery client sends a data recovery request to each database node based on the target file name and the target timestamp.

[0120] S506: The database node downloads the data snapshot of the corresponding database node from the backup storage node in response to the data recovery request.

[0121] S507: The database node filters candidate data from the data snapshot based on the data range and the target file name.

[0122] S508, the database node determines a candidate timestamp corresponding to each candidate data;

[0123] When the first candidate data is screened out from the candidate data, the first candidate data is used as a backup file; wherein the candidate timestamp of the first candidate data is equal to the target timestamp;

[0124] When the first candidate data cannot be screened out from the candidate data, the second candidate data is screened out from the candidate data and the second candidate data is used as a backup file; wherein the candidate timestamp of the second candidate data is the maximum value among the candidate timestamps that are not later than the target timestamp.

[0125] S509, writing the backup file as the target file to the disk; and importing the target file from the disk to the storage engine of the database node through a preset interface.

[0126] S510, determining whether there are any backup files that have not yet completed data recovery. If there are any backup files to be recovered, repeating the above S605 to S609 until all backup files corresponding to the sub-backup data have completed data recovery.

[0127] The embodiment of the present application receives the backup request sent by the backup database through the database node corresponding to the sub-backup data. The database node responds to the backup request, obtains the data snapshot of the database node, and based on the data range of the sub-backup data, filters the files to be backed up from the data snapshot to upload the files to be backed up to the backup storage node. The embodiment of the present application directly obtains the data snapshot of the database node, and filters the files to be backed up from the data snapshot to complete the files to be backed up related to the sub-backup data in the backup data snapshot; that is, in the embodiment of the present application, the physical files involved in the data snapshot are directly used as the files to be backed up to complete the data backup, which is different from the prior art of performing logical scanning and analysis on the data snapshot to back up the scanned data; the present application avoids the large amount of IO and CPU resource consumption caused by the logical scanning operation, effectively avoids the increase in the request delay of the online business and the decrease in the throughput during reading and writing, and solves the resource conflict problem between data backup and online business.

[0128] On the other hand, the embodiment of the present application obtains the data recovery request sent by the recovery client through the database node, responds to the data recovery request, downloads the data snapshot of the corresponding database node from the backup storage node, and screens the backup file from the data snapshot to realize the recovery of the backup file. In the embodiment of the present application, multiple versions of data snapshots (i.e., each version corresponds to a timestamp) can be stored in the backup storage node. Different from the prior art that can only perform data backup and recovery for one version of the data snapshot, the embodiment of the present application realizes data recovery for backup files of different versions, and effectively improves the adaptability of data recovery while maintaining the consistency of file data in the same version of the data snapshot.

[0129] The present application embodiment provides a data backup device, such as Figure 6 As shown, the data backup device 60 may include: a backup module 601;

[0130] The backup module 601 is used to perform the following backup operations on each sub-backup data obtained by splitting the data to be backed up:

[0131] receiving a backup request for the sub-backup data;

[0132] In response to a backup request, a data snapshot of a database node corresponding to the sub-backup data is obtained; wherein each sub-backup data is distributedly stored in a plurality of database nodes, and each sub-backup data corresponds to a database node;

[0133] Based on the data range of the sub-backup data, the files to be backed up are obtained by filtering from the data snapshot;

[0134] Upload the files to be backed up to the backup storage node.

[0135] A possible implementation method is provided in an embodiment of the present application. After uploading the files to be backed up to the backup storage node, the backup module 601 is further used to:

[0136] The timestamp, data range and file name of the data snapshot are sent to the backup client, so that the backup client saves the timestamp, data range and file name into the meta information file, and uploads the meta information file to the backup storage node; wherein, uploading the meta information file to the backup storage node occurs after the backup operation is completed for all sub-backup data.

[0137] In an embodiment of the present application, a possible implementation is provided. Before obtaining the data snapshot of the database node, the backup module 601 is further used to:

[0138] The cache data in the memory of the distributed database is written to the disk; wherein the cache data includes database read and write data to be processed that is initiated before the timestamp of the data snapshot; and the data snapshot includes the cache data.

[0139] The present application embodiment provides a data recovery device, such as Figure 7 As shown, the data recovery device 70 may include: an acquisition module 701 and a recovery module 702;

[0140] Wherein, the acquisition module 701 is used to obtain a data recovery request;

[0141] The recovery module 702 is used to download the data snapshot of the corresponding database node from the backup storage node in response to the data recovery request, and filter the backup file from the data snapshot; wherein the backup file is stored in the backup storage node through the data backup device provided by the third aspect above.

[0142] A possible implementation method is provided in an embodiment of the present application. The above recovery module 702 is further used to:

[0143] Write the backup file to disk as a target file;

[0144] Import the target file from the disk into the storage engine of the database node through the preset interface.

[0145] In an embodiment of the present application, a possible implementation is provided. When the recovery module 702 obtains a backup file by filtering from a data snapshot, it is used to:

[0146] Filter candidate data from the data snapshot based on the data range and target file name;

[0147] The backup file is selected from the candidate data according to the target timestamp of the data to be restored.

[0148] A possible implementation method is provided in an embodiment of the present application. When the recovery module 702 screens and determines the backup file from the candidate data according to the target timestamp of the data to be recovered, it is used to:

[0149] Determine a candidate timestamp corresponding to each candidate data;

[0150] When the first candidate data is screened out from the candidate data, the first candidate data is used as a backup file; wherein the candidate timestamp of the first candidate data is equal to the target timestamp;

[0151] When the first candidate data cannot be screened out from the candidate data, the second candidate data is screened out from the candidate data and the second candidate data is used as a backup file; wherein the candidate timestamp of the second candidate data is the maximum value among the candidate timestamps that are not later than the target timestamp.

[0152] The device of the embodiments of the present application can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0153] The embodiment of the present application receives the backup request sent by the backup database through the database node corresponding to the sub-backup data. The database node responds to the backup request, obtains the data snapshot of the database node, and based on the data range of the sub-backup data, filters the files to be backed up from the data snapshot to upload the files to be backed up to the backup storage node. The embodiment of the present application directly obtains the data snapshot of the database node, and filters the files to be backed up from the data snapshot to complete the files to be backed up related to the sub-backup data in the backup data snapshot; that is, in the embodiment of the present application, the physical files involved in the data snapshot are directly used as the files to be backed up to complete the data backup, which is different from the prior art of performing logical scanning and analysis on the data snapshot to back up the scanned data; the present application avoids the large amount of IO and CPU resource consumption caused by the logical scanning operation, effectively avoids the increase in the request delay of the online business and the decrease in the throughput during reading and writing, and solves the resource conflict problem between data backup and online business.

[0154] On the other hand, the embodiment of the present application obtains the data recovery request sent by the recovery client through the database node, responds to the data recovery request, downloads the data snapshot of the corresponding database node from the backup storage node, and screens the backup file from the data snapshot to realize the recovery of the backup file. In the embodiment of the present application, multiple versions of data snapshots (i.e., each version corresponds to a timestamp) can be stored in the backup storage node. Different from the prior art that can only perform data backup and recovery for one version of the data snapshot, the embodiment of the present application realizes data recovery for backup files of different versions, and effectively improves the adaptability of data recovery while maintaining the consistency of file data in the same version of the data snapshot.

[0155] In an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, the processor executes the above computer program to implement the steps of a data backup method and a data recovery method, and compared with the related art, it can be realized that: in an embodiment of the present application, a database node corresponding to the sub-backup data receives a backup request sent by a backup database, and the database node responds to the backup request, obtains a data snapshot of the database node, and based on the data range of the sub-backup data, filters out the files to be backed up from the data snapshot to upload the files to be backed up to the backup storage node. In an embodiment of the present application, a data snapshot of a database node is directly obtained, and the files to be backed up are filtered out from the data snapshot to complete the files to be backed up related to the sub-backup data in the backup data snapshot; that is, in an embodiment of the present application, the physical files involved in the data snapshot are directly used as the files to be backed up to complete the data backup, which is different from the prior art of performing logical scanning and analysis on the data snapshot to back up the scanned data; the present application avoids a large amount of IO and CPU resource consumption caused by the logical scanning operation, effectively avoids the increase in the request delay of the online business and the decrease in the throughput during reading and writing, and solves the resource conflict problem between data backup and online business.

[0156] On the other hand, the embodiment of the present application obtains the data recovery request sent by the recovery client through the database node, responds to the data recovery request, downloads the data snapshot of the corresponding database node from the backup storage node, and screens the backup file from the data snapshot to realize the recovery of the backup file. In the embodiment of the present application, multiple versions of data snapshots (i.e., each version corresponds to a timestamp) can be stored in the backup storage node. Different from the prior art that can only perform data backup and recovery for one version of the data snapshot, the embodiment of the present application realizes data recovery for backup files of different versions, and effectively improves the adaptability of data recovery while maintaining the consistency of file data in the same version of the data snapshot.

[0157] In an alternative embodiment, an electronic device is provided, such as Figure 8 As shown, Figure 8 The electronic device 80 shown includes: a processor 801 and a memory 803. The processor 801 and the memory 803 are connected, such as through a bus 802. Optionally, the electronic device 80 may also include a transceiver 804, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 804 is not limited to one, and the structure of the electronic device 80 does not constitute a limitation on the embodiments of the present application.

[0158] The processor 801 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 801 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0159] The bus 802 may include a path to transmit information between the above components. The bus 802 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 802 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0160] The memory 803 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0161] The memory 803 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 801. The processor 801 is used to execute the computer program stored in the memory 803 to implement the steps shown in the above method embodiment.

[0162] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, notebook computers, PADs, etc. and fixed terminals such as digital TVs, desktop computers, etc.

[0163] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0164] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that when the computer device executes the computer instructions, the following conditions are achieved:

[0165] For each sub-backup data obtained by splitting the data to be backed up, perform the following backup operations:

[0166] receiving a backup request for the sub-backup data;

[0167] In response to a backup request, a data snapshot of a database node corresponding to the sub-backup data is obtained; wherein each sub-backup data is distributedly stored in a plurality of database nodes, and each sub-backup data corresponds to a database node;

[0168] Based on the data range of the sub-backup data, the files to be backed up are obtained by filtering from the data snapshot;

[0169] Upload the files to be backed up to the backup storage node.

[0170] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0171] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0172] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A data backup method, applied to multiple database nodes in a distributed database, It is characterized in that include: For each sub-backup data obtained by splitting the data to be backed up, perform the following backup operations: receiving a backup request for the sub-backup data; In response to the backup request, a data snapshot of the database node corresponding to the sub-backup data is obtained; wherein each sub-backup data is distributedly stored in a plurality of database nodes, and each sub-backup data corresponds to a database node; the data snapshot includes all data content on the corresponding database node, and the data snapshot includes a plurality of files, and each file is provided with a corresponding file data range; Based on the data range of the sub-backup data, filtering the files to be backed up from the data snapshot; Uploading the files to be backed up to the backup storage node; The step of selecting the to-be-backed-up file from the data snapshot based on the data range of the sub-backup data specifically includes: For each file included in the data snapshot, if the file data range of the file overlaps with the data range of the sub-backup data, the file is used as the file to be backed up.

2. The method according to claim 1, It is characterized in that After uploading the file to be backed up to the backup storage node, the method further includes: The timestamp, data range and file name of the data snapshot are sent to the backup client, so that the backup client saves the timestamp, data range and file name into a meta-information file, and uploads the meta-information file to the backup storage node; wherein, uploading the meta-information file to the backup storage node occurs after the backup operation is completed for all sub-backup data.

3. The method according to claim 1, It is characterized in that Before obtaining the data snapshot of the database node, the method includes: The cache data in the memory of the distributed database is written to the disk; wherein the cache data includes database read and write data to be processed that is initiated before the timestamp of the data snapshot; and the data snapshot includes the cache data.

4. A data recovery method, applied to multiple database nodes in a distributed database, It is characterized in that The method further comprises: Get data recovery request; In response to the data recovery request, a data snapshot of the corresponding database node is downloaded from the backup storage node, and a backup file is obtained by screening from the data snapshot; wherein the backup file is stored in the backup storage node by the method described in any one of claims 1-3.

5. The method according to claim 4, It is characterized in that The method further comprises: Writing the backup file to disk as a target file; The target file is imported from the disk into the storage engine of the database node through a preset interface.

6. The method according to claim 4, It is characterized in that The step of screening the backup files from the data snapshot includes: Filtering candidate data from the data snapshot based on the data range and the target file name; A backup file is determined by screening from the candidate data according to the target timestamp of the data to be restored.

7. The method according to claim 6, wherein, the screening and determination of the backup file from the candidate data according to the target timestamp of the data to be restored includes: determining a candidate timestamp corresponding to each candidate data; when a first candidate data is screened out from the candidate data, using the first candidate data as the backup file; wherein, the candidate timestamp of the first candidate data is equal to the target timestamp; when the first candidate data cannot be screened out from the candidate data, screening out a second candidate data from the candidate data and using the second candidate data as the backup file; wherein, the candidate timestamp of the second candidate data is the maximum value among the candidate timestamps not later than the target timestamp.

8. A data backup device, wherein, it includes: a backup module, configured to perform the following backup operations for each sub-backup data obtained by splitting the data to be backed up: receiving a backup request for the sub-backup data; in response to the backup request, obtaining a data snapshot of the database node corresponding to the sub-backup data; wherein, the sub-backup data are distributed and stored in multiple database nodes, and each sub-backup data corresponds to a database node; the data snapshot contains all the data contents on the corresponding database node, the data snapshot contains multiple files, and each file is set with a corresponding file data range; screening out the file to be backed up from the data snapshot based on the data range of the sub-backup data; uploading the file to be backed up to the backup storage node; specifically, the backup module is configured to: for each file included in the data snapshot, if there is an intersection between the file data range of the file and the data range of the sub-backup data, using the file as the file to be backed up.

9. A data recovery device, wherein, it includes: an acquisition module, configured to acquire a data recovery request; a recovery module, configured to, in response to the data recovery request, download the data snapshot of the corresponding database node from the backup storage node and screen out the backup file from the data snapshot; wherein, the backup file is stored in the backup storage node by the data backup device according to claim 8.

10. An electronic device, including a memory, a processor, and a computer program stored on the memory, wherein, the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images