Data backup method and electronic equipment

Abstract

The invention provides a data backup method and electronic device.The method comprises the steps that a snapshot of a volume mounted on a virtual machine is obtained, and the volume comprises at least one data block; determining bitmap information according to the snapshot; acquiring data of a first data block from a storage device according to the bitmap information, and storing the data of the first data block in a cache; wherein the first data block is a data block indicated by a first identifier in the bitmap information, and the first identifier is used for indicating that data stored in the at least one data block has updated data blocks; and under the condition that a data acquisition request of the first data block is received, sending the data of the first data block.

Description

Technical Field

[0001] This application relates to the technical field of distributed storage. Specifically, it relates to a data backup method and an electronic device. Background Art

[0002] Ceph is a unified distributed storage system that can provide good performance, reliability, and scalability. The core concept of Ceph is to achieve high availability and high performance of the storage system through a decentralized architecture, store data dispersedly on multiple nodes, and ensure data consistency and reliability through intelligent algorithms. Ceph supports three storage interfaces: object storage, block storage, and file system storage.

[0003] In Ceph, data blocks can record changes (such as modifications, additions, or deletions) of data blocks based on CBT (Changing Block Tracing), so as to back up the blocks that have changed since the last backup. However, the process of related backup technologies is relatively long, resulting in a large delay and low backup performance. Summary of the Invention

[0004] At least one data backup method and an electronic device are provided in the embodiments of this application to shorten the delay of data backup, thereby improving the performance of data backup.

[0005] In a first aspect, an embodiment of this application provides a data backup method, including:

[0006] Obtain a snapshot of a volume mounted by a virtual machine, where the volume includes at least one data block;

[0007] Determine bitmap information according to the snapshot;

[0008] Obtain the data of the first data block from a storage device according to the bitmap information and store the data of the first data block in a cache; where the first data block is the data block indicated by a first identifier in the bitmap information, and the first identifier is used to indicate the data block in which the stored data has been updated among the at least one data block;

[0009] Send the data of the first data block when a data acquisition request for the first data block is received.

[0010] In the above implementation, obtaining the data of the first data block to be backed up at the current moment according to the bitmap information and caching the data can achieve pre-reading of the data of the data block to be backed up. Compared with the method of directly searching for the data of the first data block from the storage device, in this application, after receiving a data acquisition request, by searching for the data of the first data block in the cache, the delay of data backup can be shortened, thereby improving the performance of data backup.

[0011] In an alternative embodiment, obtaining the data of the first data block from the storage device according to the bitmap information includes:

[0012] Determining the starting position and size of the first data block according to the bitmap information;

[0013] Obtaining the data of the first data block in the storage device according to the starting position and size.

[0014] In the above embodiment, since the backup device normally reads the data of the data blocks to be backed up in the order of the differential bitmap, therefore, by the way of pre-reading the data of the first data block according to the starting position and size indicated by the bitmap information, the corresponding data can be obtained more accurately and efficiently, thereby achieving a significant improvement in efficiency with lower memory resources.

[0015] In an alternative embodiment, sending the data of the first data block in the case of receiving a data acquisition request for the first data block includes:

[0016] Obtaining the data acquisition request;

[0017] Determining the starting position and size of the requested data block according to the data acquisition request;

[0018] If the starting position and size of the requested data block are the same as those of the first data block in the cache, searching for the first data block in the cache;

[0019] Sending the data of the first data block.

[0020] In the above embodiment, by matching the starting position and size of the data block, the accuracy of the pre-read data in the cache can be ensured, thereby improving the performance of data backup while ensuring the accuracy of data backup.

[0021] In an alternative embodiment, the method further includes:

[0022] If the starting position and size of the requested data block are different from those of the first data block in the cache, reading a second data block in the storage device and sending the data of the second data block.

[0023] In the above embodiment, if the data of the first data block is not found in the cache, by reading the data of the first data block in the storage device, the normal execution of the data backup process can be ensured.

[0024] In an alternative embodiment, after sending the data of the first data block, the method further includes:

[0025] Obtaining the data of the third data block to be backed up from the storage device based on the bitmap information; wherein, the third data block is one or more data blocks indicated by the first identifier after the first identifier corresponding to the first data block in the bitmap information;

[0026] Storing the data of the third data block in the cache.

[0027] In the above embodiment, the way that the working node pre-reads the data of the next one or more data blocks in advance according to the currently accessed data block and in combination with the bitmap information can shorten the latency of data backup and improve the performance of data backup.

[0028] In an alternative embodiment, determining the second data block to be backed up based on the bitmap order of the bitmap information includes:

[0029] Obtaining the remaining resource information of the memory resource;

[0030] Determining the number of data blocks to be backed up based on the remaining resource information;

[0031] Determining the number of data blocks as the third data block from the data blocks indicated by the first identifier after the first identifier corresponding to the first data block based on the bitmap information, and obtaining the data of each third data block from the storage device.

[0032] In the above embodiment, by determining the number of backups through the remaining resource information, and then determining the third data block according to the bitmap order and the number of backups, the adaptive adjustment of pre-reading data can be realized, so that the data backup efficiency can be further improved and the data backup latency can be shortened while meeting the resource requirements.

[0033] In an alternative embodiment, obtaining the data of the third data block to be backed up from the storage device based on the bitmap information includes:

[0034] In the case where the number of the third data blocks is multiple, reading the data of each third data block in parallel in the storage device.

[0035] In the above embodiment, by reading the data of multiple third data blocks in parallel, the efficiency of data pre-reading can be improved and the latency of data backup can be shortened.

[0036] In an alternative embodiment, storing the data of the third data block in the cache includes:

[0037] After parallelly reading the data of each of the third data blocks from the storage device based on the bitmap information, store the data of the multiple third data blocks in the cache in the backup order of the third data blocks.

[0038] In the above embodiment, if there are multiple third data blocks, after parallelly reading the data of the multiple third data blocks from the storage device, the data of each third data block can also be stored in the cache in sequence according to the backup order. When sequentially searching for the data of each third data block in the cache, the hit rate of data search can be improved.

[0039] In an alternative embodiment, the first data block is the first data block to be backed up. Obtaining the data of the first data block from the storage device according to the bitmap information and storing the data of the first data block in the cache includes:

[0040] If an interface call request sent by the backup device is detected, read the data of the first data block in the storage device and store the data of the first data block in the cache.

[0041] In the above embodiment, by pre-reading the data of the data block, the process of data backup can be simplified, thereby shortening the latency of data backup and improving the performance of data backup.

[0042] In a second aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps in the first aspect, or any possible implementation manner in the first aspect are executed.

[0043] The present application provides a data backup method and an electronic device. In an embodiment of the present application, first, obtain a snapshot of a volume mounted by a virtual machine, where the volume includes at least one data block; then, determine bitmap information according to the snapshot; and obtain the data of the first data block from the storage device according to the bitmap information and store the data of the first data block in the cache; where the first data block is the data block indicated by a first identifier in the bitmap information, and the first identifier is used to indicate a data block in which the stored data has been updated among the at least one data block; and when a data acquisition request for the first data block is received, send the data of the first data block.

[0044] In the above embodiments, the data of the first data block to be backed up at the current moment is obtained according to the bitmap information, and the data is cached, so that the data pre-reading of the data block to be backed up can be realized. Compared with the method of directly searching for the data of the first data block from the storage device, after receiving the data acquisition request, the present application can shorten the data backup delay by searching for the data of the first data block in the cache, thereby improving the performance of data backup.

[0045] In order to make the above objects, features and advantages of the present application more obvious and understandable, the following specifically gives preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. The drawings herein are incorporated into the specification and constitute a part of this specification. These drawings show embodiments that conform to the present application and, together with the specification, are used to illustrate the technical solutions of the present application. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1 Shows a flowchart of a data backup method provided by an embodiment of the present application;

[0048] Figure 2 Shows a schematic diagram of the interaction process between a working node and a backup device provided by an embodiment of the present application;

[0049] Figure 3 Shows a schematic diagram of the backup process of a backup device provided by an embodiment of the present application;

[0050] Figure 4 Shows a schematic diagram of the interaction process of a data backup method provided by an embodiment of the present application;

[0051] Figure 5 Shows a schematic diagram of a data backup device provided by an embodiment of the present application;

[0052] Figure 6 Shows a schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only some, rather than all, of the embodiments of this application. Components of the embodiments of this application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative efforts fall within the scope of protection of this application.

[0054] It should be noted that like reference numerals and letters denote like items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0055] The term "and / or" in this document merely describes an associated relationship and indicates that three relationships may exist. For example, A and / or B may represent three cases: A exists alone, both A and B exist simultaneously, and B exists alone. In addition, the term "at least one" in this document means any one of multiple types or any combination of at least two of multiple types. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set composed of A, B, and C.

[0056] Before specific description, the terms related to the embodiments of this application are explained as follows:

[0057] Ceph: Ceph is a high-performance, highly reliable, and scalable distributed storage system that supports block storage, object storage, and file storage.

[0058] RBD (RADOS Block Device) volume: The block storage of Ceph provides RBD volumes (accessed by the RBD protocol) that support expansion, thin provisioning, snapshots, cloning, etc. RBD volumes can be accessed through the RBD driver of the Linux kernel and the RBD storage driver of the Qemu / KVM virtual machine, and the RBD driver has been incorporated into the mainline code of Linux.

[0059] CBT backup: CBT is block change tracking technology, which is suitable for recording the changed block data after a virtual machine snapshot. During backup, only the changed data blocks are backed up, improving the backup time. During recovery, only the changed data blocks are recovered, improving the recovery time.

[0060] CBT Backup Framework: The CBT backup framework can perform CBT backups on various storage devices mounted by virtual machines, including Ceph's RBD volumes, virtualized storage VIMS, volumes accessed through protocols such as iscsi and nfs for virtualization use, and local volumes, etc.

[0061] Ceph is a unified distributed storage system that can provide good performance, reliability, and scalability. The core concept of Ceph is to achieve high availability and high performance of the storage system through a decentralized architecture, store data dispersedly on multiple nodes, and ensure data consistency and reliability through intelligent algorithms. Ceph supports three storage interfaces: object storage, block storage, and file system storage.

[0062] In Ceph, data blocks can record changes (such as modifications, additions, or deletions) of data blocks based on the CBT block change tracking technology, so as to back up the blocks that have changed since the last backup. However, the process of related backup technologies is relatively long, resulting in a large delay and low backup performance.

[0063] Based on the above research, this application provides a data backup method and an electronic device. In the embodiments of this application, first, obtain a snapshot of the volume mounted by the virtual machine, where the volume includes at least one data block; then, determine bitmap information according to the snapshot; and obtain the data of the first data block from the storage device according to the bitmap information and store the data of the first data block in the cache; where the first data block is the data block indicated by the first identifier in the bitmap information, and the first identifier is used to indicate the data block in which the stored data has been updated among the at least one data block; in the case of receiving a data acquisition request for the first data block, send the data of the first data block.

[0064] In the above implementation, obtaining the data of the first data block to be backed up at the current moment according to the bitmap information and caching the data can realize the pre-reading of the data of the data block to be backed up. Compared with the method of directly searching for the data of the first data block from the storage device, in this application, after receiving the data acquisition request, by searching for the data of the first data block in the cache, the delay of data backup can be shortened, thereby improving the performance of data backup.

[0065] To facilitate the understanding of this embodiment, first, a data backup method disclosed in the embodiments of this application will be introduced in detail. The execution subject of the data backup method provided in the embodiments of this application is generally an electronic device with a certain computing ability. In some possible implementation manners, this data backup method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0066] See Figure 1As shown in the figure, it is a flowchart of a data backup method provided by an embodiment of the present application. This method is applied to a working node in a distributed storage system. The method includes steps S101 to S104, where:

[0067] S101: Obtain a snapshot of the volume mounted by the virtual machine, where the volume includes at least one data block;

[0068] Here, snapshots are usually used for data recovery and backup. A snapshot records the data state of data blocks at a specific point in time.

[0069] S102: Determine bitmap information according to the snapshot;

[0070] Here, the bitmap information includes the following information: the identifier of each data block, the offset and length of each data block; among them, the identifier is used to indicate whether the corresponding data block is a data block with updated data. For example, if the identifier is the first identifier (for example, 1), it means that the data block is a data block with updated data; the offset and length of each data block are used to indicate the starting position and size of the data block.

[0071] S103: Obtain the data of the first data block from the storage device according to the bitmap information and store the data of the first data block in the cache; where the first data block is the data block indicated by the first identifier in the bitmap information, and the first identifier is used to indicate the data block with updated data stored in the at least one data block.

[0072] In the embodiment of the present application, when writing data to a data block in the Ceph distributed storage system, the storage location of the data can be determined through consistent hash calculation. Usually, the data is allocated to different PGs (Placement Groups), and then the PGs allocate it to specific OSDs (Object Storage Devices) for storage.

[0073] For example, a virtual machine deployed on an HCI host (Hyper Converged Infrastructure) can write data to a data block in the Ceph distributed storage system. At this time, the CBT mechanism can be used to back up each data block.

[0074] In the embodiment of the present application, before the working node receives a data acquisition request sent by the backup device, the working node can pre-read the data of the first data block from the storage device and store the data of the first data block in the cache.

[0075] S104: Send the data of the first data block when receiving the data acquisition request for the first data block.

[0076] When the backup device triggers a backup of the data block to be backed up, it can send a data acquisition request to the working node (e.g., the HCI host). After receiving the data acquisition request, the working node can look up the data of the first data block in the cache and return the data of the first data block to the backup device so that the backup device can back up the first data block.

[0077] Compared with the way the working node reads data from the storage device, the technical solution of the present application can shorten the data backup latency and improve the data backup performance by pre-reading the data of the data block.

[0078] The above steps will be introduced below in combination with specific implementation manners.

[0079] In an embodiment of the present application, when the first data block is the first data block to be backed up, the steps of obtaining the data of the first data block from the storage device according to the bitmap information and storing the data of the first data block in the cache include:

[0080] If an interface call request sent by the backup device is detected, read the data of the first data block in the storage device and store the data of the first data block in the cache.

[0081] In an embodiment of the present application, after the backup device triggers a data backup of the data block to be backed up, the backup device can create a snapshot of the volume mounted by the virtual machine in the working node by calling an interface. Snapshots are usually used for data recovery and backup, and a snapshot records the data state of the data block at a specific point in time. Then, the working node can obtain the snapshot of the volume and determine the bitmap information according to the snapshot.

[0082] Next, the backup device can call the interface to prepare the backup resources. After receiving the interface call request sent by the backup device, the working node triggers to read the data of the first data block to be backed up in the storage device and store the data in the cache.

[0083] In the above embodiment, by pre-reading the data of the data block, the data backup process can be simplified, thereby shortening the data backup latency and improving the data backup performance.

[0084] In an embodiment of the present application, the above step S103 of obtaining the data of the first data block from the storage device according to the bitmap information specifically includes the following steps:

[0085] First, determine the starting position and size of the first data block according to the bitmap information;

[0086] Second, obtain the data of the first data block in the storage device according to the starting position and size.

[0087] For the first data block to be backed up at a future moment, the working node can determine the starting position and size of the first data block in advance according to the bitmap information. Then, the data of the first data block is obtained from the storage device according to the starting position and size, and the data is stored in the cache.

[0088] Here, the working node can determine the first data block to be backed up at a future moment according to the bitmap order indicated in the bitmap information. Among them, the bitmap order is the order of the first identifier in the bitmap information.

[0089] The identifier in the bitmap information bitmap can be 10011 (the first bit from the right, and each bit corresponds to a data block). The granularity of the data block to be backed up can be 1MB (1048576). Among them, the identifier "1" in the bitmap information indicates that the status of the corresponding data block is "modified", and at this time, the identifier "1" is the first identifier. The identifier "0" in the bitmap information indicates that the status of the corresponding data block is "unmodified". That is to say, the data block with the bit "1" in the bitmap information is the data block with updated data, and the data block with the bit "0" is the data block with no updated data.

[0090] As shown in Table 1 below, the offset corresponding to the first bit "1" from the right is "0", and the length is "1048576". Among them, through the offset and length, the starting position and size of the data block in all the data blocks to be backed up can be indicated. The starting position of the data block is "0", and the size is "1048576". The offset corresponding to the second bit "1" from the right is "1048576", and the length is "1048576", indicating that the starting position of the data block is "1048576" and the size is "1048576".

[0091] Table 1

[0092]

[0093]

[0094] Here, the bitmap order of the bitmap information can be the order of each bit with the value "1" from right to left.

[0095] In the embodiments of the present application, the data block to be backed up can be determined as the first data block according to the bitmap order. Then, the offset and size of the first data block are determined based on the bitmap information. Thus, according to the offset and size, the starting position and size of the first data block are determined, and the data of the first data block is obtained from the storage device according to the starting position and size.

[0096] In the above embodiments, since the backup device reads the data of the data blocks to be backed up in the order of the differential bitmap under normal circumstances, therefore, by pre-reading the data of the first data block in the manner of the starting position and size indicated by the bitmap information, the corresponding data can be obtained more accurately and efficiently, thereby achieving a significant improvement in efficiency with lower memory resources.

[0097] In the embodiment of the present application, when receiving a data acquisition request for the first data block in the above step S104, sending the data of the first data block specifically includes the following steps:

[0098] Step S11: Obtain the data acquisition request;

[0099] Step S12: Determine the starting position and size of the data block requested according to the data acquisition request;

[0100] Step S13: If the starting position and size of the requested data block are the same as the starting position and size of the first data block in the cache, then search for the first data block in the cache;

[0101] Step S14: Send the data of the first data block.

[0102] The backup device may send a data acquisition request to the working node. After the working node obtains the data acquisition request, it determines the starting position and size of the data block requested according to the data acquisition request, searches for the data that matches the starting position and size in the cache, and uses the matching data as the data of the data block requested.

[0103] During specific implementation, the starting position and size of the requested data block may be compared with the starting position and size of the first data block already cached in the cache. If they are the same, it is determined that the first data block is the requested data block. Then, the data of the first data block is searched for in the cache, and the data of the first data block is returned to the backup device.

[0104] In the above embodiments, by matching the starting position and size of the data block, the accuracy of the pre-read data in the cache can be ensured, thereby improving the performance of data backup while ensuring the accuracy of data backup.

[0105] In the embodiment of the present application, the working node may sequentially pre-read the data of each data block to be backed up from the storage device in the order of the bitmap; and after transmitting the data of the data block to be backed up to the backup device, delete the corresponding data in the cache and pre-read the data of the next data block to be backed up. In this case, there is the data of one data block in the cache. For example, at this time, the number of the first data blocks is one.

[0106] At this time, the working node can compare the starting position and size of the requested data block with those of the first data block. If they are the same, it is determined that the data of the first data block is the data of the requested data block.

[0107] Here, if it is found that the starting position and size of the requested data block are different from those of the first data block, the data of the requested data block needs to be read from the storage device and returned to the backup device.

[0108] In the embodiment of the present application, in addition to sequentially pre-reading the data of each data block to be backed up from the storage device according to the bitmap order, the working node can also read the data of all data blocks to be backed up in the storage device according to the bitmap information. Alternatively, the working node can also read the data of multiple data blocks to be backed up in the storage device according to the bitmap information. In this case, there is data of multiple data blocks in the cache. For example, at this time, the number of the second data blocks is one.

[0109] Here, by parallelly reading the data of multiple data blocks from the storage device, the efficiency of data backup can be further improved and the use of resources can be reduced.

[0110] On this basis, in a possible implementation manner, the working node can compare the starting position and size of the requested data block with those of each first data block. If a first data block with the same starting position and size as the requested data block is found in the cache, the data of the requested data block is determined according to the data of the matched first data block.

[0111] Here, if it is found that the starting position and size of the requested data block are different from those of each first data block, the data of the requested data block needs to be read from the storage device and returned to the backup device.

[0112] In another possible implementation manner, the working node can obtain the cache time of each first data block. If the cache times are different, the starting position and size of the first data block with the most recent cache time are compared with those of the requested data block, so as to determine the data of the requested data block according to the comparison result. If the cache times are the same, the starting position and size of the requested data are compared with those of each first data block to determine the data of the requested data block.

[0113] In yet another possible implementation, when the working node caches the data of each first data block, it can add a sequential label to the data of each first data block in the cache. Through this sequential label, the bitmap order of each first data block in the bitmap information can be indicated. When the backup device sends a data acquisition request to the working node, it can add the sequential label of the requested data block to the data acquisition request. Then, the working node can compare the sequential label carried in the data acquisition request with the sequential labels of the data in the cache, so as to compare the data with consistent sequential labels, and compare the start position and size of the data block corresponding to this data with the start position and size of the requested data block, so as to determine the data of the requested data block according to the comparison result.

[0114] In the above implementation, the method of parallelly reading multiple data from the storage device can further improve the efficiency of data backup and reduce resource usage. By comparing the bitmap information to determine the data, the accuracy of the cached data can be determined more quickly, improving the backup efficiency.

[0115] In the embodiment of the present application, the method further includes the following steps:

[0116] If the start position and size of the requested data block are different from the start position and size of the first data block in the cache, then read the second data block from the storage device and send the data of the second data block.

[0117] In the embodiment of the present application, if it is compared that the start position and size of the requested data block are different from the start position and size of the first data block, it is determined that the data of the requested data block is not found in the cache. In this case, the working node can read the data of the second data block (i.e., the data of the requested data block) from the storage device and return the data of the second data block to the backup device.

[0118] In the above implementation, in the case where the data of the requested data block is not found in the cache, by reading the data of the second data block from the storage device, the normal execution of the data backup process can be ensured.

[0119] In the embodiment of the present application, after sending the data of the first data block, the method further includes the following steps:

[0120] Step S105: Obtain the data of the third data block to be backed up from the storage device based on the bitmap information; wherein, the third data block is one or more data blocks indicated by the first identifier after the first identifier corresponding to the first data block in the bitmap information;

[0121] Step S106: Store the data of the third data block in the cache.

[0122] In the embodiment of the present application, after the working node returns the data of the first data block to the backup device, it is also necessary to determine the third data block to be backed up from the remaining unbacked data blocks according to the bitmap information. Among them, the number of third data blocks can be multiple or one.

[0123] After determining the third data block, the data of the third data block can be read from the storage device and stored in the cache.

[0124] Here, when storing the data of the third data block in the cache, a cache time and / or an order tag can be added to the data of the third data block. Among them, the order tag is used to indicate the backup order of the third data block among all the data blocks to be backed up, and the cache time can be the time when the data is read from the storage device, or can be the time when the data is stored in the cache.

[0125] In the above embodiment, the working node can shorten the latency of data backup and improve the performance of data backup by prefetching the data of the next data block in advance according to the currently accessed data block and combining the bitmap information.

[0126] In the embodiment of the present application, the above step S105: obtaining the data of the third data block to be backed up from the storage device based on the bitmap information specifically includes the following steps:

[0127] Step S21: Obtain the remaining resource information of the memory resource;

[0128] Step S22: Determine the backup quantity of the data blocks to be backed up based on the remaining resource information;

[0129] Step S23: Determine the backup quantity of data blocks as the third data blocks from the data blocks indicated by the first identifier after the first identifier corresponding to the first data block based on the bitmap information, and obtain the data of each third data block from the storage device.

[0130] In the embodiment of the present application, the working node can regularly obtain the remaining resource information of the memory resource, so as to determine the backup quantity of the data blocks to be backed up based on the remaining resource information. Here, the size of the cache area can be determined through the remaining resource information of the memory resource; or it can be the ability to read data from the storage device in parallel.

[0131] For example, if it is determined that the remaining resource information of the memory resource is less than or equal to threshold A, it is determined that the backup quantity is 1. At this time, according to the bitmap information, one or more data blocks indicated by the first identifier after the first identifier corresponding to the first data block can be determined as the third data blocks.

[0132] For example, if it is determined that the remaining resource information of the memory resource is greater than threshold A, it is determined that the number of backups is multiple. At this time, one or more data blocks indicated by the first identifier after the first identifier corresponding to the first data block can be determined as the second data blocks according to the bitmap information. In this case, a mapping relationship between the remaining resource information and the number of backups can also be preset in advance. At this time, the working node can determine the number of backups matching the remaining resource information based on this mapping relationship, and determine the third data blocks according to this number of backups.

[0133] In the above embodiments, by determining the number of backups through the remaining resource information, and then determining the third data blocks according to the bitmap order and the number of backups, the adaptive adjustment of prefetching data can be realized, so that the data backup efficiency can be further improved and the data backup delay can be shortened while meeting the resource requirements.

[0134] In the embodiments of the present application, the step of reading the data of the second data blocks in the storage device specifically includes the following steps:

[0135] When the number of the second data blocks is multiple, the data of each of the third data blocks is read in parallel from the storage device based on the bitmap information.

[0136] In the embodiments of the present application, if it is determined that the number of the third data blocks is multiple in the above manner, the working node can read the data of each of the third data blocks in parallel from the storage device in a parallel reading manner.

[0137] Here, for multiple data read in parallel at the same time, sequence tags can be added to the corresponding data according to the backup order of each of the third data blocks, where the backup order of the third data blocks can be understood as the bitmap order.

[0138] In the embodiments of the present application, the number N of data that the working node can read in parallel each time can be obtained. Among them, if this number is less than the number of the third data blocks, the parallel reading tasks can be executed in multiple times. Here, multiple parallel reading tasks can be created according to the backup order of the multiple third data blocks, where the number of the third data blocks corresponding to the last parallel reading task may be less than the number N.

[0139] In the above embodiments, by reading the data of multiple third data blocks in parallel, the efficiency of data prefetching can be improved and the data backup delay can be shortened.

[0140] In the embodiments of the present application, the step of storing the data of the third data blocks in the cache specifically includes the following steps:

[0141] After parallelly reading the data of each of the third data blocks from the storage device based on the bitmap information, store the data of the multiple third data blocks in the cache in the backup order of the third data blocks.

[0142] If the number of the third data blocks is multiple, after reading the data of each third data block, the third data blocks can also be sequentially stored in the cache in the backup order. Then, the working node can sequentially compare the start position and size of the data block requested by the backup device with the start position and size of the corresponding third data block in the backup order.

[0143] In the above embodiment, if there are multiple third data blocks, after parallelly reading the data of the multiple third data blocks from the storage device, the data of each third data block can also be sequentially stored in the cache in the backup order. When sequentially searching for the data of each third data block in the cache, the hit rate of data search can be improved.

[0144] Next, it will be combined with Figures 2 to 4 to introduce the above process.

[0145] As Figure 2 shown is the interaction process diagram between the working node and the backup device. As Figure 2 shown, the working node is the HCI host. Ceph provides block storage functions through RBD, supporting virtual machines to directly use the block devices provided by Ceph as storage devices. That is to say, Ceph can provide storage resources for the virtual machines in the HCI host.

[0146] As Figure 2 shown, the HCI host includes: a virtual machine (guest operating system), i.e., VM (guest OS); virtio pcidriver (virtio pci driver); virtIO device (virtIO device); virt queue (virt queue); interface layer, driver layer, manager, and working process. The virtio PCI driver is a driver used in virtualization technology, mainly for implementing efficient I / O operations in virtual machines. The Virt queue is a key data structure in the Linux kernel for implementing communication between Virtio devices. VirtIO is a virtual I / O device standard designed to simplify the communication between virtual devices and virtual machines in a virtualized environment.

[0147] As Figure 2As shown in the figure, when the virtual machine deployed in the HCI host passes the write request through VirtIO, the Qemu program in the HCI host records the dirty data blocks (i.e., the data blocks with updated data) for recording incremental data. After that, a snapshot of the volume mounted by the virtual machine is created, and the bitmap information is determined according to the snapshot. Finally, the data of the data blocks to be backed up can be prefetched according to the bitmap information, and the read data blocks are cached.

[0148] As Figure 3 shown is the schematic diagram of the backup process of the backup device. As Figure 3 shown, this process specifically includes the following steps:

[0149] Step S31: Create a snapshot of the virtual machine; among them, the backup device creates a snapshot of the virtual machine by calling an interface;

[0150] Step S32: Obtain the snapshot of the volume mounted by the virtual machine; among them, the backup device obtains the snapshot of the RBD volume by calling an interface;

[0151] Step S33: Send an interface call request to the HCI host; among them, the backup device calls the interface for preparing backup resources and sends an interface call request to the HCI host. After the HCI side receives the interface call request, a Worker process is created on the HCI side;

[0152] Step S34: Obtain the CBT differential bitmap; among them, the backup device can obtain the CBT differential bitmap (i.e., the bitmap information) by calling an interface;

[0153] Step S35: Obtain the data of the data blocks to be backed up. Among them, the backup device can obtain the data of the data blocks to be backed up from the snapshot information according to the differential bitmap;

[0154] Here, the backup device can obtain data from the HCI host side through the data backup method provided by the technical solution of this application.

[0155] Assume that the differential bitmap is the bitmap information described in Table 1 above. As Figure 4 shown, the specific process of the data backup method is as Figure 4 shown, specifically including:

[0156] Step S41: Initialize the Worker process and obtain the bitmap information of the data blocks to be backed up; here, after the Worker process is created on the HCI side, the Worker process automatically obtains the bitmap information of the data blocks to be backed up;

[0157] Step S42: The Worker process loads the data 1 of the data block with an offset address of 0 from the storage device;

[0158] Step S43: The storage device returns Data 1 to the Worker process;

[0159] Step S44: The backup device reads the data 1 of the data block with an offset address of 0 from the Worker process;

[0160] Step S45: The Worker process determines whether the cache hits;

[0161] Step S46: In the case of a hit determination, return Data 1 to the backup device;

[0162] Step S47: The Worker process loads the data 2 of the data block with an offset address of 1,048,576 from the storage device;

[0163] Step S48: The storage device returns Data 2 to the Worker process;

[0164] Step S49: The backup device reads the data 2 of the data block with an offset address of 1,048,576 from the Worker process;

[0165] Step S410: The Worker process determines whether the cache hits;

[0166] Step S411: In the case of a hit determination, return Data 2 to the backup device;

[0167] Step S412: The Worker process loads the data 3 of the data block with an offset address of 4,194,304 from the storage device;

[0168] Step S413: The storage device returns Data 3 to the Worker process;

[0169] Step S414: The backup device reads the data 3 of the data block with an offset address of 4,194,304 from the Worker process;

[0170] Step S415: The Worker process determines whether the cache hits;

[0171] Step S416: In the case of a hit determination, return Data 3 to the backup device.

[0172] Step S36: Release the backup resources. After the backup device obtains all the data according to the differential bitmap, it calls the interface to release the backup resources. When the HCI side interface receives the request, the corresponding Worker process ends;

[0173] Step S37: Delete the snapshot information of the virtual machine; Among them, the backup device calls the interface to delete the snapshot information of the virtual machine.

[0174] In the above embodiments, by prefetching the data of the data block and returning the prefetching data to the backup device when receiving a data acquisition request, the data backup process can be simplified, thereby shortening the latency of data backup and improving the performance of data backup.

[0175] Those skilled in the art can understand that in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order that constitutes any limitation on the implementation process, and the specific execution order of each step should be determined according to its function and possible internal logic.

[0176] Based on the same inventive concept, the embodiments of the present application also provide a data backup device corresponding to the data backup method. Since the principle of solving problems by the device in the embodiments of the present application is similar to the above data backup method in the embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be elaborated.

[0177] Refer to Figure 5 As shown, it is a schematic diagram of a data backup device provided by an embodiment of the present application. The device includes: an acquisition unit 10, a determination unit 20, a cache unit 30, and a sending unit 40; wherein,

[0178] The acquisition unit is used to acquire a snapshot of a volume mounted by a virtual machine, wherein the volume includes at least one data block;

[0179] The determination unit is used to determine bitmap information according to the snapshot;

[0180] The cache unit is used to acquire the data of the first data block from the storage device according to the bitmap information and store the data of the first data block in the cache; wherein, the first data block is the data block indicated by the first identifier in the bitmap information, and the first identifier is used to indicate the data block in which the data stored in the at least one data block has been updated;

[0181] The sending unit is used to send the data of the first data block when receiving a data acquisition request for the data of the first data block.

[0182] In a possible implementation manner, the cache unit is further used to:

[0183] Determine the starting position and size of the first data block according to the bitmap information;

[0184] Acquire the data of the first data block from the storage device according to the starting position and size.

[0185] In a possible implementation manner, the sending unit is further used to:

[0186] Acquire the data acquisition request;

[0187] Determine the start position and size of the requested data block according to the data acquisition request;

[0188] If the start position and size of the requested data block are the same as those of the first data block in the cache, look up the data of the first data block in the cache;

[0189] Send the data of the first data block.

[0190] In a possible implementation, the device is further configured to:

[0191] If the start position and size of the requested data block are different from those of the first data block in the cache, read a second data block from the storage device and send the data of the second data block.

[0192] In a possible implementation, the device is further configured to:

[0193] After sending the first data block, obtain the data of the third data block to be backed up from the storage device based on the bitmap information; wherein, the third data block is one or more data blocks indicated by the first identifier after the first identifier corresponding to the first data block in the bitmap information;

[0194] Store the data of the third data block in the cache.

[0195] In a possible implementation, the device is further configured to:

[0196] Obtain the remaining resource information of the memory resource;

[0197] Based on the remaining resource information, determine the number of data blocks to be backed up;

[0198] Based on the bitmap information, determine the number of data blocks as the third data block among the data blocks indicated by the first identifier after the first identifier corresponding to the first data block, and obtain the data of each third data block from the storage device.

[0199] In a possible implementation, the device is further configured to:

[0200] In the case where the number of the third data blocks is multiple, read the data of each third data block from the storage device in parallel based on the bitmap information.

[0201] In a possible implementation, the device is further configured to:

[0202] After parallelly reading the data of each of the third data blocks from the storage device based on the bitmap information, store the data of the multiple third data blocks in the cache according to the backup order of the third data blocks.

[0203] In a possible implementation, the cache unit is further configured to: in the case where the first data block is the first data block to be backed up, if it detects an interface call request sent by the backup device, read the data of the first data block in the storage device and store the data of the first data block in the cache.

[0204] For the processing flow of each module in the device and the interaction flow between modules, reference can be made to the relevant descriptions in the above method embodiments, which will not be elaborated here.

[0205] Corresponding to Figure 1 the data backup method in Figure 6 As shown in

[0206] FIG. 600 is a schematic structural diagram of an electronic device 600 provided by an embodiment of the present application, including:

[0207] Obtain a snapshot of a volume mounted by a virtual machine, where the volume includes at least one data block;

[0208] Determine bitmap information according to the snapshot;

[0209] Obtain the data of the first data block from the storage device according to the bitmap information and store the data of the first data block in the cache; where the first data block is the data block indicated by the first identifier in the bitmap information, and the first identifier is used to indicate the data block in which the data stored in the at least one data block has been updated;

[0210] Send the data of the first data block when receiving a data acquisition request for the first data block.

[0211] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the data backup method described in the above method embodiment. Among them, the storage medium may be a volatile or non-volatile computer-readable storage medium.

[0212] An embodiment of the present application further provides a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the data backup method described in the above method embodiment. For details, please refer to the above method embodiment and will not be elaborated here.

[0213] Among them, the above computer program product can be specifically implemented in a way of hardware, software or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0214] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces. The indirect coupling or communication connection of the devices or units may be in an electrical, mechanical or other form.

[0215] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0216] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit exists physically alone, or two or more units are integrated into one unit.

[0217] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0218] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of the present application, used to illustrate the technical solutions of the present application, rather than limiting them. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the technical field can still modify the technical solutions described in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data backup method, characterized in that, Including: Obtaining a snapshot of a volume mounted by a virtual machine, where the volume includes at least one data block; Determining bitmap information according to the snapshot; Obtaining data of a first data block from a storage device according to the bitmap information and storing the data of the first data block in a cache; where the first data block is the data block indicated by a first identifier in the bitmap information, and the first identifier is used to indicate a data block in which the data stored in the at least one data block has been updated; When a data acquisition request for the first data block is received, sending the data of the first data block.

2. The method according to claim 1, wherein The obtaining data of the first data block from the storage device according to the bitmap information includes: Determining the starting position and size of the first data block according to the bitmap information; Obtaining the data of the first data block from the storage device according to the starting position and size.

3. The method according to claim 1, characterized in that, The sending the data of the first data block when a data acquisition request for the first data block is received includes: Obtaining the data acquisition request; Determining the starting position and size of the requested data block according to the data acquisition request; If the starting position and size of the requested data block are the same as the starting position and size of the first data block in the cache, searching for the data of the first data block in the cache; Sending the data of the first data block.

4. The method according to claim 3, characterized in that The method further includes: If the starting position and size of the requested data block are different from the starting position and size of the first data block in the cache, reading a second data block from the storage device and sending the data of the second data block.

5. The method according to claim 1, wherein After sending the first data block, the method further includes: Obtaining data of a third data block to be backed up from the storage device based on the bitmap information; where the third data block is one or more data blocks indicated by a first identifier after the first identifier corresponding to the first data block in the bitmap information; Storing the data of the third data block in the cache.

6. The method according to claim 5, wherein The obtaining data of the third data block to be backed up from the storage device based on the bitmap information includes: Obtaining resource remaining information of memory resources; Determining the number of data blocks to be backed up based on the resource remaining information; Determining the number of data blocks as the third data block from the data blocks indicated by the first identifier after the first identifier corresponding to the first data block in the bitmap information based on the bitmap information and obtaining the data of each third data block from the storage device.

7. The method according to claim 5 or 6, characterized in that, The obtaining data of the third data block to be backed up from the storage device based on the bitmap information includes: When the number of the third data blocks is multiple, parallelly reading the data of each third data block from the storage device based on the bitmap information.

8. The method according to claim 7, wherein The storing the data of the third data block in the cache includes: After parallelly reading the data of each third data block from the storage device based on the bitmap information, storing the data of the multiple third data blocks in the cache according to the backup order of the third data blocks.

9. The method according to claim 1, wherein The first data block is the first data block to be backed up. Obtaining the data of the first data block from the storage device according to the bitmap information and storing the data of the first data block in the cache includes: If an interface call request sent by the backup device is detected, read the data of the first data block in the storage device and store the data of the first data block in the cache.

10. An electronic device, characterized in that, Including: A processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the data backup method according to any one of claims 1 to 9 are executed.

Images