Data backup method and device, equipment, medium and product
By storing bitmaps to a shared block when the virtual machine is shut down, the problem of being unable to perform incremental backups after the virtual machine is booted up is solved, achieving efficient incremental backups, saving storage space and performance loss, and supporting multiple storage formats other than QCOW2.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-18
- Publication Date
- 2026-04-07
AI Technical Summary
In existing technologies, incremental backups cannot be performed after a virtual machine is powered on, resulting in wasted storage space and performance degradation. This is especially true for backend storage in RAW format or other non-QCOW2 formats, where incremental backups fail after the virtual machine is powered off.
When the virtual machine is shut down, the target bitmap is stored in the shared block and the allocated storage space. When the virtual machine is started, the target bitmap is retrieved from the shared block and imported to achieve incremental backup.
Improve backup efficiency, save storage space and performance loss, avoid the risk of bitmap data loss or mismatch due to improper management, and support incremental backup of multiple disk formats.
Smart Images
Figure CN121807620A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, specifically to methods, apparatus, equipment, media, and products for data backup. Background Technology
[0002] QEMU virtual machines can interface with various backend storage formats, such as QCOW2 and RAW files, or local block storage devices, or block devices connected to the operating system (e.g., RADOS block device, i.e., RBD) via iSCSI (Internet Small Computer System Interface) or FC (Fibre Channel) card networks.
[0003] Currently, QEMU only provides incremental backup functionality for QCOW2 formatted backend storage, allowing virtual machines to be migrated or shut down. For RAW format or other non-QCOW2 backend storage, shutting down a virtual machine will cause incremental backups to fail; only full backups can be performed after the virtual machine is powered on. This results in significant waste of storage space and performance degradation during the backup process. Summary of the Invention
[0004] In view of this, this application provides a method, apparatus, device, medium and product for data backup to solve the problem that incremental backup is not possible after virtual machines are booted up.
[0005] In a first aspect, this application provides a data backup method, the method comprising: In response to a shutdown command for the target virtual machine, obtain the target bitmap corresponding to the target virtual machine; The target bitmap is stored in a shared block within the target storage space allocated for the target virtual machine; the shared block is a pre-allocated data block used to record bitmaps of multiple virtual machines. In response to a startup command for the target virtual machine, the target bitmap is obtained from the target storage space of the shared block and imported. In response to a backup command for the target virtual machine, an incremental backup of the target virtual machine is performed based on the target bitmap.
[0006] In some optional implementations, obtaining the target bitmap corresponding to the target virtual machine includes: If the target virtual machine has been shut down and the management process corresponding to the target virtual machine has not been exited, the target bitmap corresponding to the target virtual machine is obtained using the management process. And / or, The importing of the target bitmap includes: If the management process corresponding to the target virtual machine has been started, but the target virtual machine has not been started, the target bitmap is imported using the management process.
[0007] In some optional implementations, the method further includes: Determine the current state of the target bitmap stored in the shared block; If the current state of the target bitmap is a first state indicating that the target bitmap is valid, the operation of importing the target bitmap is performed.
[0008] In some optional implementations, the method further includes: After storing the target bitmap in the target storage space allocated for the target virtual machine in the shared block, the target bitmap stored in the shared block is set to the first state; After obtaining the target bitmap from the target storage space of the shared block, the target bitmap stored in the shared block is set to a second state indicating that the target bitmap is invalid.
[0009] In some optional implementations, the method further includes: In response to a startup command for the target virtual machine, the first disk summary information corresponding to the target virtual disk stored in the shared block is obtained; the target virtual disk is a virtual disk allocated for the target virtual machine. If the current second disk digest information of the target virtual disk is consistent with the first disk digest information, the operation of importing the target bitmap is performed.
[0010] In some optional implementations, the method further includes: In response to a shutdown command for the target virtual machine, determine the current first disk summary information of the target virtual disk; The first disk summary information is stored in the target storage space allocated for the target virtual machine in the shared block.
[0011] In some optional implementations, the metadata header of the shared block includes disk information of the virtual disks allocated to each virtual machine, the disk information including disk identifiers and storage addresses of the storage space allocated to the virtual machines; The step of storing the target bitmap into the target storage space allocated for the target virtual machine in the shared block includes: Determine the target storage address in the metadata header that corresponds to the disk identifier of the target virtual disk; the target virtual disk is a virtual disk allocated for the target virtual machine; store the target bitmap in the target storage space corresponding to the target storage address in the shared block; The step of obtaining the target bitmap from the target storage space of the shared block includes: Determine the target storage address in the metadata header that corresponds to the disk identifier of the target virtual disk; The target bitmap is obtained from the target storage space corresponding to the target storage address in the shared block.
[0012] Secondly, this application provides a data backup apparatus, the apparatus comprising: The acquisition module is used to acquire the target bitmap corresponding to the target virtual machine in response to a shutdown command for the target virtual machine; A storage module is used to store the target bitmap into a shared block within the target storage space allocated for the target virtual machine; the shared block is a pre-allocated data block used to record bitmaps of multiple virtual machines. A processing module is configured to, in response to a startup command for the target virtual machine, obtain the target bitmap from the target storage space of the shared block and import the target bitmap; A backup module is configured to perform incremental backups of the target virtual machine based on the target bitmap in response to a backup command for the target virtual machine.
[0013] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the data backup method described in the first aspect or any corresponding embodiment.
[0014] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the data backup method described in the first aspect or any corresponding embodiment.
[0015] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to perform a data backup method described in the first aspect or any corresponding embodiment.
[0016] The data backup method provided in this application pre-allocates a shared block for recording the bitmaps of each virtual machine. When the target virtual machine is shut down, its corresponding target bitmap can be stored in the target storage space allocated to it by the shared block. When the target virtual machine starts up subsequently, it can read its own target bitmap from the shared block and then continue incremental backup. Recording the bitmaps of each virtual machine by the shared block is not limited by disk format. When QEMU is connected to backend storage other than QCOW2 format, the effectiveness of incremental backup is not affected when the virtual machine is shut down; that is, incremental backup can continue after the virtual machine is booted up, which improves backup efficiency and saves storage space and performance loss during the backup process. Furthermore, unified management of the bitmaps of multiple virtual machines based on the shared block has higher reliability than local storage and can effectively avoid risks such as bitmap data loss, mismatch, and tampering caused by improper management. In addition, read and write operations can be performed directly on the storage space of the shared block without a file system, which can reduce the performance and robustness loss caused by an additional shared file system. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0018] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this application; Figure 2 This is a schematic flowchart of a first method for data backup according to an embodiment of this application; Figure 3 This is a schematic diagram illustrating a process for data backup based on shared blocks according to an embodiment of this application; Figure 4 This is a schematic diagram illustrating the principle of locally persisting a bitmap according to an embodiment of this application; Figure 5 This is a schematic diagram illustrating the principle of persistent bitmap based on shared blocks according to an embodiment of this application; Figure 6 This is a second flowchart illustrating a data backup method according to an embodiment of this application; Figure 7 This is a schematic diagram of a shared block according to an embodiment of this application; Figure 8 This is a structural block diagram of a data backup apparatus according to an embodiment of this application; Figure 9This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0020] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.
[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0022] Before providing a detailed description of the embodiments of this application, some of the terms and concepts involved in the embodiments of this application will be explained. These explanations are intended to make the embodiments of this application easier to understand and should not be considered as limiting the scope of protection claimed in this application.
[0023] (1) Full backup: A full backup refers to a complete copy of all data or applications at a specific point in time. In practical applications, it involves using a storage device to perform a full backup of the entire system, including the system and all data.
[0024] (2) Incremental backup: Incremental backup refers to the backup of files that have been added or modified since the first full backup or the last incremental backup. This means that the first incremental backup will be the files added or modified after the full backup; the second incremental backup will be the files added or modified after the first incremental backup, and so on.
[0025] Figure 1 This diagram illustrates a comparison between full backups and incremental backups. Figure 1 As can be seen, incremental backup only requires backing up less data, resulting in higher backup efficiency, especially when data increases or is modified only slightly, where the efficiency improvement is more significant.
[0026] (3) QEMU (Quick Emulator): It is an open-source machine emulator and virtualization software. QEMU can run a complete operating system as an independent virtual machine, or it can be used in conjunction with KVM (kernel-based virtual machine) to achieve hardware-accelerated virtualization.
[0027] (4) QCOW2 (QEMU Copy-On-Write): is a storage format for virtual disk images. The QCOW2 format separates the physical storage layer from the virtual layer by adding a mapping between logical and physical blocks. Each logical block is mapped to its physical offset, enabling storage over-allocation and virtual machine snapshots, where each QCOW volume represents only changes made to the underlying disk image.
[0028] (5) Bitmap: A bitmap is a data structure that uses each bit to represent a certain state, using one bit to mark the state of an element. In QEMU, bitmap is used to record dirty page blocks of block devices and is applied to various functions such as migration, backup, and snapshots. Bitmap exists in memory or in the QCOW2 image file.
[0029] Since Bitmap can indicate which data blocks are dirty (i.e., which data blocks have been added or modified compared to the previous backup), incremental backup can be performed by locating the data blocks that need to be backed up in this incremental backup based on Bitmap.
[0030] Virtual machine backend storage supports RAW format files, QCOW2 format files, local block storage devices, iSCSI block storage, RBD block storage, etc. For backend storage devices in non-QCOW2 formats such as RAW, although incremental backups of any format backend storage can be performed using Bitmap, the Bitmap in memory will be lost once the virtual machine is shut down, breaking the incremental backup chain. If you want to back up again after the virtual machine is powered on, you can only redo a full backup.
[0031] According to an embodiment of this application, a method embodiment for data backup is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0032] This embodiment provides a data backup method that can be applied to terminals or servers capable of implementing virtual machines, such as servers in a cluster. Figure 2 This is a flowchart of a data backup method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps.
[0033] Step S201: In response to the shutdown command for the target virtual machine, obtain the target bitmap corresponding to the target virtual machine.
[0034] In this embodiment, one or more virtual machines can run on a server or other device to provide the required business functions. For ease of description, one virtual machine is used as an example, and this virtual machine is referred to as the target virtual machine. During the operation of the target virtual machine, in order to perform incremental backups, a corresponding bitmap is created for the target virtual machine, namely the target bitmap. Each bit in the target bitmap is used to record whether the data pages corresponding to the target virtual machine are dirty pages, that is, whether the data pages have added or modified data.
[0035] When backing up a target virtual machine, the target bitmap can be used to identify dirty data pages that have changed compared to the previous backup (full backup or incremental backup). Only these data pages are incrementally backed up, and after the backup, these data pages become clean data pages.
[0036] Backing up a virtual machine involves backing up the data on the disk corresponding to that virtual machine. For disks in various formats such as QCOW2 and RAW, incremental backups are possible because the virtual machine's bitmap can be recorded in memory during operation. However, after the virtual machine is shut down, the data in memory is cleared, making direct incremental backups impossible upon restarting. A full backup must be performed first before incremental backups can be performed.
[0037] In this embodiment, after receiving the shutdown command for the target virtual machine, the target bitmap corresponding to the target virtual machine is obtained. Specifically, the target bitmap corresponding to the target virtual machine can be obtained from the device's memory.
[0038] Step S202: Store the target bitmap in the target storage space allocated for the target virtual machine in the shared block; the shared block is a data block that is pre-allocated to record bitmaps of multiple virtual machines.
[0039] In this embodiment, the bitmaps obtained when the virtual machines are shut down are managed uniformly, for example, they can be uploaded to the cloud. Specifically, a storage area is pre-allocated from a storage system such as a cloud platform. This storage area can be shared with multiple virtual machines to store the bitmaps of multiple virtual machines. This storage area can be implemented using block storage devices, and the block devices shared with multiple virtual machines are called shared blocks. For example, a cluster can apply for a shared block for storing bitmaps, and this shared block records the bitmaps of each virtual machine within the cluster.
[0040] Specifically, the shared block allocates corresponding storage space for each virtual machine that needs to record the bitmap; this storage space is a portion of the storage area within the shared block. Correspondingly, a target storage space is allocated for the target virtual machine within the shared block. After obtaining the target bitmap of the target virtual machine, the target bitmap can be stored in the target storage space. At this point, even if the memory data is cleared due to the shutdown of the target virtual machine, the persistence of the target bitmap can still be achieved.
[0041] In step S203, in response to the startup command for the target virtual machine, the target bitmap is obtained from the target storage space of the shared block and imported.
[0042] In this embodiment, if the target virtual machine is restarted after it is shut down, the target bitmap stored in the shared block before shutdown can be obtained by reading the target storage space of the shared block. It can be understood that, under normal circumstances, the target bitmap read from the shared block is consistent with the bitmap in the memory when the target virtual machine is shut down.
[0043] Once the target bitmap is obtained, it can be imported. For example, the target bitmap can be loaded into memory for later use.
[0044] Step S204: In response to the backup command for the target virtual machine, perform an incremental backup of the target virtual machine based on the target bitmap.
[0045] After the target virtual machine is powered on, if a backup of the target virtual machine is required, since the target bitmap has been imported at this time, an incremental backup of the target virtual machine can be performed based on the target bitmap, without the need for a full backup, thereby improving backup efficiency.
[0046] Figure 3 This diagram illustrates a process for data backup based on shared blocks. Each virtual machine is allocated a corresponding disk, i.e., a virtual disk; and a virtual machine can be allocated multiple virtual disks. Each virtual disk typically has its own bitmap. For simplicity, this diagram primarily uses a single bitmap of a virtual machine.
[0047] like Figure 3As shown, at time t1, the virtual disk of the virtual machine is backed up. If the bitmap does not exist at this time, a full backup is required, and a bitmap corresponding to the virtual machine, i.e., Bitmap1, can be created. It can be understood that in the bitmap created after a full backup, each bit represents a clean data page. After a period of time, at time t2, during the period from time t1 to time t2, the virtual disk will generally experience data changes, such as data addition or modification, causing the bitmap of the virtual machine to change from Bitmap1 to Bitmap2. During the operation of the virtual machine, its bitmap is maintained in the host machine's memory by the QEMU process.
[0048] If data backup is required at time t2, since a bitmap (Bitmap2) is available at this time, incremental backup can be performed based on the current bitmap. Multiple incremental backups can be performed after time t1, but this will not be elaborated upon in this embodiment.
[0049] If, at time t3, an instruction to shut down the virtual machine is received, meaning the virtual machine needs to be shut down, the virtual machine's bitmap at this time is Bitmap3. In this embodiment, the bitmap Bitmap3 can be retrieved from memory and stored in a shared block.
[0050] At time t4, the instruction to restart the virtual machine is received, allowing the virtual machine to boot up. At this point, the bitmap corresponding to the virtual machine is retrieved from the shared block and loaded into memory, resulting in bitmap Bitmap3' existing in memory. It can be understood that during the period from time t3 to time t4, since the virtual machine is in a shutdown state, the data in the virtual disk is not modified. Therefore, the bitmap Bitmap3' obtained at this time is consistent with the bitmap Bitmap3 when the virtual machine was shut down, and bitmap Bitmap3' accurately represents which data pages in the virtual disk are dirty pages. After time t4, if new data is modified or added, the corresponding bits in bitmap Bitmap3' can be adaptively modified, thus correctly recording whether each data page has been modified.
[0051] At time t5, the virtual machine's bitmap is Bitmap4. If a backup is needed at this time, an incremental backup can be performed directly based on this bitmap Bitmap4.
[0052] It is understandable that if the virtual machine's bitmap Bitmap3 is not persisted when the virtual machine is shut down, then when the virtual machine is powered on at time t4, the virtual machine will not have the corresponding bitmap. Therefore, when backing up at time t5, a full backup is required, which is not only inefficient but also consumes a lot of processing resources.
[0053] In this embodiment, using shared blocks to record the bitmaps of each virtual machine eliminates the limitation of virtual machine disk formats. This means the solution is applicable to incremental backups of various disk formats, including QCOW2 and RAW. While QCOW2 disks can persist bitmaps, they store them in a QCOW2 image file, essentially storing them locally. This embodiment, however, stores the bitmaps of all virtual machines uniformly in shared blocks, which not only facilitates management but also provides higher reliability, reducing the risk of bitmap data loss due to local disk or file system failures. Furthermore, by recording each bitmap in shared blocks, read and write operations can be performed directly on the shared block's storage space without requiring a file system, minimizing performance and robustness losses caused by an additional shared file system.
[0054] Figure 4 This illustrates a principle for locally persisting a bitmap, which requires saving the bitmap as a bitmap file and storing it locally. For example... Figure 4 As shown, virtual machine VM1 on host 1 is allocated two virtual disks, Disk1 and Disk2; virtual machine VM2 on host 2 is allocated one virtual disk, Disk3. On host 1, the bitmaps of each virtual disk can be recorded as files, forming corresponding bitmap files, namely Bitmap file 1 and Bitmap file 2. Similarly, bitmap file 3 corresponding to virtual disk Disk3 can be recorded on host 2. This scheme requires managing each bitmap as a file, and each host needs to record its own bitmap file.
[0055] Figure 5 This illustrates a schematic diagram of a principle based on a shared block persistent bitmap. Figure 4 The scenario is similar, with virtual machines VM1 and VM2 also existing, but as... Figure 5 As shown, in this embodiment, a shared block is requested in the storage system (e.g., a centralized or distributed storage system) for multiple virtual machines to share. This block allocates corresponding storage space to each virtual machine to record the corresponding bitmap, for example... Figure 5 The bit shown Figure 1 ,Bit Figure 2 ,Bit Figure 3 Wait. It's understandable that the bitmap in the shared block doesn't need to use a common file system.
[0056] The data backup method provided in this embodiment pre-allocates a shared block for recording the bitmaps of each virtual machine. When the target virtual machine is shut down, its corresponding target bitmap can be stored in the target storage space allocated to it by the shared block. When the target virtual machine starts up subsequently, it can read its own target bitmap from the shared block and then continue incremental backup. Recording the bitmaps of each virtual machine by the shared block is not limited by disk format. When QEMU is connected to backend storage other than QCOW2 format, the effectiveness of incremental backup is not affected when the virtual machine is shut down; that is, incremental backup can continue after the virtual machine is restarted, improving backup efficiency and saving storage space and performance loss during the backup process. Furthermore, unified management of the bitmaps of multiple virtual machines based on the shared block has higher reliability than local storage, effectively avoiding risks such as bitmap data loss, mismatch, and tampering caused by improper management. In addition, read and write operations can be performed directly on the storage space of the shared block without a file system, reducing the performance and robustness loss caused by an additional shared file system.
[0057] This embodiment provides a data backup method that can be applied to terminals or servers capable of implementing virtual machines, such as servers in a cluster. Figure 6 This is a flowchart of a data backup method according to an embodiment of this application, such as... Figure 6 As shown, the process includes the following steps.
[0058] Step S601: In response to the shutdown command for the target virtual machine, obtain the target bitmap corresponding to the target virtual machine.
[0059] Please see details Figure 2 Step S201 of the illustrated embodiment will not be described again here.
[0060] Optionally, step S601, "obtain the target bitmap corresponding to the target virtual machine", may include: obtaining the target bitmap corresponding to the target virtual machine using the management process when the target virtual machine has been shut down and the management process corresponding to the target virtual machine has not been exited.
[0061] In this embodiment, the acquisition, saving, and importing of the target bitmap can be implemented by the management process of the target virtual machine, specifically the QEMU process. Upon receiving a shutdown command for the target virtual machine, the target virtual machine is typically shut down first, and then the management process corresponding to the target virtual machine exits.
[0062] To ensure the accuracy of the bitmap stored in the shared block, after shutting down the target virtual machine, since the target virtual machine can no longer modify the data in the virtual disk, the target bitmap of the target virtual machine can be obtained through the management process at this time. This ensures that no new content is written to the corresponding virtual disk after the target bitmap is obtained; then the management process of the target virtual machine is exited to completely shut down the target virtual machine.
[0063] Step S602: Store the target bitmap in the target storage space allocated for the target virtual machine in the shared block; the shared block is a data block that is pre-allocated to record bitmaps of multiple virtual machines.
[0064] In this embodiment, a shared block is a block device, specifically a storage device that is accessed for reading and writing in fixed-size blocks (e.g., 4K, 64K, etc.). The shared block has a metadata header that records its metadata information, specifically including disk information for each virtual disk allocated to each virtual machine. This disk information includes a disk identifier and the storage address of the storage space allocated to the virtual machine, enabling reading and writing to each storage space.
[0065] Figure 7 A schematic diagram of a shared block structure is shown, such as... Figure 7 As shown, the metadata header includes Magic (a magic number used to indicate the validity of the shared block data structure), Version (a version number indicating the version number of the shared block data structure), Size (the size of the shared block), and Global mutex (a global lock, which will be explained later). Furthermore, the metadata header also includes disk information corresponding to multiple virtual disks. Figure 7 The disk information for virtual disk 1 (disk_1), virtual disk 2 (disk_2), and virtual disk n (disk_n) is shown. Specifically, the disk information may include: disk name, disk size, bitmap size, and the storage address corresponding to the storage space allocated to the virtual machine, that is, the storage address of the bitmap.
[0066] As shown above, a shared block consists of multiple fixed-size data blocks, such as 64K data blocks. Metadata headers can be stored within the corresponding data blocks and can be expanded using a linked list. Figure 7 As shown, the metadata header also includes a pointer to the next linked list (i.e., the metadata header): Next head, which can be used to expand the metadata header when there are many virtual disks.
[0067] In this embodiment, bitmap read / write operations between the virtual machine and the shared block are performed based on this metadata header. Specifically, step S602, "Storing the target bitmap into the target storage space allocated for the target virtual machine in the shared block," includes steps S6021 to S6022.
[0068] Step S6021: Determine the target storage address in the metadata header that corresponds to the disk identifier of the target virtual disk; the target virtual disk is the virtual disk allocated for the target virtual machine.
[0069] Step S6022: Store the target bitmap to the target storage space corresponding to the target storage address in the shared block.
[0070] In this embodiment, each target virtual machine is allocated a corresponding virtual disk, i.e., a target virtual disk. For example... Figure 4 As shown, the virtual disks allocated to virtual machine VM1 include Disk1 and Disk2, both of which can be used as target virtual disks. When shutting down the target virtual machine, the disk identifier of the target virtual disk can be determined, such as disk ID, disk name, etc. This disk identifier needs to be globally unique to ensure that the disk identifiers recorded in the shared block are different.
[0071] During the process of storing the target bitmap into the shared block, the metadata header of the shared block can be queried based on the disk identifier of the target virtual disk, thereby determining the disk information corresponding to the target virtual disk in the metadata header, and then determining the storage address therein, which is the target storage address.
[0072] like Figure 7 As shown, if the target virtual disk is disk_1, the corresponding target storage address (Bitmap address) can be determined in the metadata header. Then, the obtained target bitmap can be stored in the storage space corresponding to that target storage address. This storage space is the target storage space, for example, [example of target storage space]. Figure 7 median Figure 1 The corresponding storage space, the bits stored at this time Figure 1 This is the target bitmap.
[0073] In step S603, in response to the startup command for the target virtual machine, the target bitmap is obtained from the target storage space of the shared block and imported.
[0074] Similarly, step S603, "obtain the target bitmap from the target storage space of the shared block", may include steps S6031 to S6032.
[0075] Step S6031: Determine the target storage address in the metadata header that corresponds to the disk identifier of the target virtual disk.
[0076] Step S6032: Obtain the target bitmap from the target storage space corresponding to the target storage address in the shared block.
[0077] In this embodiment, similar to the process of writing the target bitmap to the shared block, when the target virtual machine boots up, it is necessary to read the target bitmap in the shared block. At this time, the corresponding target storage address is also determined based on the disk identifier of the target virtual disk, that is, the storage address used to store the target bitmap. Then, the target bitmap can be read from the target storage space corresponding to the target storage address in the shared block.
[0078] Optionally, similar to obtaining the target bitmap, importing the target bitmap also needs to be done at an appropriate time. Specifically, step S603 "importing the target bitmap" may include: importing the target bitmap using the management process when the management process corresponding to the target virtual machine has been started but the target virtual machine has not been started.
[0079] In this embodiment, to ensure that the imported target bitmap accurately represents the dirty page information of the target virtual machine, the management process corresponding to the target virtual machine (e.g., the QEMU process) is started first when the target virtual machine is started, and then the target virtual machine itself is started. Therefore, when importing the target bitmap, it can be done after the management process corresponding to the target virtual machine is started, and then the target virtual machine is started after the target bitmap is imported based on the management process. This avoids the problem of inaccurate target bitmaps caused by changes in virtual disk data due to starting the target virtual machine and then importing the target bitmap.
[0080] In some optional implementations, the shared block also records relevant verification information to verify whether the bitmap recorded in the shared block is usable. Specifically, the verification information may include status information indicating whether the bitmap is valid.
[0081] Specifically, when it is necessary to read the target bitmap in the shared block, the method may further include steps a1 to a2.
[0082] Step a1: Determine the current state of the target bitmap stored in the shared block.
[0083] Step a2: If the current state of the target bitmap is the first state indicating that the target bitmap is valid, perform the operation of importing the target bitmap.
[0084] In this embodiment, the target storage space can record not only the target bitmap but also its current state. The bitmap's state is divided into a first state (indicating the bitmap is valid) and a second state (indicating the bitmap is invalid). If the target bitmap's current state is the first state, it means the target bitmap is currently usable, and therefore can be read from the target storage space, and the subsequent import process can be executed. Conversely, if the target bitmap's current state is the first state, it means the target bitmap recorded in the shared block is invalid and should not be used. Therefore, the target virtual machine can be prevented from reading the target bitmap, and since there is no bitmap after the target virtual machine boots up, a full backup needs to be performed first.
[0085] Optionally, the bitmap state recorded in the shared block is specifically used to indicate whether the bitmap is consistent with the bitmap during virtual machine operation. Specifically, the method further includes steps a3 and a4.
[0086] Step a3: After storing the target bitmap in the target storage space allocated for the target virtual machine in the shared block, set the target bitmap stored in the shared block to the first state.
[0087] Step a4: After obtaining the target bitmap from the target storage space of the shared block, the target bitmap stored in the shared block is set to a second state indicating that the target bitmap is invalid.
[0088] In this embodiment, when the target virtual machine is shut down (e.g. Figure 3 At time t3 (as shown), after storing the target bitmap to the shared block, the target bitmap stored in the shared block is set to the first state, indicating that the target bitmap is available. Then, when the target virtual machine boots up (e.g., ...), ... Figure 3 At time t4, since the target bitmap stored in the shared block is in the first state, the target virtual machine can read and load the target bitmap normally.
[0089] Furthermore, after the target virtual machine obtains the target bitmap from the shared block, for example in Figure 3 After time t4, as shown, the target bitmap recorded in the shared block remains unchanged, but the target virtual machine's own bitmap changes in real time, meaning the two bitmaps are inconsistent. Therefore, the target bitmap in the shared block can be marked as the second state to indicate the inconsistency. Only when the target bitmap is rewritten later will this state be changed from the second state to the first state.
[0090] That is, after reading the bitmap in the shared block, the bitmap in the shared block will be temporarily marked as the second state; after writing the bitmap to the shared block, the bitmap in the shared block will be modified back to the first state. This can identify certain abnormal situations, such as the bitmap not being written to the shared block correctly due to an abnormal exit of QEMU.
[0091] For example, in Figure 3 At time t3, as shown, there is usually a process of reading the target bitmap from the shared block before this point, meaning the target bitmap in the shared block is in the second state. Furthermore, if QEMU exits abnormally at time t3, causing the latest target bitmap not to be written to the shared block, the state of the target bitmap in the shared block will not change to the first state. At the subsequent time t4, when the target virtual machine boots up, the target bitmap in the shared block is still in the unusable second state, indicating that it may be inconsistent with the bitmap when the virtual machine shut down. Therefore, the target bitmap recorded in the shared block is unusable, and the virtual machine cannot read the target bitmap from the shared block to avoid using an incorrect bitmap for incremental backup.
[0092] Optionally, the verification information recorded in the shared block may also include disk digest information of the virtual disk, such as MD5 information. Accordingly, the method may also include steps b1 to b2.
[0093] Step b1: In response to the startup command for the target virtual machine, obtain the first disk summary information corresponding to the target virtual disk stored in the shared block; the target virtual disk is a virtual disk allocated for the target virtual machine.
[0094] Step b2: If the current second disk summary information of the target virtual disk is consistent with the first disk summary information, perform the step of importing the target bitmap.
[0095] In this embodiment, for the target virtual disk of the target virtual machine, the shared block also records the summary information of the target virtual disk, namely the first disk summary information. The first disk summary information is specifically obtained by performing a corresponding summary algorithm on the data in the target virtual disk.
[0096] Specifically, the corresponding first disk digest information can be written simultaneously with the target bitmap. For example, the method also includes steps b3 and b4.
[0097] Step b3: In response to the shutdown command for the target virtual machine, determine the current first disk summary information of the target virtual disk.
[0098] Step b4: Store the first disk summary information into the target storage space allocated for the target virtual machine in the shared block.
[0099] In this embodiment, when shutting down the target virtual machine, the target bitmap and the first disk digest information corresponding to the target virtual disk can be determined, and both can be synchronously stored in the corresponding location of the shared block. Specifically, the first disk digest information of the target virtual disk can be determined based on the disk data of the target virtual disk at the current time (i.e., the time when the target virtual machine is shut down).
[0100] Subsequently, when the target virtual machine is started, the previously recorded first disk summary information can be retrieved from the shared block and compared with the latest second disk summary information of the target virtual disk. If they are the same, it means that the data on the target virtual disk has not changed, and incremental backup can be performed using the target bitmap; therefore, the step of importing the target bitmap can be executed. If they are different, it means that the data on the target virtual disk has changed, and correspondingly, the data in the target bitmap is no longer up-to-date. Therefore, the target bitmap in the shared block needs to be discarded, and a full backup needs to be performed after the target virtual machine is booted. The second disk summary information of the target virtual disk can be determined based on the disk data of the target virtual disk at the current time (i.e., the time when the target virtual machine is booted).
[0101] It is understandable that when verifying the target bitmap based on its current state, disk summary information, and other verification information, the verification information and the target bitmap can be obtained simultaneously. After confirming the target bitmap's usability based on the verification information, the target bitmap can be directly imported. Alternatively, the verification information can be obtained first for verification, and after confirming the target bitmap's usability based on the verification information, the target bitmap can be obtained from the shared block and then imported. This embodiment does not limit this approach.
[0102] Step S604: In response to the backup command for the target virtual machine, perform an incremental backup of the target virtual machine based on the target bitmap.
[0103] Please see details Figure 2 Step S204 of the illustrated embodiment will not be described again here.
[0104] In some alternative implementations, the shared block differs from a general file system in that the storage space allocated to each virtual disk is independent. When a virtual machine reads or writes its own bitmap, it only operates on its own storage space, without interference between them. Therefore, simultaneous read and write operations by multiple virtual machines are possible, meaning multiple virtual machines can simultaneously read and write bitmap data on the shared block. No global mutex lock is required when reading the bitmap, enabling fast synchronization.
[0105] In some scenarios, a global mutex lock needs to be requested. Specifically, the method may also include steps c1 to c3.
[0106] Step c1: In response to the update instruction that updates the metadata header of the shared block, acquire the global mutex lock; the update instruction is the instruction triggered when adding a virtual disk, deleting a virtual disk, or modifying the size of a virtual disk.
[0107] Step c2: Update the metadata header of the shared block according to the update instruction.
[0108] Step c3: Release the global mutex lock.
[0109] In this embodiment, when it is necessary to add, delete, or modify the size of a virtual disk, an update instruction for the metadata header of the shared block is triggered. See also Figure 7 As shown, if a new virtual disk A needs to be added, the disk information corresponding to virtual disk A needs to be added to the metadata header; if an existing virtual disk B needs to be deleted, the disk information corresponding to virtual disk B needs to be deleted from the metadata header; if the size of an existing virtual disk C needs to be modified, some disk information of virtual disk B in the metadata header needs to be modified, including disk size and bitmap size.
[0110] Upon receiving the update instruction, to avoid affecting read and write operations on the bitmap, a global mutex is acquired, which locks the metadata header to prevent other processes from performing operations and causing data corruption. After updating the metadata header of the shared block, the global mutex is released, and the process can return to normal.
[0111] Alternatively, a corresponding bitmap can be set for the shared block. This bitmap is used to indicate which data blocks in the shared block have been allocated. When creating a virtual disk later, the unallocated data blocks in the shared block can be determined based on this bitmap and then allocated to the newly created virtual disk to store the bitmap of the virtual disk.
[0112] In this embodiment, a global lock is only required for scenarios such as disk creation, deletion, and resizing, followed by exclusive writing of the metadata header. In other scenarios, simultaneous reading and writing can be achieved through sharing without the need for a lock. Disk creation, deletion, and resizing are very infrequent operations in cloud scenarios. In other words, the shared block is in a lock-free state most of the time, and the introduction of this shared block has almost no performance impact on existing cloud services.
[0113] The data backup method provided in this embodiment supports various types of backend storage and does not affect the incremental backup chain after the virtual machine is shut down, meaning that the validity of incremental backups is not affected even after the virtual machine is shut down. Furthermore, it has no impact on virtualization operations such as virtual machine migration, snapshots, and backups, preserving the original characteristics of virtualization. Through a series of verification mechanisms, the validity of the bitmap in the shared block can be determined, avoiding incremental backup recovery failures due to inconsistencies between the bitmap and backend storage. The shared block records the bitmaps of each virtual machine, supporting simultaneous read and write operations, and locking is only required in a few scenarios such as disk creation, deletion, and resizing, ensuring the processing performance of the original business.
[0114] This embodiment also provides a data backup apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0115] This embodiment provides a data backup device, such as... Figure 8 As shown, the device includes: The acquisition module 801 is used to acquire the target bitmap corresponding to the target virtual machine in response to a shutdown command for the target virtual machine; Storage module 802 is used to store the target bitmap into a shared block within the target storage space allocated for the target virtual machine; the shared block is a pre-allocated data block used to record bitmaps of multiple virtual machines; Processing module 803 is configured to, in response to a startup command for the target virtual machine, obtain the target bitmap from the target storage space of the shared block and import the target bitmap; Backup module 804 is used to perform incremental backup of the target virtual machine according to the target bitmap in response to a backup command for the target virtual machine.
[0116] In some optional implementations, obtaining the target bitmap corresponding to the target virtual machine includes: If the target virtual machine has been shut down and the management process corresponding to the target virtual machine has not been exited, the target bitmap corresponding to the target virtual machine is obtained using the management process. And / or, The importing of the target bitmap includes: If the management process corresponding to the target virtual machine has been started, but the target virtual machine has not been started, the target bitmap is imported using the management process.
[0117] In some optional embodiments, the processing module 803 is further configured to: Determine the current state of the target bitmap stored in the shared block; If the current state of the target bitmap is a first state indicating that the target bitmap is valid, the operation of importing the target bitmap is performed.
[0118] In some optional embodiments, the processing module 803 is further configured to: After storing the target bitmap in the target storage space allocated for the target virtual machine in the shared block, the target bitmap stored in the shared block is set to the first state; After obtaining the target bitmap from the target storage space of the shared block, the target bitmap stored in the shared block is set to a second state indicating that the target bitmap is invalid.
[0119] In some optional embodiments, the processing module 803 is further configured to: In response to a startup command for the target virtual machine, the first disk summary information corresponding to the target virtual disk stored in the shared block is obtained; the target virtual disk is a virtual disk allocated for the target virtual machine. If the current second disk digest information of the target virtual disk is consistent with the first disk digest information, the operation of importing the target bitmap is performed.
[0120] In some optional embodiments, the processing module 803 is further configured to: In response to a shutdown command for the target virtual machine, determine the current first disk summary information of the target virtual disk; The first disk summary information is stored in the target storage space allocated for the target virtual machine in the shared block.
[0121] In some optional implementations, the metadata header of the shared block includes disk information of the virtual disks allocated to each virtual machine, the disk information including disk identifiers and storage addresses of the storage space allocated to the virtual machines; The step of storing the target bitmap into the target storage space allocated for the target virtual machine in the shared block includes: Determine the target storage address in the metadata header that corresponds to the disk identifier of the target virtual disk; the target virtual disk is a virtual disk allocated for the target virtual machine; store the target bitmap in the target storage space corresponding to the target storage address in the shared block; The step of obtaining the target bitmap from the target storage space of the shared block includes: Determine the target storage address in the metadata header that corresponds to the disk identifier of the target virtual disk; The target bitmap is obtained from the target storage space corresponding to the target storage address in the shared block.
[0122] In some optional embodiments, the processing module 803 is further configured to: In response to an update instruction that updates the metadata header of the shared block, a global mutex lock is acquired; the update instruction is triggered when a virtual disk is added, deleted, or its size is modified. Update the metadata header of the shared block according to the update instruction; Release the global mutex lock.
[0123] The data backup apparatus provided in this disclosure can execute the data backup method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.
[0124] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0125] The following is a detailed reference. Figure 9 This diagram illustrates a suitable structural schematic for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 901, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 902 or a program loaded from memory 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device. The processor 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0126] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 9 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.
[0127] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a memory 908, or installed from a ROM 902. When the computer program is executed by the processor 901, it performs the functions defined in the data backup method of embodiments of this application.
[0128] Figure 9 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0129] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the data backup method shown in the above embodiments is implemented.
[0130] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.
[0131] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.
Claims
1. A method for data backup, characterized in that, The method includes: In response to a shutdown command for the target virtual machine, obtain the target bitmap corresponding to the target virtual machine; The target bitmap is stored in a shared block within the target storage space allocated for the target virtual machine; the shared block is a pre-allocated data block used to record bitmaps of multiple virtual machines. In response to a startup command for the target virtual machine, the target bitmap is obtained from the target storage space of the shared block and imported. In response to a backup command for the target virtual machine, an incremental backup of the target virtual machine is performed based on the target bitmap.
2. The method according to claim 1, characterized in that, The step of obtaining the target bitmap corresponding to the target virtual machine includes: If the target virtual machine has been shut down and the management process corresponding to the target virtual machine has not been exited, the target bitmap corresponding to the target virtual machine is obtained using the management process. And / or, The importing of the target bitmap includes: If the management process corresponding to the target virtual machine has been started, but the target virtual machine has not been started, the target bitmap is imported using the management process.
3. The method according to claim 1, characterized in that, The method further includes: Determine the current state of the target bitmap stored in the shared block; If the current state of the target bitmap is a first state indicating that the target bitmap is valid, the operation of importing the target bitmap is performed.
4. The method according to claim 3, characterized in that, The method further includes: After storing the target bitmap in the target storage space allocated for the target virtual machine in the shared block, the target bitmap stored in the shared block is set to the first state; After obtaining the target bitmap from the target storage space of the shared block, the target bitmap stored in the shared block is set to a second state indicating that the target bitmap is invalid.
5. The method according to claim 1, characterized in that, The method further includes: In response to a startup command for the target virtual machine, the first disk summary information corresponding to the target virtual disk stored in the shared block is obtained; the target virtual disk is a virtual disk allocated for the target virtual machine. If the current second disk digest information of the target virtual disk is consistent with the first disk digest information, the operation of importing the target bitmap is performed.
6. The method according to claim 5, characterized in that, The method further includes: In response to a shutdown command for the target virtual machine, determine the current first disk summary information of the target virtual disk; The first disk summary information is stored in the target storage space allocated for the target virtual machine in the shared block.
7. The method according to claim 1, characterized in that, The metadata header of the shared block includes disk information of the virtual disks allocated to each virtual machine, including disk identifiers and storage addresses of the storage space allocated to the virtual machines. The step of storing the target bitmap into the target storage space allocated for the target virtual machine in the shared block includes: Determine the target storage address in the metadata header that corresponds to the disk identifier of the target virtual disk; the target virtual disk is a virtual disk allocated for the target virtual machine; store the target bitmap in the target storage space corresponding to the target storage address in the shared block; The step of obtaining the target bitmap from the target storage space of the shared block includes: Determine the target storage address in the metadata header that corresponds to the disk identifier of the target virtual disk; The target bitmap is obtained from the target storage space corresponding to the target storage address in the shared block.
8. A data backup device, characterized in that, The device includes: The acquisition module is used to acquire the target bitmap corresponding to the target virtual machine in response to a shutdown command for the target virtual machine; A storage module is used to store the target bitmap into a shared block within the target storage space allocated for the target virtual machine; the shared block is a pre-allocated data block used to record bitmaps of multiple virtual machines. A processing module is configured to, in response to a startup command for the target virtual machine, obtain the target bitmap from the target storage space of the shared block and import the target bitmap; A backup module is configured to perform incremental backups of the target virtual machine based on the target bitmap in response to a backup command for the target virtual machine.
9. An electronic device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the data backup method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the data backup method according to any one of claims 1 to 7.
11. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the data backup method according to any one of claims 1 to 7.