Real-time backup method and device of virtual machine, electronic equipment and storage medium

Abstract

The application discloses a real-time backup method and device of a virtual machine, electronic equipment and a storage medium, and relates to the technical field of computers. According to the application, a modification marking mechanism is deployed at the virtual machine manager level, a consistent reference data copy is created, and a periodically modified data block can be identified and synchronized without freezing the virtual machine disk IO, and meanwhile, the state information of the modification marking mechanism is persistently stored, the continuity of the synchronization operation after the marking failure is guaranteed, the real-time performance and continuity of the virtual machine backup are improved, the business continuity is guaranteed, the incremental synchronization integrity is improved, the reliability of the data protection of the virtualization environment is further strengthened, and the technical effect of meeting the requirements of the scene of the financial transaction, the medical system and the like for the business continuity is achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to real-time backup methods, apparatus, electronic devices and storage media for virtual machines. Background Technology

[0002] Real-time virtual machine backup technology, as a core support for data protection in cloud computing environments, is widely used in scenarios with stringent business continuity requirements, such as financial transactions and medical systems. Among related technologies, a backup system combining virtual machine snapshots and incremental synchronization is constructed through the collaborative operation of the QuickEmulator (QEMU) virtualization platform and kernel-based Virtual Machine (KVM) hardware acceleration.

[0003] In existing virtual machine backup methods, creating a consistent snapshot requires freezing the virtual machine disk input / output (IO). In traditional solutions, the pause time often lasts for several seconds, leading to business anomalies such as database connection timeouts and transaction interruptions. This results in a decrease in the integrity of incremental synchronization, and these defects collectively restrict the improvement of the reliability of data protection in virtualized environments. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for real-time backup of virtual machines, in order to at least solve the problem in related technologies where excessively long pause times lead to business anomalies such as database connection timeouts and transaction interruptions.

[0005] This application provides a real-time backup method for virtual machines, including: Deploy a modification tagging mechanism at the virtual machine manager level and create a consistent baseline data copy based on the virtual machine's disk; Based on a preset period, the modified data blocks in the disk are identified according to the modification marking mechanism, and the modified data blocks are synchronized to the backup storage pool; The state information of the modification marking mechanism is persistently stored, and after the modification marking mechanism fails, the operation of identifying the modified data blocks in the disk and synchronizing them to the backup storage pool is restored based on the persistent state information according to a preset period.

[0006] Optionally, the modification tagging mechanism is implemented through a bitmap, and the mapping granularity of the bitmap is adaptively adjusted based on the input and output characteristics of the virtual machine.

[0007] Optionally, the step of deploying a modification marking mechanism at the virtual machine manager level and creating a consistent baseline data copy based on the virtual machine's disk includes: The disk access request of the virtual machine is suspended to create a disk snapshot, and the virtual machine is resumed to run after the snapshot is created; wherein the duration of suspending the disk access request is less than a preset threshold.

[0008] Optionally, the duration of the preset period can be dynamically adjusted based on the current input / output load of the virtual machine.

[0009] Optionally, the step of persistently storing the state information of the modification marking mechanism, and, after the modification marking mechanism fails, restoring the operation of identifying modified data blocks in the disk and synchronizing them to the backup storage pool according to a preset period based on the persistent state information, includes: The status information is stored simultaneously in a first storage location on the local machine where the virtual machine is located and in a second storage location in the backup storage pool.

[0010] Optionally, the backup storage pool includes a front-end cache layer and a back-end main storage layer; the step of identifying modified data blocks in the disk according to the modification marking mechanism based on a preset period and synchronizing the modified data blocks to the backup storage pool includes: Write the modified data block into the front-end cache layer; Once the amount of data stored in the front-end cache layer reaches a preset capacity threshold or meets a preset time interval condition, the data in the front-end cache layer is then migrated in batches to the back-end main storage layer.

[0011] This application also provides a real-time backup device for virtual machines, comprising: Create a unit to deploy a modification marking mechanism at the virtual machine manager level and create a consistent baseline data copy based on the virtual machine's disk; The identification unit is used to identify the modified data blocks in the disk according to the modification marking mechanism based on a preset period, and synchronize the modified data blocks to the backup storage pool; The storage unit is used to persistently store the state information of the modification marking mechanism, and after the modification marking mechanism fails, based on the persistent state information, to restore the operation of identifying the modified data blocks in the disk and synchronizing them to the backup storage pool according to a preset period.

[0012] Optionally, the modification tagging mechanism is implemented through a bitmap, and the mapping granularity of the bitmap is adaptively adjusted based on the input and output characteristics of the virtual machine.

[0013] Optionally, the creation unit is further configured to: The disk access request of the virtual machine is suspended to create a disk snapshot, and the virtual machine is resumed to run after the snapshot is created; wherein the duration of suspending the disk access request is less than a preset threshold.

[0014] Optionally, the duration of the preset period can be dynamically adjusted based on the current input / output load of the virtual machine.

[0015] Optionally, the storage unit is further used for: The status information is stored simultaneously in a first storage location on the local machine where the virtual machine is located and in a second storage location in the backup storage pool.

[0016] Optionally, the backup storage pool includes a front-end cache layer and a back-end main storage layer; the identification unit is further used for: Write the modified data block into the front-end cache layer; Once the amount of data stored in the front-end cache layer reaches a preset capacity threshold or meets a preset time interval condition, the data in the front-end cache layer is then migrated in batches to the back-end main storage layer.

[0017] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described virtual machine real-time backup methods when executing the computer program.

[0018] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described virtual machine real-time backup methods.

[0019] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described virtual machine real-time backup methods.

[0020] This application addresses the technical problems of existing virtual machine backup methods. By deploying a modification marking mechanism and creating a consistent baseline data copy at the virtual machine manager level, modified data blocks can be identified and synchronized periodically without freezing virtual machine disk I / O. Furthermore, the state information of the modification marking mechanism is persistently stored, ensuring the continuity of synchronization operations after the marking expires. This solves the problems of long pause times caused by frozen disk I / O, leading to database connection timeouts, transaction interruptions, and decreased incremental synchronization integrity. The application achieves improved real-time performance and continuity of virtual machine backups, ensures uninterrupted business operations, enhances incremental synchronization integrity, and strengthens the reliability of data protection in virtualized environments. This meets the stringent business continuity requirements of scenarios such as financial transactions and medical systems. Attached Figure Description

[0021] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a real-time backup method for a virtual machine provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating an optimized bitmap tracking mechanism that combines full backup and real-time incremental synchronization, as provided in an embodiment of this application. Figure 3 This is a schematic diagram illustrating a real-time virtual machine backup method based on QEMU bitmap optimization provided in an embodiment of this application. Figure 4 This is a schematic diagram of the structure of a real-time backup device for a virtual machine provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0024] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0025] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] The embodiments of this application provide a real-time backup method for virtual machines. The method is described in detail below in conjunction with the execution flow of the real-time backup method for virtual machines. Figure 1 This is a flowchart illustrating a real-time backup method for a virtual machine provided in an embodiment of this application.

[0027] like Figure 1 As shown, the method includes the following steps: Step 101: Deploy a modification marking mechanism at the virtual machine manager level and create a consistent baseline data copy based on the virtual machine's disk; This is implemented by deploying a dedicated dynamic bitmap management module at the hypervisor layer. This module continuously tracks and marks changed areas of virtual machine disk data. The backup process is divided into an initialization phase and a real-time synchronization phase. In the initialization phase, the modification marking mechanism is first configured at the virtual machine manager level. A consistent snapshot of the virtual machine disk is created to obtain baseline data, which is then completely transferred to the backup storage system, thus establishing the initial data foundation for incremental recovery. The modification marking mechanism, for example, in a QEMU virtualization environment, is implemented as a dynamic bitmap module supporting block-level management, capable of recording disk write operations in units of data blocks. In the real-time synchronization phase, the system periodically reads the modification marks, synchronizing only data blocks that have changed since the previous cycle to the backup storage system, and updating the mark status after synchronization is complete, thereby achieving near real-time backup of virtual machine data.

[0028] By introducing a fine-grained change tracking and periodic incremental synchronization mechanism implemented at the virtualization layer, the impact of the backup process on the performance of the production system can be significantly reduced, the data transfer volume and storage overhead of the backup task can be greatly reduced, and the potential data loss can be controlled within a very short period of time, thereby effectively improving the efficiency, economy and reliability of virtual machine data protection.

[0029] Step 102: Based on a preset period, identify the modified data blocks in the disk according to the modification marking mechanism, and synchronize the modified data blocks to the backup storage pool; This is implemented by deploying a dedicated dynamic bitmap management module at the hypervisor layer. This module continuously tracks and marks changed areas of virtual machine disk data. The backup process is divided into an initialization phase and a real-time synchronization phase. In the initialization phase, the modification marking mechanism is first configured at the virtual machine manager level. Subsequently, a consistent snapshot of the virtual machine disk is created to obtain baseline data, which is then completely transferred to the backup storage system, thus establishing the initial data foundation for incremental recovery. The modification marking mechanism, for example, implemented as a block-granular management dynamic bitmap module in a QEMU virtualization environment, can record disk write operations in units of data blocks. In the real-time synchronization phase, the system operates periodically at configurable time intervals, identifying data blocks that have changed on the disk since the last synchronization based on the modification marking mechanism. Then, only the identified changed data blocks are incrementally synchronized to the backup storage system, and the corresponding marking status is updated after synchronization is complete. This achieves continuous, low-overhead capture and backup of virtual machine data changes, ensuring near real-time consistency between backup data and production data.

[0030] By introducing a fine-grained change tracking and periodic incremental synchronization mechanism implemented at the virtualization layer, the impact of the backup process on the performance of the production system can be significantly reduced, the data transfer volume and storage overhead of the backup task can be greatly reduced, and the potential data loss can be controlled within a very short synchronization cycle, thereby effectively improving the efficiency, economy and reliability of virtual machine data protection.

[0031] Step 103: Persistently store the state information of the modification marking mechanism, and after the modification marking mechanism fails, based on the persistent state information, resume the operation of identifying the modified data blocks in the disk and synchronizing them to the backup storage pool according to a preset period.

[0032] This is implemented by deploying a dedicated dynamic bitmap management module at the hypervisor layer. This module continuously tracks and marks changed areas of virtual machine disk data. The backup process is divided into an initialization phase and a real-time synchronization phase. In the initialization phase, the modification marking mechanism is first configured at the virtual machine manager level. Subsequently, a consistent snapshot of the virtual machine disk is created to obtain baseline data, which is then completely transferred to the backup storage system, thus establishing the initial data foundation for incremental recovery. The modification marking mechanism, for example, in a QEMU virtualization environment, is implemented as a dynamic bitmap module supporting block-level management, capable of recording disk write operations in units of data blocks. In the real-time synchronization phase, the system operates periodically at configurable time intervals, identifying data blocks that have changed on the disk since the last synchronization based on the modification marking mechanism. Then, only the identified changed data blocks are incrementally synchronized to the backup storage system, and the corresponding marking status is updated after synchronization is complete. This achieves continuous, low-overhead capture and backup of virtual machine data changes. To ensure the reliability of the synchronization process, the system also periodically saves the key state information of the modification marking mechanism to persistent storage. When the modification marking mechanism fails and restarts for any reason, its working state can be reconstructed based on the persistent state information, thereby seamlessly restoring the aforementioned periodic incremental data identification and synchronization operations.

[0033] By introducing fine-grained change tracking, periodic incremental synchronization, and a mechanism for persistent and restored marked state implemented at the virtualization layer, a complete real-time backup solution is formed. This significantly reduces the impact of the backup process on production system performance, drastically reduces data transfer volume and storage overhead for backup tasks, and keeps potential data loss within a very short synchronization cycle. Furthermore, the state guarantee mechanism effectively improves the continuity and robustness of the backup process, thereby enhancing the overall efficiency, economy, and reliability of virtual machine data protection.

[0034] In some embodiments, the modification tagging mechanism is implemented using a bitmap, the mapping granularity of which is adaptively adjusted based on the input and output characteristics of the virtual machine.

[0035] Each bit in the bitmap maps to and indicates whether a specific size data block on the virtual machine disk (i.e., the mapping granularity) has been modified since the last synchronization. To achieve efficient resource utilization and tracking performance, the bitmap mapping granularity is not fixed but adaptively adjusted based on the virtual machine's real-time input / output (IO) characteristics. Specifically, the system continuously monitors the patterns and data characteristics of disk IO requests issued by the virtual machine. When the monitoring module detects that IO requests are primarily small-sized and random accesses over a continuous period (e.g., 5 seconds) (this characteristic typically corresponds to scenarios such as frequent updates to database transaction logs), it is determined to be a "small file random write" mode. In this case, the system automatically switches the bitmap mapping granularity to a finer level, for example, adjusting each bit to correspond to a 4KB disk data block, thereby achieving precise tracking of subtle changes. Conversely, when IO requests are primarily large-sized and continuous accesses (this characteristic typically corresponds to scenarios such as video file writing), it is determined to be a "large file continuous write" mode, and the system automatically switches the bitmap mapping granularity to a coarser level, for example, adjusting each bit to correspond to a 64KB disk data block. This granularity switching operation is accomplished by calling the adjustment function within the dynamic bitmap management module. This function recalculates and allocates the memory structure of the bitmap while ensuring the correct conversion and inheritance of the original marked data. This significantly reduces the amount of data that the bitmap itself needs to maintain and the CPU overhead when iterating over it in high-throughput continuous write scenarios without interrupting the backup process.

[0036] By employing the dynamic bitmap with adaptively adjusted mapping granularity based on IO characteristics, this specific implementation can intelligently adapt to different workloads of virtual machines. In fine-grained mode, it ensures the accuracy of backup data, while in coarse-grained mode, it effectively reduces system resource consumption. Thus, while guaranteeing backup real-time performance and data consistency, it achieves an optimized balance between computing resources and storage overhead, improving the overall adaptability and efficiency of the backup system.

[0037] In some embodiments, deploying a modification tagging mechanism at the virtual machine manager level and creating a consistent baseline data copy based on the virtual machine's disk includes: The disk access request of the virtual machine is suspended to create a disk snapshot, and the virtual machine is resumed to run after the snapshot is created; wherein the duration of suspending the disk access request is less than a preset threshold.

[0038] In one specific embodiment of the present invention, the step of "deploying a modification marking mechanism at the virtual machine manager level and creating a consistent baseline data copy based on the virtual machine's disk" is implemented through the following specific operations. First, an instruction is issued to the virtual machine manager (e.g., QEMU) to trigger a disk input / output (IO) freeze operation for the target virtual machine, i.e., suspending disk access requests. This operation is implemented through the virtualization manager's internal control interface (e.g., the `suspend` command or equivalent API of QEMU Monitor). Its core is to temporarily suspend all new write requests initiated by the virtual machine to the virtual disk and cache them in a temporary memory log, while ensuring that ongoing disk operations are completed. During this brief window of disk IO freezing, the virtualization manager immediately invokes its snapshot creation function (e.g., through the `drive-snapshot` command) to generate a consistent snapshot file containing the current precise state of the disk. This snapshot is the prototype of the consistent baseline data copy. To ensure the continuity of services running on the virtual machine, the duration of the entire suspended disk access request is strictly designed and controlled within a very short preset threshold, such as 100 milliseconds. This time control is achieved by optimizing the snapshot creation process (such as using copy-on-write technology) and reducing unnecessary synchronization operations. Once the snapshot file is created, the system immediately unfreezes the virtual machine's disk I / O, resumes the virtual machine's normal operation, and simultaneously starts the modification marking mechanism (such as a bitmap module) in the background to associate its initial state with the newly created disk snapshot, preparing for subsequent incremental synchronization.

[0039] While ensuring the acquisition of a highly consistent baseline copy for backup, the downtime caused by backup initialization operations on virtual machine operations is reduced to an extremely low level (e.g., on the order of hundreds of milliseconds). This significantly reduces the impact on high-availability and high-real-time services, making this backup method applicable to scenarios with stringent business continuity requirements, such as financial transactions and online services.

[0040] In some embodiments, the duration of the preset period is dynamically adjusted based on the current input / output load of the virtual machine.

[0041] In one specific embodiment of the present invention, the duration of the "preset period" in the real-time incremental synchronization process is not a fixed value, but is dynamically adjusted according to the current input / output (I / O) load of the virtual machine. Specifically, the system includes a period adjustment module that continuously monitors the I / O load metrics of the virtual machine disk, such as I / O operations per second (IOPS), data transfer bandwidth, or the depth of the I / O request queue. The adjustment logic is based on a preset load threshold: when the current I / O load is consistently higher than a set high load threshold (e.g., I / OPS exceeding a certain value or bandwidth utilization exceeding 80%), it indicates that the disk is busy. In this case, to reduce the competitive impact of backup operations on production performance, the period adjustment module will actively extend the preset period, for example, adjusting the synchronization period from a base value to 1 second. Conversely, when the current I / O load is consistently lower than a set low load threshold, it indicates that the disk is relatively idle. To further improve the real-time performance of backups and reduce the potential data loss window, the module will shorten the preset period, for example, adjusting the synchronization period to 100 milliseconds. This dynamic adjustment is achieved through a feedback control loop, where monitoring, decision-making, and periodic parameter updates are performed automatically without administrator intervention, thus enabling the frequency of incremental data synchronization to adapt to the actual workload of the virtual machine.

[0042] By implementing the aforementioned mechanism of dynamically adjusting the synchronization cycle based on real-time I / O load, this specific solution can intelligently reduce the frequency of backup operations when the virtual machine load is high, effectively alleviating the potential pressure on production business performance; and increase the backup frequency when the load is low, maximizing the real-time performance of data protection. This flexible strategy achieves an adaptive balance between backup timeliness and business performance impact, improving the overall applicability and efficiency of the backup system in diverse operating environments.

[0043] In some embodiments, the operation of persistently storing the state information of the modification marking mechanism, and, after the modification marking mechanism fails, restoring the identification of modified data blocks in the disk and synchronizing them to the backup storage pool according to a preset period based on the persistent state information includes: The status information is stored simultaneously in a first storage location on the local machine where the virtual machine is located and in a second storage location in the backup storage pool.

[0044] In one specific embodiment of the present invention, the persistent storage of the state information of the modification marking mechanism for recovery is specifically implemented using a dual-copy storage mechanism. The state information, namely the disk data block modification state recorded by the dynamic bitmap, is periodically (e.g., every 100 milliseconds) captured and serialized by the persistence processing thread in the virtual machine manager (such as QEMU). Subsequently, this serialized state information is **simultaneously** written to two independent, physically isolated storage locations: the first storage location is located on the persistent storage device of the local physical host or compute node where the virtual machine resides, for example, in a file named after the virtual machine identifier under the specified local file system directory ` / var / lib / qemu / bitmaps / `; the second storage location is located within the remote backup storage pool, for example, written to a specific namespace or directory of the backup storage system via a storage network protocol, such as ` / backup / bitmaps / `. These two write operations constitute a transactional dual-write process, designed to ensure that the two copies remain as synchronized as possible at any given time. When the modification marking mechanism fails due to events such as virtual machine restarts or QEMU process crashes, during reinitialization, the recovery module first attempts to load the latest state information from the first storage location (local). If the loading is successful, the bitmap state is directly restored based on this, and incremental synchronization continues. If the copy in the first storage location is detected to be corrupted or lost, the recovery module will automatically switch to pulling a backup copy of the state information from the second storage location (backup storage pool) and use it to restore the bitmap, thereby ensuring that even if local storage fails, incremental data identification and synchronization operations can continue uninterrupted.

[0045] By implementing the aforementioned dual-copy persistent storage and intelligent recovery mechanism for state information, this specific solution significantly enhances the fault tolerance and business continuity of the backup system. It effectively prevents the risk of the entire incremental backup chain breaking due to a single point of failure (such as local disk damage), ensuring that real-time backup tasks can recover quickly and automatically in most unexpected scenarios, thus providing a higher level of reliability for virtual machine data.

[0046] In some embodiments, the backup storage pool includes a front-end cache layer and a back-end main storage layer; the step of identifying modified data blocks in the disk according to the modification marking mechanism based on a preset period and synchronizing the modified data blocks to the backup storage pool includes: Write the modified data block into the front-end cache layer; Once the amount of data stored in the front-end cache layer reaches a preset capacity threshold or meets a preset time interval condition, the data in the front-end cache layer is then migrated in batches to the back-end main storage layer.

[0047] In one specific embodiment of the invention, the backup storage pool is configured with a specific architecture comprising a two-tier storage system, namely, a front-end cache layer and a back-end main storage layer working together. Specifically, the front-end cache layer consists of an array of high-speed storage media (e.g., solid-state drives, SSDs) and its availability is ensured through redundancy mechanisms (e.g., RAID 5); the back-end main storage layer consists of a capacity-oriented distributed storage system (e.g., a Ceph-based storage cluster). During real-time incremental synchronization, the process of "synchronizing modified data blocks to the backup storage pool" is refined into two stages. First, the modified data blocks identified by the modification marking mechanism are directly and efficiently written to the front-end cache layer through a dedicated interface (e.g., `librbd`). This step utilizes the high IOPS characteristics of the cache layer to complete the process with extremely low latency, thereby avoiding bottlenecks to the synchronization process. The data written to the cache layer is not immediately and permanently stored, but temporarily resides there. Subsequently, a separate cache management process monitors the amount of data temporarily stored in the front-end cache layer. When the accumulated data reaches a preset capacity threshold (e.g., 1GB), or when a preset time interval condition is met (e.g., every 30 seconds), the management process triggers a batch migration operation. This process efficiently migrates the incremental data accumulated in the cache layer as a whole to the back-end main storage layer for persistent storage asynchronously (e.g., by the Ceph OSD service process). This migration process is performed in the background and does not block the continuous writing of new incremental data to the cache layer from the front end.

[0048] By implementing the aforementioned two-tier storage architecture and data processing flow, which includes front-end caching and back-end batch migration, this specific solution can significantly absorb and smooth the sudden write load generated by incremental backups, transforming a large number of small-scale random writes into sequential batch writes that are more friendly to back-end storage. This effectively reduces the real-time IO pressure on the backup storage pool, increases the overall write throughput, and reduces synchronization latency. At the same time, the redundancy design of the cache layer further improves the reliability of the backup data transfer process.

[0049] The following example illustrates a real-time backup method for virtual machines provided in this application.

[0050] Preliminary preparations 1. Deploy a dynamic bitmap module in the QEMU layer. Each bit in the bitmap corresponds to a data block on the original disk (the block size can be dynamically adjusted) and is used to mark whether the data block has been modified. 2. Configure a backup storage pool that supports the data transfer protocol of the QEMU-img toolchain and adopts a distributed storage architecture to improve write performance.

[0051] Please see Figure 2 , Figure 2This is a schematic diagram illustrating an optimized bitmap tracking mechanism combining full backup and real-time incremental synchronization, as provided in an embodiment of this application. Figure 2 As shown, it includes: Initial full backup 1. Perform a virtual machine pause operation, create a consistent snapshot of the original disk through QEMU's internal interface, enable the bitmap function and initialize the bitmap (set all bits to 0), and then resume the virtual machine operation; 2. Use the QEMU-img convert tool to transfer the above snapshots completely to the backup storage pool as the baseline data for incremental synchronization.

[0052] Real-time incremental synchronization process Step A: Iterate the bitmap periodically in the QEMU layer and determine the data blocks that need to be synchronized based on the bits set to 1 in the bitmap; Step B: Read the data blocks marked as modified from the original disk and write them to the backup storage pool via asynchronous I / O; Step C: After writing is complete, reset the corresponding bitmap bit to 0 to avoid duplicate synchronization; Step D: Repeat steps A through C. The iteration period can be dynamically adjusted according to the IO load (e.g., extended to 1 second under high load, shortened to 100 milliseconds under low load).

[0053] Optimization mechanism 1. Optimize virtual machines by pausing The strategy of "memory snapshot + disk snapshot separation" is adopted: when the virtual machine is paused, only disk I / O is frozen (lasting 50-100 milliseconds), and memory operations are cached through temporary logs; after the disk snapshot is created, the virtual machine is restored immediately, and the memory operations in the log are replayed asynchronously after restoration, compressing the business interruption time to less than 100 milliseconds.

[0054] 2. Dynamic adjustment of bitmap granularity Automatically switch block size based on IO characteristics: when small file random writes are detected (such as database transaction logs), enable 4KB fine-grained bitmap; when large file continuous writes are detected (such as video files), automatically switch to 64KB coarse-grained bitmap to reduce bitmap iterations and CPU overhead.

[0055] 3. Enhanced backup performance Deploy an SSD caching layer in front of the backup storage pool: Incremental synchronization data is first written to the SSD cache, and after accumulating to 1GB, it is written in batches to the backend distributed storage to reduce IOPS pressure; at the same time, the caching layer adopts a RAID5 redundancy mechanism to avoid backup data loss due to single point of failure.

[0056] 4. Bitmap reliability assurance To achieve dual-copy persistence of bitmaps: Every 100 milliseconds, the QEMU layer synchronizes the current state of the bitmap to the virtual machine's local disk ( / var / lib / qemu / bitmaps / ) and the backup storage pool ( / backup / bitmaps / ), forming dual copies; after the virtual machine restarts or QEMU crashes, the bitmap is loaded from the local disk first, and if the local copy is corrupted, it is pulled from the backup storage pool to ensure that incremental synchronization is uninterrupted.

[0057] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating a real-time virtual machine backup method based on QEMU bitmap optimization provided in an embodiment of this application, as shown below. Figure 3 As shown, it includes: (1) Bitmap module deployment Add a new module, dynamic_bitmap.c, to the QEMU source code to implement bitmap creation, setting, resetting, and dynamic granularity adjustment functions. After compilation, replace the original QEMU binary file and start the virtual machine using qemu-system-x86_64 -enable-bitmap -bitmap-size 64MB.

[0058] (2) Initial full backup execution Executing `virsh suspend vm1` pauses the virtual machine (actually freezes disk I / O for approximately 80 milliseconds). Create a disk snapshot using the QEMU monitor command: drive-snapshot -device virtio-blk0 -snapshot-file / tmp / initial_snap.qcow2 -create; The command `qemu-img convert / tmp / initial_snap.qcow2 rbd:backup-pool / vm1-base-p` transfers the snapshot to Ceph storage, taking approximately 15 minutes (2TB of data, average transfer speed 220MB / s). Execute `virsh resume vm1` to restore the virtual machine, and QEMU will automatically enable bitmap tracing.

[0059] (3) Real-time incremental synchronization operation The QEMU background process triggers a bitmap iteration every 500 milliseconds by calling qemu-io -c "read<block_addr> <size>"Read the modified blocks from the original disk and write them to the Ceph cache layer via the librbd interface; After the cache layer accumulates 1GB of data, it is asynchronously migrated to the backend storage by the Ceph OSD process. The migration process does not block incremental synchronization. Administrators can view the current bitmap status, including the number of modified blocks and the last synchronization time, by using qemu-monitor-command vm1 --cmd "query-bitmap".

[0060] (4) Fault recovery verification Simulate a virtual machine's original disk failure and perform the following recovery operations: Pull the latest full snapshot ( / backup / vm1-base) and incremental data ( / backup / increments / ) from the backup storage pool. Merge incremental data using qemu-img rebase -b / backup / vm1-base / backup / increments / latest.qcow2 to generate a complete disk image; Create a new virtual machine based on the merged image, and verify data integrity after startup: database transaction logs, application configuration files, etc. are complete and without loss, and recovery point objective (RPO) is controlled within 500 milliseconds.

[0061] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0062] Embodiments of this application also provide a real-time backup device for virtual machines, such as... Figure 4 As shown, it includes: Creation unit 21 is used to deploy a modification marking mechanism at the virtual machine manager level and create a consistent baseline data copy based on the virtual machine's disk; The identification unit 22 is used to identify the modified data blocks in the disk according to the modification marking mechanism based on a preset period, and synchronize the modified data blocks to the backup storage pool; Storage unit 23 is used to persistently store the state information of the modification marking mechanism, and after the modification marking mechanism fails, based on the persistent state information, to restore the operation of identifying the modified data blocks in the disk and synchronizing them to the backup storage pool according to a preset period.

[0063] Furthermore, in one possible implementation of this application embodiment, the modification tagging mechanism is implemented through a bitmap, and the mapping granularity of the bitmap is adaptively adjusted based on the input and output characteristics of the virtual machine.

[0064] Furthermore, in one possible implementation of this application embodiment, the creation unit 21 is further configured to: The disk access request of the virtual machine is suspended to create a disk snapshot, and the virtual machine is resumed to run after the snapshot is created; wherein the duration of suspending the disk access request is less than a preset threshold.

[0065] Furthermore, in one possible implementation of this application embodiment, the duration of the preset period is dynamically adjusted based on the current input / output load of the virtual machine.

[0066] Furthermore, in one possible implementation of this application embodiment, the storage unit 23 is further used for: The status information is stored simultaneously in a first storage location on the local machine where the virtual machine is located and in a second storage location in the backup storage pool.

[0067] Furthermore, in one possible implementation of this application embodiment, the backup storage pool includes a front-end cache layer and a back-end main storage layer; the identification unit 22 is further used for: Write the modified data block into the front-end cache layer; Once the amount of data stored in the front-end cache layer reaches a preset capacity threshold or meets a preset time interval condition, the data in the front-end cache layer is then migrated in batches to the back-end main storage layer.

[0068] For a description of the features in the embodiment corresponding to the virtual machine real-time backup device, please refer to the relevant description of the embodiment corresponding to the virtual machine real-time backup method, which will not be repeated here.

[0069] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described embodiments of the real-time backup method for virtual machines.

[0070] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described embodiments of the real-time backup method for virtual machines at runtime.

[0071] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0072] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described embodiments of the real-time backup method for virtual machines.

[0073] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described real-time backup method embodiments for virtual machines.

[0074] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0075] The above provides a detailed description of a real-time backup method, apparatus, electronic device, and storage medium for virtual machines provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.< / size>

Claims

1. A real-time backup method for virtual machines, characterized in that, include: Deploy a modification tagging mechanism at the virtual machine manager level and create a consistent baseline data copy based on the virtual machine's disk; Based on a preset period, the modified data blocks in the disk are identified according to the modification marking mechanism, and the modified data blocks are synchronized to the backup storage pool; The state information of the modification marking mechanism is persistently stored, and after the modification marking mechanism fails, the operation of identifying the modified data blocks in the disk and synchronizing them to the backup storage pool is restored based on the persistent state information according to a preset period.

2. The method according to claim 1, characterized in that, The modification tagging mechanism is implemented through a bitmap, and the mapping granularity of the bitmap is adaptively adjusted based on the input and output characteristics of the virtual machine.

3. The method according to claim 1, characterized in that, The deployment of a modification marking mechanism at the virtual machine manager level and the creation of a consistent baseline data copy based on the virtual machine's disk include: The disk access request of the virtual machine is suspended to create a disk snapshot, and the virtual machine is resumed to run after the snapshot is created; wherein the duration of suspending the disk access request is less than a preset threshold.

4. The method according to claim 1, characterized in that, The duration of the preset period is dynamically adjusted based on the current input / output load of the virtual machine.

5. The method according to claim 1, characterized in that, The operation of persistently storing the state information of the modification marking mechanism, and, after the modification marking mechanism fails, restoring the identification of modified data blocks in the disk and synchronizing them to the backup storage pool according to a preset period based on the persistent state information, includes: The status information is stored simultaneously in a first storage location on the local machine where the virtual machine is located and in a second storage location in the backup storage pool.

6. The method according to claim 1, characterized in that, The backup storage pool includes a front-end cache layer and a back-end main storage layer; the step of identifying modified data blocks in the disk according to the modification marking mechanism based on a preset period and synchronizing the modified data blocks to the backup storage pool includes: Write the modified data block into the front-end cache layer; Once the amount of data stored in the front-end cache layer reaches a preset capacity threshold or meets a preset time interval condition, the data in the front-end cache layer is then migrated in batches to the back-end main storage layer.

7. A real-time backup device for virtual machines, characterized in that, include: Create a unit to deploy a modification marking mechanism at the virtual machine manager level and create a consistent baseline data copy based on the virtual machine's disk; The identification unit is used to identify the modified data blocks in the disk according to the modification marking mechanism based on a preset period, and synchronize the modified data blocks to the backup storage pool; The storage unit is used to persistently store the state information of the modification marking mechanism, and after the modification marking mechanism fails, based on the persistent state information, to restore the operation of identifying the modified data blocks in the disk and synchronizing them to the backup storage pool according to a preset period.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the real-time backup method for a virtual machine as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the real-time backup method for a virtual machine as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the real-time backup method for the virtual machine as described in any one of claims 1 to 6.

Images