Disaster recovery management methods, devices, equipment, media, and software products for virtual machines.
By configuring a filter driver service in the virtual disk block device chain, full data synchronization and write request forwarding are achieved, and log entries are generated to record data changes. This solves the problems of high invasiveness and high recovery point targets in virtual machine disaster recovery management, and realizes low-invasiveness and high-reliability data recovery.
Patent Information
- Application Number
- CN202611450679.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-17
- Publication Date
- 2026-11-17
AI Technical Summary
Existing virtual machine disaster recovery management methods suffer from problems such as high invasiveness, high recovery point targets, high operation and maintenance costs, and imprecise data protection, making it difficult to simultaneously meet the data protection requirements of low invasiveness, extremely low recovery point targets, and virtual machine granularity.
By configuring a filter driver service in the block device chain of the virtual disk, full data synchronization and write request forwarding are achieved, and log entries are generated. The synchronization process does not require the installation of components inside the virtual machine. The filter driver service generates log entries to record data changes, and the disaster recovery end can restore data at any point in time based on the log entries.
It enables disaster recovery management without intruding into the virtual machine, reduces operation and maintenance costs, ensures low invasiveness and high reliability of data recovery, and can recover virtual disk data at any point in time, meeting the goal of extremely low recovery point.
Smart Images

Figure CN123114630A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, medium, and program product for disaster recovery management of virtual machines. Background Technology
[0002] With the development of cloud computing technology, virtual machines are widely used to support core business operations. After core business operations are migrated to the cloud, the data assets hosted on virtual machines place increasingly higher demands on disaster recovery capabilities. In practical applications, business systems typically require an RPO (Recovery Point Objective) close to zero, meaning that the amount of data loss allowed during a disaster is as small as possible, and the system must support data recovery to any historical point in time.
[0003] Currently, one approach to achieve continuous data protection is to install agent software within the operating system of a virtual machine. However, this method is highly intrusive, resulting in a strong binding between the agent software and the operating system kernel version. Operating system upgrades or security patch updates may cause driver loading failures or even business interruptions. Furthermore, the installation, configuration, and upgrade of the agent software require operating each virtual machine individually, leading to high deployment and maintenance costs. Summary of the Invention
[0004] Therefore, it is necessary to provide a method, apparatus, device, media, and program product for disaster recovery management of virtual machines to address the aforementioned technical problems.
[0005] Firstly, this application provides a disaster recovery management method for virtual machines, applied to a business end, wherein the business end and the disaster recovery end are communicatively connected, and a filter driver service is configured in the block device chain of the virtual disk corresponding to the virtual machine in the business end. The method includes: synchronizing all data in the virtual disk to the disaster recovery end through the filter driver service; forwarding write requests initiated by the virtual machine for the virtual disk downstream to perform write operations through the filter driver service, and generating log entries based on the metadata of the write requests, wherein the metadata includes the data payload in the write request and the storage location and data length of the data payload; sending the log entries to the disaster recovery end so that the disaster recovery end stores the log entries, and upon receiving a recovery request, performing data recovery on the virtual disk based on the metadata in the log entries and the fully synchronized data.
[0006] In this embodiment, by configuring the filter driver service in the block device chain of the virtual disk, full data synchronization, write request forwarding, and log entry generation are all completed at the block device chain level on the business side. This eliminates the need to install any components inside the virtual machine, reducing the intrusiveness of disaster recovery management to the virtual machine. Furthermore, full synchronization provides the disaster recovery end with complete basic data, and the generation and sending of log entries enables the disaster recovery end to obtain data change records, thereby achieving the ability to recover virtual disk data at any point in time.
[0007] In an optional implementation, the step of fully synchronizing the data in the virtual disk to the disaster recovery terminal through the filtering driver service includes: scanning data blocks marked as pending synchronization in the virtual disk according to a preset synchronization status record for the virtual disk through the filtering driver service; reading the data in the scanned data blocks and sending it to the disaster recovery terminal; and marking the scanned data blocks as synchronized in the synchronization status record; during the full synchronization process, if a synchronized data block is overwritten by a write request, the data block is remarked as pending synchronization in the synchronization status record; after completing the current round of scanning and sending, if the data blocks marked as pending synchronization in the synchronization status record do not reach a preset convergence condition, the process returns to the step of scanning data blocks marked as pending synchronization in the virtual disk according to the preset synchronization status record for the virtual disk through the filtering driver service until the preset convergence condition is reached, and the full synchronization is determined to be completed; wherein, the preset convergence condition includes the total number of data blocks pending synchronization being less than or equal to a preset threshold, or no new data blocks pending synchronization being added in two consecutive rounds.
[0008] In this embodiment, the synchronization status of each data block in the virtual disk is represented by a synchronization status record, and the data blocks that are overwritten by write requests are remarked as pending synchronization during the full synchronization process. This allows the full synchronization and the data writing of the virtual machine to be processed in parallel, so as not to interrupt the virtual machine's business. At the same time, it ensures that the data obtained by the disaster recovery end will not miss the data writing that occurred during the full synchronization, thus guaranteeing the integrity of the data during the full synchronization.
[0009] In an optional implementation, the log entry further includes a sequence number, which indicates the execution order of the write requests; sending the log entry to the disaster recovery terminal includes: storing the log entry in a preset circular buffer in ascending order of the sequence number; wherein, the smaller the sequence number, the earlier the execution order of the write request corresponding to the log entry; during the storage of log entries, the log entries in the circular buffer are sent to the disaster recovery terminal in ascending order of the sequence number, so that the disaster recovery terminal stores the log entry according to the sequence number, and upon receiving a recovery request, performs data recovery on the virtual disk according to the sequence number based on the metadata in the log entry and the fully synchronized data.
[0010] In this embodiment, by setting a sequence number for log entries to indicate the execution order of write requests, and by temporarily storing log entries in a circular buffer and sending log entries in sequence order, the generation and sending operations of log entries are decoupled. The disaster recovery terminal stores and replays log entries in sequence order, so that the disaster recovery terminal can restore data according to the time sequence of write operations, thereby improving the reliability of data recovery.
[0011] In an optional implementation, the method further includes: during full synchronization, generating log entries based on the metadata of write requests and storing them in the circular buffer, and sending the log entries in the circular buffer to the disaster recovery terminal; during full synchronization, if the occupancy rate of the circular buffer reaches a preset critical threshold, reducing the bandwidth for sending data to the disaster recovery terminal, or, if the occupancy rate of the circular buffer reaches a preset critical threshold, migrating the overflowing log entries in the circular buffer to the non-volatile memory in the service terminal, and after sending the log entries in the non-volatile memory to the disaster recovery terminal, sending the log entries in the circular buffer to the disaster recovery terminal.
[0012] In this embodiment, by reducing the transmission bandwidth or migrating the overflowing log entries to non-volatile memory when the occupancy rate of the circular buffer reaches a preset critical threshold, the circular buffer will not lose log entries due to insufficient transmission speed, thus ensuring the continuity of the log entry link. At the same time, log entries in non-volatile memory are sent first, followed by log entries in the circular buffer, so that the log entries maintain their original order after transmission is resumed.
[0013] In an optional implementation, the method further includes: when a disk flushing operation targeting the virtual disk is captured by the filtering driver service, determining the time point corresponding to the disk flushing operation; generating a barrier log entry based on the time point and a preset barrier marker; the barrier marker indicating that the data payload in each log entry generated before the time point has been stored in the non-volatile memory of the service terminal; sending the barrier log entry to the disaster recovery terminal, so that the disaster recovery terminal generates a recovery index record based on the time point in the barrier log entry, and upon receiving a recovery request, performs data recovery on the virtual disk based on the recovery index record, the fully synchronized data, and the metadata in each log entry generated before the time point.
[0014] In this embodiment, by generating barrier log entries at the disk flushing operation and establishing recovery index records at the disaster recovery end, the log stream of the disaster recovery end is divided into multiple time points with consistency guarantees, thereby enabling the recovery of virtual disk data at any time point and reducing RPO.
[0015] In an optional implementation, the method further includes: upon receiving a log breakage notification from the disaster recovery terminal, determining a data block in the virtual disk to be retransmitted; wherein the log breakage notification is generated by the disaster recovery terminal when the sequence numbers of received log entries are not consecutive; the data block to be retransmitted includes data blocks that were overwritten by write requests during the period from the last successful reception of log entries by the disaster recovery terminal to the current time; reading data from the data block to be retransmitted and sending it to the disaster recovery terminal, so that the disaster recovery terminal updates the fully synchronized data based on the data in the data block to be retransmitted.
[0016] In this embodiment, by determining the data to be retransmitted upon receiving a log breakage notification and sending the data to be retransmitted to the disaster recovery end, the data consistency between the business end and the disaster recovery end can be restored after the log entry link is broken. At the same time, the amount of retransmitted data depends on the amount of data actually written during the log breakage, rather than on the total capacity of the virtual disk, thus shortening the time required for data consistency recovery.
[0017] Secondly, this application also provides a disaster recovery device for virtual machines, applied to a business end, wherein the business end is communicatively connected to a disaster recovery end, and a filter driver service is configured in the block device chain of the virtual disk corresponding to the virtual machine in the business end. The device includes: a synchronization module, used to fully synchronize the data in the virtual disk to the disaster recovery end through the filter driver service; a generation module, used to forward the write request initiated by the virtual machine for the virtual disk downstream to perform a write operation through the filter driver service, and generate a log entry based on the metadata of the write request, wherein the metadata includes the data payload in the write request and the storage location and data length of the data payload; and a recovery module, used to send the log entry to the disaster recovery end so that the disaster recovery end stores the log entry, and upon receiving a recovery request, perform data recovery on the virtual disk based on the metadata in the log entry and the fully synchronized data.
[0018] Thirdly, this application provides a computer device including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method described in any of the foregoing embodiments.
[0019] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing embodiments.
[0020] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing embodiments.
[0021] This application provides a virtual machine disaster recovery management method, apparatus, device, media, and program product. The method includes: synchronizing all data in a virtual disk to a disaster recovery endpoint via a filter driver service; forwarding write requests initiated by the virtual machine for the virtual disk downstream to perform write operations via the filter driver service, and generating log entries based on the metadata of the write requests, including the data payload in the write request, its storage location, and data length; sending the log entries to the disaster recovery endpoint so that the endpoint stores the log entries, and upon receiving a recovery request, performing data recovery on the virtual disk based on the metadata in the log entries and the fully synchronized data. By configuring the filter driver service in the block device chain of the virtual disk, the full data synchronization, write request forwarding, and log entry generation are all completed at the block device chain level on the business side, thus eliminating the need to install any components inside the virtual machine and reducing the intrusiveness of disaster recovery management to the virtual machine. Furthermore, by providing complete basic data to the disaster recovery endpoint through full synchronization, and then by generating and sending log entries, the disaster recovery endpoint obtains data change records, thereby enabling the recovery of virtual disk data at any point in time. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This illustration shows an application scenario provided by an embodiment of this application.
[0024] Figure 2 An example diagram of an application scenario provided by an embodiment of this application is shown;
[0025] Figure 3 A flowchart illustrating a disaster recovery management method for virtual machines provided in an embodiment of this application is shown.
[0026] Figure 4 A flowchart illustrating a disaster recovery management method for a virtual machine provided in an embodiment of this application is shown.
[0027] Figure 5 This paper illustrates a functional block diagram of a virtual machine disaster recovery management device provided in an embodiment of this application.
[0028] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation
[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0030] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.
[0031] In related technologies, there are three methods for disaster recovery management of virtual machines: The first method is to install agent software within the operating system of the virtual machine. This method requires installing an agent program within the operating system of each protected virtual machine and loading a filter driver in kernel mode to intercept read and write requests at the file system layer or block device layer. This method has the following problems: The filter driver is strongly bound to the kernel version of the operating system. Regular security patch upgrades to the operating system may cause changes in the kernel interface, leading to the failure of the filter driver to load, which in turn causes kernel crashes or system blue screens, directly causing business interruption; The filter driver runs on the critical read and write paths of the operating system kernel. Each write operation requires additional memory copying and metadata encapsulation, which will significantly increase the processor overhead of the operating system in high read and write frequency scenarios, and may increase the tail latency of read and write requests due to kernel lock contention; In addition, in cloud computing environments with thousands or even tens of thousands of virtual machines, it is necessary to install, configure and upgrade the agent program on each virtual machine, facing complex operation and maintenance problems such as version fragmentation management, gray-scale upgrades and anomaly troubleshooting. The agent program may also have kernel-level interaction conflicts with the security protection software inside the virtual machine.
[0032] The second method is to create snapshots of the virtual disk periodically. This method creates snapshots of the virtual disk at preset time intervals, but it has the following problems: The snapshot time interval is usually on the order of minutes or even hours. Data changes that occur between two adjacent snapshots will be completely lost in the event of a disaster, which cannot meet the target requirements of second-level or even sub-second-level recovery points; In addition, the creation of a snapshot requires a brief pause in the execution of the virtual machine's processor to ensure the consistency of memory and disk states. For virtual machines with large memory capacity, this pause time can reach hundreds of milliseconds or even seconds, which will cause significant lag and timeouts at the business level; Furthermore, snapshot files based on the copy-on-write chain will continue to grow over time, and the merging operation is extremely time-consuming when the snapshot chain is too long, which will further increase the storage burden.
[0033] The third method is data replication based on the underlying storage layer. This method replicates data in blocks at the distributed storage backend, but it has the following problems: Storage layer replication is at the storage pool or logical volume level, and cannot perceive the logical boundaries of individual virtual machines. When a storage pool hosts multiple virtual machines, it is impossible to configure independent continuous data protection policies, rollback, or recovery for individual virtual machines. Storage layer replication only guarantees eventual consistency of block-level data and is not aware of the write order at the virtual machine level. When a single logical write operation of a virtual machine involves multiple non-contiguous storage blocks, it may lead to inconsistent data on the target end. In addition, storage layer replication is independent of the virtualization management platform and cannot be linked with the lifecycle management of virtual machines, increasing the sense of disconnect in operation and maintenance.
[0034] It can be seen that the above three methods are all difficult to simultaneously meet the requirements of low invasiveness to virtual machines, extremely low recovery point targets, and data protection at the virtual machine granularity. Therefore, this application provides a disaster recovery management method for virtual machines to solve the above problems.
[0035] The virtual machine disaster recovery management method provided in this application embodiment can be applied to, for example, Figure 1 The application environment shown depicts a network connection between the service provider and the disaster recovery server. The service provider hosts and runs virtual machines (KVMs), and can host one or more KVMs. For example, the service provider can be a host machine, a physical server, a server cluster, or a cloud server providing cloud computing services. The disaster recovery server receives and stores data and provides data recovery capabilities. For example, the disaster recovery server can be a disaster recovery gateway or a backup server. This is understandable. Figure 1 The application environment shown is for illustrative purposes only. In actual applications, a disaster recovery terminal can communicate and connect with multiple business terminals simultaneously to perform centralized disaster recovery management of virtual machines in each business terminal.
[0036] For ease of understanding, this application provides an example diagram of an application environment. Please refer to [link / reference]. Figure 2 This includes the business end (i.e., the production end or source end) and the disaster recovery end (i.e., the target end), and the business end and the disaster recovery end establish a network channel through the network layer such as RDMA (Remote Direct Memory Access) or TCP (Transmission Control Protocol).
[0037] The business application is deployed on the host machine, running one or more KVM virtual machines. Each KVM virtual machine includes a Guest Operating System (such as Linux 5.4), and transport application layer services such as Nginx (a reverse proxy server) and MySQL (MySQL Structured Query Language). The host machine also has a hardware emulator, QEMU (QuickEmulator), which internally constructs a block device chain. This chain, from upstream to downstream, includes virtio-blk (semi-virtualization driver node), throttle (rate limiting node), cdp-filter (filter driver service node), gcow2 (image loading driver node), and Host Storage (host storage node). The host machine also has a lock-free buffer configured with a Start / Stop & Policy Configure strategy, and asynchronous worker threads.
[0038] The disaster recovery endpoint includes a cloud platform management interface, which contains a custom CDP API (Continuous Data Protection Application Programming Interface). It also includes a remote CDP (Continuous Data Protection) disaster recovery gateway, which comprises a log storage engine (Log-Structured Storage), a point-in-time index module (PITIndex Module), and an image synthesis and recovery module.
[0039] The following section will use the business side as the execution subject to describe the various steps and corresponding technical effects of a virtual machine disaster recovery management method provided in this application embodiment. For ease of understanding, this application embodiment uses a virtual machine in the business side and a virtual disk corresponding to that virtual machine as an example for illustration. Please refer to... Figure 3 This is a flowchart illustrating a disaster recovery management method for virtual machines provided in an embodiment of this application, including steps 210 to 230.
[0040] Step 210: Through the filter driver service, synchronize all data in the virtual disk to the disaster recovery terminal.
[0041] In the application side, the block device chain of the virtual disk corresponding to the virtual machine is configured with a filter driver service. A hardware emulator is installed on the application side, capable of full system simulation or process-level simulation. That is, the hardware emulator can provide virtual hardware resources such as virtual disks and virtual processors for the virtual machine. The virtual disk is a storage medium mapped as a block device by the hardware emulator, and read / write operations initiated by the virtual machine are executed on the physical storage medium of the application side via the block device.
[0042] A block device chain refers to an input / output processing path constructed by a hardware emulator, consisting of multiple nodes connected sequentially. Virtual machine I / O (input / output) requests are passed along this path from upstream to downstream, allowing each node to process the I / O requests. It can be understood that I / O requests include write I / O requests and read I / O requests; for simplicity, write I / O requests will be simply referred to as write requests below. The filter driver service, or CDP Filter, is a functional component configured in the block device chain of the virtual disk, used to process write requests flowing through that location. The filter driver service can be positioned between the paravirtualization driver node (virtio-blk or virtio-scsi) and the image loading driver node (qcow2) in the block device chain.
[0043] Full synchronization refers to the process of completely reading and sending the existing data in the virtual disk to the disaster recovery site. Its purpose is to enable the disaster recovery site to obtain a complete copy of the data from the virtual disk as the basis for subsequent data recovery.
[0044] In some embodiments, during the startup of a virtual machine or its operation via hot-plugging, the business terminal can customize command-line parameters through a hardware emulator to insert a custom bypass filter driver service into the block device chain of the virtual disk corresponding to the virtual machine. This filter driver service includes a first callback function for intercepting write requests and a second callback function for intercepting disk flushing operations.
[0045] In some embodiments, the application can filter the driver service, read data from the virtual disk block by block according to the address order of the virtual disk, and send the read data to the disaster recovery site. It is understood that full synchronization can be performed while the virtual machine continues to run the application; that is, while reading data from the virtual disk, the filter driver service continues to process write requests initiated by the virtual machine. Furthermore, the purpose of full synchronization is to send the existing data in the virtual disk completely to the disaster recovery site, thereby providing the disaster recovery site with a complete copy of the data from the virtual disk as the basis for subsequent data recovery.
[0046] Step 220: Through the filter driver service, write requests initiated by the virtual machine for the virtual disk are forwarded downstream to perform write operations, and log entries are generated based on the metadata of the write requests.
[0047] The metadata includes the data payload in the write request, as well as the storage location and length of the data payload. The data payload refers to the actual data content written in this write operation; the storage location of the data payload refers to the starting position of this write operation on the virtual disk, such as the logical block address offset (LBA Offset); and the data length refers to the size of the data in this write operation, which can be expressed in bytes. Log entries can be understood as records of data changes made to the write request.
[0048] In some embodiments, the application within the virtual machine on the business side initiates a write request to the virtual disk. This write request passes through the file system and block device driver layer in the guest OS of the virtual machine, and is then transmitted to the filter driver service via the semi-virtualized driver node, virtio-blk. The business side then calls the first callback function in the filter driver service to pass through the write request while simultaneously copying the metadata of the write request in a bypass manner and generating log entries.
[0049] It's understandable that the process of passing a write request through is as follows: the write request is forwarded downstream, that is, forwarded to the next node in the block device chain that filters the driver service, such as the image loading driver node qcow2. This means that the content of the write request is not modified during the process of passing it through, thus ensuring the normal execution of virtual machine business writes and the correctness of the write results, thereby reducing the impact on virtual machine read / write performance.
[0050] The process of generating log entries is as follows: the filter-driven service extracts metadata from write requests, namely the data payload, the storage location of the data payload, and the data length of the data payload, and generates log entries containing the metadata. It is understandable that, since each write request has an execution order, information indicating the execution order of write requests can be appended to the log entries so that the log entries can reflect the execution order of the corresponding write requests.
[0051] Step 230: Send the log entries to the disaster recovery terminal so that the disaster recovery terminal can store the log entries and, upon receiving a recovery request, perform data recovery on the virtual disk based on the metadata in the log entries and the fully synchronized data.
[0052] A recovery request refers to a request to restore the data on the virtual disk to a specific point in time. Recovery requests can be initiated by users through the management interface of the disaster recovery terminal, or triggered by the business terminal or the disaster recovery terminal according to a preset recovery strategy.
[0053] In some embodiments, the service terminal sends log entries to the disaster recovery terminal. The disaster recovery terminal persistently stores the received log entries in its local storage medium. Furthermore, upon receiving a recovery request, the disaster recovery terminal can use the data obtained from full synchronization as the base data, and write the data payload of the log entries into the corresponding storage location in the base data according to the metadata in the log entries, thereby achieving data recovery from the virtual disk.
[0054] In this embodiment, by configuring the filter driver service in the block device chain of the virtual disk, full data synchronization, write request forwarding, and log entry generation are all completed at the block device chain level on the business side. This eliminates the need to install any components inside the virtual machine, reducing the intrusiveness of disaster recovery management. Furthermore, by providing complete basic data to the disaster recovery end through full synchronization, and by generating and sending log entries, the disaster recovery end obtains data change records, thereby enabling the recovery of virtual disk data at any point in time.
[0055] In one exemplary embodiment, the present application provides a possible implementation of step 210, namely, step 210 includes steps 211 to 214.
[0056] Step 211: Using the filter driver service, scan the data blocks marked as to be synchronized in the virtual disk according to the preset synchronization status records for the virtual disk.
[0057] The virtual disk can be divided into multiple data blocks according to a preset block granularity, such as 64KB or 1MB. Understandably, the block granularity can be configured according to actual needs; for example, it can be configured to a smaller granularity to improve the fineness of transmission, or it can be configured to a larger granularity to reduce the overhead of marking and transmission.
[0058] Synchronization status records are used to document the synchronization status of each data block in the virtual disk. Synchronization status includes "synchronized" and "pending synchronization." For example, the synchronization status record can use a bitmap data structure, where one bit in the synchronization bitmap represents the synchronization status of a data block in the virtual disk. For instance, a bit value of 1 indicates the data block is pending synchronization, while a bit value of 0 indicates the data block is synchronized.
[0059] In some embodiments, the business side can set a state machine for the filter driver service. During the configuration of relevant information of the filter driver service, the filter driver service is controlled to be in the INIT state (initial state); after the configuration is completed, the filter driver service is controlled to be in the FULL_SYNC state (full synchronization state); after the full synchronization is completed, the filter driver service is controlled to be in the STEADY_STATE state (stable running state).
[0060] In some embodiments, a dedicated full synchronization coroutine can be pre-configured on the business side for full data synchronization. After the filter driver service enters the full synchronization state, it can create a bitmap in the business side's memory that matches the capacity of the virtual disk, thus obtaining the synchronization status record. Subsequently, the full synchronization coroutine can scan the data blocks marked as pending synchronization in the virtual disk according to the synchronization status record. For example, if all data blocks in the synchronization status record are marked as pending synchronization during the first round of scanning, the business side can scan each data block in the virtual disk block by block in order from low address to high address.
[0061] Understandably, to avoid excessive storage I / O bandwidth consumption during full synchronization, the full synchronization coroutine can be configured as a coroutine that supports I / O bandwidth rate limiting and I / O priority.
[0062] Step 212: Read the data in the scanned data block and send it to the disaster recovery terminal, and mark the scanned data block as synchronized in the synchronization status record.
[0063] In some embodiments, during the initial scan, the service terminal can scan and read each data block in the virtual disk block by block, from low address to high address. After reading each data block, the data in that block is sent to the disaster recovery terminal via a pre-established asynchronous transmission channel, and the synchronization status of that data block is updated to "synchronized" in the synchronization status record. For example, the value of the corresponding bit in the synchronization bitmap is modified to the second value, i.e., 0.
[0064] Step 213: During the full synchronization process, if a synchronized data block is overwritten by a write request, the data block is remarked as pending synchronization in the synchronization status record.
[0065] Understandably, during full synchronization, the virtual machine does not need to interrupt business operations, so it can normally initiate write requests to the virtual disk. However, this can lead to a situation where, after data in a data block has already been sent to the disaster recovery center, a write request initiated by the virtual machine covers that data block. This results in the sent data for that data block not being the latest data. Therefore, in this embodiment, during full synchronization, the business end will re-mark the synchronized data block covered by the write request as pending synchronization in the synchronization status record. This ensures that the data block is reread and sent in subsequent scans, allowing the disaster recovery center to receive the latest data.
[0066] In some embodiments, during the process of calling the filter driver service to process write requests, the business side can determine the data block to be overwritten based on the storage location and data length in the write request, and update the synchronization status of that data block in the synchronization status record to "pending synchronization". It is understood that during full synchronization, write requests initiated by the virtual machine will also be intercepted by the filter driver service, generating corresponding log entries and sending them to the disaster recovery end.
[0067] Step 214: After completing the current round of scanning and sending, if the data blocks marked in the synchronization status record that are to be synchronized have not reached the preset convergence condition, then return to execute steps 211 to 213 until the preset convergence condition is reached, and confirm that the full synchronization is completed.
[0068] The preset convergence condition is used to determine whether full synchronization is complete. It may include the total number of data blocks to be synchronized being less than or equal to a preset threshold, or no new data blocks to be synchronized being added in two consecutive rounds.
[0069] In some embodiments, after the service terminal completes the current round of scanning and sending, it counts the total number of data blocks marked as pending synchronization in the synchronization status record. If this total number is greater than a preset threshold, it indicates that there are still a large number of data blocks pending synchronization. In this case, the service terminal will reread the data in the data blocks marked as pending synchronization according to the synchronization status record and send it to the disaster recovery terminal, as well as update the synchronization status of the data blocks. If this total number is less than or equal to the preset threshold, it indicates that the number of remaining data blocks pending synchronization is very small. In this case, the remaining data blocks pending synchronization can be sent to the disaster recovery terminal together, and the full synchronization can be confirmed as complete.
[0070] In some other embodiments, after the application completes the current round of scanning and sending, it compares the total number of data blocks to be synchronized in the previous round with the total number of data blocks to be synchronized in the current round. If the total number in the current round is greater than the total number in the previous round, it indicates that new data blocks to be synchronized have been added during the two consecutive rounds of scanning, meaning that the full synchronization of data and the write operations of the virtual machine have not yet reached a balance. In this case, the application will reread the data in the data blocks marked as to be synchronized and send it to the disaster recovery end, and update the synchronization status of the data blocks, based on the synchronization status record. If the total number in the current round is less than or equal to the total number in the previous round, it indicates that no new data blocks to be synchronized have been added during the two consecutive rounds of scanning, meaning that the full synchronization of data and the write operations of the virtual machine have reached a balance, and then it is determined that the full synchronization is complete.
[0071] This can be understood as follows: by scanning, reading and sending data in data blocks in multiple rounds, the number of data blocks to be synchronized in each round will gradually decrease and tend to converge. When the preset convergence condition is reached, the full synchronization will be terminated.
[0072] In some embodiments, after confirming that full synchronization is complete, the application can invoke the filter driver service to send a base image completion marker to the disaster recovery server to indicate that full synchronization is complete, and send the sequence number corresponding to the current full data. Upon receiving the base image completion marker, the disaster recovery server persists the received full data, i.e., persists it as a complete base disk image. Then, the application controls the filter driver service to switch from the full synchronization state to a stable running state, so that the filter driver service only generates and sends log entries based on write requests, i.e., only performs incremental backups of the data in the virtual disk.
[0073] In this embodiment, the synchronization status of each data block in the virtual disk is represented by a synchronization status record, and the data blocks that are overwritten by write requests are remarked as pending synchronization during the full synchronization process. This allows the full synchronization and the data writing of the virtual machine to be processed in parallel, so as not to interrupt the virtual machine's business. At the same time, it ensures that the data obtained by the disaster recovery end will not miss the data writing that occurred during the full synchronization, thus guaranteeing the integrity of the data during the full synchronization.
[0074] In an exemplary embodiment, step 230 may include the following steps: storing log entries in a preset circular buffer in ascending order of sequence number; during the storage of log entries, sending the log entries in the circular buffer to the disaster recovery terminal in ascending order of sequence number, so that the disaster recovery terminal stores log entries according to sequence number, and upon receiving a recovery request, performs data recovery on the virtual disk according to the sequence number based on the metadata in the log entries and the fully synchronized data.
[0075] The log entries also include sequence numbers, which indicate the execution order of the write requests corresponding to the log entry. The sequence numbers can increment sequentially according to the order in which the filter driver service intercepts write requests; a smaller sequence number indicates an earlier execution order. For example, the filter driver service can maintain an N-bit (e.g., 64-bit) atomic counter. Each time the filter driver service intercepts a write request, the counter increments by 1, and the current value of the counter is determined as the sequence number of the write request, which is also the sequence number of the log entry corresponding to that write request. In other words, the sequence number reflects the sequential relationship between log entries. It can be understood that the sequence number is globally unique and monotonically increasing within a single virtual disk dimension; therefore, the sequence number can be understood as a global epoch / sequence number.
[0076] A circular buffer refers to a pre-allocated storage area in the application's memory for temporarily storing log entries. For example, a lock-free circular buffer can be used. A lock-free circular buffer can employ either an SPSC (Single Producer Single Consumer) model or an MPSC (Multi Producer Single Consumer) model. Here, the producer can be a coroutine in the hardware emulator used to handle read and write requests, and the consumer can be a background asynchronous transmission worker thread. A lock-free circular buffer can use a lock-free enqueue method based on atomic operations, allowing log entries to be written without locking, thus avoiding the thread context switching and priority inversion overhead caused by locking and unlocking.
[0077] Furthermore, the lock-free circular buffer can allocate a fixed-size contiguous memory region during initialization. This region's size can be configured according to actual needs, for example, from 64MB to 256MB. When the occupied space in the lock-free circular buffer approaches its full capacity, a backpressure mechanism can be triggered to notify the log entry generator to wait or expand the storage area, without discarding log entries that have not yet been acknowledged by the disaster recovery end. Moreover, the write cursor indicating the producer's write position and the read cursor indicating the consumer's read position in the lock-free circular buffer can be aligned to independent processor cache lines, such as 64-byte cache lines, respectively. This avoids false sharing caused by two cursors residing in the same cache line and eliminates the overhead of maintaining cache consistency.
[0078] Understandably, a lock-free circular buffer can decouple data copying operations from network transmission operations on the read / write request interception path. The read / write request interception process only needs to complete one memory copy to enqueue the log entry, which usually takes microseconds. Meanwhile, the time-consuming network transmission is completed by a background asynchronous transmission worker thread, so that network transmission latency fluctuations will not be transmitted to the virtual machine's business read / write request processing process.
[0079] In some embodiments, after generating log entries, the service end stores the log entries sequentially into a circular buffer according to their sequence numbers in ascending order. Furthermore, during the storage of log entries, the service end can also send the log entries in the circular buffer to the disaster recovery end sequentially, also in ascending order of sequence numbers. For example, the service end can batch aggregate log entries according to a preset data volume threshold or a preset time window before sending them to reduce the number of network interactions. It is understood that the generation and sending of log entries operate on two parallel processing lines, thus decoupling the log entry generation and sending operations.
[0080] In some embodiments, a background asynchronous transmission worker thread can sequentially retrieve log entries from the consumer end of the circular buffer and transmit them to the disaster recovery end via the network. To reduce the processor resources consumed by data transmission, a zero-copy method can be used. For example, in a network environment supporting RDMA, the worker thread can pre-register the memory region where the circular buffer is located as an RDMA Memory Region (Remote Direct Memory Access Memory Region), and directly transmit the data in the circular buffer to the receiving buffer of the disaster recovery end through RDMA Write operations, thereby bypassing the network protocol stack of the service end and reducing the number of data copies performed by the processor. For example, in a TCP-based network environment, the zero-copy transmission capability of operating systems such as Linux can be utilized to reduce the number of data copies between user space and kernel space.
[0081] In some embodiments, worker threads can send log entries in a batch aggregation manner. That is, instead of sending log entries one by one, a batch sending is triggered when a preset time window is reached, such as every 5 milliseconds, or when the accumulated data volume reaches a preset threshold, such as every 4MB, in order to reduce the overhead of network RTT (Round-Trip Time) and protocol header. Furthermore, worker threads can dynamically adjust the amount of data sent in each batch and the sending frequency based on the real-time utilization of network bandwidth and the level of the circular buffer.
[0082] In some embodiments, after receiving log entries, the disaster recovery endpoint can append them to a high-performance log storage medium, such as a dedicated log volume composed of NVMe (Non-Volatile Memory express) and SSD (Solid State Drive), in sequence according to their sequence numbers. By adopting a log-structured storage model, sequential write performance can be fully utilized. Furthermore, when recovering data, the disaster recovery endpoint can replay each log entry in ascending order of its sequence number, thereby ensuring that the recovered data is consistent with the data write order of the virtual machine at the corresponding point in time.
[0083] In some embodiments, after the disaster recovery terminal persistently stores the received log entries, it can generate an acknowledgment message based on the largest persistent sequence number and send it to the service terminal. The service terminal releases the buffer space of the log entries matching the sequence number in the circular buffer according to the sequence number in the received acknowledgment message. Furthermore, if the service terminal does not receive acknowledgment messages for these log entries within a preset time after sending them in batches, a retransmission mechanism is triggered, i.e., starting from the sequence number that the disaster recovery terminal has not acknowledged, the service terminal retransmits the sequence number and all subsequent log entries to the disaster recovery terminal.
[0084] In this embodiment, by setting a sequence number for log entries to indicate the execution order of write requests, and by temporarily storing log entries in a circular buffer and sending log entries in sequence order, the generation and sending operations of log entries are decoupled. The disaster recovery terminal stores and replays log entries in sequence order, so that the disaster recovery terminal can restore data according to the time sequence of write operations, thereby improving the reliability of data recovery.
[0085] In an exemplary embodiment, the following steps may be included after step 230: during the full synchronization process, generating log entries based on the metadata of the write request and storing them in a circular buffer, and sending the log entries in the circular buffer to the disaster recovery terminal; during the full synchronization process, reducing the bandwidth for sending data to the disaster recovery terminal when the occupancy rate of the circular buffer reaches a preset critical threshold.
[0086] The occupancy rate of the circular buffer is the ratio of the occupied space to the total space in the circular buffer, reflecting the difference between the log entry generation rate and the transmission rate. The preset critical threshold is the occupancy rate threshold that triggers the protection policy, and it can be set according to actual needs.
[0087] In some embodiments, during full synchronization, the filter driver service also forwards write requests initiated by virtual machines, generates log entries based on the metadata of the write requests, stores the log entries in a circular buffer according to the sequence number, and sends the log entries in the circular buffer to the disaster recovery end.
[0088] Understandably, during full synchronization, the application sends data and log entries to the disaster recovery server. If network outages or disaster recovery server failures occur, the ring buffer's occupancy rate may reach a preset critical threshold, meaning the log entry sending rate is much lower than the generation rate. Therefore, to avoid log entry overflow, the application can reduce the bandwidth used to send data to the disaster recovery server, thereby reducing the network bandwidth consumed by full synchronization and allocating more bandwidth to log entry sending, thus accelerating the release of space in the ring buffer.
[0089] In an exemplary embodiment, after step 230, the following steps may be included: when the occupancy rate of the ring buffer reaches a preset critical threshold, the log entries overflowing in the ring buffer are migrated to the non-volatile memory in the service terminal, and after the log entries in the non-volatile memory are sent to the disaster recovery terminal, the log entries in the ring buffer are sent to the disaster recovery terminal.
[0090] Overflowing log entries can be understood as log entries that have not yet been sent to the disaster recovery site when the remaining space in the circular buffer is insufficient to accommodate newly generated log entries. Non-volatile memory refers to storage media whose data will not be lost after a power outage, such as hard disks, solid-state drives, or flash memory.
[0091] In some embodiments, when the occupancy rate of the circular buffer reaches a preset critical threshold, the service can invoke the filter driver service to append log entries to a log file on the service's local disk, ensuring the order of log entries. After network recovery or the occupancy rate of the circular buffer decreases, log entries in this log file are read first and sent to the disaster recovery end. After all log entries in this log file have been sent, log entries in the circular buffer are sent, ensuring that log entries are sent in order. Optionally, after all log entries in the log file have been sent, the log file can be deleted to free up storage space.
[0092] In some other embodiments, when the occupancy rate of non-volatile memory reaches a preset critical threshold, an alarm message can be generated and displayed in the management interface to notify the administrator to troubleshoot the fault and implement corresponding protection measures to avoid data loss.
[0093] In this embodiment, by reducing the transmission bandwidth or migrating the overflowing log entries to non-volatile memory when the occupancy rate of the circular buffer reaches a preset critical threshold, the circular buffer will not lose log entries due to insufficient transmission speed, thus ensuring the continuity of the log entry link. At the same time, log entries in non-volatile memory are sent first, followed by log entries in the circular buffer, so that the log entries maintain their original order after transmission is resumed.
[0094] In an exemplary embodiment, the following steps may be included after step 230: if a disk flushing operation for a virtual disk is captured by the filtering driver service, determine the time point corresponding to the disk flushing operation; generate a barrier log entry based on the time point and a preset barrier marker; send the barrier log entry to the disaster recovery terminal so that the disaster recovery terminal generates a recovery index record based on the time point in the barrier log entry, and upon receiving a recovery request, performs data recovery on the virtual disk based on the recovery index record, the fully synchronized data, and the metadata in each log entry generated before the time point.
[0095] Here, a flush operation refers to the operation initiated by the virtual machine's guest operating system to write data from the virtual disk to the physical storage medium, such as a Flush operation or a FUA operation. A barrier marker indicates that the data payload of all log entries generated before a certain point in time has been stored in the non-volatile memory at the application end. A barrier log entry is a log entry containing a barrier marker; it does not carry a data payload but records the time point corresponding to the flush operation.
[0096] Recovery index records refer to the index records created by the disaster recovery terminal for barrier log entries. These records document the time point within each barrier log entry and the corresponding range of log entries, such as the correspondence between sequence number ranges and log storage locations. Furthermore, after creating recovery index records for each barrier log entry, the disaster recovery terminal can arrange these records chronologically to form a selectable timeline.
[0097] In some embodiments, the business side can call the second callback function in the filter driver service to capture the disk flushing operation for the virtual disk and determine the time point corresponding to the disk flushing operation. It can be understood that the time point corresponding to the disk flushing operation can be used to represent a data boundary, that is, data written to the virtual disk before that time point has been persisted to disk.
[0098] In some embodiments, the service can generate barrier log entries based on a time point and a preset barrier marker. In other embodiments, the service can also generate barrier log entries based on a time point, a preset barrier marker, and the sequence number of the last log entry before that time point. Subsequently, the service sends the barrier log entries to the disaster recovery terminal, or stores the barrier log entries in a circular buffer to retrieve barrier log entries from the circular buffer and send them to the disaster recovery terminal.
[0099] It is understandable that since the data payload in each log entry before the time point has already been stored in the non-volatile memory on the business side, using this time point as the recovery target can ensure that the recovered data is at least in a crash-consistent state.
[0100] In some embodiments, after receiving a barrier log entry, the disaster recovery terminal generates a recovery index record based on the time point in the barrier log entry. Furthermore, after establishing a recovery index record for each barrier log entry, the disaster recovery terminal can arrange these recovery index records in chronological order to form a selectable timeline.
[0101] In some embodiments, the disaster recovery endpoint can display the timeline. When a user initiates a recovery request, they can select any point in time on the timeline as the recovery target. The disaster recovery endpoint then uses the data obtained from full synchronization as the base data to replay all log entries generated before that point in time, thereby synthesizing the disk image corresponding to that point in time to recover the data in the virtual disk at that point in time. Optionally, the recovery index record can also record the log file offset corresponding to the barrier log entry, so that the disaster recovery endpoint can quickly locate the log entry that needs to be replayed.
[0102] In this embodiment, by generating barrier log entries at the disk flushing operation and establishing recovery index records at the disaster recovery end, the log stream of the disaster recovery end is divided into multiple time points with consistency guarantees, thereby enabling the recovery of virtual disk data at any time point and reducing RPO.
[0103] In an exemplary embodiment, the following steps may be included after step 230: upon receiving a log breakage notification from the disaster recovery terminal, determining the data blocks to be retransmitted in the virtual disk; reading the data in the data blocks to be retransmitted and sending it to the disaster recovery terminal, so that the disaster recovery terminal updates the fully synchronized data based on the data in the data blocks to be retransmitted.
[0104] The log breakage notification is generated by the disaster recovery site when the sequence numbers of received log entries are not consecutive. The data blocks to be retransmitted include data blocks that were overwritten by write requests between the last successful reception of log entries at the disaster recovery site and the current time.
[0105] In some embodiments, since log entries are sent in sequence according to their sequence numbers, the disaster recovery terminal can verify whether the sequence number of a received log entry is continuous with the sequence number of the previously received log entry. If they are not continuous, it means that the sequence number has changed, that is, the log entries within the change range have not been successfully received by the disaster recovery terminal. In this case, the disaster recovery terminal will generate a log break notification carrying the missing sequence number range and send it to the service terminal.
[0106] In some embodiments, during the stable operation of the filter driver service, the business side can maintain a write status record, such as using a bitmap data structure to obtain an active bitmap. One bit in this active bitmap represents the write status of a data block in the virtual disk, including "written" and "not written." "Written" indicates that after the filter driver service reaches a stable operating state, the data block has been overwritten by a write request, meaning new data has been written to the data block; "not written" indicates that after the filter driver service reaches a stable operating state, the data block has not been overwritten by a write request, meaning no new data has been written to the data block.
[0107] When the application receives a log break notification from the disaster recovery server, it can determine the data blocks marked between the last successful log entry received by the disaster recovery server and the current time based on the write status record, thus obtaining the data blocks to be retransmitted. Then, a full synchronization coroutine is invoked to scan and read the data in the data blocks to be retransmitted from the virtual disk and send it to the disaster recovery server, eliminating the need for a full disk scan. Furthermore, after data transmission is complete, the application sends a resynchronization completion flag to the disaster recovery server to indicate that retransmission is complete, and also sends the sequence number range of the log entries covered by the retransmission, as well as the latest sequence number.
[0108] In some embodiments, after receiving the data in the data blocks to be retransmitted, the disaster recovery terminal merges them into the stored base data and reconnects the log entry link from the point of log break. It can be understood that since data changes corresponding to log entries ultimately manifest as changes in data block content, retransmitting these data blocks can restore the disaster recovery terminal's data to a state consistent with the business end. Furthermore, since the number of data blocks to be retransmitted is related to the actual amount of data written during the log break, when less data is written during the log break, the amount of data required for retransmission is correspondingly smaller.
[0109] In this embodiment, by determining the data to be retransmitted upon receiving a log breakage notification and sending the data to be retransmitted to the disaster recovery end, the data consistency between the business end and the disaster recovery end can be restored after the log entry link is broken. At the same time, the amount of retransmitted data depends on the amount of data actually written during the log breakage, rather than on the total capacity of the virtual disk, thus shortening the time required for data consistency recovery.
[0110] For ease of understanding, this application provides an example diagram, please refer to it. Figure 4 The full synchronization and incremental anomaly recovery state machine demonstrates the three operating states of the filter driver service: INIT (initial state), FULL_SYNC (full synchronization state), and STEADY_STATE (stable operating state). The filter driver service, i.e., the CDP Filter, automatically triggers upon mounting, transitioning from the initial state to the full synchronization state. The filter driver service triggers the anomaly recovery mechanism after the base image is marked and confirmed by the remote end (i.e., the disaster recovery end), transitioning from the full synchronization state to the stable operating state. Furthermore, a full resynchronization is triggered when the log chain is broken and incremental synchronization cannot be repaired.
[0111] Furthermore, the anomaly recovery mechanism demonstrates a three-level overflow protection strategy for the lock-free circular buffer. The first level is backpressure rate limiting, the second level is local overflow logging, and the third level is degradation alerting. The incremental resynchronization optimization mechanism performs only a local scan based on the Resync Dirty Bitmap, i.e., the active bitmap. The complete full synchronization loop is as follows: during the parallel execution of incremental I / O capture and full read coroutines, the dirty bitmap is first initialized (i.e., the synchronization state record is initialized first), then block-by-block reading and zeroing (i.e., data in the virtual disk is read block by block and sent to the disaster recovery end, while simultaneously zeroing out synchronized data blocks in the synchronization state record), then business write I / O re-dirtying (i.e., when a write request overwrites the data block, it is remarked as pending synchronization), and finally, convergence iterative scanning (i.e., convergence iterative scanning is performed on the data blocks to be synchronized).
[0112] To facilitate understanding, a complete example of a virtual machine disaster recovery management method is presented below.
[0113] Step 1: Initialization and Mounting of the Bypass Filter Driver. When the target virtual machine starts up or runs via hot-plugging, a custom bypass filter driver service (i.e., CDP Filter) is dynamically inserted into the block device chain of the target virtual disk within the QEMU process using custom QEMU command-line parameters. This includes registering write I / O interception callback functions and disk flush event interception callback functions, allocating and initializing a lock-free circular buffer, establishing a network connection channel with the remote CDP disaster recovery gateway, and starting one or more background asynchronous transmission worker threads.
[0114] Step 2: Full Synchronization of Base Image and Dirty Block Bitmap Tracking. Step 2.1: CDP Protection State Machine Initialization: CDPFilter maintains a three-state state machine: INIT->FULL_SYNC->STEADY_STATE. After the bypass filter driver is mounted in Step 1, CDP Filter enters the FULL_SYNC state. Step 2.2: Dirty Block Bitmap Creation: CDP Filter creates a bitmap data structure in the host memory corresponding to the target virtual disk capacity. Each bit corresponds to a fixed-size data block on the virtual disk (granularity is configurable, defaulting to 64KB or 1MB). Initially, all bits are marked "dirty" (i.e., awaiting synchronization). Step 2.3: Background Full Read and Transfer: CDP Filter starts a dedicated full synchronization coroutine, reading the full data of the virtual disk block by block in ascending order of address. Each data block read is sent to the remote disaster recovery gateway via an asynchronous transmission channel, and the corresponding bitmap bits are cleared (marked as "synchronized"). To avoid excessive storage I / O bandwidth consumption during full read operations, this coroutine supports configurable I / O bandwidth throttle and I / O priority degradation. Step 2.4, Incremental I / O Parallel Capture During Full Synchronization: While full synchronization is in progress, virtual machine service write I / O is continuously intercepted through the CDP Filter (logical synchronization in step S3 takes effect). When a write I / O falls into an address range that has not yet been read by the full synchronization coroutine, the CDP Filter, in addition to enqueuing the write I / O normally into the lock-free circular buffer, will also re-mark the data block covered by the write I / O as "dirty" in the dirty block bitmap. This ensures that when the full synchronization coroutine scans this area later, it will reread the latest data that has been covered by service writes, rather than outdated old data. Step 2.5, Convergence Iteration and Synchronization Completion Determination: After the full synchronization coroutine completes the first round of full scan, it checks whether there are still "dirty" bits in the dirty block bitmap (i.e., blocks that were re-dirtyed by service write I / O during the first round of scan). If dirty blocks exist, incremental rescanning is performed on these dirty blocks, only rereading and transmitting the dirty block data, and clearing the corresponding bitmap bits again. This process is repeated iteratively, with the number of dirty blocks to be scanned monotonically decreasing (convergence) in each round until the number of remaining dirty blocks drops below a preset threshold or no new dirty blocks are added between two consecutive scans. Step 2.6, State Switching and Base Image Confirmation: When the convergence condition is met, the CDP Filter sends a base image completion marker to the remote disaster recovery gateway, carrying the sequence number corresponding to the full data at this moment. After the disaster recovery gateway confirms the marker, it persists the received full data into a complete base disk image. The CDP Filter then switches the state machine from FULL_SYNC to STEADY_STATE, and only performs incremental CDP log capture and transmission thereafter.
[0115] Step 3: Real-time Interception of Write I / O. When an application within the virtual machine initiates a disk write operation, the write request is passed through the Guest OS's file system and block device driver layer, via the virtio-blk or virtio-scsi semi-virtualization driver, to the QEMU's block device layer. When the write I / O request reaches the CDP Filter's callback function, the CDP Filter performs the following operations: Step 3.1: Raw I / O Pass-through: Immediately forward the raw write I / O request without any modification to the next level node in the block storage chain (such as the qcow2 driver), ensuring that normal business I / O of the virtual machine is not blocked. This step ensures that the impact of this application on virtual machine I / O performance is minimized. Step 3.2: Bypass Copy of I / O Metadata and Data: While forwarding the raw I / O (or immediately after the raw I / O is completed), the CDP Filter extracts the following information from the write request to construct a CDP log entry. These include the Logical Block Address Offset (LBA Offset), which is the starting sector offset of this write operation on the virtual disk; the Data Length, which is the size of the data in this write operation (in bytes); the Data Payload, which is the actual data content of this write operation; and the Global Epoch / Sequence Number, which is allocated according to the I / O operation order to satisfy the first-come, first-served principle.
[0116] Step 4: Lock-free circular buffer storage and timing consistency marking. The CDP Filter writes the constructed CDP log entries to a lock-free circular buffer pre-allocated in the host machine's memory. To ensure that the remote CDP disaster recovery gateway strictly adheres to the original I / O write order of the virtual machine during data replay, the CDP Filter embeds a globally monotonically increasing sequence number in each CDP log entry. The sequence number generation process is as follows: The CDP Filter maintains a 64-bit atomic counter. Each time a write I / O operation is intercepted, this counter increments by 1, and its current value is the sequence number of that log entry. This sequence number is globally unique and monotonically increasing within a single virtual disk dimension. The Epoch barrier marking process is as follows: When the CDP Filter captures a Flush / FUA operation issued by the Guest OS, a special Epoch barrier log entry is generated. The semantics of this barrier mark are: "All write I / Os before this barrier have been confirmed by the Guest OS to have been written to disk, constituting a crash consistency point." Furthermore, the remote replay order-preserving process is as follows: after receiving the log stream, the remote CDP disaster recovery gateway strictly executes the write replay operation in ascending order of sequence number, and generates a recoverable snapshot index at the Epoch barrier marker. Therefore, when performing recovery at any point in time, users can choose any Epoch barrier point as the recovery target, ensuring that the recovered data is at least in a crash-consistent state.
[0117] Step 5: Asynchronous Zero-Copy Network Transmission. The background asynchronous transmission worker thread continuously retrieves CDP log entries from the consumer end in the lock-free circular buffer and transmits them over the network to the remote CDP disaster recovery gateway. After persisting the log entries to its local storage, the remote CDP disaster recovery gateway returns an acknowledgment (ACK) to the sender, carrying the largest confirmed and persisted sequence number. The service end advances the consumer cursor based on the ACK, releasing the confirmed buffer space. If the service end does not receive an ACK within the timeout period, it triggers a retransmission mechanism, retransmitting from the unacknowledged sequence number position.
[0118] Step 6, Incremental Synchronization Anomaly Detection and Self-Healing Recovery. Step 6.1, Sequence Number Continuity Detection: When receiving each CDP log entry, the remote CDP disaster recovery gateway verifies whether its sequence number is strictly continuous with the sequence number of the previously received entry (i.e., the difference is 1). If a sequence number jump is detected, a log break notification is immediately sent to the source CDP Filter, carrying the missing sequence number range. Step 6.2, Incremental Resynchronization Based on Dirty Block Bitmap: When the log chain is broken beyond repair (i.e., log entries corresponding to some sequence numbers are lost and cannot be recovered from the overflow log file), the system triggers the incremental resynchronization process: During normal operation, the CDP Filter continuously maintains an active dirty block bitmap. Whenever a write I / O is intercepted, the corresponding data block is marked in this bitmap. When the log break is confirmed, the system extracts all marked dirty blocks from the previous confirmed ACK sequence number to the current sequence number from the active dirty block bitmap, forming a dirty block bitmap to be resynchronized. Furthermore, the full synchronization coroutine uses the dirty block bitmap to be resynchronized as the scanning range (rather than a full disk scan), only reading and retransmitting dirty block data to the remote disaster recovery gateway. This mechanism ensures that the data transfer volume during the recovery process is only proportional to the amount of data actually written during the break, rather than the total virtual disk capacity, significantly shortening the recovery time. Step 6.3, Resynchronization Completed and Log Chain Connection: After incremental resynchronization is completed, CDP Filter sends a resynchronization complete marker to the remote disaster recovery gateway, carrying the sequence number range covered by resynchronization and the latest sequence number. The disaster recovery gateway applies the resynchronized data to the base image and reconnects the incremental log chain from the sequence number corresponding to the marker, restoring the normal continuous data protection state. Step 6.4, When the water level of the unlocked ring buffer continues to rise to the overflow threshold due to insufficient network transmission speed or remote response timeout, CDP Filter initiates the following hierarchical protection strategy: The first-level response is backpressure flow limiting, that is, reducing the I / O bandwidth quota of the full synchronization coroutine and giving more buffer space to the incremental log. The level 2 response involves local overflow log persistence. When backpressure fails to prevent the buffer level from rising further, a background worker thread temporarily writes the overflow log entries to the overflow log file on the host machine's local disk. This file uses append-only writing mode with order preservation as the first priority. After network recovery, the worker thread prioritizes reading from the overflow log file and resuming data transmission. The file is then deleted after transmission is complete. The level 3 response is a CDP degradation alarm. If the disk usage of the overflow log file exceeds a preset limit, the system sends a CDP protection degradation alarm to the management plane, notifying the administrator to intervene.
[0119] Step 7: Remote Log Persistence and Point-in-Time Index Construction. After receiving the log stream, the remote CDP disaster recovery gateway performs the following operations: Step 7.1: Sequential Append to Log Storage: CDP log entries are appended to high-performance log storage media, such as a dedicated log volume composed of NVMe and SSDs, in sequence number order, using a log structured storage mode to fully leverage sequential write performance. Step 7.2: Build Point-in-Time Recovery Index: An index record is generated for each Epoch barrier marker, recording its corresponding sequence number range, timestamp, and log file offset. This index forms a timeline from which users can select any crash consistency point for recovery. Step 7.3: On-Demand Recovery Image Synthesis: When a user selects a specific point in time for recovery, the disaster recovery gateway uses the base image of the target virtual machine as a basis to sequentially replay all CDP log entries before that point in time, synthesizing a complete disk image of the target point in time for recovery.
[0120] As can be seen, the embodiments of this application achieve continuous data protection for virtual machines by fully linking full synchronization, incremental logs, consistency time points and fracture recovery, and no components need to be installed in the virtual machine's operating system throughout the process.
[0121] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0122] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of a virtual machine disaster recovery management device is given below. Please refer to... Figure 5 This is a functional block diagram of the virtual machine disaster recovery management device provided in this application embodiment. It should be noted that the basic principle and technical effects of the virtual machine disaster recovery management device 300 provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The virtual machine disaster recovery management device 300 includes:
[0123] Synchronization module 310 is used to synchronize all data in the virtual disk to the disaster recovery terminal through the filter driver service.
[0124] The generation module 320 is used to forward write requests initiated by the virtual machine for the virtual disk downstream to perform write operations through the filter driver service, and generate log entries based on the metadata of the write request. The metadata includes the data payload in the write request, as well as the storage location and data length of the data payload.
[0125] The recovery module 330 is used to send log entries to the disaster recovery terminal so that the disaster recovery terminal can store the log entries and, upon receiving a recovery request, perform data recovery on the virtual disk based on the metadata in the log entries and the fully synchronized data.
[0126] Optionally, the synchronization module 310 is specifically used for: scanning data blocks marked as pending synchronization in the virtual disk according to a preset synchronization status record for the virtual disk through a filter driver service; reading data from the scanned data blocks and sending it to the disaster recovery end, and marking the scanned data blocks as synchronized in the synchronization status record; during the full synchronization process, if a synchronized data block is overwritten by a write request, the data block is remarked as pending synchronization in the synchronization status record; after completing the current round of scanning and sending, if the data blocks marked as pending synchronization in the synchronization status record do not reach the preset convergence condition, the process returns to executing the filter driver service to scan data blocks marked as pending synchronization in the virtual disk according to the preset synchronization status record for the virtual disk until the preset convergence condition is reached, thus determining that the full synchronization is complete; wherein, the preset convergence condition includes the total number of data blocks pending synchronization being less than or equal to a preset threshold, or no new data blocks pending synchronization being added in two consecutive rounds.
[0127] Optionally, the recovery module 330 is specifically used to: store log entries in a preset circular buffer in ascending order of sequence number; wherein, the smaller the sequence number, the earlier the execution order of the write request corresponding to the log entry; during the storage of log entries, the log entries in the circular buffer are sent to the disaster recovery end in ascending order of sequence number, so that the disaster recovery end stores log entries according to sequence number, and upon receiving a recovery request, performs data recovery on the virtual disk according to the sequence number based on the metadata in the log entry and the fully synchronized data.
[0128] Optionally, the generation module 320 is further configured to: generate log entries based on the metadata of the write request and store them in a circular buffer during the full synchronization process, and send the log entries in the circular buffer to the disaster recovery terminal; during the full synchronization process, if the occupancy rate of the circular buffer reaches a preset critical threshold, reduce the bandwidth for sending data to the disaster recovery terminal, or, if the occupancy rate of the circular buffer reaches a preset critical threshold, migrate the overflowing log entries in the circular buffer to the non-volatile storage in the business terminal, and after sending the log entries in the non-volatile storage to the disaster recovery terminal, send the log entries in the circular buffer to the disaster recovery terminal.
[0129] Optionally, the recovery module 330 is further configured to: determine the time point corresponding to the disk flushing operation when a disk flushing operation for a virtual disk is captured by the filter driver service; generate a barrier log entry based on the time point and a preset barrier marker; the barrier marker is used to indicate that the data payload in each log entry generated before the time point has been stored in the non-volatile memory in the business terminal; send the barrier log entry to the disaster recovery terminal so that the disaster recovery terminal generates a recovery index record based on the time point in the barrier log entry, and upon receiving a recovery request, performs data recovery on the virtual disk based on the recovery index record, the fully synchronized data, and the metadata in each log entry generated before the time point.
[0130] Optionally, the recovery module 330 is further configured to: upon receiving a log breakage notification from the disaster recovery terminal, determine the data blocks to be retransmitted in the virtual disk; wherein the log breakage notification is generated by the disaster recovery terminal when the sequence numbers of the received log entries are not consecutive; the data blocks to be retransmitted include data blocks that were overwritten by write requests during the period from the last successful reception of log entries by the disaster recovery terminal to the current time; read the data in the data blocks to be retransmitted and send it to the disaster recovery terminal so that the disaster recovery terminal updates the fully synchronized data according to the data in the data blocks to be retransmitted.
[0131] The various modules in the aforementioned virtual machine disaster recovery management device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0132] This application also provides a computer device, which can be a terminal or a server, and its internal structure diagram can be as follows. Figure 6As shown, the computer device includes a processor, memory, input / output interfaces, and a communication interface. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a virtual machine disaster recovery management method.
[0133] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0134] This application also provides a computer device, including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the virtual machine disaster recovery management method disclosed in this application.
[0135] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the virtual machine disaster recovery management method disclosed in this application.
[0136] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the virtual machine disaster recovery management method disclosed in this application.
[0137] In summary, the disaster recovery management method, apparatus, device, media, and program products for virtual machines provided in this application have the following technical effects: (1) RPO approaches zero, achieving true continuous data protection. This application intercepts every write I / O operation of the virtual machine in real time and forwards it to remote storage immediately. The RPO can reach the second level or even the sub-second level, which is fundamentally different from the minute-level or hour-level RPO of traditional timed snapshot schemes. In addition, combined with the barrier marking mechanism, it supports accurate recovery at any crash consistency time point. (2) Completely zero intrusion on the Guest OS, completely eliminating compatibility risks. All technical components of this application run inside the QEMU process of the host machine and the host machine user space. There is no need to install any agent software or load any kernel driver in the virtual machine Guest OS. The version updates and security patch upgrades of the operating system inside the virtual machine, such as Linux and Windows, are completely decoupled from the CDP functional logic of this application. It completely eliminates the risk of blue screen and crash caused by kernel driver compatibility issues, and is completely transparent and imperceptible to cloud platform tenants. (3) Extremely low business performance loss. This application can reduce the impact on the I / O performance of virtual machine services. The lock-free ring buffer controls the data copying time on the I / O interception path to the microsecond level, avoiding the context switching overhead caused by locking. The asynchronous transmission architecture completely removes time-consuming network transmission operations from the I / O critical path, and network jitter will not affect the business I / O latency. Zero-copy transmission can reduce the number of data moves and reduce the CPU utilization of the host machine. (4) Adapt to the centralized management scenario of large-scale cloud computing platforms. The CDP function of this application uses virtual disks as protection units and performs centralized policy configuration and lifecycle management through the cloud platform management plane. No manual intervention is required for each virtual machine, which is naturally adapted to the scenario of large-scale cloud computing data centers with thousands to tens of thousands of virtual machines, and can significantly reduce the deployment and maintenance costs of disaster recovery systems. (5) Fine-grained protection at the virtual machine level. The I / O interception point of this application is located inside the QEMU process, and naturally uses a single virtual disk as the protection boundary. Different virtual machines can be independently configured with differentiated CDP policies, such as different RPO targets, log retention periods, remote storage targets, etc., supporting hierarchical protection of core business virtual machines and ordinary virtual machines, which is superior to the coarse-grained management mode of storage layer replication schemes.
[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0139] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0140] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for disaster management of a virtual machine, the method comprising: Applied to the business end, which communicates with the disaster recovery end, the business end has a filter driver service configured in the block device chain of the virtual disk corresponding to the virtual machine in the business end. The method includes: The filtering driver service is used to fully synchronize the data in the virtual disk to the disaster recovery terminal. The filter driver service forwards the write requests initiated by the virtual machine for the virtual disk downstream to perform write operations, and generates log entries based on the metadata of the write requests. The metadata includes the data payload in the write request, as well as the storage location and data length of the data payload. The log entries are sent to the disaster recovery terminal so that the disaster recovery terminal stores the log entries and, upon receiving a recovery request, performs data recovery on the virtual disk based on the metadata in the log entries and the fully synchronized data.
2. The method of claim 1, wherein, The step of synchronizing all data in the virtual disk to the disaster recovery terminal through the filtering driver service includes: The filtering driver service scans data blocks marked as pending synchronization in the virtual disk based on the preset synchronization status records for the virtual disk. Read the data in the scanned data block and send it to the disaster recovery terminal, and mark the scanned data block as synchronized in the synchronization status record; During full synchronization, if a synchronized data block is overwritten by a write request, the data block will be remarked as pending synchronization in the synchronization status record. After completing the current round of scanning and sending, if the data blocks marked as pending synchronization in the synchronization status record do not meet the preset convergence condition, the process returns to the step of scanning the data blocks marked as pending synchronization in the virtual disk according to the preset synchronization status record for the virtual disk through the filtering driver service until the preset convergence condition is met, thus confirming that full synchronization is complete. The preset convergence condition includes the total number of data blocks pending synchronization being less than or equal to a preset threshold, or no new data blocks pending synchronization being added in two consecutive rounds.
3. The method of claim 1, wherein, The log entry also includes a sequence number, which indicates the execution order of the write requests; Sending the log entries to the disaster recovery terminal includes: Log entries are stored in a preset circular buffer in ascending order of their sequence numbers; wherein, the smaller the sequence number, the earlier the write request corresponding to the log entry is executed. During the storage of log entries, the log entries in the circular buffer are sent to the disaster recovery terminal in ascending order of the sequence number, so that the disaster recovery terminal stores the log entries according to the sequence number, and performs data recovery on the virtual disk according to the sequence number based on the metadata in the log entries and the fully synchronized data.
4. The method according to claim 3, characterized in that, The method further includes: During the full synchronization process, log entries are generated based on the metadata of the write request and stored in the circular buffer, and the log entries in the circular buffer are sent to the disaster recovery terminal. During full synchronization, if the occupancy rate of the circular buffer reaches a preset critical threshold, the bandwidth for sending data to the disaster recovery terminal is reduced, or... When the occupancy rate of the circular buffer reaches a preset critical threshold, the log entries overflowing from the circular buffer are migrated to the non-volatile memory in the service terminal, and the log entries in the non-volatile memory are sent to the disaster recovery terminal. Then, the log entries in the circular buffer are sent to the disaster recovery terminal.
5. The method according to claim 1, characterized in that, The method further includes: If a disk flushing operation for the virtual disk is captured by the filter driver service, the time point corresponding to the disk flushing operation is determined. Based on the time point and the preset barrier marker, a barrier log entry is generated; the barrier marker is used to indicate that the data payload in each log entry generated before the time point has been stored in the non-volatile memory in the service terminal. The barrier log entries are sent to the disaster recovery terminal so that the disaster recovery terminal can generate a recovery index record based on the time point in the barrier log entries, and upon receiving a recovery request, perform data recovery on the virtual disk based on the recovery index record, the fully synchronized data, and the metadata in each log entry generated before the time point.
6. The method according to claim 1, characterized in that, The method further includes: Upon receiving a log breakage notification from the disaster recovery terminal, the data blocks to be retransmitted in the virtual disk are determined; wherein, the log breakage notification is generated by the disaster recovery terminal when the sequence numbers of the received log entries are not consecutive; the data blocks to be retransmitted include data blocks that were overwritten by write requests during the period from the last successful reception of log entries by the disaster recovery terminal to the current time. The data in the data block to be retransmitted is read and sent to the disaster recovery terminal so that the disaster recovery terminal can update the fully synchronized data according to the data in the data block to be retransmitted.
7. A disaster recovery device for virtual machines, characterized in that, Applied to the business end, which communicates with the disaster recovery end, the business end has a filter driver service configured in the block device chain of the virtual disk corresponding to the virtual machine in the business end. The device includes: The synchronization module is used to fully synchronize the data in the virtual disk to the disaster recovery terminal through the filtering driver service; The generation module is used to forward the write request initiated by the virtual machine for the virtual disk to the downstream through the filter driver service to perform the write operation, and generate log entries based on the metadata of the write request, wherein the metadata includes the data payload in the write request and the storage location and data length of the data payload; The recovery module is used to send the log entries to the disaster recovery terminal so that the disaster recovery terminal stores the log entries and, upon receiving a recovery request, performs data recovery on the virtual disk based on the metadata in the log entries and the fully synchronized data.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
