Virtual machine agent-free real-time backup method and device for privatized cloud platform

Abstract

The invention discloses an agent-free real-time backup method for a virtual machine of a privatized cloud platform, which comprises the following steps of: starting a backup service of a transmission unit, loading an eBPF program and initializing a configuration file; creating and initializing a shared memory; an eBPF program is loaded on a host machine, and the eBPF program is attached to a related function for processing a virtio-blk write IO request; the eBPF program intercepts a write IO (input / output) of virtio-blk equipment in the host machine, and data parameters are analyzed; the eBPF program encapsulates the write IO request and adds the encapsulated write IO request data structure into a shared memory; the backup service of the transmission unit reads the data from the shared memory and transmits the data to the storage unit; according to the method, complex snapshot operation and related resource consumption work such as data comparison and merging do not need to be carried out, the influence of the backup process on the performance of a service environment is small, the dependence on a virtualization platform is reduced, the method has higher universality and portability, and the limitation of a traditional backup method is effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data protection, and particularly to a method and device for agentless real-time backup of virtual machines in a private cloud platform. Background Art

[0002] In the host environment of a virtualization platform, full and incremental backups of guest machines are usually achieved by reading virtual disk image files and using the snapshot function provided by the virtualization platform. The advantage of this method is that the implementation process is relatively simple. However, its defects are also relatively obvious. Different virtualization platforms have differences in the processing modes of snapshot management and image file merging. Commonly, when a platform deletes a snapshot, only the snapshot itself is deleted, and the next image file of the corresponding virtual disk image file is merged into the image file corresponding to the currently deleted snapshot. While for other platforms, when a snapshot is deleted, all snapshots from this snapshot to the latest snapshot will be deleted, and at the same time, the image files originally corresponding to these snapshots are merged into the active virtual disk image file;

[0003] However, if the conventional method of combining reading virtual disk image files and the snapshot function of the virtualization platform is used to achieve incremental backup, a snapshot must be created and not deleted every time a backup operation is performed. In this way, the number of snapshots will continue to accumulate, which will inevitably have a serious negative impact on the performance of the guest machine. This situation is obviously unacceptable. And another backup solution is to create a snapshot every time a backup is performed and delete the snapshot after the backup is completed. However, this solution not only requires consistency comparison of each piece of data at the source and target ends, but also when deleting the snapshot, merging the image file containing all historical data will bring huge I / O pressure to the host machine. Summary of the Invention

[0004] The present invention provides a method and device for agentless real-time backup of virtual machines in a private cloud platform, which can effectively solve the problems proposed in the above background art that the number of snapshots will continue to accumulate, inevitably having a serious negative impact on the performance of the guest machine, and when deleting the snapshot, merging the image file containing all historical data will bring huge I / O pressure to the host machine.

[0005] To achieve the above object, the present invention provides the following technical solution: A method for agentless real-time backup of virtual machines in a private cloud platform, which loads an eBPF program on the host machine, monitors the write I / O requests of the guest machine in real time, and continuously backs up these write I / O requests through a backup program in the user space to achieve backup, without relying on the snapshot function of the virtualization platform itself;

[0006] It includes the following processes:

[0007] S1. Start the backup service of the transmission unit, load the eBPF program, and initialize the configuration file;

[0008] S2. Create and initialize the shared memory;

[0009] S3. Load the eBPF program on the host and attach it to the relevant functions that handle virtio - blk write IO requests;

[0010] S4. The eBPF program intercepts the write IO of the virtio - blk device in the host and parses the data parameters;

[0011] S5. The eBPF program encapsulates the write IO request and adds the encapsulated write IO request data structure to the shared memory;

[0012] S6. The backup service of the transmission unit reads the data from the shared memory and transfers it to the storage unit;

[0013] S7. Create a snapshot;

[0014] In the above S1, the backup service is responsible for starting, managing, and monitoring the entire backup process. This service listens to the data intercepted by the eBPF program, continuously transfers the data in the shared memory to the backup target, and performs error handling and traffic control.

[0015] According to the above technical solution, in the above S1, eBPF is a small program that runs in the Linux kernel and is commonly used for tasks such as network monitoring, performance analysis, and debugging;

[0016] The eBPF program can be attached to specific hooks in the kernel or user space, interact with the kernel through specific interfaces, execute tasks, and access the data between the kernel and user space. eBPF has the ability to efficiently capture, filter, and modify kernel data. It can be attached to specific functions or system calls through user - space probes uprobes to capture detailed information about specific processes or kernel operations. In virtualization scenarios, eBPF is mainly used to trace and analyze the interaction between the guest and the host;

[0017] And KVM is a virtualization function provided by the Linux kernel that allows the host to run multiple guest machines, and the guest machines interact with the host through virtual devices;

[0018] In the above S1, it specifically includes the following steps:

[0019] S11. Start the backup service of the transmission unit and perform a service self - check to confirm that all dependencies and environment variables have been correctly configured to ensure that the service has the basic conditions for performing backup operations;

[0020] S12. Use tools to write and load eBPF programs;

[0021] S13. Initialize the configuration file, which contains various parameters and settings required during the backup process for subsequent use in the backup process.

[0022] According to the above technical solution, in S2, the shared memory serves as an intermediary for data transmission, temporarily storing or recording the write IO request data of the virtio-blk device intercepted by the eBPF program from the host machine for subsequent reading by the backup service of the transmission unit. The shared memory includes a Buffer and a Bitmap, playing an indispensable role in the data transmission and status management of the entire agentless real-time backup process of the virtual machine, ensuring the smooth and accurate progress of the backup work;

[0023] The buffer Buffer is used to temporarily store the write IO request data structure intercepted and encapsulated by the eBPF program. This data structure includes source disk information, the offset position of the IO, the data length, and the business data content;

[0024] The disk Bitmap is used to assist in recording the changes of data blocks on the guest machine's disk. It maintains a bit for each data block. One bit corresponds to one data block, and the size of the data block can be flexibly set according to the actual situation. Its value is used to indicate whether the corresponding disk data block needs to be synchronized to the target backup storage. 0 indicates that the data block corresponding to this bit does not need to be synchronized, and 1 indicates that the data block corresponding to this bit needs to be synchronized;

[0025] In S2, it specifically includes the following steps:

[0026] S21. Create a shared memory area, allocate a sufficiently large memory space for the shared memory on the host machine. This space will be used to store the Buffer and the Bitmap, and ensure that both the eBPF program and the backup service have permission to access this shared memory area;

[0027] The size of the buffer Buffer is determined according to the business characteristics of the virtual machine, the expected scale of write IO requests, and the processing capacity of the backup service;

[0028] The size of the disk Bitmap is determined according to the data block partitioning strategy of the guest machine's disk. Based on the total disk capacity and the set data block size, calculate the total number of data blocks partitioned on the disk, and then obtain the number of bits required for the Bitmap;

[0029] S22. Initialize the Buffer, perform an initialization operation on the allocated Buffer memory space, set each field to its initial default value, and ensure that the data structure in the Buffer is in a known and predictable state in the initial state;

[0030] S23, initialize Bitmap, and initialize all bits of Bitmap to 0, which means that in the initial state, all data blocks on the default disk have not changed, and no synchronous backup operation is required. The values ​​of these bits will be dynamically updated according to the actual write IO events later.

[0031] According to the above technical solution, in S3, virtio-blk is a virtualized block device interface used for disk I / O communication between the client and the host, and the client operating system accesses the virtual disk through the virtio-blk driver to perform read and write operations;

[0032] virtio-blk is a block device virtualization driver based on VirtIO. It provides clients with a block device access interface similar to a physical hard disk. By using virtio-blk, clients can perform random storage access like accessing ordinary disks, thus supporting the creation and management of file systems and the application of database storage.

[0033] The working principle of virtio-blk is as follows: virtio-blk uses a ring buffer and shared memory mechanism to achieve efficient data exchange. The client initiates an I / O request to the virtio-blk device, the client driver transfers the data to the shared memory, and the host reads the data from the shared memory and operates it.

[0034] According to the above technical solution, in S4, the eBPF program can be attached to a function or code segment related to QEMU to capture detailed information on processing a client virtio-blk I / O request in the QEMU process;

[0035] QEMU is a user space process responsible for handling client I / O requests, especially those related to virtio-blk devices. Through uprobes, the eBPF program is attached to the relevant functions in QEMU responsible for handling virtio-blk I / O requests. The eBPF program can capture function calls in the QEMU process, access local variables and function parameters in QEMU memory, and parse the written offset, length, device information, and written data.

[0036] In S4, the IO between the client and the host is intercepted by eBPF, and the specific implementation steps include:

[0037] S41. Use uprobes to attach the eBPF program to the relevant functions in QEMU responsible for handling virtio-blk I / O requests.

[0038] S42. Capture the written offset, length, device path, and written data;

[0039] S43. Use a tool to write and load the eBPF program;

[0040] S44. Achieve precise interaction with the QEMU process through uprobes.

[0041] According to the above technical solution, in S5, it can be subdivided into two scenarios. One is a full backup or data blocks that need to be retransmitted due to exceptions, and the other is an incremental backup of only business IOs;

[0042] When performing a full backup or there are data blocks that need to be retransmitted due to exceptions, the specific steps are as follows:

[0043] S51. If it is a full backup, set the bit values of all bits in the disk Bitmap to 1. If it is a data block that needs to be retransmitted due to an exception, only set the bit value of the corresponding bit of this data block to 1;

[0044] S52. eBPF captures business IOs in real time, and the backup service of the transmission unit processes them in ascending order of the offset position. Assume that the currently processed position is denoted as offsetmiddle, and new IOs are generated at different offsets at this time;

[0045] If the new IO generated by the business is on the left side of the disk offsetmiddle, the position is denoted as offsetleft, and the new data is recorded in the buffer. If the buffer is full at this time, it is updated to the Bitmap, and the bit corresponding to the data block is set to 1;

[0046] If the new IO generated by the business is on the right side of the disk offsetmiddle, the position is denoted as offsetright, and the new data is not recorded in the buffer and no other processing is required;

[0047] If the new IO generated by the business just falls on the position of offsetmiddle, the data being transmitted is denoted as datamiddle, and the new data is denoted as datamiddle′, then discard the old IO data and retransmit the new IO data of the data block corresponding to this bit;

[0048] S53. Determine that all data blocks corresponding to all bits in the Bitmap have been successfully synchronized to the target backup storage, that is, all bits are 0. The transmission unit obtains data from the buffer for synchronization, and at this time, creating a snapshot is allowed;

[0049] If there are unsynchronized data blocks in the Bitmap, continue to process the data blocks with bit 1 in the Bitmap. To ensure the consistency of backup data, snapshot creation needs to be disabled at this time;

[0050] When performing incremental data synchronization, that is, when only the intercepted business IOs need to be processed, the specific steps are as follows:

[0051] S51′. Determine whether the buffer is full. If the buffer is not full, execute S52′; if the buffer is full, execute S53′;

[0052] S52′. Write the business IO data captured by eBPF in real time into the buffer;

[0053] S53′. Update the Bitmap, and set the bit corresponding to the business IO data block captured by eBPF in real time to 1;

[0054] S54′. The backup service of the transfer unit continuously obtains data from the buffer and synchronizes it to the target backup storage;

[0055] S55′. Update the data blocks marked as 1 in the disk bitmap Bitmap into the buffer. Similarly, when there are data blocks with bit 1 marked in the Bitmap, to ensure the consistency of backup data, snapshot creation is disabled at this time;

[0056] The business IO data intercepted by the eBPF program will first be recorded in the Buffer. If the volume of business IO data is huge, or the processing speed of the transfer unit is relatively slow, the Buffer may reach the full load state. Once the Buffer is full, the changes of subsequent business IO data blocks will be updated to the Bitmap;

[0057] If there are data blocks with transmission failures due to exceptions, they will also be updated and recorded in the Bitmap. When there is free space in the Buffer, the data blocks marked as 1 in the Bitmap will be put into the Buffer, and the backup service of the transfer unit continuously obtains data from the Buffer and synchronizes it.

[0058] According to the above technical solution, in S3, S4, and S5, the processing flow of write IOs is specifically as follows:

[0059] A. The application program in the client initiates a write operation through the file system;

[0060] B. The virtio - blk driver in the client receives and encapsulates the write request and puts it into the virtqueue;

[0061] C. The write IO request is passed from the client to QEMU through the virtqueue;

[0062] D. The QEMU decodes the request, determines the offset and data of the write operation, and prepares the data for writing.

[0063] E. Write the data to the storage backend.

[0064] F. After QEMU completes the write operation, it notifies the guest through interrupt or polling.

[0065] According to the above technical solution, step S7 specifically includes the following steps:

[0066] S71. Determine whether there is a data block to be synchronized in the disk bitmap Bitmap, that is, whether there is a data block with the bit marked as 1. If there is no bit marked as 1, allow creating a snapshot and execute S72. If there is a bit marked as 1, do not allow creating a snapshot and end the task.

[0067] S72. Insert a snapshot instruction mark into the buffer according to the backup task plan.

[0068] S73. The transmission unit synchronizes the data before the snapshot instruction mark in the buffer to the target backup storage.

[0069] S74. Notify the target side to create a snapshot.

[0070] S75. After receiving the request to create a snapshot, the target side creates a snapshot.

[0071] A proxy-free real-time backup device for virtual machines in a private cloud platform includes an eBPF service IO interception unit, a Buffer buffer, a disk Bitmap, a transmission unit, a storage unit, and a snapshot creation unit inserted between the IO channels of the virtual machine VM and the host.

[0072] According to the above technical solution, the eBPF service IO interception unit is responsible for loading the eBPF program on the host, which is used to track and analyze the interaction between the guest and the host to intercept business IO operations.

[0073] The eBPF program can be attached to functions or code segments related to QEMU, capture function calls in the QEMU process, access local variables and function parameters in the QEMU memory, and parse out device information, write offsets, lengths, and written data.

[0074] The Buffer buffer refers to caching the business IO intercepted by the eBPF program that needs to be transmitted for backup services to transmit.

[0075] The disk Bitmap is used to record the change status of disk data blocks. 0 indicates unchanged, and 1 indicates changed. By using the bitmap, it can record which blocks have been changed. For the data blocks where the business I / O is located after the Buffer is full, or for the data blocks that need to be retransmitted due to exceptions, the corresponding bit in their Bitmap is marked as 1;

[0076] The backup service of the transmission unit is responsible for obtaining data from the Buffer and transmitting it to the backup storage;

[0077] The storage unit refers to the destination of the backup data. It is responsible for receiving the data sent by the transmission unit and storing it persistently;

[0078] The snapshot creation unit is responsible for creating a snapshot of the data in the storage unit at a specific time point, which can capture the state of the virtual machine at a certain moment, including its file system and data, so as to restore the data to the historical state at the time when the snapshot was created when needed.

[0079] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0080] 1. The real-time backup method based on the eBPF technology of the present invention loads an eBPF program on the host machine. This program can monitor the write I / O requests of the guest machine in real time. At the same time, through the backup program in the user space, these write I / O requests are continuously backed up to achieve the backup purpose. Moreover, this method no longer depends on the snapshot function of the virtualization platform itself. In this way, it can effectively avoid the performance loss caused by snapshots in the virtualization platform and significantly improve the compatibility with different virtualization platforms, providing a more generally applicable solution for data backup in the virtualization environment with little impact on the performance of the business environment;

[0081] 2. The present invention does not require complex snapshot operations and related data comparison, merging and other tasks that may consume a large amount of system resources. The entire backup process has little impact on the performance of the business environment, effectively reduces the dependence on the virtualization platform, is more general and portable, effectively solves the limitations of traditional backup methods, and while ensuring the data backup requirements, does not interfere with the business applications running on the guest machine, enabling the business to run continuously and stably, providing a reliable data backup guarantee for the production and operation of enterprises, and taking into account the requirements of business continuity at the same time.

[0082] 3. The present invention tightly couples the traditional backup method with specific functions of the virtualization platform. When switching to other virtualization platforms or when the platform undergoes functional upgrades or changes, the backup solution often requires significant adjustments or even re - design. In contrast, the backup based on eBPF technology is relatively independent and has a low dependence on the functions of the virtualization platform itself, making it easier to adapt to various changes in different platforms and enhancing the generality and portability of the backup solution in different virtualization scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, but do not constitute a limitation to the present invention.

[0084] In the accompanying drawings:

[0085] Figure 1 is the step - flow chart of the backup method of the present invention;

[0086] Figure 2 is the schematic diagram of the backup device of the present invention;

[0087] Figure 3 is the working principle diagram of the eBPF program of the present invention;

[0088] Figure 4 is the schematic diagram of the real - time IO transmission rule of the business of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0089] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0090] Embodiment: As Figure 1 shown, the present invention provides a technical solution, a method for agent - less real - time backup of virtual machines in a private cloud platform. An eBPF program is loaded on the host machine to monitor the write I / O requests of the guest machine in real time, and these write I / O requests are continuously backed up by a backup program in the user space to achieve backup, without relying on the snapshot function of the virtualization platform itself;

[0091] The process includes the following:

[0092] S1. Start the backup service of the transmission unit, load the eBPF program and initialize the configuration file;

[0093] S2. Create and initialize the shared memory;

[0094] S3. Load the eBPF program on the host machine and attach it to the relevant functions for processing virtio - blk write IO requests;

[0095] S4. The eBPF program intercepts the write I / O of the virtio-blk device in the host and parses the data parameters;

[0096] S5. The eBPF program encapsulates the write I / O request and adds the encapsulated write I / O request data structure to the shared memory;

[0097] S6. The backup service of the transmission unit reads the data from the shared memory and transmits it to the storage unit;

[0098] S7. Create a snapshot;

[0099] In S1, the backup service is responsible for starting, managing, and monitoring the entire backup process. This service listens for the data intercepted by the eBPF program, continuously transmits the data in the shared memory to the backup target, and performs error handling and traffic control.

[0100] Based on the above technical solution, in S1, eBPF is a small program that runs in the Linux kernel and is commonly used for tasks such as network monitoring, performance analysis, and debugging;

[0101] The eBPF program can be attached to specific hooks in the kernel or user space, interact with the kernel through specific interfaces, execute tasks, and access the data between the kernel and user space. eBPF has the ability to efficiently capture, filter, and modify kernel data. It can be attached to specific functions or system calls through user space probes uprobes to capture detailed information about specific processes or kernel operations. In virtualization scenarios, eBPF is mainly used for tracing and analyzing the interactions between guest machines and the host;

[0102] While KVM is a virtualization function provided by the Linux kernel that allows the host to run multiple guest machines. The guest machines interact with the host through virtual devices, and the virtual devices include virtual disks and network interfaces;

[0103] In S1, it specifically includes the following steps:

[0104] S11. Start the backup service of the transmission unit and perform a service self-check to confirm that all dependencies and environment variables are correctly configured to ensure that the service has the basic conditions for performing backup operations;

[0105] S12. Use tools to write and load the eBPF program. The specific tools include bcc and libbpf;

[0106] S13. Initialize the configuration file, which contains various parameters and settings required during the backup process for use in subsequent backup processes. The specific parameters include the backup frequency, the storage location of the backup data, and the size of the shared memory.

[0107] Based on the above technical solution, in S2, the shared memory serves as an intermediary for data transmission, temporarily storing or recording the write IO request data of the virtio-blk device intercepted by the eBPF program from the host, so that it can be read by the backup service of the transmission unit later. The shared memory includes Buffer and Bitmap, which plays an indispensable role in the data transmission and status management of the entire virtual machine agentless real-time backup process, ensuring the smooth and accurate implementation of the backup work;

[0108] The buffer Buffer is used to temporarily store the write IO request data structure intercepted and encapsulated by the eBPF program. This data structure includes source disk information, the offset position of the IO, the data length, and the business data content. Among them, the source disk information includes the disk identifier, the data length includes the number of bytes written, and the business data content refers to the actual data block written;

[0109] The disk Bitmap is used to assist in recording the changes of data blocks on the guest machine disk. It maintains a bit for each data block. One bit corresponds to one data block. The size of the data block can be flexibly set according to the actual situation. Its value is used to indicate whether the corresponding disk data block needs to be synchronized to the target backup storage. 0 means that the data block corresponding to this bit does not need to be synchronized, and 1 means that the data block corresponding to this bit needs to be synchronized;

[0110] In S2, it specifically includes the following steps:

[0111] S21. Create a shared memory area, allocate a sufficiently large memory space for the shared memory on the host. This space will be used to store Buffer and Bitmap, and ensure that both the eBPF program and the backup service have permission to access this shared memory area;

[0112] The size of the buffer Buffer is determined according to the business characteristics of the virtual machine, the expected scale of write IO requests, and the processing capacity of the backup service;

[0113] The size of the disk Bitmap is determined according to the data block division strategy of the guest machine disk. According to the total disk capacity and the set data block size, calculate the total number of data blocks divided by the disk, and then obtain the number of bits required for Bitmap;

[0114] S22. Initialize Buffer, perform an initialization operation on the allocated Buffer memory space, and set each field to its initial default value. Each field specifically includes the memory areas corresponding to the disk identifier, the IO offset position, and the data length. Specifically, initialize the disk identifier field to an empty string, initialize the IO offset position and the data length to 0, and clear the business data content area to ensure that the data structure in Buffer is in a known and predictable state in the initial state;

[0115] S23. Initialize the Bitmap and set all bits of the Bitmap to 0, which means that by default all data blocks on the disk have not changed in the initial state and no synchronization backup operation is required. Subsequently, the values of these bits will be dynamically updated according to actual write I / O events.

[0116] Based on the above technical solution, in S3, virtio-blk is a virtualized block device interface for disk I / O communication between the guest and the host. The guest operating system accesses the virtual disk through the virtio-blk driver for read and write operations.

[0117] virtio-blk is a block device virtualization driver based on VirtIO. It provides a block device access interface similar to a physical hard disk for the guest. By using virtio-blk, the guest can perform random storage access as if it were accessing a normal disk, thus supporting the creation and management of file systems and the application of database storage.

[0118] The working principle of virtio-blk is as follows: virtio-blk uses a ring buffer and a shared memory mechanism to achieve efficient data exchange. The guest initiates an I / O request to the virtio-blk device, and the guest driver transfers the data to the shared memory. The host then reads the data from the shared memory and performs operations.

[0119] As Figure 3 shown, based on the above technical solution, in S4, the eBPF program can be attached to functions or code segments related to QEMU to capture detailed information about handling guest virtio-blk I / O requests in the QEMU process.

[0120] QEMU is a user-space process responsible for handling guest I / O requests, especially requests related to virtio-blk devices. Through uprobes, the eBPF program is attached to functions in QEMU responsible for handling virtio-blk I / O requests. The eBPF program can then capture function calls in the QEMU process, access local variables and function parameters in QEMU memory, and parse out the written offsets, lengths, device information, and written data.

[0121] In S4, intercepting the I / O between the guest and the host through eBPF, the specific implementation steps include:

[0122] S41. Use uprobes to attach the eBPF program to functions in QEMU responsible for handling virtio-blk I / O requests.

[0123] S42. Capture the written offset, length, device path, and written data;

[0124] S43. Use tools to write and load eBPF programs, where the tools include bcc and libbpf;

[0125] S44. Achieve precise interaction with the QEMU process through uprobes.

[0126] Based on the above technical solution, in S5, it can be subdivided into two scenarios. One is full backup or there are data blocks that need to be retransmitted due to exceptions, and the other is incremental backup of only business I / O;

[0127] When performing full backup or there are data blocks that need to be retransmitted due to exceptions, the specific steps are as follows:

[0128] S51. If it is a full backup, set the bit values of all bits in the disk Bitmap to 1. If it is a data block that needs to be retransmitted due to an exception, only set the bit value of the corresponding bit of this data block to 1;

[0129] S52. eBPF captures business I / O in real time, and the backup service of the transmission unit processes it in ascending order of the offset position. Assume that the currently processed position is denoted as offsetmiddle, and new I / O is generated at different offsets at this time;

[0130] If the new I / O generated by the business is on the left side of the disk offsetmiddle, and the position is denoted as offsetleft, the new data is recorded in the buffer. If the buffer is full at this time, it is updated to the Bitmap, and the bit corresponding to the data block is set to 1;

[0131] If the new I / O generated by the business is on the right side of the disk offsetmiddle, and the position is denoted as offsetright, the new data is not recorded in the buffer and no other processing is required;

[0132] If the new I / O generated by the business just falls on the position of offsetmiddle, the data being transmitted is denoted as datamiddle, and the new data is denoted as datamiddle′, then discard the old I / O data and retransmit the new I / O data of the data block corresponding to this bit;

[0133] As Figure 4 shown, S53. Determine that all data blocks corresponding to all bits in the Bitmap have been successfully synchronized to the target backup storage, that is, all bits are 0. The transmission unit obtains data from the buffer for synchronization, and at this time, creating a snapshot is allowed;

[0134] If there are unsynchronized data blocks in the Bitmap, continue to process the data blocks with bit 1 in the Bitmap. To ensure the consistency of backup data, snapshot creation needs to be disabled at this time;

[0135] When performing incremental data synchronization, that is, when only the intercepted business IOs need to be processed, the specific steps are as follows:

[0136] S51′. Judge whether the buffer is full. If the buffer is not full, execute S52′; if the buffer is full, execute S53′;

[0137] S52′. Write the business IO data captured by eBPF in real time into the buffer;

[0138] S53′. Update the Bitmap, and set the bit corresponding to the business IO data block captured by eBPF in real time to 1;

[0139] S54′. The backup service of the transmission unit continuously obtains data from the buffer and synchronizes it to the target backup storage;

[0140] S55′. Update the data blocks marked as 1 in the disk bitmap Bitmap into the buffer. Similarly, when there are data blocks with bit 1 marked in the Bitmap, to ensure the consistency of backup data, snapshot creation is disabled at this time;

[0141] The business IO data intercepted by the eBPF program will first be recorded in the Buffer. If the volume of business IO data is huge, or the processing speed of the transmission unit is relatively slow, the Buffer may reach the full load state. Once the Buffer is full, the changes of subsequent business IO data blocks will be updated to the Bitmap;

[0142] If there are data blocks with transmission failures caused by exceptions, they will also be updated and recorded in the Bitmap. When there is free space in the Buffer, the data blocks marked as 1 in the Bitmap will be put into the Buffer, and the backup service of the transmission unit continuously obtains data from the Buffer and synchronizes it.

[0143] Based on the above technical solutions, in S3, S4, and S5, the processing flow of write IOs is specifically as follows:

[0144] A. The application program in the client initiates a write operation through the file system;

[0145] B. The virtio - blk driver in the client receives and encapsulates the write request and puts it into the virtqueue;

[0146] C. The write IO request is passed from the client to QEMU through the virtqueue;

[0147] D. The QEMU decodes the request, determines the offset and data of the write operation, and prepares the data for writing.

[0148] E. Write the data to the storage backend.

[0149] F. After QEMU completes the write operation, it notifies the guest through interrupt or polling.

[0150] Based on the above technical solution, step S7 specifically includes the following steps:

[0151] S71. Determine whether there is a data block to be synchronized in the disk bitmap Bitmap, that is, whether there is a data block with the bit marked as 1. If there is no bit marked as 1, allow the creation of a snapshot and execute S72. If there is a bit marked as 1, do not allow the creation of a snapshot and end the task.

[0152] S72. Insert a snapshot instruction flag into the buffer according to the backup task plan.

[0153] S73. The transmission unit synchronizes the data before the snapshot instruction flag in the buffer to the target backup storage.

[0154] S74. Notify the target end to create a snapshot.

[0155] S75. After receiving the request to create a snapshot, the target end creates a snapshot.

[0156] As Figure 2 shown, a proxy-free real-time backup device for virtual machines in a private cloud platform includes an eBPF service IO interception unit, a Buffer buffer, a disk Bitmap, a transmission unit, a storage unit, and a snapshot creation unit inserted between the IO channels of the virtual machine VM and the host.

[0157] Based on the above technical solution, the eBPF service IO interception unit is responsible for loading the eBPF program on the host, which is used to trace and analyze the interaction between the guest and the host to intercept business IO operations.

[0158] The eBPF program can be attached to functions or code segments related to QEMU, capture function calls in the QEMU process, access local variables and function parameters in the QEMU memory, and parse out device information, written offsets, lengths, and written data.

[0159] The Buffer buffer refers to caching the business IO intercepted by the eBPF program that needs to be transmitted for backup services to transmit.

[0160] The disk Bitmap is used to record the change status of disk data blocks. 0 indicates no change, and 1 indicates a change. By using the bitmap, it can record which blocks have been changed. For the data blocks where the business I / O is located after the Buffer is full, or for the data blocks that need to be retransmitted due to exceptions, the corresponding bit in their Bitmap is marked as 1;

[0161] The backup service of the transmission unit is responsible for obtaining data from the Buffer buffer and transmitting it to the backup storage;

[0162] The storage unit refers to the destination of the backup data. It is responsible for receiving the data sent by the transmission unit and storing it persistently;

[0163] The snapshot creation unit is responsible for creating a snapshot of the data in the storage unit at a specific time point. It can capture the state of the virtual machine at a certain moment, including its file system and data, so as to restore the data to the historical state at the time when the snapshot was created when needed.

[0164] Finally, it should be noted that the above are only preferred examples of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for real-time agentless backup of virtual machines on a private cloud platform, characterized in that: Load the eBPF program on the host machine to monitor the client's write I / O requests in real time, and continuously back up these write I / O requests through the backup program in the user space to achieve backup, no longer relying on the snapshot function of the virtualization platform itself; The process includes: S1. Start the backup service of the transmission unit, load the eBPF program and initialize the configuration file; S2, create and initialize shared memory; S3. Load the eBPF program on the host and attach it to the relevant function that handles virtio-blk write IO requests. S4. The eBPF program intercepts the write IO of the virtio-blk device in the host and parses the data parameters; S5. The eBPF program encapsulates the write IO request and adds the encapsulated write IO request data structure to the shared memory. S6. The backup service of the transmission unit reads the data from the shared memory and transmits it to the storage unit; S7, create a snapshot; In S1, the backup service is responsible for starting, managing and monitoring the entire backup process. The service listens to the data intercepted by the eBPF program, continuously transfers the data in the shared memory to the backup target, and performs error handling and flow control.

2. The method for real-time agentless backup of virtual machines on a private cloud platform according to claim 1, characterized in that: In S1, eBPF is a small program running in the Linux kernel for network monitoring, performance analysis and debugging tasks; eBPF programs are attached to specific hooks in the kernel or user space, interact with the kernel through specific interfaces, perform tasks and access data between the kernel and user space, and are attached to specific functions or system calls through user space probe uprobes to capture detailed information about specific processes or kernel operations. In virtualization scenarios, eBPF is used to track and analyze interactions between clients and hosts. KVM is a virtualization function provided by the Linux kernel, which allows the host to run multiple clients, and the clients interact with the host through virtual devices; The S1 specifically includes the following steps: S11. Start the backup service of the transmission unit and perform a service self-check to confirm that all dependencies and environment variables are correctly configured to ensure that the service has the basic conditions for performing backup operations; S12. Use tools to write and load eBPF programs. S13. Initialize the configuration file, which contains various parameters and settings required in the backup process, so as to be used in the subsequent backup process.

3. The method for real-time agentless backup of virtual machines on a private cloud platform according to claim 1, characterized in that: In S2, the shared memory acts as a data transmission medium to temporarily store or record the write IO request data of the virtio-blk device intercepted by the eBPF program from the host machine. The shared memory includes Buffer and Bitmap. The buffer is used to temporarily store the write IO request data structure intercepted and encapsulated by the eBPF program. The data structure includes source disk information, IO offset position, data length and business data content; The disk bitmap is used to assist in recording the changes of data blocks on the client disk. It maintains a bit for each data block, and one bit corresponds to one data block. The size of the data block can be flexibly set according to the actual situation. Its value is used to indicate whether the corresponding disk data block has changed and needs to be synchronized to the target backup storage. 0 means that the data block corresponding to this bit does not need to be synchronized, and 1 means that the data block corresponding to this bit needs to be synchronized; The S2 specifically includes the following steps: S21. Create a shared memory area and allocate a large enough memory space for the shared memory on the host. This space will be used to store buffers and bitmaps. Ensure that both the eBPF program and the backup service have access to this shared memory area. The size of the buffer is determined based on the business characteristics of the virtual machine, the expected scale of write IO requests, and the processing capacity of the backup service; The size of the disk bitmap is determined by the data block division strategy of the client disk. The total number of data blocks divided by the disk is calculated according to the total capacity of the disk and the set data block size, and then the number of bits required for the Bitmap is obtained; S22, initialize the buffer, initialize the allocated buffer memory space, set each field to the initial default value, and ensure that the data structure in the buffer is in a known and predictable state in the initial state; S23, initialize Bitmap, and initialize all bits of Bitmap to 0, which means that in the initial state, all data blocks on the default disk have not changed, and no synchronous backup operation is required. The values ​​of these bits will be dynamically updated according to the actual write IO events later.

4. The method for real-time agentless backup of virtual machines of a private cloud platform according to claim 1, characterized in that: In S3, virtio-blk is a virtualized block device interface used for disk I / O communication between the client and the host. The client operating system accesses the virtual disk through the virtio-blk driver to perform read and write operations; virtio-blk is a block device virtualization driver based on VirtIO. It provides clients with a block device access interface similar to a physical hard disk. By using virtio-blk, clients can perform random storage access like accessing ordinary disks, thus supporting the creation and management of file systems and the application of database storage. The working principle of virtio-blk is as follows: virtio-blk uses a ring buffer and shared memory mechanism to achieve efficient data exchange. The client initiates an I / O request to the virtio-blk device, the client driver transfers the data to the shared memory, and the host reads the data from the shared memory and operates it.

5. The agentless real-time backup method for virtual machines of a private cloud platform according to claim 1, characterized in that: In S4, the eBPF program can be attached to a QEMU-related function or code segment to capture detailed information about processing a client virtio-blk I / O request in the QEMU process; QEMU is a user space process responsible for handling client I / O requests, especially those related to virtio-blk devices. Through uprobes, the eBPF program is attached to the relevant functions in QEMU responsible for handling virtio-blk I / O requests. The eBPF program can capture function calls in the QEMU process, access local variables and function parameters in QEMU memory, and parse the written offset, length, device information, and written data. In S4, the IO between the client and the host is intercepted by eBPF, and the specific implementation steps include: S41. Use uprobes to attach the eBPF program to the relevant functions in QEMU responsible for handling virtio-blk I / O requests. S42, capturing the offset, length, device path and written data; S43. Use tools to write and load eBPF programs; S44. Accurate interaction with QEMU process through uprobes.

6. The method for real-time agentless backup of virtual machines of a private cloud platform according to claim 1, characterized in that: In the S5, there are two scenarios: one is full backup or data blocks that need to be retransmitted due to an exception, and the other is incremental backup of only business IO; When performing a full backup or when an exception occurs and a data block needs to be retransmitted, the specific steps are as follows: S51, if it is a full backup, set the bit values ​​of all bits of the disk bitmap to 1. If there is an abnormal data block that needs to be retransmitted, only the bit value corresponding to the data block needs to be set to 1; S52, eBPF captures service IO in real time, and the backup service of the transmission unit is processed in order from small to large offset positions. Assume that the position currently being processed is recorded as offsetmiddle. At this time, new IO is generated at different offsets; If the new IO generated by the business is on the left side of the disk offsetmiddle, the position is recorded as offsetleft, and the new data is recorded in the buffer. If the buffer is full at this time, it is updated in the Bitmap and the bit position corresponding to the data block is set to 1; If the new IO generated by the business is to the right of the disk offsetmiddle, the position is recorded as offsetright, and the new data is not recorded in the buffer, and no other processing is required; If the new IO generated by the business happens to fall at the offsetmiddle position, the data being transmitted is recorded as datamiddle, and the new data is recorded as datamiddle′, then the old IO data is discarded and the new IO data of the data block corresponding to the bit position is retransmitted; S53, it is determined that the data blocks corresponding to all the bits of the Bitmap have been successfully synchronized to the target backup storage, that is, all the bits are 0, and the transmission unit obtains data from the buffer for synchronization, and the creation of the snapshot is allowed at this time; If there are unsynchronized data blocks in the Bitmap, the data blocks with bit 1 in the Bitmap will continue to be processed. To ensure the consistency of the backup data, snapshot creation needs to be disabled at this time; When performing incremental data synchronization, only the intercepted business IO needs to be processed. The specific steps are as follows: S51', determine whether the buffer is full, if the buffer is not full, execute S52', if the buffer is full, execute S53'; S52′, write the service IO data captured by eBPF in real time into the buffer; S53′, update the Bitmap, and set the bit position corresponding to the business IO data block captured in real time by eBPF to 1; S54′, the backup service of the transmission unit continuously obtains data from the buffer and synchronizes it to the target backup storage; S55′, updating the data block marked as 1 in the disk bitmap Bitmap to the buffer. Similarly, when there is a data block marked as 1 in the bit position of the Bitmap, in order to ensure the consistency of the backup data, the snapshot creation is disabled at this time; The business IO data intercepted by the eBPF program will first be recorded in the Buffer. If the amount of business IO data is huge or the processing speed of the transmission unit is relatively slow, the Buffer may be full. Once the Buffer is full, the subsequent changes in the business IO data blocks will be updated to the Bitmap; If there is an exception that causes a data block to fail in transmission, it will also be updated and recorded in the Bitmap. When the Buffer is free, the data block marked as 1 in the Bitmap will be placed in the Buffer, and the transmission unit backup service will continue to obtain data from the Buffer and synchronize it.

7. The method for real-time agentless backup of virtual machines of a private cloud platform according to claim 1, characterized in that: In S3, S4, and S5, the processing flow of writing IO is as follows: A. The application in the client initiates a write operation through the file system; B. The virtio-blk driver in the client receives and encapsulates the write request and puts it into the virtqueue; C. Write IO requests are passed from the client to QEMU via virtqueue; D. QEMU decodes the request, determines the offset and data for the write operation, and prepares the data for writing. E. Data is written to the storage backend; F. After QEMU completes the write operation, it notifies the client through interruption or polling.

8. The agentless real-time backup method for virtual machines of a private cloud platform according to claim 1, characterized in that: The S7 specifically includes the following steps: S71, determine whether there are data blocks to be synchronized in the disk bitmap, that is, whether there are data blocks with a bit marked as 1. If no bit is marked as 1, the snapshot is allowed to be created, and S72 is executed. If a bit is marked as 1, the snapshot is not allowed to be created, and the task is terminated. S72, inserting a snapshot instruction mark into the buffer according to the backup task plan; S73, the transmission unit synchronizes the data before the snapshot instruction mark in the buffer to the target backup storage; S74, notifying the target end to create a snapshot; S75. The target end creates a snapshot after receiving the request to create a snapshot.

9. An agentless real-time backup device for virtual machines of a private cloud platform, characterized in that: It includes an eBPF service IO interception unit, a Buffer buffer, a disk Bitmap bitmap, a transmission unit, a storage unit and a snapshot creation unit inserted between the IO channel of the virtual machine VM and the host machine.

10. The agentless real-time backup device for virtual machines of a private cloud platform according to claim 9, characterized in that: The eBPF service IO interception unit is responsible for loading the eBPF program on the host machine to track and analyze the interaction between the client and the host machine to intercept service IO operations; eBPF programs can be attached to QEMU-related functions or code segments, capture function calls in the QEMU process, access local variables and function parameters in QEMU memory, and parse device information, written offsets, lengths, and written data; The Buffer buffer is used to cache the business IO that needs to be transmitted and is intercepted by the eBPF program, so as to backup service transmission; The disk Bitmap is used to record the change status of disk data blocks. 0 indicates no change, and 1 indicates change. The bitmap is used to record which blocks have been changed. When the buffer is full, the data block where the business IO is located, or the data block that needs to be retransmitted due to an exception, the corresponding bit in its Bitmap is marked as 1; The backup service of the transmission unit is responsible for obtaining data from the Buffer buffer and transmitting it to the backup storage; The storage unit refers to the destination of the backup data, which is responsible for receiving the data sent by the transmission unit and storing it persistently; The snapshot creation unit is responsible for creating a snapshot of the data in the storage unit at a specific time point, and can capture the state of the virtual machine at a certain moment, including its file system and data, so as to restore the data to the historical state when the snapshot was created when necessary.

Images