Virtual machine non-intrusive real-time copying method, device and equipment based on eBPF

Abstract

The invention relates to an eBPF-based virtual machine non-intrusive real-time copying method, device and equipment. The method comprises the steps that a user mode probe captures an IO context, generates a serial number associated IO context and stores the serial number associated IO context in an eBPF serial number main table; the kernel mode probe intercepts the write operation, and looks up the IO context by using the thread identifier; and reading write data corresponding to the IO context, writing the write data into the annular buffer area in a fragmented manner, and reading the data from the annular buffer area to recombine the complete IO. The device comprises a context capture module, a mapping storage module, a write-in interception module, a context matching module, a data transmission module and a data recombination module. According to the method, reading and transmission paths of large-scale data are completely completed in the kernel mode probe, zero-copy transmission is achieved in combination with the eBPF annular buffer area, the context switching overhead is remarkably reduced, and the system keeps extremely low performance loss and higher stability in a high-concurrency and large-data-volume writing scene.

Description

Technical Field

[0001] This invention belongs to the field of disaster recovery and backup technology, and relates to a non-intrusive real-time replication method, apparatus, computer equipment, computer storage medium and computer program product based on eBPF. Background Technology

[0002] With the widespread application of cloud computing and virtualization technologies, real-time replication of virtual machines has gradually become an important means of ensuring business continuity and data security. Real-time replication refers to capturing and transmitting disk write data and related context information generated by a virtual machine in real time during its operation to a backup device. This allows for rapid recovery of the virtual machine's running state in the event of a host failure, achieving continuous protection with near-zero data loss. Traditional real-time replication solutions typically rely on modifying the source code of agents or hypervisors, which often suffers from high system invasiveness and poor compatibility.

[0003] eBPF (extended Berkeley Packet Filter), a highly efficient programmable framework developed in the Linux kernel in recent years, has been widely used in performance monitoring, security detection, and I / O interception. Through eBPF, developers can dynamically inject lightweight probe logic into the kernel without modifying the kernel source code, enabling data interaction between kernel mode and user mode. Kernel-mode pointer technology allows probes to be attached at kernel function entry points to capture process behavior in kernel mode; while user-mode probe technology allows probes to be attached at user-mode binary function entry points to intercept context information of specific application function calls. The combination of these two technologies makes cross-user-mode and kernel-mode data path monitoring possible.

[0004] In the prior art, a method and apparatus for agentless real-time backup of virtual machines on a private cloud platform (application number CN120196397A) is disclosed. This solution attaches a user-space pointer probe to the virtio-blk device IO request function of QEMU to obtain context information such as the write offset and data length of the virtual machine, and combines it with an eBPF ring buffer to transmit the captured data to the user-space application in real time, thereby realizing agentless real-time backup of virtual machine disk data. This type of solution avoids deploying agent programs within the virtual machine or modifying the QEMU source code, and has a certain degree of non-intrusiveness and versatility.

[0005] However, this solution still has the following shortcomings: First, its monitoring points only rely on the virtio-blk device IO request function, which has limited scalability and is difficult to adapt to various storage interfaces and complex virtualization scenarios; Second, large-scale data is directly captured and transmitted through user-space probes, resulting in frequent switching between kernel mode and user mode, which brings high performance overhead and is prone to performance degradation or unstable data transmission in high-concurrency, high-volume write scenarios.

[0006] Therefore, there is an urgent need for a new real-time virtual machine replication technology based on eBPF that can be compatible with multiple storage interfaces and complex virtualization scenarios, and can reduce the performance overhead in the data capture and transmission process, thereby realizing a high-performance and versatile real-time virtual machine replication solution. Summary of the Invention

[0007] In order to solve the technical problems in the background art, the present invention provides a non-intrusive real-time copying method, apparatus, computer device, computer storage medium and computer program product based on eBPF for virtual machines.

[0008] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: The first aspect provides a non-intrusive real-time replication method for virtual machines based on eBPF, the method comprising: The context capture step involves attaching a user-space probe to the QEMU block device I / O function to capture the virtual machine I / O context information. The mapping and storage step involves assigning a unique sequence number to each captured virtual machine IO, storing the sequence number and IO context information in the eBPF sequence number master table, obtaining the thread identifier that initiated the IO operation, and storing the mapping relationship between the thread identifier and the sequence number in the eBPF thread index table. The write interception step involves attaching a kernel-mode probe to the system call function written on the host machine to intercept QEMU write operations; In the context matching step, the kernel-mode probe queries the eBPF thread index table with the current thread identifier, and then locates the write operation context in the eBPF sequence number main table. If a match is found, the data transfer step is executed first, and then the system original write operation is executed; otherwise, the system original write operation is executed directly. The data transmission process involves reading data segment by segment from the user-space buffer of the write operation according to the preset fragment size. Each time a data segment of the fragment size is read, a corresponding data fragment is constructed and immediately written to the eBPF circular buffer until the complete data processing is finished. In the data reconstruction step, the user-space application receives data fragments from the eBPF circular buffer and then reconstructs and restores the complete virtual machine I / O data based on the sequence number and fragment information.

[0009] Secondly, a non-intrusive real-time virtual machine replication device based on eBPF is provided, comprising: The context capture module is used to attach user-space probes to QEMU block device I / O functions to capture virtual machine I / O context information; The mapping storage module is used to assign a unique sequence number to each captured virtual machine IO, store the sequence number and IO context information in the eBPF sequence number master table; obtain the thread identifier that initiated the IO operation, and store the mapping relationship between the thread identifier and the sequence number in the eBPF thread index table. The write interception module is used to attach a kernel-mode probe to the system call function written on the host machine to intercept QEMU write operations; The context matching module is used by the kernel-mode probe to query the eBPF thread index table with the current thread identifier, and then locate the write operation context in the eBPF sequence number main table. If a match is found, the data transfer module is executed first, and then the system original write operation is executed; otherwise, the system original write operation is executed directly. The data transmission module is used to read data segment by segment from the user-space buffer of the write operation according to the preset fragment size. Each time a data segment of the fragment size is read, the corresponding data fragment is constructed and immediately written into the eBPF circular buffer until the complete data processing is completed. The data reassembly module is used by user-space applications to receive data fragments from the eBPF ring buffer and then reassemble and restore complete virtual machine I / O data based on the sequence number and fragment information.

[0010] Thirdly, a computer device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the non-intrusive real-time copying method for virtual machines based on eBPF as described in any of the preceding claims.

[0011] Fourthly, a computer-readable storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the non-intrusive real-time copying method for virtual machines based on eBPF as described above.

[0012] Fifthly, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the non-intrusive real-time copying method for virtual machines based on eBPF as described in any of the preceding claims.

[0013] The beneficial effects of this invention are: (1) This invention places the reading and transmission path of large-scale data entirely in the kernel-mode probe, and combines the eBPF ring buffer to achieve zero-copy transmission, which significantly reduces the context switching overhead and enables the system to maintain extremely low performance loss and higher stability in high-concurrency, large-scale data writing scenarios. (2) This invention deploys probes at two points: the QEMU general block device layer IO function and the host machine's underlying write system call. This enables indiscriminate coverage of various file and block backend formats, and has complete device independence and format independence. This greatly improves the versatility of the solution and its adaptability to complex virtualization scenarios. (3) This invention constructs a precise mapping from thread to IO context through the eBPF serial number master table and the thread index table, and only intercepts and transmits data for virtual machine IO write operations, effectively avoiding invalid data processing and circular buffer pollution, and significantly improving data processing efficiency and system resource utilization. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of the non-intrusive real-time virtual machine replication method based on eBPF in Embodiment 1 of the present invention.

[0016] Figure 2 This is a schematic diagram of the user-state probe hook position in Embodiment 1 of the present invention.

[0017] Figure 3 This is a schematic diagram of the eBPF serial number master table and eBPF thread index table structure in Embodiment 1 of the present invention.

[0018] Figure 4 This is a schematic diagram of the data sharding structure in Embodiment 1 of the present invention.

[0019] Figure 5 This is a structural diagram of the non-intrusive real-time virtual machine replication device based on eBPF in Embodiment 2 of the present invention.

[0020] Figure 6 This is a structural diagram of the mapping storage module in Embodiment 2 of the present invention.

[0021] Figure 7 This is a structural diagram of the context matching module in Embodiment 2 of the present invention.

[0022] Figure 8 This is a structural diagram of the data transmission module in Embodiment 2 of the present invention.

[0023] Figure 9 This is a schematic diagram of the structure of the electronic device provided in Embodiment 3 of the present invention.

[0024] The attached diagram lists the components represented by each number as follows: 201. Context capture module; 202. Mapped storage module; 203. Write interception module; 204. Context matching module; 205. Data transmission module; 206. Data reconstruction module; 2021. Serial number generation unit; 2022. Context encapsulation unit; 2023. Thread mapping unit; 2041. Thread identifier acquisition unit; 2042. Context processing unit; 2051. Segment calculation unit; 2052. Segment reading unit; 2053. Segment encapsulation unit; 2054. Segment writing unit; 2055. Loop execution unit; 310. Processor; 320. Communication interface; 330. Memory; 340. Communication bus. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0026] Example 1

[0027] Existing technologies disclose a method and apparatus for agentless real-time backup of virtual machines on a private cloud platform. This solution attaches a user-space pointer probe to the QEMU virtio-blk device I / O request function to obtain context information such as the virtual machine's write offset and data length. Combined with an eBPF circular buffer, the captured data is transmitted to the user-space application in real time, achieving agentless real-time backup of virtual machine disk data. This type of solution avoids deploying agent programs within the virtual machine or modifying the QEMU source code, exhibiting a degree of non-intrusiveness and versatility.

[0028] However, this solution still has the following shortcomings: First, its monitoring points only rely on the virtio-blk device IO request function, which has limited scalability and is difficult to adapt to various storage interfaces and complex virtualization scenarios; Second, large-scale data is directly captured and transmitted through user-space probes, resulting in frequent switching between kernel mode and user mode, which brings high performance overhead and is prone to performance degradation or unstable data transmission in high-concurrency, high-volume write scenarios.

[0029] To address the aforementioned issues, this invention provides a non-intrusive real-time virtual machine replication method based on eBPF. Figure 1 This is a schematic flowchart of a non-intrusive real-time replication method for virtual machines based on eBPF, provided in an embodiment of the present invention. The method provided in this embodiment is used in the following environment: the virtualization platform is QEMU, and the operating system platform is Ubuntu. like Figure 1As shown, a non-intrusive real-time replication method for virtual machines based on eBPF is described, which includes: Step S101: Attach a user-space probe to the block device I / O function of QEMU to capture the context information of virtual machine I / O. The context information includes: logical offset, data length, virtual device identifier, and timestamp.

[0030] As is understandable, the context information includes four key fields: the logical offset, which indicates the logical address location of the data in the virtual device; the data length, which specifies the number of bytes written; the virtual device identifier, which uniquely identifies the virtual block device to which the I / O belongs; and the timestamp, which records the precise moment when QEMU initiated the I / O. These four key fields ensure that the reconstructed data can be accurately restored to its original logical location in the target replica, thus achieving high-fidelity real-time replication.

[0031] Preferably, step S101 further includes: S1011, intercepts QEMU's block device I / O function by hooking the user-space probe mechanism based on eBPF, specifically the intercepted function is blk_aio_pwritev; S1012 extracts the logical offset, data length, virtual device identifier, and timestamp from the intercepted IO request.

[0032] It is worth noting that, such as Figure 2 As shown, this embodiment selects the block device IO function blk_aio_pwritev as the hook point because this function is located in the block device backend layer. The block device backend layer is located between the virtual device layer and the block device state layer. It is the only necessary path after all front-end virtual devices submit write requests and before all backend image format processing. It is independent of the front-end device type and the backend image format. A single hook can completely cover the write operations of virtual devices such as virtio-blk, virtio-scsi, and IDE, as well as various virtual disks such as raw, qcow2, and vmdk. At this time, the accurate logical offset, data length, and virtual device identifier can still be directly obtained. It has the best versatility and the lowest implementation and maintenance cost. Therefore, it is the optimal position for capturing virtual machine IO context information.

[0033] Step S102: Assign a unique sequence number to each captured virtual machine IO, and store the sequence number and IO context information in the eBPF sequence number master table; obtain the thread identifier that initiated the IO operation, and store the mapping relationship between the thread identifier and the sequence number in the eBPF thread index table.

[0034] Understandably, the unique sequence number serves as an association identifier, playing a crucial role in establishing context throughout the entire replication process. The unique sequence number links the virtual machine I / O metadata captured in user space with the actual write operation data subsequently intercepted in kernel space, ensuring that every I / O request issued by the virtual machine can be correctly identified, tracked, and reassembled.

[0035] It's also understandable that in the eBPF thread index table, the thread identifier and sequence number have a one-to-one or one-to-many mapping relationship. When a thread initiates I / O serially, it's a one-to-one relationship; when a thread initiates I / O in parallel, it's a one-to-many relationship. This relationship provides an efficient query basis for subsequent context matching, enabling precise matching between write operations and raw I / O.

[0036] Optionally, step S102 further includes: S1021, Generate a globally unique sequence number for each captured virtual machine I / O based on the timestamp and virtual device identifier; S1022, using the above serial number as the key, the corresponding context information is stored as a data item in the eBPF serial number master table, and the context information in the eBPF serial number master table is encapsulated in a structured data format; S1023: Obtain the thread identifier currently performing the IO operation, and query the eBPF thread index table to see if a mapping entry has been established for the thread identifier; if a mapping entry already exists for the thread identifier, append the newly generated sequence number to the sequence number array corresponding to the thread identifier; if no mapping entry exists for the thread identifier, create a new mapping relationship, using the thread identifier as the key and the array containing the current sequence number as the value for storage.

[0037] Understandably, a globally unique sequence number is generated for each captured virtual machine I / O operation to ensure unique identification between different I / O operations. This sequence number is unique throughout the entire system's lifecycle, not only distinguishing I / O operations initiated by the same virtual machine at different times, but also effectively identifying concurrent I / O requests from different virtual machines. This strong uniqueness ensures that in complex, high-concurrency virtualization environments, any captured I / O context and its corresponding data fragment can be unambiguously identified and associated.

[0038] It is worth noting that, such as Figure 3As shown, the eBPF sequence number main table uses the sequence number as the key and a custom structure as the value to store the complete context information of each virtual machine I / O. The custom structure contains the following fields: logical offset, data length, virtual device identifier, and timestamp. The eBPF thread index table uses the thread identifier as the key and a dynamic array as the value to maintain one or more sequence numbers that the thread is currently processing. This is used to quickly locate candidate I / O requests in the kernel-mode write probe by the current thread, thereby achieving a precise cross-layer association between user-mode virtual machine I / O and actual kernel-mode disk write operations.

[0039] Step S103: Attach a kernel-mode probe to the write system call function of the QEMU host operating system to intercept write operations initiated by the QEMU process.

[0040] It is worth noting that, in order to ensure accurate interception of the actual disk write operations of the QEMU process, and to avoid mistakenly capturing the write behavior of other processes on the host machine, this embodiment uses the thread identifier of the current thread to precisely match the QEMU process in the kernel-mode eBPF probe, and only processes the write operations initiated by QEMU.

[0041] In step S104, the kernel-mode probe queries the eBPF thread index table using the current thread identifier, and then locates the write operation context in the eBPF sequence number master table. If a match is found, step S105 is executed first, followed by the system original write operation; otherwise, the system original write operation is executed directly.

[0042] Optionally, step S104 further includes: Step S1041: When the kernel-mode probe intercepts a write system call, obtain the thread identifier of the currently executing thread; In step S1042, the associated sequence number array is queried in the eBPF thread index table using the thread identifier as the key. Each sequence number in the array is traversed, and the corresponding virtual machine IO context information is retrieved in the eBPF sequence number master table. If a sequence number matching the target write operation is found, its corresponding context information is saved for subsequent processing in step S105, and then the original system write operation is executed. If no sequence number matching the write operation is found, the original system write operation is executed directly.

[0043] It is worth noting that when the kernel-mode probe intercepts a write system call, this embodiment first obtains the current thread identifier and uses it as the key to perform a lookup in the eBPF thread index table. If there is an IO operation being traced, the corresponding sequence number array is obtained. Then, the current write operation context obtained by the kernel-mode probe is compared one by one with the IO context corresponding to each sequence number in the sequence number array in the eBPF sequence number master table. If a match is found, the data transmission step is executed. If there is no associated entry or the comparison fails, the original write operation is executed directly, thereby avoiding the unnecessary analysis overhead of irrelevant write operations.

[0044] This two-level table structure dynamically associates threads with I / O contexts: the thread index table stores only a lightweight sequence number array, serving a fast filtering function; the main sequence number table maintains complete virtual machine I / O metadata. In this way, the system only needs to process pre-marked I / O streams, concentrating computational resources on valuable scenarios, significantly reducing the overhead of context switching, memory traversal, and invalid matches. Simultaneously, leveraging the kernel-mode execution characteristics of eBPF avoids frequent data copying between user mode and kernel mode, thereby improving overall data processing efficiency and throughput.

[0045] Preferably, step S104 further includes step S1043: after the write operation is completed, the used sequence number is removed from the sequence number array of the eBPF thread index table. If the array becomes empty as a result, the entire mapping entry of the thread identifier is deleted.

[0046] This "execute first, clean up later" order ensures the atomicity and fault tolerance of I / O operations—even if the write operation fails, the cleanup logic can still be executed correctly, while avoiding race conditions or data inconsistency problems that may be caused by prematurely releasing resources before I / O is completed.

[0047] It should be noted that if no matching sequence number for the write operation is found, there is no need to execute step S1043, because there is no cleanup object.

[0048] Step S105: Read data segment by segment from the user-space buffer of the write operation according to the preset fragment size. For each data segment of the fragment size read, construct the corresponding data fragment and write it to the eBPF circular buffer immediately until the complete data processing is completed.

[0049] Optionally, step S105 further includes: S1051, determine the total length of the data to be processed based on the context information of the successful match, and calculate the total number of fragments according to the preset fragment size; S1052, based on the current read offset position, read data segments sequentially from the user-mode buffer according to the preset segment size, and update the read offset position to the starting position of the next data segment to be processed; S1053, construct the corresponding data fragment for the currently read data segment, such as Figure 4 As shown, the structure of a data shard includes at least the following information: sequence number, shard number, total number of shards, virtual machine IO context information, and IO data of the current shard; S1054 After writing the constructed data fragment structure into the eBPF circular buffer, update the read offset position of the user-space buffer. S1055, Repeat steps S1051 to S1054 until all data has been processed.

[0050] It is worth noting that the information contained in the data shard is the core to achieving accurate data shard transmission and subsequent complete reassembly. The sequence number is a globally unique identifier for this IO operation, used to associate all shards of the same task; the shard sequence number and the total number of shards define the position and total quantity of the current shard, ensuring the reassembly order; the virtual machine IO context information carries location and timing parameters, supporting accurate restoration on the replica side; the IO data of the current shard is the core business carrier of the replication. The above fields together constitute a data unit that can be independently identified and reassembled in an orderly manner.

[0051] It is also worth noting that after a target write operation is matched, the core purpose of prioritizing the execution of fragment construction and ring buffer transfer is to achieve parallelization of data collection and IO execution paths, thereby hiding transmission latency and reducing performance loss.

[0052] Specifically, the fragmentation operation divides the complete I / O context information into multiple data blocks of a preset size, allowing them to tightly fill consecutive slots in the circular buffer, thus achieving batch aggregation and transmission. The circular buffer, as a shared memory area between kernel and user space, allows user-space consumers to asynchronously read in batches without triggering a separate system call for each I / O event. If the original system write operation is executed first, the entire process degenerates into a serial mode—transmission cannot start until the write operation is complete, causing the kernel space to wait for user space processing to finish before releasing resources, frequently triggering context switches and synchronization overhead. However, by triggering fragmented transmission first, the I / O waiting time during the original write operation, such as disk scheduling and network packet transmission, can be used to asynchronously push the fragmented data to user space. The overlapping time windows of the two operations effectively remove the cost of tracking data collection from the critical path.

[0053] This embodiment is particularly suitable for high-concurrency scenarios: multi-threaded virtual machine write operations can be accurately captured and written to the eBPF circular buffer in batches. User space interacts in real time through polling but reads buffer data in batches without processing each piece, which significantly reduces the frequency of switching between kernel space and user space and multi-threaded lock contention, and avoids the additional performance loss caused by IO monitoring / copying.

[0054] It is also worth noting that the eBPF program stack space is limited to 512 bytes, while virtual machine I / O data is typically an integer multiple of 4096 bytes. To balance stack safety and integer fragmentation, this embodiment preferably uses a fragment size of 256 bytes. This keeps the single fragment structure within 512 bytes to avoid overflow, and is also divisible by 4KB, achieving efficient and concise uniform fragmentation.

[0055] In step S106, the user-space application receives data fragments from the eBPF circular buffer and then reassembles and restores the complete virtual machine I / O data based on the sequence number and fragment information.

[0056] Optionally, step S106 further includes: S1061, the user-space application receives data fragments from the eBPF ring buffer, then distinguishes and processes data fragments from different virtual machines based on the context information in the data fragments, and distributes the fragments to the corresponding independent reassembly queues. S1062, within each independent reassembly queue, the reassembly order is determined based on the data fragment sequence number and data fragment information, and the data fragments are sorted and spliced ​​in sequence to restore the complete virtual machine IO data.

[0057] It is worth noting that this step involves the user-space application receiving data fragments pushed by the eBPF ring buffer in real time using an event-driven approach, avoiding CPU idle time caused by busy-wait polling. The application automatically distributes data fragments from different virtual machines to their respective independent reassembly queues based on the virtual device identifiers carried in the fragments. Within each queue, the fragments are sorted and concatenated according to their sequence number and fragment sequence number, quickly and accurately reconstructing the original complete IO write operation of each virtual machine, thereby achieving efficient and low-overhead real-time replication in multi-virtual machine concurrent scenarios.

[0058] This embodiment places the reading and transmission path of large-scale data entirely within the kernel-mode probe, and combines it with the eBPF ring buffer to achieve zero-copy transmission, significantly reducing context switching overhead. This allows the system to maintain extremely low performance loss and higher stability in high-concurrency, large-volume write scenarios. At the same time, by deploying probes at two points—the QEMU general block device layer IO function and the host machine's underlying write system call—it achieves indiscriminate coverage of various file and block backend formats, exhibiting complete device independence and format independence. This greatly improves the versatility of the solution and its adaptability to complex virtualization scenarios.

[0059] Example 2

[0060] In this embodiment, as Figure 5 As shown, a non-intrusive real-time virtual machine replication device based on eBPF is provided, the device comprising: The context capture module 201 is used to attach user-mode probes to QEMU block device I / O functions to capture virtual machine I / O context information; The mapping storage module 202 assigns a unique sequence number to each captured virtual machine IO, stores the sequence number and IO context information in the eBPF sequence number master table, obtains the thread identifier that initiated the IO operation, and stores the mapping relationship between the thread identifier and the sequence number in the eBPF thread index table. The write interception module 203 is used to attach a kernel-mode probe to the host machine's write system call function to intercept QEMU write operations; The context matching module 204 is used by the kernel-mode probe to query the eBPF thread index table with the current thread identifier, and then locate the write operation context in the eBPF sequence number main table. If a match is found, the data transmission step is executed first, and then the system original write operation is executed; otherwise, the system original write operation is executed directly. The data transmission module 205 is used to read data segment by segment from the user-space buffer of the write operation according to the preset fragment size. Each time a data segment of the fragment size is read, the corresponding data fragment is constructed and immediately written into the eBPF circular buffer until the complete data processing is completed. The data reconstruction module 206 is used by user-space applications to receive data fragments from the eBPF ring buffer and then reconstruct and restore complete virtual machine I / O data based on the sequence number and fragment information.

[0061] like Figure 6 As shown, in one embodiment, the mapping storage module 202 further includes: The serial number generation unit 2021 is used to generate a globally unique serial number for each captured virtual machine I / O based on the timestamp and the virtual device identifier; The context encapsulation unit 2022 is used to store the corresponding context information as a data item in the eBPF serial number master table using the serial number as the key value. The stored context information is encapsulated in a structured data format. The thread mapping unit 2023 is used to obtain the thread identifier currently performing IO operation, and to query the eBPF thread index table to see if a mapping entry has been established for the thread identifier. If a mapping entry already exists for the thread identifier, the newly generated sequence number is appended to the sequence number array corresponding to the thread identifier. If no mapping entry exists for the thread identifier, a new mapping relationship is created, with the thread identifier as the key and the array containing the current sequence number as the value for storage.

[0062] like Figure 7 As shown, in one embodiment, the context matching module 204 further includes: The thread identifier acquisition unit 2041 is used to acquire the thread identifier of the currently executing thread when the kernel-mode probe intercepts a write system call. The context processing unit 2042 is used to query the associated sequence number array in the eBPF thread index table with the thread identifier as the key, traverse each sequence number in the array and retrieve the corresponding virtual machine IO context information in the eBPF sequence number master table. If a sequence number matching the target write operation is found, its corresponding context information is saved for subsequent data transmission module 205 to process and then the original system write operation is executed; if no sequence number matching the write operation is found, the original system write operation is executed directly.

[0063] like Figure 8 As shown, in one embodiment, the data transmission module 205 includes: The fragment calculation unit 2051 is used to determine the total length of the data to be processed based on the context information of the successful matching of the target write operation, and to calculate the total number of fragments according to the preset fragment size. The segmented reading unit 2052 is used to read data segments sequentially from the user-mode buffer according to a preset segment size based on the current reading offset position, and update the reading offset position to the starting position of the next data segment to be processed; The fragment encapsulation unit 2053 is used to construct a corresponding data fragment for the currently read data segment. The data fragment contains at least the following information: sequence number, fragment sequence number, total number of fragments, virtual machine IO context information, and IO data of the current fragment. The fragmented write unit 2054 is used to update the read offset position of the user-mode buffer after writing the constructed data fragments into the eBPF circular buffer. The loop execution unit 2055 is used to repeatedly run the slice read unit 2052, the slice encapsulation unit 2053 and the slice write unit 2054 until all data is processed, and then the original system write operation is executed.

[0064] Below, we provide some comparative experiments to further illustrate this embodiment, as follows: All experiments were run on a virtual machine in a VMware ESXi managed virtualization environment. The virtual machine operating system was Ubuntu 22.04, kernel version 6.8.0-57-generic. Table 1 shows the specific parameters of the test environment.

[0065] Table 1 Test Environment Information

[0066] In this experiment, the test objects were this embodiment and the comparative example, which were designed and implemented according to the patent document "A Method and Apparatus for Agentless Real-Time Backup of Virtual Machines on a Private Cloud Platform". The test item was: the impact on virtual machine I / O performance when the services of both were enabled.

[0067] In this experiment, the fio tool was used. Under the experimental environment described in Table 1, the performance of random writes was compared between this embodiment and the comparative example. The random write data sizes were 100M, 500M, 1G, 5G and 10G. The results are shown in Table 2.

[0068] Table 2 Comparison of the impact of this embodiment and the comparative embodiment on virtual machine I / O performance.

[0069] As shown in Table 2, as the data volume increases from 100 MB to 10 GB, the virtual machine IO rate in this embodiment is consistently higher than that in the comparative embodiment, and the advantage increases with scale: leading by 105 MB / s (9.9%) at 100 MB, leading by 144 MB / s (14.3%) at 500 MB, leading by 192 MB / s (20.2%) at 1 GB, leading by 257 MB / s (29.4%) at 5 GB, and leading by 301 MB / s at 10 GB, an improvement of 36.9%. This indicates that the scheme in this embodiment can significantly reduce the drag on IO performance caused by real-time replication, and the benefits are more obvious with larger scales; although different experimental environments may bring numerical fluctuations, comparative tests confirm that its efficiency improvement trend is stable, verifying the universality and adaptability of the scheme to large-scale data processing.

[0070] The non-intrusive real-time virtual machine replication device based on eBPF provided in this embodiment constructs a precise mapping from thread to IO context through the eBPF sequence number master table and thread index table. It intercepts and transmits data only for virtual machine IO write operations, effectively avoiding invalid data processing and circular buffer pollution, and significantly improving data processing efficiency and system resource utilization.

[0071] Example 3

[0072] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 9 As shown, the electronic device may include a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor, communication interface, and memory communicate with each other through the communication bus. The processor can call logical instructions in the memory to execute the non-intrusive real-time virtual machine copying method based on eBPF as described in Embodiment 1.

[0073] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] Example 4

[0075] In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it implements the non-intrusive real-time copying method of virtual machines based on eBPF as described in Embodiment 1.

[0076] Example 5

[0077] In this embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the non-intrusive real-time copying method for virtual machines based on eBPF as described in Embodiment 1.

[0078] The computer storage medium in this invention embodiment can take any combination of one or more computer-readable media. Computer-readable media can be divided into two categories: signal-type media and physical storage media. Physical storage media specifically refers to physical devices that carry data in physical form, and their implementation is based on electromagnetic field effects, photoelectric conversion characteristics, semiconductor material characteristics, or a fusion of the above technologies. Specifically, the forms of physical storage media include, but are not limited to, the following technical forms (non-limiting list): such as hard disk drives, solid-state drives, optical disks, magnetic tapes, flash memory chips, memory cards, USB devices, EEPROMs, and eMMC or UFS memories in embedded systems. These media can all store program code in a tangible way and can be recognized and read by computing devices to realize the installation, loading, or execution of programs. Whether integrated inside a device or connected externally, any medium that can stably store and run programs can be considered a qualified physical storage medium.

[0079] Computer signal media can carry data signals propagated in baseband or as part of a carrier wave, containing executable computer program code. The forms of propagated signals include, but are not limited to, electromagnetic signals, optical signals, or any combination thereof. Computer-readable signal media can also be media other than storage media, capable of transmitting or sending programs for use by instruction execution systems, devices, or equipment.

[0080] The transmission of program code can be achieved through various media, including but not limited to wireless networks, wired connections, fiber optic cables, radio frequency signals, or any combination thereof. Program code can be written in one or more programming languages, including object-oriented languages ​​(such as Python, Java, and C++) and procedural languages ​​(such as C and Go). The program code can be executed entirely on the local computer, partially locally, as a standalone software package, partially locally and partially on a remote server, or entirely on a remote server. When remote execution is involved, a connection to the remote computer can be made through any network, including a local area network (LAN), a wide area network (WAN), or the Internet.

[0081] The description of this embodiment is merely an example of some implementations of the present invention, intended to illustrate the implementation path of the technical solution, and not to limit the scope of protection of the present invention. For those skilled in the art, various improvements and adjustments can be made without departing from the technical concept of the present invention, and these improvements and adjustments should all be considered within the scope of protection of the present invention. The scope of protection of this patent is determined by the claims.

Claims

1. A non-intrusive real-time virtual machine replication method based on eBPF, characterized in that, include: The context capture step involves attaching a user-space probe to the QEMU block device I / O function to capture the virtual machine I / O context information. The mapping and storage step involves assigning a unique sequence number to each captured virtual machine IO, storing the sequence number and IO context information in the eBPF sequence number master table, obtaining the thread identifier that initiated the IO operation, and storing the mapping relationship between the thread identifier and the sequence number in the eBPF thread index table. The write interception step involves attaching a kernel-mode probe to the system call function written on the host machine to intercept QEMU write operations; In the context matching step, the kernel-mode probe queries the eBPF thread index table with the current thread identifier, and then locates the write operation context in the eBPF sequence number main table. If a match is found, the data transfer step is executed first, and then the system original write operation is executed; otherwise, the system original write operation is executed directly. The data transmission process involves reading data segment by segment from the user-space buffer of the write operation according to the preset fragment size. Each time a data segment of the fragment size is read, a corresponding data fragment is constructed and immediately written to the eBPF circular buffer until the complete data processing is finished. In the data reconstruction step, the user-space application receives data fragments from the eBPF circular buffer and then reconstructs and restores the complete virtual machine I / O data based on the sequence number and fragment information.

2. The non-intrusive real-time virtual machine replication method based on eBPF according to claim 1, characterized in that, The mapping and storage step further includes: Generate a globally unique sequence number for each captured virtual machine I / O based on the timestamp and virtual device identifier in the context information; Using the aforementioned serial number as the key, the corresponding context information is stored as a data item in the eBPF serial number master table, and the stored context information is encapsulated in a structured data format. Get the thread identifier currently performing the I / O operation. Check the eBPF thread index table to see if a mapping entry has been established for this thread identifier. If a mapping entry already exists for this thread identifier, append the newly generated sequence number to the sequence number array corresponding to this thread identifier. If no mapping entry exists for this thread identifier, create a new mapping relationship, using the thread identifier as the key and the array containing the current sequence number as the value for storage.

3. The non-intrusive real-time virtual machine replication method based on eBPF according to claim 1 or 2, characterized in that, The context matching step further includes: When the kernel-mode probe intercepts a write system call, it obtains the thread identifier of the currently executing thread. Using the thread identifier as the key, the system queries the associated sequence number array in the eBPF thread index table. It then iterates through each sequence number in the array and retrieves the corresponding virtual machine I / O context information from the eBPF sequence number master table. If a sequence number matching the target write operation is found, its corresponding context information is saved for subsequent data transmission steps, and the original system write operation is executed. If no matching sequence number is found, the original system write operation is executed directly.

4. The non-intrusive real-time virtual machine replication method based on eBPF according to claim 3, characterized in that, The context matching step further includes: After the write operation is completed, the used sequence number is removed from the sequence number array of the eBPF thread index table, and the entire mapping entry for the thread identifier is deleted when the sequence number array becomes empty.

5. The non-intrusive real-time virtual machine replication method based on eBPF according to claim 1 or 2, characterized in that, The data transmission step further includes: The total length of the data to be processed is determined based on the context information of the successful matching of the target write operation, and the total number of fragments is calculated according to the preset fragment size. Based on the current read offset position, read data segments sequentially from the user-space buffer according to the preset segment size, and update the read offset position to the starting position of the next data segment to be processed; Construct a corresponding data fragment for the currently read data segment. The data fragment must contain at least the following information: sequence number, fragment sequence number, total number of fragments, virtual machine IO context information, and IO data of the current fragment. After writing the constructed data fragments into the eBPF circular buffer, update the read offset position of the user-space buffer; Repeat the loop of reading, constructing, writing, and offset updating until all data has been processed.

6. The non-intrusive real-time virtual machine replication method based on eBPF according to claim 1 or 2, characterized in that, The data reconstruction step further includes: User-space applications receive data fragments from the eBPF circular buffer, then differentiate and process data fragments from different virtual machines based on the context information in the data fragments, and distribute the fragments to the corresponding independent reassembly queues. Within each independent reassembly queue, the reassembly order is determined based on the data fragment sequence number and data fragment information, and the data fragments are sorted and spliced ​​in sequence to restore the complete virtual machine IO data.

7. A non-intrusive real-time virtual machine replication device based on eBPF, characterized in that, Including the following steps: The context capture module is used to attach user-space probes to QEMU block device I / O functions to capture virtual machine I / O context information; The mapping storage module is used to assign a unique sequence number to each captured virtual machine IO, and store the sequence number and IO context information in the eBPF sequence number master table; Obtain the thread identifier that initiated the IO operation, and store the mapping relationship between the thread identifier and the sequence number in the eBPF thread index table; The write interception module is used to attach a kernel-mode probe to the system call function written on the host machine to intercept QEMU write operations; The context matching module is used by the kernel-mode probe to query the eBPF thread index table with the current thread identifier, and then locate the write operation context in the eBPF sequence number main table. If a match is found, the data transfer module is executed first, and then the system original write operation is executed; otherwise, the system original write operation is executed directly. The data transmission module is used to read data segment by segment from the user-space buffer of the write operation according to the preset fragment size. Each time a data segment of the fragment size is read, the corresponding data fragment is constructed and immediately written into the eBPF circular buffer until the complete data processing is completed. The data reassembly module is used by user-space applications to receive data fragments from the eBPF ring buffer and then reassemble and restore complete virtual machine I / O data based on the sequence number and fragment information.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the non-intrusive real-time replication method for virtual machines based on eBPF as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the non-intrusive real-time virtual machine replication method based on eBPF as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of non-intrusive real-time copying of virtual machines based on eBPF as described in any one of claims 1 to 6.

Images