Block-level data real-time backup method and device based on eBPF

Abstract

The invention discloses a block-level data real-time backup method based on eBPF. The method specifically comprises the following steps: starting a data replication service; an eBPF program is loaded, and configuration is initialized; the method comprises the following steps: creating and initializing an io data cache ring buffer; the method comprises the following steps of: registering an eBPF (evolved Business Program Format) program to a submitter trace point; an ebpf program intercepts bio, and a bio target disk object is analyzed; judging whether the bio target disk is a backup disk or not; packaging the io operation type, data, offset, length and target disk information according to the bio structural body, and writing the information into ring buffer; the copying service receives and transmits the data; according to the method, the eBPF technology is utilized, the method can be universally used on different Linux-based operating system release versions and kernel versions, adaptation for a specific environment is not needed, the method is not limited by a specific kernel version, dependence on a specific operating system is reduced, the workload and complexity of adaptation of different kernel versions are reduced, and the method has universal applicability and is suitable for popularization and application. And the eBPF program is executed in a kernel mode without modifying a kernel source code or recompiling a kernel, so that the requirement of UEFI secure startup is met.

Description

Technical Field

[0001] The present invention relates to the technical field of data protection, and in particular to a method and device for real-time block-level data backup based on eBPF. Background Art

[0002] With the rapid development of modern information technology, the importance of data has become increasingly prominent. Data backup has become a key measure to ensure data security. In the Linux system, disk block-level backup is one of the key means to ensure data security and reliability. Traditional disk block-level backup methods usually intercept IO operations through disk filter drivers and perform backup processing. However, with the diversification of Linux distribution versions and the continuous update of kernel code, this traditional method faces many challenges. At the same time, the widespread use of UEFI secure boot has further exacerbated these problems. In recent years, with the maturity and wide application of eBPF technology, using eBPF technology to intercept IO operations for data backup processing has become a better solution. eBPF allows new functions to be dynamically added to existing systems, so it can be used as a lightweight way to add new service features without restarting the server or recompiling the kernel. The eBPF infrastructure not only enhances the functionality and flexibility of the Linux kernel, but also provides a powerful and secure platform for developers to build innovative solutions;

[0003] In traditional disk block-level backup methods, disk filter drivers need to be adapted to different operating systems and kernel versions, resulting in large development and testing workloads and complex maintenance. Each kernel update may require re-adapting the driver, increasing the long-term maintenance cost. And with the widespread use of UEFI secure boot, all code loaded at system startup must be digitally signed and verified. As a kernel module, traditional disk filter drivers may not be loaded without being signed, thus limiting their use in production environments. Even if the driver passes the signature verification, due to the strictness of UEFI secure boot, any modification to the kernel or driver may trigger security warnings or cause the system to fail to start. Summary of the Invention

[0004] The present invention provides a method and device for real-time block-level data backup based on eBPF, which can effectively solve the problems proposed in the above background technology. The disk filter driver needs to be adapted to different operating systems and kernel versions, resulting in a large amount of development and testing work and complex maintenance. Each kernel update may require re-adapting the driver, increasing the long-term maintenance cost. Moreover, with the widespread use of UEFI secure boot, all code loaded during system startup must be digitally signed and verified. As a traditional disk filter driver is a kernel module, it may not be loaded without being signed, thus limiting its use in the production environment. Even if the driver passes the signature verification, due to the strictness of UEFI secure boot, any modification to the kernel or driver may trigger a security warning or cause the system to fail to start.

[0005] To achieve the above object, the present invention provides the following technical solution: A method for real-time block-level data backup based on eBPF uses eBPF to hook into the key function calls of the block device layer to capture the data about to be written to the disk. When a write operation is detected, the eBPF program intercepts this data and sends it to a pre-configured backup target. At the same time, the original write operation continues, ensuring that the application is not interfered with. Specifically, it includes the following steps:

[0006] S1. Start the data replication service module, and call the eBPF program loading and initialization module to load the eBPF program and perform initialization configuration;

[0007] S2. After the eBPF program loading and initialization module completes the loading, call the IO data cache module to create and initialize a ring buffer;

[0008] S3. The eBPF system call management module registers the eBPF program to the submit_bio trace point;

[0009] S4. The bio capture and analysis module captures the submit_bio event in the eBPF program and analyzes the bio target disk object;

[0010] S5. If the ring buffer resources are insufficient or an error is detected, record the IO that has not been successfully captured in the Bitmap;

[0011] S6. Determine whether the target disk is the disk to be backed up. If so, encapsulate the relevant information of the IO operation; otherwise, ignore the bio operation;

[0012] S7. The data transmission module writes the encapsulated IO data into the ring buffer and notifies the data replication service module to read the data from the ring buffer and Bitmap for transmission;

[0013] In S1, the data replication service module is the core control unit of the entire backup system, responsible for starting, managing, and monitoring the entire backup process. This service listens for data from the eBPF program, transfers the data to the backup target according to the configuration record, and performs error handling and traffic control. At the same time, the data replication service module listens for data from the eBPF program, transfers the data to the backup target according to the configuration record, and performs error handling and traffic control;

[0014] After the data replication service module is started, it will first perform a series of self-checks, including resource availability checks and configuration parameter readings. These configuration parameters include the backup target path, source disk information, and backup strategy. The backup strategy includes full backup, incremental backup, and differential backup;

[0015] In S2, the IO data cache module is a component responsible for caching the data captured from disk IO operations. Since disk IO operations are usually frequent and the data volume is large, directly processing these data may have a greater impact on system performance. Therefore, a caching mechanism needs to be introduced to temporarily store these data for subsequent efficient processing;

[0016] In S3, the eBPF system call management module is responsible for registering the eBPF program to a specific trace point to ensure that the eBPF program can intercept bio requests at the correct time. Based on the ebpf infrastructure, a kprobe hook point is defined. submit_bio is a key function in the Linux kernel, which is responsible for submitting I / O requests to the block device layer. A kprobe hook will be set at the entrance of this function. The eBPF program is triggered every time submit_bio is called, and the actual I / O buffer address and size, and the information of the target device are extracted from the bio;

[0017] In S4, the bio interception and analysis module obtains information about the IO operation by parsing the bio structure in the submit_bio event, including the target disk object, data size, offset length, and operation type. Among them, the operation type is read and write operations.

[0018] According to the above technical solution, in S1, eBPF is an efficient and dynamically loadable kernel-level programming technology that allows user-space programs to execute custom code in the kernel space without modifying the kernel source code or recompiling the kernel. The eBPF program loading and initialization module is responsible for loading the eBPF-based program into the kernel and performing necessary initialization configurations to ensure that the eBPF program can correctly interact with the kernel;

[0019] eBPF programs are written in user space and then loaded into the kernel for execution. The kernel contains a dedicated eBPF virtual machine that can interpret these programs. Most modern architectures also support just-in-time compilation, which converts eBPF bytecode into native machine code to improve performance. eBPF programs have strict restrictions, and even if a program has defects, it will not cause the system to crash or be maliciously exploited. eBPF programs can interact with the kernel by calling helper functions provided by the kernel;

[0020] An important feature of eBPF is mapping, which allows data to be shared between user space and kernel space. Mapping is a key-value store that can be used to pass information, cache data, or implement cross-process communication. At the same time, eBPF supports multiple types of programs, and each type has its specific usage scenarios and interfaces;

[0021] Before an eBPF program is executed, the validator in the kernel checks the program to ensure that it follows all rules and meets security requirements, preventing potential security vulnerabilities. There is a complete set of toolchains and libraries around eBPF, which simplifies the development process and enables developers to more conveniently create, debug, and deploy eBPF programs;

[0022] In S1, it specifically includes the following steps:

[0023] S11. Start the data replication service module, check whether the system resources meet the requirements of the backup task. The system resources specifically include memory, CPU, and disk space to avoid task failure due to insufficient resources during the backup process, and read the specific parameters of the backup task from the configuration file or database, including the backup source and target, backup strategy, and logging level;

[0024] S12. Call the eBPF program loading and initialization module to load the compiled eBPF program into the kernel, specifying the eBPF program type and the kernel function or event to which it is to be attached;

[0025] S13. After the eBPF program is loaded, perform initialization configuration, set the necessary data structures, specifically the map table and the ring buffer, and configure the parameters of the eBPF program for subsequent data capture and processing;

[0026] S14. Complete initialization and status monitoring, start a status monitoring thread to monitor the running status of the eBPF program and the data replication service in real time, and promptly detect and handle exceptions.

[0027] According to the above technical solution, in S2, it specifically includes the following steps:

[0028] S21. Create a ring buffer as a temporary storage area for IO data, define the size and the number of buffers of the ring buffer, and the size and the number of buffers of the ring buffer can be set according to the actual requirements and resource conditions of the system;

[0029] S22. Configure the initialization parameters of the ring buffer to control the read and write operations of the ring buffer and data overflow handling;

[0030] S23. Start a status monitoring thread to monitor the usage of the ring buffer in real time, including the used space and the remaining space, which helps to detect and handle potential memory leaks and overflow problems in a timely manner.

[0031] According to the above technical solution, in S3, it specifically includes the following steps:

[0032] S31. Identify the definition and trigger conditions of the submit_bio event point in the kernel source code. submit_bio is an event point in the Linux kernel used to submit block IO operations. When the upper-layer file system or block device layer submits an IO request to the underlying block device, the submit_bio event will be triggered. Since this event point is directly related to data read and write operations, it can be used to capture block-level data changes;

[0033] S32. Use the interface provided by the eBPF system call management module to register the eBPF program to the submit_bio event point, and the type of the eBPF program and the kernel function or event to which it is to be attached need to be specified;

[0034] S33. Start a status monitoring thread to monitor the running status and performance metrics of the eBPF program in real time. These include key metrics such as the memory occupancy, CPU occupancy rate, and event capture rate of the program. By monitoring these key metrics, potential performance bottlenecks or abnormal behaviors can be detected and handled in a timely manner.

[0035] According to the above technical solution, in S4, it specifically includes the following steps:

[0036] S41. Capture the submit_bio event. In the eBPF program, the submit_bio event is captured through the hook function registered to the submit_bio event point. When the submit_bio event is triggered, the hook function will be called and a pointer to the bio structure will be passed as a parameter;

[0037] S42. When obtaining the bio structure, the content of the bio structure is obtained through the passed pointer. This structure contains all the information about the IO operation, specifically including the target disk object, the data buffer pointer, and the data size.

[0038] S43. Analyze the target disk object, including target disk identification, operation type judgment, and data size analysis. Identify the target disk object of the IO operation by parsing the target disk field in the bio structure, distinguish different disks and partitions for subsequent backup processing.

[0039] Judge whether the IO operation is a read operation or a write operation according to the operation type field in the bio structure. Backup pays more attention to the write operation of data. Obtain the data size of the IO operation by parsing the data size field in the bio structure, which is used to evaluate the scale of the backup task and the required time resources.

[0040] According to the above technical solution, in S5, the Bitmap disk bitmap is a data structure used to record the IOs that were not successfully intercepted due to insufficient ringbuffer resources or detection of errors. 1 indicates that there are un-intercepted IOs in this data block, and 0 indicates that all IOs in this data block were successfully intercepted.

[0041] When the data replication service performs data transmission, it is necessary to re-transmit the data blocks marked as 1 in the Bitmap. When the Bitmap module traces the submit_bio function in the eBPF program, it records the I / O requests when encountering errors or abnormal situations through the bitmap to effectively identify and manage these error messages, providing metadata information for subsequent backup and recovery.

[0042] According to the above technical solution, in S6, it specifically includes the following steps:

[0043] S61. Obtain the list of disks to be backed up. This list is passed in from the user space during the initialization of the eBPF program, or dynamically obtained through configuration files and environment variables during runtime.

[0044] S62. Match the disks to be backed up. Compare the bio target disk with the device numbers or file paths in the backup disk list to make a backup decision. If the target disk is a disk to be backed up, perform backup processing; otherwise, ignore this bio operation.

[0045] S63. Package the IO operation information. Further extract the IO operation information in the submit_bio event through the eBPF program, including the type, start address, data length, and timestamp of the IO operation. Package the extracted IO operation information into a structure or object for subsequent processing and transmission.

[0046] According to the above technical solution, in S7, it specifically includes the following steps:

[0047] S71. Use the bpf_ringbuf_reserve function to allocate space from the ring buffer, write the encapsulated IO operation information into the allocated space, and then use the bpf_ringbuf_submit function to submit the data to the ring buffer;

[0048] S72. Notify the data replication service module to read data from the ring buffer. The notification signal contains the address of the ringbuffer and the length of the written data;

[0049] S73. Read and transfer data. The data replication service module will read the encapsulated IO operation information from the ring buffer and further process the read IO operation information according to the backup policy, specifically including format conversion, compression, and encryption, to improve the efficiency and security of data transmission;

[0050] S74. Transmit the processed data to the backup storage system or remote backup server for storage and retention;

[0051] S75. Error handling and recovery. During the entire data transmission process, the data replication service module will perform error detection and recovery processing. When an error or abnormal situation is detected, measures will be taken for recovery or retry, and detailed log information will be recorded for troubleshooting and positioning in case of problems. In the log information, configuration operation logs, backup process logs, backup status, and abnormal situation alerts of resource usage are recorded.

[0052] A block-level data real-time backup device based on eBPF includes a data replication service module, an eBPF system call management module, an IO operation encapsulation module, an eBPF program loading and initialization module, a bio capture and analysis module, a data transmission module, an IO data cache module, a backup disk judgment module, and an error handling and logging module.

[0053] According to the above technical solution, the data replication service module is the core control unit of the entire backup system, responsible for starting, managing, and monitoring the entire backup process. The data replication service module listens to data from the eBPF program, transfers the data to the backup target according to the configuration record, and performs error handling and traffic control;

[0054] The eBPF system call management module is responsible for registering the eBPF program to a specific trace point to ensure that the eBPF program can intercept bio requests at the correct timing;

[0055] The IO operation encapsulation module refers to encapsulating IO operation information. By using an eBPF program, it further extracts the IO operation information in the submit_bio event, and encapsulates the extracted IO operation information into a structure or object for subsequent processing and transmission;

[0056] The eBPF program loading and initialization module is responsible for loading the eBPF-based program into the kernel and performing necessary initialization configurations to ensure that the eBPF program can correctly interact with the kernel;

[0057] The bio capture and analysis module captures the submit_bio event in the eBPF program, analyzes the bio target disk object, and obtains information about the IO operation by parsing the bio structure in the submit_bio event;

[0058] The data transmission module writes the encapsulated IO data into the ring buffer and notifies the data replication service module to read the data from the ring buffer and Bitmap for transmission;

[0059] The IO data caching module is a component responsible for caching the data captured from disk IO operations. By introducing a caching mechanism, it temporarily stores this data for subsequent efficient processing;

[0060] The backup disk judgment module is used to judge whether the target disk is a disk to be backed up;

[0061] In the entire data transmission process, the data replication service module in the error handling and logging module will perform error detection and recovery processing. When an error or abnormal situation is detected, it will take measures for recovery or retry and record detailed log information.

[0062] Compared with the prior art, the beneficial effects of the present invention are:

[0063] 1. By using eBPF technology, it can be universal on different Linux-based operating system distributions and kernel versions, without the need for adaptation to a specific environment, not restricted by a specific kernel version, reducing the dependence on a specific operating system, reducing the workload and complexity of adapting different kernel versions, having general applicability, and the eBPF program executes in the kernel state, but without modifying the kernel source code or recompiling the kernel, meeting the requirements of UEFI secure boot. At the same time, the disk block device backup based on eBPF can effectively avoid the performance bottlenecks brought by traditional backup methods and also maintain high flexibility and scalability.

[0064] 2. The block-level data real-time backup method and device based on eBPF are not restricted by specific kernel versions, reducing the dependence on specific operating systems. This not only simplifies the development, testing, and maintenance work but also improves the real-time performance and efficiency of backup. At the same time, it enhances the generality and security of system backup, providing an efficient, flexible, and secure solution for data protection under the Linux system.

[0065] 3. The disk block device backup technology based on eBPF can achieve real-time backup of disk data by intercepting and replicating the data stream written to the block device without changing the existing storage architecture. eBPF is a kernel technology that allows users to write secure and high-performance program fragments without modifying the kernel code. These programs can be executed at different positions in the kernel. After being JIT-compiled into machine code, eBPF programs run directly on the CPU, providing the characteristics of low latency and high throughput. In addition, eBPF also supports complex data structures, making it very suitable for building backup systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation to the present invention.

[0067] In the drawings:

[0068] Figure 1 is the step flowchart of the backup method of the present invention;

[0069] Figure 2 is the structural schematic diagram of the backup device of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] The following describes the preferred embodiments of the present invention with reference to the drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0071] Embodiment: As Figure 1 shown, the present invention provides a technical solution, a block-level data real-time backup method based on eBPF. By using eBPF to hook into the key function calls of the block device layer to capture the data about to be written to the disk. When a write operation is detected, the eBPF program will intercept this data and send it to a pre-configured backup target, which can be a local file, a remote server, or other persistent storage media. At the same time, the original write operation continues, ensuring that the application program is not interfered with. Specifically, it includes the following steps:

[0072] S1. Start the data replication service module, and call the eBPF program loading and initialization module to load the eBPF program and perform initialization configuration;

[0073] After the S2, eBPF program loading and initialization module finishes loading, it calls the IO data cache module to create and initialize a ring buffer;

[0074] In S3, the eBPF system call management module registers the eBPF program to the submit_bio trace point;

[0075] In S4, the bio interception and analysis module intercepts the submit_bio event in the eBPF program and analyzes the bio target disk object;

[0076] In S5, if the ring buffer resource is insufficient or an error is detected, the unsuccessfully intercepted IOs are recorded in the Bitmap;

[0077] In S6, it is judged whether the target disk is the disk to be backed up. If so, the relevant information of the IO operation is encapsulated. Otherwise, the bio operation is ignored;

[0078] In S7, the data transmission module writes the encapsulated IO data into the ring buffer and notifies the data replication service module to read the data from the ring buffer and Bitmap for transmission;

[0079] In S1, the data replication service module is the core control unit of the entire backup system, responsible for starting, managing, and monitoring the entire backup process. This service listens to the data from the eBPF program, transmits the data to the backup target according to the configuration record, and performs error handling and traffic control. At the same time, the data replication service module listens to the data from the eBPF program, transmits the data to the backup target according to the configuration record, and performs error handling and traffic control;

[0080] After the data replication service module starts, it will first perform a series of self-checks, including resource availability checks and configuration parameter readings. These configuration parameters include the backup target path, source disk information, and backup strategy. The backup strategy includes full backup, incremental backup, and differential backup;

[0081] In S2, the IO data cache module is a component responsible for caching the data captured from disk IO operations. Since disk IO operations are usually frequent and the data volume is large, directly processing these data may have a greater impact on system performance. Therefore, a caching mechanism needs to be introduced to temporarily store these data for subsequent efficient processing;

[0082] In S3, the eBPF system call management module is responsible for registering the eBPF program to a specific trace point, such as submit_bio, to ensure that the eBPF program can intercept bio requests at the correct time. Based on the ebpf infrastructure, a kprobe hook point is defined. submit_bio is a key function in the Linux kernel, which is responsible for submitting I / O requests to the block device layer. A kprobe hook will be set at the entrance of this function. The eBPF program is triggered every time submit_bio is called, and extracts the actual I / O buffer address and size, and information about the target device from the bio;

[0083] In S4, the bio interception and analysis module obtains information about the IO operation by parsing the bio structure in the submit_bio event, including the target disk object, data size, offset length, and operation type. Among them, the operation types are read and write operations.

[0084] Based on the above technical solution, in S1, eBPF is an efficient and dynamically loadable kernel-level programming technology that allows user-space programs to execute custom code in the kernel space without modifying the kernel source code or recompiling the kernel. The eBPF program loading and initialization module is responsible for loading the eBPF-based program into the kernel and performing necessary initialization configurations to ensure that the eBPF program can interact correctly with the kernel;

[0085] The eBPF program is written in user space and then loaded into the kernel to run. The kernel contains a dedicated eBPF virtual machine that can interpret these programs. Most modern architectures also support just-in-time compilation, which converts the eBPF bytecode into native machine code to improve performance. eBPF programs have strict limitations, such as they must terminate within a limited time, that is, they cannot have infinite loops, and can only access specific data structures. Even if the program is defective, it will not cause the system to crash or be maliciously exploited. The eBPF program can interact with the kernel by calling auxiliary functions provided by the kernel, and these auxiliary functions provide capabilities such as accessing packet content, querying timestamps, and operating on maps;

[0086] An important feature of eBPF is mapping, which allows sharing data between user space and kernel space. Mapping is a key-value store that can be used to transfer information, cache data, or implement cross-process communication. At the same time, eBPF supports multiple types of programs, and each type has its specific usage scenarios and interfaces. Common types include socket filters, trace points, kernel / user space probes, cgroup programs, and XD Express Data Path;

[0087] Before the eBPF program is executed, the validator in the kernel checks the program to ensure that it follows all rules and meets security requirements, preventing potential security vulnerabilities. There is a complete set of toolchains and libraries around eBPF, such as bcc (BPF Compiler Collection), libbpf, and bpftool, which simplify the development process and enable developers to create, debug, and deploy eBPF programs more conveniently.

[0088] In S1, it specifically includes the following steps:

[0089] S11. Start the data replication service module, check whether the system resources meet the requirements of the backup task. The system resources specifically include memory, CPU, and disk space, to avoid task failure due to insufficient resources during the backup process, and read the specific parameters of the backup task from the configuration file or database, including the backup source and target, backup strategy, and logging level.

[0090] S12. Call the eBPF program loading and initialization module to load the compiled eBPF program into the kernel, specify the eBPF program type and the kernel function or event to which it is to be attached. The specified eBPF program types include KPROBE and TRACEPOINT.

[0091] S13. After the eBPF program is loaded, perform initialization configuration, set the necessary data structures, specifically the map (mapping table) and ring buffer, and configure the parameters of the eBPF program for subsequent data capture and processing.

[0092] S14. Complete initialization and status monitoring, start a status monitoring thread to monitor the running status of the eBPF program and the data replication service in real time, and detect and handle exceptions in a timely manner.

[0093] Based on the above technical solution, in S2, it specifically includes the following steps:

[0094] S21. Create a ring buffer as a temporary storage area for IO data, define the size and the number of buffers of the ring buffer. The size and the number of buffers of the ring buffer can be set according to the actual system requirements and resource conditions.

[0095] S22. Configure the initialization parameters of the ring buffer, such as the maximum number of entries and buffer size, to control the read and write operations of the ring buffer and data overflow handling.

[0096] S23. Start a status monitoring thread to monitor the usage of the ring buffer in real time, including the used space and the remaining space, which helps to detect and handle potential memory leaks and overflows in a timely manner.

[0097] Based on the above technical solution, in S3, it specifically includes the following steps:

[0098] S31. Identify the definition and triggering conditions of the submit_bio event point in the kernel source code. submit_bio is an event point in the Linux kernel used to submit block I / O operations. When the upper-layer file system or block device layer submits an I / O request to the underlying block device, the submit_bio event will be triggered. Since this event point is directly related to data read and write operations, it can be used to capture block-level data changes.

[0099] S32. Use the interface provided by the eBPF system call management module to register the eBPF program to the submit_bio event point. It is necessary to specify the type of the eBPF program and the kernel function or event to which it is to be attached. Among them, the type of the eBPF program is such as the KPROBE type, which is used to capture the calls of kernel functions, and the kernel function is the submit_bio function.

[0100] S33. Start a status monitoring thread to monitor the running status and performance metrics of the eBPF program in real time. These include key metrics such as the memory occupancy, CPU occupancy rate, and event capture rate of the program. By monitoring these key metrics, potential performance bottlenecks or abnormal behaviors can be detected and handled in a timely manner.

[0101] Based on the above technical solution, in S4, it specifically includes the following steps:

[0102] S41. Capture the submit_bio event. In the eBPF program, the submit_bio event is captured through the hook function registered to the submit_bio event point. When the submit_bio event is triggered, the hook function will be called and a pointer to the bio structure will be passed as a parameter.

[0103] S42. When obtaining the bio structure, obtain the content of the bio structure through the passed pointer. This structure contains all the information about the I / O operation, specifically including the target disk object, the data buffer pointer, and the data size.

[0104] S43. Analyze the target disk object, including target disk identification, operation type judgment, and data size analysis. By parsing the target disk field in the bio structure, identify the target disk object of the I / O operation. The target disk field is such as the device number and partition number, which are used to distinguish different disks and partitions for subsequent backup processing.

[0105] Determine whether the IO operation is a read operation or a write operation according to the operation type field in the bio structure. The backup pays more attention to the write operation of data. Obtain the data size of the IO operation by parsing the data size field in the bio structure, which is used to evaluate the scale of the backup task and the required time resources.

[0106] Based on the above technical solution, in S5, the Bitmap disk bitmap is a data structure used to record the IO that was not successfully intercepted due to insufficient ringbuffer resources or detected errors. 1 indicates that there is un-intercepted IO for this data block, and 0 indicates that all IOs for this data block were successfully intercepted;

[0107] When the data replication service performs data transmission, it is necessary to retransmit the data blocks marked as 1 in the Bitmap. When the Bitmap module traces the submit_bio function in the eBPF program, it records the I / O requests encountered during errors or abnormal situations through the bitmap to effectively identify and manage these error messages, providing metadata information for subsequent backup and recovery.

[0108] Based on the above technical solution, in S6, it specifically includes the following steps:

[0109] S61. Obtain the list of disks to be backed up. This list is passed in from the user space during the initialization of the eBPF program, or dynamically obtained at runtime through configuration files and environment variables;

[0110] S62. Match the disks to be backed up. Compare the bio target disk with the device numbers or file paths in the backup disk list to make a backup decision. If the target disk is a disk to be backed up, perform backup processing; otherwise, ignore this bio operation;

[0111] S63. Package the IO operation information. Further extract the IO operation information in the submit_bio event through the eBPF program, including the type, starting address, data length, and timestamp of the IO operation. Package the extracted IO operation information into a structure or object for subsequent processing and transmission.

[0112] Based on the above technical solution, in S7, it specifically includes the following steps:

[0113] S71. Use the bpf_ringbuf_reserve function to allocate space from the ring buffer, write the packaged IO operation information into the allocated space, and then use the bpf_ringbuf_submit function to submit the data to the ring buffer;

[0114] S72. Notify the data replication service module to read data from the ring buffer. The notification signal contains the address of the ring buffer and the length of the written data;

[0115] S73. Read and transmit data. The data replication service module reads the encapsulated IO operation information from the ring buffer and further processes the read IO operation information according to the backup policy, including format conversion, compression, and encryption, to improve the efficiency and security of data transmission;

[0116] S74. Transmit the processed data to the backup storage system or remote backup server for storage and retention;

[0117] S75. Error handling and recovery. During the entire data transmission process, the data replication service module performs error detection and recovery processing. When an error or abnormal situation is detected, such as data corruption or transmission failure, measures are taken for recovery or retry, and detailed log information is recorded for troubleshooting and positioning in case of problems. In the log information, configuration operation logs, backup process logs, backup status, and alerts for abnormal resource usage are recorded.

[0118] As Figure 2 shown, a block-level data real-time backup device based on eBPF includes a data replication service module, an eBPF system call management module, an IO operation encapsulation module, an eBPF program loading and initialization module, a bio capture and analysis module, a data transmission module, an IO data cache module, a backup disk judgment module, and an error handling and logging module.

[0119] Based on the above technical solution, the data replication service module is the core control unit of the entire backup system, responsible for starting, managing, and monitoring the entire backup process. The data replication service module listens to data from the eBPF program, transmits the data to the backup target according to the configuration record, and performs error handling and traffic control;

[0120] The eBPF system call management module is responsible for registering the eBPF program to a specific trace point to ensure that the eBPF program can intercept bio requests at the correct timing;

[0121] The IO operation encapsulation module refers to encapsulating IO operation information, further extracting the IO operation information in the submit_bio event through the eBPF program, and encapsulating the extracted IO operation information into a structure or object for subsequent processing and transmission;

[0122] The eBPF program loading and initialization module is responsible for loading the eBPF-based program into the kernel and performing necessary initialization configurations to ensure that the eBPF program can interact with the kernel correctly;

[0123] The bio capture and analysis module captures the submit_bio event in the eBPF program, analyzes the bio target disk object, and obtains information about the IO operation by parsing the bio structure in the submit_bio event;

[0124] The data transmission module writes the encapsulated IO data into the ring buffer and notifies the data replication service module to read the data from the ring buffer and Bitmap for transmission;

[0125] The IO data caching module is a component responsible for caching the data captured from disk IO operations. By introducing a caching mechanism, it temporarily stores this data for subsequent efficient processing;

[0126] The backup disk judgment module is used to judge whether the target disk is the disk to be backed up. If so, it encapsulates the relevant information of the IO operation; otherwise, it ignores the bio operation;

[0127] During the entire data transmission process, the data replication service module in the error handling and logging module will perform error detection and recovery processing. When an error or abnormal situation is detected, it will take measures to recover or retry and record detailed log information.

[0128] Finally, it should be noted that the above are only preferred examples of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A block-level data real-time backup method based on eBPF, characterized in that: Use eBPF to hook into key function calls of the block device layer to capture the data to be written to the disk. When a write operation is detected, the eBPF program will intercept the data and send it to the pre-configured backup target. At the same time, the original write operation continues to ensure that the application is not disturbed. The specific steps include the following: S1. Start the data replication service module, call the eBPF program loading and initialization module to load the eBPF program and perform initialization configuration; S2. After the eBPF program loading and initialization module is loaded, the IO data cache module is called to create and initialize the ringbuffer; S3. The eBPF system call management module registers the eBPF program to the submit_bio trace point. S4, the bio interception and analysis module intercepts the submit_bio event in the eBPF program and analyzes the bio target disk object; S5. If the ring buffer resources are insufficient or a detection error occurs, the IO that was not successfully intercepted will be recorded in the Bitmap; S6, determine whether the target disk is a disk to be backed up, if so, encapsulate the relevant information of the IO operation, otherwise, ignore the bio operation; S7, the data transmission module writes the encapsulated IO data into the ring buffer, and notifies the data replication service module to read data from the ring buffer and Bitmap for transmission; In S1, the data replication service module is the core control unit of the entire backup system, responsible for starting, managing and monitoring the entire backup process. The service listens to data from the eBPF program, transfers the data to the backup target according to the configuration record, and performs error handling and flow control. At the same time, the data replication service module listens to data from the eBPF program, transfers the data to the backup target according to the configuration record, and performs error handling and flow control; After the data replication service module is started, it will first perform a series of self-tests, including resource availability checks and configuration parameter reading. These configuration parameters include the backup target path, source disk information, and backup strategies, among which the backup strategies include full backup, incremental backup, and differential backup. In S2, the IO data cache module is a component responsible for caching data captured from disk IO operations, and introduces a cache mechanism to temporarily store this data for efficient subsequent processing; In S3, the eBPF system call management module is responsible for registering the eBPF program to a specific trace point to ensure that the eBPF program can intercept the bio request at the right time. Based on the ebpf infrastructure, a kprobe hook point is defined. submit_bio is a key function in the Linux kernel, which is responsible for submitting I / O requests to the block device layer. A kprobe hook will be set at the entrance of the function. The eBPF program is triggered each time submit_bio is called, and the actual I / O buffer address and size, and target device information are extracted from bio. In S4, the bio interception and analysis module obtains information about the IO operation by parsing the bio structure in the submit_bio event, including the target disk object, data size, offset length, and operation type, wherein the operation type is read and write operation.

2. The block-level data real-time backup method based on eBPF according to claim 1 is characterized in that: In S1, eBPF is an efficient, dynamically loadable kernel-level programming technology that allows user-mode programs to execute custom code in kernel mode without modifying kernel source code or recompiling the kernel. The eBPF program loading and initialization module is responsible for loading eBPF-based programs into the kernel and performing necessary initialization configuration to ensure that the eBPF program can interact with the kernel correctly. eBPF programs are written in user space and then loaded into the kernel to run. The kernel contains a dedicated eBPF virtual machine that can interpret these programs. Most modern architectures also support just-in-time compilation, which converts eBPF bytecode into native machine code to improve performance. eBPF programs have strict restrictions, and even if the program has defects, it will not cause the system to crash or be maliciously exploited. eBPF programs can interact with the kernel by calling auxiliary functions provided by the kernel; The feature of eBPF is mapping, which allows data to be shared between user space and kernel space. Mapping is a key-value storage used to pass information, cache data, or implement cross-process communication. At the same time, eBPF supports multiple types of programs, each with its own specific usage scenarios and interfaces. Before the eBPF program is executed, the verifier in the kernel checks the program to ensure that it follows all rules and meets security requirements to prevent potential security vulnerabilities. There is a complete set of tool chains and libraries around eBPF, which makes it easier for developers to create, debug, and deploy eBPF programs. In S1, the following steps are specifically included: S11, start the data replication service module, check whether the system resources meet the requirements of the backup task, the system resources specifically include memory, CPU, and disk space, so as to avoid task failure due to insufficient resources during the backup process, and read the specific parameters of the backup task from the configuration file or database, including the backup source and target, backup strategy, and log record level; S12. Call the eBPF program loading and initialization module to load the compiled eBPF program into the kernel, specify the eBPF program type and the kernel function or event to be attached to; S13, after the eBPF program is loaded, it is initialized and configured, and necessary data structures are set, including mapping table map, ring buffer, and parameters of the eBPF program, which are used for subsequent data capture and processing; S14, initialization completion and status monitoring, start a status monitoring thread to monitor the running status of the eBPF program and data replication service in real time, and detect and handle exceptions in time.

3. The block-level data real-time backup method based on eBPF according to claim 1 is characterized in that: The S2 specifically includes the following steps: S21, create a ring buffer as a temporary storage area for IO data, define the size of the ring buffer and the number of buffers, the size of the ring buffer and the number of buffers can be set according to actual system requirements and resource conditions; S22, configure the initialization parameters of the ring buffer to control the read and write operations of the ring buffer and data overflow processing; S23. Start a status monitoring thread to monitor the usage of the ring buffer in real time, including the used space and the remaining space, which helps to timely detect and handle potential memory leaks and overflows.

4. The method for real-time block-level data backup based on eBPF according to claim 1, characterized in that: The S3 specifically includes the following steps: S31. Identify the definition and triggering conditions of the submit_bio event point in the kernel source code. submit_bio is an event point in the Linux kernel for submitting block IO operations. When the upper file system or block device layer submits an IO request to the underlying block device, the submit_bio event is triggered. Because this event point directly involves the read and write operations of data, it can be used to capture changes in block-level data. S32. Use the interface provided by the eBPF system call management module to register the eBPF program to the submit_bio event point. The type of the eBPF program and the kernel function or event to be attached to it must be specified. S33. Start a status monitoring thread to monitor the running status and performance indicators of the eBPF program in real time, including key indicators such as the program's memory usage, CPU usage, and event capture rate. By monitoring these key indicators, potential performance bottlenecks or abnormal behaviors can be discovered and handled in a timely manner.

5. The block-level data real-time backup method based on eBPF according to claim 1 is characterized in that: The S4 specifically includes the following steps: S41. Capture submit_bio event. In the eBPF program, the submit_bio event is captured by registering a hook function to the submit_bio event point. When the submit_bio event is triggered, the hook function is called and a pointer to the bio structure is passed in as a parameter. S42, when obtaining the bio structure, obtain the content of the bio structure through the passed-in pointer. This structure contains all the information about the IO operation, including the target disk object, the data buffer pointer and the data size; S43, analyzing the target disk object, including target disk identification, operation type determination and data size analysis, identifying the target disk object of the IO operation by parsing the target disk field in the bio structure, distinguishing different disks and partitions, so as to perform subsequent backup processing; The operation type field in the bio structure is used to determine whether the IO operation is a read operation or a write operation. Backup focuses more on data write operations. The data size of the IO operation is obtained by parsing the data size field in the bio structure to evaluate the scale of the backup task and the time resources required.

6. The block-level data real-time backup method based on eBPF according to claim 1 is characterized in that: In S5, Bitmap is a data structure used to record the IOs that were not successfully intercepted due to insufficient ring buffer resources or detection errors. 1 indicates that there are IOs that are not intercepted for this data block, and 0 indicates that all IOs for this data block are successfully intercepted. When the data replication service performs data transmission, the data block with the Bitmap identifier of 1 needs to be retransmitted. When the eBPF program traces the submit_bio function, the Bitmap module uses the bitmap to record I / O requests when errors or exceptions occur, so as to effectively identify and manage these error messages and provide metadata information for subsequent backup and recovery.

7. The method for real-time block-level data backup based on eBPF according to claim 1, characterized in that: The S6 specifically includes the following steps: S61. Obtain a list of disks to be backed up. The list is passed in from the user space when the eBPF program is initialized, or is dynamically obtained at runtime through configuration files and environment variables. S62, matching the disk to be backed up, comparing the device number or file path of the bio target disk with the device number or file path in the backup disk list, making a backup decision, if the target disk is the disk to be backed up, performing the backup process, otherwise, ignoring the bio operation; S63. Encapsulate the IO operation information, and further extract the IO operation information in the submit_bio event through the eBPF program, including the type, start address, data length and timestamp of the IO operation, and encapsulate the extracted IO operation information into a structure or object.

8. The block-level data real-time backup method based on eBPF according to claim 1, characterized in that: The S7 specifically includes the following steps: S71. Use the bpf_ringbuf_reserve function to allocate space from the ring buffer, write the encapsulated IO operation information into the allocated space, and then use the bpf_ringbuf_submit function to submit the data to the ring buffer. S72, notifying the data replication service module to read data from the ring buffer, the notification signal including the address of the ring buffer and the length of the written data; S73, reading and transmitting data. The data replication service module reads the encapsulated IO operation information from the ring buffer and further processes the read IO operation information according to the backup strategy, including format conversion, compression, and encryption, to improve the efficiency and security of data transmission. S74, transmitting the processed data to a backup storage system or a remote backup server for storage and retention; S75. Error handling and recovery. During the entire data transmission process, the data replication service module will perform error detection and recovery processing. When an error or abnormal situation is detected, measures will be taken to recover or retry, and detailed log information will be recorded to facilitate troubleshooting and location when problems occur. The log information will record configuration operation logs, backup process logs, backup status, and abnormal resource usage alerts.

9. A block-level data real-time backup device based on eBPF, characterized in that: It includes data replication service module, eBPF system call management module, IO operation encapsulation module, eBPF program loading and initialization module, bio interception and analysis module, data transmission module, IO data cache module, backup disk judgment module, error handling and logging module.

10. The block-level data real-time backup device based on eBPF according to claim 9, characterized in that: The data replication service module is the core control unit of the entire backup system, responsible for starting, managing and monitoring the entire backup process. The data replication service module listens to data from the eBPF program, transfers data to the backup target according to the configuration record, and performs error handling and flow control. The eBPF system call management module is responsible for registering the eBPF program to a specific trace point to ensure that the eBPF program can intercept the bio request at the right time; The IO operation encapsulation module encapsulates IO operation information, further extracts IO operation information in the submit_bio event through the eBPF program, and encapsulates the extracted IO operation information into a structure or object for subsequent processing and transmission; The eBPF program loading and initialization module is responsible for loading the eBPF-based program into the kernel and performing necessary initialization configuration to ensure that the eBPF program can interact with the kernel correctly; The bio interception and analysis module intercepts the submit_bio event in the eBPF program, analyzes the bio target disk object, and obtains information about the IO operation by parsing the bio structure in the submit_bio event; The data transmission module writes the packaged IO data into the ring buffer, and notifies the data replication service module to read data from the ring buffer and Bitmap for transmission; The IO data cache module is a component responsible for caching data captured from disk IO operations, and temporarily stores the data by introducing a cache mechanism for subsequent efficient processing; The backup disk determination module is used to determine whether the target disk is a disk to be backed up; The error handling and logging module is used to copy data during the entire data transmission process. The service module performs error detection and recovery. When an error or abnormality is detected, If the error occurs, measures are taken to recover or retry, and detailed log information is recorded.

Images