A multi-platform data acquisition method based on eBPF

Abstract

This invention relates to the field of computer system monitoring and data acquisition technology, and discloses a multi-platform data acquisition method based on eBPF, applicable to Windows, Android, and Linux operating systems. First, eBPF programs are loaded on different platforms to capture kernel events, thereby achieving comprehensive monitoring of system calls, network activity, and file operations. Next, by extending the data processing capabilities of user-space applications, the collected kernel event data is standardized and transmitted to a cross-platform data processing interface. To ensure the security of data transmission, this invention introduces an encryption protocol to protect the multi-platform data. Finally, the transmitted data is centrally stored and analyzed to detect system anomalies and potential security threats. By utilizing eBPF technology, this invention achieves efficient data acquisition and secure data transmission in multi-platform systems, effectively improving the security and reliability of multi-platform kernel event monitoring and data analysis.

Description

Technical Field

[0001] This invention relates to the field of computer system monitoring and data acquisition technology, specifically a multi-platform data acquisition method based on eBPF (Extended Berkeley Packet Filter). Background Technology

[0002] With the rapid development of information technology and the increasing complexity of network environments, data acquisition technology plays an increasingly important role in various fields. Traditional data acquisition technologies are typically designed and optimized for single environments, leading to numerous challenges when applied in multi-platform environments. Especially with the increasing prevalence of cloud computing and virtualization technologies, the coexistence of various operating systems within the same network environment makes multi-platform data acquisition particularly crucial. Specifically, traditional technologies struggle to achieve synchronous data acquisition in multi-platform environments, suffer from inconsistent data storage methods, cannot perform simultaneous acquisition, and have poor adaptability, failing to meet the needs of complex application scenarios.

[0003] eBPF, as a powerful kernel technology, enables efficient and flexible data acquisition and is applicable to various operating systems. eBPF not only enhances kernel functionality but also provides real-time monitoring and data acquisition capabilities. However, existing eBPF applications are typically limited to a single platform, lacking a unified solution for multi-platform data acquisition. Therefore, developing a multi-platform eBPF data acquisition method to achieve security monitoring and data acquisition across different platforms is of significant practical importance. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a multi-platform data acquisition method based on eBPF, designed to achieve secure monitoring and data acquisition across multiple operating system platforms such as Linux, Windows, and Android, thereby improving the security and stability of information systems. Furthermore, it enables synchronized tasks across multiple platforms by issuing commands through a unified control center. The technical solution is as follows:

[0005] A multi-platform data acquisition method based on eBPF includes the following steps:

[0006] Step 1: Write eBPF source code in the Power-eBPF editor, define specific data acquisition logic for different platforms, and generate eBPF programs suitable for each platform through conditional compilation.

[0007] Step 2: The generated eBPF program is distributed to target devices on different platforms through the central controller. The central controller is responsible for managing the distribution of the eBPF program so that each platform receives the corresponding version of the eBPF program.

[0008] Step 3: Start the daemon process on each platform. After receiving the eBPF program from the central controller, the daemon process dynamically loads the eBPF program and runs it in the kernel of the target device to collect target data such as system calls, network activities, and file operations.

[0009] Step 4: The daemon process encapsulates the collected target data and uploads it to the central controller through an encrypted channel. The central controller receives the collected data from different platforms, standardizes and unifies the format of the data to ensure data consistency across multiple platforms, and stores the data in a unified data storage module for subsequent analysis and processing.

[0010] Step 5: In the data processing module, perform real-time monitoring and in-depth analysis of the stored data, identify potential security threats or abnormal behaviors through machine learning algorithms, and generate monitoring reports and security recommendations.

[0011] Furthermore, step 1 specifically includes:

[0012] Step 1.1: In the Power-eBPF editor, write the data acquisition logic according to the characteristics of different platforms. Design the eBPF program modules for the operating system characteristics of Linux, Windows and Android platforms. Linux platform uses tc and xdp to implement network data capture, Windows platform uses ETW to handle event tracking, and Android uses the JNI interface to interact with eBPF, integrating the characteristics of each platform.

[0013] Step 1.2: Generate eBPF programs that meet the requirements of different platforms through conditional compilation, and import the code into the compiler for processing; specifically:

[0014] First, the entire eBPF code is imported into the Power-eBPF compiler, which performs preliminary syntax and semantic analysis on the code. Then, using the conditional compilation mechanism, preprocessing directives are used to compile the code for different platforms such as Linux, Windows, and Android. The compiler first parses the conditional compilation directives in the code, checks the definition of each platform, and thus identifies the code blocks for specific platforms.

[0015] Step 1.3: After conditional compilation, the generated eBPF program is optimized. The optimization process includes removing redundant code, improving performance, and reducing program size. The optimized eBPF program is then linked to generate the final eBPF program, ensuring efficient and stable operation on various platforms.

[0016] Step 1.4: Configure the triggering conditions and data acquisition parameters of the eBPF program according to the system characteristics of each platform; specifically:

[0017] First, we analyze the characteristics of system call interfaces, network protocol stacks, and file system calls on various platforms. On the Linux platform, we select specific system calls as triggering conditions and use socket filters to capture network packets. On the Windows platform, we monitor critical system activities by registering specific ETW events. On the Android platform, we use the Binder IPC mechanism or JNI interface calls to interact with eBPF.

[0018] Step 1.5: Set the frequency and type of data acquisition to achieve effective data acquisition on different operating systems.

[0019] Furthermore, step 2 specifically includes:

[0020] The central controller first identifies the operating system type of each target device and selects the appropriate version to distribute based on the previously compiled eBPF program. The central controller matches the identity information of the target device with its corresponding eBPF program to ensure that the program received by each device is compatible with its operating system. By establishing long or short connections, the central controller monitors the distribution status in real time and confirms that the target device has successfully received and loaded the corresponding eBPF program.

[0021] Furthermore, step 3 specifically includes:

[0022] Step 3.1: Start the daemon process on each platform to maintain continuous communication with the central controller;

[0023] Step 3.2: Receive and parse instructions from the central controller to determine whether to load a new eBPF program or unload an old program. If the central controller sends a new eBPF program, the daemon first checks the currently loaded version and performs a version comparison. If a new version is detected, the daemon will safely unload the old version of the eBPF program and dynamically load the newly received eBPF program to run in the kernel.

[0024] Furthermore, step 4 specifically includes:

[0025] Step 4.1: The daemon process encapsulates the collected data and uploads it to the central controller through an encrypted channel;

[0026] Step 4.2: During the data upload process, the daemon monitors the network status and implements a retry mechanism in case of network anomalies; after the upload is completed, the upload status and timestamp are recorded;

[0027] Step 4.3: The central controller receives collected data from different platforms, verifies, standardizes, and unifies the format of the data; during the data reception process, it verifies the integrity and authenticity of the data; finally, it parses the received data into the standard format used internally and converts the timestamp into a unified time zone format.

[0028] Furthermore, step 5 specifically includes:

[0029] Step 5.1: In the data processing module, perform real-time monitoring and analysis of the stored data, and identify potential security threats by comparing the performance of different devices;

[0030] Step 5.2: Utilize machine learning algorithms to perform in-depth data analysis, identify threat patterns, and generate monitoring reports to support security decisions;

[0031] Step 5.3: Use the generated analysis results for real-time monitoring, system log recording, and compliance auditing, and have the central controller implement corresponding security measures or provide feedback to the administrator.

[0032] The beneficial effects of this invention are:

[0033] 1. Multi-platform compatibility: Through conditional compilation, dedicated eBPF programs adapted to Linux, Windows and Android platforms are generated, ensuring compatibility across different platforms and simplifying multi-platform development and maintenance.

[0034] 2. Intelligent compilation and distribution mechanism: The central controller enables centralized management of eBPF programs, automatically selects the appropriate eBPF program version based on the target device's operating system, ensures the security and efficiency of program distribution, and supports subsequent updates and version control.

[0035] 3. Dynamic monitoring and update capabilities: The daemon continuously communicates with the central controller and dynamically loads or unloads eBPF programs according to instructions, adapting to the specific needs of different platforms and enabling flexible adjustment of real-time data acquisition strategies.

[0036] 4. Secure Data Transmission and Storage: Encrypted channels ensure data security during transmission. The central controller verifies, standardizes, and unifies the format of received data to ensure consistency and availability. Attached Figure Description

[0037] Figure 1 This is a flowchart of the multi-platform data acquisition method based on eBPF of the present invention. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0039] like Figure 1 The flowchart shown is for a multi-platform data acquisition method based on eBPF. The process is as follows:

[0040] 1) Data Processing: In the data processing module, user-written Power-eBPF programs are processed by a conditional compiler to generate eBPF programs suitable for different operating systems (such as Linux, Windows, and Android).

[0041] 2) Central Controller: The central controller is responsible for managing and coordinating the eBPF programs on each platform. It receives instructions and distributes the corresponding eBPF programs to the target devices.

[0042] 3) eBPF program distribution: eBPF programs for different operating systems (eBPF Program For Linux, eBPF Program For Android, eBPF Program For Windows) are distributed from the central controller to the corresponding daemons.

[0043] 4) Daemon process: Each platform (Android, Windows, Linux) has a daemon process responsible for communicating with the central controller, receiving instructions, dynamically loading and unloading eBPF programs, and performing related data acquisition tasks.

[0044] 5) Unified storage: After processing, the collected data is stored in a unified data storage module for subsequent data analysis and monitoring.

[0045] The specific steps are as follows:

[0046] Step 1: Write an eBPF program in the Power-eBPF editor, define specific data acquisition logic for different platforms, and generate an eBPF program suitable for each platform through conditional compilation.

[0047] Step 1.1: In the Power-eBPF editor, write the data acquisition logic according to the characteristics of different platforms. First, design separate eBPF program modules for the operating system characteristics of Linux, Windows, and Android platforms. The Linux platform uses tc and xdp to achieve network data capture, while the Windows platform uses ETW to handle event tracking. Android uses a specific JNI interface to interact with eBPF, integrating the characteristics of each platform.

[0048] Step 1.2: The completed eBPF code is processed in the Power-eBPF compiler to generate eBPF programs compatible with different platforms through conditional compilation. In this step, the entire eBPF codebase is first imported into the Power-eBPF compiler. The compiler performs preliminary syntax and semantic analysis to ensure the code is error-free and conforms to programming standards. Next, using conditional compilation, preprocessor directives (such as #ifdef, #elif, and #endif) are used to compile the code specifically for different platforms such as Linux, Windows, and Android. The compiler first parses the conditional compilation directives in the code, checking the definitions of each platform (such as #define LINUX, #define WINDOWS, #define ANDROID) to identify code blocks specific to each platform. Through conditional compilation, the compiler only compiles code segments applicable to the current target platform, ensuring the generated code meets the requirements of each platform.

[0049] Step 1.3: After conditional compilation, the compiler optimizes the generated eBPF program, including removing unused functions and simplifying control flow, to improve the execution efficiency of the final binary file. After optimization, the compiler links all object files together to form an executable eBPF program. For each platform, the compiler generates an eBPF program suitable for that platform.

[0050] Step 1.4: Configure the triggering conditions and data acquisition parameters of the eBPF program according to the system characteristics of each platform. First, analyze the characteristics of the system call interfaces, network protocol stacks, and file system calls of each platform. On the Linux platform, specific system calls (such as open(), read(), and write()) can be selected as triggering conditions, and socket filters can be used to capture network packets. For the Windows platform, monitoring of critical system activities is achieved by registering specific ETW events. On the Android platform, interaction with eBPF needs to be achieved using the Binder IPC mechanism or through JNI interface calls.

[0051] Step 1.5: When configuring data acquisition parameters, explicitly set the data acquisition frequency to ensure the real-time performance and accuracy of the data. Adjust the acquisition frequency and key time points according to the system load and the required real-time monitoring accuracy. Furthermore, define the required data types, such as network traffic data, file access records, and system call logs, to achieve effective data acquisition on various platforms. These configurations will ensure the compatibility and accuracy of data acquisition across different operating systems, thereby meeting the needs of real-time monitoring.

[0052] Step 2: The generated eBPF program is distributed to target devices on different platforms through the central controller. The central controller is responsible for managing the distribution of the eBPF program, so that each platform receives the corresponding version of the eBPF program.

[0053] The central controller distributes eBPF programs from different platforms to the corresponding devices. In this process, the central controller first identifies the operating system type of each target device and selects the appropriate program for distribution based on the previously compiled eBPF program version. Secure communication protocols (such as HTTPS or MQTT) are used to ensure the security and integrity of data transmission. The central controller matches the target device's identity information with its corresponding eBPF program, ensuring that the program received by each device is compatible with its operating system. By establishing long or short connections, the central controller can monitor the distribution status in real time, confirming that the target device has successfully received and loaded the corresponding eBPF program.

[0054] Step 3: Start the daemon process on each platform (Linux, Windows, Android). After receiving the eBPF program from the central controller, the daemon process dynamically loads the eBPF program and runs it in the kernel of the target device to collect target data such as system calls, network activities, and file operations.

[0055] Step 3.1: Start a daemon process on each platform. This process is responsible for maintaining communication with the central controller. Once started, the daemon process initializes the connection with the central controller, maintaining a stable, long-lived connection to receive instructions and updates in real time. The daemon process periodically sends heartbeat packets to the central controller to ensure connection validity and prevent disconnection due to prolonged inactivity. The daemon process is ready to process instructions from the central controller, including loading new eBPF applications or uninstalling existing ones.

[0056] Step 3.2: When instructions arrive at the central controller, the daemon process receives and parses these instructions to determine whether to load a new eBPF program or uninstall an existing one. If the central controller sends a new eBPF program, the daemon process first checks the currently loaded version and performs a version comparison. If a new version is detected, the daemon process safely uninstalls the old version of the eBPF program to avoid system instability due to program conflicts. Subsequently, the daemon process dynamically loads the newly received eBPF program, registers the relevant hooks, and enables it to run in the kernel.

[0057] Step 4: The daemon process encapsulates the collected target data and uploads it to the central controller through an encrypted channel. The central controller receives the collected data from different platforms, standardizes and unifies the data format to ensure data consistency across multiple platforms, and stores the data in a unified data storage module for subsequent analysis and processing.

[0058] Step 4.1: The daemon process encapsulates the collected data. After data collection, the daemon process first formats the data, converting it into a unified standard format for subsequent parsing and processing. During data encapsulation, the daemon process cleans and filters the collected data, uploading only useful information and removing redundant and sensitive data. After data encapsulation, the daemon process uploads the encapsulated data to the central controller via an encrypted channel (such as TLS or HTTPS).

[0059] Step 4.2: During the data upload process, the daemon monitors network conditions and implements a retry mechanism in case of network anomalies, ensuring reliable data transmission to the central controller. Furthermore, after upload completion, the daemon records the upload status and timestamp for subsequent auditing and troubleshooting. In this way, the daemon can efficiently and accurately transmit the collected data to the central controller.

[0060] Step 4.3: The central controller receives collected data from different platforms, verifies, standardizes, and unifies the format of the data to ensure data consistency across multiple platforms. During data reception, the central controller verifies the integrity and authenticity of the data by checking the hash value and signature to ensure that the received data has not been tampered with. Next, the central controller parses the received data into its internal standard format, handles potential field differences between different platforms, and converts the timestamp to a unified timezone format to ensure data time consistency.

[0061] Step 4.4: During the standardization process, the central controller deduplicates and cleans the data, removing invalid or redundant records to ensure data quality for subsequent analysis. After standardization, the central controller stores the data in a unified data storage module for convenient processing and analysis.

[0062] Save the standardized data to a unified storage module (such as a database, file system, or data lake) for subsequent data processing and analysis.

[0063] Step 5: In the data processing module, perform real-time monitoring and in-depth analysis of the stored data, identify potential security threats or abnormal behaviors through machine learning algorithms, and generate monitoring reports and security recommendations.

[0064] Step 5.1: In the data processing module, perform real-time monitoring and analysis of the data. By comparing the performance of the same behavior on multiple devices, differences between different devices can be identified, and key behaviors can be found.

[0065] Step 5.2: Perform in-depth analysis of the stored data using machine learning algorithms. First, select a suitable machine learning model, such as a decision tree, random forest, or support vector machine, to extract valuable information from the data. By training on historical data, the model can identify potential threat patterns and anomalous behaviors. The analysis results will be used to generate real-time monitoring reports and support security decisions.

[0066] Step 5.3: The generated analysis results can be used for security management purposes such as real-time monitoring, system log recording, and compliance auditing. The central controller will then send the results to the instruction module to execute corresponding security measures or provide feedback to the administrator.

[0067] In summary, this invention is applicable to Windows, Android, and Linux operating systems. First, it loads eBPF programs on different platforms to capture kernel events, thereby achieving comprehensive monitoring of system calls, network activity, and file operations. Then, by extending the data processing capabilities of user-space applications, the collected kernel event data is standardized and transmitted to a cross-platform data processing interface. To ensure data transmission security, an encryption protocol is introduced to protect data across multiple platforms. Finally, the transmitted data is centrally stored and analyzed to detect system anomalies and potential security threats. By utilizing eBPF technology, this invention achieves efficient data acquisition and secure data transmission in multi-platform systems, effectively improving the security and reliability of multi-platform kernel event monitoring and data analysis.

Claims

1. A multi-platform data acquisition method based on eBPF, characterized in that, Includes the following steps: Step 1: Write eBPF source code in the Power-eBPF editor, define specific data acquisition logic for different platforms, and generate eBPF programs suitable for each platform through conditional compilation. Step 2: The generated eBPF program is distributed to target devices on different platforms through the central controller. The central controller is responsible for managing the distribution of the eBPF program so that each platform receives the corresponding version of the eBPF program. Step 3: Start the daemon process on each platform. After receiving the eBPF program from the central controller, the daemon process dynamically loads the eBPF program and runs it in the kernel of the target device to collect target data such as system calls, network activities, and file operations. Step 4: The daemon process encapsulates the collected target data and uploads it to the central controller through an encrypted channel. The central controller receives the collected data from different platforms, standardizes and unifies the data format to ensure data consistency across multiple platforms, and stores the data in a unified data storage module for subsequent analysis and processing. Step 5: In the data processing module, perform real-time monitoring and in-depth analysis of the stored data, identify potential security threats or abnormal behaviors through machine learning algorithms, and generate monitoring reports and security recommendations; Step 1 specifically involves: Step 1.1: In the Power-eBPF editor, write the data acquisition logic according to the characteristics of different platforms, and design their respective eBPF program modules for the operating system characteristics of Linux, Windows and Android platforms. The Linux platform uses tc and xdp to capture network data, the Windows platform uses ETW to handle event tracking, and Android uses the JNI interface to interact with eBPF, integrating the features of each platform. Step 1.2: Generate eBPF programs that meet the requirements of different platforms through conditional compilation, and import the programs into the compiler for processing; specifically: First, the entire eBPF code is imported into the Power-eBPF compiler, which performs preliminary syntax and semantic analysis on the code. Then, using conditional compilation, preprocessing directives are used to compile the code specifically for different platforms such as Linux, Windows, and Android. The compiler first parses the conditional compilation directives in the code, checks the definition of each platform, and thus identifies the code blocks that are specific to a particular platform. Step 1.3: After conditional compilation is completed, the generated eBPF program is further optimized to remove redundant parts and improve performance; the compiler links the optimized code to generate the final executable eBPF program. Step 1.4: Configure the triggering conditions and data acquisition parameters of the eBPF program according to the system characteristics of each platform; specifically: First, we analyze the characteristics of system call interfaces, network protocol stacks, and file system calls on various platforms. On the Linux platform, we select specific system calls as triggering conditions and use socket filters to capture network packets. On the Windows platform, we monitor critical system activities by registering specific ETW events. On the Android platform, we use the Binder IPC mechanism or JNI interface calls to interact with eBPF. Step 1.5: Set the data acquisition frequency and type to achieve effective data acquisition on different operating systems. Step 2 specifically involves: The central controller first identifies the operating system type of each target device and selects the appropriate version to distribute based on the previously compiled eBPF program. The central controller matches the identity information of the target device with its corresponding eBPF program to ensure that the program received by each device is compatible with its operating system. By establishing long or short connections, the central controller monitors the distribution status in real time and confirms that the target device has successfully received and loaded the corresponding eBPF program.

2. The multi-platform data acquisition method based on eBPF according to claim 1, characterized in that, Step 3 specifically involves: Step 3.1: Start the daemon process on each platform to maintain continuous communication with the central controller; Step 3.2: Receive and parse instructions from the central controller to determine whether to load a new eBPF program or unload an old program. If the central controller sends a new eBPF program, the daemon first checks the currently loaded version and performs a version comparison. If a new version is detected, the daemon will safely unload the old version of the eBPF program and dynamically load the newly received eBPF program to run in the kernel.

3. The multi-platform data acquisition method based on eBPF according to claim 1, characterized in that, Step 4 specifically involves: Step 4.1: The daemon process encapsulates the collected data and uploads it to the central controller through an encrypted channel; Step 4.2: During the data upload process, the daemon monitors the network status and implements a retry mechanism in case of network anomalies; after the upload is completed, the upload status and timestamp are recorded; Step 4.3: The central controller receives collected data from different platforms, verifies, standardizes, and unifies the format of the data; during the data reception process, it verifies the integrity and authenticity of the data; finally, it parses the received data into the standard format used internally and converts the timestamp into a unified time zone format.

4. The multi-platform data acquisition method based on eBPF according to claim 1, characterized in that, Step 5 specifically involves: Step 5.1: In the data processing module, perform real-time monitoring and analysis of the stored data, and identify potential security threats by comparing the performance of different devices; Step 5.2: Utilize machine learning algorithms to perform in-depth data analysis, identify threat patterns, and generate monitoring reports to support security decisions; Step 5.3: Use the generated analysis results for real-time monitoring, system log recording, and compliance auditing, and have the central controller implement corresponding security measures or provide feedback to the administrator.

Images