EBPF-based multi-platform data acquisition method

Abstract

The invention relates to the technical field of computer system monitoring and data acquisition, and discloses a multi-platform data acquisition method based on eBPF, which is suitable for Windows, Android and Linux operating systems. The method comprises the following steps: firstly, loading eBPF programs on different platforms to capture kernel events so as to realize comprehensive monitoring on system calling, network activities and file operation; next, standardizing the collected kernel event data by expanding a data processing function of a user mode application program, and transmitting the standardized kernel event data to a cross-platform data processing interface; in order to ensure the security of data transmission, an encryption protocol is introduced to protect multi-platform data. Finally, the transmitted data will be centrally stored and analyzed in order to detect system anomalies and potential security threats. By applying the eBPF technology, efficient data collection and safe data transmission are achieved in a multi-platform system, and the safety and reliability of multi-platform kernel event monitoring and data analysis are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer system monitoring and data collection, in particular to a multi-platform data collection method based on eBPF (Extended Berkeley Packet Filter). Background Art

[0002] With the rapid development of information technology, the network environment is becoming increasingly complex, and data collection technology plays an increasingly important role in various fields. Traditional data collection technology is usually designed and optimized for a single environment, resulting in many challenges when applied in a multi-platform environment. Especially in the context of the increasing popularity of cloud computing and virtualization technology, the coexistence of various operating systems in the same network environment makes multi-platform collection particularly important. Specifically, traditional technology is difficult to achieve synchronous collection in a multi-platform environment, the data storage method is not unified, it cannot be collected simultaneously, and the adaptability is poor, which cannot meet the needs of complex application scenarios.

[0003] As a powerful kernel technology, eBPF can realize data collection in an efficient and flexible way and is applicable to various operating systems. eBPF can not only enhance kernel functions, but also provide real-time monitoring and data collection capabilities. However, existing eBPF applications are usually limited to a single platform and lack a unified solution for multi-platform data collection. Therefore, it is of great practical significance to develop a multi-platform eBPF data collection method to achieve security monitoring and data collection on different platforms. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a multi-platform data collection method based on eBPF, aiming to realize security monitoring and data collection of multiple operating system platforms such as Linux, Windows and Android, so as to improve the security and stability of the information system, and to implement synchronization tasks of multiple platforms by issuing instructions through a unified control center. The technical solution is as follows:

[0005] A multi-platform data collection method based on eBPF, comprising the following steps:

[0006] Step 1: Write the eBPF source code in the Power-eBPF editor, define specific data collection logic for different platforms, and generate the eBPF source code into an eBPF program suitable for each platform through conditional compilation;

[0007] Step 2: Send the generated eBPF program to target devices on different platforms through the central controller. The central controller is responsible for managing the distribution of the eBPF program so that each platform receives the corresponding version of the eBPF program.

[0008] Step 3: Start the daemon process on each platform. After receiving the eBPF program sent by the central controller, the daemon process dynamically loads the eBPF program and runs it in the kernel of the target device to collect target data of system calls, network activities, and file operations.

[0009] Step 4: The daemon encapsulates the collected target data and uploads it to the central controller through an encrypted channel. The central controller receives the collected data from different platforms, standardizes the data and unifies the format to ensure data consistency on multiple platforms, and stores the data in a unified data storage module for subsequent analysis and processing;

[0010] Step 5: Perform real-time monitoring and in-depth analysis of the stored data in the data processing module, identify potential security threats or abnormal behaviors through machine learning algorithms, and generate monitoring reports and security recommendations.

[0011] Furthermore, the step 1 is specifically as follows:

[0012] Step 1.1: In the Power-eBPF editor, write data collection logic according to the characteristics of different platforms, and design eBPF program modules for Linux, Windows, and Android platforms according to the operating system characteristics. The Linux platform uses tc and xdp to capture network data, and the Windows platform uses ETW to handle event tracking. Android uses the JNI interface to interact with eBPF and integrate the characteristics of each platform.

[0013] Step 1.2: Generate eBPF programs that meet the requirements of different platforms through the conditional compilation mechanism, and import the code into the compiler for processing; specifically:

[0014] First, the entire eBPF code is imported into the Power-eBPF compiler, and the compiler performs preliminary syntax and semantic analysis on the code. The conditional compilation mechanism is used to perform targeted code compilation for different platforms such as Linux, Windows, and Android through preprocessing instructions. The compiler first parses the conditional compilation instructions in the code and checks the definition of each platform to identify the code blocks for specific platforms.

[0015] Step 1.3: After conditional compilation is completed, the generated eBPF program is optimized. The optimization process includes removing redundant code, improving performance and reducing program size. The optimized eBPF program will be linked to generate the final eBPF program to ensure efficient and stable operation on various platforms;

[0016] Step 1.4: Configure the trigger conditions and data collection parameters of the eBPF program according to the system characteristics of each platform; specifically:

[0017] First, we analyze the characteristics of the system call interface, network protocol stack, and file system call of each platform. On the Linux platform, we select specific system calls as trigger conditions and use socket filters to capture network data packets. For the Windows platform, we register specific ETW events to monitor key system activities. On the Android platform, we use the Binder IPC mechanism or JNI interface calls to interact with eBPF.

[0018] Step 1.5: Set the frequency and type of data collection to achieve effective data collection on different operating systems.

[0019] Furthermore, the step 2 is specifically as follows:

[0020] The central controller first identifies the operating system type of each target device, and selects the appropriate version for distribution based on the previously compiled eBPF program; the central controller matches the identity information of the target device with its corresponding eBPF program, so that the program received by each device is compatible with its operating system; by establishing a long connection or a short connection, the central controller monitors the distribution status in real time to confirm that the target device has successfully received and loaded the corresponding eBPF program.

[0021] Furthermore, the step 3 is specifically as follows:

[0022] Step 3.1: Start a daemon on each platform to communicate continuously with the central controller;

[0023] Step 3.2: Receive and parse instructions from the central controller to decide whether to load a new eBPF program or uninstall an old program. If the central controller sends a new eBPF program, the daemon first checks the currently loaded version and performs a version comparison. If a new version is detected, the daemon safely uninstalls the old version of the eBPF program and dynamically loads the newly received eBPF program to run in the kernel.

[0024] Furthermore, the step 4 is specifically as follows:

[0025] Step 4.1: The daemon encapsulates the collected data and uploads it to the central controller through an encrypted channel;

[0026] Step 4.2: During the data upload process, the daemon monitors the network status and implements a retry mechanism in case of network anomalies; after the upload is completed, the status and timestamp of the upload are recorded;

[0027] Step 4.3: The central controller receives the collected data from different platforms, verifies, standardizes and unifies the format of the data; in the process of receiving the data, verifies the integrity and authenticity of the data; finally, parses the received data into a standard format for internal use and converts the timestamp into a unified time zone format.

[0028] Furthermore, the step 5 is specifically as follows:

[0029] Step 5.1: Monitor and analyze the stored data in real time in the data processing module, and identify potential security threats by comparing the performance of different devices;

[0030] Step 5.2: Use machine learning algorithms to deeply analyze data, identify threat patterns and generate monitoring reports to support security decision-making;

[0031] Step 5.3: The generated analysis results are used for real-time monitoring, system logging, and compliance auditing, and the central controller executes corresponding security measures or feeds back to the administrator.

[0032] The beneficial effects of the present invention are:

[0033] 1. Multi-platform compatibility: Through the conditional compilation mechanism, dedicated eBPF programs are generated for Linux, Windows and Android platforms to ensure compatibility between different platforms and simplify multi-platform development and maintenance.

[0034] 2. Intelligent compilation and distribution mechanism: Use the central controller to realize the centralized management of eBPF programs, automatically select the appropriate eBPF program version according to the operating system of the target device, ensure the security and efficiency of program distribution, and support subsequent updates and version control.

[0035] 3. Dynamic monitoring and updating capabilities: The daemon process continuously communicates with the central controller, dynamically loads or unloads eBPF programs according to instructions, adapts to the specific needs of different platforms, and realizes flexible adjustment of real-time data collection strategies.

[0036] 4. Secure data transmission and storage: Encrypted channels are used to ensure the security of data during transmission. The central controller verifies, standardizes and unifies the format of the received data to ensure data consistency and availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of the multi-platform data collection method based on eBPF of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] like Figure 1 The flowchart of the multi-platform data collection method based on eBPF is shown, and the process is as follows:

[0040] 1) Data processing: In the data processing module, the Power-eBPF program written by the user is processed by the conditional compiler to generate eBPF programs suitable for different operating systems (such as Linux, Windows, and Android).

[0041] 2) Central Controller: The central controller is responsible for managing and coordinating the eBPF programs of each platform. It receives instructions and distributes the corresponding eBPF programs to the target devices.

[0042] 3) eBPF program distribution: eBPF programs for different operating systems (eBPF Program For Linux, eBPFProgram For Android, eBPF Program For Windows) are sent from the central controller to the corresponding daemon process.

[0043] 4) Daemon: Each platform (Android, Windows, Linux) has a daemon responsible for maintaining communication with the central controller, receiving instructions, dynamically loading and unloading eBPF programs, and performing related data collection tasks.

[0044] 5) Unified storage: After being processed, the collected data is stored in a unified data storage module for subsequent data analysis and monitoring.

[0045] The specific steps are as follows:

[0046] Step 1: Write the eBPF program in the Power-eBPF editor, define specific data collection logic for different platforms, and generate the eBPF program into an eBPF program suitable for each platform through conditional compilation.

[0047] Step 1.1: In the Power-eBPF editor, write data collection logic according to the characteristics of different platforms. First, the eBPF program modules of Linux, Windows and Android platforms are designed according to the operating system characteristics. The Linux platform uses tc and xdp to capture network data, while the Windows platform uses ETW to handle event tracing. Android uses a specific JNI interface to interact with eBPF and integrate the characteristics of each platform.

[0048] Step 1.2: The completed eBPF code is processed in the Power-eBPF compiler to generate eBPF programs that conform to different platforms through the conditional compilation mechanism. In this step, the entire eBPF code base is first imported into the Power-eBPF compiler, and the compiler performs preliminary syntax and semantic analysis on the code to ensure that there are no errors and that it complies with programming specifications. Next, the conditional compilation mechanism is used to perform targeted code compilation for different platforms such as Linux, Windows, and Android through preprocessing instructions (such as #ifdef, #elif, and #endif). The compiler first parses the conditional compilation instructions in the code and checks the definitions of each platform (such as #define LINUX, #define WINDOWS, #define ANDROID) to identify code blocks for specific platforms. Through conditional compilation, the compiler only compiles code segments applicable to the current target platform, so that the generated code meets the requirements of each platform.

[0049] Step 1.3: After conditional compilation, the compiler optimizes the generated eBPF program, including removing unused functions, simplifying control flow, etc., to improve the execution efficiency of the final binary file. After optimization, the compiler links all target files together to form an executable eBPF program. For each platform, the compiler generates an eBPF program suitable for that platform.

[0050] Step 1.4: Configure the trigger conditions and data collection parameters of the eBPF program according to the system characteristics of each platform. First, analyze the characteristics of the system call interface, network protocol stack, file system calls, etc. of each platform. On the Linux platform, you can select specific system calls (such as open(), read(), and write()) as trigger conditions, and use socket filters to capture network packets. For the Windows platform, monitor key system activities by registering specific ETW events. On the Android platform, you need to use the Binder IPC mechanism or call through the JNI interface to interact with eBPF.

[0051] Step 1.5: When configuring data collection parameters, explicitly set the data collection frequency to ensure the real-time and accuracy of the data. Adjust the collection frequency and time key points according to the system load and the required real-time monitoring accuracy. In addition, define the required data types, such as network traffic data, file access records, and system call logs, to achieve effective data collection on various platforms. These configurations will ensure the compatibility and accuracy of data collection on different operating systems, thereby meeting the needs of real-time monitoring.

[0052] Step 2: The generated eBPF program is sent to target devices on different platforms through the central controller. The central controller is responsible for managing the distribution of the eBPF program so that each platform receives the corresponding version of the eBPF program.

[0053] The eBPF programs of different platforms are distributed to the corresponding devices through the central controller. In this process, the central controller first identifies the operating system type of each target device, and selects the appropriate program for distribution based on the previously compiled eBPF program version. The distribution process uses secure communication protocols (such as HTTPS or MQTT) to ensure the security and integrity of data transmission. The central controller will match the identity information of the target device with its corresponding eBPF program so that the program received by each device is compatible with its operating system. By establishing a long connection or a short connection, the central controller can monitor the distribution status in real time and confirm that the target device has successfully received and loaded the corresponding eBPF program.

[0054] Step 3: Start the daemon process on each platform (Linux, Windows, Android). After receiving the eBPF program sent by the central controller, the daemon process dynamically loads the eBPF program and runs it in the kernel of the target device to collect target data such as system calls, network activities, and file operations.

[0055] Step 3.1: Start a daemon on each platform, which is responsible for maintaining communication with the central controller. When the daemon is started, it will initialize a connection with the central controller and maintain a stable long connection to receive instructions and update information in real time. The daemon will periodically send heartbeat packets to the central controller to ensure the validity of the connection and avoid being disconnected due to long-term inactivity. The daemon will be ready to process instructions from the central controller, including loading new eBPF programs or uninstalling existing programs.

[0056] Step 3.2: When instructions arrive at the central controller, the daemon can receive and parse these instructions to decide whether to load a new eBPF program or uninstall an existing program. If the central controller sends a new eBPF program, the daemon will first check the currently loaded version and perform a version comparison. If a new version is detected, the daemon will safely uninstall the old version of the eBPF program to avoid system instability due to program conflicts. The daemon will then dynamically load the newly received eBPF program, register the relevant hooks, and enable it to run in the kernel.

[0057] Step 4: The daemon process encapsulates the collected target data and uploads it to the central controller through an encrypted channel. The central controller receives the collected data from different platforms, standardizes and unifies the data format to ensure data consistency across multiple platforms, and stores the data in a unified data storage module for subsequent analysis and processing.

[0058] Step 4.1: The daemon encapsulates the collected data. After the data collection is completed, the daemon first formats the data and converts it into a unified standard format for subsequent parsing and processing. During the data encapsulation process, the daemon will perform necessary cleaning and filtering on the collected data, upload only useful information, and remove redundant data and sensitive information. After completing the data encapsulation, the daemon uploads the encapsulated data to the central controller through an encrypted channel (such as TLS or HTTPS).

[0059] Step 4.2: During the data upload process, the daemon monitors the network status and implements a retry mechanism in case of network anomalies, so that the data can be reliably transmitted to the central controller. In addition, after the upload is completed, the daemon records the status and timestamp of the upload for subsequent auditing and troubleshooting. In this way, the daemon can efficiently and accurately transmit the collected data to the central controller.

[0060] Step 4.3: The central controller receives the collected data from different platforms, verifies, standardizes and unifies the data format to ensure data consistency across multiple platforms. In the process of receiving data, the central controller verifies the integrity and authenticity of the data, and ensures that the received data has not been tampered with by checking the hash value and signature of the data. Then, the central controller parses the received data into a standard format for internal use, handles possible field differences on different platforms, and converts the timestamp into a unified time zone format to ensure the time consistency of the data.

[0061] Step 4.4: During the standardization process, the central controller will deduplicate and clean the data, remove invalid or redundant data records, and ensure the data quality for subsequent analysis. After standardization, the central controller will store the data in a unified data storage module for subsequent processing and analysis.

[0062] Save the standardized data to a unified storage module (such as a database, file system, or data lake) for subsequent data processing and analysis.

[0063] Step 5: Perform real-time monitoring and in-depth analysis of the stored data in the data processing module, identify potential security threats or abnormal behaviors through machine learning algorithms, and generate monitoring reports and security recommendations.

[0064] Step 5.1: In the data processing module, real-time monitoring and analysis of data is performed. By comparing the performance of the same behavior on multiple devices, the differences between different devices can be identified and key behaviors can be found.

[0065] Step 5.2: Use machine learning algorithms to deeply analyze the stored data. First, select a suitable machine learning model, such as decision tree, random forest, or support vector machine, to extract valuable information from the data. By training on historical data, the model can identify potential threat patterns and abnormal behaviors. The analysis results will be used to generate real-time monitoring reports and provide support for security decisions.

[0066] Step 5.3: The generated analysis results can be used for security management purposes such as real-time monitoring, system log recording, compliance auditing, etc., and are further sent to the instruction module by the central controller to execute corresponding security measures or feedback to the administrator.

[0067] In summary, the present invention is applicable to Windows, Android and Linux operating systems. First, the eBPF program is loaded on different platforms to capture kernel events, thereby realizing comprehensive monitoring of system calls, network activities and file operations; then, by expanding the data processing function of the user-mode application, the collected kernel event data is standardized and passed to the cross-platform data processing interface; in order to ensure the security of data transmission, an encryption protocol is also introduced to protect multi-platform data; finally, the transmitted data will be centrally stored and analyzed to detect system anomalies and potential security threats. By using eBPF technology, the present invention realizes efficient data collection and secure data transmission in a multi-platform system, effectively improving the security and reliability of multi-platform kernel event monitoring and data analysis.

Claims

1. A multi-platform data collection method based on eBPF, characterized in that: The steps include: Step 1: Write the eBPF source code in the Power-eBPF editor, define specific data collection logic for different platforms, and generate the eBPF source code into an eBPF program suitable for each platform through conditional compilation; Step 2: Send the generated eBPF program to target devices on different platforms through the central controller. The central controller is responsible for managing the distribution of the eBPF program so that each platform receives the corresponding version of the eBPF program. Step 3: Start the daemon process on each platform. After receiving the eBPF program sent by the central controller, the daemon process dynamically loads the eBPF program and runs it in the kernel of the target device to collect target data of system calls, network activities, and file operations. Step 4: The daemon encapsulates the collected target data and uploads it to the central controller through an encrypted channel. The central controller receives the collected data from different platforms, standardizes the data and unifies the format to ensure data consistency on multiple platforms, and stores the data in a unified data storage module for subsequent analysis and processing; Step 5: Perform real-time monitoring and in-depth analysis of the stored data in the data processing module, identify potential security threats or abnormal behaviors through machine learning algorithms, and generate monitoring reports and security recommendations.

2. The multi-platform data collection method based on eBPF according to claim 1, characterized in that: The step 1 is specifically as follows: Step 1.1: In the Power-eBPF editor, write data collection logic according to the characteristics of different platforms, and design respective eBPF program modules according to the operating system characteristics of Linux, Windows, and Android platforms; The Linux platform uses tc and xdp to capture network data, and the Windows platform uses ETW to handle event tracking. Android uses the JNI interface to interact with eBPF and integrate the features of each platform. Step 1.2: Generate eBPF programs that meet the requirements of different platforms through the conditional compilation mechanism, and import the programs into the compiler for processing; specifically: First, the entire eBPF code is imported into the Power-eBPF compiler, and the compiler performs preliminary syntax and semantic analysis on the code. The conditional compilation mechanism is used to perform targeted code compilation for different platforms such as Linux, Windows, and Android through preprocessing instructions. The compiler first parses the conditional compilation directives in the code and checks the definitions for each platform to identify the code blocks for a specific platform; Step 1.3: After conditional compilation, the generated eBPF program is further optimized to remove redundant parts and optimize performance; the compiler links the optimized code to generate the final executable eBPF program; Step 1.4: Configure the trigger conditions and data collection parameters of the eBPF program according to the system characteristics of each platform; specifically: First, we analyze the characteristics of the system call interface, network protocol stack, and file system call of each platform. On the Linux platform, we select specific system calls as trigger conditions and use socket filters to capture network data packets. For the Windows platform, we register specific ETW events to monitor key system activities. On the Android platform, we use the Binder IPC mechanism or JNI interface calls to interact with eBPF. Step 1.5: Set the frequency and type of data collection to achieve effective data collection on different operating systems.

3. The multi-platform data collection method based on eBPF according to claim 1 is characterized in that: The step 2 is specifically as follows: The central controller first identifies the operating system type of each target device, and selects the appropriate version for distribution based on the previously compiled eBPF program; the central controller matches the identity information of the target device with its corresponding eBPF program, so that the program received by each device is compatible with its operating system; by establishing a long connection or a short connection, the central controller monitors the distribution status in real time to confirm that the target device has successfully received and loaded the corresponding eBPF program.

4. The multi-platform data collection method based on eBPF according to claim 1, characterized in that: The step 3 is specifically as follows: Step 3.1: Start a daemon on each platform to communicate continuously with the central controller; Step 3.2: Receive and parse instructions from the central controller to decide whether to load a new eBPF program or uninstall an old program. If the central controller sends a new eBPF program, the daemon first checks the currently loaded version and performs a version comparison. If a new version is detected, the daemon safely uninstalls the old version of the eBPF program and dynamically loads the newly received eBPF program to run in the kernel.

5. The multi-platform data collection method based on eBPF according to claim 1 is characterized in that: The step 4 is specifically as follows: Step 4.1: The daemon encapsulates the collected data and uploads it to the central controller through an encrypted channel; Step 4.2: During the data upload process, the daemon monitors the network status and implements a retry mechanism in case of network anomalies; after the upload is completed, the status and timestamp of the upload are recorded; Step 4.3: The central controller receives the collected data from different platforms, verifies, standardizes and unifies the format of the data; in the process of receiving the data, verifies the integrity and authenticity of the data; finally, parses the received data into a standard format for internal use and converts the timestamp into a unified time zone format.

6. The multi-platform data collection method based on eBPF according to claim 1, characterized in that: The step 5 is specifically as follows: Step 5.1: Monitor and analyze the stored data in real time in the data processing module, and identify potential security threats by comparing the performance of different devices; Step 5.2: Use machine learning algorithms to deeply analyze data, identify threat patterns and generate monitoring reports to support security decision-making; Step 5.3: The generated analysis results are used for real-time monitoring, system logging, and compliance auditing, and the central controller executes corresponding security measures or feeds back to the administrator.

Images