Method and system for optimizing eBPF monitoring service based on dynamic subscription mechanism

Abstract

The invention provides a method and system for optimizing eBPF monitoring service based on a dynamic subscription mechanism, and the method comprises the steps: an SDK packaging interface function is newly added for a user to carry out topic event subscription, and the SDK is based on a GRPC frame and comprises a GRPC server and a client component; a subscription value used for monitoring subscription / cancellation events of other users is set, and an eBPF map object B storing the subscription value is newly added and used for dynamic subscription state synchronization; optimizing an eBPF kernel program, and setting execution logic after a kernel hook function is triggered according to the subscription value; and the eBPF user mode program dynamically updates the eBPF map object B through subscription value change to realize dynamic control. According to the method, system resource waste caused by non-subscribed topic is reduced, the processing logic of the kernel hook function is dynamically adjusted, and the flexibility and real-time performance of monitoring service are guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of computer information technology, and specifically relates to a method and system for optimizing the system activity monitoring service performance of eBPF based on a dynamic subscription mechanism. Background Art

[0002] Currently, the eBPF-based system activity monitoring service on the market has been widely used in security-aware services. By subscribing to different activity information (topics), refined monitoring of system behavior can be achieved. However, this monitoring mechanism has obvious shortcomings in performance optimization.

[0003] In the prior art, the system activity monitoring service based on eBPF usually loads all monitoring logic when the system starts. Regardless of whether the topic is subscribed, all hook functions will execute the complete processing flow when triggered, including data parsing, filtering, formatting and other operations. This design lacks flexibility. When the security-aware service does not subscribe to any topic or only subscribes to a small number of topics, the unsubscribed topics will still trigger the eBPF program in the kernel and the user-mode processing flow, resulting in the processing burden of unsubscribed topics still existing, causing a waste of system resources and affecting the overall performance of the system. In particular, in high-frequency triggering scenarios, this redundant processing has a particularly significant impact on system performance. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for optimizing eBPF monitoring services based on a dynamic subscription mechanism, reduce the waste of system resources caused by unsubscribed topics, dynamically adjust the processing logic of kernel hook functions, and ensure the flexibility and real-time performance of monitoring services.

[0005] In order to achieve the above object, the technical solution of the present invention is as follows: A method for optimizing eBPF monitoring service based on a dynamic subscription mechanism, comprising: S1. Add SDK encapsulation interface functions for users to subscribe to topic events. The SDK is based on the GRPC framework and includes GRPC server and client components. S2. Set the subscription value used to monitor other users’ subscription / cancellation events, and add a new eBPFmap object B to save the subscription value for dynamic subscription state synchronization; S3, optimize the eBPF kernel program, and set the execution logic after the kernel hook function is triggered according to the subscription value; S4. The eBPF user-state program dynamically updates the eBPF map object B by subscribing to value changes to achieve dynamic control.

[0006] Furthermore, the interface function in step S1 includes subscribing to events, unsubscribing, publishing events, and reading events. The interface function is directly mapped to the corresponding operations in the GRPC framework for users to call seamlessly after connecting to the GRPC client.

[0007] Furthermore, the method for eBPF map object B to save subscription value in step S2 includes: S201, after the eBPF user state program is started, it first subscribes itself as a system user, calls the interface function for reading events and waits to obtain event information subscribed by other users; S202. When other users call the interface function for subscribing to events or unsubscribing, the GRPC server calls the interface function for publishing events to publish user information, subscription events, or types of unsubscriptions. At this time, the eBPF user-mode program receives the subscription value. S203. After receiving the subscription value, the eBPF user-mode program writes a newly added eBPF map object B for the eBPF kernel program to read and use.

[0008] Furthermore, the optimization of the eBPF kernel program in step S3 includes: S301, after the kernel hook function is triggered, the eBPF kernel program is executed, firstly reading the current subscription value from the eBPF map object B and determining whether the subscription value corresponding to the current kernel hook function is in the subscription state; S302, if not subscribed, directly terminate the eBPF kernel program and return; S303: If subscribed, continue with the subsequent data processing flow.

[0009] Further, step S4 includes: When the user's subscription status changes, the eBPF user-state program obtains the changed subscription value through the GRPC server and dynamically updates the eBPF map object B.

[0010] On the other hand, the present invention also proposes a system for optimizing eBPF monitoring services based on a dynamic subscription mechanism, comprising: SDK module: Added SDK encapsulation interface functions for users to subscribe to topic events. The SDK is based on the GRPC framework and includes GRPC server and client components. Subscription value module: sets the subscription value used to monitor other users' subscription / cancellation events, and adds an eBPF map object B to save the subscription value for dynamic subscription status synchronization; Optimization module: optimizes the eBPF kernel program and sets the execution logic after the kernel hook function is triggered according to the subscription value; Dynamic control module: The eBPF user-state program dynamically updates the eBPF map object B by subscribing to value changes to achieve dynamic control.

[0011] Furthermore, the interface functions in the SDK module include subscribing to events, unsubscribing, publishing events, and reading events. The interface functions are directly mapped to corresponding operations in the GRPC framework for users to call seamlessly after connecting to the GRPC client.

[0012] Furthermore, the subscription value module includes: Waiting unit: After the eBPF user-mode program is started, it first subscribes itself as a system user, calls the interface function for reading events, and waits to obtain event information subscribed by other users; Receiving unit: When other users call the interface function of subscribing to events or unsubscribing, the GRPC server calls the interface function of publishing events to publish user information, subscription events, or unsubscribe types. At this time, the eBPF user-mode program receives the subscription value. Writing unit: After receiving the subscription value, the eBPF user-mode program writes the newly added eBPF map object B for the eBPF kernel program to read and use.

[0013] Furthermore, the optimization module includes: Reading unit: After the kernel hook function is triggered, the eBPF kernel program is executed. First, the current subscription value is read from the eBPF map object B and whether the subscription value corresponding to the current kernel hook function is in the subscription state; Return unit: If not subscribed, the eBPF kernel program ends directly and returns; Continue unit: If subscribed, continue the subsequent data processing flow.

[0014] Furthermore, the dynamic control module includes: When the user's subscription status changes, the eBPF user-state program obtains the changed subscription value through the GRPC server and dynamically updates the eBPF map object B.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention reduces redundant processing: the kernel and user-mode processing flows corresponding to unsubscribed topics are immediately interrupted, thus avoiding unnecessary CPU and memory consumption.

[0016] 2. The present invention has dynamic adjustment flexibility: when the subscription status changes, the kernel behavior can be adjusted instantly without restarting the service or manual intervention.

[0017] 3. The performance of the present invention is significantly optimized: in high-frequency triggering scenarios, the utilization rate of system resources is significantly improved, especially for large-scale monitoring scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the process of Example 1 of the present invention.

[0019] Figure 2 It is an overall framework diagram of Example 1 of the present invention.

[0020] Figure 3 It is a schematic diagram of the dynamic control flow of Example 1 of the present invention. DETAILED DESCRIPTION

[0021] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0022] First, the professional terms involved in the embodiments are explained.

[0023] eBPF (Extended Berkeley Packet Filter): An efficient mechanism for executing user-defined programs in the kernel, used to monitor and debug system behavior.

[0024] GRPC: A high-performance, cross-language remote procedure call framework for communication between services in distributed systems.

[0025] Hook function: A specific trigger point in the operating system kernel that is used to intercept system events and perform custom processing.

[0026] Map object: A data structure in eBPF used to share data between the kernel and userland.

[0027] SDK (Software Development Kit): Software development kit used to develop applications.

[0028] The present invention is described in detail below in conjunction with specific embodiments and drawings.

[0029] Embodiment 1: This embodiment proposes a method for optimizing the performance of the eBPF system activity monitoring service based on a dynamic subscription mechanism, such as Figure 1 As shown, including: Step 1: Add a new SDK for users to subscribe to events.

[0030] This step provides a new set of software development kits (SDKs) that allow users to interact with services built on the GRPC framework through simple interface functions. The SDK supports mainstream programming languages ​​such as C / C++ and Python, and encapsulates the following four core interface functions: 1. Subscribe to events subscribe(): Users can subscribe to specific topic events through this interface to receive relevant notifications.

[0031] 2. Unsubscribe(): Allows the user to stop receiving previously subscribed event notifications.

[0032] 3. Publish events publish_events(): used to send new events to the system so that other subscribers can receive these events.

[0033] 4. Read events read_events(): Users can query and obtain event records that have occurred through this interface.

[0034] The SDK implements a complete GRPC framework, including GRPC server and client components, to ensure efficient and reliable message delivery. Figure 2 As shown in the overall framework diagram, each interface function of the SDK is directly mapped to the corresponding operation in the GRPC framework, so that users can call these functions seamlessly after connecting to the GRPC client.

[0035] In addition, the SDK is designed to simplify the user's development process, allowing users to easily integrate event processing functions into their own applications without having to deeply understand the specific implementation details of GRPC.

[0036] Step 2: Add a new eBPF map object B for dynamic subscription state synchronization.

[0037] First, in this embodiment, each system call is assigned a subscription value, such as file reading 0x01, file writing 0x02, file creation 0x04, file deletion 0x08, process creation 0x10, network connection 0x100, etc. Subsequent operations will pass this subscription value to determine different processing.

[0038] Set a special subscription value: 0, which is used to monitor other users' subscription / cancellation events.

[0039] Next, the logic of the user-mode program saving the subscription value through the eBPF map object B includes: (1) After the eBPF user-mode program is started, it first subscribes itself as a system user (subscription value: 0), calls the interface function for reading events (read_events) and waits to obtain event information subscribed by other users, such as Figure 2 The overall framework diagram is shown in the figure.

[0040] (2) When other users call the subscription event (subscribe) / unsubscription (unsubscribe) interface function, the GRPC server will call the publish event (publish_events) interface function to publish user information and subscription / unsubscription type events. At this time, the eBPF user-mode program can receive the subscription value.

[0041] (3) After receiving the subscription value, the eBPF user-mode program writes the newly added eBPF map object B.

[0042] (4) eBPF map object B is a data structure map provided by eBPF. After being defined in the kernel program, it can share data between user-mode programs and kernel programs. It is the main data exchange method of eBPF. eBPF map object B is used to save the user's subscription value, which will be read and used in the kernel program.

[0043] Step 3: Optimize the execution logic after the kernel hook function is triggered.

[0044] (1) The eBPF kernel program uses kprobe, fentry, tracepoints, etc. to establish a connection with the hook function in the kernel. It can capture and execute custom logic when the hook function in the kernel is triggered.

[0045] (2) When the corresponding event occurs and the hook function is triggered, the eBPF kernel program will also be executed. First, it reads the current subscription value from the eBPF map object B to determine whether the subscription value corresponding to the current hook function is in the subscribed state.

[0046] (3) If the subscription value corresponding to the current hook function is not in the subscription state, the eBPF kernel program is terminated directly, and the system returns to the original processing flow, and continues to wait for the capture hook function to be triggered.

[0047] (4) If the subscription value corresponding to the current hook function is in the subscribed state, the subsequent data processing flow continues, including parsing, filtering, and sending to the eBPF user-mode program through map A to achieve interaction with the user-mode.

[0048] Step 4: Dynamic control process.

[0049] When the subscription status changes, the eBPF user-mode program receives the changed subscription value through the GRPC server and dynamically updates the eBPF map object B.

[0050] This mechanism ensures that data processing of unsubscribed topic events is interrupted immediately without restarting the service or reloading the eBPF program.

[0051] The complete dynamic control process is as follows Figure 3 As shown, including: 1. After the eBPF user-mode program is started, it first subscribes itself as a system user; 2. Call the event reading interface function (read_events) to wait for event information subscribed by other users; 3. User A calls the subscribe() interface function to subscribe to the file read (0x01) event, or calls the unsubscribe() interface function to cancel the subscription to the file read event; 4. The GRPC server sends (user A, 0x01) or (user A, 0x0) to the eBPF user program; 5. The eBPF user-mode program saves 0x01 or 0x0 to map B; 6. When the system call to read a file is triggered in the system, the vfs_read function in the kernel will be executed, and the corresponding fentry_vfs_read function in the eBPF kernel program will also be executed. The fentry_vfs_read function first reads the subscription value from map B; when the subscription value is 0x0 and the corresponding 0x01 is inconsistent, it will exit directly; when the subscription value is 0x01, it will continue to process, collect the uid, exe, file name and other information of the current process, and pass it to the user program through other means. If a file write event occurs, the hook function vfs_write is triggered, and the value 0x01 in map B is also read in the fentry_vfs_write function, which is inconsistent with the corresponding file write value (0x02), so it will also exit directly.

[0052] In the method of this embodiment, GRPC may also be replaced by other cross-process communication mechanisms (such as pipelines, semaphores, socket communication, etc.).

[0053] The method described in this embodiment solves the problem of how to reduce the waste of system resources caused by unsubscribed topics, solves the problem of how to dynamically adjust the processing logic of the kernel hook function to adapt to changes in subscription status, and solves the problem of how to ensure the flexibility and real-time performance of the monitoring service while reducing the system burden.

[0054] Embodiment 2: This embodiment 2 proposes a system for optimizing eBPF monitoring services based on a dynamic subscription mechanism, including: SDK module: Added SDK encapsulation interface functions for users to subscribe to topic events. The SDK is based on the GRPC framework and includes GRPC server and client components. Subscription value module: sets the subscription value used to monitor other users' subscription / cancellation events, and adds an eBPF map object B to save the subscription value for dynamic subscription status synchronization; Optimization module: optimizes the eBPF kernel program and sets the execution logic after the kernel hook function is triggered according to the subscription value; Dynamic control module: The eBPF user-state program dynamically updates the eBPF map object B by subscribing to value changes to achieve dynamic control.

[0055] The interface functions in the SDK module include subscribing to events, unsubscribing, publishing events, and reading events. The interface functions are directly mapped to the corresponding operations in the GRPC framework for users to call seamlessly after connecting to the GRPC client.

[0056] Subscription value modules include: Waiting unit: After the eBPF user-mode program is started, it first subscribes itself as a system user, calls the interface function for reading events, and waits to obtain event information subscribed by other users; Receiving unit: When other users call the interface function of subscribing to events or unsubscribing, the GRPC server calls the interface function of publishing events to publish user information, subscription events, or unsubscribe types. At this time, the eBPF user-mode program receives the subscription value. Writing unit: After receiving the subscription value, the eBPF user-mode program writes the newly added eBPF map object B for the eBPF kernel program to read and use.

[0057] The optimization modules include: Reading unit: After the kernel hook function is triggered, the eBPF kernel program is executed. First, the current subscription value is read from the eBPF map object B and whether the subscription value corresponding to the current kernel hook function is in the subscription state; Return unit: If not subscribed, the eBPF kernel program ends directly and returns; Continue unit: If subscribed, continue the subsequent data processing flow.

[0058] The dynamic control module includes: When the user's subscription status changes, the eBPF user-state program obtains the changed subscription value through the GRPC server and dynamically updates the eBPF map object B.

[0059] The system for optimizing the eBPF monitoring service based on the dynamic subscription mechanism proposed in this embodiment can implement the method for optimizing the eBPF monitoring service based on the dynamic subscription mechanism described in Embodiment 1, and has the same technical effect as Embodiment 1.

[0060] The above-mentioned embodiments are only preferred implementations of the present invention, and are only used to help understand the method and core ideas of the present application. The protection scope of the present invention is not limited to the above-mentioned embodiments. All technical solutions under the idea of ​​the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for optimizing eBPF monitoring service based on a dynamic subscription mechanism, characterized in that: include: S1. Add SDK encapsulation interface functions for users to subscribe to topic events. The SDK is based on the GRPC framework and includes GRPC server and client components. S2. Set the subscription value used to monitor other users’ subscription / cancellation events, and add an eBPF map object B to save the subscription value for dynamic subscription state synchronization. S3, optimize the eBPF kernel program, and set the execution logic after the kernel hook function is triggered according to the subscription value; S4. The eBPF user-state program dynamically updates the eBPF map object B by subscribing to value changes to achieve dynamic control.

2. The method for optimizing eBPF monitoring service based on dynamic subscription mechanism according to claim 1, characterized in that: The interface functions in step S1 include subscribing to events, unsubscribing, publishing events, and reading events. The interface functions are directly mapped to corresponding operations in the GRPC framework for users to call seamlessly after connecting to the GRPC client.

3. The method for optimizing eBPF monitoring service based on dynamic subscription mechanism according to claim 1, characterized in that: The method for eBPF map object B to save subscription value in step S2 includes: S201, after the eBPF user state program is started, it first subscribes itself as a system user, calls the interface function for reading events and waits to obtain event information subscribed by other users; S202. When other users call the interface function for subscribing to events or unsubscribing, the GRPC server calls the interface function for publishing events to publish user information, subscription events, or types of unsubscriptions. At this time, the eBPF user-mode program receives the subscription value. S203. After receiving the subscription value, the eBPF user-mode program writes a newly added eBPF map object B for the eBPF kernel program to read and use.

4. The method for optimizing eBPF monitoring service based on dynamic subscription mechanism according to claim 1, characterized in that: The optimization of the eBPF kernel program in step S3 includes: S301, after the kernel hook function is triggered, the eBPF kernel program is executed, firstly reading the current subscription value from the eBPF map object B and determining whether the subscription value corresponding to the current kernel hook function is in the subscription state; S302, if not subscribed, directly terminate the eBPF kernel program and return; S303: If subscribed, continue with the subsequent data processing flow.

5. The method for optimizing eBPF monitoring service based on dynamic subscription mechanism according to claim 1, characterized in that: Step S4 includes: When the user's subscription status changes, the eBPF user-state program obtains the changed subscription value through the GRPC server and dynamically updates the eBPF map object B.

6. A system for optimizing eBPF monitoring services based on a dynamic subscription mechanism, characterized in that: include: SDK module: Added SDK encapsulation interface functions for users to subscribe to topic events. The SDK is based on the GRPC framework and includes GRPC server and client components. Subscription value module: sets the subscription value used to monitor other users' subscription / cancellation events, and adds an eBPF map object B to save the subscription value for dynamic subscription status synchronization; Optimization module: optimizes the eBPF kernel program and sets the execution logic after the kernel hook function is triggered according to the subscription value; Dynamic control module: The eBPF user-state program dynamically updates the eBPF map object B by subscribing to value changes to achieve dynamic control.

7. The system for optimizing eBPF monitoring services based on a dynamic subscription mechanism according to claim 6, characterized in that: The interface functions in the SDK module include subscribing to events, unsubscribing, publishing events, and reading events. The interface functions are directly mapped to the corresponding operations in the GRPC framework for users to call seamlessly after connecting to the GRPC client.

8. The system for optimizing eBPF monitoring services based on a dynamic subscription mechanism according to claim 6, characterized in that: Subscription value modules include: Waiting unit: After the eBPF user-mode program is started, it first subscribes itself as a system user, calls the interface function for reading events, and waits to obtain event information subscribed by other users; Receiving unit: When other users call the interface function of subscribing to events or unsubscribing, the GRPC server calls the interface function of publishing events to publish user information, subscription events, or unsubscribe types. At this time, the eBPF user-mode program receives the subscription value. Writing unit: After receiving the subscription value, the eBPF user-mode program writes the newly added eBPF map object B for the eBPF kernel program to read and use.

9. The system for optimizing eBPF monitoring services based on a dynamic subscription mechanism according to claim 6, characterized in that: The optimization modules include: Reading unit: After the kernel hook function is triggered, the eBPF kernel program is executed. First, the current subscription value is read from the eBPF map object B and whether the subscription value corresponding to the current kernel hook function is in the subscription state; Return unit: If not subscribed, the eBPF kernel program ends directly and returns; Continue unit: If subscribed, continue the subsequent data processing flow.

10. The system for optimizing eBPF monitoring services based on a dynamic subscription mechanism according to claim 6, characterized in that: The dynamic control module includes: When the user's subscription status changes, the eBPF user-state program obtains the changed subscription value through the GRPC server and dynamically updates the eBPF map object B.

Images