Log collection method based on eBPF, medium and system

Abstract

The invention discloses an eBPF-based log collection method, a medium and a system. The method comprises the following steps: S1, positioning a tracking point; s2, compiling an eBPF program based on the tracking points; s3, compiling and deducing the compiled eBPF program, and then loading the compiled eBPF program; s4, capturing information when the project program runs based on the eBPF program; s5, the captured information is written into the BPF Maps; and S6, the captured information is read from the BPF Maps and formatted. According to the method, on the premise that the system performance is not affected, key events can be directly captured on the kernel level, and valuable information is selectively collected; by defining a unified log format standard, the consistency and readability of log data are improved, and the troubleshooting efficiency and accuracy are enhanced; most log processing work is completed in a kernel space, dependence on user space resources is reduced, the storage performance overhead is effectively reduced, and the operation efficiency of the system is improved.

Description

Technical Field

[0001] The present invention mainly relates to, specifically, a log collection method, medium and system based on eBPF. Background Art

[0002] In the current technology environment, application log management on most Linux servers usually follows a traditional model, such as Figure 1 As shown in the figure: first record the log information into a file, and then use various command line tools or professional log analysis software to process and analyze the log data. However, this approach faces many challenges. First, since a large amount of log data needs to be written to the storage system frequently, this leads to significant storage performance overhead, which may become a performance bottleneck of the system, especially in high-concurrency scenarios. Secondly, the log formats and contents generated by different applications vary greatly, and there is a lack of unified standards, which makes it difficult to standardize the management and efficient retrieval of logs. Finally, when the system fails or anomalies, it becomes extremely complex and time-consuming to accurately and quickly locate the cause of the problem from the massive log data. Summary of the invention

[0003] In view of the technical problems existing in the prior art, the present invention provides a log collection method, medium and system based on eBPF, which reduces the dependence on user space resources, effectively reduces storage performance overhead, and improves the operating efficiency of the overall system.

[0004] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0005] A log collection method based on eBPF, comprising the steps of:

[0006] S1, positioning tracking point;

[0007] S2. Write eBPF program based on tracepoints;

[0008] S3. Compile the written eBPF program and then load it;

[0009] S4, capturing information about the project program when it is running based on the eBPF program;

[0010] S5. Write the captured information into BPF Maps;

[0011] S6. Read the captured information from BPF Maps and format it.

[0012] Preferably, in step S2, writing the eBPF program includes defining the type of tracing point, clarifying the data source, the information collection method and the exception handling method; the tracing point type includes kernel function call, system call and user space function call;

[0013] Kernel function calls: Use kprobes to insert probes at the entry and exit of kernel functions to track the behavior of specific kernel functions;

[0014] System calls: lightweight tracing of kernel events using tracepoints;

[0015] Userspace function calls: Use uprobes to insert probes at the entry and exit of userspace functions to trace the behavior of your application.

[0016] Preferably, in step S2, clarifying the data source includes: clarifying the method of obtaining data from the project application to ensure that the eBPF program can correctly capture relevant information; obtaining the PID and TID of the current process: using the bpf_get_current_pid_tgid() auxiliary function to obtain the PID and TID of the current process; obtaining the command name of the current process: using the bpf_get_current_comm() auxiliary function to obtain the command name of the current process.

[0017] Preferably, in step S2, the exception handling method is:

[0018] Define a BPF Hash Map to store the error count of each process, and count the number of errors by updating the Map when the system call returns an error. Specifically, check the return value when the system call exits. If the return value is less than zero, it means an error has occurred. At this time, get the PID, TID and current timestamp of the current process. Then find the counter corresponding to the PID in the error count Map. If it exists, increment the count, otherwise initialize the count to 1.

[0019] Define a data structure to record detailed exception logs, including the process PID, TID, command name, system call ID, return value, and timestamp; pass exception logs from kernel space to user space, and define callback functions in user space to process these logs and output exception information in real time.

[0020] Preferably, in step S2, the exception handling method further includes: inserting a probe when the process exits, capturing the exit code and exit time of the process, recording them in another BPF Map, and passing this information to the user space for subsequent analysis and processing.

[0021] Preferably, the specific process of step S4 is: the eBPF program runs in the kernel space, monitors and captures specified events or data; the events include system calls, function calls and network data packets; the data includes one or more of timestamps, process IDs, system call parameters and exception counts.

[0022] Preferably, the specific process of step S5 is: using the BPF Helper function to write the captured information into BPF Maps.

[0023] The present invention also discloses a computer program product, comprising a computer program, wherein the computer program executes the steps of the method described above when executed by a processor.

[0024] The present invention further discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described above are executed.

[0025] The present invention also discloses a log collection system based on eBPF, comprising a memory and a processor connected to each other, wherein a computer program is stored on the memory, and when the computer program is run by the processor, the steps of the method described above are executed.

[0026] Compared with the prior art, the advantages of the present invention are:

[0027] The present invention uses eBPF technology to achieve real-time collection of application logs and efficient management of exception information. Specifically, by inserting eBPF programs when system calls enter and exit, the execution time of system calls can be efficiently recorded and calculated in kernel space, and the exception count can be updated when the system call returns. User space programs can provide powerful log management and error monitoring capabilities by reading data in BPF Maps, summarizing and displaying this information. This solution can not only significantly reduce storage performance overhead and improve the standardization of log formats, but also greatly improve the efficiency and accuracy of error location, providing strong technical support for building a high-performance, highly reliable modern application log management system.

[0028] The present invention uses eBPF technology to realize real-time collection and intelligent interception of application logs. The method can capture key events directly at the kernel level without affecting system performance, and selectively collect valuable information through efficient filtering rules. At the same time, by defining a unified log format standard, the consistency and readability of log data are improved, and the efficiency and accuracy of troubleshooting are further enhanced. In addition, since most of the log processing work is completed in the kernel space, the dependence on user space resources is reduced, thereby effectively reducing storage performance overhead and improving the operating efficiency of the overall system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a traditional log analysis flow chart.

[0030] Figure 2 The flowchart of the log collection method in the present invention is an embodiment.

[0031] Figure 3 The flowchart of information data transmission is captured by the eBPF program in the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0033] Under the premise that the project program is running normally, using eBPF programs to track the project running status, obtain log content and display log file information requires custom development. Although eBPF technology was originally designed for event tracking and performance analysis, its powerful flexibility and efficiency make it an ideal choice for log management and error monitoring.

[0034] like Figure 2 As shown, the log collection method based on eBPF provided by the embodiment of the present invention specifically includes the following steps:

[0035] Prerequisite: The overall project program runs normally, without program crashes or component operation abnormalities.

[0036] Step 1: Locate the tracking point. First, it is necessary to deeply understand and analyze the structure and operation mechanism of the project program. This step is crucial because it directly affects the selection of subsequent tracking points and the accuracy of log information. Specifically, it is necessary to identify and understand the key functions, modules, libraries and expected error types in the program. In the present invention, the clear positioning target of the tracking point is the function call and system call of the project program. For this purpose, kprobes is used as the kernel hook point, and the data source of the tracking point is clearly defined as the project application.

[0037] Key points: (1) Function calls: Identify key functions that are frequently called in the project, which may be performance bottlenecks or potential sources of errors.

[0038] (2) System calls: Pay attention to the system calls frequently used in the project, which may affect the performance and stability of the system.

[0039] Step 2: Write eBPF programs. eBPF allows developers to write secure and efficient kernel programs without modifying kernel code or restarting the system. Common eBPF tools include bpfcc-tools, bpftrace, and bcc (BPF Compiler Collection), which provide rich interfaces and functions, making it easier to write and debug eBPF programs.

[0040] When writing eBPF programs, use eBPF technology to implement log management and error monitoring, including the following aspects:

[0041] (1) Tracking point type.

[0042] Kernel function calls: Use kprobes to insert probes at the entry and exit of kernel functions to track the behavior of specific kernel functions;

[0043] System calls: lightweight tracing of kernel events using tracepoints;

[0044] Userspace function calls: Use uprobes to insert probes at the entry and exit of userspace functions to trace the behavior of your application.

[0045] (2) Data source: Clarify how to obtain data from the project application to ensure that the eBPF program can correctly capture relevant information.

[0046] Get the PID and TID of the current process: Use the bpf_get_current_pid_tgid() auxiliary function to get the PID and TID of the current process;

[0047] Get the command name of the current process: Use the bpf_get_current_comm() auxiliary function to get the command name of the current process.

[0048] (3) Information collection: Write eBPF programs to capture key information in the project program, such as function call stacks, system call parameters, etc.

[0049] Capture the name and parameters of the system call: Use tracepoints to capture the name and parameters of the system call;

[0050] Capture the return value of the system call: Use tracepoints to capture the return value of the system call;

[0051] Get timestamp: Use the bpf_ktime_get_ns() helper function to get the current timestamp.

[0052] (4) Exception handling: Exception handling is a key step when writing eBPF programs to implement log management and error monitoring. In order to fully capture and handle exceptions in project programs, a variety of technical means are used, specifically:

[0053] First, define a BPF Hash Map to store the error count of each process, and count the number of errors by updating the Map when the system call returns an error. Specifically, check the return value when the system call exits. If the return value is less than zero, it means an error has occurred. At this time, get the PID and TID of the current process through bpf_get_current_pid_tgid(), and use bpf_ktime_get_ns() to get the current timestamp; then find the counter of the corresponding PID in the error count Map, and increment the count if it exists, otherwise initialize the count to 1.

[0054] In addition, a data structure is defined to record detailed exception logs, including the process PID, TID, command name, system call ID, return value and timestamp. A Perf Event Array is defined through BPF_PERF_OUTPUT to pass the exception logs from the kernel space to the user space, and a callback function is defined in the user space to process these logs and output exception information in real time.

[0055] In order to further enhance the exception handling capability, an exception exit tracking function is also implemented. By inserting a probe when the process exits, the process exit code and exit time are captured, recorded in another BPF Map, and the information is passed to the user space through a similar mechanism for subsequent analysis and processing.

[0056] Through these measures, it is possible to comprehensively monitor and record abnormal situations in project procedures, timely discover and solve potential problems, and improve the stability and reliability of the system.

[0057] Step 3: Load the eBPF program. Use clang to compile the written eBPF program to generate eBPF bytecode that can run on the Linux kernel version. Use tools such as bpftool, ip command, tc command, or custom loading scripts to load the eBPF program into the kernel's eBPF virtual machine (BPF VM). Loading the eBPF program requires administrator privileges, and the kernel will verify it to ensure legitimacy to prevent crashes or security issues.

[0058] Key point: Permission management, ensure that the operation of loading eBPF program has sufficient permissions to avoid loading failure caused by insufficient permissions.

[0059] Step 4: Capture information. The eBPF program captures information according to the defined conditions when the project is running, including but not limited to network packet information, system call parameters, function call stack, memory access, system events, errors and exception information, etc. This step is a key data collection process, and the eBPF program is used to obtain key information when the project program is running. For example, the timestamp is recorded when the system call enters, and the execution time is calculated when the system call exits, and the results are stored in BPF Maps.

[0060] Step 5: Data transfer. eBPF programs are executed in kernel space, while human-computer interaction usually takes place in user space. Therefore, the captured information needs to be efficiently transferred to user space for further processing. This process is done through shared memory areas. Specifically, the captured information is written to BPF Maps using BPF Helper functions, so that user space applications can access these Maps and read the information stored in the kernel.

[0061] Step 6: Format log output display. User space applications use tools such as the libbpf library to read the captured information from BPFMaps and perform corresponding processing, such as analysis and visualization. By formatting the log output display, users can more clearly understand the running status of the project program, obtain the project program running log information, and perform error location and problem analysis.

[0062] like Figure 3 As shown, the above data transmission process is specifically as follows:

[0063] Capture information: eBPF programs run in kernel space, listening to and capturing specified events or data. These events can be system calls, function calls, network packets, etc. The captured information includes but is not limited to timestamps, process IDs, system call parameters, exception counts, etc. This information is critical for performance analysis and error diagnosis.

[0064] Writing BPF Maps: Use BPF Helper functions (such as bpf_map_update_elem()) to write the captured information to BPF Maps. BPF Maps is an efficient data structure for passing and sharing data between kernel space and user space. The written data can be in the form of key-value pairs, for example, the key is the process ID and the value is a timestamp or anomaly count. BPFMaps supports a variety of data types, such as arrays, hash tables, etc. You can choose the appropriate type according to your specific needs.

[0065] Reading information: Userspace applications read information stored in the kernel through the BPF Maps interface. Using the libbpf library or other tools, userspace programs can easily access data in BPF Maps. The data read can be a single entry or batch data, depending on the application scenario. For example, you can read the data of a single entry through the bpf_map_lookup_elem() function, or use the bpf_map_lookup_batch() function to batch read the data of multiple entries.

[0066] Data processing: User space applications process the read information, such as aggregation, analysis, and visualization. The processed information can be output to the console, log files, or other storage media for subsequent viewing and analysis. Data processing steps include data cleaning, aggregation, statistical analysis, etc. to extract useful information and generate reports. These reports can help developers and operation and maintenance personnel better understand the operating status of the system and discover and solve problems in a timely manner.

[0067] The present invention uses eBPF technology to achieve real-time collection of application logs and efficient management of exception information. Specifically, by inserting eBPF programs when system calls enter and exit, the execution time of system calls can be efficiently recorded and calculated in kernel space, and the exception count can be updated when the system call returns. User space programs can provide powerful log management and error monitoring capabilities by reading data in BPF Maps, summarizing and displaying this information. This solution can not only significantly reduce storage performance overhead and improve the standardization of log formats, but also greatly improve the efficiency and accuracy of error location, providing strong technical support for building a high-performance, highly reliable modern application log management system.

[0068] The present invention uses eBPF technology to realize real-time collection and intelligent interception of application logs. The method can capture key events directly at the kernel level without affecting system performance, and selectively collect valuable information through efficient filtering rules. At the same time, by defining a unified log format standard, the consistency and readability of log data are improved, and the efficiency and accuracy of troubleshooting are further enhanced. In addition, since most of the log processing work is completed in the kernel space, the dependence on user space resources is reduced, thereby effectively reducing storage performance overhead and improving the operating efficiency of the overall system.

[0069] The eBPF technology of the present invention allows developers to customize data capture and processing logic according to specific needs, so as to better meet specific monitoring and error reporting requirements. By optimizing the log format and providing more accurate error location, the efficiency of problem location can be greatly improved.

[0070] In specific applications, the log collection method based on eBPF of the present invention includes the following corresponding contents:

[0071] (1) eBPF program tracking and log information acquisition

[0072] The present invention uses eBPF technology to monitor the error and abnormal exit of the application in real time, and efficiently stores the relevant information in the kernel space, avoiding frequent disk write operations. This process is achieved by writing an eBPF program. The following is a related pseudo code example:

[0073]

[0074] Method 1: When the system call starts, an eBPF program is triggered. The program first uses the bpf_get_current_pid_tgid() function to get the PID and TID of the current process; then, it uses the bpf_ktime_get_ns() function to get the current timestamp in nanoseconds; then, the obtained PID and timestamp are stored as key-value pairs in BPF_MAP; finally, 0 is returned to indicate that the program is executed successfully.

[0075] Method 2: When the system call ends, another eBPF program is triggered. The program first uses the bpf_get_current_pid_tgid() function to obtain the PID and TID of the current process; then, use

[0076] The bpf_map_lookup_elem() function looks up the stored timestamp from the BPF_MAP. If the timestamp is found, it proceeds to the next step. Otherwise, it skips the subsequent steps. Then, it uses the bpf_ktime_get_ns() function to get the current timestamp in nanoseconds. Next, it calculates the execution time of the system call, which is the current timestamp minus the stored timestamp. Finally, it prints the PID and execution time to the kernel log using the bpf_trace_printk() function and returns 0 to indicate successful program execution.

[0077] Method 3: In the user space program, first read the stored system call timestamp and execution time information from BPF_MAP, then summarize and analyze the read information to generate the required statistical report; finally, the summary results can be output to the console, file or other storage media for further analysis and use.

[0078] Key function description:

[0079] bpf_get_current_pid_tgid(): Get the PID and TID of the current process;

[0080] bpf_ktime_get_ns(): Get the current timestamp in nanoseconds;

[0081] bpf_map_update_elem(): inserts or updates an element into BPF_MAP;

[0082] bpf_map_lookup_elem(): Look up an element in BPF_MAP;

[0083] bpf_trace_printk(): Prints information to the kernel log.

[0084] (2) Application stack error information display

[0085] In order to further improve the efficiency and accuracy of error monitoring, the present invention also provides a mechanism for tracking the number of exceptions when the system call exits, and summarizing and displaying the exception information in the user space. The following is a related pseudo code example:

[0086]

[0087] Method 1: Create a hash table named exception_counts with keys of type u32 to store exception counts.

[0088] Method 2: When the system call returns, an eBPF program is triggered. The program first uses the bpf_get_current_pid_tgid() function to get the PID and TID of the current process. Next, the exception_counts.lookup_or_init() function is used to find or initialize the exception count corresponding to the PID in the exception_counts hash table. Then, the exception count found is increased by 1. Finally, 0 is returned to indicate that the program execution is successful.

[0089] Method 3: In the user space program, first define the key and value of the hash table, and use the bpf_table_lookup_batch() function to batch retrieve the data in the exception_counts hash table. Then, traverse the batched data and check the exception count in each key-value pair. If the exception count of a process is greater than 0, use the printf() function to print the exception information to the user space. Finally, continue traversing until all data has been processed.

[0090] Key function description:

[0091] BPF_PROG_KRETPROBE: eBPF program tied to the return of a system call to update the exception count when the system call returns.

[0092] exception_counts.lookup_or_init: Look up or initialize the exception count corresponding to the given PID in the exception_counts hash table.

[0093] BPF_TABLE_LOOKUP_BATCH: Get data from the exception_counts hash table in batches to improve performance.

[0094] printf: Print exception information to user space.

[0095] Explanation of relevant technical terms:

[0096] Tracepoint: It is a named event marker inserted in the kernel or user space application. These markers allow developers to define places of interest during program execution, such as function entry, exit, system call, etc. Each tracepoint has a unique identifier and event description information associated with it. This description information includes the event name, parameter type and other related information.

[0097] Probe: refers to the specific location where the eBPF program is inserted, used to monitor a specific event or perform a specific operation. eBPF programs can have different types, such as Kprobe, Tracepoint, XDP, etc. In the context of eBPF, probes are usually associated with specific eBPF program types.

[0098] BPF Map: It is a data structure used to pass and share data between different parts of the eBPF program. BPF Map provides a mechanism for exchanging data between kernel space and user space, allowing eBPF programs to access and modify shared data without directly communicating with other parts of the kernel space or user space.

[0099] The present invention also discloses a computer program product, including a computer program, which executes the steps of the above method when executed by a processor. The present invention further discloses a computer-readable storage medium, on which a computer program is stored, which executes the steps of the above method when executed by a processor. The present invention also discloses a log collection system based on eBPF, including an interconnected memory and a processor, on which a computer program is stored, which executes the steps of the above method when executed by a processor. The product, medium and system of the present invention correspond to the above method and also have the advantages described in the above method.

[0100] The present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiment when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Computer-readable storage media include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. The memory is used to store computer programs and / or modules, and the processor implements various functions by running or executing computer programs and / or modules stored in the memory, and calling data stored in the memory. The memory may include high-speed random access memory and may also include non-volatile memory, such as a hard disk, an internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0101] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A log collection method based on eBPF, characterized in that: Includes steps: S1, positioning tracking point; S2. Write eBPF program based on tracepoints; S3. Compile the written eBPF program and then load it; S4, capturing information about the project program when it is running based on the eBPF program; S5. Write the captured information into BPF Maps; S6. Read the captured information from BPF Maps and format it.

2. The log collection method based on eBPF according to claim 1, characterized in that: In step S2, writing the eBPF program includes defining the type of tracing point, clarifying the data source, information collection method and exception handling method; the tracing point type includes kernel function call, system call and user space function call; Kernel function calls: Use kprobes to insert probes at the entry and exit of kernel functions to track the behavior of specific kernel functions; System calls: lightweight tracing of kernel events using tracepoints; Userspace function calls: Use uprobes to insert probes at the entry and exit of userspace functions to trace the behavior of your application.

3. The log collection method based on eBPF according to claim 2, characterized in that: In step S2, clarifying the data source includes: clarifying the method of obtaining data from the project application to ensure that the eBPF program can correctly capture relevant information; obtaining the PID and TID of the current process: using the bpf_get_current_pid_tgid() auxiliary function to obtain the PID and TID of the current process; obtaining the command name of the current process: using the bpf_get_current_comm() auxiliary function to obtain the command name of the current process.

4. The log collection method based on eBPF according to claim 3 is characterized in that: In step S2, the exception handling method is: Define a BPF Hash Map to store the error count of each process, and count the number of errors by updating the Map when the system call returns an error. Specifically, check the return value when the system call exits. If the return value is less than zero, it means an error has occurred. At this time, get the PID, TID and current timestamp of the current process. Then find the counter corresponding to the PID in the error count Map. If it exists, increment the count, otherwise initialize the count to 1. Define a data structure to record detailed exception logs, including process PID, TID, command name, system call ID, return value, and timestamp; Pass exception logs from kernel space to user space, and define callback functions in user space to process these logs and output exception information in real time.

5. The log collection method based on eBPF according to claim 4, characterized in that: In step S2, the exception handling method also includes: inserting a probe when the process exits, capturing the exit code and exit time of the process, recording them in another BPF Map, and passing this information to the user space for subsequent analysis and processing.

6. The log collection method based on eBPF according to any one of claims 1 to 5, characterized in that: The specific process of step S4 is: the eBPF program runs in the kernel space, monitors and captures specified events or data; the events include system calls, function calls and network data packets; the data includes one or more of timestamps, process IDs, system call parameters and exception counts.

7. The log collection method based on eBPF according to claim 6, characterized in that: The specific process of step S5 is: using the BPF Helper function to write the captured information into BPF Maps.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are performed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program performs the steps of the method according to any one of claims 1 to 7.

10. A log collection system based on eBPF, comprising a memory and a processor connected to each other, wherein a computer program is stored in the memory, characterized in that: When the computer program is executed by a processor, the computer program performs the steps of the method according to any one of claims 1 to 7.

Images