An eBPF-based process crash information collection method and device

Abstract

The application discloses a process crash information collection method and device based on eBPF, comprising the following steps: using an eBPF user state component to mount a pre-compiled eBPF kernel state component on a kernel state function triggered when a process exits in a Linux system; when a process running in the Linux system exits, triggering the eBPF kernel state component to intercept the process and judging whether the process is an abnormal exit; if the process is an abnormal exit, using the eBPF kernel state component to obtain process crash data of the process and sending the process crash data into a BPF ring buffer; using the eBPF user state component to obtain the process crash data from the BPF ring buffer, and generating corresponding process crash logs according to the process crash data, so as to accurately locate the fault cause. The application can automatically obtain key data of a crash process directly in the kernel, can greatly reduce the system load in a production environment, and is convenient for operation and maintenance personnel to quickly analyze crash information and accurately locate the fault cause.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to a method and apparatus for collecting process crash information based on eBPF. Background Technology

[0002] With the rapid development of the Internet, cloud computing, the Internet of Things, e-commerce, and other fields, more and more applications (APPs) are being widely used. During the operation of these applications, important operational data, namely log data, is generated. Operation and maintenance personnel can use log data for anomaly detection and fault diagnosis.

[0003] In reality, applications sometimes encounter unrecoverable exceptions, causing them to crash. In Linux environments, system administrators typically use the core dump mechanism to generate crash logs for abnormally exiting processes in order to reproduce the system crash scenario. However, the core file generated by the core dump mechanism is a memory image of the crashed process, which may consume a high system load. Furthermore, the information in the core file is unstructured, making it difficult to quickly and accurately locate the problem. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for collecting process crash information based on eBPF, so as to solve the technical problem that the existing technology requires a high system load when collecting process crash information and is not conducive to quickly and accurately locating the cause of the failure.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A method for collecting process crash information based on eBPF, comprising:

[0007] By using eBPF user-space components, pre-compiled eBPF kernel-space components are mounted onto kernel-space functions triggered when a process exits in the Linux system;

[0008] When a process running in the Linux system exits, the eBPF kernel-mode component is triggered to intercept the process and determine whether the process exited abnormally.

[0009] If the process exits abnormally, the process crash data of the process is obtained using the eBPF kernel-mode component and the process crash data is sent into the BPF circular buffer.

[0010] The process crash data is obtained from the BPF circular buffer using the eBPF user-space component, and a corresponding process crash log is generated based on the process crash data to accurately locate the cause of the failure.

[0011] Optionally, the kernel-mode function is the do_exit() function.

[0012] Optionally, the eBPF user-space component includes an eBPF user-space initialization component and an eBPF user-space output component. Mounting the pre-compiled eBPF kernel-space component onto a kernel-space function triggered when a process exits in the Linux system using the eBPF user-space component includes:

[0013] By using eBPF user-mode initialization components and BPF system calls, pre-compiled eBPF kernel-mode components are mounted onto kernel-mode functions triggered when a process exits in the Linux system.

[0014] Optionally, triggering the eBPF kernel-mode component to intercept the process and determine whether the process exited abnormally includes:

[0015] The eBPF kernel-mode component is triggered to intercept the process and obtain the corresponding process exit code. Based on the process exit code, it is determined whether the process exited abnormally.

[0016] Optionally, using the eBPF kernel-mode component to send the process crash data into the BPF circular buffer includes:

[0017] The process crash data is sent to the BPF circular buffer in the form of an event structure using the eBPF kernel-mode component.

[0018] Optionally, the process crash data includes at least:

[0019] The process exit code, kernel-mode function call stack, user-mode function call stack, and virtual address space information of the dynamic library files that the process depends on, wherein the virtual address space information includes at least several contiguous virtual memory segments.

[0020] Optionally, the eBPF user-space component includes an eBPF user-space initialization component and an eBPF user-space output component. Generating a corresponding process crash log based on the process crash data using the eBPF user-space component includes:

[0021] The process crash data is analyzed and reconstructed using the eBPF user-space output component, and a corresponding process crash log is generated at a specified path.

[0022] Optionally, analyzing and reconstructing the process crash data using the eBPF user-space output component includes:

[0023] The eBPF user-mode output component is used to convert each virtual address in the kernel-mode function call stack into a unique kernel function name plus a first offset.

[0024] Each virtual address in the user-mode function call stack is converted into a unique user function name plus a second offset;

[0025] The absolute path of the dynamic library file that the process depends on is obtained from the virtual memory segment.

[0026] Optionally, after generating the corresponding process crash log based on the process crash data using the eBPF user-space component, the process crash log further includes:

[0027] The process crash log is output to the terminal using the eBPF user-space component.

[0028] The present invention also provides an eBPF-based process crash information collection device, comprising:

[0029] The component initialization module is used to mount pre-compiled eBPF kernel-mode components onto kernel-mode functions triggered when a process exits in the Linux system using eBPF user-mode components.

[0030] The abnormal process exit judgment module is used to trigger the eBPF kernel-mode component to intercept the process and determine whether the process exited abnormally when a process running in the Linux system exits.

[0031] The process crash data acquisition module is used to acquire the process crash data of the process and send the process crash data into the BPF circular buffer if the process exits abnormally.

[0032] The process crash log generation module is used to obtain the process crash data from the BPF circular buffer using the eBPF user-space component, and generate the corresponding process crash log based on the process crash data in order to accurately locate the cause of the failure.

[0033] This invention provides a method and apparatus for collecting process crash information based on eBPF. The method includes: using an eBPF user-space component to mount a pre-compiled eBPF kernel-space component onto a kernel-space function triggered when a process exits in a Linux system; when a running process in the Linux system exits, triggering the eBPF kernel-space component to intercept the process and determine whether the process exited abnormally; if the process exited abnormally, using the eBPF kernel-space component to obtain the process crash data and sending the process crash data into a BPF circular buffer; using the eBPF user-space component to obtain the process crash data from the BPF circular buffer and generating a corresponding process crash log based on the process crash data to accurately locate the cause of the failure.

[0034] In view of this, the beneficial effects of this invention are:

[0035] This invention utilizes eBPF's kprobe technology to mount eBPF kernel-mode components onto kernel-mode functions triggered when a process exits. This automatically intercepts all abnormally exiting processes, bypassing the generation and analysis of core files. It directly and automatically retrieves key data of abnormally exiting processes from the kernel and transmits it back to the BPF circular buffer, consuming very few I / O and disk resources, thus significantly reducing system load in production environments. Furthermore, by using eBPF user-mode components to retrieve key data of crashed processes from the BPF circular buffer and generate corresponding process crash logs, this invention helps system administrators quickly analyze crash information and accurately locate the cause of the failure. Attached Figure Description

[0036] Figure 1 This is a schematic flowchart of the method of the present invention;

[0037] Figure 2 This is a flowchart of the core architecture of the present invention;

[0038] Figure 3 This is a flowchart illustrating the virtual address translation process of the kernel-mode function call stack in this invention.

[0039] Figure 4 This is a flowchart illustrating the virtual address translation process of the user-mode function call stack in this invention.

[0040] Figure 5 This is a schematic diagram of the process crash log generated under a specified path in an embodiment of the method of the present invention;

[0041] Figure 6 This is a schematic diagram of the logs generated for a test case of a crash process in this invention;

[0042] Figure 7 This is a schematic diagram showing the size of the Core file generated by the crash process test sample of this invention;

[0043] Figure 8 This is a schematic diagram showing the size of the process crash log generated for the crash process test sample of this invention;

[0044] Figure 9 This is a schematic diagram of the structure of an embodiment of the device of the present invention. Detailed Implementation

[0045] eBPF: eBPF is a popular Linux tool that allows user-defined programs to run at the Linux kernel level. eBPF programs are attached to specified locations in the kernel (hook points), and whenever a program reaches that location, the attached eBPF program is also triggered to execute. eBPF can be used for operations such as filtering traffic, classifying traffic, classifying networks, and modifying socket settings. eBPF programs use eBPF Maps to store data status, statistics, etc. eBPF Maps can be accessed by both eBPF programs and user space, thus enabling information sharing between eBPF programs and between user space and eBPF programs. eBPF programs can only run specified helper functions; most Linux system functions are unavailable.

[0046] Core dump: When the operating system crashes due to an abnormal situation, it saves the state information in memory and synchronizes it to disk or sends it to a remote host over the network. This information is used to diagnose the fault and find the cause of the system crash.

[0047] This invention provides a method and apparatus for collecting process crash information based on eBPF, in order to solve the technical problem that the prior art requires a high system load when collecting process crash information and is not conducive to quickly and accurately locating the cause of the failure.

[0048] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0050] In reality, applications sometimes encounter unrecoverable exceptions, leading to crashes. In real-world production environments, exception handling frameworks used by applications running in non-virtual environments often fail to capture and handle system errors such as stack overflows and dangling pointers accessing invalid memory. In Linux environments, system administrators typically use core dumps to reproduce system crashes. However, correctly and accurately using core dumps in production environments requires consideration of numerous details and production-specific issues, and system administrators often go through a cumbersome process to locate and resolve errors. Furthermore, the unstructured information in core dumps hinders the rapid and accurate identification of system problems by administrators.

[0051] Specifically, the core dump mechanism needs to monitor whether there are new files in the path where core files are generated when processes exit abnormally. If so, it indicates that a new abnormally exited process has occurred. A script then uses debugging tools like gdb to extract key information about the abnormal process exit from the newly generated core file and writes it to the log. However, this approach still relies on the generation of core files, so the problem of excessive system resource consumption still exists in certain scenarios.

[0052] In a Unix environment, the primary method for locating the cause of an application's abnormal termination due to an exception or receiving certain signals in a non-virtualized runtime environment is to utilize the core dump mechanism to save the context of the abnormal process termination and then use system debugging tools such as gdb to debug that context. The specific process is as follows: When the core dump mechanism is enabled, a core file is generated when a process terminates abnormally. To accurately reproduce the crash scenario using the core file, system administrators need a binary file of the crashed program with a symbol table (or a .idy file and a binary file without a symbol table). The debugging environment must be consistent with the production environment, including the application's dependent package versions, operating system architecture and version, etc. Then, using gdb and the core file, we can obtain the signal that triggered the process crash, the memory layout at the time of the crash, the user-level function call stack backtrace, and the contents of user-mode registers.

[0053] The main drawback of existing methods for generating abnormal process exit logs using the core dump mechanism is the high system load. The core file generated by the core dump mechanism is a memory image of the crashed process. For complex programs, this core file can be quite large, and the workload of generating this file can also consume a significant amount of system resources. For example, in some producer-consumer scenarios, if the consumer thread crashes and repeatedly restarts and crashes again, the frequent generation of core files can lead to excessive system resource consumption, such as disk and I / O.

[0054] The increasingly popular Linux eBPF technology allows user-defined programs to be mounted onto kernel functions. After a process terminates abnormally, before the kernel functions reclaim system resources, user-defined programs can be obtained to retrieve the process's memory data, without needing to debug the file image within the core file. Therefore, eBPF can directly bypass the coredump mechanism to generate process crash logs, avoiding the system resource consumption associated with generating and debugging core files. Furthermore, eBPF provides structured information at the process level, facilitating rapid analysis of crash information and accurate identification of the cause of the failure.

[0055] Addressing the shortcomings of existing technologies, this application utilizes the eBPF mechanism to skip the core file generation stage, directly obtaining the crucial information needed when a process abnormally exits, thereby reducing system load in a production environment. This application leverages eBPF technology to achieve low-load automatic acquisition of on-site information when a program encounters a system error, providing system administrators with timely log feedback to help quickly locate and resolve problems.

[0056] Please see Figure 1 This invention provides an embodiment of a process crash information collection method based on eBPF, comprising:

[0057] S100: Using eBPF user-space components, pre-compiled eBPF kernel-space components are mounted onto kernel-space functions triggered when a process exits in the Linux system;

[0058] S200: When a process running in the Linux system exits, the eBPF kernel-mode component is triggered to intercept the process and determine whether the process exited abnormally.

[0059] S300: If the process exits abnormally, the process crash data of the process is obtained using the eBPF kernel-mode component and the process crash data is sent into the BPF circular buffer.

[0060] S400: Use the eBPF user-space component to obtain the process crash data from the BPF ring buffer, and generate a corresponding process crash log based on the process crash data to accurately locate the cause of the failure.

[0061] A complete eBPF program typically consists of two parts: user space and kernel space. The user-space program interacts with the kernel through BPF system calls to complete tasks such as loading the eBPF program, mounting events, and creating and updating mappings. In kernel space, the eBPF program cannot arbitrarily call kernel functions; instead, it needs to use BPF helper functions to complete the required tasks. Especially when accessing memory addresses, it is essential to use the `bpf_probe_read` family of functions to read memory data to ensure safe and efficient memory access. When an eBPF program requires large blocks of storage, we also need to introduce specific types of BPF mappings based on the application scenario and use them to provide runtime status data to the user-space program.

[0062] This invention implements a system crash log generation component using eBPF technology. It collects process crash information from crashed processes in kernel space and then sends this information back to user space, where a user-space program persists the information as a crash log. The entire system crash log generation component consists of two parts: an eBPF user-space component and an eBPF kernel-space component. The eBPF user-space component includes an eBPF user-space initialization component and an eBPF user-space output component.

[0063] Please see Figure 2 The core architecture flowchart of this invention includes two dimensions: data space and process time. Looking at the vertical axis, the process data space is divided into user data space and kernel data space, indicating that the system crash log generation component spans the user mode and kernel mode of the operating system; the component reads key information about the crashed process from the kernel mode and analyzes and displays the cause of the process crash in user mode. Looking at the horizontal axis, the component's workflow is mainly divided into three stages:

[0064] (1) Component deployment phase: The pre-compiled eBPF kernel-mode component is injected into the Linux system kernel through BPF system calls to complete the component initialization. Specifically, the pre-compiled eBPF kernel-mode component is mounted on a kernel-mode function (e.g., the do_exit() function) triggered when a process exits in the Linux system.

[0065] (2) Crash Detection: The `do_exit()` kernel-mode function is triggered whenever any running thread (normal / abnormal) exits. Leveraging this feature, the eBPF kernel-mode component anchors the trigger point to the `do_exit()` function. When this function is executed, the eBPF kernel-mode component is triggered, and the original crashed process is blocked. This allows the eBPF kernel-mode component to capture key information about the crashed process—i.e., process crash data—from user-mode applications and kernel process information. Process crash data includes at least: the process exit code, the process's function call stack, and the process's dynamic dependencies.

[0066] (3) Information summarization: After the eBPF kernel-mode component reads the key information, it integrates the key information and sends it into the BPF ring buffer in the form of an event structure; the eBPF user-mode output component polls the BPF ring buffer to obtain and analyze the key information of the crashed process and generate the process crash log file in the specified path with a preset structure.

[0067] In this embodiment, the entire system crash log generation component consists of two parts: an eBPF user-space component and an eBPF kernel-space component.

[0068] (1) eBPF user-space components (eBPF user-space initialization component and eBPF user-space output component)

[0069] In this embodiment, the eBPF user-space component may include an eBPF user-space initialization component and an eBPF user-space output component.

[0070] The main functions of the user-mode program include: ① During component initialization, the user-mode component is initialized using eBPF to mount the pre-compiled eBPF kernel-mode component to the kernel-mode function (e.g., do_exit() function) triggered when the process exits in the Linux system. It also sets the path for the process crash log to be generated based on the command-line parameters when running the component (e.g., the default log generation path is / var / log / crashlog).

[0071] During component initialization, the eBPF user-mode initialization component in this embodiment injects the pre-compiled eBPF kernel-mode component into the Linux system kernel through the BPF system call, and then enters the running state. It will continuously listen for data returned by the kernel-mode eBPF program in the ring buffer.

[0072] Specifically, listening can be implemented in the following ways:

[0073] rb=ring_buffer_new(bpf_map_fd(skel->maps.rb),handle_event,NULL,NULL);

[0074] while(!exiting)

[0075] {

[0076] err=ring_buffer_poll(rb,100);

[0077] }

[0078] This embodiment defines an `event` structure as shown in the code below, based on the key data of the abnormally exiting process that needs to be obtained. This `event` structure can be used as a carrier for data returned from the eBPF kernel-mode component to the user-mode program. It can be understood that in this embodiment, the eBPF kernel-mode component sends process crash data into the BPF circular buffer in the form of an `event` structure.

[0079] struct mmap_struct{

[0080] unsigned long start; / / Starting address of memory mapping

[0081] unsigned long end; / / Memory mapping termination address

[0082] unsigned long flags; / / Permissions

[0083] unsigned long long pgoff; / / Page offset

[0084] unsigned long ino; / / memory-mapped incode

[0085] dev_t dev; / / Device number corresponding to the memory mapping

[0086] char name[MAX_LEVEL][MAXLEN_VMA_NAME+1]; / / Absolute path of the file corresponding to the memory mapping

[0087] };

[0088] struct event{

[0089] pid_t pid; / / Process ID information

[0090] pid_t tid; / / Thread ID information

[0091] pid_t ppid; / / Parent process ID information

[0092] int sig; / / Process termination signal

[0093] int exit_code; / / Process exit code

[0094] char comm[TASK_COMM_LEN]; / / Command corresponding to the process

[0095] unsigned long stack_id; / / The index in the corresponding stacktrace map

[0096] unsigned long count; / / Memory-mapped count

[0097] struct mmap_struct mmap[MAX_VMA_ENTRY]; / / Records memory mappings

[0098] };

[0099] In this embodiment, the event structure defines the key information that we want to obtain from the crashed process, namely the process crash data, which includes at least the following information:

[0100] (1) Process exit code

[0101] According to Linux documentation, the reason for a process exiting in a Linux system can be indicated by the `exitcode` parameter of the `do_exit()` function, which is the process exit code. The reason for the process crash can be obtained from the process exit code. If the process behaves abnormally, such as division by zero or accessing invalid memory, it will trigger an exception and enter the exception handler. The operating system will identify the type of abnormal behavior and include the reason in the `exitcode`. It can be understood that there is a one-to-one correspondence between process exit codes and the types of exceptions that caused the process crash.

[0102] When a process's exit code is between 129 and 165, it indicates that the process terminated abnormally because the operating system sent a signal with sequence number n (called signal n) to the abnormal process. The process termination caused by signal n has an exit code of 128 + n. For example, if an abnormal process has an exit code of 139, then n in signal n can be calculated to be 11. According to the Linux system's signal definition, this signal can be found to be SIGSEGV. If developers see SIGSEGV as the signal that caused the process crash in the process crash log, they can determine that the program has encountered a segmentation fault.

[0103] It should be noted that in Linux systems, in addition to exit codes in the range of 129 to 165 indicating abnormal process termination caused by a certain signal, there are also exit codes in the ranges of 1 to 2, 126 to 128, and 255. The operating system assigns special meanings to these exit codes. For example, when exit code is 1, it indicates that the process termination was caused by a logical error in the process itself, such as a division-by-zero exception or the process actively exiting by calling exit(1).

[0104] In this embodiment, the eBPF user-space output component can be used to put the process exit code into the process crash log, allowing developers to determine the cause of the crash based on the content of the process exit code.

[0105] (2) Function call stacks (backtrace) in user mode and kernel mode

[0106] In one embodiment of the present invention, the bpf_get_stackid() function provided by the Linux system can be used to obtain the function call stack in user mode or kernel mode. Each jump address in the function call stack in user mode and kernel mode when the crashing process crashes is stored in a MAP data structure of type BPF_MAP_TYPE_STACK_TRACE. That is, the MAP stores the corresponding return address of the function call chain.

[0107] The addresses stored in the MAP are virtual addresses within the crash program's address space. To provide developers with useful information, these virtual addresses need to be translated into function symbol names plus offsets that developers can recognize. It should be noted that function symbol names include user-defined function symbol names (referred to as user function names) and kernel function symbol names provided by the Linux system (referred to as kernel function names). This work is performed by the eBPF user-space output component after obtaining the event structure returned from kernel space, and the specific operations are as follows:

[0108] Please see Figure 3 For the kernel-mode function call stack, the ` / proc / kallsyms` kernel symbol table provided by the Linux system is used directly. Since the virtual address space of kernel symbols is shared by all processes, and there is a one-to-one correspondence between their virtual addresses and kernel function names in the `kallsyms` symbol table, the virtual address of the kernel-mode function call stack can be directly used to find the closest kernel function name in the kernel symbol table, and then the offset is calculated. Ultimately, a unique kernel function name plus the corresponding offset (first offset) can be obtained for each virtual address of the kernel-mode function call stack.

[0109] Please see Figure 4For the user-mode function call stack, the absolute path of the corresponding binary file of the crashed process, extracted from the kernel-mode memory of the crashed process by the eBPF kernel-mode component, needs to be passed back to the eBPF circular buffer so that the eBPF user-mode output component can access and use it. The eBPF user-mode output component checks whether the corresponding binary file exists on the obtained absolute path. If it does not exist, it cannot resolve the function name belonging to the file and throws an error. If it exists, the symbol table of the program can be obtained using the nm command in the Linux system. Then, the segment start address mapped to the virtual address space of each segment of the program can be obtained using the readelf instruction. Next, each virtual address in the function call stack is converted into an offset address in the binary file according to the mapping relationship obtained from the readelf command. Then, the user function name with the closest offset address is found in the symbol table obtained from the nm command, and the offset is calculated. In this way, each virtual address in the user-mode function call stack can be located with a unique user function name + corresponding offset (second offset).

[0110] (3) Dynamic library information that the process depends on

[0111] In this embodiment, the procfs provided by the Linux system can be used to view information about running processes. Specifically, / proc / <pid>The ` / maps` attribute displays the layout of the process's virtual address space: the starting virtual address, ending virtual address, access permissions, absolute path of the binary file to which the memory-mapped segment belongs, and the file offset of the memory-mapped segment within the binary file for each memory-mapped segment. All dynamic libraries that a process depends on are mapped to several contiguous virtual memory segments. This embodiment can analyze all memory segments in a process's virtual address space to obtain the absolute path of the binary file to which it belongs, and further, the absolute paths of the dynamic libraries that the process depends on.

[0112] In this embodiment, obtaining the paths to the dynamic libraries that the crashed process depends on helps to obtain the version information of its dependent libraries and helps to analyze program errors caused by potential environment variable configuration errors. However, procfs does not retain information about crashed processes, so it is not possible to directly read / proc / <pid>The ` / maps` command is used to obtain the dynamic library path. When a process exits and triggers the eBPF kernel-mode component, the process's memory information has not yet been destroyed or reclaimed. At the same time, the `task_struct` structure in the process's kernel-mode data space stores all the process's memory information, including information about each memory-mapped segment in the virtual address space.

[0113] The following is the specific process for eBPF kernel-mode components to obtain the dynamic library paths that the crashed process depends on:

[0114] In this embodiment, the eBPF kernel-mode component reads the virtual memory mapping list `vm_area_struct*` from the `mm_struct` structure (which stores memory management information) within the process structure `task_struct` through a BPF helper function. Specifically, it reads the linked list starting with `task_struct->mm_struct->mmap`, where each element is a `vm_area_struct` structure. Each `vm_area_struct` structure in the virtual memory mapping list represents a contiguous segment of memory in the process's virtual address space.

[0115] In this embodiment, to determine whether an item in the virtual memory mapping list originates from a dynamic library, it is first necessary to determine whether the memory mapping segment is a file mapping. If the file pointer of type `structfile*` (`vm_file`) in the `vm_area_struct` structure is not null, it indicates that the memory mapping segment is derived from a file mapping; otherwise, it indicates that the memory mapping segment is not derived from a file mapping. If it is a file mapping, the `dentry` object in the `f_path` of the `file` structure pointed to by the `vm_file` pointer can be used, referring to the logic of the `d_path()` kernel function, to store the directory names of each level of the directory structure where the `dentry` resides in the `name` member of the `mmap_struct` array in the `event` structure. After the eBPF user-space output component obtains the `event` structure from the BPF circular buffer, it will reconstruct the absolute path corresponding to the memory mapping based on the directory names at each level stored in the `name` member of the `mmap_struct` in the `event` structure.

[0116] It's important to note that the `task_struct` structure is the Process Control Block (PCB) in Linux systems, containing all the information about a process. Each process has its own virtual memory space, represented by an `mm_struct` structure. `mm_struct` uses two pointers to represent a segment of virtual address space, which is then mapped to actual physical memory via page tables. The `mmap` member of the `mm_struct` structure is of type `vm_area_struct`, which describes a range within the virtual address space. Specifically, the `vm_start` member of the `vm_area_struct` structure represents the starting address of the virtual memory space, and the `vm_end` member represents the first byte after the ending address of the virtual memory space.

[0117] It should be noted that this embodiment mimics the implementation of procfs and the d_path() kernel function in Linux when collecting process-related dependency library information. However, other paths within the kernel can also be used to obtain process-related dependency library information, such as mimicking the ELF linking source code of the glibc library. This invention utilizes the eBPF kernel-mode component to obtain process crash information; other similar methods can also be used to obtain process crash information.

[0118] ② During component runtime, in user space, the eBPF user space output component receives and analyzes the crash process data extracted from the kernel by the eBPF kernel space component from the BPF ring buffer, and generates the corresponding process crash log in the specified path.

[0119] Understandably, by using the eBPF user-space output component to analyze and reassemble process crash data, corresponding process crash logs can be generated at a specified path.

[0120] Specifically, the analysis and reorganization process crash data using eBPF user-space output components includes:

[0121] The eBPF user-mode output component is used to convert each virtual address in the kernel-mode function call stack into a unique kernel function name plus a first offset.

[0122] Translate each virtual address in the user-mode function call stack into a unique user function name plus a second offset;

[0123] The absolute path of the dynamic library files that the process depends on is obtained from the virtual memory segment.

[0124] (2) eBPF kernel-mode components

[0125] The function of the eBPF kernel-mode component is to intercept all processes exiting in the Linux system, analyze their exit codes to determine whether they are exiting abnormally; if they are exiting abnormally, then, under the constraints of the eBPF mechanism, probe the memory space of the process to obtain the kernel-mode and user-mode data that we are interested in.

[0126] In this embodiment, the specific implementation scheme of the eBPF kernel-mode component is as follows:

[0127] The first step is to select the kernel-mode function to mount the eBPF kernel-mode component. This is a necessary operation for using the eBPF mechanism. Ideally, the eBPF kernel-mode component mount point should be triggered when all processes exit. In a Linux environment, system resources after all processes terminate—such as process-allocated memory and open files—are released in the `do_exit()` function. Information about the `do_exit()` function can be found in the path ` / sys / kernel / debug / tracing / available_filter_functions`. Therefore, the entry point of the `do_exit()` function can be the mount point of the eBPF kernel-mode component in this embodiment.

[0128] When an eBPF kernel-mode component is mounted to this mount point, all processes will trigger this mount point before their resources are reclaimed by the operating system, thereby intercepting all process exit information.

[0129] After intercepting all process exit operations, it is necessary to determine whether the process exited normally or abnormally. This embodiment uses the kprobe mechanism in eBPF. In an eBPF program, crucial information about process exit, namely the process exit code (exitcode), can be directly obtained, indicating the process's exit status. Based on the exit code and the Linux manual on process exit codes, this embodiment can determine whether the process terminated due to normal or abnormal exit.

[0130] For example, if the process exit code is 0, it indicates that the process terminated normally; if the process exit code is 130, it indicates that the process terminated due to Ctrl+C; and if the process exit code is 139, it indicates that the process terminated due to a segmentation fault. After determining that the process exited abnormally (terminated) based on the process exit code, the eBPF kernel-mode component can use the bpf_helper_function provided by the Linux system to obtain data related to the crashed process, and then send it back to the user-mode component in the form of a predefined event structure through the BPF circular buffer. The specific data acquisition process has been mentioned when explaining the key data and processing methods of the crashed process.

[0131] Please see Figure 5 In this embodiment, the effect of the entire system crash log generation component is as follows: during the component's operation, that is, after processing the log generation path specified by the command line parameter and loading the eBPF kernel-mode component, any abnormally terminated process in the system will generate a process crash log in the specified directory (each time a process exits abnormally, a log file named with the binary file name corresponding to the process + the process crash time will be generated separately).

[0132] Please see Figure 6 Using eBPF user-space components, the generated process crash logs can be output to the terminal, or dynamically output to stdout. This can be developed into a way to dynamically notify developers whether a crash has occurred via a remote terminal.

[0133] Please see Figure 7 and Figure 8 In this embodiment, when obtaining key information about the crashed process, the generation of the Core file is skipped, and the data of interest to the developers is directly obtained from the memory of the crashed process. Since the iostat and iotop tools provided by Linux cannot monitor the IO load caused by generating the Core file, the file size of the Core file and the process crash log file generated in this embodiment are directly compared to reflect the IO resource consumption and disk space consumption. It can be seen that for the single-threaded simple process in the test example, the generated Core file size is 372KB, and the Core file of the multi-threaded process even reaches 57MB. However, in this embodiment, when generating the crash log, a bloated Core file is not needed; the relevant data is directly obtained from memory, and the final log size is only 2.3KB. Therefore, enabling Core dump to generate the Core file has a large IO load and consumes a lot of disk resources. In contrast, this embodiment bypasses the generation of the Core file, still obtains the more critical information of the crashed process, and has a very small IO load and much less disk resource consumption. Figure 7 and Figure 8 The low-load characteristics of this invention can be verified.

[0134] This embodiment utilizes eBPF's kprobe technology. By attaching a custom program to the do_exit() kernel function, it can intercept all abnormally exiting processes and obtain key data about the process crash in the process's memory through related auxiliary functions to further locate the cause of the failure, while using relatively low system resources.

[0135] This embodiment utilizes eBPF's kprobe technology to mount eBPF kernel-mode components onto kernel-mode functions that are triggered when a process exits. This allows for the interception of all abnormally exiting processes, bypassing the generation and analysis of core files. It directly and automatically retrieves key data of abnormally exiting processes from the kernel and sends it back to the BPF circular buffer, consuming very few I / O and disk resources, thus significantly reducing system load in production environments. Furthermore, by using eBPF user-mode components to retrieve key data of crashed processes from the BPF circular buffer and generate corresponding process crash logs, it helps system administrators quickly analyze crash information and accurately locate the cause of the failure.

[0136] Please see Figure 9 The present invention also provides an embodiment of a process crash information collection device based on eBPF, comprising:

[0137] Component initialization module 11 is used to mount pre-compiled eBPF kernel-mode components onto kernel-mode functions triggered when a process exits in the Linux system using eBPF user-mode components;

[0138] The abnormal process exit judgment module 22 is used to trigger the eBPF kernel-mode component to intercept the process and determine whether the process exited abnormally when the process running in the Linux system exits.

[0139] The process crash data acquisition module 33 is used to acquire the process crash data of the process and send the process crash data into the BPF ring buffer if the process exits abnormally.

[0140] The process crash log generation module 44 is used to obtain the process crash data from the BPF ring buffer using the eBPF user-space component, and generate the corresponding process crash log based on the process crash data in order to accurately locate the cause of the failure.

[0141] Compared to the existing technology that uses the Core dump mechanism to locate abnormal process exits and analyze the causes, this invention uses eBPF technology to bypass the generation and analysis of Core files, directly obtaining the desired data from the kernel without having to extract information from the bloated Core file, thereby reducing system load.

[0142] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0143] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0146] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0147] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.< / pid> < / pid>

Claims

1. An eBPF-based process crash information collection method, characterized by, include: By using eBPF user-space components, pre-compiled eBPF kernel-space components are mounted onto kernel-space functions triggered when a process exits in the Linux system; When a process running in the Linux system exits, the eBPF kernel-mode component is triggered to intercept the process and determine whether the process exited abnormally. If the process exits abnormally, the process crash data of the process is obtained using the eBPF kernel-mode component and the process crash data is sent into the BPF circular buffer. The process crash data is obtained from the BPF circular buffer using the eBPF user-space component, and a corresponding process crash log is generated based on the process crash data to accurately locate the cause of the failure. The kernel-mode function is the do_exit() function; The process crash data includes at least the following: The process exit code, kernel-mode function call stack, user-mode function call stack, and virtual address space information of the dynamic library files that the process depends on, wherein the virtual address space information includes at least several contiguous virtual memory segments; The eBPF user-space component includes an eBPF user-space initialization component and an eBPF user-space output component. Generating a corresponding process crash log based on the process crash data using the eBPF user-space component includes: The process crash data is analyzed and reconstructed using the eBPF user-space output component, and a corresponding process crash log is generated in the specified path. Analyzing and reconstructing the process crash data using the eBPF user-space output component includes: The eBPF user-mode output component is used to convert each virtual address in the kernel-mode function call stack into a unique kernel function name plus a first offset. Each virtual address in the user-mode function call stack is converted into a unique user function name plus a second offset; The absolute path of the dynamic library file that the process depends on is obtained from the virtual memory segment.

2. The method for collecting process crash information based on eBPF according to claim 1, characterized in that, The eBPF user-mode component includes an eBPF user-mode initialization component and an eBPF user-mode output component. The use of the eBPF user-mode component to mount the pre-compiled eBPF kernel-mode component onto the kernel-mode function triggered when a process exits in the Linux system includes: By using eBPF user-mode initialization components and BPF system calls, pre-compiled eBPF kernel-mode components are mounted onto kernel-mode functions triggered when a process exits in the Linux system.

3. The method for collecting process crash information based on eBPF according to claim 1, characterized in that, Triggering the eBPF kernel-mode component to intercept the process and determine whether the process exited abnormally includes: The eBPF kernel-mode component is triggered to intercept the process and obtain the corresponding process exit code. Based on the process exit code, it is determined whether the process exited abnormally.

4. The method for collecting process crash information based on eBPF according to claim 1, characterized in that, Using the eBPF kernel-mode component to send the process crash data into the BPF circular buffer includes: The process crash data is sent to the BPF circular buffer in the form of an event structure using the eBPF kernel-mode component.

5. The method for collecting process crash information based on eBPF according to claim 1, characterized in that, After generating the corresponding process crash log based on the process crash data using the eBPF user-space component, the process crash log also includes: The process crash log is output to the terminal using the eBPF user-space component.

6. A process crash information collection device based on eBPF, characterized in that, The apparatus is used to implement the eBPF-based process crash information collection method as described in any one of claims 1-5, the apparatus comprising: The component initialization module is used to mount pre-compiled eBPF kernel-mode components onto kernel-mode functions triggered when a process exits in the Linux system using eBPF user-mode components. The abnormal process exit judgment module is used to trigger the eBPF kernel-mode component to intercept the process and determine whether the process exited abnormally when a process running in the Linux system exits. The process crash data acquisition module is used to acquire the process crash data of the process and send the process crash data into the BPF circular buffer if the process exits abnormally. The process crash log generation module is used to obtain the process crash data from the BPF circular buffer using the eBPF user-space component, and generate the corresponding process crash log based on the process crash data in order to accurately locate the cause of the failure.

Images