Data processing method, electronic device, storage medium, and computer program product

Abstract

This application discloses a data processing method, electronic device, storage medium, and computer program product. The method includes: upon receiving a first read request from a first user, locking a first buffer using a first lock, wherein the first read request requests to read first information, the first information including information related to each type of kernel event in one or more types of kernel events, and the first information is stored in the first buffer; copying the first information to temporary memory to generate a data snapshot of the first information; copying the data snapshot to a buffer corresponding to the first user; and after the data snapshot is successfully copied to the buffer corresponding to the first user, releasing the first buffer and releasing the first lock. This application improves the security of the process of reading information related to kernel events.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, electronic device, storage medium, and computer program product. Background Technology

[0002] During the Linux kernel's operation, numerous kernel events reflecting the system's state are generated. These events can include memory management events, process scheduling events, and input / output (I / O) operations. Kernel events are crucial for understanding the system's operational status, troubleshooting performance bottlenecks, and locating faults. Therefore, it's necessary to extract relevant information from these kernel events so that it can be read when needed. However, this process can be compromised by issues such as duplicate readings of kernel event information and residual information, compromising the security of the kernel event information retrieval process. Summary of the Invention

[0003] To address the related technical problems, embodiments of this application provide a data processing method, an electronic device, a storage medium, and a computer program product.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a data processing method applied to a first node. The method includes: Upon receiving a first read request from a first user, the first buffer is locked using a first lock. The first read request is used to request the reading of first information, which includes information related to each type of kernel event in one or more types of kernel events. The first information is stored in the first buffer. The first information is copied into temporary memory to generate a data snapshot of the first information; Copy the data snapshot to the buffer corresponding to the first user; After the data snapshot is successfully copied to the buffer corresponding to the first user, the first buffer is released and the first lock is released.

[0005] In the above scheme, before locking the first buffer with the first lock upon receiving the first read request from the first user, the method further includes: The first information is written into the first buffer, the storage capacity of the first buffer is a fixed capacity, wherein when the space required by the first information is greater than the remaining space of the first buffer, the first information overwrites the historical information in the first buffer when the first information is written into the first buffer.

[0006] In the above scheme, before writing the first information into the first buffer, the method further includes: Acquire a second lock, which is used to lock the first buffer; After writing the first information into the first buffer, the method further includes: Release the second lock.

[0007] In the above scheme, before locking the first buffer with the first lock upon receiving the first read request from the first user, the method further includes: Register a corresponding processing function for each type of kernel event. Different types of kernel events in the one or more types of kernel events correspond to different processing functions, and different processing functions are bound to different trace points in the kernel. When a kernel event of any type among the one or more types of kernel events occurs, the corresponding trace point is triggered, and the corresponding first information is extracted through the target processing function, wherein the target processing function is the processing function bound to the corresponding trace point.

[0008] The method in the above scheme further includes: Receive the first instruction; Determine the status control information according to the first instruction; The step of extracting the corresponding first information through the target processing function includes: When the state control information takes the first value, the corresponding first information is extracted through the target processing function; Correspondingly, the method further includes: If the state control information takes the second value, the corresponding first information is not extracted through the target processing function.

[0009] In the above scheme, before locking the first buffer with the first lock upon receiving the first read request from the first user, the method further includes: Create a first directory, whose contents include one or more of the following: The first file includes state control information, and the processing of relevant information for each type of kernel event is controlled by the state control information. The second file; the second file includes the first information for each type of kernel event.

[0010] In the above scheme, the method is applied to a first device, the first device including a SysFS interface, the SysFS interface being used to control the processing of relevant information of each type of kernel event by reading and writing the first file; and / or, the SysFS interface being used to obtain the first information of each type of kernel event by reading the second file.

[0011] This application provides a data processing apparatus, including: The first locking unit is used to lock the first buffer with a first lock when a first read request is received from a first user, wherein the first read request is used to request to read first information, the first information including relevant information of each type of kernel event in one or more types of kernel events, and the first information is stored in the first buffer; The first copying unit is used to copy the first information into temporary memory to generate a data snapshot of the first information; The second copying unit is used to copy the data snapshot to the buffer corresponding to the first user; The first processing unit is configured to release the first buffer and unlock the first lock after the data snapshot is successfully copied to the buffer corresponding to the first user.

[0012] This application also provides an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor. The processor is used to execute the steps of any of the above-mentioned technical solutions when running the computer program.

[0013] This application also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0014] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0015] As can be seen from the above, in this embodiment, upon receiving a first read request from a first user, the first buffer is locked using a first lock. The first read request requests to read first information, which includes information related to each type of kernel event in one or more types of kernel events. This first information is stored in the first buffer. The first information is copied to temporary memory to generate a data snapshot of the first information. The data snapshot is copied to the buffer corresponding to the first user. After the data snapshot is successfully copied to the buffer corresponding to the first user, the first buffer is released, and the first lock is released. Therefore, by locking the first buffer with a first lock, this embodiment ensures that the reading of kernel event-related information and the release of the buffer are both completed under the same lock protection, guaranteeing the atomicity of reading and releasing kernel event-related information, reducing residual and repeated readings of kernel event-related information, and improving the security of reading kernel event-related information. Attached Figure Description

[0016] Figure 1 A schematic diagram of the overall architecture provided for embodiments of this application; Figure 2 A flowchart illustrating a data processing method provided in an embodiment of this application; Figure 3 A flowchart illustrating Embodiment 1 provided for the purposes of this application; Figure 4 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0018] It should be understood that the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, the term "one or more" in this document is an exemplary expression and can be replaced with any possible expression, such as one or more, at least one, or at least one item, etc.

[0019] It should also be understood that the term "instruction" mentioned in the embodiments of this application can be a direct instruction, an indirect instruction, or an indication of a relationship. For example, A instructing B can mean that A directly instructs B, such as B being able to obtain information through A; it can also mean that A indirectly instructs B, such as A instructing C, so B can obtain information through C; or it can mean that there is a relationship between A and B.

[0020] It should also be understood that the term "correspondence" mentioned in the embodiments of this application may indicate a direct or indirect correspondence between the two, or an association between the two, or a relationship of instruction and being instructed, configuration and being configured, etc.

[0021] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0022] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0023] During the Linux kernel's operation, numerous events reflecting the system's state are generated, such as memory management (e.g., out-of-memory (OOM) killing processes), process scheduling (e.g., process switching), I / O operations (e.g., disk read / write), network events (e.g., packet sending / receiving), power management (e.g., sleep / wake-up), and file system events (e.g., file creation / deletion). These kernel events are crucial for understanding the system's operational status, troubleshooting performance bottlenecks, and locating faults.

[0024] In related technologies, monitoring Linux kernel events mainly relies on the following technical solutions: The first approach involves using the ftrace tool. ftrace establishes a built-in tracing framework within the Linux kernel, supporting event monitoring through static tracepoints and dynamic probes (such as kprobe). However, ftrace events are scattered across tracepoints in different subsystems, requiring users to manually specify specific event types. Furthermore, the output data formats are inconsistent, necessitating secondary processing for aggregation and analysis.

[0025] Secondly, the perf tool can be used. Based on performance counters and kernel traces, perf can collect events and generate statistical data. However, perf focuses more on the quantitative analysis of performance indicators, and does not record the detailed context of events (such as process identifier (ID) and operation status) completely, and lacks a unified log management mechanism.

[0026] Thirdly, some kernel subsystems provide dedicated monitoring interfaces by using independent monitoring tools. However, these monitoring interfaces are scattered across different system paths, requiring users to obtain data across multiple interfaces, which is complex and makes it difficult to achieve multi-event linkage analysis.

[0027] Fourth, third-party kernel monitoring modules can be used: Some enterprise-developed kernel modules can monitor specific events, but they are mostly for single types of events (such as monitoring only process scheduling) and lack standardized user interaction interfaces, resulting in poor compatibility.

[0028] The main drawbacks of the related technologies can be summarized as follows: First, the tracking points of kernel events are scattered and difficult to aggregate. In related technologies, the tracking points of kernel events are distributed in different subsystems and lack a unified aggregation mechanism. Users need to manually integrate multi-source data, which is inefficient.

[0029] Second, the log management mechanism is imperfect: the log storage of most tools consumes too much memory. For example, the log storage may consume too much memory because the size of the log cache area is not limited, and there is a lack of concurrency safety protection. When multiple threads write, the log may be corrupted.

[0030] Third, the user interaction interface is not uniform. The monitoring status control (divided into enabling or disabling) and log reading of the log interaction interface need to be implemented through different commands or paths, which makes the operation complex.

[0031] The technical problems that need to be solved by the embodiments of this application include: First, how to achieve unified aggregation and monitoring of multiple types of kernel events to solve the problems of scattered events and difficulty in integration in existing technologies.

[0032] Second, how to design an efficient log management mechanism to achieve circular storage of logs with a fixed memory footprint and ensure concurrency safety in a multi-threaded environment.

[0033] Third, how to provide standardized user interaction interfaces to simplify monitoring status control and log reading operations and improve ease of use.

[0034] To address at least one of the aforementioned technical problems, this application proposes a data processing method. In this method, upon receiving a first read request from a first user, a first buffer is locked using a first lock. The first read request requests the reading of first information, which includes information related to each type of kernel event in one or more types of kernel events. The first information is stored in the first buffer. The first information is copied to temporary memory to generate a data snapshot of the first information. The data snapshot is copied to the buffer corresponding to the first user. After the data snapshot is successfully copied to the buffer corresponding to the first user, the first buffer is released, and the first lock is released. Therefore, by locking the first buffer with a first lock, this application ensures that the reading of kernel event-related information and the release of the buffer are both completed under the same lock protection, guaranteeing the atomicity of reading and releasing kernel event-related information, reducing residual and repeated readings of kernel event-related information, and improving the security of reading kernel event-related information.

[0035] First, an exemplary architecture of an embodiment of this application is introduced: Please see Figure 1 The embodiments of this application can be applied to kernel modules, the overall architecture of which includes one or more of the following components: an event processing unit; a log management unit; and a SysFS interaction unit. These three units can work together to realize the entire process of "monitoring-storage-interaction" of kernel events.

[0036] The architecture of this application embodiment is based on the core principles of "modular decoupling and low operational intrusion"; Modular decoupling refers to splitting "event capture", "log management" and "user interaction" into three independent units: the event processing unit, the log management unit and the SysFS interaction unit. These three units communicate with each other through explicit function interfaces, which facilitates subsequent function expansion (such as adding a new network subsystem event type only requires modifying the event processing unit).

[0037] The low operational intrusion is reflected in the fact that it can rely solely on the kernel's native tracepoint mechanism (without modifying the kernel source code) and the SysFS file system (without needing to add new mount points). In addition, after the kernel module is loaded, it does not affect the original kernel subsystem logic, such as process scheduling and I / O processes.

[0038] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0039] Please see Figure 2 The data processing method provided in this application includes: Step 201: Upon receiving a first read request from a first user, lock the first buffer using a first lock, wherein the first read request is used to request the reading of first information, the first information including relevant information of each type of kernel event in one or more types of kernel events, and the first information is stored in the first buffer; For example, the statement that the first user's first read request is received in this application embodiment is an exemplary statement and can be replaced with any possible statement, such as triggering the first read request, etc.

[0040] For example, in this embodiment of the application, the first information is stored in the first file (also known as the first folder) under the first directory. For example, the first directory can be sys / kernel / event_monitor, the name of the first file can be log, and the storage address of the first file can be sys / kernel / event_monitor / log.

[0041] For example, when the first user executes `cat / sys / kernel / event_monitor / log`, it can be considered as receiving the first read request from the first user.

[0042] For example, kernel events include one or more of the following: OOM process killing event; page allocation failure event; process creation event; process destruction event; process state switching event (e.g., switching to ready, running, or blocked); disk read / write completion event; I / O error event; packet sending event; packet receiving event; connection establishment event; connection termination event; system hibernation event; system wake-up event; central processing unit (CPU) frequency adjustment event; file creation event; file deletion event; file modification event; directory permission change event.

[0043] For example, kernel events can be categorized into one or more of the following major categories: memory management; process scheduling; I / O operations; network events; power management; and file system events. Memory management includes one or more of the following: OOM (Out of Memory) events that kill processes; page allocation failure events. Process scheduling includes one or more of the following: process destruction events; process state transition events. I / O operations include one or more of the following: disk read / write completion events; I / O error events. Network events include one or more of the following: packet sending events; packet receiving events; connection establishment events; connection termination events. Power management events include one or more of the following: system hibernation events; system wake-up events; CPU frequency adjustment events. File system events include one or more of the following: file creation events; file deletion events; file modification events; and directory permission change events.

[0044] For example, the kernel event is a kernel event of the first operating system, such as the Linux operating system.

[0045] For example, in the embodiments of this application, one or more types of kernel events define corresponding first information. Taking an OOM kill event as an example, the first information corresponding to an OOM kill event includes one or more of the following: process ID, process name; taking a process state switching event as an example, the first information corresponding to a process state switching event includes one or more of the following: prev_pid (previous process ID); prev_comm (previous process name); next_pid (next process ID); next_comm (next process name).

[0046] For example, the first information may also be referred to as the extracted fields of the log or kernel event, that is, the fields that need to be extracted from the kernel event, which are used to assist in the analysis process of the kernel based on the kernel event.

[0047] For example, the first lock includes one or more of the following: a mutex lock; a standard mutex lock; a recursive mutex lock; and a read-write lock.

[0048] For example, taking the first lock as a mutex lock, after the first read request is triggered, that is, after receiving the first read request from the first user, mutex_lock(&ctx.lock) is called to acquire the lock, that is, the first lock is acquired through mutex_lock(&ctx.lock), and then the first buffer is locked through the first lock.

[0049] For example, the first buffer is a fixed-size buffer. Fixed size means that the size of the cache space of the first buffer is fixed. When the storage space of the first buffer is insufficient, the oldest log will be automatically overwritten. The oldest log is the earliest data saved in the first buffer.

[0050] For example, the first buffer is a circular buffer.

[0051] For example, the first buffer structure includes one or more of the following: a log data area for extracting information related to kernel events; a head pointer for pointing to the next write position; and a mutex lock for protecting concurrent access.

[0052] It is understandable that users need to avoid "data being overwritten during the reading process" when reading logs, and at the same time ensure that the buffer can be safely cleared after reading. To this end, this application embodiment designs an atomic reading process, which includes generating a data snapshot, copying the data snapshot to the user space (i.e., the user's corresponding buffer), and clearing the first buffer, etc.

[0053] For example, when the first user executes `cat / sys / kernel / event_monitor / log`, that is, when the first user's first read request is received, the `log_show()` function of the log management unit is triggered through the SysFS interaction unit to add a mutex lock to the first buffer.

[0054] Step 202: Copy the first information to temporary memory to generate a data snapshot of the first information.

[0055] For example, the size of the temporary memory is equal to the size of the first buffer.

[0056] For example, after locking the first buffer with the first lock (e.g., after locking by calling mutex_lock(&ctx.lock)), a temporary memory block (temp_buf) of the same size as the first buffer can be allocated, and the current log data can be copied to the temporary memory by memcpy(temp_buf, ctx.log_buffer, len) to generate a data snapshot.

[0057] Step 203: Copy the data snapshot to the buffer corresponding to the first user; It is understood that copying the data snapshot to the buffer corresponding to the first user means copying the data content in the data snapshot (i.e., snapshot data) to the buffer corresponding to the first user.

[0058] For example, the buffer corresponding to the first user is the buffer provided by the first user. For instance, the buffer corresponding to the first user can be the buffer of the user device used by the first user.

[0059] For example, copying the data snapshot to the buffer corresponding to the first user includes: copying the snapshot data to user space using copy_to_user(). For instance, taking the buffer corresponding to the first user as buf and the first temporary memory as temp_buf, copy_to_user(buf, temp_buf, len) can be called to copy the snapshot data to the user-provided buffer and return the actual number of copies.

[0060] Step 204: After the data snapshot is successfully copied to the buffer corresponding to the first user, release the first buffer and unlock the first lock.

[0061] It is understood that in this application embodiment, the release operation and the read operation of the first buffer are completed under the same lock protection to ensure the atomicity of "read-release" and avoid data residue or repeated reading.

[0062] For example, after the data snapshot is successfully copied to the buffer corresponding to the first user, ctx.log_head = 0 is reset, and then the temporary memory is automatically released and the first lock is released.

[0063] The above describes how to handle successful copying. For handling copying failures, please refer to the following example: For example, if the data snapshot is not successfully copied to the buffer corresponding to the first user (e.g., a return value > 0, indicating that the copy was partially successful), the -EFAULT error code is returned, and the number of bytes of data that failed to be copied in the data snapshot is recorded.

[0064] For example, the complete interaction process of this application embodiment follows the linear logic of "event triggering - data flow - user interaction". The reading process of the first information in this application embodiment (also known as the log viewing stage) has been introduced above. The initialization stage, event monitoring stage (i.e. the first information collection stage) and destruction stage in this application embodiment are introduced below.

[0065] First, let's introduce the initialization phase: Taking the execution of this application's embodiment by a kernel module in an electronic device as an example, when the kernel module is loaded, the log management unit first pre-allocates a first buffer.

[0066] In addition to pre-allocating the first buffer, the initialization phase also requires creating the first directory, as detailed in the following practical application: In practical applications, before locking the first buffer with the first lock upon receiving the first read request from the first user, the method further includes: Create a first directory, whose contents include one or more of the following: The first file includes state control information, and the processing of relevant information for each type of kernel event is controlled by the state control information. The second file; the second file includes the first information for each type of kernel event.

[0067] Understandably, to address the issue of "dispersed operations and high complexity" in related technologies—namely, the need for monitoring status control (e.g., enabling or disabling monitoring) and log reading through different commands or paths, resulting in high operational complexity—this application's embodiment addresses this problem. The monitoring status of each type of kernel event is controlled through status control information in a first file, while the relevant information for each type of kernel event is located in a second file. Both the first and second files are located in a first directory. Therefore, this application's embodiment, through a "single directory, dual files" design, aggregates status control and log reading into the same directory, enabling the reading of status control and kernel event-related information through a single path, thereby reducing operational dispersion and complexity.

[0068] It should be noted that the statement "the processing of relevant information for each type of kernel event is controlled by the state control information" is an exemplary statement and can be replaced by one or more of the following statements: the relevant information for each type of kernel event is controlled by the state control information; the extraction of relevant information for each type of kernel event is controlled by the state control information; the monitoring status of relevant information for each type of kernel event is controlled by the state control information, etc.

[0069] For example, the processing of information related to kernel events includes one or more of the following: extracting information related to kernel events; saving information related to kernel events to a first buffer, etc.

[0070] Understandably, when one or more types of kernel events occur, the kernel module will detect the kernel event, check the status control information, and determine whether to process the kernel event accordingly based on the status control information, such as whether to extract relevant information about the kernel event.

[0071] For example, creating the first directory includes: creating the first directory through the SysFS interaction unit. For example, the first directory can be / sys / kernel / event_monitor / , which contains two files, namely a first file and a second file. The first file is named enabled, and the second file is named log. Correspondingly, the storage address of the status control information is / sys / kernel / event_monitor / enabled, and the storage address of the first information is / sys / kernel / event_monitor / log.

[0072] For example, when the monitoring status is enabled (e.g., when enabled=1), the extraction and writing of the first information (i.e., the relevant information of the kernel event) will be triggered when any one of the one or more kernel events occurs; when the monitoring status is disabled (e.g., when enabled=0), the extraction and writing of the first information will not be triggered when any one of the one or more kernel events occurs.

[0073] For example, the SysFS interaction unit is the "user entry point" of the solution, that is, users can interact with the SysFS interaction unit, such as instructing the user to enable monitoring status through the SysFS interaction unit.

[0074] In practical applications, the method is applied to a first device, which includes a SysFS interface. The SysFS interface is used to control the processing of information related to each type of kernel event by reading and writing the first file; and / or, the SysFS interface is used to obtain the first information of each type of kernel event by reading the second file.

[0075] In this embodiment of the application, a standardized interactive interface scheme based on SysFS is provided. The SysFS interactive unit solves the problems of "dispersed operations and high complexity" in related technologies. Through the "single directory and dual file" design, the standardized operation of reading information related to monitoring status control and kernel events is realized. The reading of information related to status control and kernel events is realized through a single path, simplifying user operations.

[0076] For example, the first file is the sole control entry point for the monitoring status. The monitoring status supports two states: enabled and disabled. For instance, when the status control information is 1, the monitoring status is enabled; when the status control information is 1, the monitoring status is disabled. The design of the first file balances security and ease of use, and the specific implementation is as follows: For example, during module initialization, the `event_monitor` directory is created under ` / sys / kernel / ` using `sysfs_create_dir()`, and then the `enabled` file, i.e., the first file, is created in that directory using `sysfs_create_file()`. The permissions of the first file are set to 0644, meaning that users can write, while groups and others are read-only, ensuring that ordinary users cannot modify it and that only root privileges are required to modify it, thus avoiding accidental operations.

[0077] For information regarding the source and function of the status control information in the first file, please refer to the following practical applications: In practical applications, the method further includes: Receive the first instruction; Determine the status control information according to the first instruction; The step of extracting the corresponding first information through the target processing function includes: When the state control information takes the first value, the corresponding first information is extracted through the target processing function; Correspondingly, the method further includes: If the state control information takes the second value, the corresponding first information is not extracted through the target processing function.

[0078] In this embodiment of the application, the status of all tracking points can be uniformly controlled through centralized status control information, thereby reducing the dispersion and complexity of operations.

[0079] For example, the first instruction can be data written by the user, such as data written by the user to a first file.

[0080] For example, determining the state control information according to the first instruction includes: reading the first instruction, converting the first instruction into state control information using kstrtoint(), where the state control information can be an integer val; if val is neither 0 nor 1, returning the -EINVAL error code (illegal input); if val=1, enabling monitoring and processing the monitored kernel events through the event handling unit; if val=0, disabling monitoring, not processing the monitored kernel events, and stopping the writing of logs (i.e., information related to each type of kernel event). For example, to ensure state consistency, the atomic_t type is used to store the monitoring state to avoid state competition under multi-threading; for example, the processing function of the event handling unit will first check atomic_read(&ctx.enabled), and only execute the first information writing when the state is 1, ensuring no redundant operations after disabling.

[0081] For example, the second file is the sole entry point for reading the first information, supporting on-demand reading and automatic clearing.

[0082] For example, when creating the second file, the second file is created in the same directory as the first file, that is, both are created in the first directory with permissions set to 0444 (i.e., read-only for all users) to prevent accidental writing and corruption of log data.

[0083] For example, the initialization phase also includes registering processing functions, as detailed in the following practical application: In practical applications, before locking the first buffer with the first lock upon receiving the first read request from the first user, the method further includes: Register a corresponding processing function for each type of kernel event. Different types of kernel events in the one or more types of kernel events correspond to different processing functions, and different processing functions are bound to different trace points in the kernel. When a kernel event of any type among the one or more types of kernel events occurs, the corresponding trace point is triggered, and the corresponding first information is extracted through the target processing function, wherein the target processing function is the processing function bound to the corresponding trace point.

[0084] In this embodiment, standardized information extraction can be achieved through the traceevent mechanism, meaning that relevant information about kernel events is extracted according to certain standards. Unified aggregation of events from multiple subsystems is achieved through "precise binding of trace points + standardized information extraction," where standardized information extraction refers to extracting relevant information about kernel events according to certain standards.

[0085] It is understood that the embodiments of this application are based on the traceevent mechanism. Registering a processing function means registering the processing function with the corresponding kernel event in traceevent, so that the processing function can report the relevant information of the corresponding kernel event when it runs.

[0086] For example, the above trace points are kernel trace points.

[0087] For example, the processing function for all tracking points is registered through the event processing unit, that is, the corresponding processing function is registered for each type of kernel event.

[0088] For example, the event processing unit is the "data entry point" in the embodiments of this application. The core objective of the event processing unit is to solve the problem of "dispersed events and incomplete capture" in related technologies. It achieves unified aggregation of events from multiple subsystems through "precise binding of tracking points + standardized information extraction". Standardized information extraction refers to extracting relevant information of kernel events according to certain standards.

[0089] For example, registering a corresponding processing function for each type of kernel event includes: registering a dedicated processing function for each type of kernel event among one or more kernel events through a kernel-predefined tracepoint mechanism.

[0090] For example, the registration function is called to bind a custom handler function to the corresponding trace point. For instance, the handler function bc_oom_kill_handler corresponds to the OOM event, and the handler function bc_oom_kill_handler is bound to trace point 1, which is the trace point corresponding to the OOM event.

[0091] For example, this application embodiment also provides a registration failure rollback mechanism. Specifically, if a certain event registration fails during kernel module initialization (such as the kernel not supporting the trace point), all registered trace points are immediately deregistered and -ENODEV is returned, ensuring that the module is in a "fully available" or "fully unavailable" state without the risk of partial initialization.

[0092] It is understood that the above refers to the corresponding processing function for each type of kernel event registration, and when any type of kernel event occurs, triggering the corresponding trace point belongs to the event monitoring stage.

[0093] The event monitoring phase in the embodiments of this application is described in detail below.

[0094] For example, when any type of kernel event occurs among the one or more types of kernel events, the kernel automatically triggers the corresponding trace point, calls the bound processing function, and realizes real-time capture of the event.

[0095] For example, after the processing function is triggered by the tracepoint, it needs to extract the corresponding first information from the context of the corresponding kernel event: For example, as mentioned above, one or more types of kernel events define corresponding first information. Therefore, it is necessary to define exclusive extraction fields for different event types. For example, for OOM events, pid (process ID) and comm (process name) need to be extracted; for process switching events, prev_pid (previous process ID), prev_comm (previous process name), next_pid (next process ID), and next_comm (next process name) need to be extracted.

[0096] For example, all fields are read directly from the kernel structure to ensure data authenticity.

[0097] For example, the kernel structure includes one or more of the following: struct task_struct; struct oom_control.

[0098] For example, after a user writes `echo 1> / sys / kernel / event_monitor / enabled` to enable monitoring, when the kernel subsystem triggers a target event, that is, when any of the kernel events of the one or more types mentioned above occur, the tracepoint automatically calls the bound processing function to extract the event information and write it to the first buffer.

[0099] The following describes the writing of the first information in an embodiment of this application: For example, the storage and management of the first information can be performed by the log management unit, which is the "data hub" of the solution. The core of the log management unit is to solve the problems of "log memory overflow and concurrent write corruption" that may occur in related technologies. In this application embodiment, a "ring buffer + concurrency control" design can be used to achieve secure storage with fixed memory usage. In practical applications, before writing the first information into the first buffer, the method further includes: Acquire a second lock, which is used to lock the first buffer; After writing the first information into the first buffer, the method further includes: Release the second lock.

[0100] For example, the second lock includes one or more of the following: a mutex lock; a standard mutex lock; a recursive mutex lock; and a read-write lock.

[0101] For example, taking the second lock as a mutex lock, the embodiments of this application can protect the write operation by calling mutex lock. Before writing, mutex_lock(&ctx.lock) is called to acquire the mutex lock, and after writing, mutex_unlock(&ctx.lock) is called to release the mutex lock. For example, when the second lock is a mutex, the second lock can be of type struct mutex.

[0102] Understandably, compared to spinlocks, struct mutexes offer better performance when held for longer periods, making them suitable for non-atomic operations such as log writing.

[0103] In practical applications, before locking the first buffer with the first lock upon receiving the first read request from the first user, the method further includes: The first information is written into the first buffer, the storage capacity of the first buffer is a fixed capacity, wherein when the space required by the first information is greater than the remaining space of the first buffer, the first information overwrites the historical information in the first buffer when the first information is written into the first buffer.

[0104] For example, historical information refers to data written to the first buffer before the first information. For instance, historical information can be the oldest log, which is the earliest data saved in the first buffer.

[0105] For example, this application embodiment supports write length checking. Specifically: after writing the first information to the first buffer using the snprintf() function, the length of the first information is checked first; ctx.log_head indicates the position where the next data will be inserted, and LOG_BUF_SIZE indicates the size of the first buffer. If ctx.log_head > LOG_BUF_SIZE, it means that the remaining space in the first buffer is insufficient, so the overwrite logic is triggered.

[0106] The destruction stage in the embodiments of this application is described below: For example, when a kernel module is unloaded, monitoring is first disabled, for example by forcibly setting enabled=0, then all trace point handlers are unregistered, files are deleted (e.g., the first and second files), and finally memory is released and sysfs nodes are removed to ensure no resource leaks.

[0107] The present application will be further described in detail below with reference to application examples.

[0108] Example 1 Example 1 uses a circular buffer as the first buffer and a mutex lock as the first lock. Please refer to [link to example]. Figure 3 , Figure 3 Here is an exemplary flowchart of Example 1. Figure 3 Includes the following steps: S301: When a kernel module is loaded, the log management unit pre-allocates a circular buffer; S302: The SysFS interactive unit creates a first directory and its first and second files.

[0109] For example, the first directory could be / sys / kernel / event_monitor / , the first file could be the enabled file, and the second file could be the log file.

[0110] S303: Call the processing functions bound to all preset tracking points; For example, the event handling unit is an implementation of the processing function of the tracking point. For instance, the event handling unit can be a software unit corresponding to the processing function of the tracking point.

[0111] For example, read the data written by the first user. The data written by the first user can be the first instruction received from the first user. Convert the data written by the first user to an integer val using kstrtoint(). The integer val is an example of status control information. If val is not 0 and not 1, return the -EINVAL error code (illegal input). If val=1, it means that the monitoring status is enabled. Call all trace points and extract relevant information of kernel events through the handling function. If val=0, it means that the monitoring status needs to be disabled. Then stop writing relevant information of kernel events without extracting relevant information of kernel events through the handling function.

[0112] S304: After the first user writes echo 1> / sys / kernel / event_monitor / enabled to enable monitoring, when the kernel subsystem triggers a target event, it triggers the corresponding tracepoint and calls the processing function bound to the tracepoint through the registered function to extract the corresponding first information and write it to the circular buffer.

[0113] For example, the target event can be any type of one or more kernel events, and the corresponding processing function for the one or more types of kernel events has been registered. Accordingly, the kernel subsystem triggering the target event means that any type of one or more kernel events occurs.

[0114] For example, before writing the first information to the circular buffer, you can call mutex_lock(&ctx.lock) to acquire the mutex lock, and call mutex_unlock(&ctx.lock) to release the mutex lock after writing the first information to the circular buffer.

[0115] S305: When the first user executes cat / sys / kernel / event_monitor / log, the SysFS interaction unit triggers the log_show() function of the log management unit, first adding a mutex lock to the circular buffer, and then copying the first information to user space through copy_to_user().

[0116] For example, before copying the first information to user space using copy_to_user(), a temporary memory block of the same size as the circular buffer can be allocated first. The current log data can be copied to the temporary memory using memcpy(temp_buf, ctx.log_buffer, len) to generate a data snapshot. "Copying the first information to user space using copy_to_user()" means copying the data snapshot of the first information to the buffer corresponding to the first user. The buffer corresponding to the first user can be the buffer (buf) provided by the first user.

[0117] S306: Clear the circular buffer and release the mutex lock.

[0118] S307: When a kernel module is unloaded, monitoring is first disabled, then all tracepoint-bound handlers are unregistered, files are deleted, and finally memory is released and the sysfs node is removed.

[0119] For example, disabling monitoring means not processing information related to kernel events, that is, suspending the extraction of information related to kernel events.

[0120] Understandably, the purpose of the above steps is to ensure that no resources are leaked.

[0121] This application proposes a kernel event aggregation and monitoring method and system based on traceevent. By designing relevant kernel modules, it achieves unified aggregation, secure storage and convenient interaction of multiple types of kernel events, thus overcoming the shortcomings of related technologies.

[0122] First, this application proposes a unified aggregation monitoring mechanism for multiple types of kernel events. By registering multiple subsystem trace points and binding dedicated processing functions, it achieves unified collection and formatting of events such as memory, processes, and I / O, solving the problem of scattered events. Second, this application proposes a secure and efficient log management mechanism. It uses a fixed-size first buffer and combines it with specific concurrent safe write and atomic read clearing logic to ensure the integrity and persistence of kernel event-related information under limited resources, and to ensure that kernel event-related data is not duplicated or left behind. Third, this application proposes a standardized interactive interface scheme based on SysFS: monitoring status is controlled through the enabled file, and kernel event-related information is read from the log file, simplifying user operations.

[0123] The following is a comparison between related technologies (such as ftrace, perf) and the embodiments of this application: First, the two differ in their event aggregation capabilities. Related technical events are scattered across multiple tracking points, requiring manual integration and unified registration of multiple subsystem tracking points. In contrast, the embodiments of this application automatically aggregate multiple types of events, such as memory and processes.

[0124] Second, there is a difference in log management capabilities between the two. The relevant information of kernel events in related technologies is relatively scattered and may grow without limit, while the relevant information of kernel events in the embodiments of this application is aggregated and the size of the first buffer is fixed. When the memory of the first buffer is insufficient, the old logs will be automatically overwritten.

[0125] Third, the two differ in their user interaction methods. The related technologies require multiple command / path operations, which are highly complex. In contrast, the embodiments of this application standardize the SysFS interface and implement state control and log reading through a single path.

[0126] In summary, the embodiments of this application, through unified aggregation, efficient storage, and convenient interaction, are significantly superior to related technologies and are more suitable for kernel monitoring scenarios of enterprise-level servers and cloud computing platforms.

[0127] In addition, firstly, the embodiments of this application can improve system operation and maintenance efficiency, helping enterprise administrators quickly locate kernel faults (such as process crashes caused by OOM, and performance degradation caused by I / O blocking), shortening troubleshooting time. Secondly, the embodiments of this application can optimize system performance. By aggregating and analyzing kernel events, performance bottlenecks (such as frequent process switching, unreasonable memory allocation) can be identified, providing data support for system optimization and improving the resource utilization of servers / cloud computing platforms. Thirdly, the embodiments of this application can enhance system reliability, monitor kernel abnormal events in real time (such as network connection failures, file system errors), support early warning, reduce the risk of system downtime, and are suitable for scenarios with high stability requirements such as finance and e-commerce. Fourthly, the embodiments of this application can adapt to the trend of automated operation and maintenance. The standardized SysFS interface can be seamlessly integrated into the enterprise's automated operation and maintenance platform, supporting batch server monitoring configuration and log analysis, and is suitable for large-scale deployment.

[0128] Based on the embodiments described above, this application also provides a data processing apparatus, see [link to previous document]. Figure 4 The data processing device includes: The first locking unit 401 is used to lock the first buffer with a first lock when a first read request is received from a first user. The first read request is used to request the reading of first information, which includes information related to each type of kernel event in one or more types of kernel events. The first information is stored in the first buffer. The first copying unit 402 is used to copy the first information into a temporary memory to generate a data snapshot of the first information; The second copying unit 403 is used to copy the data snapshot to the buffer corresponding to the first user; The first processing unit 404 is configured to release the first buffer and unlock the first lock after the data snapshot is successfully copied to the buffer corresponding to the first user.

[0129] In one embodiment, the apparatus further includes a first writing unit, which, before locking the first buffer with a first lock upon receiving a first read request from a first user, is configured to: The first information is written into the first buffer, the storage capacity of the first buffer is a fixed capacity, wherein when the space required by the first information is greater than the remaining space of the first buffer, the first information overwrites the historical information in the first buffer when the first information is written into the first buffer.

[0130] In one embodiment, the device further includes a first acquisition unit and a first release unit. Before writing the first information into the first buffer, the first acquisition unit is used to acquire a second lock, and the second lock is used to lock the first buffer. After the first information is written into the first buffer, the first release unit is used to release the second lock.

[0131] In one embodiment, the device further includes a first registration unit and a first triggering unit. Before locking the first buffer with a first lock upon receiving a first read request from a first user, the first registration unit is used to register a corresponding processing function for each type of kernel event. Different types of kernel events in the one or more types of kernel events correspond to different processing functions, and the different processing functions are respectively bound to different trace points in the kernel. The first triggering unit is used to trigger the corresponding trace point when any type of kernel event occurs in the one or more types of kernel events, and to extract the corresponding first information through the target processing function, wherein the target processing function is the processing function bound to the corresponding trace point.

[0132] In one embodiment, the device further includes a first receiving unit and a first determining unit, wherein the first receiving unit is configured to receive a first instruction; and the first determining unit is configured to determine state control information based on the first instruction. The first registration unit registers a corresponding processing function for each type of kernel event among one or more types of kernel events, including: When the state control information takes the first value, the corresponding first information is extracted through the target processing function; Correspondingly, the device further includes a second processing unit, the second processing unit being used for: If the state control information takes the second value, the corresponding first information is not extracted through the target processing function.

[0133] In one embodiment, the apparatus further includes a creation unit, which, before locking the first buffer with a first lock upon receiving a first read request from a first user, is configured to: Create a first directory, whose contents include one or more of the following: The first file includes state control information, and the processing of relevant information for each type of kernel event is controlled by the state control information. The second file; the second file includes the first information for each type of kernel event.

[0134] In one embodiment, the apparatus further includes a SysFS interface, which is used to control the processing of information related to each type of kernel event by reading and writing the first file; and / or, the SysFS interface is used to obtain the first information of each type of kernel event by reading the second file.

[0135] In practical applications, the first locking unit 401, the first copying unit 402, the second copying unit 403, the first processing unit 404, the first writing unit, the first acquiring unit, the first unlocking unit, the first registration unit, the first triggering unit, the first determining unit, the second processing unit, the creation unit, etc., can be implemented by the processor in the data processing device, and the first receiving unit can be implemented by the processor in the data processing device in combination with the communication interface.

[0136] It should be noted that the data processing apparatus provided in the above embodiments is only illustrated by the division of the above program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the apparatus can be divided into different program modules to complete all or part of the processing described above. In addition, the data processing apparatus and data processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0137] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, this application also provides an electronic device, see [link to relevant documentation]. Figure 5 The electronic device includes: a first communication interface 1, a first processor 2, and a first memory 3.

[0138] Specifically, the first processor 2 is configured to lock the first buffer using a first lock upon receiving a first read request from a first user, wherein the first read request is for requesting to read first information, the first information including relevant information of each type of kernel event in one or more types of kernel events, and the first information is stored in the first buffer; The first processor 2 is further configured to copy the first information into temporary memory to generate a data snapshot of the first information; The first processor 2 is further configured to copy the data snapshot to the buffer corresponding to the first user; The first processor 2 is further configured to release the first buffer and unlock the first lock after the data snapshot is successfully copied to the buffer corresponding to the first user.

[0139] In one embodiment, before locking the first buffer with the first lock upon receiving a first read request from the first user, the first processor 2 is further configured to: The first information is written into the first buffer, the storage capacity of the first buffer is a fixed capacity, wherein when the space required by the first information is greater than the remaining space of the first buffer, the first information overwrites the historical information in the first buffer when the first information is written into the first buffer.

[0140] In one embodiment, before writing the first information into the first buffer, the first acquisition unit is used to acquire a second lock, and the first processor 2 is also used to lock the first buffer; After writing the first information into the first buffer, the first processor 2 is also used to release the second lock.

[0141] In one embodiment, before locking the first buffer with the first lock upon receiving a first read request from the first user, the first processor 2 is further configured to: Register a corresponding processing function for each type of kernel event. Different types of kernel events in the one or more types of kernel events correspond to different processing functions, and different processing functions are bound to different trace points in the kernel. In the event of any of the kernel events of the one or more types mentioned above, the corresponding trace point is triggered, and the first information is extracted through the target processing function, wherein the target processing function is the processing function bound to the corresponding trace point.

[0142] In one embodiment, the first communication interface 1 is further configured to receive a first instruction; the first determining unit is configured to determine state control information based on the first instruction. The first processor 2 extracts the corresponding first information through a target processing function, including: When the state control information is set to a first value, the first processor 2 is further configured to extract the corresponding first information through the target processing function; Correspondingly, the first processor 2 is also used for: If the state control information takes the second value, the corresponding first information is not extracted through the target processing function.

[0143] In one embodiment, before locking the first buffer with the first lock upon receiving a first read request from the first user, the first processor 2 is further configured to: Create a first directory, whose contents include one or more of the following: The first file includes state control information, and the processing of relevant information for each type of kernel event is controlled by the state control information. The second file; the second file includes the first information for each type of kernel event.

[0144] In one embodiment, the first communication interface includes a SysFS interface, which is used to control the processing of information related to each type of kernel event by reading and writing the first file; and / or, the SysFS interface is used to obtain the first information of each type of kernel event by reading the second file.

[0145] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 The general will label all buses as Bus System 4.

[0146] The first memory 3 in this embodiment is used to store various types of data to support operation in the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0147] The methods disclosed in the embodiments of this application can be applied to the first processor 2, or implemented by the first processor 2. The first processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the first processor 2. The first processor 2 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The first processor 2 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the first memory 3. The first processor 2 reads the information in the first memory 3 and completes the steps of the aforementioned method in combination with its hardware.

[0148] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0149] It is understood that the first memory 3 in this application embodiment can be a volatile memory pool or a non-volatile memory pool, or both. The non-volatile memory pool can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory pool can be a disk storage pool or a magnetic tape storage pool. The volatile memory pool can be a random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The storage pools described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of storage pools.

[0150] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a first memory 3 storing a computer program, which can be executed by a first processor 2 to complete the steps described in the aforementioned method.

[0151] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a first processor 2 to perform the steps described in the foregoing method.

[0152] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A data processing method, characterized in that, The data processing method includes: Upon receiving a first read request from a first user, the first buffer is locked using a first lock. The first read request is used to request the reading of first information, which includes information related to each type of kernel event in one or more types of kernel events. The first information is stored in the first buffer. The first information is copied into temporary memory to generate a data snapshot of the first information; Copy the data snapshot to the buffer corresponding to the first user; After the data snapshot is successfully copied to the buffer corresponding to the first user, the first buffer is released and the first lock is released.

2. The method according to claim 1, characterized in that, Before locking the first buffer with the first lock upon receiving the first read request from the first user, the method further includes: The first information is written into the first buffer, the storage capacity of the first buffer is a fixed capacity, wherein when the space required by the first information is greater than the remaining space of the first buffer, the first information overwrites the historical information in the first buffer when the first information is written into the first buffer.

3. The method according to claim 2, characterized in that, Before writing the first information into the first buffer, the method further includes: Acquire a second lock, which is used to lock the first buffer; After writing the first information into the first buffer, the method further includes: Release the second lock.

4. The method according to claim 1, characterized in that, Before locking the first buffer with the first lock upon receiving the first read request from the first user, the method further includes: Register a corresponding processing function for each type of kernel event. Different types of kernel events in the one or more types of kernel events correspond to different processing functions, and different processing functions are bound to different trace points in the kernel. When a kernel event of any type among the one or more types of kernel events occurs, the corresponding trace point is triggered, and the corresponding first information is extracted through the target processing function, wherein the target processing function is the processing function bound to the corresponding trace point.

5. The method according to claim 4, characterized in that, The method further includes: Receive the first instruction; Determine the status control information according to the first instruction; The step of extracting the corresponding first information through the target processing function includes: When the state control information takes the first value, the corresponding first information is extracted through the target processing function; Correspondingly, the method further includes: If the state control information takes the second value, the corresponding first information is not extracted through the target processing function.

6. The method according to claim 1, characterized in that, Before locking the first buffer with the first lock upon receiving the first read request from the first user, the method further includes: Create a first directory, whose contents include one or more of the following: The first file includes state control information, and the processing of relevant information for each type of kernel event is controlled by the state control information. The second file; the second file includes the first information for each type of kernel event.

7. The method according to claim 6, characterized in that, The method is applied to a first device, the first device including a SysFS interface, the SysFS interface being used to control the processing of information related to each type of kernel event by reading and writing the first file; and / or, the SysFS interface being used to obtain the first information of each type of kernel event by reading the second file.

8. A data processing apparatus, characterized in that, include: The first locking unit is used to lock the first buffer with a first lock when a first read request is received from a first user, wherein the first read request is used to request to read first information, the first information including relevant information of each type of kernel event in one or more types of kernel events, and the first information is stored in the first buffer; The first copying unit is used to copy the first information into temporary memory to generate a data snapshot of the first information; The second copying unit is used to copy the data snapshot to the buffer corresponding to the first user; The first processing unit is configured to release the first buffer and unlock the first lock after the data snapshot is successfully copied to the buffer corresponding to the first user.

9. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images