A hardware request scheduling method and system based on kernel ioctl hijacking

Abstract

This application provides a hardware request scheduling method and system based on kernel ioctl hijacking. The method includes: intercepting character device ioctl requests, obtaining the calling process attributes, request command codes, and parameters; rewriting the parameters of matching requests according to a pre-defined security equivalence rewriting rule verified by formal verification and differential fuzzing testing, while directly passing through unmatched requests; allocating the processed requests to corresponding priority queues according to process attributes, retrieving requests sequentially by priority and calling the device's original ioctl processing function, thereby enabling the request to be sent to the device driver for execution. This application does not require modification of existing drivers and user programs, and relies on verifiable ioctl hijacking and request aggregation mechanisms to avoid priority inversion problems and stably improve hardware scheduling throughput performance.

Description

Technical Field

[0001] This application belongs to the field of Linux kernel request scheduling and device driver interaction technology, and in particular relates to a hardware request scheduling method and system based on kernel ioctl hijacking. Background Technology

[0002] In modern Linux operating systems, when user-space programs interact with character devices (such as GPUs, NPUs, and other AI accelerator cards) via ioctl system calls, the kernel virtual file system layer directly passes the requests to the device driver, which typically processes them on a first-come, first-served basis. When multiple processes compete for hardware resources simultaneously, the lack of request queue management and priority scheduling mechanisms causes high-priority tasks (such as real-time inference) to have their ioctl requests blocked by low-priority, long-running tasks (such as background training data transfer), resulting in severe priority inversion and uncontrollable tail latency issues. To address this deficiency, existing solutions based on user-space LD_PRELOAD hijacking can be bypassed by static linking or direct system calls, but introduce significant inter-process communication overhead. Solutions based on hardware virtualization or driver-internal priority scheduling rely on specific hardware capabilities or driver support, making them unsuitable for all types of character devices. Furthermore, while theoretically ioctl calls can be optimized by merging small DMA copy requests to improve hardware throughput, the proprietary ioctl parameter structures of different device drivers make it extremely easy to corrupt the driver's internal state machine without modifying the driver itself, leading to data corruption or system crashes. Therefore, the field has long lacked a general technology that can both transparently inject priority scheduling capabilities and safely perform semantic-level optimization of ioctl requests. Summary of the Invention

[0003] This application provides a hardware request scheduling method and system based on kernel ioctl hijacking. Without modifying existing device drivers and user-space applications, it can both implement priority scheduling of character device ioctl requests to prevent priority inversion and safely rewrite request parameters to improve hardware throughput.

[0004] This application discloses a hardware request scheduling method based on kernel ioctl hijacking, including: Intercept the ioctl request of the target character device, and obtain the process attribute information of the initiating call, as well as the command code and parameters of the request; When the features of the request match the verified rewrite rule, the parameters of the request are rewritten according to the verified rewrite rule; otherwise, the request is passed through. The rewritten or passed-through request is allocated to the corresponding priority queue according to the process attribute information. The verified rewrite rule refers to the rule for rewriting ioctl requests securely and equivalently, which is determined in advance through formal verification and differential fuzzing. Request scheduling based on the priority queue includes: retrieving requests from the queue in priority order and calling the original ioctl processing function of the target character device to send the request to the device driver for execution.

[0005] Optionally, before intercepting the ioctl request of the target character device, the method further includes: obtaining the verified rewrite rule, wherein obtaining the verified rewrite rule includes: Construct Hall logic preconditions and postconditions for the ioctl command of the target character device; The original ioctl sequence and candidate rewrite sequence are initiated to the device through differential fuzzy testing. The device state after execution is compared to see if the postcondition is met. If it is met, a verified rewrite rule is generated.

[0006] Optionally, when the features of the request match a verified rewrite rule, performing a rewrite operation on the parameters of the request according to the verified rewrite rule includes: When the command code and parameter value of the request meet the merging conditions specified in the verified rewrite rules, and there is a similar request that is consecutive to the request among the pending requests intercepted by the same process, the request is merged with the similar requests into a new aggregate ioctl request.

[0007] Optionally, after calling the original ioctl processing function of the target character device, the following may also be included: The system receives the execution result of the aggregated ioctl request, splits the execution result into sub-results corresponding to each request before merging according to the verified rewrite rules, and wakes up the processes waiting for each sub-result.

[0008] Optionally, the process attribute information includes a process identifier or the control group identifier to which the process belongs; the step of allocating the rewritten or pass-through request to the corresponding priority queue according to the process attribute information includes: The process identifier or control group identifier is mapped to a preset priority level, and the request is inserted into the queue corresponding to that priority level.

[0009] Optionally, the request scheduling based on the priority queue includes: A kernel thread is created as a scheduler. The kernel thread retrieves requests from the priority queue in priority order and calls the original ioctl processing function using the context of the kernel thread.

[0010] Optionally, the request scheduling based on the priority queue includes: When the request from the initiating process is at the head of the queue, the original ioctl processing function is called directly using the kernel context of that process; otherwise, the process is blocked until its request becomes the head of the queue and is scheduled.

[0011] Optional, also includes: Runtime monitoring is performed on the applied and verified rewrite rules. When the anomaly rate of a request associated with a rule is detected to exceed the threshold, the rule is disabled and subsequent requests matching the rule are downgraded to pass-through processing.

[0012] Optionally, the verified rewrite rule is dynamically injected from user space via eBPF mapping; after intercepting the ioctl request of the target character device, the method further includes: The eBPF program mounted at the hijacking point is invoked to classify and rewrite the request.

[0013] This application also discloses a hardware request scheduling system based on kernel ioctl hijacking, including: The hijacking module is used to intercept ioctl requests from the target character device, obtain the process attribute information of the initiating call, and the command code and parameters of the request; The rewrite module is used to perform a rewrite operation on the parameters of the request according to the verified rewrite rule when the features of the request match the verified rewrite rule; otherwise, the request is passed through. The queue and scheduling module is used to allocate the rewritten or transparent requests to the corresponding priority queues according to the process attribute information, and retrieve the requests from the queues in priority order, and call the original ioctl processing function of the target character device to send them to the device driver for execution. The verified rewrite rules refer to the rules pre-determined through formal verification and differential fuzzing tests for rewriting ioctl requests securely and equivalently.

[0014] As described in the above technical solution, all ioctl requests sent to the target character device are intercepted at the hijacking point, obtaining the request command code, parameters, and priority attributes of the initiating process. Then, pre-verified rewrite rules obtained through formal verification and differential fuzzing are used to match the requests. If the request characteristics meet the conditions of a certain verified rule, multiple similar requests are merged into a single aggregate request, provided the rule guarantees equivalent execution results; otherwise, they are passed through as is. The rewritten or passed-through requests are allocated to different priority queues based on process attributes. The scheduler strictly retrieves requests according to priority order and calls the original device driver's ioctl processing function to complete the hardware operation. Throughout this process, the mechanism of intercepting and replacing device file operation pointers ensures that neither the user program nor the driver needs any modification to be included in the scheduling. The rewrite action is limited to rules that have been verified offline and proven to be equivalent to the original call sequence in the final state of the device, thus eliminating the risk of abnormal driver behavior caused by parameter tampering. The priority queue ensures that high-priority requests are always submitted to the driver first, fundamentally avoiding the blocking of urgent tasks by low-priority, long-running tasks. Meanwhile, request merging reduces the number of driver and hardware traps, lowers interrupt load and lock contention, and improves the effective bandwidth for data transport tasks. These features interact to allow the system to achieve both precise priority isolation and a significant increase in hardware throughput without sacrificing stability. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a hardware request scheduling method based on kernel ioctl hijacking in an embodiment of this application; Figure 2 This application embodiment describes a hardware request scheduling method based on kernel ioctl hijacking. Before intercepting the ioctl request of the target character device, the method obtains the verified rewrite rule. The flowchart of obtaining the verified rewrite rule is provided. Figure 3 This is a schematic diagram of the structure of a hardware request scheduling system based on kernel ioctl hijacking, provided in an embodiment of this application. Detailed Implementation

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not limiting, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without such specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0018] This application provides a hardware request scheduling method based on kernel ioctl hijacking, such as... Figure 1 As shown, it includes: S100: Intercept the ioctl request of the target character device, obtain the process attribute information of the initiating call, as well as the command code and parameters of the request; S200: When the features of the request match the verified rewrite rule, the parameters of the request are rewritten according to the verified rewrite rule; otherwise, the request is passed through. The rewritten or passed-through request is allocated to the corresponding priority queue according to the process attribute information. The verified rewrite rule refers to the rule for rewriting ioctl requests securely and equivalently, which is determined in advance through formal verification and differential fuzzing. S300: Request scheduling is performed according to the priority queue, including: retrieving requests from the queue in priority order and calling the original ioctl processing function of the target character device to send the request to the device driver for execution.

[0019] This application provides a hardware request scheduling method based on kernel ioctl hijacking. It intercepts all ioctl requests sent to the target character device through hijacking points, obtaining the request command code, parameters, and priority attributes of the initiating process. Then, it uses pre-verified rewrite rules obtained through formal verification and differential fuzzing to match the requests. If the request characteristics meet the conditions of a verified rule, multiple similar requests are merged into a single aggregate request, provided the rule guarantees equivalent execution results; otherwise, they are passed through as is. Rewritten or passed-through requests are allocated to different priority queues based on process attributes. The scheduler strictly retrieves requests according to priority order and calls the original device driver's ioctl processing function to complete the hardware operation. Throughout the process, the mechanism of intercepting and replacing device file operation pointers ensures that neither the user program nor the driver needs any modification to be included in the scheduling. The rewrite action is limited to rules that have been offline verified to be equivalent to the original call sequence in the final state of the device, thus eliminating the risk of abnormal driver behavior due to parameter tampering. The priority queue ensures that high-priority requests are always submitted to the driver first, fundamentally avoiding the blocking of urgent tasks by low-priority, long-running tasks. Meanwhile, request merging reduces the number of driver and hardware traps, lowers interrupt load and lock contention, and improves the effective bandwidth for data transport tasks. These features interact to allow the system to achieve both precise priority isolation and a significant increase in hardware throughput without sacrificing stability.

[0020] This application provides a hardware request scheduling method based on Linux kernel ioctl hijacking. This method runs in the Linux operating system kernel space and is applicable to various hardware devices based on the character device model. By establishing a unified request interception point at the kernel virtual file system layer, this method manages all ioctl requests sent to the target character device in a unified manner, achieving priority scheduling and secure equivalent rewriting of requests without modifying existing device drivers and user-space applications.

[0021] Specifically, intercepting ioctl requests from the target character device is achieved by replacing the function pointer in the file operation structure corresponding to the device. In the Linux kernel, each character device corresponds to a `file_operations` structure, which contains function pointers for various file operations supported by the device. Among them, the `unlocked_ioctl` function pointer is used to handle ioctl system calls initiated from user space. In this embodiment, when the kernel module is loaded, it first looks up the corresponding `file_operations` structure using the character device's device number or device node path, saves the original `unlocked_ioctl` function pointer in that structure to a global variable of the kernel module, and then uses an atomic operation to replace this function pointer with a custom hook function defined by the kernel module. The purpose of using an atomic operation to replace the function pointer is to avoid pointer inconsistency issues caused by concurrent access during the replacement process, ensuring that all ioctl requests initiated after the replacement can be intercepted by the hook function, while requests that have already entered the execution flow before the replacement continue to be processed by the original function without causing execution exceptions.

[0022] In this embodiment, when replacing the `unlocked_ioctl` function pointer in the `file_operations` structure, the issue of concurrent access needs to be considered. In the Linux kernel, multiple processes may access the same character device simultaneously. Therefore, during the function pointer replacement process, other processes may be calling the original `unlocked_ioctl` function. To avoid pointer inconsistency, this embodiment uses the `atomic_cmpxchg` function for atomic comparison and swapping. The kernel module first reads the original `unlocked_ioctl` function pointer, then calls the `atomic_cmpxchg` function to compare the `unlocked_ioctl` field in the `file_operations` structure with the original pointer. If they are equal, the pointer is replaced with a custom hook function pointer. If they are not equal, it means that the field has been modified by another kernel module after the original pointer was read. In this case, the kernel module abandons the replacement operation and returns the corresponding error code.

[0023] Furthermore, to further ensure the security of concurrent access, this embodiment of the application can also use the RCU mechanism to protect access to the `file_operations` structure. RCU is a synchronization mechanism for read-heavy, write-light scenarios. It allows read operations to be performed without locks, while write operations ensure consistency by delaying the release of old data. When using the RCU mechanism, after the kernel module replaces the `unlocked_ioctl` function pointer, it needs to wait for all read operations currently accessing the old pointer to complete before releasing the resources associated with the old pointer. This ensures that all executing ioctl requests can complete normally during the replacement process, preventing null pointer dereferences.

[0024] In this embodiment, when a user-space process initiates an ioctl system call, the execution flow, after processing by the kernel virtual file system layer, enters a custom hook function defined in this application. The hook function first obtains the attribute information of the process initiating the call. This process attribute information includes the process identifier, thread group identifier, control group identifier, security context, and nice value. The process identifier uniquely identifies the process initiating the request; the thread group identifier identifies the thread group to which the process belongs; the control group identifier retrieves the configuration information of the resource control group to which the process belongs; the security context performs access control checks; and the nice value helps determine the priority of the request. The hook function then extracts the ioctl request command code and user-space parameter address from the system call parameters. The command code is a 32-bit integer used to distinguish different types of ioctl operations, typically containing information such as device type, serial number, direction, and data length. The user-space parameter address points to a buffer in user space that stores the parameters of this ioctl request.

[0025] It's important to note that before retrieving user-space parameters, the hook function must first check the validity of the user-space address to ensure it belongs to an accessible region in user space, preventing kernel crashes caused by accessing illegal addresses. After successful verification, the hook function uses the `copy_from_user` function to copy the parameter data from the user-space buffer to a pre-allocated buffer in kernel space. The `copy_from_user` function automatically handles page faults that may exist at user-space addresses and returns an error code if the copy fails. If the copy fails, the hook function directly returns the error code to the user-space process, terminating the processing flow of this request.

[0026] In this embodiment, the following points should be noted when using the `copy_from_user` function: First, the `copy_from_user` function can only be called in process context, not interrupt context. Second, the `copy_from_user` function automatically handles page faults at user-space addresses. If the page corresponding to the user-space address is not in memory, the kernel will automatically swap the page into memory. Third, the `copy_from_user` function returns the number of bytes that were not successfully copied. A return value of 0 indicates a successful copy. A return value greater than 0 indicates a partial successful copy, in which case the kernel module should return the corresponding error code to user space. Fourth, the `copy_from_user` function does not access kernel-space addresses. If the passed address is a kernel-space address, the `copy_from_user` function will return an error to avoid kernel information leakage.

[0027] In alternative implementations, such as Figure 2 As shown, before intercepting the ioctl request of the target character device, the method further includes: obtaining the verified rewrite rule, wherein obtaining the verified rewrite rule includes: S210: Construct Hall logic preconditions and postconditions for the ioctl command of the target character device; S220: Initiate the original ioctl sequence and candidate rewrite sequence to the device through differential fuzzy testing, compare whether the device state after execution meets the postcondition, and if it does, generate a verified rewrite rule.

[0028] In this embodiment, a verified rewrite rule refers to a rule pre-determined through formal verification and differential fuzzing for securely and equivalently rewriting ioctl requests. Each verified rewrite rule comprises three parts: applicable conditions, rewrite actions, and verification information. The applicable conditions describe the request characteristics to which the rule can be applied, including the matching command code, the value range of request parameters, the characteristics of the process initiating the request, and the current state of the device. The rewrite actions describe how to modify the request parameters, how to merge multiple requests into a new request, or how to split a request into multiple new requests. The verification information proves the security and equivalence of the rewrite action, including the index of the formal specification, the test identifier of the differential fuzzing test, and test environment information.

[0029] Specifically, after obtaining the command code and kernel space parameter buffer of the ioctl request, the hook function matches the request characteristics against all rules in the verified rewrite rule base one by one. The matching process first compares whether the command code matches the command code specified in the rule; if they do not match, the rule is skipped. If the command code matches, it further checks whether the request parameter values ​​conform to the parameter value range specified in the rule, whether the characteristics of the process initiating the request conform to the process characteristics specified in the rule, and whether the current state of the device conforms to the state requirements specified in the rule. If all applicable conditions are met, the request is considered to match the rule.

[0030] In this embodiment, the verified rewrite rule base is stored using a hash table structure to improve rule matching speed. The key of the hash table is the ioctl command code, and the value is a linked list containing all verified rewrite rules applicable to that command code. When a new request arrives, the rewrite module first calculates the hash value based on the request's command code and finds the corresponding linked list in the hash table. Then, the rewrite module traverses all rules in the linked list, comparing the request's characteristics with the rules' applicability conditions one by one. Once the first matching rule is found, the corresponding rewrite action is executed. If no matching rule is found after traversing the entire linked list, the request is passed through for processing. The advantage of this hash table structure is its fast rule matching speed, with a time complexity of constant time plus the length of the linked list. Since the number of rules corresponding to each command code is usually not large, the overhead of rule matching is very small and will not significantly impact the performance of the online system.

[0031] For example, a rule for merging multiple data transfer requests might have the following applicable conditions: the command code is a data transfer command, the requested data length is a fixed value, the requested device target addresses are consecutive, the requesting processes are the same process, and the device is currently in a state where it can receive data transfer requests. When a new data transfer request arrives, the hook function checks whether the request's command code is a data transfer command and whether the data length meets the fixed value specified by the rule. It then checks whether there are other data transfer requests from the same process with consecutive device target addresses in the pending request queue maintained in the kernel module. If so, the request is considered to match the merging rule.

[0032] In this embodiment, if a request matches a verified rewrite rule, the hook function will perform a rewrite operation on the request parameters according to the rewrite action defined in the rule. The rewrite operation may include modifying the parameter values ​​of the request, merging multiple requests into a new aggregate request, or splitting a request into multiple new requests. If the request does not match any verified rewrite rule, or although it matches a rule but is not currently suitable for performing a rewrite operation (e.g., the rewrite operation might cause the request processing delay to exceed a preset threshold), the hook function will leave the request unchanged, i.e., pass-through processing.

[0033] In this embodiment, the kernel module maintains multiple priority queues, each corresponding to a priority level. The number of priority levels can be configured according to actual needs; for example, it can be configured to have three priority levels, corresponding to real-time tasks, normal tasks, and batch tasks, respectively. Each priority queue is a first-in-first-out linked list structure used to store requests waiting to be scheduled.

[0034] Specifically, after completing the rewriting or pass-through processing of the request, the hook function determines the priority level of the request based on the attribute information of the process that initiated the request. The priority level can be determined based on various factors such as the control group identifier to which the process belongs, the process's nice value, and the process type. For example, the mapping relationship between control groups and priority levels can be pre-configured, mapping requests initiated by processes belonging to the real-time control group to the highest priority queue, requests initiated by processes belonging to the batch control group to the lowest priority queue, and requests initiated by processes belonging to other control groups to the intermediate priority queue. Alternatively, adjustments can be made based on the process's nice value, allocating requests initiated by processes with lower nice values ​​to higher priority queues and requests initiated by processes with higher nice values ​​to lower priority queues.

[0035] In this embodiment, the specific process for obtaining the control group identifier to which a process belongs is as follows. Each process's `task_struct` structure contains a `cgroups` pointer, which points to a `css_set` structure. The `css_set` structure contains information about all control groups to which the process belongs. For the IO control group, its corresponding subsystem ID is `io_subsys_id`. The kernel module can obtain the `cgroup_subsys_state` structure pointer of the IO control group to which the process belongs by calling the `task_css` function, passing in the process's `task_struct` pointer and `io_subsys_id`. Then, by calling the `cgroup_path` function, passing in the `cgroup_subsys_state` structure pointer, the complete path of the control group can be obtained. The kernel module compares the obtained control group path with a pre-configured mapping table of control groups and priority levels to find the corresponding priority level.

[0036] In this embodiment, after determining the priority level of a request, the hook function encapsulates the request into a scheduling entity. The scheduling entity is a kernel data structure containing all relevant information about the request, such as the original request's command code, parameter buffer pointer, the rewritten request's command code and parameter buffer pointer, the attribute information of the process initiating the request, the request's priority level, the request's arrival time, the request's timeout, a rewrite flag, and an associated list of original requests. The rewrite flag indicates whether the request has undergone a rewrite operation, and the associated list of original requests stores information about all merged original requests when the request is an aggregated request. After encapsulation, the hook function inserts the scheduling entity into the tail of the queue corresponding to the priority level.

[0037] In this embodiment, the priority queue is implemented using a doubly linked list structure, with each queue having a spinlock to protect concurrent access. When a new request is inserted into the queue, the kernel module first acquires the queue's spinlock, then adds the request's scheduling entity to the tail of the linked list, and finally releases the spinlock. When the scheduler retrieves a request from the queue, it first acquires the queue's spinlock, then retrieves the first scheduling entity from the head of the linked list, and finally releases the spinlock. Using spinlocks ensures that in a multi-CPU system, multiple CPUs accessing the queue simultaneously will not cause data inconsistency issues. Furthermore, to improve queue access efficiency, this embodiment can also employ a lock-free queue implementation. Lock-free queues use atomic operations to implement queue insertion and deletion operations, eliminating the need for spinlocks, thus offering better performance in high-concurrency scenarios.

[0038] In this embodiment, the kernel module includes a scheduler for selecting the next request to be executed from multiple priority queues. The scheduler's scheduling strategy can be configured according to actual needs, such as a strict priority scheduling strategy, a weighted fair queue scheduling strategy, or a token bucket rate-limiting scheduling strategy. A strict priority scheduling strategy means the scheduler always prioritizes retrieving requests from the highest priority queue for execution. Only when the highest priority queue is empty will a request be retrieved from the next highest priority queue, and so on. A weighted fair queue scheduling strategy means the scheduler assigns a weight to each priority queue and retrieves requests from each queue according to the weight ratio, ensuring that requests of different priorities receive a certain amount of processing time. A token bucket rate-limiting scheduling strategy means the scheduler sets a token bucket for each priority queue. Only when the token bucket has enough tokens will a request be retrieved from that queue for execution, thus limiting the maximum request processing rate of each priority queue.

[0039] In this embodiment, the weighted fair queue scheduling strategy is implemented as follows: The scheduler assigns a weight to each priority queue; the higher the weight, the more CPU time the queue receives. The scheduler maintains a virtual time variable to record the service time of each queue. When a request is retrieved from a queue for execution, the scheduler calculates the virtual time increment corresponding to the request based on the expected execution time of the request and the weight of the queue. Then, the scheduler adds this increment to the virtual time of the queue. The scheduler always selects the queue with the smallest virtual time to execute the next request. This ensures that the CPU time received by each queue is proportional to its weight, achieving weighted fair scheduling. The advantage of the weighted fair queue scheduling strategy is that it avoids the starvation problem of low-priority queues. Even if high-priority requests continue to arrive, low-priority requests can still obtain a certain amount of CPU time and will not be indefinitely unexecuted. This is crucial for ensuring the fairness and stability of the system.

[0040] For example, when using a strict priority scheduling strategy, the scheduler checks each priority queue in descending order of priority. If a priority queue is not empty, the scheduler retrieves a scheduling entity from the head of that queue, then calls the original `unlocked_ioctl` function pointer saved during the interception phase. The scheduler passes the request's command code and parameter buffer pointer stored in the scheduling entity to this function, which then forwards the request to the device driver for execution. After the original function completes execution, it returns an execution result code, which the scheduler saves to the scheduling entity.

[0041] It's important to note that when the scheduler calls the original `unlocked_ioctl` function, it must ensure that the parameters passed to the function are valid and comply with the device driver's requirements. For rewritten requests, the scheduler passes the rewritten command code and a pointer to the parameter buffer. For pass-through requests, the scheduler passes the original command code and a pointer to the parameter buffer.

[0042] In this embodiment, after the original `unlocked_ioctl` function completes execution, the scheduler performs different follow-up processing depending on whether the request has been rewritten. For requests that are passed through, the scheduler writes the execution result code returned by the original function back to the return value address specified by the user-space process using the `copy_to_user` function, and then wakes up the user-space process that was sleeping while waiting for the request to complete. For requests that have been rewritten, the scheduler processes the execution result returned by the original function according to the result processing logic defined in the rewriting rules, then writes the processed result back to user space and wakes up the corresponding waiting process.

[0043] In this embodiment, the `copy_to_user` function is used to copy data from kernel space to user space. It can only be called within a process context, automatically handles page faults, and returns the number of bytes that were not successfully copied. If the copy fails, the kernel module should return the corresponding error code to user space. The `copy_to_user` function also does not access kernel space addresses; if the passed user space address is a kernel space address, the `copy_to_user` function will return an error to prevent kernel information leakage.

[0044] In this embodiment, before intercepting the ioctl request of the target character device, a step of obtaining verified rewrite rules is included. The process of obtaining verified rewrite rules is completed offline, that is, before deploying the kernel module of this application to the production environment, it is executed on a dedicated test node for the target character device and its corresponding driver. Obtaining verified rewrite rules includes constructing Hall logic preconditions and postconditions for the ioctl command of the target character device, and initiating the original ioctl sequence and candidate rewrite sequences to the device through differential fuzzing testing, comparing whether the device state after execution meets the postconditions, and if so, generating a verified rewrite rule.

[0045] Specifically, constructing Hall logic preconditions and postconditions for the ioctl command of the target character device is fundamental to obtaining verified rewrite rules. Hall logic is a formal logic used to reason about the correctness of a program. It uses triples to describe the behavior of a program, and each triple contains three parts: a precondition, an execution statement, and a postcondition. The precondition describes the state that the system must satisfy before the execution statement, the execution statement describes the operation to be performed, and the postcondition describes the state that the system should satisfy after the execution statement. In this embodiment, the execution statement corresponds to an ioctl command of the target character device. The precondition describes the state that the device and system must satisfy before executing the ioctl command, and the postcondition describes the state that the device and system should achieve after executing the ioctl command.

[0046] For example, the preconditions for an ioctl command used to copy user-space data to device memory might include: the user-space buffer has been allocated and has read permissions; the length of the user-space buffer is equal to the requested data length; the target address in device memory has been allocated and has write permissions; and the device is currently in a state where it can receive data transfer requests. The postconditions might include: the content of the continuous data length in device memory starting from the target address is consistent with the content in the user-space buffer; the ioctl command returning a value of 0 indicates successful execution; and the flag indicating data transfer completion in the device's status register being set.

[0047] In this embodiment, the construction of Hall logic preconditions and postconditions can be completed semi-automatically. First, a list of ioctl commands and the parameter structure of each command are extracted from sources such as the target character device's public header files, technical documentation, and exported symbols of the driver. Then, based on the functional description of each command, the abstract preconditions and postconditions are defined by technical personnel. For commands with complex behaviors, the definitions of preconditions and postconditions can be improved by dynamically tracking the device's behavior and observing the device's output and state changes under different inputs.

[0048] Furthermore, those skilled in the art will understand that the definitions of Hall logic preconditions and postconditions do not need to completely cover all internal states of the device, but only the states related to the externally visible behavior of the ioctl command. This is because the goal of this application is to ensure that the rewritten ioctl request is equivalent to the original request sequence in terms of externally visible device states, without needing to concern themselves with the specific implementation details of the device's internals. This abstraction approach can significantly reduce the complexity of formal modeling while ensuring the accuracy of the rewrite equivalence determination.

[0049] In this embodiment, after constructing the Hall logic preconditions and postconditions, the next step is to automatically generate and verify equivalent rewrite rules through differential fuzzing. Differential fuzzing is a testing method that discovers vulnerabilities or equivalent behaviors in a system by inputting a large number of randomly generated test cases into the target system and observing the differences in the target system's behavior. In this embodiment, the goal of differential fuzzing is to discover rewrite rules that can rewrite one ioctl request sequence into another ioctl request sequence, and the execution results before and after the rewrite are equivalent in terms of device state.

[0050] Specifically, differential fuzzing is executed on dedicated test nodes equipped with the same target character devices and drivers as the production environment. The kernel module runs offline on these test nodes, with a built-in differential fuzzing agent. The agent first generates a large number of random but compliant raw ioctl request sequences based on predefined Hall logic preconditions. Each raw ioctl request sequence consists of one or more ioctl commands, with parameter values ​​for each command within the legal range specified by the preconditions. Then, the agent generates corresponding candidate rewrite sequences for each raw ioctl request sequence based on preset rewrite patterns. These preset rewrite patterns include various common optimization methods such as request merging, request splitting, request rearrangement, and parameter replacement.

[0051] In this embodiment, the process of generating test cases by the differential fuzzy testing agent is as follows. First, the differential fuzzy testing agent generates random parameter values ​​for each ioctl command based on the parameter range defined in the Hall logic preconditions. The parameter values ​​can be generated using various distribution methods, such as uniform distribution, normal distribution, and boundary value distribution, to cover as many parameter value scenarios as possible. For integer type parameters, boundary values ​​such as minimum, maximum, median, zero, and negative values ​​are generated. For pointer type parameters, valid user space addresses, null pointers, and invalid user space addresses are generated. For structure type parameters, random values ​​are generated for each field of the structure to ensure that each field covers its possible value range. Then, the differential fuzzy testing agent combines the generated individual ioctl commands into an original ioctl request sequence. The length of the sequence can vary randomly within a preset range. The commands in the sequence can be commands of the same type or combinations of commands of different types. The differential fuzzy testing agent generates a reasonable command sequence based on the dependencies between ioctl commands. For example, for commands that require memory allocation before data transfer, the differential fuzz test agent will first generate the memory allocation command and then generate the data transfer command to avoid generating invalid command sequences.

[0052] In this embodiment, the differential fuzzing agent also employs coverage-guided fuzzing techniques to improve test coverage. The differential fuzzing agent tracks the execution path of the driver, recording which code paths have been tested and which have not. Then, the agent adjusts its test case generation strategy, prioritizing the generation of test cases that cover untested code paths. This significantly improves testing efficiency, enabling the discovery of more equivalent rewrite rules and potential driver vulnerabilities in a shorter time.

[0053] For example, for an original sequence consisting of multiple consecutive small data transfer requests, the differential fuzzing agent generates a candidate rewrite sequence containing an aggregated data transfer request whose parameters are a scatter-aggregate list. Each entry in the list corresponds to the source address, destination address, and data length of a small data transfer request in the original sequence. For an original sequence consisting of multiple independent control commands, the differential fuzzing agent generates multiple candidate rewrite sequences, each corresponding to a different command execution order, used to verify whether these commands can be reordered without affecting the final result.

[0054] In this embodiment, for each pair of original sequences and candidate rewrite sequences, the differential fuzzing agent executes both sequences under the same initial device state. To ensure consistency of the initial state, the differential fuzzing agent performs a device reset operation before executing each sequence, restoring the device to a known initial state. During sequence execution, the differential fuzzing agent records all system call return values, device-generated interrupts, changes in device registers, and other information. After sequence execution, the differential fuzzing agent collects the complete device state after execution through various methods, including hardware debugging interfaces, PCIe bus analyzers, and device memory snapshots.

[0055] It should be noted that the purpose of collecting the complete device state is to comprehensively compare whether the execution results of the original sequence and the candidate rewrite sequence are equivalent. The complete device state includes all register values ​​of the device, all contents of the device memory, and the state of the device's internal state machine. For some internal states that cannot be directly read, they can be obtained indirectly by sending specific query commands to the device.

[0056] In this embodiment, the differential fuzzing agent compares the device state after executing the original sequence and the device state after executing the candidate rewrite sequence with the Hall logic postcondition. If both execution results satisfy the same postcondition, i.e., starting from the same preceding state and reaching an equivalent following state, the candidate rewrite sequence is considered semantically equivalent to the original sequence, and the corresponding rewrite mode is safe. At this time, the differential fuzzing agent generates a verified rewrite rule, which includes applicable conditions, rewrite actions, and verification information. The applicable conditions describe the characteristics of the original request sequence to which the rule can be applied, the rewrite actions describe how to convert the original sequence into a candidate rewrite sequence, and the verification information includes the test identifier of this differential fuzzing test, test environment information, and the index of the Hall logic specification.

[0057] For example, for the rewrite pattern that merges multiple small data transmission requests, if the differential fuzzing test passes, the applicable conditions for the generated verified rewrite rules might include that all requests in the original sequence are data transmission commands, each request has the same data length, the device target addresses of the requests are consecutive, and the number of requests is within a preset range. The rewrite action describes how to combine the parameters of these requests into a scattered aggregated list of aggregated data transmission requests. The verification information includes the test number of this test, the driver version used during the test, the device firmware version, and the index of the corresponding Hall logic specification, etc.

[0058] In this embodiment, the differential fuzzing testing agent continuously generates and tests a large number of original sequences and candidate rewrite sequences until all preset rewrite patterns and all key ioctl commands are covered. All generated verified rewrite rules are compiled into a kernel module-loadable format and stored in the verified rewrite rule library. When the kernel module is deployed to the production environment, the verified rewrite rule library is loaded into the kernel space along with the kernel module for use by the rewrite module in the online phase.

[0059] Furthermore, this application is not limited to this; the verified rewrite rule base can also be dynamically updated during production environment operation. When the target character device driver is upgraded or the device firmware is updated, differential fuzz testing can be re-executed on the test node to generate new verified rewrite rules. These new rules can then be dynamically injected into the kernel module of the production environment without reloading the entire kernel module or restarting the system.

[0060] In an optional implementation, when the features of the request match a verified rewrite rule, performing a rewrite operation on the parameters of the request according to the verified rewrite rule includes: When the command code and parameter value of the request meet the merging conditions specified in the verified rewrite rules, and there is a similar request that is consecutive to the request among the pending requests intercepted by the same process, the request is merged with the similar requests into a new aggregate ioctl request.

[0061] In this embodiment of the application, when the features of the request match the verified rewrite rules, the rewrite operation on the parameters of the request according to the verified rewrite rules includes merging the request with the requests of the same type into a new aggregate ioctl request when the command code and parameter value of the request meet the merging conditions specified by the verified rewrite rules, and there are similar requests that are consecutive to the request among the pending requests intercepted by the same process.

[0062] Specifically, the merging conditions are part of the applicable conditions defined in the verified rewrite rules, used to determine whether multiple requests can be safely merged into a single aggregate request. Merging conditions typically include the following aspects: First, all requests have the same command code, meaning they are all the same type of ioctl operation. Second, all request parameter values ​​conform to the range specified in the rules, such as a fixed data length or a data length within a certain range. Third, all requests have consecutive device target addresses, meaning the target address of the previous request plus the data length equals the target address of the next request. Fourth, all requests originate from the same process, meaning the process identifiers initiating the requests are the same. Fifth, the time interval between the initiation of all requests is within a preset threshold to avoid excessive processing delays due to waiting for merging. Sixth, the current state of the device allows the execution of the aggregate request, meaning the device supports this type of aggregation operation and currently has sufficient resources to process the aggregate request.

[0063] In this embodiment, to achieve request merging, the kernel module maintains a per-task rewrite cache for each process. The rewrite cache is a temporary storage area used to store requests that have been intercepted but have not yet met the merging conditions. When a new request arrives, the rewrite module first checks if the request matches a certain merging rule. If it matches, the rewrite module further checks if there is a similar request consecutively with the current request in the rewrite cache corresponding to that process. If it exists, the current request is added to the rewrite cache and waits to be merged along with existing requests. If it does not exist, the current request is added to the rewrite cache as a new merging candidate.

[0064] For example, for a rule used to merge data transfer requests, when a new data transfer request arrives, the rewrite module checks whether the request's command code is a data transfer command and whether the data length conforms to the fixed value specified by the rule. If they do, the rewrite module checks whether a data transfer request exists in the process's rewrite buffer, and whether its device target address plus the data length equals the device target address of the current request. If it exists, the current request is added to the request list in the buffer. If it does not exist, a new request list is created in the buffer, and the current request is added to the list.

[0065] In this embodiment, the rewrite module continuously monitors the request list in each rewrite cache to determine whether the merge trigger conditions are met. The merge trigger conditions include the following: First, the number of requests in the request list reaches the maximum number of requests to be merged as specified by the rules. Second, the waiting time of the first request in the request list reaches the maximum timeout threshold specified by the rules. Third, a new request arrives, which is not consecutive with the last request in the request list and cannot be merged further. Fourth, the scheduler is ready to retrieve a request from the priority queue corresponding to the process for execution. When any one of the merge trigger conditions is met, the rewrite module merges all requests in the request list into a new aggregate ioctl request.

[0066] Specifically, the merging process includes the following steps: First, a new parameter buffer is allocated in kernel space according to the parameter structure of the aggregated request defined in the rules. Second, the parameter information of each original request in the request list is combined according to the parameter structure of the aggregated request and filled into the new parameter buffer. For example, for aggregated data transmission requests, the source address, destination address, and data length of each original request need to be combined into a scattered aggregation list and filled into the parameter buffer of the aggregated request. Third, a new scheduling entity is created, whose command code is the command code of the aggregated request, and whose parameter buffer pointer points to the newly allocated kernel parameter buffer. Fourth, the information of the scheduling entities of all original requests in the request list is associated with the new aggregated scheduling entity, including the initiating process information, return value address, and user space parameter buffer address of each original request. Fifth, the new aggregated scheduling entity is inserted into the queue of the corresponding priority level. Sixth, the corresponding request list in the process's rewrite buffer is cleared.

[0067] It's important to note that during the merging process, the rewrite module sets a flag for the scheduling entity of each original request, indicating that the request has been merged into the aggregate request and will no longer be scheduled for execution separately. Simultaneously, the rewrite module adds the scheduling entity of each original request to the associated original request list of the aggregate scheduling entity, ensuring that the results are correctly split and the corresponding waiting processes are awakened after the aggregate request is executed.

[0068] Furthermore, those skilled in the art will understand that the merging operation can be applied not only to data transfer requests but also to other types of ioctl requests, provided that such requests can be safely merged and the equivalence of the merging has been verified through differential fuzzing. For example, multiple consecutive device register write requests can be merged into a single bulk register write request. Multiple consecutive memory allocation requests can be merged into a single large memory allocation request before internal allocation.

[0069] In an optional implementation, after calling the target character device's original ioctl processing function, the method further includes: The system receives the execution result of the aggregated ioctl request, splits the execution result into sub-results corresponding to each request before merging according to the verified rewrite rules, and wakes up the processes waiting for each sub-result.

[0070] In this embodiment of the application, after calling the original ioctl processing function of the target character device, the method further includes receiving the execution result of the aggregated ioctl request, splitting the execution result into sub-results corresponding to each request before merging according to the verified rewrite rules, and waking up the processes waiting for each sub-result respectively.

[0071] Specifically, after the scheduler calls the original `unlocked_ioctl` function to execute the aggregation request, it receives the execution result code returned by the original function. For an aggregation request, the execution result code indicates the execution status of the entire aggregation operation. If the execution result code is 0, it means that the entire aggregation operation was executed successfully. If the execution result code is non-zero, it means that the aggregation operation failed, and the reason for the failure is indicated by the specific value of the result code.

[0072] In this embodiment, the rewriting module splits the execution result of the aggregation request into sub-results corresponding to each original request before merging, according to the result splitting logic defined in the verified rewriting rules that generated the aggregation request. The result splitting logic is part of the verified rewriting rules and is determined by the differential fuzzy testing agent based on the behavior of the aggregation request when generating the rules.

[0073] For example, for an aggregated data transfer request generated by merging multiple data transfer requests, if the aggregated request executes successfully, the sub-results of each original data transfer request are successful, and the return value is 0. If the aggregated request fails, the sub-results of each original request need to be determined based on the specific reason for the failure and the location of the failure. For instance, if the aggregated request fails to transfer data from the third original request, the sub-results of the first two original requests are successful, and the sub-results of the third and subsequent original requests are failed, with the return value being the corresponding error code.

[0074] In this embodiment, after splitting the original request into sub-results, the rewrite module writes the sub-results back to the user space return value address corresponding to each original request. If the original request has data that needs to be written back, such as the ioctl command reading data from the device into user space, the rewrite module also needs to split the data returned by the aggregation request according to the boundaries of the original request, and then use the copy_to_user function to write the split data back to the user space buffer corresponding to each original request.

[0075] It should be noted that when writing back user-space data, the rewrite module needs to ensure that the user-space buffer address of each original request is valid and has write permissions. If the user-space buffer address of an original request is invalid, or an error occurs during the write-back process, the sub-result of that original request will be set to the corresponding error code.

[0076] In this embodiment, after writing back the sub-result, the rewrite module wakes up the user-space processes waiting for each original request to complete one by one. Each original request's scheduling entity stores the corresponding wait queue head, and the rewrite module wakes up all processes in the wait queue by calling the `wake_up` function. The woken-up process returns from the `ioctl` system call and receives the corresponding sub-result code.

[0077] Furthermore, this application is not limited to this. For partially successful aggregation requests, the rewrite module can also perform retry operations based on the actual situation. For example, if an aggregation request fails to transmit due to temporary device congestion, the rewrite module can reinsert the unsuccessfully transmitted original request into the priority queue, waiting for the next scheduled execution. The number of retries and the interval can be configured according to actual needs to avoid request processing failures due to temporary faults.

[0078] In an optional implementation, the process attribute information includes a process identifier or the control group identifier to which the process belongs; the step of allocating the rewritten or pass-through request to the corresponding priority queue according to the process attribute information includes: The process identifier or control group identifier is mapped to a preset priority level, and the request is inserted into the queue corresponding to that priority level.

[0079] In this embodiment, process attribute information includes a process identifier or a control group identifier to which the process belongs. Allocating the rewritten or transparently transmitted request to the corresponding priority queue based on the process attribute information includes mapping the process identifier or control group identifier to a preset priority level and inserting the request into the queue corresponding to that priority level.

[0080] Specifically, control groups are a mechanism in the Linux kernel used for grouping and managing processes and controlling resources. Control groups allow a group of processes to be organized together, and then their use of resources such as CPU, memory, and I / O can be restricted and scheduled. In this embodiment, using control groups to prioritize requests has a natural advantage because control groups are widely used in containerized environments and cloud computing platforms for resource isolation and QoS assurance for different applications and services.

[0081] In this embodiment, the mapping relationship between control groups and priority levels is pre-configured. The mapping relationship can be configured through kernel module parameters or dynamically modified at runtime via the sysfs interface. For example, a control group for a specific path can be mapped to the highest priority level, a control group for another path to the lowest priority level, and the default control group to an intermediate priority level. When a process belongs to a control group, all ioctl requests initiated by that process will be assigned to the priority level queue corresponding to that control group.

[0082] In addition, priority mapping can be configured based on process identifiers. For certain special processes, such as critical system processes, the mapping between process identifiers and priority levels can be configured to allocate requests from these processes to higher-priority queues, ensuring that their requests are processed with priority. The mapping between process identifiers and priority levels can also be configured through module parameters or the sysfs interface.

[0083] In this embodiment, the priority mapping process is as follows: When a request is intercepted, the hook function first obtains the control group identifier of the process that initiated the request. Then, the hook function queries the pre-configured mapping table between control groups and priority levels to find the priority level corresponding to the control group. If a corresponding priority level is found, the request is assigned to the queue corresponding to that priority level. If no corresponding priority level is found, the process identifier and priority level mapping table is further queried. If a corresponding priority level is found, the request is assigned to the queue corresponding to that priority level. If none of these are found, the request is assigned to the queue corresponding to the default priority level.

[0084] For example, in a containerized environment, each container corresponds to an independent control group. Platform administrators can configure different priority levels for each container's control group based on the container's service type. For instance, the control group for containers running real-time video analytics can be configured with the highest priority, while the control group for containers running model training can be configured with the lowest priority. This way, ioctl requests initiated by the real-time video analytics container will be assigned to the highest priority queue and processed first. ioctl requests initiated by the model training container will be assigned to the lowest priority queue and will only be processed if there are no high-priority requests.

[0085] It should be noted that the priority mapping method based on control groups offers better scalability and manageability. In large-scale cluster environments, administrators can configure different priority levels for tens of thousands of containers through a unified control group management platform, without needing to configure each process individually. Furthermore, the resource control mechanism of control groups can work in conjunction with the ioctl scheduling mechanism of this application to achieve more comprehensive QoS guarantees.

[0086] Furthermore, those skilled in the art will understand that priority mapping is not limited to control group identifiers and process identifiers, but may also include other process attribute information such as the process's nice value, user identifier, and group identifier. For example, requests initiated by processes with nice values ​​below a certain threshold can be assigned to a higher priority queue, while requests initiated by processes with nice values ​​above a certain threshold can be assigned to a lower priority queue. Requests initiated by processes of a specific user or user group can be assigned to a specified priority queue.

[0087] In an optional implementation, the request scheduling based on the priority queue includes: A kernel thread is created as a scheduler. The kernel thread retrieves requests from the priority queue in priority order and calls the original ioctl processing function using the context of the kernel thread.

[0088] In this embodiment of the application, request scheduling based on the priority queue includes creating a kernel thread as a scheduler, having the kernel thread retrieve requests from the priority queue in priority order, and using the context of the kernel thread to call the original ioctl processing function.

[0089] Specifically, a kernel thread is an independent execution flow running in kernel space. It has no user-space context and can only execute kernel code. In this embodiment, using a kernel thread as the scheduler has the following advantages: First, the priority of the kernel thread can be configured independently. The priority of the scheduler kernel thread can be set to a higher level, ensuring that the scheduler can retrieve requests from the queue and execute them in a timely manner. Second, kernel threads can run continuously in the background without relying on the context of user processes. Even if all user processes are in a sleep state, the scheduler can continue to process requests in the queue. Third, kernel threads can easily implement asynchronous scheduling, separating request processing from user process execution and improving the overall concurrency of the system.

[0090] In this embodiment, when the kernel module is loaded, it creates one or more scheduler kernel threads by calling the kthread_run function. The kthread_run function is a function provided by the Linux kernel for creating kernel threads. It accepts a thread function, a thread parameter, and a thread name as parameters, and returns a task_struct structure pointer representing the created kernel thread.

[0091] For example, a kernel module can create a single scheduler kernel thread to handle requests from all priority queues. Alternatively, it can create a separate scheduler kernel thread for each priority queue, with each thread handling requests only for its corresponding priority queue. Creating a separate kernel thread for each priority queue improves scheduling parallelism, especially in multi-CPU systems, where different kernel threads can be bound to different CPUs to achieve true parallel processing.

[0092] In this embodiment, the scheduler kernel thread's main loop performs the following operations: First, it checks each priority queue in priority order to determine if there are any requests waiting to be executed. Second, if there are any requests waiting to be executed, it retrieves a scheduling entity from the head of the queue. Third, it calls the original `unlocked_ioctl` function, passing the command code and parameter buffer pointer from the scheduling entity to the function to execute the request. Fourth, it processes the execution result of the request, including writing it back to user space and waking up waiting processes. Fifth, it repeats the above steps until the kernel thread is stopped.

[0093] It's important to note that when all priority queues are empty, the scheduler kernel thread calls the `schedule` function to enter a sleep state, releasing CPU resources. When a new request is inserted into the priority queue, the kernel module calls the `wake_up` function to wake up the scheduler kernel thread and allow it to continue processing requests. This approach prevents the scheduler kernel thread from idling and wasting CPU resources when there are no requests.

[0094] In this embodiment, calling the original ioctl handler function using the kernel thread's context means that the request is executed within the scheduler kernel thread's context, rather than within the context of the user process initiating the request. The advantage of this approach is that it frees the user process from handling ioctl requests. After initiating a request, the user process can immediately enter a sleep state, waiting for the request to complete, without being blocked in the driver's processing. This allows the user process to perform other computational tasks while waiting for the request to complete, improving CPU utilization.

[0095] Furthermore, this application is not limited to this; the number of scheduler kernel threads and the CPUs they are bound to can be dynamically adjusted according to the actual hardware configuration and business needs. For example, in a multi-CPU system, multiple scheduler kernel threads can be created, and each thread can be bound to a separate CPU to fully utilize the parallel processing capabilities of multiple CPUs. Under lighter loads, the number of scheduler kernel threads can be reduced to save system resources.

[0096] In an optional implementation, the request scheduling based on the priority queue includes: When the request from the initiating process is at the head of the queue, the original ioctl processing function is called directly using the kernel context of that process; otherwise, the process is blocked until its request becomes the head of the queue and is scheduled.

[0097] In this embodiment, request scheduling based on a priority queue includes directly invoking the original ioctl handler function using the kernel context of the process that initiated the call if its request is at the head of the queue. Otherwise, the process is blocked until its request becomes the head of the queue and is scheduled.

[0098] Specifically, utilizing the kernel context of a user process to execute requests is a low-latency scheduling method. When a user process initiates an ioctl request, it is already in kernel context and has its own complete process address space. If the process's request is at the head of the priority queue, and all higher-priority queues are empty, the original ioctl handler function can be called directly within the current process's context to execute the request, without context switching or waiting for scheduling by the scheduler kernel thread. This method can significantly reduce request processing latency, making it particularly suitable for real-time tasks with high latency requirements.

[0099] In this embodiment, the execution process of this scheduling method is as follows: When a new request is intercepted and inserted into the priority queue, the hook function first checks whether the request is at the head of its priority queue and whether all higher-priority queues are empty. If so, the hook function directly calls the original `unlocked_ioctl` function to execute the request in the context of the current process. After execution, the hook function writes the result back to user space and then returns it to the user process; the user process does not need to enter a sleep state. If the request is not at the head of the queue, or if a higher-priority request is waiting to be processed, the hook function adds the current process to the waiting queue of the request and then calls the `schedule` function to block the process until the request is scheduled and executed.

[0100] For example, when a high-priority real-time process initiates an ioctl request, if the highest-priority queue is empty, the request will be inserted at the head of the highest-priority queue. The hook function checks that the request is at the head of the queue and there are no higher-priority queues, so it directly calls the original ioctl handler function to execute the request within the context of the current process. After execution, the process returns immediately to continue executing subsequent tasks. The entire process involves no context switching and has low latency.

[0101] It's important to note that this scheduling method can be combined with using kernel threads as the scheduler. For high-priority real-time requests, execution using user process context can be prioritized to reduce latency. For low-priority batch requests, kernel thread scheduling can be used to improve overall system throughput. The kernel module can automatically select the appropriate scheduling method based on the request's priority level.

[0102] Furthermore, those skilled in the art will understand that executing requests using user process context requires careful attention to avoid priority inversion. When a low-priority process's request is at the head of the queue and is being executed by its context, if a high-priority process initiates a new request, the high-priority process's request will be inserted at the head of the high-priority queue. However, because the low-priority process is currently executing its request and holding the driver's mutex lock, the high-priority process's request cannot be executed immediately, resulting in priority inversion. To solve this problem, a priority inheritance protocol can be used. When a high-priority process waits for a lock held by a low-priority process, the low-priority process's priority is increased to the high-priority process's priority until the low-priority process releases the lock.

[0103] In an optional implementation, the method further includes: Runtime monitoring is performed on the applied and verified rewrite rules. When the anomaly rate of a request associated with a rule is detected to exceed the threshold, the rule is disabled and subsequent requests matching the rule are downgraded to pass-through processing.

[0104] In this embodiment of the application, runtime monitoring of the applied and verified rewrite rules is also included. When the abnormal rate of request execution associated with a rule is detected to exceed a threshold, the rule is disabled and subsequent requests matching the rule are downgraded to pass-through processing.

[0105] Specifically, although the rewrite rules have been verified to be predetermined through formal verification and differential fuzz testing, in actual production environments, due to changes in device state, implicit issues in drivers, and the triggering of extreme boundary conditions, some rules may cause request execution anomalies under certain circumstances. To avoid these anomalies affecting the stability and availability of the system, this application embodiment introduces runtime monitoring and rule circuit breaking mechanisms to monitor the execution of applied rewrite rules in real time. Once an anomaly is detected, the rule is immediately disabled, and subsequent requests are downgraded to pass-through processing.

[0106] In this embodiment, the runtime monitoring module maintains a set of statistical metrics for each verified rewrite rule, including the number of times the rule was applied, the number of successful executions, the number of failed executions, the average execution latency, the maximum execution latency, and the number of latency anomalies. Each time a rule is applied to perform a rewrite operation, the monitoring module updates the corresponding statistical metrics. Successful execution means that after the request is executed, the return value meets expectations, and the device status is normal. Execution failure means that after the request is executed, the return value is an error code, or the device status is abnormal. Latency anomalies mean that the execution latency of the request exceeds a preset maximum latency threshold.

[0107] For example, for a rule that merges data transmission requests, the monitoring module will count how many times the rule is applied, how many times the merged aggregated request is executed successfully, how many times it fails, the average execution latency of each aggregated request, and how many times the execution latency of the aggregated request exceeds a preset threshold.

[0108] In this embodiment, the monitoring module periodically calculates the execution anomaly rate for each rule. The execution anomaly rate is the sum of the number of execution failures and the number of latency anomalies, divided by the total number of times the rule is applied. The monitoring module compares the calculated execution anomaly rate with a preset anomaly rate threshold. If the execution anomaly rate of a rule exceeds the threshold for several consecutive statistical periods, the rule is considered to have stability issues in the current production environment and needs to be disabled.

[0109] In this embodiment, the specific process by which the runtime monitoring module calculates the execution anomaly rate is as follows: The monitoring module maintains a sliding window for each rule, with the length of the sliding window being a preset number of statistical periods. The length of each statistical period is a preset time. At the end of each statistical period, the monitoring module calculates the number of applications, execution failures, and latency anomalies for that rule within that period. Then, the monitoring module adds up the number of applications, execution failures, and latency anomalies across multiple statistical periods to obtain the total number of applications, the total number of execution failures, and the total number of latency anomalies. Finally, the monitoring module calculates the execution anomaly rate as the sum of the total number of execution failures and the total number of latency anomalies divided by the total number of applications. The advantage of using a sliding window to calculate the execution anomaly rate is that it can avoid false circuit breakers caused by occasional anomalies. If an occasional anomaly occurs in a certain statistical period, but there are no anomalies in other statistical periods, the average anomaly rate of the sliding window will be relatively low and will not trigger a circuit breaker. A circuit breaker will only be triggered when the anomaly rate exceeds the threshold for multiple consecutive statistical periods, thus improving the accuracy of the circuit breaker mechanism.

[0110] When a rule is disabled, the rewrite module will ignore it during subsequent request processing, no longer using it for request matching and rewriting. All requests that should have matched the rule will be passed through and processed as is, sent to the device driver for execution. This avoids further exceptions caused by rule issues and ensures the normal operation of the system's basic functions.

[0111] It's important to note that even after a rule is disabled, the monitoring module continues to record relevant information about that rule, including the time the exception occurred, the type of exception, the device state at the time of the exception, and the requested parameters. This information is saved to the kernel logs for technical personnel to analyze and troubleshoot later. After locating and fixing the rule issue, technical personnel can re-enable the rule through the sysfs interface or update it to the fixed rule via dynamic injection.

[0112] Furthermore, this application is not limited to this; the runtime monitoring and rule-based circuit breaker mechanism can also support tiered circuit breaking. For example, rules can be divided into three levels—normal, warning, and circuit breaker—based on different anomaly rates. When the anomaly rate exceeds the warning threshold but does not exceed the circuit breaker threshold, the rule enters the warning state. In this case, the rewrite module will reduce the application frequency of the rule or only apply the rule when the load is low. When the anomaly rate exceeds the circuit breaker threshold, the rule enters the circuit breaker state and is completely disabled. The tiered circuit breaker mechanism can maximize the performance improvement brought by rule rewriting while ensuring system stability.

[0113] In an optional implementation, the verified rewrite rule is dynamically injected from user space via eBPF mapping; after intercepting the ioctl request of the target character device, the method further includes: The eBPF program mounted at the hijacking point is invoked to classify and rewrite the request.

[0114] In this embodiment, it has been verified that the rewrite rules are dynamically injected from user space via eBPF mapping. After intercepting the ioctl request of the target character device, the process also includes invoking the eBPF program mounted at the hijacking point to classify the request and make rewrite decisions.

[0115] Specifically, eBPF is a technology that allows sandboxed programs to run within the Linux kernel. It allows user-space programs to inject custom code into the kernel, extending kernel functionality without modifying kernel source code or loading kernel modules. eBPF mapping is the mechanism for data exchange between eBPF programs and user-space programs; it's a key-value pair storage structure within the kernel that can be accessed by both user-space and eBPF programs simultaneously.

[0116] In this embodiment, using eBPF mapping to dynamically inject verified rewrite rules has the following advantages: First, it allows adding, deleting, or modifying verified rewrite rules in the kernel space without reloading kernel modules or restarting the system, greatly improving the flexibility and efficiency of rule updates. Second, access to eBPF mappings is secure; the kernel performs strict permission checks on user-space programs' access to eBPF mappings to prevent unauthorized access and malicious modification. Third, eBPF programs run in the kernel's sandbox environment, preventing kernel crashes; even if an eBPF program encounters an error, it only affects its own execution and does not impact the stability of the entire system.

[0117] In this embodiment, the dynamic injection process of verified rewrite rules is as follows: First, new verified rewrite rules are generated in user space, or existing rules are modified. Then, the user space program writes the new or modified rules into a pre-created eBPF mapping via an eBPF system call. The eBPF mapping is a hash mapping where the key is a unique identifier for the rule, and the value is the specific content of the rule, including applicable conditions, rewrite actions, and verification information. When performing rule matching, the kernel-space rewrite module iterates through all rules in the eBPF mapping, compares the requested features with the applicable conditions of the rules, and executes the corresponding rewrite action after finding a matching rule.

[0118] In this embodiment, the writing and loading process of the eBPF program is as follows: First, the source code of the eBPF program is written in C language, containing custom request classification and rewrite decision logic. Then, the C language source code is compiled into eBPF bytecode using the clang compiler and LLVM toolchain. During compilation, the target architecture needs to be specified as BPF, and the corresponding optimization options need to be enabled. Next, the user-space program loads the eBPF bytecode into the kernel via the BPF system call. The kernel verifies the eBPF bytecode to ensure that it is safe and will not cause the kernel to crash. After successful verification, the kernel allocates a file descriptor for the eBPF program. Finally, the user-space program mounts the eBPF program to the ioctl hijacking point via the BPF system call. After mounting, the kernel module automatically calls the eBPF program when it intercepts an ioctl request.

[0119] In this embodiment, the eBPF program can access the eBPF mapping and read and modify verified rewrite rules through functions such as bpf_map_lookup_elem and bpf_map_update_elem. The eBPF program can also obtain attribute information of the requesting process through auxiliary functions such as bpf_get_current_pid_tgid and bpf_get_current_cgroup_id, and obtain a pointer to the process's task_struct structure through the bpf_get_current_task function, thereby acquiring more process information.

[0120] Furthermore, this application embodiment also supports mounting an eBPF program at the hijacking point, whereby the eBPF program classifies and makes rewriting decisions for requests. The eBPF program can access the command code, parameters, and attribute information of the initiating process of the ioctl request, classify the request based on this information, and then decide whether to rewrite the request and which rewriting rule to apply. The advantage of mounting an eBPF program is that it allows for more flexible rewriting decision logic. Users can write custom eBPF programs according to their business needs to implement personalized request processing logic without modifying the kernel module code.

[0121] For example, a user can write an eBPF program that determines whether to merge requests based on the request size and the priority of the initiating process. Large requests are passed through directly. Small requests are merged if the initiating process has a low priority, and not merged if the initiating process has a high priority, to avoid increased latency due to waiting for merging. After the user attaches this eBPF program to the ioctl interception point, the kernel module, upon intercepting a request, will first call the eBPF program, which will then make a rewrite decision. The kernel module will then execute the corresponding operation based on the decision result.

[0122] It should be noted that mounting and unmounting eBPF applications can be done dynamically at runtime without requiring a system restart or reloading of kernel modules. Users can mount or unmount different eBPF applications at any time according to actual business needs, and adjust the request processing logic accordingly.

[0123] In this application embodiment, this application also provides a hardware request scheduling system based on kernel ioctl hijacking, such as... Figure 3 As shown, the system includes: The hijacking module is used to intercept ioctl requests from the target character device, obtain the process attribute information of the initiating call, and the command code and parameters of the request; The rewrite module is used to perform a rewrite operation on the parameters of the request according to the verified rewrite rule when the features of the request match the verified rewrite rule; otherwise, the request is passed through.

[0124] The queue and scheduling module is used to allocate the rewritten or transparent requests to the corresponding priority queues according to the process attribute information, and retrieve the requests from the queues in priority order, and call the original ioctl processing function of the target character device to send them to the device driver for execution.

[0125] The verified rewrite rules refer to the rules pre-determined through formal verification and differential fuzzing tests for rewriting ioctl requests securely and equivalently.

[0126] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0127] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A hardware request scheduling method based on kernel ioctl hijacking, characterized in that, include: Intercept the ioctl request of the target character device, and obtain the process attribute information of the initiating call, as well as the command code and parameters of the request; When the features of the request match the verified rewrite rule, the parameters of the request are rewritten according to the verified rewrite rule; otherwise, the request is passed through. The rewritten or passed-through request is allocated to the corresponding priority queue according to the process attribute information. The verified rewrite rule refers to the rule for rewriting ioctl requests securely and equivalently, which is determined in advance through formal verification and differential fuzzing. Request scheduling based on the priority queue includes: retrieving requests from the queue in priority order and calling the original ioctl processing function of the target character device to send the request to the device driver for execution.

2. The method according to claim 1, characterized in that, Before intercepting the ioctl request of the target character device, the method further includes: obtaining the verified rewrite rule, wherein obtaining the verified rewrite rule includes: Construct Hall logic preconditions and postconditions for the ioctl command of the target character device; The original ioctl sequence and candidate rewrite sequence are initiated to the device through differential fuzzy testing. The device state after execution is compared to see if the postcondition is met. If it is met, a verified rewrite rule is generated.

3. The method according to claim 1, characterized in that, When the features of the request match a verified rewrite rule, performing a rewrite operation on the parameters of the request according to the verified rewrite rule includes: When the command code and parameter value of the request meet the merging conditions specified in the verified rewrite rules, and there is a similar request that is consecutive to the request among the pending requests intercepted by the same process, the request is merged with the similar requests into a new aggregate ioctl request.

4. The method according to claim 3, characterized in that, After calling the original ioctl processing function of the target character device, the following is also included: The system receives the execution result of the aggregated ioctl request, splits the execution result into sub-results corresponding to each request before merging according to the verified rewrite rules, and wakes up the processes waiting for each sub-result.

5. The method according to claim 1, characterized in that, The process attribute information includes a process identifier or the control group identifier to which the process belongs; The step of allocating the rewritten or transparent requests to the corresponding priority queues based on the process attribute information includes: The process identifier or control group identifier is mapped to a preset priority level, and the request is inserted into the queue corresponding to that priority level.

6. The method according to claim 1, characterized in that, The request scheduling based on the priority queue includes: A kernel thread is created as a scheduler. The kernel thread retrieves requests from the priority queue in priority order and calls the original ioctl processing function using the context of the kernel thread.

7. The method according to claim 1, characterized in that, The request scheduling based on the priority queue includes: When the request from the initiating process is at the head of the queue, the original ioctl processing function is called directly using the kernel context of that process; otherwise, the process is blocked until its request becomes the head of the queue and is scheduled.

8. The method according to claim 1, characterized in that, Also includes: Runtime monitoring is performed on the applied and verified rewrite rules. When the anomaly rate of a request associated with a rule is detected to exceed the threshold, the rule is disabled and subsequent requests matching the rule are downgraded to pass-through processing.

9. The method according to claim 1, characterized in that, The verified rewrite rules are dynamically injected from user space via eBPF mapping; after intercepting the ioctl request of the target character device, the method further includes: The eBPF program mounted at the hijacking point is invoked to classify and rewrite the request.

10. A hardware request scheduling system based on kernel ioctl hijacking, characterized in that, include: The hijacking module is used to intercept ioctl requests from the target character device, obtain the process attribute information of the initiating call, and the command code and parameters of the request; The rewrite module is used to perform a rewrite operation on the parameters of the request according to the verified rewrite rule when the features of the request match the verified rewrite rule; otherwise, the request is passed through. The queue and scheduling module is used to allocate the rewritten or transparent requests to the corresponding priority queues according to the process attribute information, and retrieve the requests from the queues in priority order, and call the original ioctl processing function of the target character device to send them to the device driver for execution. The verified rewrite rules refer to the rules pre-determined through formal verification and differential fuzzing tests for rewriting ioctl requests securely and equivalently.

Images