An off-line mixed-traffic scheduling method based on ext scheduling class
By using the BPF callback function of the ext scheduling class in user space, task scheduling in offline and mixed deployment scenarios is realized, which solves the problems of kernel modification complexity and maintenance burden in existing technologies and improves the service quality and resource utilization of online tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- KYLIN CORP
- Filing Date
- 2026-03-19
- Publication Date
- 2026-07-31
AI Technical Summary
In offline and mixed deployment scenarios, existing technologies require deep and intrusive modifications to the kernel scheduling mechanism, resulting in high complexity, heavy maintenance burden, and the risk of kernel crashes. At the same time, they cannot effectively guarantee the service quality and resource utilization of online tasks.
We adopt an ext-based scheduling class approach and implement task scheduling strategies in user space through BPF callback functions. This eliminates the need for intrusive kernel modifications. By utilizing core selection, queuing, dispatching, and pre-task callback processes, we ensure the priority and resource utilization of online tasks.
It reduces system complexity and maintenance burden, improves the service quality of online tasks and system resource utilization, simplifies solution iteration and adaptation to upper-layer ecosystems, and avoids the risk of kernel crashes.
Smart Images
Figure CN122489217A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of task scheduling technology, and more specifically, to an online / offline hybrid scheduling method based on the ext scheduling class. Background Technology
[0002] In the context of hybrid deployment, the concept of "online service" and "offline job" refers to running two very different types of computing tasks ("online service" and "offline job") on the same physical machine in the same cluster in order to maximize resource utilization and reduce costs.
[0003] Online services are characterized by high interactivity, real-time performance, large traffic fluctuations, and extreme sensitivity to stability. Examples include web search, online payment, and short video playback. They require low latency and high availability, ensuring users experience no buffering or service interruptions. Offline jobs, on the other hand, are characterized by non-real-time processing, batch processing, computational intensity, insensitivity to short-term fluctuations, and a focus on high throughput. Examples include scientific computing, machine learning, and video transcoding and processing.
[0004] The biggest challenge in offline-to-offline hybrid deployment is resource contention. When an offline task is diligently performing calculations, an online service request suddenly arrives (e.g., a user suddenly searches). If the system doesn't manage this, the offline task might preempt the CPU, memory, and network I / O resources needed by the online task, causing the online service to slow down and impacting the user experience. Therefore, a sophisticated task scheduling mechanism is needed to ensure that online tasks always have the highest priority, and that when online tasks need resources, they can quickly suppress or even evict offline tasks.
[0005] Currently, the scheduling mechanism for online and offline mixed deployment tasks at the kernel level mainly involves modifying the kernel space based on the Fair Scheduler (CFS) class, or adding new scheduling classes to ensure the priority of online tasks. This approach requires deep and intrusive modifications to the kernel scheduling core, which not only introduces significant complexity and maintenance burden, but also increases the potential risk of kernel crashes, making it difficult to iterate the solution and adapt it to higher-level ecosystems. Summary of the Invention
[0006] The main objective of this invention is to provide a method for online / offline hybrid deployment scheduling based on the ext scheduling class. This method utilizes the BPF callback functions provided by the ext scheduling class to implement task scheduling strategies in user-space programming for online / offline hybrid deployment scenarios. This eliminates the need for intrusive kernel modifications, reducing complexity and maintenance burden, and effectively improving system resource utilization while ensuring the quality of online task services.
[0007] This invention provides an online / offline hybrid scheduling method based on the ext scheduling class, including a core selection process; the first CPU performs a core selection process for tasks based on a core selection callback function, and performs the following steps: Step S1: First, determine whether the CPU used in the last task execution was an idle CPU; If the CPU used to run the task in the previous run was an idle CPU, the task proceeds directly to step S3; if the CPU used to run the task in the previous run was not an idle CPU, the task proceeds to step S2. Step S2: Determine whether the task is marked as synchronous wake-up; If the task is marked as synchronous wake-up, then an idle CPU is selected from the system range starting with the first CPU, and then proceed to step S3; if the task is not marked as synchronous wake-up, then an idle CPU is selected from the system range starting with the CPU from which the task was last run, and then proceed to step S3. Step S3: Determine if the task can select an idle CPU; If the task can select an idle CPU, proceed to step S4; if the task cannot select an idle CPU, proceed to step S5. Step S4: Perform a hyper-threading logic check on the task; if the hyper-threading logic check passes, the task is directly enqueued into the first scheduling queue of the idle CPU; if the hyper-threading logic check fails, the task is treated as an un-enqueued task and the enqueueing process is executed. Step S5: The task performs a preemption logic check; if the preemption logic check passes, the task initiates preemption and enters the first scheduling queue of the CPU that previously ran the task; if the preemption logic check fails, the task is enqueued as an un-enqueued task.
[0008] Furthermore, the joining process includes the following steps: Step B1: Does the task have a CPU with which it is bound? If the task has a CPU with which it is bound, the task is directly enqueued into the first scheduling queue of the CPU with which it is bound; if the task does not have a CPU with which it is bound, proceed to step B2. Step B2: Determine whether the task has been screened through the selection process; if the task has not been screened through the selection process, proceed to step B3; if the task has been screened through the selection process, proceed to step B4. Step B3: Determine if the task can select an idle CPU within the range of the NUMA node where the CPU of the last running task was located; If the task selects an idle CPU within the range of the NUMA node where the CPU of the previous task was located, the task performs a hyper-threading logic check; if the hyper-threading logic check passes, the task enters the first scheduling queue of idle CPUs within the range of the NUMA node where the CPU of the previous task was located; if the hyper-threading logic check fails, proceed to step B4. If a task cannot select an idle CPU within the range of the NUMA node where the CPU of the previously running task is located, the task will perform a preemption logic check. If the preemption logic check passes, the task will enter the first scheduling queue of the CPU of the previously running task; if the preemption logic check fails, the task will proceed to step B4. Step B4: Determine if the task is an online task; if the task is an online task, enqueue it to the online task queue of the NUMA node where the CPU that last ran the task resides; if the task is not an online task, enqueue it to the offline task queue of the NUMA node where the CPU that last ran the task resides.
[0009] Furthermore, the preemption logic refers to: determining whether the task is an online task and whether the CPU that previously ran the task is currently running an offline task and whether there are online tasks queued in the first scheduling queue of the CPU that previously ran the task; if the task is an online task and the CPU that previously ran the task is currently running an offline task and there are no online tasks queued in the first scheduling queue of the CPU that previously ran the task, then the preemption logic's judgment passes; otherwise, the preemption logic's judgment fails.
[0010] Furthermore, it also includes the dispatch process and the callback process before task execution; after the enqueue process, the current CPU performs the task dispatch process and the callback process before task execution; the current CPU refers to the CPU that the task enters after the core selection process or the enqueue process.
[0011] Furthermore, the distribution process includes the following steps: Step C1: Obtain the first offline task in the offline task queue of the NUMA node where the current CPU is located; Step C2: Determine whether the unscheduled time of the first offline task has exceeded a set scheduling threshold; Step C3: If the unscheduled time of the first offline task exceeds the scheduling threshold, mark the offline_need_run flag of the first offline task and increase the scheduling priority of the first offline task so that the first offline task can be dispatched first. Step C4: If the unscheduled time of the first offline task does not exceed the scheduling threshold, then the suppression logic is used to determine the first offline task.
[0012] Furthermore, the judgment of the suppression logic refers to the following steps: Step E1: Determine whether the first scheduling queue of the current CPU and the online task queue of the NUMA node where the current CPU is located are both empty; if the first scheduling queue of the current CPU and the online task queue of the NUMA node where the current CPU is located are both empty, proceed to step E2; otherwise, proceed to step E3. Step E2: Determine whether the task currently running on the CPU is an online task and whether the task status of the task currently running on the CPU is marked as runnable. If yes, proceed to step E3; otherwise, dispatch the first offline task from the offline task queue of the NUMA node where the current CPU is located to the current CPU. Step E3: Continue dispatching the first online task in the online task scheduling queue of the NUMA node where the current CPU is located, or the current CPU continues to run online tasks.
[0013] Furthermore, the callback process before task execution follows these steps: Step F1: The task performs a hyper-threading logic check; if the hyper-threading logic check passes, the current CPU is allowed to run the task, and then proceed to step F2; if the hyper-threading logic check fails, the current CPU is not allowed to run the task, and proceed to step F3. Step F2: Determine if the task has the offline_need_run flag set; if the task has the offline_need_run flag set, clear the task's time slice to zero, and then proceed to step F3; Step F3: Set the task to be schedulable, trigger the current CPU to schedule, and switch to the next task; otherwise, end directly.
[0014] Furthermore, in step S4, the judgment of the hyper-threading logic refers to: Step D1: Does the system support hyper-threading? If the system does not support hyper-threading, the hyper-threading logic check passes; if the system supports hyper-threading, proceed to step D2. Step D2: Is the sibling CPU of the idle CPU currently running a task? If the sibling CPU is not running a task, the hyper-threading logic check passes; otherwise, proceed to step D3. Step D3: Determine whether the task is an online task and whether the task being run on the sibling CPU of the idle CPU is an offline task; if the task is an online task and the task being run on the sibling CPU is an offline task, the hyper-threading logic judgment passes; otherwise, proceed to step D4. Step D4: Determine whether the task is an offline task and whether the task being run on the sibling CPU of the idle CPU is an online task; if the task is an offline task and the task being run on the sibling CPU is an online task, then the hyper-threading logic check fails; otherwise, the hyper-threading logic check passes.
[0015] Furthermore, in step B4, after a task is enqueued into the online task queue of the NUMA node where the CPU of the previous task was located, or into the offline task queue of the NUMA node where the CPU of the previous task was located, it is then queued in the offline task queue or the online task queue based on the virtual weight virtual deadline calculation formula.
[0016] Furthermore, the virtual deadline calculation formula is: deadline = vruntime + wake_vtime; deadline refers to the virtual deadline; vruntime refers to the total cumulative running time of online or offline tasks; wake_vtime is the cumulative running time of online or offline tasks since they entered a sleep event after the last run.
[0017] The beneficial effects of this invention are as follows: 1. Implemented based on the ext scheduling class, it requires no intrusive modification to the kernel, reducing complexity and maintenance burden; 2. Customized scheduling strategies are more flexible, can be customized and deeply optimized according to business scenarios, can be quickly experimented and deployed, and the iteration of the solution and the adaptation to the upper-layer ecosystem become simple, and there is no risk of kernel crash. 3. While ensuring the quality of online task services, effectively improve system resource utilization and reduce business deployment costs.
[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort. Attached Figure Description
[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0020] Figure 1 This is a diagram of the architecture of the present invention; Figure 2 This is a schematic diagram of the core selection process of the present invention; Figure 3 This is a schematic diagram of the queuing process of the present invention; Figure 4 This is a schematic diagram of the distribution process of the present invention; Figure 5 This is a schematic diagram of the callback process before the task runs according to the present invention; Figure 6 This is a schematic diagram of the hyper-threading logic of the present invention. Detailed Implementation
[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Many specific details are set forth in the following description to provide a thorough understanding of the present invention; however, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0022] This invention proposes a hybrid online / offline deployment scheduling method based on the ext scheduling class. The method utilizes the BPF callback functions provided by the ext scheduling class to implement task scheduling strategies in user space programming for hybrid online / offline deployment scenarios. This eliminates the need for intrusive kernel modifications, reducing complexity and maintenance burden, and effectively improving system resource utilization while ensuring the quality of online task services.
[0023] Example 1: The user space reads the CPU topology logic to construct CPU hierarchy information and loads the kernel-space BPF module. The user space implementation uses Rust, but other languages such as C and C++ can also be used. The BPF module is implemented in C. The CPU hierarchy information includes NUMA nodes and the corresponding CPUs for those nodes.
[0024] Specifically, user-mode modules in user mode (corresponding to...) Figure 1 The user-space program reads the number of NUMA nodes in the system under the ` / sys / devices / system / node` path. It iterates through the CPUs within each NUMA node, reading the hierarchical structure information (CPU topology logic) under the ` / sys / devices / system / cpu / cpux / topology` path for each CPU. It records information such as the CPU's `thread_siblings` and `core_siblings`, and the corresponding NUMA node information. This information (CPU hierarchy information) is initialized into global variables and the BPF map in the BPF module. The user-space module communicates with the BPF module based on the BPF map.
[0025] 2. BPF module (corresponding to) Figure 1The bpf program in the program constructs a first scheduling queue (percpu scheduling queue) for each CPU; tasks include online tasks and offline tasks; the bpf module constructs an online task scheduling queue and an offline task scheduling queue for each NUMA node.
[0026] Specifically, when the BPF module is loaded, it iterates through all CPUs in the system and creates a per-CPU scheduling queue for each CPU: bpf_for(cpu, 0, nr_cpu_ids) { scx_bpf_create_dsq(cpu, scx_bpf_cpu_node(cpu)); } Create an online task scheduling queue and an offline task scheduling queue for each numa node: bpf_for(node, 0, nr_nodes) { scx_bpf_create_dsq(DSQ_ONLINE_ID | node, node); scx_bpf_create_dsq(DSQ_OFFLINE_ID | node, node); } If the NUMA node where an online or offline task last ran has an idle CPU, it can be assigned to the first scheduling queue of that idle CPU. If the NUMA node does not have an idle CPU, but other NUMA nodes across the system have idle CPUs, both online and offline tasks can be assigned to the first scheduling queue of the corresponding idle CPU. If there are no idle CPUs across the system, online tasks can enter the online task scheduling queue of the NUMA node, and offline tasks can enter the offline task scheduling queue of the NUMA node. The CPU prioritizes tasks from the first scheduling queue. If the first scheduling queue is empty, it continues to select tasks from the online task scheduling queue of the corresponding NUMA node, and finally selects tasks from the offline task scheduling queue of the corresponding NUMA node.
[0027] An online / offline hybrid scheduling method based on the ext scheduling class includes: core selection process, enqueue process, dispatch process, and callback process before task execution.
[0028] After a task is created, the first CPU (this_cpu) performs a core selection process for the task to ensure it can be placed on an idle CPU. In this embodiment, the first CPU refers to the CPU currently selecting a core for the task. If an idle CPU cannot be selected for the task, the task enters the enqueue process as an unqueued task to ensure it can be enqueued into a suitable queue. After the enqueue process is completed, the task waits to be dispatched. A suitable task is selected for dispatch within each CPU's dispatch process, and after the dispatch process is completed, the callback process before the task execution is executed.
[0029] like Figure 2 As shown, the first CPU performs the core selection process for the task based on the core selection callback function, and proceeds with the following steps: Step S1: First, determine whether the CPU on which the task was last run (the CPU on which the task was last run may contain hot cached data for the task, which can improve the task execution performance; therefore, the CPU on which the task was last run is the optimal CPU for the task) is an idle CPU. If the CPU (prev_cpu) that ran the task last time was an idle CPU, the task proceeds directly to step S3; if the CPU that ran the task last time was not an idle CPU, the task proceeds to step S2. Step S2: Determine whether the task is marked as synchronous wake-up (the type of synchronous wake-up is SCX_WAKE_SYNC); If the task is marked for synchronous wake-up, then an idle CPU is selected from the system range starting with the first CPU, and then proceed to step S3; if the task is not marked for synchronous wake-up, then an idle CPU is selected from the system range starting with the CPU from which the task was last run, and then proceed to step S3.
[0030] The process of selecting an idle CPU within the system, starting with the first CPU, involves the following steps: First, determine if the first CPU is idle. If it is, the task enters the task scheduling queue of the first CPU. If the first CPU is not idle, check if there is an idle CPU within the range of the NMAA node where the first CPU resides. If so, the task enters the first scheduling queue of idle CPUs within the range of the NMAA node where the first CPU resides. If there is no idle CPU within the range of the NMAA node where the first CPU resides, search for an idle CPU in other ranges of the system. If an idle CPU is found in other ranges of the system, the task enters the first scheduling queue of idle CPUs in other ranges of the system. If no idle CPU is found in other ranges of the system, the task cannot find an idle CPU within the system.
[0031] The process of selecting idle CPUs within the system, starting from the CPU of the last running task, involves: first determining if there are any idle CPUs within the range of the NMAA node where the CPU of the last running task resides; if so, entering the first scheduling queue of idle CPUs within the range of the NMAA node where the CPU of the last running task resides; if not, searching for idle CPUs in other ranges within the system. If an idle CPU is found in other ranges within the system, entering the first scheduling queue of idle CPUs in other ranges within the system; if no idle CPU is found in other ranges within the system, the task cannot find an idle CPU within the system.
[0032] Step S3: Determine if the task can select an idle CPU; If the task can select an idle CPU, proceed to step S4; if the task cannot select an idle CPU, proceed to step S5. Step S4: Perform a hyper-threading logic check on the task; if the hyper-threading logic check passes, the task is directly enqueued into the first scheduling queue of the idle CPU; if the hyper-threading logic check fails, the task is treated as an un-enqueued task and the enqueueing process is executed. Step S5: The task performs a preemption logic check; if the preemption logic check passes, the task initiates preemption and enters the first scheduling queue of the CPU that previously ran the task; if the preemption logic check fails, the task is enqueued as an un-enqueued task.
[0033] like Figure 3 As shown, the enqueue process, based on the enqueue callback function, involves the following steps: Step B1: Does the task have a CPU with which it is bound? If the task has a CPU with which it is bound, the task is directly enqueued into the first scheduling queue of the CPU with which it is bound; if the task does not have a CPU with which it is bound, proceed to step B2. Step B2: Determine whether the task has been screened through the selection process; if the task has not been screened through the selection process, proceed to step B3; if the task has been screened through the selection process, proceed to step B4.
[0034] The reason for proceeding to step B2 is that tasks may involve migration, relinquishing CPU time slices and re-enqueuing, modifying task attributes, or being preempted by higher-priority tasks. In these scenarios, tasks are directly enqueued without going through the core selection phase. Therefore, in these scenarios, it's also necessary to consider finding suitable CPUs for task core selection.
[0035] Step B3: Determine if the task can select an idle CPU within the range of the NUMA node where the CPU of the last running task was located; If the task selects an idle CPU within the range of the NUMA node where the CPU of the previous task was located, the task performs a hyper-threading logic check; if the hyper-threading logic check passes, the task enters the first scheduling queue of idle CPUs within the range of the NUMA node where the CPU of the previous task was located; if the hyper-threading logic check fails, proceed to step B4. If a task cannot select an idle CPU within the range of the NUMA node where the CPU of the previously running task is located, the task will perform a preemption logic check. If the preemption logic check passes, the task will initiate preemption and enter the first scheduling queue of the CPU of the previously running task; if the preemption logic check fails, proceed to step B4. Step B4: Determine if the task is an online task; if the task is an online task, enqueue it to the online task queue of the NUMA node where the CPU that last ran the task resides; if the task is not an online task, enqueue it to the offline task queue of the NUMA node where the CPU that last ran the task resides.
[0036] In the core selection and queuing process, the preemption logic is as follows: determine whether the task is an online task and whether the CPU that previously ran the task is currently running an offline task and whether the first scheduling queue of the CPU that previously ran the task has an online task in queue; if the task is an online task and the CPU that previously ran the task is currently running an offline task and the first scheduling queue of the CPU that previously ran the task has no online task in queue, then the preemption logic passes the judgment; otherwise, the preemption logic fails the judgment.
[0037] If the preemption logic passes the check, the task immediately initiates preemption by calling `scx_bpf_kick_cpu(prev_cpu, SCX_KICK_PREEMPT)`. Then, `prev_cpu` (the CPU that previously ran the task) removes the currently running offline task, thus yielding `prev_cpu` to the preempting task. Specifically, `prev_cpu` resets the time slice of the currently running offline task and sets a schedulable flag. When the schedulable flag arrives, the currently running offline task yields `prev_cpu` and checks if there are any idle CPUs within the NUMA node containing `prev_cpu`. If there are idle CPUs, the offline task that yielded `prev_cpu` is enqueued into the first scheduling queue of the idle CPU; otherwise, the offline task that yielded `prev_cpu` is enqueued into the offline scheduling queue within the NUMA node containing `prev_cpu`.
[0038] After a task completes its enqueueing process, the CPU where the task was enqueued is called the current CPU, which then performs the task dispatch process and the subsequent pre-run callback process. The pre-run callback process is the callback procedure performed based on the task's running function after task dispatch and before execution.
[0039] like Figure 4 As shown, the dispatch process refers to the priority logic judgment based on the task dispatch callback function. When an offline task is preempted, there is a situation where the offline task has exclusive access to shared resources in kernel mode. Because all CPUs are 100% occupied by online tasks, it is "absolutely suppressed," preventing it from running and releasing shared resources. If an online task also needs to access the shared resource, it will be forced to wait for a low-priority task that will never release it, thus causing a priority inversion problem. Therefore, priority logic judgment is required in the dispatch process.
[0040] The priority logic is determined by the following steps: Step C1: Get the first offline task in the offline task queue of the NUMA node where the current CPU is located; the current CPU refers to the CPU that is currently dispatching the task, which can be the first CPU, the prev_cpu, or other CPUs, depending on which CPU in the system the task was enqueued on.
[0041] Step C2: Determine whether the unscheduled time of the first offline task has exceeded a set scheduling threshold; Step C3: If the unscheduled time of the first offline task exceeds the scheduling threshold, mark the offline_need_run flag of the first offline task and increase the scheduling priority of the first offline task so that the first offline task can be dispatched first. Step C4: If the unscheduled time of the first offline task does not exceed the scheduling threshold, then the suppression logic is used to determine the first offline task.
[0042] Suppression logic refers to performing the following steps: Step E1: Determine whether the first scheduling queue of the current CPU and the online task queue of the NUMA node where the current CPU is located are both empty; if the first scheduling queue of the current CPU and the online task queue of the NUMA node where the current CPU is located are both empty, proceed to step E2; otherwise, proceed to step E3. Step E2: Determine whether the task currently running on the CPU is an online task and whether the status of the task currently running on the CPU is SCX_TASK_QUEUED (runnable status flag). If yes, proceed to step E3; otherwise, dispatch the first offline task of the offline task queue of the NUMA node where the current CPU is located to the current CPU. Step E3: Continue dispatching the first online task in the online task scheduling queue or continue running online tasks on the current CPU.
[0043] After dispatching is complete, the current CPU performs the callback process before task execution based on the task running callback function, executing the following steps, such as... Figure 5 As shown: Step F1: The task performs a hyper-threading logic check; if the hyper-threading logic check passes, the current CPU is allowed to run the task, and then proceed to step F2; if the hyper-threading logic check fails, the current CPU is not allowed to run the task, and proceed to step F3. Step F2: Determine whether the task has the offline_need_run flag set; if the task has the offline_need_run flag set, then clear the task's time slice (clearing the task's time slice gives the task the opportunity to run in kernel mode, which can release shared resources), and then proceed to step F3; Step F3: Set the task to be schedulable, trigger the current CPU to schedule, and switch to the next task; otherwise, end directly.
[0044] Hyper-threading logic is involved in the core selection process, the queuing process, and the callback process before task execution.
[0045] The hyper-threading logic determines whether the online task exclusively uses the SMT hyper-thread pair and evicts the offline task. Specifically, in step S4, the hyper-threading logic performs the following steps: Figure 6 As shown: Step G1: Does the system support hyper-threading? If the system does not support hyper-threading, the hyper-threading logic check passes; if the system supports hyper-threading, proceed to step G2. Step G2: Is the sibling CPU of the idle CPU currently running a task? If the sibling CPU is not running a task, the hyper-threading logic check passes; otherwise, proceed to step G3. Step G3: Determine whether the task is an online task and whether the task being run by the sibling CPU of the idle CPU is an offline task; if the task is an online task and the task being run by the sibling CPU is an offline task, the hyper-threading logic judgment passes, and the sibling CPU initiates preemption to drive away the running offline task. After the drive-away is completed, the sibling CPU selects an online task. If it cannot select an online task, the sibling CPU becomes an idle CPU; otherwise, proceed to step G4. Step G4: Determine whether the task is an offline task and whether the task currently running on the sibling CPU of the idle CPU is an online task; if the task is an offline task and the task currently running on the sibling CPU is an online task, the hyper-threading logic fails the judgment and the task is not allowed to enter the first scheduling queue of the idle CPU; otherwise, the hyper-threading logic passes the judgment.
[0046] It should be noted that if the judgment of the hyper-threading logic of the task in step S4 is based on the fact that the CPU of the previous task was an idle CPU, then the idle CPU in the judgment of the hyper-threading logic refers to the CPU of the previous task.
[0047] If the judgment of the hyper-threading logic of the task in step S4 is based on the first CPU being an idle CPU, then the idle CPU in the judgment of the hyper-threading logic refers to the first CPU.
[0048] If the judgment of the hyper-threading logic of the task in step S4 is based on the idle CPUs within the range of the numa node where the first CPU is located, then the idle CPUs in the judgment of the hyper-threading logic refer to the idle CPUs within the range of the numa node where the first CPU is located.
[0049] If the judgment of the hyper-threading logic of the task in step S4 is based on the idle CPUs within the range of the numa node where the CPU of the previous running task is located, then the idle CPUs in the judgment of the hyper-threading logic refer to the idle CPUs within the range of the numa node where the CPU of the previous running task is located.
[0050] If the judgment of the hyper-threading logic of the task in step S4 is based on the idle CPUs in other ranges of the system, then the idle CPUs in the judgment of the hyper-threading logic refer to the idle CPUs in other ranges of the system.
[0051] Example 2: In this embodiment, the BPF module creates a high-precision timer for the NUMA nodes. This high-precision timer is used to calculate the average CPU utilization of all NUMA nodes. The timing period of the high-precision timer can be set to 1 second (which can be modified based on actual needs). When the timing period arrives, the high-precision timer iterates through all NUMA nodes, updates the utilization of each NUMA node, and marks busy or idle NUMA nodes.
[0052] If the average CPU utilization of a numa node is below the lower utilization threshold (25%), then the numa node is marked as an idle numa node. If the average CPU utilization of a numa node is above the upper utilization threshold (75%) and there are other idle numa nodes, then the numa node with the average CPU utilization above the upper utilization threshold is marked as a busy numa node, and the tasks in the busy numa node are evenly distributed to the idle numa nodes. During task core selection and enqueue callback, if no idle CPU is found in the busy numa node, then other numa nodes are traversed, the numa node with the lowest average utilization is found, and the task is enqueued to the numa node with the lowest average utilization.
[0053] Example 3: In this embodiment, tasks are assigned to queues (first scheduling queue, online task scheduling queue, and offline task scheduling queue) based on weighted virtual deadlines.
[0054] The formula for calculating the virtual deadline is: deadline = vruntime + wake_vtime; Where: vruntime: the total cumulative running time of the task. vruntime is scaled inversely based on the task's weight (priority) to ensure fairness; wake_vtime: The cumulative runtime since the task last sleep. wake_vtime is also scaled inversely based on the task's weight. Deadline: The virtual deadline. A lower limit needs to be set for the deadline to prevent tasks that sleep for extended periods from gaining excessive priority. This lower limit is configured by the user-space module as a slice_lag value based on the task's weight. The slice_lag value is scaled based on the task's wake-up frequency; tasks with higher wake-up frequencies have lower deadline lower limits. Interactive tasks typically feature frequent sleeps and short execution times, resulting in smaller slice_lag values and thus smaller virtual deadlines.
[0055] Example 4: This embodiment provides an online / offline hybrid scheduling system based on the ext scheduling class. The system includes kernel mode and user mode, such as... Figure 1 As shown.
[0056] User space includes user space modules and interface modules. Users can access the interface module (corresponding to...). Figure 1The cpu.qos_state interface file (the cgroup CPU subsystem interface file in the kernel) configures tasks in the cgroup CPU subsystem in kernel space as online tasks (business) or offline tasks (business). The user-space module is used to read the CPU topology logic to construct the CPU hierarchy, load the BPF program, and communicate with the BPF module through the BPF map.
[0057] Kernel mode includes the bpf module (corresponding to...) Figure 1 (BPF program), cgroup CPU subsystem, scheduling core.
[0058] The BPF module is used to construct the first scheduling queue, the online task scheduling queue, and the offline task scheduling queue, as well as to execute the first, second, and third scheduling policies. The BPF module can also set high-precision timers for load balancing scheduling of NUMA nodes and prioritize tasks based on virtual deadlines.
[0059] The bpf module identifies the task type by reading the qos_state member variable of the task group (process group) in the cgroup cpu subsystem, thereby implementing different scheduling strategies.
[0060] The scheduling core includes the ext scheduling class, an extensible scheduling class built on BPF. It innovatively shifts the implementation of scheduling strategies from the kernel to user space, allowing for dynamic definition of scheduling behavior by loading BPF programs, and providing highly customizable capabilities based on business needs. The ext scheduling class provides a series of callback functions, including core selection callback functions, enqueue callback functions, task dispatch callback functions, and task running callback functions.
[0061] Abbreviations: SMT (Simultaneous Multithreading): is a processor hardware design technique that simulates multiple logical cores (usually two) within a single physical processor core, enabling that physical core to concurrently execute instructions from multiple threads within the same clock cycle.
[0062] NUMA (Non-Conformal Memory Access Architecture): A multiprocessor computer architecture. Its characteristics include each NUMA node having local memory with extremely fast access speeds, while also being able to access the remote memory of other NUMA nodes, albeit at a slower speed.
[0063] BPF (Berkeley Packet Filter): A revolutionary kernel technology originating from network filtering. It allows user-mode programs to securely inject bytecode into the kernel without recompiling the kernel or loading kernel modules, thereby efficiently and securely extending kernel functionality. It is now widely used in various fields such as tracing, observation, networking, and security.
[0064] CFS (Completely Fair Scheduler): The default CPU scheduler in the Linux kernel. Its core goal is to allocate CPU time slices fairly to all processes through the calculation of virtual runtime (vruntime).
[0065] cgroups (Control Groups) are a resource management mechanism provided by the Linux kernel. Subgroups are the core components that implement specific resource control capabilities (such as CPU, memory, disk I / O, etc.). Each subgroup is responsible for limiting, counting, or isolating a type of resource. It acts like a "resource allocation manager" for the system, allowing you to finely control the sharing of resources by multiple processes. Figure 1 In this context, the cgroup cpu subsystem is a component that manages CPU resources.
[0066] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
Claims
1. An off-line mixed-traffic scheduling method based on ext scheduling class, characterized in that, This includes the core selection process; the first CPU performs the core selection process for the task based on the core selection callback function, and the following steps are taken: Step S1: First, determine whether the CPU used in the last task execution was an idle CPU; If the CPU used to run the task in the previous run was an idle CPU, the task proceeds directly to step S3; if the CPU used to run the task in the previous run was not an idle CPU, the task proceeds to step S2. Step S2: Determine whether the task is marked as synchronous wake-up; If the task is marked as synchronous wake-up, then an idle CPU is selected from the system range starting with the first CPU, and then proceed to step S3; if the task is not marked as synchronous wake-up, then an idle CPU is selected from the system range starting with the CPU from which the task was last run, and then proceed to step S3. Step S3: Determine if the task can select an idle CPU; If the task can select an idle CPU, proceed to step S4; if the task cannot select an idle CPU, proceed to step S5. Step S4: Perform a hyper-threading logic check on the task; if the hyper-threading logic check passes, the task is directly enqueued into the first scheduling queue of the idle CPU; if the hyper-threading logic check fails, the task is treated as an un-enqueued task and the enqueueing process is executed. Step S5: The task performs a preemption logic check; if the preemption logic check passes, the task initiates preemption and enters the first scheduling queue of the CPU that previously ran the task; if the preemption logic check fails, the task is enqueued as an un-enqueued task.
2. The off-line hybrid scheduling method based on ext scheduling class according to claim 1, wherein, The joining process includes the following steps: Step B1: Does the task have a CPU with which it is bound? If the task has a CPU with which it is bound, the task is directly enqueued into the first scheduling queue of the CPU with which it is bound; if the task does not have a CPU with which it is bound, proceed to step B2. Step B2: Determine whether the task has been screened through the selection process; if the task has not been screened through the selection process, proceed to step B3; if the task has been screened through the selection process, proceed to step B4. Step B3: Determine if the task can select an idle CPU within the range of the NUMA node where the CPU of the last running task was located; If the task selects an idle CPU within the range of the NUMA node where the CPU of the previous task was located, the task performs a hyper-threading logic check; if the hyper-threading logic check passes, the task enters the first scheduling queue of idle CPUs within the range of the NUMA node where the CPU of the previous task was located; if the hyper-threading logic check fails, proceed to step B4. If a task cannot select an idle CPU within the range of the NUMA node where the CPU of the previously running task is located, the task will perform a preemption logic check. If the preemption logic check passes, the task will enter the first scheduling queue of the CPU of the previously running task; if the preemption logic check fails, the task will proceed to step B4. Step B4: Determine if the task is an online task; if the task is an online task, enqueue it to the online task queue of the NUMA node where the CPU that last ran the task resides; if the task is not an online task, enqueue it to the offline task queue of the NUMA node where the CPU that last ran the task resides.
3. The online / offline hybrid scheduling method based on the ext scheduling class according to claim 1, characterized in that, The preemption logic refers to: determining whether the task is an online task and whether the CPU that previously ran the task is currently running an offline task, and whether the first scheduling queue of the CPU that previously ran the task has an online task in the queue; If the task is an online task and the CPU that previously ran the task is currently running an offline task and there are no online tasks queued in the first scheduling queue of the CPU that previously ran the task, then the preemption logic will pass the test; otherwise, the preemption logic will fail the test.
4. The online / offline hybrid scheduling method based on the ext scheduling class according to claim 2, characterized in that, It also includes the dispatch process and the callback process before the task runs; after the enqueue process, the current CPU performs the task dispatch process and the callback process before the task runs; the current CPU refers to the CPU that the task enters after the core selection process or the enqueue process.
5. The online / offline hybrid scheduling method based on the ext scheduling class according to claim 4, characterized in that, The distribution process includes the following steps: Step C1: Obtain the first offline task in the offline task queue of the NUMA node where the current CPU is located; Step C2: Determine whether the unscheduled time of the first offline task has exceeded a set scheduling threshold; Step C3: If the unscheduled time of the first offline task exceeds the scheduling threshold, mark the offline_need_run flag of the first offline task and increase the scheduling priority of the first offline task so that the first offline task can be dispatched first. Step C4: If the unscheduled time of the first offline task does not exceed the scheduling threshold, then the suppression logic is used to determine the first offline task.
6. The online / offline hybrid scheduling method based on the ext scheduling class according to claim 5, characterized in that, The suppression logic involves the following steps: Step E1: Determine whether the first scheduling queue of the current CPU and the online task queue of the NUMA node where the current CPU is located are both empty; if the first scheduling queue of the current CPU and the online task queue of the NUMA node where the current CPU is located are both empty, proceed to step E2. Otherwise proceed to step E3; Step E2: Determine whether the task currently running on the CPU is an online task and whether the task status of the task currently running on the CPU is marked as runnable. If yes, proceed to step E3; otherwise, dispatch the first offline task from the offline task queue of the NUMA node where the current CPU is located to the current CPU. Step E3: Continue dispatching the first online task in the online task scheduling queue of the NUMA node where the current CPU is located, or the current CPU continues to run online tasks.
7. The online / offline hybrid scheduling method based on the ext scheduling class according to claim 6, characterized in that, The callback process before task execution executes the following steps: Step F1: The task performs a hyper-threading logic check; if the hyper-threading logic check of the task passes, the current CPU is allowed to run the task, and then proceed to step F2. If the hyper-threading logic of the task fails the check, the current CPU is not allowed to run the task, and the process proceeds to step F3. Step F2: Determine if the task has the offline_need_run flag set; If the task has the offline_need_run flag set, then the task's time slice is cleared to zero, and then proceed to step F3; Step F3: Set the task to be schedulable, trigger the current CPU to schedule, and switch to the next task; otherwise, end directly.
8. The online / offline hybrid scheduling method based on the ext scheduling class according to claim 1, characterized in that, In step S4, the judgment of the hyper-threading logic refers to: Step D1: Does the system support hyper-threading? If the system does not support hyper-threading, the hyper-threading logic check passes; if the system supports hyper-threading, proceed to step D2. Step D2: Is the sibling CPU of the idle CPU currently running a task? If the sibling CPU of the idle CPU is not running a task, the hyper-threading logic check passes; otherwise, proceed to step D3. Step D3: Determine whether the task is an online task and whether the task being run on the idle CPU's sibling CPU is an offline task; if the task is an online task and the task being run on the idle CPU's sibling CPU is an offline task, then the hyper-threading logic judgment passes; otherwise, proceed to step D4. Step D4: Determine whether the task is an offline task and whether the task being run on the idle CPU's sibling CPU is an online task; if the task is an offline task and the task being run on the idle CPU's sibling CPU is an online task, then the hyper-threading logic check fails; otherwise, the hyper-threading logic check passes.
9. The online / offline hybrid scheduling method based on the ext scheduling class according to claim 1, characterized in that, In step B4, after a task is enqueued into the online task queue of the NUMA node where the CPU of the previous task was located, or into the offline task queue of the NUMA node where the CPU of the previous task was located, it is queued in the offline task queue or the online task queue based on the virtual weight virtual deadline calculation formula. The formula for calculating the virtual deadline is: deadline = vruntime + wake_vtime; deadline refers to the virtual deadline; vruntime refers to the total cumulative running time of online or offline tasks; wake_vtime is the cumulative running time of online or offline tasks since they entered a sleep event after the last run.
10. The online / offline hybrid scheduling method based on the ext scheduling class according to claim 4, characterized in that, During the task enqueueing process, core selection process, dispatch process, and callback process before task execution, a high-precision timer is used to measure the average CPU utilization of all numa nodes. If the average CPU utilization of a numa node is lower than the lower utilization threshold, the numa node is marked as an idle numa node. If the average CPU utilization of a numa node is higher than the upper utilization threshold and there are idle numa nodes, the numa nodes with average CPU utilization higher than the upper utilization threshold are marked as busy numa nodes, and the tasks in the busy numa nodes are evenly distributed to the idle numa nodes.