Dynamic thread scheduling and management method based on real-time resource feedback

Abstract

The invention provides a thread dynamic scheduling and management method based on real-time resource feedback, which belongs to the field of operating system resource scheduling, and comprises the following steps of: acquiring resource utilization rate, load queue length and thread delay data of a CPU (Central Processing Unit), a memory and a hardware accelerator in real time, and combining task type (calculation / I / O (Input / Output) intensive type), priority and deadline characteristics; constructing a dynamic scheduling strategy: firstly, distributing an initial priority and a time slice by adopting an improved multi-level feedback queue algorithm, secondly, optimizing a resource matching degree through a particle swarm algorithm, and dynamically adjusting a thread priority and a migration strategy according to real-time resource fluctuation in task execution; and finally, realizing cross-processor load balancing based on a minimum residual load migration principle and flow table type queue management. According to the method, through multi-dimensional resource feedback closed-loop control, the problems of non-uniform resource utilization, long-range starvation and low cooperation efficiency of heterogeneous equipment in traditional scheduling are solved.

Description

Technical Field

[0001] This invention relates to the fields of computer operating systems, embedded real-time systems, and high-performance computing, and is particularly applicable to the dynamic scheduling and resource management of threads in multi-core processor and heterogeneous computing (such as GPU / NPU) environments. Specifically, it is a thread dynamic scheduling and management method based on real-time resource feedback. Background Technology

[0002] Current computer systems face the following core scheduling problems in high-concurrency, heterogeneous computing scenarios:

[0003] 1. Insufficient dynamic adaptability of resources:

[0004] Static scheduling policy refers to a scheduling method that allocates time slices, priorities, or binds compute nodes to tasks before execution (such as Linux's default CFS Completely Fair Scheduler). It cannot dynamically adjust based on runtime resource fluctuations (such as sudden CPU loads or insufficient GPU memory), leading to uneven resource utilization (e.g., some CPU cores have an idle rate exceeding 50%, while other cores remain fully loaded).

[0005] Long thread starvation: In a multilevel feedback queue (MLFQ), computationally intensive long threads are frequently preempted by short tasks and remain in the low-priority queue for a long time, ultimately failing to complete within the deadline.

[0006] 2. Low efficiency in heterogeneous resource collaboration:

[0007] Heterogeneous resources refer to computing units (such as CPUs, GPUs, NPUs, and FPGAs) that coexist in a system, with significantly different architectural characteristics (CPUs are suitable for logic control, GPUs are suitable for parallel computing, and NPUs focus on AI inference). Existing schedulers typically treat different devices as independent resource pools, lacking cross-device coordination strategies, resulting in uneven load distribution among devices (e.g., CPUs are already overloaded when GPU utilization is only 30%).

[0008] Examples of defects in existing solutions

[0009] 1. Example of Static Scheduling Strategy – Linux CFS Scheduler

[0010] It employs a completely fair virtual time slice (vruntime) allocation, without distinguishing between compute-intensive and I / O-intensive task characteristics.

[0011] Fixed time slices cause I / O tasks to frequently yield the CPU, while computational tasks occupy cores for extended periods, exacerbating context switching overhead (actually increasing switching frequency by 40%). When a sudden influx of high-priority tasks arrives, CFS cannot quickly preempt low-priority long threads, leading to timeouts and delays in critical task responses.

[0012] 2. Defects of the Multilevel Feedback Queue (MLFQ) Algorithm

[0013] Traditional MLFQ uses a fixed priority scaling rule (e.g., decreasing by one level after each time slice), causing long threads to be continuously downgraded due to multiple executions. Without dynamic adjustment based on task deadlines, continuous preemption by short tasks leads to uncontrollable completion times for long threads. For example, in tests on an 8-core server, the average latency for long threads in traditional MLFQ reached 320ms, while short tasks only took 28ms, a fairness difference of over 10 times.

[0014] 3. Deficiencies in heterogeneous resource scheduling schemes – Kubernetes default scheduler

[0015] Node selection is based solely on CPU / Memory resource declarations, without considering the actual computing power status of the GPU / NPU (such as Tensor Core utilization). There is a lack of inter-device task migration mechanisms; once GPU tasks are bound to nodes, they cannot be dynamically adjusted according to load changes. When a GPU node runs out of memory, new tasks queue up instead of migrating to other idle GPU nodes, resulting in a resource utilization loss of up to 65%. Summary of the Invention

[0016] To address the problems of poor adaptability of static strategies, single resource monitoring dimensions, susceptibility to starvation of long threads, and insufficient coordination among heterogeneous devices in existing thread scheduling methods, this invention provides a dynamic thread scheduling and management method based on real-time resource feedback. Through task feature modeling, dynamic priority adjustment, and elastic load migration strategies, it achieves coordinated optimization of resource utilization and task response efficiency in high-concurrency scenarios.

[0017] The technical solution of this invention is:

[0018] A thread dynamic scheduling and management method based on real-time resource feedback is proposed. By collecting real-time data on CPU, memory and hardware accelerator resource utilization, load queue length and thread latency, and combining task type (i.e., computation / I / O intensive), priority and deadline characteristics, a dynamic scheduling strategy is constructed.

[0019] Furthermore,

[0020] First, an improved multi-level feedback queue algorithm is used to allocate initial priorities and time slices. Second, the resource matching degree is optimized through particle swarm optimization. During task execution, thread priorities and migration strategies are dynamically adjusted according to real-time resource fluctuations. Finally, cross-processor load balancing is achieved based on the minimum remaining load migration principle and stream table queue management.

[0021] Furthermore,

[0022] Includes the following steps:

[0023] The resource monitoring agent periodically collects real-time utilization data of CPU cores, memory, hardware accelerators, load queue length, and thread execution latency. The sampling period can be configured from 10ms to 1s.

[0024] The thread task type is classified based on the support vector machine model, and task feature vectors containing priority, deadline, and resource requirement matrix are generated.

[0025] A dynamic scheduling algorithm that integrates multi-level feedback queues and particle swarm optimization is adopted. An initial scheduling strategy is generated based on the current resource status and task feature vectors. The objective function of particle swarm optimization is to minimize the weighted sum of resource idle rate and task deadline default rate.

[0026] During task execution, when the utilization rate of any resource dimension exceeds the preset threshold (e.g., CPU > 85%), a feedback adjustment mechanism is triggered: dynamically adjusting thread priority weights, reallocating time slices, and migrating low-remaining-load threads to idle computing nodes through flow table matching.

[0027] A dual-queue buffering mechanism is used to handle migration tasks. During the migration process, the original queue context is retained until the target node confirms receipt, thus avoiding task interruption.

[0028] in,

[0029] The priority calculation in the dynamic scheduling algorithm uses the following formula:

[0030]

[0031] Where Tremaining is the reciprocal of the remaining deadline, Sresource-match is the cosine similarity between the task's resource requirements and the node's available resources, Hstarvation is the penalty for the number of times the task has been preempted in the past, and Rcurrent-load and Ddependency are the current node load and task dependency, respectively.

[0032] The hardware accelerators include GPUs / NPUs / FPGAs, and their resource status is monitored by obtaining computing power utilization, video memory usage, and task queue depth through the device driver layer API.

[0033] The flow table matching uses a five-tuple rule (source node, target resource type, minimum remaining load, task tolerance latency, and device compatibility) to filter migration target nodes.

[0034] The beneficial effects of this invention are:

[0035] This invention achieves deep integration of resource status awareness, dynamic priority adjustment, and cross-device load balancing through a closed-loop feedback mechanism, a hybrid scheduling algorithm (MLFQ+PSO), and a heterogeneous resource collaboration model. Simultaneously, it supports elastic scaling from edge computing nodes to cloud servers, providing an efficient scheduling solution for scenarios such as AI inference and real-time data processing. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the overall architecture of the present invention;

[0037] Figure 2 This is a schematic diagram of the overall process of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0039] This invention discloses a thread dynamic scheduling and management method based on real-time resource feedback. It uses a multi-level storage architecture as its core foundation, combined with intelligent tiering strategies and access optimization mechanisms, to form a highly efficient and collaborative system. Hot, warm, and cold data layers are responsible for data storage and scheduling; the intelligent tiering strategy dynamically manages data location; the access optimization mechanism improves user experience; and edge nodes and CDN further extend the system's performance and coverage.

[0040] The technical solution of this invention solves the problems of uneven resource utilization, long-thread starvation, and inefficient collaboration of heterogeneous devices in traditional scheduling by constructing a dynamic scheduling mechanism based on real-time resource feedback, driven by a dynamic priority calculation model and resource matching optimization, combined with a hierarchical resource monitoring and closed-loop feedback control system. Through a quantitative decision-making model and elastic migration strategy, dynamic adaptation of task priority, time slice, and device binding is achieved, significantly improving resource utilization and real-time task response in high-concurrency scenarios.

[0041] See the overall architecture diagram. Figure 1 The following is a detailed description of the technical solution:

[0042] 1. Resource monitoring layer

[0043] ●Function: Provides real-time data input for formula parameters

[0044] ●Data Collection:

[0045] Data collection parameters:

[0046] Tremaining (remaining deadline): Obtained through the task management module.

[0047] Rcurrent-load (current node load): Normalized CPU / GPU utilization

[0048] Vnode (node ​​resource vector): [Ucpu,Umem,Ugpu,Qdepth]

[0049] ●Preprocessing module:

[0050] Weighted calculations are performed on the raw data (e.g., GPU utilization is weighted 1.2 times that of CPU).

[0051] 2. Task Analysis Layer

[0052] ●Function: Generates the feature vectors required for formula calculation.

[0053] ●Core Computing:

[0054] Construct the task feature vector Vtask = [Rgpu, Rnpu, Ddeadline]

[0055] Calculate resource matching degree:

[0056]

[0057] Output parameters:

[0058] Ddependency (Task Dependency): The percentage of subtasks in the dependency graph. 3. Dynamic Scheduling Layer

[0059] ●Function: Performs priority calculation and scheduling decisions

[0060] ●Formula Implementation Module:

[0061] Dynamic Priority Calculator:

[0062]

[0063] Coefficient settings: α = 0.6, β = 0.3, γ = 0.1 (can be dynamically adjusted)

[0064] Hstarvation (starvation penalty): Number of times preemption has occurred in the past × 0.05. Scheduling rule:

[0065] Queue upgrade is triggered when Pnew / Pold > 1.2.

[0066] Degradation or migration is triggered when Pnew / Pold < 0.8.

[0067] 4. Load Balancing Layer

[0068] ●Function: Perform thread migration based on formula results

[0069] Migration decision formula:

[0070] Migration benefit assessment:

[0071]

[0072] Triggering conditions: Benefit > 0.5 and target node Rcurrent-load < 70% (II) Process description

[0073] See the overall flowchart Figure 2 The following is a detailed description of the process:

[0074] Phase 1: Task Registration and Parameter Calculation

[0075] Input: New task submission (declaring GPU memory requirements, deadline, etc.)

[0076] Formula preprocessing:

[0077] Calculate Tremaining = Deadline - Current Time

[0078] Example of extracting Vtask and traversing the node list to calculate Sresource-match:

[0079] Task requirements: Vtask = [4GB VRAM, 50 TOPS computing power, 200ms]

[0080] Node status: Vnode1 = [3.8GB, 48TOPS, 180ms]

[0081] Match score: Smatch = 0.92

[0082] Phase 2: Initial Scheduling Decision

[0083] Priority calculation:

[0084] Assume initial parameters:

[0085] Tremaining = 200ms

[0086] Hstarvation=0

[0087] Rcurrent-load = 0.65

[0088] Ddependency = 0.2

[0089] Substitute into the formula:

[0090] P = 0.354

[0091] Queue allocation:

[0092] Mapping priority P to queue Q2 (P∈[0.3,0.4))

[0093] Time slice allocation: TimeSlice = 80ms × (1 + Smatch) = 153.6ms Phase 3: Dynamic feedback adjustment

[0094] Periodic monitoring (every 10ms):

[0095] The GPU utilization was detected to have risen to Rcurrent-load = 0.85.

[0096] Recalculation priority:

[0097] New parameters: Rcurrent-load = 0.85, Hstarvation = 0.2

[0098] New priority:

[0099] Pnew = 0.281

[0100] Trigger migration:

[0101] Calculate migration benefits:

[0102] Target node Starget-match = 0.95, MigrationCost = 5ms

[0103] Benefit = (0.95 - 0.92) / 5 = 0.006 > 0.5 (condition met)

[0104] Perform RDMA zero-copy migration

[0105] Phase 4: Exception Handling and Optimization

[0106] Formula adaptive adjustment:

[0107] When the system is under high load (Ravg-load > 80%):

[0108] Adjustment coefficients α = 0.7, β = 0.2 (increasing the weight of the deadline).

[0109] When a starvation task (Hstarvation > 3) is detected:

[0110] Forced priority increase: Padjusted = P × 1.5

[0111] The above description is merely a preferred embodiment of the present invention and is used only to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for dynamic thread scheduling and management based on real-time resource feedback, characterized in that, By collecting real-time data on CPU, memory, and hardware accelerator resource utilization, load queue length, and thread latency, and combining this data with task type (i.e., computation / I / O intensive), priority, and deadline characteristics, a dynamic scheduling strategy is constructed.

2. The method according to claim 1, characterized in that, First, an improved multi-level feedback queue algorithm is used to allocate initial priorities and time slices. Second, the resource matching degree is optimized through particle swarm optimization. During task execution, thread priorities and migration strategies are dynamically adjusted according to real-time resource fluctuations. Finally, cross-processor load balancing is achieved based on the minimum remaining load migration principle and stream table queue management.

3. The method according to claim 2, characterized in that, Includes the following steps: 1) Periodically collect real-time utilization rates of CPU cores, memory, and hardware accelerators, as well as load queue length and thread execution latency data through a resource monitoring agent; 2) Classify thread task types based on the support vector machine model and generate task feature vectors containing priority, deadline, and resource requirement matrix; 3) A dynamic scheduling algorithm that integrates multi-level feedback queues and particle swarm optimization is adopted. An initial scheduling strategy is generated based on the current resource status and task feature vectors. The objective function of particle swarm optimization is to minimize the weighted sum of resource idle rate and task deadline default rate. 4) During task execution, if the utilization rate of any resource dimension exceeds the preset threshold, a feedback adjustment mechanism is triggered; 5) A dual-queue buffering mechanism is used to handle migration tasks. During the migration process, the original queue context is retained until the target node confirms receipt, thus avoiding task interruption.

4. The method according to claim 3, characterized in that, The sampling period is configured to be 10ms-1s.

5. The method according to claim 3, characterized in that, The priority calculation in the dynamic scheduling algorithm uses the following formula: Where Tremaining is the reciprocal of the remaining deadline, Sresource-match is the cosine similarity between the task's resource requirements and the node's available resources, Hstarvation is the penalty for the number of times the task has been preempted in the past, and Rcurrent-load and Ddependency are the current node load and task dependency, respectively.

6. The method according to claim 3, characterized in that, The hardware accelerators include GPUs / NPUs / FPGAs, and their resource status is monitored by obtaining computing power utilization, video memory usage, and task queue depth through the device driver layer API.

7. The method according to claim 3, characterized in that, Feedback adjustment mechanism: dynamically adjust thread priority weights, reallocate time slices, and migrate low-remaining-load threads to idle computing nodes through flow table matching.

8. The method according to claim 7, characterized in that, The flow table matching uses a five-tuple rule, namely source node, target resource type, minimum remaining load, task tolerance latency, and device compatibility to filter the target node for migration.

Images