A heterogeneous computing power intelligent scheduling method and system based on reinforcement learning technology
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING WENYUE SUANHAI TECHNOLOGY CO LTD
- Filing Date
- 2026-05-14
- Publication Date
- 2026-08-07
AI Technical Summary
Linux CFS调度器仅支持CPU，无法感知GPU/NPU状态，负载均衡存在“组失衡”问题；Kubernetes原生调度器仅支持容器级调度，缺乏对异构算力性能的精细感知；传统强化学习调度存在冷启动慢、探索成本高、难以处理复杂语义任务的问题；基于大语言模型的调度（如SchedCP）采用单Agent决策，缺乏多Agent协同能力，且与内核交互存在安全风险
[0036]本发明的有益效果为：本发明采用轻量级大语言模型推理引擎（模型蒸馏+量化）以及大语言模型与强化学习异步解耦架构，使大语言模型仅用于任务解析和候选策略生成（低频），强化学习负责实时决策（高频），将调度延迟控制在50ms以内，实现了大语言模型实时调度的效果；
Smart Images
Figure CN122526809A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computing power scheduling technology, and in particular to a method and system for intelligent scheduling of heterogeneous computing power based on reinforcement learning technology. Background Technology
[0002] With the rapid development of large-scale AI model training and "East-to-West computing" scenarios, computing power scheduling faces three major challenges: heterogeneous computing power, specifically manifested in the simultaneous deployment of multiple chips within a single node and inconsistent resource descriptions; dynamic load, specifically manifested in the suddenness, periodicity, and uncertainty of AI tasks, making traditional fixed strategies difficult to adapt; and large-scale scheduling, specifically manifested in the multi-card clusters and cross-regional collaboration, making manual scheduling no longer feasible.
[0003] Currently, several scheduling technologies exist in this field, but all have significant drawbacks. The Linux CFS scheduler only supports CPU and cannot perceive GPU / NPU status, resulting in a "group imbalance" problem in load balancing; the Kubernetes native scheduler only supports container-level scheduling and lacks fine-grained perception of heterogeneous computing power performance; traditional reinforcement learning scheduling suffers from slow cold starts, high exploration costs, and difficulty in handling complex semantic tasks; scheduling based on large language models (such as SchedCP) uses single-agent decision-making, lacks multi-agent collaborative capabilities, and poses security risks when interacting with the kernel.
[0004] Therefore, there is an urgent need for a heterogeneous computing power intelligent scheduling method and system based on reinforcement learning technology that can integrate the semantic understanding ability of large language models with the sequential decision-making ability of reinforcement learning, and to build a safe, efficient and interpretable heterogeneous computing power scheduling system to solve the above problems. Summary of the Invention
[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:
[0006] A heterogeneous computing power intelligent scheduling method based on reinforcement learning technology includes the following steps: Step 1, State acquisition and context construction: Real-time acquisition of the state information of each computing node in the heterogeneous computing power cluster, including CPU utilization, GPU utilization, NPU utilization, DPU utilization, task queue length of each node, node temperature, node power consumption, inter-node network bandwidth, latency, packet loss rate, and historical scheduling records; combining the acquired state information with user task descriptions to form a structured context;
[0007] Step 2, LLM candidate strategy generation: The structured context is input into a distilled and quantized lightweight large language model engine. The large language model engine parses the semantic information in the user task at a rate of seconds and outputs structured scheduling requirements and one or more candidate scheduling strategies. The candidate scheduling strategies include task placement node suggestions, resource quota schemes, priority adjustment values and scheduling timing indications.
[0008] Step 3, State Space and Action Mask Mapping: The state information collected in Step 1 is used to construct the state space of the reinforcement learning engine, and the candidate scheduling policy generated in Step 2 is converted into an action mask. The action mask is used to limit the subset of legal actions of the reinforcement learning engine at the current decision time.
[0009] Step 4, Reinforcement Learning Real-time Decision Making: The reinforcement learning engine receives the state space and action mask at a millisecond frequency, and selects the optimal scheduling action from the subset of legal actions under the guidance of the multi-objective reward function; the hot start strategy of the reinforcement learning engine adopts the candidate scheduling strategy generated in step 2 instead of random initialization, which shortens the cold start time to the minute level.
[0010] Step 5, Level 3 Security Execution: The optimal scheduling action will be executed with the following security checks in sequence: L1 Policy Verification: Execute the candidate policy in the offline simulator to detect whether resource contention or deadlock is triggered; L2 Progressive Deployment: Apply the new policy only to a preset proportion (initially 5%) of the scheduling traffic and monitor system stability indicators; L3 Anomaly Rollback: Monitor CPU utilization and task deadlock events in real time through a watchdog. If an anomaly is detected, roll back to the previous stable policy within 2 seconds.
[0011] Step 6, Kernel Dynamic Injection: The scheduling policy that has passed the security verification in Step 5 is dynamically injected into the scheduler in the kernel space through the sched_ext interface of the Linux kernel and the eBPF program. The kernel then performs task selection, load balancing and priority management.
[0012] As an improvement, the lightweight large language model engine in step 2 is asynchronously decoupled from the reinforcement learning engine in step 4: the large language model engine generates candidate policies at a frequency of 1 to 5 seconds, and the reinforcement learning engine executes real-time decisions at a frequency of 10 to 50 milliseconds, so that the end-to-end latency from task parsing to the final scheduling action execution is controlled within 50 milliseconds.
[0013] As an improvement, the multi-objective reward function in step 4 is:
[0014] in Rewards are given for completing the task within the specified time. As a resource cost incentive, Incentives for resource utilization As a reward for energy efficiency, For multi-tenant fairness rewards; each weight to The large language model engine automatically configures the parameters based on the user's preferred keywords in natural language ("cost priority", "performance priority", "green and low carbon").
[0015] As an improvement, the reinforcement learning engine achieves a 63% increase in average reward on the first 100 scheduled tasks compared to the random initialization strategy.
[0016] As an improvement, cross-domain collaborative scheduling steps are also included: each data center node periodically reports available computing resources and network status through the global scheduler; the global scheduler parses user tasks and determines whether they meet the conditions for cross-domain execution; the optimal execution node is selected by comprehensively considering computing power price, network bandwidth, and task queue length; and RDMA technology is used for data preheating and transmission to achieve cross-domain task execution.
[0017] A heterogeneous computing power intelligent scheduling system based on reinforcement learning technology includes:
[0018] Status acquisition module: Deployed on each heterogeneous computing node, used to collect real-time information on node load, temperature, power consumption, queue length and network status;
[0019] The LLM semantic understanding engine has an input end connected to the state acquisition module and the user interface, used to receive structured context and user natural language descriptions, and an output end that outputs structured scheduling requirements and a set of candidate scheduling strategies. The LLM semantic understanding engine uses a lightweight model that has been distilled and quantized, and triggers inference at a frequency of seconds.
[0020] Action mask conversion module: The input end is connected to the output end of the LLM semantic understanding engine, and it is used to convert the candidate scheduling strategy into an action mask that can be recognized by the reinforcement learning engine. The action mask limits the subset of legal actions at the current moment.
[0021] The reinforcement learning optimization engine has a first input terminal connected to the state acquisition module to obtain the state space, a second input terminal connected to the output terminal of the action mask conversion module to receive the action mask, and an output terminal that outputs the optimal scheduling action. The reinforcement learning optimization engine executes decisions at a millisecond frequency, and its initial policy is a hot start of candidate policies generated by the LLM semantic understanding engine.
[0022] Safe Execution Layer: The input end connects to the output end of the reinforcement learning optimization engine, and the output end connects to the kernel interaction interface; the safe execution layer includes: a policy verification unit: with a built-in offline simulator, used to detect whether candidate policies cause resource contention or deadlock; a progressive deployment unit: used to control the traffic ratio of new policies and monitor system stability; an anomaly rollback unit: including a watchdog timer, used to roll back to the previous stable policy within 2 seconds when an anomaly is detected;
[0023] Kernel Interaction Interface: Based on the Linux kernel's sched_ext and eBPF technologies, it is used to dynamically inject scheduling policies verified by the security execution layer into the kernel scheduler, which performs task selection, load balancing, and priority management.
[0024] As an improvement, an asynchronous message queue is set up between the LLM semantic understanding engine and the reinforcement learning optimization engine: the LLM semantic understanding engine writes the generated candidate policies into the queue at a period of 1 to 5 seconds, and the reinforcement learning optimization engine reads the latest candidate policies from the queue at a period of 10 to 50 milliseconds and converts them into action masks, thereby achieving temporal decoupling between the two.
[0025] As an improvement, the reinforcement learning optimization engine incorporates a multi-objective reward function calculation unit, wherein the multi-objective reward function is:
[0026]
[0027] The LLM semantic understanding engine is also used to parse weight preferences from the user's natural language and dynamically configure the weight parameters of the multi-objective reward function calculation unit.
[0028] As an improvement, the system also includes a multi-agent collaborative scheduling module:
[0029] Main Agent: Used to receive user tasks, break down the tasks into several sub-tasks, and distribute them to dedicated Agents according to the type of sub-task;
[0030] Dedicated Agents: These include training agents, inference agents, and data transmission agents. Each dedicated agent synchronizes its state and coordinates its progress through a message bus.
[0031] Result aggregation unit: used to collect the subtask execution results returned by each dedicated agent, merge them and return them to the user; wherein, the main agent and the dedicated agents are respectively connected to the LLM semantic understanding engine and the reinforcement learning optimization engine to obtain their respective scheduling strategies.
[0032] As an improvement, the system also includes a cross-domain computing power collaborative scheduling unit:
[0033] Computing power awareness subunit: Deployed at each data center node, used to periodically report available computing power resources and network status to the global scheduler;
[0034] Cross-domain decision subunit: connected to the LLM semantic understanding engine, used to analyze whether the user task has the conditions for cross-domain execution, and to select the optimal execution node by comprehensively considering computing power price, network bandwidth, and task queuing length;
[0035] Data migration subunit: Employs RDMA technology for data preheating and transmission, and supports encrypted channel return of execution results.
[0036] The beneficial effects of this invention are as follows: This invention adopts a lightweight large language model inference engine (model distillation + quantization) and an asynchronous decoupled architecture between the large language model and reinforcement learning, so that the large language model is only used for task parsing and candidate policy generation (low frequency), and reinforcement learning is responsible for real-time decision-making (high frequency), and the scheduling latency is controlled within 50ms, thus achieving the effect of real-time scheduling of the large language model.
[0037] By designing a unified state-action space mapping mechanism, the output of a large language model is transformed into an action mask for reinforcement learning. For the first time, a seamless integration of semantic understanding of a large language model and numerical optimization of reinforcement learning is achieved, thereby realizing the technical effect of input space matching.
[0038] By combining a large language model with reinforcement learning as a dual-engine approach, we achieved cold start acceleration, online optimization, anomaly handling, and interpretability.
[0039] By setting up a three-tiered security execution layer—sandbox simulation verification, canary traffic deployment, and watchdog rollback—we ensure that AI-generated strategies do not cause system crashes. Attached Figure Description
[0040] Figure 1 This is a schematic diagram illustrating the steps of a heterogeneous computing power intelligent scheduling method based on reinforcement learning technology according to the present invention.
[0041] Figure 2 This is a block diagram of a heterogeneous computing power intelligent scheduling system based on reinforcement learning technology according to the present invention. Detailed Implementation
[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0043] It should be noted that the terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.
[0044] like Figure 1As shown, a heterogeneous computing power intelligent scheduling method based on reinforcement learning technology includes the following steps: Step 1, State Acquisition and Context Construction: Real-time acquisition of state information of each computing node in the heterogeneous computing power cluster, including CPU utilization, GPU utilization, NPU utilization, DPU utilization, task queue length of each node, node temperature, node power consumption, inter-node network bandwidth, latency, packet loss rate, and historical scheduling records; combining the acquired state information with the user task description to form a structured context; Step 2, LLM Candidate Policy Generation: inputting the structured context into a distilled and quantized lightweight large language model engine, which parses the semantic information in the user task at a rate of seconds and outputs structured scheduling requirements and one or more candidate scheduling policies, including task placement node suggestions, resource quota schemes, priority adjustment values, and scheduling timing indicators; Step 3, State Space and Action Mask Mapping: constructing the state space of the reinforcement learning engine from the state information acquired in Step 1, and converting the candidate scheduling policies generated in Step 2 into action masks, which are used to limit the reinforcement learning engine. Step 4: Real-time decision-making in reinforcement learning: The reinforcement learning engine receives the state space and action mask at millisecond frequency, and selects the optimal scheduling action from the subset of legal actions under the guidance of the multi-objective reward function; The hot start strategy of the reinforcement learning engine adopts the candidate scheduling strategy generated in step 2 instead of random initialization, so that the cold start time is shortened to the minute level; Step 5: Three-level safe execution: The optimal scheduling action is executed with the following safety checks in sequence: L1 policy verification: The candidate policy is executed in the offline simulator to detect whether resource contention or deadlock is triggered; L2 progressive deployment: The new policy is applied only to a preset proportion (initially 5%) of the scheduling traffic, and the system stability indicators are monitored; L3 abnormal rollback: The CPU utilization and task dead events are monitored in real time by the watchdog. If an abnormality is detected, it is rolled back to the previous stable policy within 2 seconds; Step 6: Kernel dynamic injection: The scheduling policy that has passed the safety check in step 5 is dynamically injected into the scheduler in the kernel space through the sched_ext interface of the Linux kernel and the eBPF program, and the kernel performs task selection, load balancing and priority management. By using large language models to generate warm-start strategies to replace the random initialization exploration of reinforcement learning, the cold start time of reinforcement learning is reduced from hours to minutes, and the average reward of the first 100 tasks is increased by 63%, thus overcoming the technical bias that "reinforcement learning must be trained from scratch".
[0045] In step 2, the lightweight large language model engine and the reinforcement learning engine in step 4 are asynchronously decoupled: the large language model engine generates candidate policies at a frequency of 1 to 5 seconds, and the reinforcement learning engine executes real-time decisions at a frequency of 10 to 50 milliseconds, so that the end-to-end latency from task parsing to the final scheduling action execution is controlled within 50 milliseconds.
[0046] The multi-objective reward function in step 4 is:
[0047] in Rewards are given for completing the task within the specified time. As a resource cost incentive, Incentives for resource utilization As a reward for energy efficiency, For multi-tenant fairness rewards; each weight to The large language model engine automatically configures the scheduling based on the user's preferred keywords in natural language ("cost priority", "performance priority", "green and low-carbon"). The reinforcement learning engine achieves a 63% improvement in average reward on the first 100 scheduled tasks compared to a random initialization strategy. It also includes a cross-domain collaborative scheduling step: each data center node periodically reports available computing resources and network status through the global scheduler; the global scheduler parses the user's task to determine if it meets the conditions for cross-domain execution; it selects the optimal execution node based on a combination of computing power price, network bandwidth, and task queue length; and it uses RDMA technology for data preheating and transmission to achieve cross-domain task execution.
[0048] like Figure 2 As shown, a heterogeneous computing power intelligent scheduling system based on reinforcement learning technology includes:
[0049] The system comprises the following modules: a State Acquisition Module, deployed on heterogeneous computing nodes, for real-time acquisition of node load, temperature, power consumption, queue length, and network status information; an LLM Semantic Understanding Engine, whose input is connected to the State Acquisition Module and the user interface, for receiving structured context and user natural language descriptions, and whose output is structured scheduling requirements and a set of candidate scheduling strategies; the LLM Semantic Understanding Engine employs a lightweight model that has undergone distillation and quantization, and triggers inference at a rate of seconds; an Action Mask Conversion Module, whose input is connected to the output of the LLM Semantic Understanding Engine, for converting candidate scheduling strategies into action masks recognizable by the reinforcement learning engine, wherein the action mask limits the subset of legal actions at the current moment; and a Reinforcement Learning Optimization Engine, whose first input is connected to the State Acquisition Module to obtain the state space, and whose second input is connected to the output of the Action Mask Conversion Module to receive the action mask. The output terminal outputs the optimal scheduling action; the reinforcement learning optimization engine executes decisions at a millisecond frequency, and its initial policy is a hot-start candidate policy generated by the LLM semantic understanding engine; Safe execution layer: the input terminal connects to the output terminal of the reinforcement learning optimization engine, and the output terminal connects to the kernel interaction interface; the safe execution layer includes: a policy verification unit: with a built-in offline simulator, used to detect whether candidate policies cause resource contention or deadlock; a progressive deployment unit: used to control the traffic ratio of new policies and monitor system stability; an anomaly rollback unit: containing a watchdog timer, used to roll back to the previous stable policy within 2 seconds when an anomaly is detected; kernel interaction interface: based on the Linux kernel's sched_ext and eBPF technologies, used to dynamically inject the scheduling policy verified by the safe execution layer into the kernel scheduler, which performs task selection, load balancing, and priority management. In use, the system can adapt to a variety of application scenarios: achieving millisecond-level decision-making in real-time inference scenarios to meet online service latency requirements; supporting stable scheduling of long-term, high-computing-power tasks in large model training scenarios; improving throughput and resource utilization in batch processing jobs; supporting energy consumption optimization through multi-objective reward functions in green scheduling scenarios; and ensuring scheduling fairness in multi-tenant scenarios.
[0050] An asynchronous message queue is set up between the LLM semantic understanding engine and the reinforcement learning optimization engine: the LLM semantic understanding engine writes the generated candidate policies into the queue at a period of 1 to 5 seconds, and the reinforcement learning optimization engine reads the latest candidate policies from the queue at a period of 10 to 50 milliseconds and converts them into action masks, thereby achieving temporal decoupling between the two.
[0051] The reinforcement learning optimization engine has a built-in multi-objective reward function calculation unit, and the multi-objective reward function is:
[0052]
[0053] The LLM semantic understanding engine is also used to parse weight preferences from the user's natural language and dynamically configure the weight parameters of the multi-objective reward function calculation unit.
[0054] The system also includes a multi-agent collaborative scheduling module: a main agent, used to receive user tasks, decompose the tasks into several sub-tasks, and distribute them to dedicated agents according to the sub-task type; dedicated agents, including training agents, inference agents, and data transmission agents, with each dedicated agent synchronizing its state and coordinating its progress through a message bus; and a result aggregation unit, used to collect the sub-task execution results returned by each dedicated agent, merge them, and return them to the user; wherein, the main agent and dedicated agents are respectively connected to the LLM semantic understanding engine and the reinforcement learning optimization engine to obtain their respective scheduling strategies.
[0055] The system also includes a cross-domain computing power collaborative scheduling unit: a computing power awareness subunit, deployed at each data center node, used to periodically report available computing power resources and network status to the global scheduler; a cross-domain decision-making subunit, connected to the LLM semantic understanding engine, used to analyze whether user tasks have the conditions for cross-domain execution, and to select the optimal execution node by comprehensively considering computing power price, network bandwidth, and task queue length; and a data migration subunit, which uses RDMA technology for data preheating and transmission, and supports returning execution results through encrypted channels.
[0056] Example: Step 1: State Acquisition and Context Construction
[0057] The system collects real-time status information of each computing node in the heterogeneous computing power cluster through monitoring modules deployed on each computing node. This status information includes, but is not limited to: CPU utilization, GPU utilization, NPU utilization, DPU utilization, task queue length for each node, node temperature, node power consumption, inter-node network bandwidth, latency, packet loss rate, and historical scheduling records. The collected status information is combined with the user-submitted natural language task description to form a structured context. For example, if a user inputs "Run a deep learning training task with a limited budget, preferably completed within 2 hours," the system will extract the task type as "training," with constraints of "cost priority" and "latest completion time of 2 hours."
[0058] Step 2: LLM Candidate Strategy Generation
[0059] The structured context is input into a lightweight large language model engine that undergoes distillation and 4-bit quantization. In this embodiment, the LLaMA-7B model is used for fine-tuning and quantization, with inference latency controlled within 1-5 seconds. The engine parses the semantic information in the user task at a rate of seconds, outputting structured scheduling requirements and multiple candidate scheduling strategies. Each candidate scheduling strategy includes: a task placement node suggestion (e.g., "suggest assigning to node A"), a resource quota scheme (e.g., "allocate 4 GPUs, 32GB of video memory, and 8 CPU cores"), a priority adjustment value (e.g., "priority +5"), and a scheduling timing indication (e.g., "execute immediately"). In this embodiment, LLM generates 3 candidate strategies, along with expected benefit scores.
[0060] Step 3: Mapping State Space to Action Mask
[0061] The state information collected in step 1 is used to construct the state space of the reinforcement learning engine. The dimensions of the state space include: computational load (node utilization, queue length), task characteristics (type, resource requirements, remaining time), network status (bandwidth, latency), energy consumption data (power consumption, PUE), and historical experience (historical scheduling results of similar tasks). Simultaneously, the candidate scheduling policies generated in step 2 are converted into action masks. The action mask is a binary vector used to limit the subset of legal actions of the reinforcement learning engine at the current decision moment. For example, if the LLM recommends candidate nodes {node A, node B}, the action mask sets A and B in the node selection action space as legal (value 1), and the remaining nodes as illegal (value 0). Action dimensions such as resource quotas and priority adjustments are also limited by the mask to the suggested range of the candidate policies.
[0062] Step 4: Reinforcement Learning Real-Time Decision Making
[0063] The reinforcement learning engine employs a Deep Q-Network (DQN) or Proximal Policy Optimization (PPO) algorithm, receiving the state space and action mask at millisecond-level frequencies of 10-50 milliseconds. The engine selects the optimal scheduling action from the subset of legal actions defined by the action mask, outputting the specific node ID, resource quantity, priority adjustment value, and execution time. Notably, this embodiment employs a warm-start strategy: its initial policy network parameters are not randomly initialized, but rather pre-trained using candidate policies generated by LLM and their expected rewards as supervision signals, reducing the cold-start time from hours in traditional reinforcement learning to minutes. Testing showed that, in the first 100 scheduling tasks, the average reward in this embodiment was 63% higher than that of the randomly initialized strategy.
[0064] Step 5: Level 3 Security Execution
[0065] The optimal scheduling action output by the reinforcement learning engine is sequentially subjected to three levels of safety checks:
[0066] L1 policy verification: The candidate scheduling policy is executed in an offline simulator to simulate its effect on the current system state. If any abnormalities such as resource contention (e.g., two tasks competing for the same GPU), deadlock (e.g., circular waiting) or node overload occur in the simulation results, the policy is rejected, and LLM is triggered to regenerate or select a suboptimal policy.
[0067] L2 Incremental Deployment: For new strategies validated through L1, a canary release mechanism is adopted. Initially, the strategy is applied to only 5% of the scheduled request traffic, and system stability metrics (such as task failure rate and average response time) are continuously monitored. If no anomalies are detected within 5 minutes, the application ratio is gradually increased to 20%, 50%, and finally 100%.
[0068] L3 anomaly rollback: Deploy an independently running watchdog thread to monitor key metrics such as CPU utilization, task dead events, and scheduling latency in real time, based on kernel feedback. If a CPU spike exceeding a threshold (e.g., >90% for 3 seconds) or a task dead for more than 2 seconds is detected, the watchdog will automatically roll back the scheduling policy to the previous stable version within 2 seconds and issue an alarm.
[0069] Step 6: Kernel Dynamic Injection
[0070] A scheduling policy with three levels of security verification is dynamically injected into the kernel-space scheduler via the Linux kernel's `sched_ext` interface and the eBPF program. Specifically, the user-space AI decision engine compiles the scheduling policy into eBPF bytecode, which is then loaded into the kernel via the `bpf` system call. The kernel's `sched_ext` scheduler parses the bytecode and dynamically replaces the current task selection, load balancing, and priority management logic. The entire process requires no modification to the kernel source code and no system reboot. After successful injection, the kernel executes task scheduling according to the new policy.
[0071] This embodiment achieves real-time scheduling requirements by asynchronously decoupling LLM and RL (LLM generates candidates in seconds and RL makes decisions in milliseconds) and controlling the end-to-end scheduling latency to within 50ms.
[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A heterogeneous computing power intelligent scheduling method based on reinforcement learning technology, characterized in that, Includes the following steps: Step 1, Status Acquisition and Context Construction: Real-time acquisition of status information of each computing node in the heterogeneous computing power cluster. The status information includes CPU utilization, GPU utilization, NPU utilization, DPU utilization, task queue length of each node, node temperature, node power consumption, inter-node network bandwidth, latency, packet loss rate, and historical scheduling records; the acquired status information is combined with the user task description to form a structured context. Step 2, LLM candidate strategy generation: The structured context is input into a distilled and quantized lightweight large language model engine. The large language model engine parses the semantic information in the user task at a rate of seconds and outputs structured scheduling requirements and one or more candidate scheduling strategies. The candidate scheduling strategies include task placement node suggestions, resource quota schemes, priority adjustment values and scheduling timing indications. Step 3, State Space and Action Mask Mapping: The state information collected in Step 1 is used to construct the state space of the reinforcement learning engine, and the candidate scheduling policy generated in Step 2 is converted into an action mask. The action mask is used to limit the subset of legal actions of the reinforcement learning engine at the current decision time. Step 4, Reinforcement Learning Real-time Decision Making: The reinforcement learning engine receives the state space and action mask at a millisecond frequency, and selects the optimal scheduling action from the subset of legal actions under the guidance of the multi-objective reward function; the hot start strategy of the reinforcement learning engine adopts the candidate scheduling strategy generated in step 2 instead of random initialization, which shortens the cold start time to the minute level. Step 5, Level 3 Security Execution: The optimal scheduling action will be executed with the following security checks in sequence: L1 Policy Verification: Execute the candidate policy in the offline simulator to detect whether resource contention or deadlock is triggered; L2 Progressive Deployment: Apply the new policy only to a preset proportion (initially 5%) of the scheduling traffic and monitor system stability indicators; L3 Anomaly Rollback: Monitor CPU utilization and task deadlock events in real time through a watchdog. If an anomaly is detected, roll back to the previous stable policy within 2 seconds. Step 6, Kernel Dynamic Injection: The scheduling policy that has passed the security verification in Step 5 is dynamically injected into the scheduler in the kernel space through the sched_ext interface of the Linux kernel and the eBPF program. The kernel then performs task selection, load balancing and priority management.
2. The heterogeneous computing power intelligent scheduling method based on reinforcement learning technology according to claim 1, characterized in that, In step 2, the lightweight large language model engine and the reinforcement learning engine in step 4 are asynchronously decoupled: the large language model engine generates candidate policies at a frequency of 1 to 5 seconds, and the reinforcement learning engine executes real-time decisions at a frequency of 10 to 50 milliseconds, so that the end-to-end latency from task parsing to the final scheduling action execution is controlled within 50 milliseconds.
3. The heterogeneous computing power intelligent scheduling method based on reinforcement learning technology according to claim 1, characterized in that, The multi-objective reward function in step 4 is: in Rewards are given for completing the task within the specified time. As a resource cost incentive, Incentives for resource utilization As a reward for energy efficiency, For multi-tenant fairness rewards; each weight to It is automatically configured by the large language model engine based on the user's preferred keywords in natural language ("cost priority", "performance priority", "green and low carbon").
4. The heterogeneous computing power intelligent scheduling method based on reinforcement learning technology according to claim 1, characterized in that, The reinforcement learning engine achieved an average reward improvement of 63% over the first 100 scheduled tasks compared to the random initialization strategy.
5. The heterogeneous computing power intelligent scheduling method based on reinforcement learning technology according to claim 1, characterized in that, It also includes cross-domain collaborative scheduling steps: each data center node periodically reports available computing resources and network status through the global scheduler; the global scheduler parses user tasks and determines whether they meet the conditions for cross-domain execution; the optimal execution node is selected by comprehensively considering computing power price, network bandwidth, and task queue length; and RDMA technology is used for data preheating and transmission to realize cross-domain task execution.
6. A heterogeneous computing power intelligent scheduling system based on reinforcement learning technology according to claims 1 to 5, characterized in that, include: Status acquisition module: Deployed on each heterogeneous computing node, used to collect real-time information on node load, temperature, power consumption, queue length and network status; The LLM semantic understanding engine has an input end connected to the state acquisition module and the user interface, used to receive structured context and user natural language descriptions, and an output end that outputs structured scheduling requirements and a set of candidate scheduling strategies. The LLM semantic understanding engine uses a lightweight model that has been distilled and quantized, and triggers inference at a frequency of seconds. Action mask conversion module: The input end is connected to the output end of the LLM semantic understanding engine, and it is used to convert the candidate scheduling strategy into an action mask that can be recognized by the reinforcement learning engine. The action mask limits the subset of legal actions at the current moment. The reinforcement learning optimization engine has a first input terminal connected to the state acquisition module to obtain the state space, a second input terminal connected to the output terminal of the action mask conversion module to receive the action mask, and an output terminal that outputs the optimal scheduling action. The reinforcement learning optimization engine executes decisions at a millisecond frequency, and its initial policy is a hot start of candidate policies generated by the LLM semantic understanding engine. The secure execution layer has an input end connected to the output end of the reinforcement learning optimization engine, and an output end connected to the kernel interaction interface. The secure execution layer includes: a policy verification unit with a built-in offline simulator for detecting whether candidate policies cause resource contention or deadlock; Progressive deployment unit: used to control the traffic ratio of new policies and monitor system stability; Anomaly rollback unit: includes a watchdog timer, used to roll back to the previous stable policy within 2 seconds when an anomaly is detected; Kernel Interaction Interface: Based on the Linux kernel's sched_ext and eBPF technologies, it is used to dynamically inject scheduling policies verified by the security execution layer into the kernel scheduler, which performs task selection, load balancing, and priority management.
7. A heterogeneous computing power intelligent scheduling system based on reinforcement learning technology according to claim 6, characterized in that, An asynchronous message queue is set up between the LLM semantic understanding engine and the reinforcement learning optimization engine: the LLM semantic understanding engine writes the generated candidate policies into the queue at a period of 1 to 5 seconds, and the reinforcement learning optimization engine reads the latest candidate policies from the queue at a period of 10 to 50 milliseconds and converts them into action masks, thereby achieving temporal decoupling between the two.
8. A heterogeneous computing power intelligent scheduling system based on reinforcement learning technology according to claim 6, characterized in that, The reinforcement learning optimization engine has a built-in multi-objective reward function calculation unit, and the multi-objective reward function is: The LLM semantic understanding engine is also used to parse weight preferences from the user's natural language and dynamically configure the weight parameters of the multi-objective reward function calculation unit.
9. A heterogeneous computing power intelligent scheduling system based on reinforcement learning technology according to claim 6, characterized in that, The system also includes a multi-agent collaborative scheduling module: Main Agent: Used to receive user tasks, break down the tasks into several sub-tasks, and distribute them to dedicated Agents according to the type of sub-task; Dedicated Agents: These include training agents, inference agents, and data transmission agents. Each dedicated agent synchronizes its state and coordinates its progress through a message bus. Result aggregation unit: used to collect the subtask execution results returned by each dedicated agent, merge them and return them to the user; wherein, the main agent and the dedicated agents are respectively connected to the LLM semantic understanding engine and the reinforcement learning optimization engine to obtain their respective scheduling strategies.
10. A heterogeneous computing power intelligent scheduling system based on reinforcement learning technology according to claim 6, characterized in that, The system also includes a cross-domain computing power collaborative scheduling unit: Computing power awareness subunit: Deployed at each data center node, used to periodically report available computing power resources and network status to the global scheduler; Cross-domain decision subunit: connected to the LLM semantic understanding engine, used to analyze whether the user task has the conditions for cross-domain execution, and to select the optimal execution node by comprehensively considering computing power price, network bandwidth, and task queuing length; Data migration subunit: Employs RDMA technology for data preheating and transmission, and supports encrypted channel return of execution results.