Heterogeneous computing task scheduling method based on quantized fitness and predicted scheduling

Abstract

This invention discloses a heterogeneous computing power task scheduling method based on quantized adaptability and predictive scheduling, comprising: dividing cluster nodes into a GPU computing power pool and a CPU computing power pool; constructing a task-resource adaptability scoring matrix; calculating the adaptability score of tasks on various types of nodes and determining the GPU-sensitive, CPU-adaptable, or hybrid category based on the ratio; generating scheduling decisions using a two-layer decoupled scheduling framework consisting of a priority decision layer and a node allocation layer, and performing closed-loop calibration; collecting cluster telemetry data and inputting it into a load prediction model to trigger expansion, weight adjustment, and preemption intervention in the event of resource bottlenecks. This invention solves the problems of low utilization due to the mixed use of GPU and CPU resources and the difficulty in converging end-to-end latency for high-priority artificial intelligence tasks due to the lack of predictive scheduling capabilities.

Description

Technical Field

[0001] This invention relates to the field of cloud-native heterogeneous computing power scheduling technology, specifically to a heterogeneous computing power task scheduling method based on quantized adaptability and predictive scheduling. Background Technology

[0002] Kubernetes has become the de facto standard for orchestrating heterogeneous computing power in cloud-native environments. Its native scheduler allocates container groups to compute nodes based on resource requests and limits, combined with mechanisms such as node affinity and taint tolerance. In the production pipeline of scientific research artificial intelligence datasets, the task chain covers web crawling, document format parsing, multi-agent semantic annotation, data cleaning, and quality auditing. The dependence of each stage on Graphics Processing Units (GPUs) and Central Processing Units (CPUs) differs fundamentally. GPU resources are scarce and expensive, while CPU resources are relatively abundant. However, existing scheduling schemes typically include all compute nodes in a unified resource pool, relying on manual experience to set static affinity labels, lacking a quantitative perception of task computing power affinity. On the one hand, GPU inference services (such as optical character recognition and visual language model inference) are queued due to insufficient GPUs, while CPU crawling and cleaning tasks occupy GPU nodes, resulting in GPU resources being squeezed out by low-computing-power tasks and overall low utilization. On the other hand, the scheduling system lacks the ability to predict future resource contention and can only respond passively after tasks are submitted in a burst. The end-to-end scheduling latency of high-priority inference tasks is difficult to converge to the target range, and the dataset delivery cycle is uncontrollable. Therefore, a heterogeneous computing power scheduling scheme that can quantitatively model the fit between tasks and resources and proactively intervene based on prediction is needed. Summary of the Invention

[0003] The purpose of this invention is to provide a cross-institutional collaboration method and system based on graph federation learning, which aims to solve the problems of the lack of task-resource quantification and adaptation modeling in the existing Kubernetes heterogeneous computing power scheduling, which leads to the mixed occupation and low utilization of GPU and CPU resources, and the lack of prediction and proactive intervention capabilities for future load in the scheduling system, making it difficult to converge the end-to-end scheduling latency of high-priority artificial intelligence tasks.

[0004] To address the aforementioned technical problems, this invention provides a heterogeneous computing power task scheduling method based on quantization fit and predictive scheduling, comprising:

[0005] Computing nodes in a heterogeneous computing power cluster are grouped by computing power type to obtain computing power pools. A task-resource adaptability scoring matrix is ​​constructed and initialized based on a preset task type and the type of each node in the computing power pool.

[0006] Calculate the suitability score of the task to be scheduled on the computing power pool node to update the task-resource suitability score matrix, and obtain the updated task-resource suitability score matrix; determine the computing power suitability category according to the ratio of the suitability score on the nodes corresponding to different computing power pools;

[0007] A scheduling framework comprising a priority decision layer and a node allocation layer is adopted. The priority decision layer outputs the priority score of the task to be scheduled based on the cluster state vector, and generates a priority ranking based on the priority score. The node allocation layer solves the node allocation scheme based on the priority ranking, the updated task-resource adaptability score matrix and the computing power adaptability category, and generates a scheduling decision based on the node allocation scheme.

[0008] The task to be scheduled is executed according to the scheduling decision, the measured performance indicators are collected, and the updated task-resource fit score matrix is ​​calibrated to obtain the calibrated task-resource fit score matrix.

[0009] Collect cluster status data and input it into the load prediction model, output prediction values, and when the output prediction values ​​meet the preset intervention conditions, execute scheduling intervention actions, including adjusting the node allocation scheme according to the calibrated task-resource adaptability score matrix.

[0010] Further, the step of grouping computing nodes in the heterogeneous computing power cluster according to computing power type to obtain computing power pools, and constructing and initializing a task-resource compatibility scoring matrix based on preset task types and the types of nodes in the computing power pools, includes: reading the hardware configuration information of each computing node in the heterogeneous computing power cluster; if the computing node is equipped with a graphics processing unit (GPU) accelerator card, then the computing node is marked as a GPU computing power pool node; if the computing node is only equipped with a CPU, then the computing node is marked as a CPU computing power pool node, thus obtaining the computing power pool; constructing the task-resource compatibility scoring matrix with the preset task type as the row and the types of nodes in the computing power pool as the column; setting the initial score of the corresponding element in the task-resource compatibility scoring matrix based on the preset affinity relationship between the preset task type and the types of nodes in the computing power pools; and marking all the initial scores as pending calibration, thus completing the initialization.

[0011] Further, the step of calculating the adaptation score of the task to be scheduled on the nodes of the computing power pool to update the task-resource adaptation score matrix, obtaining an updated task-resource adaptation score matrix; determining the computing power adaptation category based on the ratio of the adaptation score on nodes corresponding to different computing power pools includes: for the task to be scheduled belonging to each of the preset task types, obtaining the task feature vector of the task to be scheduled and the resource status parameters of each node in the computing power pool, and using the adaptation evaluation function to calculate the task feature vector and the resource status parameters of each node in the computing power pool, to obtain the computing power adaptation category of the task to be scheduled on each of the nodes corresponding to different computing power pools. The task-resource matching score matrix is ​​updated by scoring the matching score on the node and updating the row corresponding to the task to be scheduled in the task-resource matching score matrix. The updated task-resource matching score matrix is ​​obtained. The ratio of the matching score of the task to be scheduled on the graphics processor computing power pool node to the matching score on the central processing unit computing power pool node is calculated. The ratio is compared with at least one preset threshold interval. The task to be scheduled is marked as the corresponding computing power matching category according to the preset threshold interval to which the ratio belongs. The computing power matching category includes at least: graphics processor sensitive type, central processing unit compatible type and hybrid type.

[0012] Furthermore, the scheduling framework employing a priority decision layer and a node allocation layer, wherein the priority decision layer outputs a priority score for the task to be scheduled based on the cluster state vector, and generates a priority ranking based on the priority score, includes: obtaining the waiting time, computational load, and deadline of the task to be scheduled as task attribute information; constructing the cluster state vector, which contains multiple feature components representing the computing power pool resource status, task queue status, and the adaptation status of the task to be scheduled; inputting the task attribute information and the cluster state vector into the reinforcement learning policy network constituting the priority decision layer, whereby the reinforcement learning policy network performs forward inference, outputs the priority score for the task to be scheduled, and generates a priority ranking based on the priority score, wherein the reinforcement learning policy network is trained using an ε-greedy exploration strategy and an experience replay mechanism.

[0013] Further, the step of obtaining a node allocation scheme by the node allocation layer based on the priority sorting, the updated task-resource fit score matrix, and the computing power fit category, and generating a scheduling decision based on the node allocation scheme, includes: the node allocation layer sequentially traversing the tasks to be scheduled according to the priority sorting; for each task to be scheduled, filtering the corresponding computing power pool according to the computing power fit category to obtain a candidate node set; using the allocation relationship between the tasks to be scheduled and the nodes in the candidate node set as a decision variable; and based on the decision variable, combined with the available resources of each node in the candidate node set, minimizing... The objective function is the weighted sum of the completion times of the tasks to be scheduled. A combinatorial optimization problem is constructed and solved, with constraints including the available resources and unique node allocation. The optimal values ​​of the decision variables are obtained, where unique node allocation restricts the tasks to be scheduled to be assigned to only one node. Based on the allocation relationship indicated by the optimal values ​​of the decision variables, the target allocation node for the tasks to be scheduled is determined from the candidate node set. After traversing all the tasks to be scheduled, a node allocation scheme is generated based on the correspondence between the tasks to be scheduled and the target allocation node, and a scheduling decision is generated based on the node allocation scheme.

[0014] Furthermore, the constraints also include: if the computing power adaptation category of the task to be scheduled is graphics processor sensitive, then the task to be scheduled is restricted to be allocated only to the graphics processor computing power pool node; if the computing power adaptation category of the task to be scheduled is central processing unit adapted, then it is preferentially allocated to the central processing unit computing power pool node; when the available resources of the central processing unit computing power pool node are insufficient, it is allowed to be allocated to the graphics processor computing power pool node and a penalty term is applied to the objective function.

[0015] Further, the step of generating a scheduling decision based on the node allocation scheme includes: if the computing power adaptation category of the task to be scheduled is graphics processor sensitive, then generating a scheduling decision including a first affinity rule, wherein the first affinity rule is used to force the task to be scheduled to be scheduled to a graphics processor computing power pool node; if the computing power adaptation category of the task to be scheduled is central processing unit adapted, then generating a scheduling decision including a second affinity rule, wherein the weight value of the second affinity rule is determined based on the corresponding adaptation score in the task-resource adaptation score matrix.

[0016] Further, the step of executing the task to be scheduled according to the scheduling decision, collecting measured performance indicators, and calibrating the updated task-resource fit score matrix to obtain a calibrated task-resource fit score matrix includes: during the execution of the task to be scheduled according to the scheduling decision, collecting the actual execution time and actual resource occupancy rate of the task to be scheduled on the execution node indicated by the scheduling decision as the measured performance indicators; extracting the element at the intersection of the row corresponding to the task to be scheduled and the column corresponding to the computing pool to which the execution node belongs from the updated task-resource fit score matrix as the target element; comparing the measured performance indicators with the fit score indicated by the target element to calculate the score error; if the score error exceeds a preset threshold, updating the target element based on the score error using a preset exponential smoothing algorithm, clearing the uncalibrated status marker of the target element, completing the calibration, and obtaining the calibrated task-resource fit score matrix.

[0017] Furthermore, the step of executing the task to be scheduled according to the scheduling decision also includes: for the task to be scheduled whose computing power adaptation category is CPU-adapted, scheduling the task to be scheduled to the CPU computing power pool node according to the second affinity rule in the scheduling decision, thereby achieving physical isolation from the inference task on the graphics processor computing power pool node.

[0018] Further, the step of collecting cluster status data and inputting it into the load prediction model to output predicted values ​​includes: collecting historical load time series data of each computing pool in the heterogeneous computing power cluster at fixed time periods as the cluster status data; obtaining the number of nodes in the heterogeneous computing power cluster, selecting a corresponding load prediction model based on the number of nodes, and if the number of nodes is less than a preset threshold, selecting a random forest regression model as the load prediction model, and if the number of nodes is greater than or equal to the preset threshold, selecting a two-layer fully connected neural network model as the load prediction model; inputting the cluster status data into the load prediction model, and having the load prediction model output the predicted load value and prediction completion time of each computing pool within a preset future time period as the predicted values.

[0019] Further, the execution of the scheduling intervention action includes: when the predicted load value exceeds a preset high load threshold, triggering the nodes of the heterogeneous computing power cluster to perform automatic expansion operations; when the first prediction time is less than the second prediction time, and the difference between the first prediction time and the second prediction time is greater than a preset time difference threshold, increasing the weight of the task to be scheduled in the CPU computing power pool corresponding to the task-resource fit score matrix, wherein the first prediction time is the predicted completion time of the task to be scheduled in the CPU computing power pool, and the second prediction time is the predicted completion time of the task to be scheduled in the GPU computing power pool; when the priority score of the task to be scheduled is higher than a preset priority threshold, and the minimum value of the predicted completion time is greater than a preset service level agreement threshold, generating a preemption suggestion to prompt the task with a priority score lower than the preset priority threshold to be evicted; obtaining the task currently in the scheduled but not yet executed state as the intervention task, withdrawing the intervention task to the queue of tasks to be scheduled, recalculating the computing power fit category according to the calibrated task-resource fit score matrix, and triggering the scheduling framework to solve again.

[0020] The beneficial effects of this invention are as follows:

[0021] 1. This invention uses a task-resource suitability scoring matrix to quantitatively classify GPU-sensitive and CPU-suitable tasks, and precisely schedules GPU-sensitive inference tasks to GPU processor computing power pool nodes through hard affinity constraints, while isolating CPU-suitable crawling and cleaning tasks to CPU processor computing power pool nodes through soft affinity constraints. This prevents GPU resources from being squeezed out by low-computing-power tasks, and increases the effective inference utilization of GPU nodes from about 52% in the hybrid scheduling baseline to about 89%. The overall resource scheduling efficiency is significantly better than the static affinity rule scheme.

[0022] 2. This invention employs a two-layer decoupled scheduling framework that includes a priority decision layer and a node allocation layer, separating task priority ranking and precise node mapping into independent sub-problems. This significantly reduces the complexity of the joint search, allows the node allocation scheme to be solved in sub-second time, and enables it to respond to dynamic load changes in the cluster.

[0023] 3. By introducing a load prediction model based on telemetry feature vectors, this invention triggers pre-expansion before the average utilization of the GPU pool reaches the 85% threshold and dynamically increases the affinity weight of the central processing unit computing power pool according to the prediction completion time. This significantly reduces the queuing time of high-priority inference tasks, and the end-to-end scheduling latency can converge to the target range of the service level agreement. The service level agreement achievement rate of high-priority tasks is increased from approximately 71% to approximately 97%.

[0024] 4. This invention constructs a four-stage self-learning closed loop covering cold start default score initialization, quantitative modeling update, actual data write-back, and dynamic adjustment of prediction weights. This enables the scheduling strategy to automatically adapt to scenarios such as dataset type switching, model version upgrade, and cluster expansion and contraction, significantly reducing operation and maintenance configuration costs. Furthermore, in a completely new deployment scenario without historical data, the system can be put into use immediately through the default score and exploration mechanism, and automatically converges as it accumulates during operation. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the overall process of the heterogeneous computing power task scheduling method based on quantized adaptability and predictive scheduling of the present invention.

[0027] Figure 2 This is a schematic diagram of the reinforcement learning scheduling process of the priority decision layer of the present invention;

[0028] Figure 3 This is a schematic diagram of the task-resource suitability scoring matrix calibration process of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0030] Example 1

[0031] like Figure 1 As shown, this invention provides a heterogeneous computing power task scheduling method based on quantization fit and predictive scheduling, comprising:

[0032] Step S1: Group the computing nodes in the heterogeneous computing power cluster according to computing power type to obtain computing power pools, and construct and initialize the task-resource adaptability scoring matrix based on the preset task type and the node type in the computing power pool.

[0033] The node information listener of the container orchestration system is invoked to periodically read the hardware specifications of each compute node. For compute nodes equipped with graphics processing unit (GPU) accelerator cards, a resource-tier tag `resource-tier=gpu-pool` is written, along with the accelerator card model (e.g., A100, H800, RTX4090), memory capacity (in GB), and the number of accelerator cards per node. The node is then assigned to the GPU compute pool. For compute nodes equipped only with CPUs, the tag `resource-tier=cpu-pool` is written, along with the number of physical cores and memory bandwidth (in GB / s), and the node is assigned to the CPU compute pool. In the example cluster, 8 nodes with accelerator cards are assigned to the GPU compute pool, and 32 pure CPU nodes are assigned to the CPU compute pool, resulting in a dual-pool compute resource view consisting of the GPU compute pool and the CPU compute pool. While forming a dual-pool computing power resource view, the scheduling controller initializes a two-dimensional task-resource adaptation score matrix (TRASM) using preset task types as row indices and two types of node types as column indices. In this embodiment, the preset task types include four categories: paper crawling, format parsing, semantic annotation, and data cleaning. Therefore, the matrix is ​​4 rows and 2 columns, and the matrix elements represent the normalized performance adaptation score of a certain type of task on a certain type of node, with values ​​ranging from 0 to 1. During the cold start phase when historical execution data is lacking, default scores are set according to the computing power characteristics of each task: GPU-sensitive tasks (format parsing, semantic annotation) have an initial adaptation score of 1.0 in the GPU computing power pool column and 0.1 in the CPU computing power pool column; CPU-adaptive tasks (paper crawling, data cleaning) have a score of 0.9 in the CPU computing power pool column and 0.3 in the GPU computing power pool column; and mixed tasks have an initial score of 0.5 in both types of node columns. All default scores are marked as pending calibration. The initial task-resource fit score matrix is ​​as follows: Paper crawling task has a fit score of 0.3 in the GPU computing power pool and 0.9 in the CPU computing power pool (to be calibrated); Format parsing task has a fit score of 1.0 in the GPU computing power pool and 0.1 in the CPU computing power pool (to be calibrated); Semantic annotation task has a fit score of 1.0 in the GPU computing power pool and 0.1 in the CPU computing power pool (to be calibrated); Data cleaning task has a fit score of 0.3 in the GPU computing power pool and 0.9 in the CPU computing power pool (to be calibrated).

[0034] Step S2: Calculate the suitability score of the task to be scheduled on the computing power pool node to update the task-resource suitability score matrix, and obtain the updated task-resource suitability score matrix; determine the computing power suitability category according to the ratio of the suitability score on the nodes corresponding to different computing power pools.

[0035] After obtaining the initial task-resource fit score matrix, the matrix elements need to be updated based on the actual performance of tasks in historical execution. For tasks to be scheduled belonging to each preset task type, their throughput, latency, and resource utilization on the GPU computing pool and CPU computing pool are collected. The fit score of the task on each type of node is calculated using the fit evaluation function, and the score is written into the row corresponding to the task type in the score matrix, thus obtaining the updated task-resource fit score matrix. Taking the format parsing task as an example, its actual throughput on the GPU computing pool node is about 15,000 pages per hour per node, the actual first-word latency is low, and the accelerator card utilization is high, resulting in a comprehensive fit score of about 0.95. On the CPU computing pool node, due to the lack of accelerator cards, the throughput is less than 1,500 pages per hour per node, and the latency is significantly increased, resulting in a comprehensive fit score of about 0.08. This row is updated to 0.95 for the GPU computing pool and 0.08 for the CPU computing pool. Based on the updated matrix, the ratio ρ of the adaptation score between GPU computing power pool nodes and CPU computing power pool nodes is calculated for each task type. This ratio is compared with a threshold range to classify the tasks to be scheduled as GPU-sensitive, CPU-adapted, or hybrid: when ρ is greater than the threshold of 3.0, it is marked as GPU-sensitive; when ρ is less than the threshold of 0.33, it is marked as CPU-adapted; and when ρ falls between 0.33 and 3.0, it is marked as hybrid. Continuing the example, the format parsing task has ρ = 0.95 / 0.08 ≈ 11.9, which is much greater than the threshold of 3.0, and is therefore classified as GPU-sensitive; the paper crawling task has ρ = 0.3 / 0.9 ≈ 0.33, which is close to the lower threshold, and is therefore classified as CPU-adapted; semantic annotation and data cleaning are similarly classified as GPU-sensitive and CPU-adapted, respectively. The updated task-resource adaptation score matrix contains computing power adaptation category information for each type of task.

[0036] Step S3: A scheduling framework including a priority decision layer and a node allocation layer is adopted. The priority decision layer outputs the priority score of the task to be scheduled based on the cluster state vector, and generates a priority ranking based on the priority score. The node allocation layer solves the node allocation scheme based on the priority ranking, the updated task-resource adaptability score matrix and the computing power adaptability category, and generates a scheduling decision based on the node allocation scheme.

[0037] After obtaining the updated task-resource fit score matrix and computing power fit category, the two-layer decoupled scheduling phase begins. The priority decision layer constructs a cluster state vector by integrating the current cluster resource occupancy, task queue depth, and the fit status of the tasks to be scheduled. A reinforcement learning policy network performs forward reasoning on the state vector and task attribute information, outputting the priority score for each task to be scheduled and generating a priority ranking from high to low. The node allocation layer traverses the tasks to be scheduled sequentially according to their priority ranking, filtering candidate node sets based on the computing power fit category of each task. It performs a combined optimization solution with the objective of minimizing the weighted sum of completion times and constraints of node resource capacity and unique allocation, obtaining the target allocation node for each task, thus forming a node allocation scheme. Based on the node allocation scheme, hard affinity rules are generated for GPU-sensitive tasks, forcing them to be scheduled to the GPU computing power pool; soft affinity rules are generated for CPU-fit tasks, pointing them to the CPU computing power pool. These two rules together constitute the scheduling decision.

[0038] Step S4: Execute the task to be scheduled according to the scheduling decision, collect measured performance indicators, and calibrate the updated task-resource fit score matrix to obtain the calibrated task-resource fit score matrix.

[0039] After the scheduling decision is submitted to the Application Programming Interface (API) server of the container orchestration system, the task to be scheduled is scheduled to the target allocation node for execution. During task execution, the actual execution time and resource utilization on the execution node are collected as measured performance indicators. This measured performance is then compared with the adaptation score indicated by the corresponding cross-element in the updated task-resource fit scoring matrix to calculate the scoring error. When the scoring error exceeds the calibration threshold of 10%, an exponential smoothing algorithm is used to update the corresponding elements of the matrix and clear the calibration status markers, thereby obtaining the calibrated task-resource fit scoring matrix.

[0040] Step S5: Collect cluster status data and input it into the load prediction model, output the prediction value, and when the output prediction value meets the preset intervention conditions, execute the scheduling intervention action, which includes adjusting the node allocation scheme according to the calibrated task-resource adaptability score matrix.

[0041] During continuous task execution, load time-series data from each computing pool is collected periodically as cluster status data and input into the load prediction model. The model outputs predicted load values ​​for each computing pool and predicted completion times for each scheduled task over a future period. When the predicted values ​​meet the corresponding intervention trigger conditions (e.g., the average utilization rate of the graphics processor computing pool exceeds 85% threshold in the next 5 minutes), scheduling intervention actions are executed: triggering automatic cluster node expansion, increasing the affinity weight of some tasks in the central processing unit computing pool, generating high-priority task preemption suggestions, and withdrawing scheduled but unexecuted tasks for re-solution. All of the above intervention actions adjust the node allocation scheme based on the calibrated task-resource fit scoring matrix, transforming the scheduling system from a passive response to an active intervention.

[0042] Example 2

[0043] Based on Example 1, the step of grouping computing nodes in the heterogeneous computing power cluster according to computing power type to obtain computing power pools, and constructing and initializing a task-resource suitability scoring matrix based on preset task types and the types of nodes in the computing power pools, includes:

[0044] Step S1.1: Read the hardware configuration information of each computing node in the heterogeneous computing power cluster. If the computing node is equipped with a graphics processor accelerator card, mark the computing node as a graphics processor computing power pool node. If the computing node is only equipped with a central processing unit, mark the computing node as a central processing unit computing power pool node to obtain the computing power pool.

[0045] The system calls the node information listener of the container orchestration to read the hardware configuration list of each compute node. For compute nodes equipped with graphics processing unit (GPU) accelerator cards, a resource-tier tag `resource-tier=gpu-pool` is written to them, along with three sub-tags: accelerator card model, video memory capacity, and number of accelerator cards per node. For example, a node's tag set might be `{resource-tier: gpu-pool, gpu-model: A100, gpu-memory: 80, gpu-count: 4}`. The node is then assigned to the GPU computing power pool. For compute nodes equipped only with a CPU, the tag `resource-tier=cpu-pool` is written, along with two sub-tags: the number of physical cores and memory bandwidth. For example, `{resource-tier:cpu-pool, cpu-cores: 128, mem-bandwidth: 200}`. The node is then assigned to the CPU computing power pool.

[0046] Step S1.2: Construct the task-resource fit score matrix with the preset task type as the row and each node type in the computing power pool as the column. Based on the preset affinity relationship between the preset task type and each node type in the computing power pool, set the initial score of the corresponding element in the task-resource fit score matrix, and mark all the initial scores as pending calibration to complete the initialization.

[0047] Based on the dual-pool computing power resource view, the scheduling controller constructs a two-dimensional scoring matrix TRASM, with the number of rows equal to the number of task types and the number of columns equal to the number of node types, using preset task types as row indices and the GPU and CPU computing power pool node types as column indices. The matrix elements range from 0 to 1, representing the normalized performance adaptation score of a certain type of task on a certain type of node. During the cold start phase, when historical execution data is lacking, the matrix elements are assigned default scores based on the preset affinity relationship between tasks and nodes: GPU-sensitive tasks (such as format parsing and semantic annotation) have an initial adaptation score of 1.0 in the GPU computing power pool column and 0.1 in the CPU computing power pool column; CPU-adaptive tasks (such as paper crawling and data cleaning) have a score of 0.9 in the CPU computing power pool column and 0.3 in the GPU computing power pool column; and mixed tasks have an initial score of 0.5 in both types of node columns. All default scores are uniformly marked as pending calibration, indicating that they have not yet been verified by actual test data. Taking the example cluster and four types of tasks in Example 1 as examples, the initialized task-resource adaptation score matrix is ​​as follows: Paper crawling task has an adaptation score of 0.3 in the GPU computing power pool and 0.9 in the CPU computing power pool (to be calibrated); Format parsing task has an adaptation score of 1.0 in the GPU computing power pool and 0.1 in the CPU computing power pool (to be calibrated); Semantic annotation task has an adaptation score of 1.0 in the GPU computing power pool and 0.1 in the CPU computing power pool (to be calibrated); Data cleaning task has an adaptation score of 0.3 in the GPU computing power pool and 0.9 in the CPU computing power pool (to be calibrated).

[0048] In one implementation, the preset task types are not a fixed enumeration, but are declaratively registered to the scheduling controller by the system administrator through a configuration file during the deployment phase. Each task type corresponds to a type identifier (such as "ocr-inference", "data-crawl") and several descriptive attributes (such as typical computing mode, whether it depends on an accelerator card, historical throughput reference value, etc.). When a container group is submitted, the user writes the corresponding task type identifier in the orchestration manifest metadata field, thereby completing the task classification and passing the type information along with the task to the suitability evaluation process. This declarative registration mechanism allows the task type system to be dynamically expanded during cluster operation. When adding a new task type, it is only necessary to register the type identifier, append the corresponding row to the scoring matrix, and set a default initial score, without stopping and restarting the scheduling controller.

[0049] The granularity of the computing power pool is not limited to graphics processing units (GPUs) and central processing units (CPUs). In heterogeneous clusters equipped with dedicated acceleration hardware such as Field-Programmable Gate Array (FPGA) accelerator cards or Tensor Processing Units (TPUs), the same node information detection logic can be used to write corresponding resource-level tags (e.g., resource-tier=fpga-pool) to the corresponding nodes, and add corresponding columns to the task-resource suitability scoring matrix. This extends the computing power pool and scoring matrix mechanism to heterogeneous hardware scenarios with more than two categories, and the overall process is consistent with the GPU and CPU pool scenarios.

[0050] Example 3

[0051] Based on Examples 1 and 2, the step of calculating the suitability score of the task to be scheduled on the computing power pool node to update the task-resource suitability score matrix, resulting in an updated task-resource suitability score matrix; and determining the computing power suitability category based on the ratio of the suitability scores on nodes corresponding to different computing power pools, including:

[0052] Step S2.1: For the scheduled tasks belonging to each of the preset task types, obtain the task feature vector of the scheduled task and the resource status parameters of each node in the computing power pool. Use the fit evaluation function to calculate the task feature vector and the resource status parameters of each node in the computing power pool to obtain the fit score of the scheduled task on each node in the computing power pool. Update the row corresponding to the scheduled task in the task-resource fit score matrix with the fit score to obtain the updated task-resource fit score matrix.

[0053] The task-resource fit evaluation function takes the actual throughput, actual latency, and actual resource utilization of the task on the node as input. After normalizing the three types of performance, they are weighted and fused to obtain the updated value of the corresponding element in the task-resource fit score matrix. The update formula is as follows:

[0054]

[0055] in, The normalized adaptation score for task i on node j is denoted by w1, with a value ranging from 0 to 1. w1, w2, and w3 are the fusion weights of the three components: throughput, latency, and utilization, respectively, satisfying w1+w2+w3=1. The initial values ​​are set to 0.4, 0.4, and 0.2, respectively, and can be adjusted by the administrator according to business priority. Scoring is given for the throughput component; Score for delayed components; The utilization rate component is scored.

[0056] The formulas for calculating the scores of the three components are as follows:

[0057]

[0058]

[0059]

[0060] in, The measured throughput of task i on node type j; The throughput of task i on the optimal hardware is used for normalization; The measured latency of task i on node type j; The minimum latency for task i on optimal hardware; The resource utilization rate when task i runs on node type j; This represents the maximum available resources for a node, used for normalization.

[0061] Taking the optical character recognition inference task as an example, its measured throughput on GPU pool nodes is approximately 15,000 pages per node per hour, and the optimal hardware throughput is taken as the same order of magnitude. Approximately 1.0, due to a lack of accelerator cards in the CPU pool nodes. The score is below 0.1; similarly, the latency and utilization components are analyzed together, resulting in an adaptation score of approximately 0.95 for the GPU pool and approximately 0.08 for the CPU pool. These scores are then written into the corresponding rows of the scoring matrix to obtain the updated task-resource adaptation score matrix.

[0062] Specifically, the task feature vector is constructed as follows: For each task to be scheduled, the task type identifier, input data size (in MB), and estimated computational load (number of floating-point operations) are read from the container orchestration list. The current adaptation score of the task type on various types of nodes is then queried from the current state of the scoring matrix. This information is concatenated into a task feature vector, with the dimension consistent with the task feature sub-vector in the telemetry feature vector. Resource status parameters are provided by real-time telemetry data from each computing pool node, including CPU utilization, memory usage ratio, accelerator card utilization, and network bandwidth utilization. The adaptation evaluation function takes the task feature vector and resource status parameters as input, outputs the task's adaptation score on the corresponding node type using the aforementioned three-component weighted formula, and writes the score into the corresponding column of the corresponding row of the matrix, thus obtaining the updated task-resource adaptation score matrix. This matrix serves as the basic data for calculating the task adaptation ratio and determining the computing power adaptation category.

[0063] Step S2.2: Calculate the ratio of the adaptation score of the task to be scheduled on the graphics processor computing power pool node to the adaptation score on the central processing unit computing power pool node, compare the ratio with at least one preset threshold interval, and mark the task to be scheduled as the corresponding computing power adaptation category according to the preset threshold interval to which the ratio belongs, wherein the computing power adaptation category includes at least: graphics processor sensitive type, central processing unit adapted type and hybrid type.

[0064] Based on the updated matrix, calculate the ratio of the task's fit score in the GPU pool to that in the CPU pool:

[0065]

[0066] in, The GPU to CPU adaptation ratio for the i-th type of task; The adaptation score for task i on the GPU pool node; This is the adaptation score for task i on the CPU pool node. Continuing the example above, this optical character recognition task... The ratio is approximately 11.9, significantly higher than the threshold of 3.0. Comparing this ratio to the threshold range: when... When the value is greater than 3.0, it is marked as graphics processor sensitive. When ρ is less than 0.33, it is marked as CPU-adapted type; when ρ is between 0.33 and 3.0, it is marked as hybrid type.

[0067] The adaptation scores and ratios of the four types of tasks in the example cluster of Example 1 are summarized, and the classification results are as follows: For the paper crawling task, the graphics processor computing power pool adaptation score is 0.28, and the central processing unit computing power pool adaptation score is 0.91. =0.31, indicating it's a CPU-adapted type; Format parsing task GPU computing power pool 0.95, CPU computing power pool 0.08, =11.9, classified as GPU-sensitive; semantic annotation task GPU computing power pool 0.96, CPU computing power pool 0.07, =13.7, classified as GPU-sensitive; data cleaning task GPU computing power pool 0.31, CPU computing power pool 0.88, =0.35, indicating it is CPU-adapted. After the computing power adaptation categories for various tasks are marked, the node allocation layer can use these categories to filter the set of candidate nodes for the tasks to be scheduled.

[0068] It should be noted that the preset threshold θ=3.0 is not a fixed constant. System administrators can adjust it based on the value ratio of GPU / CPU resources and the scheduling flexibility required by the business scenario. Increasing the θ value (e.g., setting it to 5.0) will narrow the scope of GPU-sensitive tasks, causing more tasks to fall into the hybrid category, thus achieving greater scheduling flexibility. Conversely, decreasing the θ value (e.g., setting it to 2.0) will force more tasks to be bound to the GPU computing power pool, suitable for scenarios with abundant GPU resources and extremely strict requirements on inference latency. After adjusting the threshold, no other modifications are needed to take effect in the next scheduling window, providing real-time dynamic response capabilities.

[0069] Example 4

[0070] like Figure 2 As shown, based on Embodiment 1, the scheduling framework employing a priority decision layer and a node allocation layer is described. The priority decision layer outputs a priority score for the task to be scheduled based on the cluster state vector, and generates a priority ranking based on the priority score. This includes:

[0071] Step S3.1: Obtain the waiting time, computational load, and deadline of the task to be scheduled as task attribute information.

[0072] The waiting time, computational cost, and deadline for each task to be scheduled are obtained and, together with real-time cluster telemetry, constitute the input for scheduling decisions. The waiting time reflects the task's lingering position in the queue, the computational cost is used to estimate the completion cost, and the deadline is used to identify time-sensitive, high-priority tasks. These three attributes, along with the cluster state vector, are input into the reinforcement learning policy network to participate in the forward inference for priority scoring.

[0073] Step S3.2: Construct the cluster state vector, which contains multiple feature components representing the computing power pool resource status, task queue status, and the adaptation status of tasks to be scheduled.

[0074] The cluster state vector s consists of eight feature components:

[0075]

[0076] Wherein, s1 is the average utilization of the GPU pool; s2 is the peak utilization of the GPU pool; s3 is the average utilization of the CPU pool; s4 is the GPU task queue depth; s5 is the CPU task queue depth; s6 is the highest GPU adaptation score of the currently scheduled task; s7 is the highest CPU adaptation score of the currently scheduled task; and s8 is the number of high-priority scheduled tasks. The state vector is derived from the real-time telemetry and the queue of tasks to be scheduled for each computing power pool, and is refreshed every 60 seconds. This cluster state vector, together with the aforementioned task attribute information, constitutes the input of the priority decision layer reinforcement learning policy network, which is used for forward inference to generate the priority ranking of each task to be scheduled.

[0077] Step S3.3: Input the task attribute information and the cluster state vector into the reinforcement learning policy network that constitutes the priority decision layer. The reinforcement learning policy network performs forward inference, outputs the priority score of the task to be scheduled, and generates a priority ranking based on the priority score. The reinforcement learning policy network is trained using an ε-greedy exploration strategy and an experience replay mechanism.

[0078] The priority decision layer employs a reinforcement learning policy network based on a multilayer perceptron (a deep Q-network framework approximating the action-value function). This policy network has an input dimension of 8 (corresponding to the state vector), and two fully connected hidden layers: the first hidden layer has 64 neurons, and the second hidden layer has 32 neurons, both using the Rectified Linear Unit (ReLU) activation function. The output dimension equals the number of tasks currently to be scheduled, and each output is a priority score (value from 0 to 1) for that task. During training, an ε-greedy exploration strategy is used, with the exploration rate ε linearly decreasing from 0.3 to 0.05 every 1000 steps. An experience replay buffer with a capacity of 10,000 trajectories is maintained, and network parameters are periodically updated through batch sampling to mitigate distribution drift. The reward function r consists of three composite terms:

[0079]

[0080] Where r is the single-step reward value; α, β, The reward weights for the three items—improved task completion time, increased utilization, and breach of service level agreement—are set to initial values ​​of 0.5, 0.3, and 0.2, respectively. This represents the increment of task completion time relative to the baseline plan for the current scheduling decision; a negative value indicates improvement. This represents the increase in GPU utilization. This refers to the number of service level agreement (SLA) breaches caused by scheduling delays.

[0081] The network training employs the mean squared loss function of temporal difference error, that is, the squared difference between the temporal difference objective and the estimated value of the online network action is used as the loss for optimization; the temporal difference objective is composed of the single-step reward r and the discounted estimate of the value of the optimal action in the next state, i.e. , where r is the aforementioned single-step reward value (defined in the aforementioned reward function). Set the discount factor to 0.9. This is for estimating the action value of the optimal action a' in the next state s' for the target network. The parameters are for the target network. The training hyperparameters are configured as follows: the initial learning rate is 2×10⁻⁶. -5 The batch size is 64, the training epochs are 200, the optimizer used is Adam, and the weight decay factor is 1×10. -4 The experience replay buffer capacity is 10,000. The training dataset consists of the cluster's historical scheduling trajectories, divided into training, validation, and test sets in an 8:1:1 ratio, totaling approximately 200,000 state-action-reward samples.

[0082] During inference, task attribute information and cluster state vectors are input into the trained policy network. The network performs forward inference and outputs priority scores for each task to be scheduled. These scores are then sorted from highest to lowest to generate a priority ranking. For example, after inputting the state vectors, the network outputs priority scores of 0.92, 0.61, and 0.34 for three tasks, resulting in the ranking as Task 1, Task 2, and Task 3. The priority decision layer then submits this ranking to the node allocation layer.

[0083] Specifically, the cold start phase of the reinforcement learning policy network ensures scheduling stability through the following methods: In the early stages of training (when the number of samples in the experience replay buffer is less than the minimum batch size of 64), the network outputs an initial priority ranking based on a heuristic rule of task deadlines and waiting times (i.e., a weighted combination of earliest deadline priority and longest waiting time priority), while simultaneously exploring random actions with an exploration rate ε=0.3 and writing the trajectory into the replay buffer; once the number of samples in the buffer reaches the minimum batch size, the network automatically switches to inference mode and continuously updates the parameters. This mechanism ensures that a reasonable priority ranking can be generated immediately in a completely new deployment scenario with zero historical data, and gradually converges to the steady-state output of the policy network as it accumulates operational experience.

[0084] It should be understood that the task computation volume in the task attribute information can be declared by the user when the task is submitted (e.g., through the orchestration list annotation field), or it can be automatically estimated by the scheduling controller based on the historical average of the input data size and task type. For new task types without historical data, the global average computation volume of the category to which the task type belongs (GPU sensitive, CPU adapted, or hybrid) is used as the initial estimate, which is then replaced by the actual test results after the first execution.

[0085] Example 5

[0086] Based on Embodiments 1, 3, and 4, the node allocation layer solves for a node allocation scheme according to the priority ranking, the updated task-resource suitability score matrix, and the computing power suitability category, and generates a scheduling decision based on the node allocation scheme, including:

[0087] Step S3.4: The node allocation layer traverses the tasks to be scheduled in sequence according to the priority order. For each task to be scheduled, the corresponding computing power pool is selected according to the computing power adaptation category to obtain a set of candidate nodes. The allocation relationship between the tasks to be scheduled and the nodes in the set of candidate nodes is used as a decision variable.

[0088] After receiving the priority ranking submitted by the priority decision layer, the node allocation layer processes each scheduled task sequentially, starting with the highest priority. For GPU-sensitive tasks, the candidate node set is limited to GPU pool nodes; for CPU-adaptive tasks, the candidate node set is CPU pool nodes (allowing overflow to the GPU pool when resources are insufficient); for mixed tasks, the candidate set is all nodes. Whether task k is assigned to node n is represented as a binary decision variable (denoted as ). (A value of 1 indicates that task k is assigned to node n, and a value of 0 indicates that it is not assigned). The node assignment layer solves for this binary decision variable when constructing the objective function and constraints to obtain its optimal value.

[0089] Step S3.5: Based on the decision variables and the available resources of each node in the candidate node set, with the objective function of minimizing the weighted sum of the completion times of the tasks to be scheduled, and with the available resources and unique node allocation as constraints, construct a combinatorial optimization problem and solve it to obtain the optimal value of the decision variables. The unique node allocation is used to restrict the tasks to be scheduled to be assigned to only one node.

[0090] The optimization objective is to minimize the sum of the weighted completion times of all tasks, where The priority weight of task k is obtained by mapping the priority score output by the priority decision layer, with higher priority tasks having higher weights. Let $k$ be the estimated completion time for task $k$. Constraints include: each task is assigned to exactly one node; the sum of GPU, CPU, and memory resource requests for each node does not exceed its corresponding capacity; GPU-sensitive tasks can only be assigned to GPU computing pool nodes (hard constraint); CPU-adaptive tasks are preferentially assigned to CPU computing pool nodes, and are allowed to be assigned to GPU computing pool nodes when resources are insufficient, with a unit penalty applied to the objective function through slack variables. The combinatorial optimization problem is solved by a Mixed Integer Linear Programming (MILP) solver, which finds the optimal decision variable values ​​in sub-second time. The optimal values ​​of the decision variables indicate the target node for each task. The combinatorial optimization problem is also constrained by node resource capacity: the sum of GPU, CPU, and memory resource requests for each node does not exceed its corresponding capacity, where the number of GPUs, CPU cores, and memory requested by task $k$ is denoted as $k$. , , The capacity of node n is denoted as , , In the example cluster of Example 1, each of the 8 graphics processor nodes contains 4 accelerator cards and each of the 32 central processing unit nodes contains 128 cores. The solver constructs constraints based on the above node resource capacity, obtains the optimal values ​​of the decision variables, and specifies the target allocation node for each task.

[0091] Specifically, the objective function of the above combinatorial optimization problem is formally expressed as follows:

[0092]

[0093] in, , This is the estimated processing time for task k on node n (estimated by the task's computational load and the node's computing power characteristics). Let n be the current queuing time for node n; the unique allocation constraint for nodes is: (For all tasks k); node resource capacity constraints are: , , (For all nodes n). The solver uses the branch and bound method to search for the optimal integer solution in the constraint space. For the typical scale of the example cluster in Example 1, where the number of tasks to be scheduled does not exceed 100 and the number of candidate nodes does not exceed 40, the solution time can be controlled within 500 milliseconds.

[0094] Step S3.6: Determine the target allocation node for the task to be scheduled from the candidate node set according to the allocation relationship indicated by the optimal value of the decision variable.

[0095] From decision variables The optimal value of indicates the node n to which task k is assigned (i.e., the node corresponding to a value of 1), thus ensuring that each task to be scheduled has a unique target assignment node in the candidate node set. Taking a single solution as an example, three high-priority fusion and alignment tasks are mapped to the three least utilized nodes in the GPU pool to meet load balancing constraints, eight image segmentation tasks are assigned to the remaining GPU nodes, some point cloud annotation tasks are temporarily assigned to the CPU pool for accelerated processing because the CPU pool adaptation ratio is close to the threshold, and all twenty cleaning tasks are mapped to the CPU pool. Therefore, each task to be scheduled obtains a unique target assignment node in the candidate node set.

[0096] Step S3.7: After traversing the tasks to be scheduled, generate the node allocation scheme according to the correspondence between the tasks to be scheduled and the target allocation node, and generate a scheduling decision based on the node allocation scheme.

[0097] After traversing all tasks to be scheduled and obtaining the target allocation node indicated by the optimal value of the decision variable for each task, a node allocation scheme is generated based on the correspondence between tasks and target allocation nodes. The node allocation scheme is recorded in the form of a structured mapping table: each row of the mapping table corresponds to a task to be scheduled, and the fields in the row include the task identifier, the node identifier of the target allocation node, the resource quota of three types on the target node (number of graphics processing units / number of CPU cores / amount of memory), and the affinity type label derived from the computing power adaptation category, where graphics processing unit sensitive type is denoted as hard affinity, CPU adaptation type is denoted as soft affinity, and hybrid type is denoted as double soft affinity. Taking the example cluster solution result of Example 1 as an example, the node allocation scheme includes: the 8 replicas of the format parsing task are mapped to 8 graphics processing units nodes respectively, each replica occupies 2 accelerator cards and is marked as hard affinity; the paper crawling and data cleaning tasks are distributed to 32 CPU nodes and are marked as soft affinity; a small number of point cloud annotation tasks are temporarily mapped to CPU nodes because the adaptation ratio is close to the threshold, and are also marked as soft affinity. This mapping table fully records the target locations of all tasks within this scheduling window.

[0098] Based on the node allocation scheme, scheduling decisions are further generated. These decisions are expressed as a set of container orchestration lists. For each row in the node allocation scheme, a corresponding draft container orchestration list is generated. The draft list includes the task's type identifier, target node identifier or node selector, and affinity constraint placeholders determined by affinity type flags. Hard affinity placeholders constrain the placement in the graphics processor (GPU) computing pool, while soft affinity placeholders favor the placement in the CPU computing pool. The scheduling decisions are summarized into a set of pending lists that can be received by the container orchestration system's application programming interface (API). The placement target and affinity preference for each task are determined within this set.

[0099] Example 6

[0100] Based on Example 5, the constraints also include:

[0101] Step S3.5.1: If the computing power adaptation category of the task to be scheduled is graphics processor sensitive, then the task to be scheduled is restricted to be assigned only to the graphics processor computing power pool node.

[0102] When constructing combinatorial optimization problems, the mixed-integer linear programming solver forces the decision variable of a scheduled task k with a GPU-sensitive computing power adaptation category to be 0 on all CPU computing power pool nodes, and only allows it to be 1 on GPU computing power pool nodes. In other words, the decision variable of this task is always 0 on all CPU pool nodes. Here, k is the GPU-sensitive scheduled task number. A collection of central processing unit computing power pool nodes. The binary decision variable defined in Example 5 (a value of 1 indicates that task k is assigned to node n, and a value of 0 indicates that it is not assigned) ensures that GPU-sensitive inference tasks are not mistakenly assigned to CPU nodes lacking accelerator cards, fundamentally preventing GPU computing power from being squeezed out by low-performance tasks. Taking the format parsing task in Example 1 as an example, it is determined to be GPU-sensitive. After the solver sets all 32 CPU nodes to 0, only 8 GPU nodes remain to participate in the allocation, and finally, its 8 replicas are accurately mapped to 8 GPU nodes.

[0103] Step S3.5.2: If the computing power adaptation category of the task to be scheduled is CPU adaptation type, it is preferentially allocated to the CPU computing power pool node. When the available resources of the CPU computing power pool node are insufficient, it is allowed to be allocated to the GPU computing power pool node and a penalty term is applied to the objective function.

[0104] For scheduled tasks k with a CPU-adapted computing power category, the solver defaults to setting its candidate node set as CPU computing power pool nodes, thus physically isolating crawling and cleaning tasks from GPU inference workloads. When the sum of available resources in the CPU computing power pool nodes is lower than the total number of task requests, the solver relaxes the restrictions, allowing tasks to be allocated to GPU computing power pool nodes. Simultaneously, a slack variable is introduced into the objective function, and a unit penalty term is applied, ensuring that overflow allocation only occurs when CPU resources are truly unavailable, and that the overflow scale is suppressed by the penalty term. This mechanism, based on the objective function that minimizes the sum of weighted completion times, introduces a unit penalty sub-term for overflow allocation of CPU-adapted tasks: Indicates a set of central processing unit-adapted tasks. This is a binary overflow indicator variable (1 when task k is overflowed and allocated to a GPU computing power pool node, 0 otherwise). Each overflow adds a unit penalty φ=1.0 to the objective function. Taking the paper crawling task in Example 1 as an example, when the total available resources of its 32 CPU nodes are lower than the total number of crawling job requests, the solver allows some crawling jobs to overflow to GPU nodes. However, for each additional overflow allocation, a penalty term φ=1.0 is added to the objective function, thus forcing the solver to prioritize exhausting CPU resources before considering overflow, ensuring that GPU computing power primarily serves inference tasks.

[0105] In one implementation, the overflow penalty coefficient φ can be adjusted by the administrator based on the business's sensitivity to GPU resource exclusivity. Setting φ to a larger value (e.g., 10.0) will cause the solver to reject overflow allocations in almost all scenarios, allowing only a small amount of overflow in extreme cases where no available CPU resources are available. Setting φ to a smaller value (e.g., 0.5) will cause the solver to actively choose overflow to shorten task waiting time when the completion time benefit is significant. (Penalty coefficient and task weight) Together, they determine the threshold for triggering overflow behavior. Both parameters are used in the objective function as floating-point numbers, and adjusting either parameter will take effect immediately in the next scheduling window without restarting the solver.

[0106] Example 7

[0107] Based on Example 5, the step of generating scheduling decisions according to the node allocation scheme includes:

[0108] Step S3.7.1: If the computing power adaptation category of the task to be scheduled is graphics processor sensitive, then a scheduling decision containing a first affinity rule is generated. The first affinity rule is used to force the task to be scheduled to be scheduled to a graphics processor computing power pool node.

[0109] For GPU-sensitive tasks, a hard node affinity constraint, namely the `requiredDuringSchedulingIgnoredDuringExecution` rule, is written into the container orchestration list. This rule uses the node tag key `resource-tier` and the value set `{gpu-pool}` to form a matching expression, requiring the container group to be scheduled to nodes with the GPU computing pool tag. Simultaneously, the number of accelerator cards required by the GPU device plugin is declared in the resource limit field `resources.limits` (field name `nvidia.com / gpu`), and the device plugin handles the allocation and binding of accelerator cards. Taking the format parsing service in Example 1 as an example, the container orchestration list of its eight replicas all includes the above hard affinity constraint and declares `nvidia.com / gpu=2`. The first affinity rule ensures that the optical character recognition and visual language model inference services are accurately positioned to GPU computing pool nodes, with each node occupying two accelerator cards.

[0110] Step S3.7.2: If the computing power adaptation category of the task to be scheduled is CPU adaptation type, then generate a scheduling decision containing a second affinity rule. The weight value of the second affinity rule is determined based on the corresponding adaptation score in the task-resource adaptation score matrix.

[0111] For CPU-adaptive tasks, a soft node affinity preference is written, namely the `preferredDuringSchedulingIgnoredDuringExecution` rule. This rule also uses the node tag key `resource-tier` and the value set `{cpu-pool}` to construct a matching expression, but it is a soft constraint rather than a mandatory one. The preference weight of the second affinity rule is an integer, calculated as the task's node adaptation score in the CPU computing power pool multiplied by 100 and then rounded down. .

[0112] Taking a certain web crawling task as an example, its adaptation score in the CPU computing power pool is about 0.28, so the soft affinity weight is set to 28, guiding the scheduler to allocate it to the CPU computing power pool node without forcing it, thus allowing overflow when the graphics processor resources are idle to take into account elasticity; another data cleaning task has an adaptation score of about 0.91, and the soft affinity weight is set to 91, so the scheduler strongly prefers to fix it to the CPU computing power pool node.

[0113] The affinity rules for hybrid tasks are generated as follows: Two soft affinity preferences, pointing to both the GPU computing pool and the CPU computing pool, are simultaneously written into the container group orchestration list for the task. The weights of each rule are determined by... When selecting a node for a task, the container orchestration system comprehensively evaluates the scores of each candidate node based on the weight difference between the two rules, and prioritizes scheduling the task to the node type with the higher comprehensive score. This enables dynamic allocation between GPU and CPU resources based on the measured adaptation score, taking into account both current resource availability and historical performance.

[0114] Furthermore, the weight values ​​in the affinity rules are updated in real time with each calibration of the scoring matrix: whenever the corresponding element of the scoring matrix changes due to the write-back of actual test data, the affinity weight of the corresponding task type is recalculated synchronously, and the new weight is written to the deployed but not yet completed container group orchestration list through the resource update interface of the container orchestration system, ensuring that the weight dynamically reflects the latest adaptation relationship without resubmitting the task.

[0115] Example 8

[0116] like Figure 3 As shown, based on Embodiments 1 and 2, the step of executing the task to be scheduled according to the scheduling decision, collecting measured performance indicators, and calibrating the updated task-resource fit scoring matrix to obtain the calibrated task-resource fit scoring matrix includes:

[0117] Step S4.1: During the execution of the task to be scheduled according to the scheduling decision, the actual execution time and actual resource utilization rate of the task to be scheduled on the execution node indicated by the scheduling decision are collected as the measured performance index.

[0118] During the execution of the task to be scheduled on the target allocation node, its runtime data is continuously collected as a measured performance indicator. Data is collected at a 10-second sampling period, recording continuously throughout the entire execution lifecycle of the task: Regarding actual execution time, data is recorded from the moment the task container group enters the running state until all subtasks exit, yielding the end-to-end actual execution time. (i.e., the actual measured delay during this execution) (The measured latency sign is consistent with that of the adaptation evaluation function in Example 3), and the segmented time consumption of each stage of data retrieval, preprocessing, core calculation and result write-back is recorded synchronously; in terms of resource utilization, the monitoring endpoint of the container runtime interface is read, and the time series of CPU utilization, memory usage, accelerator card utilization (including VRAM usage and tensor core usage) and network throughput are collected, and the mean and peak values ​​are calculated according to the execution window. In order to suppress the noise introduced by short-term jitter, the resource utilization sequence is smoothed by removing extreme values ​​after the task ends, and the mean value after removing extreme values ​​is taken as the steady-state resource utilization of this execution. The above end-to-end actual execution time and steady-state resource utilization together constitute the measured performance index of this execution, which serves as the basis for comparing the measured performance with the estimated value of the scoring matrix, calculating the scoring error and determining whether to update the matrix elements.

[0119] Step S4.2: Extract the element at the intersection of the row corresponding to the task to be scheduled and the column corresponding to the computing power pool to which the execution node belongs from the updated task-resource fit score matrix as the target element.

[0120] After obtaining the measured performance metrics, locate and extract the target element corresponding to this task from the updated task-resource fit score matrix. The localization process is as follows: the row index is determined by the type identifier of the task to be scheduled. This identifier was registered to the scheduling context with the task submission in Example 2 and can be directly mapped to the corresponding task type row in the scoring matrix; the column index is determined by the computing pool type to which the execution node belongs. The resource-tier tag (gpu-pool or cpu-pool) of the node is read, and the node is assigned to the corresponding computing pool column. The intersection of the row index and the column index is the current fitness estimate of this task on this node type, denoted as the target element. This element is the current fitness estimate written by the fitness evaluation function in Example 3 and possibly adjusted in subsequent scheduling windows, representing the expected performance of the task on this type of node before this test.

[0121] Step S4.3: Compare the measured performance index with the fit score indicated by the target element, and calculate the score error.

[0122] Compare the measured performance indicators with the target elements The indicated fit score is compared to the actual fit score, and the scoring error is calculated. Specifically, the actual fit score of the task on this node type is first deduced from the measured performance. (The calculation method is obtained by weighting the actual execution time and steady-state resource utilization rate with the same weights as the fitness evaluation function, see the following paragraphs of this implementation for details); then the scoring error is calculated. It is defined as the ratio of the absolute value of the difference between the measured back-calculation score and the estimated value of the target element to the value of the target element:

[0123]

[0124] when When the value exceeds the calibration threshold by 10%, the current estimated value of the matrix is ​​considered to deviate too much from the true performance and requires correction; when... If the percentage is no greater than 10%, the estimated value is considered to be within the confidence range and the original value is maintained. Taking an optical character recognition task in Example 1 as an example, its GPU pool node target element... =0.95, obtained by reverse calculation from actual measurement. =1.0, scoring error The value did not exceed the 10% threshold, so the original value was maintained; another format parsing task =0.40、 =0.55, The value is approximately 27%, which significantly exceeds the threshold and requires matrix element updates.

[0125] Step S4.4: If the scoring error exceeds a preset threshold, the target element is updated using a preset exponential smoothing algorithm based on the scoring error, and the calibration status marker of the target element is cleared to complete the calibration and obtain the calibrated task-resource fit score matrix.

[0126] When the scoring error exceeds the calibration threshold of 10%, an exponential smoothing algorithm is used to weight the new observations and historical estimates with a coefficient of 0.3 to update the target element. Specifically, the updated value is the sum of 0.7 times the historical estimate and 0.3 times the measured back-calculated value, where the measured back-calculated value is denoted as... After the update is complete, the pending calibration status marker for the target element is cleared. Taking an optical character recognition task as an example, the original value of the target element in its GPU pool node is 0.95, and the actual measured value is... The original value for the task-resource fit score is 1.0, with a scoring error of 5%, which is within the 10% threshold and is therefore maintained. Another task had an original value of 0.40, a measured value of 0.55, and an error of approximately 27%, exceeding the threshold. It was updated to 0.445 with λ=0.3, and the calibration mark was removed, thus obtaining the calibrated task-resource fit score matrix. Taking a paper crawling task as another example, the original value of the target element in the central processing unit's computing power pool node was 0.90, and the measured value was 0.86, with an error of approximately 4%, which is within the 10% threshold and is therefore maintained. Another mixed task had an original value of 0.50, a measured value of 0.68, and an error of approximately 36%, exceeding the threshold. It was updated to 0.554 with λ=0.3, and the calibration mark was removed.

[0127] Specifically, the score was calculated by reverse engineering from the actual measurement. The calculation method is as follows: based on the actual throughput measured in this execution. Compared with the optimal throughput defined in Example 3 The ratio ( ) as the throughput component; the measured delay defined in Example 3 (i.e., the actual end-to-end execution time) ) and optimal delay The ratio ( () as the time component; based on the measured resource utilization rate With the node's maximum available resources The ratio ( As a utilization component, it is obtained by weighting and fusing w1, w2, and w3 with the same weights as the fitness evaluation function. ,Right now:

[0128]

[0129] in, This is the actual throughput measured during this execution. The throughput of the task defined in Example 3 on the optimal hardware (throughput component normalization benchmark, reused). The actual execution delay (i.e., the actual end-to-end execution time) was measured in this instance. ), Minimum latency (multiplexing) of the task defined in Example 3 on optimal hardware; This is the actual resource utilization rate measured during this execution. The maximum available resource amount (reuse) for a node is defined in Example 3; the weights w1, w2, and w3 are consistent with the weights used in the fit evaluation function in Example 3. The exponential smoothing update formula is:

[0130]

[0131] in, The updated matrix element values, These are the matrix element values ​​before the update. The smoothing coefficient is fixed at 0.3. This is achieved by controlling... The value of λ strikes a balance between the response speed to new samples and the stability of historical estimates: a larger λ results in matrix elements that are more sensitive to new samples and converge faster, but weakens the resistance to noise; a smaller λ results in higher weights for historical estimates and more stable matrix values, but slows down the adaptation speed to load distribution drift. The current value of λ=0.3 has been verified on the example cluster in Example 1 to converge the initial default score to within 5% of the error range of the true fit level after about 20 executions.

[0132] Example 9

[0133] Based on Example 7, the step of executing the scheduled task according to the scheduling decision further includes:

[0134] S4.5: For the scheduled task whose computing power adaptation category is CPU-adapted, the scheduled task is scheduled to the CPU computing power pool node according to the second affinity rule in the scheduling decision, so as to achieve physical isolation from the inference task on the GPU computing power pool node.

[0135] The container orchestration system schedules crawling jobs, Hypertext Transfer Protocol (HTTP) page extraction, and periodic cleaning tasks to CPU pool nodes based on the soft affinity preference in the second affinity rule. When selecting nodes for this batch of tasks, the scheduler prioritizes the soft affinity preference of resource-tier=cpu-pool and performs load balancing within the CPU pool nodes based on the current CPU utilization, memory utilization, and network throughput of each node, distributing tasks to idle nodes. Because these tasks and GPU inference services reside on different physical nodes, memory bandwidth contention and disk I / O contention on the CPU nodes do not affect the first-word latency and steady-state throughput of the inference service on the GPU nodes, thus ensuring the quality of the inference service. Taking the example cluster of Example 1 as an example, the two types of inference services, format parsing and semantic annotation, run on 8 GPU nodes, while paper crawling and data cleaning run on 32 CPU nodes. When the crawling job generates a large amount of disk input / output and memory bandwidth usage during peak periods, the GPU nodes are not affected by this due to physical isolation. The measured first-word latency of the inference service is stably maintained at about 300 milliseconds, and the steady-state throughput is about 120,000 pages / hour. The resource utilization (CPU utilization, memory utilization, network throughput) and the actual completion time of each task of the CPU pool nodes are continuously collected, and the collected results are periodically written into the scoring matrix calibration set and telemetry database, forming a closed-loop data flow from execution to calibration and then to prediction training. This closed-loop data flow provides samples for online calibration of the scoring matrix on the one hand, and labeled historical execution data for training the load prediction model on the other hand, so that the scheduling strategy continues to evolve towards the optimal adaptation as the operation accumulates.

[0136] Example 10

[0137] Based on Examples 1 and 2, the process of collecting cluster status data and inputting it into the load prediction model, outputting predicted values, and the load prediction model selecting between random forest regression and a two-layer fully connected neural network based on the cluster node size includes:

[0138] S5.1: Collect historical load time series data of each computing pool in the heterogeneous computing power cluster at fixed time intervals as the cluster status data.

[0139] With a fixed period of 60 seconds, load time series data of each computing pool is collected as cluster status data. Each sample consists of a 20-dimensional feature vector: 8-dimensional node features (average GPU utilization over 1 minute and 5 minutes, GPU memory usage ratio, average CPU utilization over 1 minute and 5 minutes, memory usage ratio, network bandwidth utilization, disk I / O utilization), 8-dimensional task features (5-dimensional task type one-hot encoding, input data size, estimated computational load, and the task's adaptation score on the node), and 4-dimensional queue features (GPU queue depth, CPU queue depth, average waiting time over 15 minutes, and number of GPU requests to be scheduled). This cluster status data serves as the input to the load prediction model for subsequent model selection and predictive inference.

[0140] S5.2: Obtain the number of nodes in the heterogeneous computing power cluster, and select the corresponding load prediction model based on the number of nodes. If the number of nodes is less than a preset threshold, select a random forest regression model as the load prediction model. If the number of nodes is greater than or equal to the preset threshold, select a two-layer fully connected neural network model as the load prediction model.

[0141] Select the load prediction model based on the cluster size: When the number of cluster nodes is less than the preset threshold of 50, select the random forest regression model (containing 100 decision trees), which has fast training speed and strong interpretability, and is suitable for cold start and rapid deployment; when the number of nodes is greater than or equal to 50, select the two-layer fully connected neural network model. The selected load prediction model is used to perform inference prediction on the cluster state data.

[0142] Specifically, the training method for the random forest regression model is as follows: The training set is based on historical execution data from the last 72 hours. Each sample consists of the aforementioned 20-dimensional feature vector and its corresponding actual completion time or load value. Bagging is used to resample the training sample set 100 times with replacement, each resampling yielding a subset of the training data. A decision tree is trained on this subset (maximum depth set to 15, minimum leaf node sample count set to 5). During inference, the average prediction results from the 100 trees are output to reduce overfitting risk and improve generalization ability. In small-scale clusters with fewer than 50 nodes, the 20-dimensional feature vector typically covers thousands to tens of thousands of training samples. A single full training run takes no more than 30 seconds, and incremental updates can be completed at the end of each scheduling window without affecting online inference.

[0143] S5.3: Input the cluster status data into the load prediction model, and the load prediction model outputs the predicted load value and prediction completion time of each computing pool within a preset time period as the predicted value.

[0144] The two-layer fully connected neural network model is a feedforward multilayer perceptron. The input layer has a dimension of 20, with 64 neurons in the first hidden layer and 32 neurons in the second hidden layer, both using the Rectified Linear Unit (ReLU) activation function. The output layer has a dimension of 1, corresponding to the prediction completion time or prediction load value. Each hidden layer first multiplies the output of the previous layer with its weight matrix and adds a bias, then activates the ReLU to obtain the hidden layer's output. The two-layer fully connected neural network model is trained using the mean squared error loss function.

[0145]

[0146] in, The mean squared error loss value; N is the batch size; The actual completion time or load value of the nth sample; These are the model's predicted values. The training dataset consists of execution history from the last 72 hours, totaling approximately 500,000 labeled samples, divided in an 8:1:1 ratio into a training set of 400,000 samples, a validation set of 50,000 samples, and a test set of 50,000 samples. Incremental training is triggered every 30 minutes using a sliding window. Training hyperparameters: Initial learning rate 2×10⁻⁶. -5 Batch size 128, training epochs 100, Adam optimizer used, weight decay factor 1×10⁻⁶ -4 The dropout rate is 0.1. During inference, the current 20-dimensional feature vector is input into a pre-trained two-layer fully connected neural network model, which outputs the predicted load values ​​of each computing pool and the predicted completion time of each scheduled task within the next 5 minutes as predicted values. Taking a cluster with 40 nodes as an example, after inputting the current feature vector, the model outputs a predicted average utilization rate of the GPU pool for the next 5 minutes of 0.91 and a predicted completion time of 150 seconds for a high-priority task in the GPU pool, providing a basis for subsequent intervention actions.

[0147] It's important to note that the switching between the two load prediction models doesn't only occur during the initial cluster deployment. When the cluster's node count fluctuates frequently around the threshold of 50 due to elastic scaling, a model switching hysteresis mechanism is implemented to avoid training overhead caused by frequent model switching: a switch to a two-layer fully connected neural network model is triggered only when the number of nodes continuously exceeds the threshold of 50 for more than 10 minutes; a rollback to a random forest regression model is triggered only when the number of nodes continuously falls below the threshold of 50 for more than 10 minutes. During model switching, the new model is initialized using transfer learning—the prediction residuals of the old model on the training set are used as additional feature channels to supplement the input vector of the new model, shortening the convergence time of the new model in the initial stage of switching.

[0148] Example 11

[0149] Based on Example 10, the execution of the scheduling intervention action includes:

[0150] Step A: When the predicted load value exceeds the preset high load threshold, the nodes of the heterogeneous computing power cluster are triggered to perform automatic expansion operations.

[0151] When the load prediction model outputs a GPU pool average utilization forecast for the next 5 minutes that exceeds the preset high load threshold of 85%, a scaling request is sent to the cluster auto-scaling controller. The target number of nodes is the larger of the current number of nodes plus 1 and the rounded-up value (predicted demand divided by 0.75). Where current is the current number of nodes in the graphics processor's computing pool. To predict the amount of graphics processor resources required, This is the floor function operator. Taking a prediction that the graphics processing unit (GPU) pool utilization will reach 0.91, with 6 nodes currently, as an example, the target number of nodes is... However, due to the constraint of the elastic scaling upper limit of 6, the queuing of high-priority tasks is avoided after the expanded nodes are ready. The node allocation scheme after expansion is readjusted according to the calibrated task-resource fit score matrix.

[0152] Step B: When the first prediction time is less than the second prediction time, and the difference between the first prediction time and the second prediction time is greater than a preset time difference threshold, the weight of the task to be scheduled in the CPU computing power pool corresponding to the task in the task-resource fit rating matrix is ​​increased. The first prediction time is the predicted completion time of the task to be scheduled in the CPU computing power pool, and the second prediction time is the predicted completion time of the task to be scheduled in the GPU computing power pool.

[0153] If the load forecasting model gives the predicted completion time of a task in the CPU pool... Less than its prediction completion time in the GPU pool. If the difference between the two is greater than a preset time difference threshold of 20%, then the adaptation score of the corresponding element in the CPU pool of the task in the scoring matrix will be temporarily increased (e.g., from 0.30 to 0.45), guiding the scheduler to prioritize allocating the task to the CPU pool and alleviate the pressure on the GPU pool. Taking the paper crawling task as an example, its predicted completion time in the central processing unit computing power pool... The predicted completion time is 40 seconds, based on the graphics processing unit's computing power pool. The time limit is 90 seconds. The difference between the two is 55.6%, which is greater than the threshold of 20%. Therefore, the adaptation score of the corresponding element of the task in the central processing unit computing power pool is temporarily increased from 0.30 to 0.45, so that the subsequent scheduling decision will be more inclined to fix it in the central processing unit computing power pool node, thereby relieving the load pressure on the graphics processing unit computing power pool.

[0154] Step C: When the priority score of the task to be scheduled is higher than the preset priority threshold, and the minimum predicted completion time is greater than the preset service level agreement threshold, a preemption suggestion is generated to prompt the task with a priority score lower than the preset priority threshold to be evicted.

[0155] If a high-priority task has a priority score higher than the preset priority threshold of 0.80 and its minimum predicted completion time is greater than the preset service level agreement threshold of 120 seconds, a preemption suggestion is generated, prompting the container group to be evicted from low-priority tasks with a priority score lower than 0.80, so as to ensure that the scheduling delay of high-priority inference tasks converges to the service level agreement target range.

[0156] Step D: Obtain the tasks that are currently scheduled but not yet executed as intervention tasks, withdraw the intervention tasks to the queue of tasks to be scheduled, recalculate the computing power adaptation category based on the calibrated task-resource adaptation score matrix, and trigger the scheduling framework to solve again.

[0157] By periodically scanning scheduled but not yet executed tasks, these tasks are returned to the queue of tasks awaiting scheduling. Their computing power suitability category is recalculated based on a calibrated task-resource suitability score matrix, and the two-layer decoupled scheduling framework is triggered to re-execute priority sorting and node mapping, allowing the node allocation scheme to dynamically adjust according to the latest suitability and load status. Taking the example cluster in Implementation Example 1 as an example, by scanning the queue of scheduled but not yet executed tasks every 60 seconds, if a low-priority cleaning task is found to be mistakenly occupying a GPU node seat, it is returned to the queue of tasks awaiting scheduling. Its suitability category is recalculated based on the calibrated score matrix as CPU-suitable, and the scheduling framework is resubmitted to solve the problem, allowing it to obtain a more suitable node allocation in the next scheduling window.

[0158] The four types of intervention actions mentioned above can be triggered concurrently within the same scheduling window, with the following execution priorities: preemption and eviction have the highest priority to immediately release high-value resources; followed by pre-expansion; then task withdrawal and recalculation; and affinity weight adjustment has the lowest priority as a gradual optimization method. When multiple intervention actions simultaneously meet the triggering conditions, they are executed sequentially according to the above priority order to avoid decision conflicts caused by concurrent modifications to the node allocation scheme. After each intervention action is completed, an intervention event record (including intervention type, trigger time, list of involved tasks, and prediction basis) is written to the telemetry database for subsequent incremental training, forming a complete data loop from intervention to training.

[0159] Furthermore, the weight modifications introduced by the temporary increase in affinity weights are temporary overwrites and are not directly written into the persistent storage of the task-resource fit scoring matrix. Each temporary increase automatically expires after the corresponding task is completed or the next scheduling window ends, restoring the original fit score in the calibrated scoring matrix. If the experimental results show that the allocation scheme after the temporary increase brings a significant improvement in completion time (improvement exceeding 20%), the calibration process is triggered, and the experimental data is written into the scoring matrix as a new sample, so that the temporary adjustment is gradually accumulated into a persistent matrix update, thereby forming a coordinated adaptive mechanism between temporary response and long-term learning.

[0160] Example 12

[0161] Based on Examples 1 to 11, the following uses three typical application scenarios as examples to illustrate the synergistic application effect of each enhancement layer of the present invention in a production line.

[0162] Scenario 1: Construction of a large-scale multimodal dataset of academic literature.

[0163] A research institution needs to construct a multimodal dataset (containing text, formulas, charts, and chemical structures) of 5 million academic papers to train large-scale models specific to the scientific field. The dataset system production cluster is configured with 8 GPU nodes (each equipped with 4 A100 80GB accelerator cards) and 32 CPU nodes (each with 128 physical cores). The task pipeline and scheduling process for each stage are as follows:

[0164] Phase 1: Paper Crawling and Format Parsing (CPU Adaptive): Deploy Scrapy crawler jobs and PDF text coarse extraction Cron jobs, and assess the compatibility ratio of their task types in the task-resource fit rating matrix. The value is approximately 0.28, which is below the 1 / θ threshold (θ=3.0), and it is determined to be CPU-adapted. The scheduling controller automatically schedules it to 32 CPU nodes and writes the preferredDuringSchedulingIgnoredDuringExecution soft affinity rule into the container group orchestration list. The soft affinity weight is TRASM[crawl][cpu_pool]×100=28. The 32 CPU nodes run in parallel, with a coarse PDF extraction rate of approximately 80,000 documents / hour, and do not consume any GPU resources.

[0165] Phase 2: DeepSeek-OCR Fine-grained Analysis (GPU-Sensitive): For complex pages containing formulas and chemical structures, deploy the DeepSeek-OCR inference service (Deployment, 8 copies). This task ranks in the task-resource fit scoring matrix. =12.6, far exceeding the threshold of θ=3.0, and was determined to be a graphics processor sensitive type by Example 3; the requiredDuringSchedulingIgnoredDuringExecution hard affinity rule was written, and the 8 container groups were precisely scheduled to 8 graphics processor nodes, each node occupying 2 A100s, with an OCR inference throughput of about 15,000 pages / hour / node, and a total of about 120,000 pages / hour for 8 nodes.

[0166] Phase 3: GLM-5V-Turbo Multi-Agent Semantic Annotation (Graphics Processor Sensitive): Deploy the GLM-5V-Turbo annotation service (Deployment, 8 replicas), which is also bound to the graphics processor computing power pool through hard affinity rules. Each container group occupies 2 A100s and is responsible for annotation tasks such as entity recognition, relation extraction and graph semantic understanding. The annotation rate is about 3,000 annotations / hour / node.

[0167] Phase 4: ETL Cleaning and Quality Filtering (CPU Adapted): Deploy SparkETLCronJob (executes every 4 hours), The value is approximately 0.21, which is determined by Example 3 to be CPU-adapted and scheduled to the CPU computing power pool, without interfering with the OCR and annotation services on the graphics processor node.

[0168] Scheduling effect: Through the TRASM quantization modeling (invention point 1) and two-layer decoupled scheduling (invention point 2) of this invention, the effective inference utilization of the graphics processing unit node is increased from about 52% to about 89% of the baseline hybrid scheduling scheme, the central processing unit node crawling and ETL tasks are scheduled with zero waiting time, and the overall dataset construction cycle is shortened by about 40%.

[0169] Scenario 2: Dynamic load scheduling of medical image annotation datasets.

[0170] A medical AI company needs to continuously produce labeled datasets for three modalities: chest CT scans, fundus images, and pathological slides. The demand for graphics processing units (GPUs) varies among the three tasks and exhibits significant diurnal fluctuations during weekdays (daytime GPU demand is approximately three times that at night). The cluster is configured with 4 GPU nodes and 16 CPU nodes, with an elastic scaling limit of 8 GPU nodes.

[0171] The following is an example of TRASM quantization modeling (reference values ​​after convergence): CT image DICOM resolution: Graphics processor pool fit score 0.95, Central Processing Unit pool fit score 0.08, Fit ratio... =11.9, classified as GPU-sensitive; RGB normalized fundus image GPU pool adaptation score 0.72, CPU pool adaptation score 0.55, =1.31, classified as mixed type; Graphics processor pool adaptation score for pathological slide super-resolution is 0.97, and CPU pool adaptation score is 0.04. =24.3, classified as GPU-sensitive; metadata ETL cleaning GPU pool adaptation score 0.28, CPU pool adaptation score 0.91, =0.31, classifying it as CPU-adapted; the quality score shows a GPU pool adaptation score of 0.35 and a CPU pool adaptation score of 0.88. =0.40, indicating it is a CPU-adapted type.

[0172] Taking predictive proactive intervention as an example, at 09:00 on a weekday, the reinforcement learning state vector of Example 4 detected gpu_util_5m=0.78, queue_depth_gpu=24, and pending_high_prio_count=8. The load prediction model predicted based on historical data that the graphics processor utilization would reach 0.91 (exceeding the 85% threshold) in 10 minutes. At 09:02, the Cluster Autoscaler was preemptively expanded to 6 graphics processor nodes (target number of nodes = rounded up (6×0.91 / 0.75)=8, constrained by the elastic upper limit to 6). The expanded nodes were actually ready at 09:08, successfully avoiding queuing for CT analysis and pathological super-resolution tasks, and the service level agreement achievement rate for high-priority tasks increased from 71% in the baseline plan to 97%.

[0173] Scenario 3: Concurrent scheduling of multiple pipelines for autonomous driving perception datasets.

[0174] An autonomous driving company is simultaneously running three dataset production pipelines: LiDAR point cloud annotation, camera image segmentation annotation, and multimodal fusion alignment. The tasks are diverse and have different priorities (fusion alignment > image segmentation > point cloud annotation), resulting in fierce real-time competition for cluster resources.

[0175] Taking the two-layer decoupled scheduling operation as an example, the priority decision layer inputs the state vector s=[0.82,0.91,0.45,18,6,0.96,0.88,5] (high GPU utilization, 5 high-priority tasks backlog), and outputs the priority order as follows: fusion alignment task ×3 (highest) > image segmentation task ×8 (medium) > point cloud annotation task ×12 (low) > ETL task ×20 (lowest); the node allocation layer (Example 5) precisely maps the 3 fusion alignment tasks to the 3 GPU nodes with the lowest utilization to meet the load balancing constraint, allocates the 8 image segmentation tasks to the remaining GPU nodes, and schedules 8 of the 12 point cloud annotation tasks to the GPU computing power pool for queuing, while the other 4 are scheduled because the central processing unit pool adaptation ratio is close to the threshold ( =3.2≈θ) was temporarily scheduled to the central processing unit's computing power pool for accelerated processing, and all 20 ETL tasks were mapped to the central processing unit's computing power pool.

[0176] Scheduling results: The fusion alignment task had a waiting time of 0 seconds, the image segmentation task had an average waiting time of 28 seconds, and the point cloud annotation task had its waiting time reduced from the expected 300 seconds to 85 seconds due to the elastic allocation to the central processing unit's computing power pool. The ETL task did not need to wait for graphics processor resources.

[0177] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention.

Claims

1. A heterogeneous computing power task scheduling method based on quantization fit and predictive scheduling, characterized in that, include: Computing nodes in a heterogeneous computing power cluster are grouped by computing power type to obtain computing power pools. A task-resource adaptability scoring matrix is ​​constructed and initialized based on a preset task type and the type of each node in the computing power pool. Calculate the suitability score of the task to be scheduled on the computing power pool node to update the task-resource suitability score matrix, and obtain the updated task-resource suitability score matrix; determine the computing power suitability category according to the ratio of the suitability score on the nodes corresponding to different computing power pools; A scheduling framework comprising a priority decision layer and a node allocation layer is adopted. The priority decision layer outputs a priority score for the task to be scheduled based on the cluster state vector, and generates a priority ranking based on the priority score. The node allocation layer solves the node allocation scheme based on the priority sorting, the updated task-resource adaptability score matrix, and the computing power adaptability category, and generates a scheduling decision based on the node allocation scheme. The task to be scheduled is executed according to the scheduling decision, the measured performance indicators are collected, and the updated task-resource fit score matrix is ​​calibrated to obtain the calibrated task-resource fit score matrix. Collect cluster status data and input it into the load prediction model, output prediction values, and when the output prediction values ​​meet the preset intervention conditions, execute scheduling intervention actions, including adjusting the node allocation scheme according to the calibrated task-resource adaptability score matrix.

2. The method according to claim 1, characterized in that, The process of grouping computing nodes in a heterogeneous computing power cluster according to computing power type to obtain computing power pools, and constructing and initializing a task-resource suitability scoring matrix based on preset task types and the types of nodes in the computing power pools, includes: Read the hardware configuration information of each computing node in the heterogeneous computing power cluster. If the computing node is equipped with a graphics processor accelerator card, mark the computing node as a graphics processor computing power pool node. If the computing node is only equipped with a central processing unit, mark the computing node as a central processing unit computing power pool node to obtain the computing power pool. The task-resource fit score matrix is ​​constructed with the preset task type as the row and each node type in the computing power pool as the column. Based on the preset affinity relationship between the preset task type and each node type in the computing power pool, the initial score of the corresponding element in the task-resource fit score matrix is ​​set, and all the initial scores are marked as pending calibration to complete the initialization.

3. The method according to claim 2, characterized in that, The task-resource fit score of the task to be scheduled on the computing power pool node is calculated to update the task-resource fit score matrix, and the updated task-resource fit score matrix is ​​obtained. The computing power adaptation category is determined based on the ratio of the adaptation score across nodes corresponding to different computing power pools, including: For each scheduled task belonging to a preset task type, the task feature vector of the scheduled task and the resource status parameters of each node in the computing power pool are obtained. The suitability evaluation function is used to calculate the suitability score of the scheduled task on each node in the computing power pool. The row corresponding to the scheduled task in the task-resource suitability score matrix is ​​updated with the suitability score to obtain the updated task-resource suitability score matrix. The ratio of the adaptation score of the task to be scheduled on the graphics processor computing power pool node to the adaptation score on the central processing unit computing power pool node is calculated. The ratio is compared with at least one preset threshold interval. The task to be scheduled is marked as the corresponding computing power adaptation category according to the preset threshold interval to which the ratio belongs. The computing power adaptation category includes at least: graphics processor sensitive type, central processing unit adapted type and hybrid type.

4. The method according to claim 1, characterized in that, The scheduling framework employs a priority decision layer and a node allocation layer. The priority decision layer outputs a priority score for the tasks to be scheduled based on the cluster state vector, and generates a priority ranking based on the priority score. This includes: The waiting time, computational load, and deadline of the task to be scheduled are obtained as task attribute information. Construct the cluster state vector, which includes multiple feature components representing the computing power pool resource status, task queue status, and the adaptation status of tasks to be scheduled. The task attribute information and the cluster state vector are input into the reinforcement learning policy network that constitutes the priority decision layer. The reinforcement learning policy network performs forward inference, outputs the priority score of the task to be scheduled, and generates a priority ranking based on the priority score. The reinforcement learning policy network is trained using an ε-greedy exploration strategy and an experience replay mechanism.

5. The method according to claim 3, characterized in that, The node allocation layer solves for the node allocation scheme based on the priority sorting, the updated task-resource suitability score matrix, and the computing power suitability category, and generates a scheduling decision based on the node allocation scheme, including: The node allocation layer traverses the tasks to be scheduled in sequence according to the priority order. For each task to be scheduled, the corresponding computing power pool is selected according to the computing power adaptation category to obtain a set of candidate nodes. The allocation relationship between the tasks to be scheduled and the nodes in the set of candidate nodes is used as a decision variable. Based on the decision variables and the available resources of each node in the candidate node set, the objective function is to minimize the sum of the weighted completion times of the tasks to be scheduled. The constraints are the available resources and the unique allocation of nodes. A combinatorial optimization problem is constructed and solved to obtain the optimal values ​​of the decision variables. The unique allocation of nodes is used to restrict the tasks to be scheduled to be assigned to only one node. Based on the allocation relationship indicated by the optimal values ​​of the decision variables, the target allocation node for the task to be scheduled is determined from the set of candidate nodes; After traversing the tasks to be scheduled, a node allocation scheme is generated based on the correspondence between the tasks to be scheduled and the target allocation nodes, and a scheduling decision is generated based on the node allocation scheme.

6. The method according to claim 5, characterized in that, The constraints also include: If the computing power adaptation category of the task to be scheduled is graphics processor sensitive, then the task to be scheduled can only be assigned to the graphics processor computing power pool node. If the computing power adaptation category of the task to be scheduled is CPU-adapted, it will be preferentially allocated to the CPU computing power pool node. When the available resources of the CPU computing power pool node are insufficient, it is allowed to be allocated to the GPU computing power pool node and a penalty term will be applied to the objective function.

7. The method according to claim 5, characterized in that, The step of generating scheduling decisions based on the node allocation scheme includes: If the computing power adaptation category of the task to be scheduled is graphics processor sensitive, a scheduling decision containing a first affinity rule is generated. The first affinity rule is used to force the task to be scheduled to be scheduled to a graphics processor computing power pool node. If the computing power adaptation category of the task to be scheduled is CPU-adapted, a scheduling decision containing a second affinity rule is generated, and the weight value of the second affinity rule is determined based on the corresponding adaptation score in the task-resource adaptation score matrix.

8. The method according to claim 2, characterized in that, The process of executing the task to be scheduled according to the scheduling decision, collecting measured performance indicators, and calibrating the updated task-resource fit score matrix to obtain a calibrated task-resource fit score matrix includes: During the execution of the scheduled task according to the scheduling decision, the actual execution time and actual resource utilization rate of the scheduled task on the execution node indicated by the scheduling decision are collected as the measured performance index. From the updated task-resource fit score matrix, extract the element at the intersection of the row corresponding to the task to be scheduled and the column corresponding to the computing power pool to which the execution node belongs as the target element; The measured performance index is compared with the fit score indicated by the target element, and the scoring error is calculated. If the scoring error exceeds a preset threshold, the target element is updated using a preset exponential smoothing algorithm based on the scoring error, and the calibration status marker of the target element is cleared to complete the calibration and obtain the calibrated task-resource fit score matrix.

9. The method according to claim 7, characterized in that, The step of executing the task to be scheduled according to the scheduling decision further includes: For the scheduled task whose computing power adaptation category is CPU-adapted, the scheduled task is scheduled to the CPU computing power pool node according to the second affinity rule in the scheduling decision, so as to achieve physical isolation from the inference task on the GPU computing power pool node.

10. The method according to claim 2, characterized in that, The process of collecting cluster status data and inputting it into the load prediction model to output predicted values ​​includes: Historical load time series data of each computing pool in the heterogeneous computing power cluster are collected at fixed time intervals as the cluster status data. The number of nodes in the heterogeneous computing power cluster is obtained, and a corresponding load prediction model is selected based on the number of nodes. If the number of nodes is less than a preset threshold, a random forest regression model is selected as the load prediction model. If the number of nodes is greater than or equal to the preset threshold, a two-layer fully connected neural network model is selected as the load prediction model. The cluster status data is input into the load prediction model, and the load prediction model outputs the predicted load value and prediction completion time of each computing pool within a preset future time period as the predicted value.

11. The method according to claim 10, characterized in that, The execution of the scheduling intervention action includes: When the predicted load value exceeds the preset high load threshold, the nodes of the heterogeneous computing power cluster are triggered to perform automatic expansion operations. When the first prediction time is less than the second prediction time, and the difference between the first prediction time and the second prediction time is greater than a preset time difference threshold, the weight of the task to be scheduled in the CPU computing power pool corresponding to the task in the task-resource adaptability scoring matrix is ​​increased. Here, the first prediction time is the predicted completion time of the task to be scheduled in the CPU computing power pool, and the second prediction time is the predicted completion time of the task to be scheduled in the GPU computing power pool. When the priority score of the task to be scheduled is higher than the preset priority threshold, and the minimum predicted completion time is greater than the preset service level agreement threshold, a preemption suggestion is generated to prompt the task with a priority score lower than the preset priority threshold to be evicted. The task currently scheduled but not yet executed is identified as an intervention task, and the intervention task is withdrawn to the queue of tasks to be scheduled. The computing power adaptation category is recalculated based on the calibrated task-resource adaptation score matrix, and the scheduling framework is triggered to solve the problem again.

Images