An elastic resource quota and routing weight-based algorithmic electricity collaborative scheduling method

Abstract

The application discloses a kind of based on elastic resource quota and routing weight's algorithm electricity collaborative scheduling method, it is related to energy management technical field, it solves the serious mismatch of prior art cross-system scheduling interface, reduces the technical problem of algorithm power and electricity collaborative linkage efficiency;The present application constructs demand tensor based on training task demand and inference service demand;Establish the algorithm electricity power consumption mapping model between the utilization rate of heterogeneous GPU and the power consumption of PD level power meter;Get the environment variable of power side to form optimization constraint;Construct cluster level resource allocation optimization model and output decision variable;The decision variable corresponding to training task is converted into elastic resource quota;According to the target resource allocation of inference service Generation cross-cluster routing weight;Elastic resource quota and cross-cluster routing weight are used to realize resource access control of training task and cross-cluster traffic scheduling of inference service respectively;The present application improves the executable control efficiency of algorithm electricity collaborative strategy.

Description

Technical Field

[0001] This invention belongs to the field of power system energy management and relates to flexible resource quota management technology, specifically a smart computing cluster-based power scheduling method based on flexible resource quotas. Background Technology

[0002] With the rapid development of "East-to-West data processing" and artificial intelligence large-scale model training and inference services, intelligent computing centers are evolving from traditional rigid electricity loads into new types of adjustable loads that can participate in grid peak shaving and valley filling, green electricity consumption, and demand response. Multiple intelligent computing clusters are usually deployed in different regions with different electricity price levels, renewable energy output characteristics, network latency, GPU / NPU hardware types, task queue structures, and resource utilization rates. Therefore, simply relying on traditional computing power schedulers to allocate tasks based on GPU utilization, queue order, or static cluster rules is no longer sufficient to meet the needs of collaborative optimization between computing power systems and power systems.

[0003] In existing technologies, there is a significant semantic gap between power systems and intelligent computing platforms in the scheduling and control dimension. Specifically, the scheduling instructions output by power system models are usually represented by macroscopic physical or economic variables such as power, cost, and response quantity; while the control actions that intelligent computing platforms can execute at the underlying level depend on microscopic computing resource variables such as queue admission, resource quotas, routing weights, and scaling up and down. Because these two types of variables lack a direct mapping relationship in terms of data dimension and control logic, there is a serious mismatch between cross-system scheduling interfaces, which reduces the efficiency of the collaborative linkage between computing power and power. At the same time, existing scheduling mechanisms face the technical bottleneck of mismatch between load characteristics and execution interfaces: because they fail to distinguish between the time-shifting requirements of training tasks and the spatial routing migration requirements of inference services, and uniformly map them into single control instructions such as "task migration" or "task delay", the scheduling strategy is difficult to accurately match the actual execution interface, thus causing the technical problem of flexible mixing of training and inference. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a computing and power cooperative scheduling method based on elastic resource quota and routing weight to solve the technical problems existing in the prior art.

[0005] To achieve the above objectives, a first aspect of the present invention provides a computing-electricity collaborative scheduling method based on elastic resource quotas and routing weights, comprising:

[0006] Acquire computing power-side operation data, power-side operation data, and business-side data from multiple intelligent computing clusters to construct requirement tensors for training task requirements and inference service requirements;

[0007] The GPU resource usage is obtained as input to the computing power consumption mapping model, and the total power at the cluster level is output. Environmental variables on the power side are obtained to form optimization constraints. The demand tensor, computing power consumption mapping model and optimization constraints are input into the cluster-level resource allocation optimization model, and the target resource allocation amount is output.

[0008] Based on the target resource allocation amount of the training task, an elastic resource quota is generated and distributed to the original computing power scheduling platform as the resource access boundary for the training task.

[0009] Based on the target resource allocation of the inference service, a cross-cluster routing weight is generated, and the cross-cluster routing weight is distributed to the traffic scheduling device to adjust the request distribution of the inference service among different intelligent computing clusters; wherein, the original computing power scheduling platform continues to perform node selection and resource binding for training tasks within the resource boundaries limited by the elastic resource quota.

[0010] Preferably, the step of constructing the training task requirements and inference service requirements into a requirement tensor includes:

[0011] Based on historical task submission sequences, task queue status, GPU model preferences, number of GPUs requested for tasks, and task type tags, the training task requirements for different clusters and GPU models in future time periods are obtained; these training task requirements include: multi-GPU training task requirements. and the need for low-calorie training tasks If a training task requirement is marked as non-delayable or not allowed to be queued, the corresponding training task requirement will be included in the rigid load baseline; otherwise, the corresponding training task requirement will be included in the flexible adjustable training load.

[0012] By analyzing the API gateway's historical query per second (QPS), user access regions, current service instance status, and historical load changes, we can obtain the inference service requirements for different model services in different future time periods. ;

[0013] Based on training task requirements and inference service requirements, a demand tensor for the future scheduling window is generated according to time, cluster, GPU model, and task type. , Where T is the decision time window, D is the set of physical clusters of the intelligent computing center, G is the set of heterogeneous GPU hardware models, and C is the task type classification.

[0014] It should be noted that multi-GPU training tasks typically have high resource requirements, but their startup time can be modified to some extent by controlling the admission quota; multi-GPU training tasks can be used to fill resource gaps and improve GPU utilization; inference services are essentially spatially flexible loads; the demand tensor can be further decomposed into a rigid load baseline and a flexible adjustable space; among them, the rigid load baseline includes non-delayable training tasks, non-portable inference services, the minimum service instance that must be retained, and the platform's guaranteed resources; the flexible adjustable space includes delayable training tasks, training tasks with adjustable concurrency, inference services that can be routed across clusters, and business resources that can be compressed or expanded in the short term.

[0015] Preferably, the step of obtaining the GPU resource usage as input to the computing power consumption mapping model and outputting the cluster-level total power includes:

[0016] Through formula The fitted power consumption slope of the g-model GPU was calculated. ;in, This represents the total power consumption of cluster d in a PD-level meter during time period t. Indicates the number of GPUs in the g model that are enabled; This represents the average utilization of GPU model g during time period t; This represents the power consumption coefficient corresponding to CPU utilization. This indicates the average CPU utilization. Indicates base power consumption;

[0017] The fitted power consumption slope of the g-type GPU As input to the power consumption mapping model, the power consumption is output through the power consumption mapping model.

[0018] It should be noted that the relationship between the fitted power consumption slope and the calculated power in the power consumption mapping model is obtained by fitting historical data of the actual cluster.

[0019] Preferably, the step of obtaining environmental variables from the power side to form optimization constraints includes:

[0020] The environmental variables obtained from the power sector include: spot electricity price forecast, time-of-use electricity price, renewable energy output forecast, green electricity availability, carbon emission factor, demand response invitation, ancillary service signals, power ceiling, power floor and grid ramping constraints.

[0021] The optimization constraints include: upper power limit constraint and lower power limit constraint;

[0022] When a peak shaving invitation is received, the target response quantity on the grid side is converted into a power upper limit constraint for the target cluster during the target time period, including: when the grid requires cluster d to be within the time period... Internal reduction When the load is high, the requirement is converted to: ;in, express Predicted baseline power under unadjusted time period conditions Indicates the target peak-shaving response;

[0023] When receiving valley filling invitations or green energy consumption demands, converting the target load increase into a lower limit constraint for the target time period includes: when it is necessary to increase power during periods of abundant green energy. When the load is high, the system converts the requirement into: ;in, This indicates the target valley filling load increase.

[0024] Preferably, the method for obtaining the cluster-level resource allocation optimization model includes:

[0025] The demand tensor, power consumption mapping model and optimization constraints are used as inputs to the cluster-level resource allocation optimization model. An objective function is constructed based on electricity cost and penalty function. The cluster-level resource allocation optimization model is solved by the objective function and main constraints.

[0026] The objective function is: ;in, To minimize overall cost, Indicates electricity cost. This indicates a penalty for queuing during training tasks. This indicates a penalty for cross-region routing in the inference service. This indicates a penalty for idle computing resources; , and These are the weight coefficients for the preset training task queuing penalty, inference service cross-region routing penalty, and resource idle penalty, respectively.

[0027] The decision variables optimized by the cluster-level resource allocation optimization model are denoted as follows: , and ;in, This indicates the optimal number of g-type GPUs allocated to cluster d for multi-GPU training tasks during time period t, and determines the temporal distribution of the multi-GPU training load. This represents the optimal number of g-type GPUs allocated to cluster d for low-GPU training tasks during time period t. This variable determines the temporal distribution of the low-GPU training load. and Allocate resources to the target of the training task; This represents the optimal number of g-type GPUs allocated to cluster d for inference services during time period t. This variable determines the spatial distribution of the inference load. The amount of target resources allocated for the reasoning task.

[0028] It should be noted that the decision variables represent the optimal resources allocated to different task types for each time period, each cluster, and each GPU model in the future.

[0029] Preferably, the method for obtaining the electricity cost and penalty function includes:

[0030] The electricity cost: ;in, This represents the electricity price of cluster d during time period t; Indicates a time interval;

[0031] The penalty functions include: a training task queuing penalty function, an inference service cross-region routing penalty function, and a computing resource idle penalty function;

[0032] The training task queuing penalty function: ;in, This represents the penalty function caused by the backlog of training tasks on GPUs of type d and g in time period t. This indicates the preset weights for training task latency costs corresponding to different GPU models. This is used to prevent the optimizer from suppressing training tasks for a long time in order to save electricity, which would lead to an indefinite backlog of training tasks.

[0033] The cross-region routing penalty function for the inference service: ;in, This indicates the network latency, user experience loss, and migration costs caused by cross-regional routing for inference services; This represents the preset routing penalty coefficient for cluster d area;

[0034] The idle computing resources penalty function: ;in, , as well as They correspond to as , and The process quantity; This indicates the idle cost corresponding to different GPU models; This indicates the maximum capacity of the g-type GPUs in cluster d, used to prevent excessive resource restrictions that could lead to low cluster resource utilization.

[0035] Preferably, the main constraints include: capacity constraints, power safety constraints, training task queue conservation constraints, rigid load constraints, and inference service SLA constraints.

[0036] The capacity constraint: Capacity constraints mean that, at any given time period t, any given cluster d, and any given GPU model g, the total amount of resources allocated to various tasks cannot exceed the capacity limit of the corresponding GPU model in cluster d.

[0037] The power safety constraints: ;in, This represents the maximum allowable power for cluster d during time period t; if grid ramping constraints exist, the following constraint is added: Limit the rate of increase and decrease of power; among which, This represents the total power of cluster d in the previous scheduling period. This indicates the preset power ramp-up limit for cluster d;

[0038] The rigid load constraint: ;in, This represents the number of g-type computing cards allocated to cluster d at time t for class c tasks; This represents a rigid load baseline; for non-delayable tasks, non-migratable services, or guaranteed inference instances, the optimization result must not fall below the rigid load baseline.

[0039] The training task queue conservation constraint: ;in, This represents the backlog of training tasks corresponding to GPU model g in cluster d at the beginning of time period t; This indicates the new training demand during time period t; express and The total execution volume is used to prevent training tasks from being delayed indefinitely;

[0040] The inference service SLA constraints are as follows: ;in, This represents the number of inference requests for prediction during time period t.

[0041] It should be noted that the total execution volume is and The arithmetic sum of the two.

[0042] Preferably, the step of generating elastic resource quotas based on the target resource allocation amount of the training task includes:

[0043] The elastic resource quota includes: multi-GPU training task ERQ and low-GPU training task ERQ;

[0044] Retrieve the output of the cluster-level resource allocation optimization model and For multi-card training tasks, the formula is used. The ERQ of the multi-GPU training task allocated to cluster d during time period t is calculated.

[0045] For training tasks with low calorie requirements, the formula is used. The ERQ training task with fewer GPUs allocated to cluster d during time period t is calculated.

[0046] It should be noted that ERQ stands for Elastic Resource Quota; It can be expressed in kW, or converted into the number of cards, card time, or namespace resource quota; if power unit is used, the value represents the maximum equivalent power that cluster d is allowed to occupy for multi-card training tasks within time period t; the ERQ of training tasks with few cards can be used to control the concurrent scale of gap filling load, so that it is appropriately increased during periods of low electricity price or abundant green electricity, and appropriately decreased during periods of high electricity price or peak shaving; when ERQ is expressed in power form, its corresponding GPU power consumption parameters can be corrected according to the predicted average utilization rate of the target time period, or pre-calculated into equivalent single-card power consumption parameters based on the historical running status of the corresponding task type, so as to realize the conversion of the target GPU resource quantity into power form ERQ.

[0047] Preferably, generating cross-cluster routing weights based on the target resource allocation amount of the inference service includes:

[0048] Retrieve the output of the cluster-level resource allocation optimization model Through formula The ideal QPS capacity of cluster d for inference services during the target time period was calculated; among which, This indicates the single-card throughput of the g-model GPU for this inference service;

[0049] The target QPS capacity of each cluster is normalized to obtain the cross-cluster routing weight of the gateway: ;in, This represents the inference service traffic allocation ratio for cluster d, calculated based on the target QPS capacity of each cluster. ,and ; i represents a cluster, i=1,2,3,…,d.

[0050] A second aspect of the present invention provides a computing and power consumption collaborative scheduling system based on elastic resource quotas and routing weights, comprising: a data acquisition and demand modeling module, a computing and power consumption mapping module, a resource decision optimization module, and a strategy conversion and execution module;

[0051] The data acquisition and demand modeling module is used to acquire computing power-side operation data, power-side operation data, and business-side data from multiple intelligent computing clusters, and to construct cluster-level demand tensors based on training task requirements and inference service requirements, according to time, cluster, GPU model, and task type.

[0052] The computing power mapping module is used to fit the power consumption parameters of different GPU models based on PD-level power data, the number and utilization of different GPU models, and CPU utilization, and to establish a computing power mapping relationship between GPU resource usage and cluster power consumption.

[0053] The resource decision optimization module takes the demand tensor, the computing power consumption mapping relationship, and power-side optimization parameters and constraints as inputs to construct a cluster-level resource allocation optimization model with the objective of minimizing the overall cost. Based on this model, it outputs decision variables, including: the number of computing cards allocated to multi-GPU training tasks, few-GPU training tasks, and inference services for each time period t, each cluster d, and each GPU model g, respectively. , and ;

[0054] The strategy conversion and execution module is used to convert the target resource allocation of training tasks into elastic resource quotas, and issue the elastic resource quotas through the original computing power scheduling platform interface as resource admission boundaries or concurrency boundaries for training tasks; and to calculate the target QPS capacity of each cluster based on the target resource allocation of inference services and the single-card throughput of different GPU models, further generate cross-cluster routing weights, and issue the cross-cluster routing weights through the traffic scheduling interface; after issuing elastic resource quotas and cross-cluster routing weights through the strategy package, the status is verified and marked as completed for intelligent computing cluster computing power collaborative scheduling strategy issuance.

[0055] Compared with the prior art, the beneficial effects of the present invention are:

[0056] 1. This invention establishes a highly interpretable physical statistical model between heterogeneous GPU resource usage and overall power consumption based on PD-level or cluster-level electricity meter data, without relying on single-task power consumption prediction. In actual intelligent computing centers, it is usually difficult to obtain accurate power consumption labels for single tasks, and there is also a lack of independent electricity meters for each GPU. Therefore, this invention uses PD-level electricity meters and cluster operation data to establish a non-intrusive power consumption mapping, reducing the dependence on single-card electricity meters and single-task power consumption labels. At the same time, this invention can convert power-side adjustment requirements and cluster-level resource optimization results into resource admission boundaries for training tasks and cross-cluster flow control parameters for inference services without obtaining accurate power consumption labels for single tasks or rewriting the core scheduling logic of the original computing power scheduler. This enables the executable control of computing power coordination strategies in existing intelligent computing infrastructure.

[0057] 2. This invention, while ensuring the operation of the intelligent computing center (cluster utilization rate) and user experience (SLA), utilizes computational power coordination scheduling based on elastic resource quotas and routing weights. It controls inference tasks through cross-cluster routing weights and training tasks through elastic resource quotas. This means the system can dynamically adjust the spatiotemporal distribution of inference traffic and training tasks based on the current spot electricity price or green electricity supply, thereby reducing carbon emissions and energy costs. It can also leverage the adjustable resources formed by the temporal flexibility of training tasks and the spatial flexibility of inference services. Under the condition of ensuring cluster resource utilization and business SLA, these resources can be used to reduce power demand during high-price periods, improve resource utilization during low-price or green electricity-sufficient periods, and support peak shaving, valley filling, and demand response, thereby obtaining additional subsidy revenue and optimizing energy costs. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of the computer-powered collaborative scheduling process of the present invention;

[0060] Figure 2 This is a schematic diagram illustrating the specific steps of data acquisition in this invention;

[0061] Figure 3 This is a schematic diagram illustrating the specific steps of resource decision optimization in this invention;

[0062] Figure 4 This is a schematic diagram of the module relationships in the computer-powered collaborative scheduling of the present invention. Detailed Implementation

[0063] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Please see Figure 1 The first aspect of this invention provides a computing-electricity collaborative scheduling method based on elastic resource quotas and routing weights, comprising:

[0065] Acquire computing power-side operation data, power-side operation data, and business-side data from multiple intelligent computing clusters to construct training task requirements and inference service requirements into a requirement tensor;

[0066] The GPU resource usage is obtained as input to the computing power consumption mapping model, and the total power at the cluster level is output.

[0067] The environmental variables on the power side are obtained to form optimization constraints. The demand tensor, the power consumption mapping model, and the optimization constraints are input into the cluster-level resource allocation optimization model, and the target resource allocation amount is output.

[0068] Based on the target resource allocation amount of the training task, an elastic resource quota is generated and distributed to the original computing power scheduling platform as the resource access boundary for the training task.

[0069] Based on the target resource allocation of the inference service, a cross-cluster routing weight is generated and distributed to the traffic scheduling device to adjust the request distribution of the inference service among different intelligent computing clusters. The original computing power scheduling platform continues to perform node selection and resource binding for training tasks within the resource boundary limited by the elastic resource quota, and marks the completion of intelligent computing cluster computing power collaborative scheduling.

[0070] Please see Figure 2 The specific steps for data acquisition include obtaining training task requirements for different clusters and different GPU models in future time periods based on historical task submission sequences, task queue status, GPU model preferences, number of task application cards, and task type labels; these training task requirements include: multi-GPU training task requirements. and the need for low-calorie training tasks If a training task requirement is marked as non-delayable or not allowed to be queued, the corresponding training task requirement will be included in the rigid load baseline; otherwise, the corresponding training task requirement will be included in the flexible adjustable training load.

[0071] By analyzing the API gateway's historical query per second (QPS), user access regions, current service instance status, and historical load changes, we can obtain the inference service requirements for different model services in different future time periods. ;

[0072] Based on training task requirements and inference service requirements, a demand tensor for the future scheduling window is generated according to time, cluster, GPU model, and task type. , Where T is the decision-making time window, D is the set of physical clusters of the intelligent computing center, G is the set of heterogeneous GPU hardware models, and C is the task type classification.

[0073] Establish a computational power consumption mapping model between heterogeneous GPU utilization and PD-level electricity meter power consumption, including:

[0074] Through formula The fitted power consumption slope of the g-model GPU was calculated. ;in, This represents the total power consumption of cluster d in a PD-level meter during time period t. Indicates the number of GPUs in the g model that are enabled; This represents the average utilization of GPU model g during time period t; This represents the power consumption coefficient corresponding to CPU utilization. This indicates the average CPU utilization. Indicates base power consumption;

[0075] The fitted power consumption slope of the g-type GPU As input to the power consumption mapping model, the power consumption is output through the power consumption mapping model;

[0076] The environmental variables obtained from the power sector include: spot electricity price forecast, time-of-use electricity price, renewable energy output forecast, green electricity availability, carbon emission factor, demand response invitation, ancillary service signals, power ceiling, power floor and grid ramping constraints.

[0077] The optimization constraints include: upper power limit constraint and lower power limit constraint;

[0078] When a peak shaving invitation is received, the target response quantity on the grid side is converted into the upper limit constraint of the target cluster power during the target time period; when a valley filling invitation or green electricity consumption demand is received, the target load increase is converted into the lower limit constraint of the target time period.

[0079] For example, first configure the basic objects involved in this scheduling, including the scheduling time window, intelligent computing cluster set, GPU model set, task type set, power consumption metering object, and execution interface information.

[0080] The scheduling time window can be configured as either day-ahead scheduling or intraday rolling scheduling based on the actual scenario. In day-ahead scheduling, a scheduling strategy for the next 24 hours is generated with a 1-hour time granularity, where t∈T=1,2,…,24. In intraday scheduling, the strategy for several future time periods is updated every 15 minutes or every hour. Day-ahead and intraday scheduling can use the same technical approach, differing only in the real-time nature of the input data and the prediction window.

[0081] Intelligent computing resources are divided into multiple physical clusters d. Each cluster d can correspond to a data center in a region, or to different server rooms, different campuses, or different resource pools within the same region. Specifically, computing resources that are network-connected and uniformly managed are considered to belong to the same cluster. Each cluster can contain one or more PDs. PD is an abbreviation for PowerDomain, representing a power domain whose electrical power can be independently metered, such as a server room, a micro-module, a rack group, or a power supply zone. A PD can contain multiple GPU resources of different models, and these GPU resources may not be regularly partitioned and racked in actual deployment.

[0082] Task type c is divided into at least three categories: multi-GPU training tasks ("Train_multi"), few-GPU training tasks ("Train_few"), and inference services ("Inf"). Multi-GPU training tasks typically consume large amounts of resources, such as requesting 8, 16, or more GPUs at once, and are suitable for altering their time distribution by controlling startup time or admission quotas. Few-GPU training tasks typically have lower resource requirements, such as 1, 2, or 4-GPU tasks, and can serve as filler loads to fill resource fragmentation and improve GPU utilization. Inference services are typically designed for online access and have stronger real-time requirements, making them suitable for altering their spatial distribution through cross-cluster traffic routing.

[0083] Simultaneously configure execution interfaces, including the original computing power scheduling platform interface and the gateway interface. The original computing power scheduling platform interface is used to issue ERQ, namespace resource quotas, maximum queue concurrency, admission control rules, or task priority rules; the gateway interface is used to issue cross-cluster routing weights for inference services, such as the weight configurations for APIGateway, DNS, service mesh, or global load balancer.

[0084] The output of this step is the basic configuration used in subsequent steps, including the correspondence between T, D, G, C, Capd, g, PD and the cluster, the original scheduling platform interface, the API gateway interface, and the scheduling operation frequency;

[0085] At the beginning of each scheduling cycle, three types of data—computing power, power, and business—are collected through a data interface to form a data snapshot for the current scheduling cycle.

[0086] Data collection on the computing power side: Read the computing resource operation status of each cluster from the intelligent computing platform, cluster monitoring system, container orchestration platform, or job management platform. This includes the total number of GPUs of different models in each cluster, the number of available GPUs, the number of allocated GPUs, GPU utilization, CPU utilization, memory utilization, task queue length, number of running tasks, number of tasks to be started, number of GPUs requested for training tasks, training task queuing status, number of inference service instances, inference service QPS, service response latency, service health status, and historical throughput of different services. This data is used to identify the current load status, resource availability, training task queuing pressure, and inference service carrying capacity of different clusters.

[0087] Data collection on the power side: This involves reading power-side data from meters, energy management systems, power trading systems, microgrid systems, or virtual power plant services. This includes PD-level power, cluster-level power, historical electricity consumption, current power, power safety limits, power ramp-up constraints, time-of-use pricing, spot price forecasts, renewable energy output forecasts, green electricity availability, carbon emission factors, energy storage status, and information on demand response, peak shaving, valley filling, or ancillary service invitations. PD-level and cluster-level power are used to establish a power consumption mapping; electricity prices and green electricity signals are used to optimize objectives; and power limits, power lower limits, response duration, and target response quantities are used to form constraints.

[0088] Collect business-side data: Read information such as task type, tenant type, service name, inference service access area, business priority, SLA requirements, non-delayed task marker, non-migratable service marker, acceptable maximum queuing time, model service single-card throughput, and historical call patterns.

[0089] For training tasks, business-side data is used to determine whether a task is a rigid or flexible task. Tasks that cannot be delayed, administrator-specified tasks, and tasks with strong SLAs are included in the rigid load baseline; training tasks that can be delayed or allowed to be queued are included in the flexible, adjustable load. For inference services, business-side data is used to determine whether the service allows cross-cluster routing and whether different clusters have the capacity to handle the service.

[0090] The output of this step is a data snapshot of the current scheduling period, which serves as input for subsequent demand forecasting, power mapping, and optimization solutions.

[0091] When a certain inference service has a single-card throughput of [amount] on a [model] GPU, [percentage]... The predicted target QPS is Then, the theoretical number of GPUs required for the service on the corresponding GPU model can be expressed as: If the service can be hosted on multiple clusters, the system will determine the number of inference GPUs that each cluster should handle during subsequent optimizations. .

[0092] By constructing demand tensors for training tasks and inference services, the output includes demand tensors, rigid load baselines, a deferred resource pool for training tasks, a transferable resource pool for inference services, and SLA constraints for various tasks.

[0093] In establishing a computation-power mapping model between heterogeneous GPU resource consumption and PD-level power consumption, it is often difficult to obtain accurate power consumption labels for single tasks in actual intelligent computing centers, and independent power meters for each GPU are also lacking. Therefore, this invention does not rely on single-task power consumption prediction, but instead establishes a highly interpretable physical-statistical model between heterogeneous GPU resource consumption and overall power consumption based on PD-level or cluster-level power meter data. Furthermore, in fitting the power consumption slope of heterogeneous GPUs, ridge regression, tree models, or other statistical learning methods can be used to fit the power consumption slopes of different GPU models. The core function of the computing power mapping model is to extract the fitted power consumption slope of various computing cards. This can be used as a conversion relationship between computing power resources and electrical power. For example, it can be fitted to obtain... , The specific values ​​were obtained by fitting historical data from the actual cluster.

[0094] The output of this step includes the fitted power consumption slope for different GPU models. Cluster basic power consumption Power consumption mapping model parameters, and the amount of resources allocated calculate The method involves using this mapping relationship to calculate electricity consumption, electricity costs, power cap, and demand response capability in subsequent optimization models.

[0095] Please see Figure 3 The specific steps for resource decision optimization include retrieving the output of the cluster-level resource allocation optimization model. and For multi-card training tasks, the formula is used. The ERQ of the multi-GPU training task allocated to cluster d during time period t is calculated.

[0096] For training tasks with low calorie requirements, the formula is used. The ERQ of the low-card training task allocated to cluster d during time period t is calculated.

[0097] Retrieve the output of the cluster-level resource allocation optimization model Through formula The ideal QPS capacity of cluster d for inference services during the target time period was calculated; among which, This indicates the single-card throughput of the g-model GPU for this inference service;

[0098] The target QPS capacity of each cluster is normalized to obtain the cross-cluster routing weight of the gateway: ;in, This represents the inference service traffic allocation ratio for cluster d, calculated based on the target QPS capacity of each cluster. ,and ; i represents a cluster, i=1,2,3,…,d.

[0099] For example, cluster-level resource allocation optimization models can be solved in various ways. For deterministic scenarios, they can be modeled as linear programming, mixed-integer linear programming, or other operations research optimization problems and solved using a solver. For scenarios with uncertainties in electricity prices, renewable energy output, or demand, distributed bar optimization (DRO) can be used to incorporate prediction errors into the uncertainty set. For scenarios with high real-time requirements or rapidly changing environments, reinforcement learning methods can also be used to transform the objective function and hard constraints into reward and safety constraints, but the hard constraints still need to be protected by rules or optimizers.

[0100] Regardless of the solution method used, the resource decision optimization module ultimately outputs the target resource allocation for each time period, each cluster, each GPU model, and each task type. , and Simultaneously output the corresponding target power curve And the satisfaction of peak shaving, valley filling or green electricity consumption constraints.

[0101] Set the priority of training and inference adjustments. In one implementation, this can be achieved by setting... This setting prioritizes inference service migration by the optimizer, first reducing the target cluster's power consumption through spatial migration. Only when the inference load is reduced to its limit will the resource consumption of training tasks be further reduced, i.e., time shifting. This configuration can minimize excessive backlog of training tasks while ensuring the SLA of inference services.

[0102] For external C-end user scenarios, platforms typically aim to maintain high resource utilization, which can be achieved by improving... This makes the optimizer prioritize resource utilization. Only when electricity cost savings or demand response benefits are sufficiently large is it permissible to sacrifice some resource utilization in exchange for cost reductions or grid response benefits.

[0103] In the process of calculating the ERQ for a low-GPU training task, a more granular implementation can also generate the ERQ according to the GPU model: ; This allows for different limits on the size of training tasks that can be admitted for different GPU models.

[0104] ERQ Issuance and Admission Control: The system issues the calculated ERQ to the existing computing power scheduling platform. The execution unit can be kW, the number of GPUs, or GPU time. If kW is used as the execution unit, when a new training task is submitted, the system converts it into an equivalent power requirement based on the GPU model and number of GPUs requested for the task. .in, This indicates the number of g-model GPUs requested for this task.

[0105] when When the sum of the currently occupied training ERQs exceeds the remaining ERQ quota for the current time period, the original scheduling platform will not start the training task, but will keep it in the Pending state, low-priority queue, or waiting-to-be-scheduled queue. When an existing training task completes and releases the quota, or when the ERQ is increased in the next scheduling cycle, the task will then enter the normal scheduling process of the original computing power scheduling platform.

[0106] This invention does not require rewriting the core node selection, resource binding, or scheduling scoring algorithms of the original computing power scheduling platform. ERQ only serves as an external admission boundary or queue concurrency boundary, while the original computing power scheduling platform remains responsible for specific node selection and task initiation.

[0107] The process of calculating cross-cluster routing weights is as follows: For example, if an eastern cluster D1 can handle 3000 QPS and a western cluster D2 can handle 7000 QPS during the target time period, then... , That is, the system can route 30% of the requests to the D1 cluster and 70% of the requests to the D2 cluster; or, if the binary value is used, all requests can be directed to D2.

[0108] Routing weight distribution process: The system calls the global API gateway, DNS, service mesh, or load balancer interface to update the routing weights of different clusters. After traffic enters the gateway, it first selects the target cluster based on the routing weight, and then determines whether to allow access based on the VCC or resource limits within the target cluster. If the target cluster is fully loaded, has an abnormal service health status, or the network latency exceeds the threshold, the system selects other available clusters according to the weight order, or lowers the weight of the current cluster.

[0109] The routing weights in this invention differ from ordinary load balancing strategies. Ordinary load balancing typically allocates traffic based on request volume, instance health status, service latency, or server load; the routing weights in this invention are derived from the inference target resource allocation output by the computational power optimization model, combined with the single-card throughput of different GPU models. The calculation yielded the result. Therefore, this routing weight simultaneously reflects factors related to electricity pricing, green electricity, power constraints, cluster capacity, and service SLA.

[0110] The system packages the training ERQ and inference route weights into a policy package. The policy package includes at least the policy version number, effective time, expiration time, target cluster, target GPU model, training task ERQ, inference service route weights, power boundaries, rollback conditions, and exception handling rules.

[0111] For training tasks, the edge execution module calls the existing computing power scheduling platform interface to convert ERQs into resource quotas, admission control rules, maximum queue concurrency, or namespace quotas. The existing computing power scheduling platform is still responsible for specific node selection, resource binding, and task initiation; this invention only provides external admission boundaries.

[0112] For the inference service, the edge execution module calls the API gateway, DNS, service mesh, or load balancer interfaces to update the routing weights of different clusters. If the target cluster is not yet capable of handling the load, the system can first trigger the inference service to scale up or the shadow instance to warm up, and then gradually increase the routing weight of the cluster after the service health check passes.

[0113] The output of this step is the ERQ of the training task and the routing weights for the inference service, which have already taken effect. This output can be directly applied to the task admission and traffic allocation processes of the intelligent computing platform.

[0114] Please see Figure 4 The second aspect of the present invention provides a computing and power collaborative scheduling system based on elastic resource quotas and routing weights, including: a data acquisition and demand modeling module, a computing and power consumption mapping module, a resource decision optimization module, and a strategy conversion and execution module;

[0115] The data acquisition and demand modeling module is used to acquire computing power-side operation data, power-side operation data, and business-side data from multiple intelligent computing clusters, and to construct cluster-level demand tensors based on training task requirements and inference service requirements, according to time, cluster, GPU model, and task type.

[0116] The computing power mapping module is used to fit the power consumption parameters of different GPU models based on PD-level power data, the number and utilization of different GPU models, and CPU utilization, and to establish a computing power mapping relationship between GPU resource usage and cluster power consumption.

[0117] The resource decision optimization module takes the demand tensor, the computing power consumption mapping relationship, and power-side optimization parameters and constraints as inputs to construct a cluster-level resource allocation optimization model with the objective of minimizing the overall cost. Based on this model, it outputs decision variables, including: the number of computing cards allocated to multi-GPU training tasks, few-GPU training tasks, and inference services for each time period t, each cluster d, and each GPU model g, respectively. , and ;

[0118] The strategy conversion and execution module is used to convert the target resource allocation of training tasks into elastic resource quotas, and issue the elastic resource quotas through the original computing power scheduling platform interface as resource admission boundaries or concurrency boundaries for training tasks; and to calculate the target QPS capacity of each cluster based on the target resource allocation of inference services and the single-card throughput of different GPU models, further generate cross-cluster routing weights, and issue the cross-cluster routing weights through the traffic scheduling interface; after issuing elastic resource quotas and cross-cluster routing weights through the strategy package, the status is verified and marked as completed for intelligent computing cluster computing power collaborative scheduling strategy issuance.

[0119] It should be noted that the power consumption parameters in the computational power consumption mapping model , and The parameters are obtained by fitting historical PD-level power data and GPU / CPU operating status data after time alignment. During rolling scheduling, these parameters can be periodically updated based on the deviation between predicted and actual power. The weight parameters and security constraint thresholds in the objective function are configured according to the business SLA, cluster security boundary, and scheduling objectives.

[0120] The working principle of this invention is as follows: Historical data and current operating status of training tasks and inference services are obtained from the database. Based on this historical data and current operating status, the requirements for training tasks and inference services are predicted, and a cluster-level requirement tensor is constructed according to time, cluster, GPU model, and task type. A computing power consumption mapping model is established between heterogeneous GPU utilization and PD-level electricity meter power consumption. Environmental variables from the power side are obtained to form optimization constraints. The requirement tensor, computing power consumption mapping model, and optimization constraints are used as inputs to construct a cluster-level resource allocation optimization model with the objective of minimizing overall cost. Decision variables are output based on the cluster-level resource allocation optimization model. These decision variables include: the number of computing cards allocated to multi-GPU training tasks, few-GPU training tasks, and inference services for each time period t, each cluster d, and each GPU model g, respectively. , and The decision variables corresponding to the training task are converted into elastic resource quotas; cross-cluster routing weights are generated based on the target resource allocation amount of the inference service; the elastic resource quotas are distributed to the original computing power scheduling platform, and the cross-cluster routing weights are distributed to the traffic scheduling device to realize resource access control for the training task and cross-cluster traffic scheduling for the inference service, respectively, and the intelligent computing cluster computing power collaborative scheduling is marked as complete.

[0121] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A computational-powered collaborative scheduling method based on elastic resource quotas and routing weights, characterized in that, include: Acquire computing power-side operation data, power-side operation data, and business-side data from multiple intelligent computing clusters to construct training task requirements and inference service requirements into a requirement tensor; The GPU resource usage is obtained as input to the computing power consumption mapping model, and the total power at the cluster level is output. The environmental variables on the power side are obtained to form optimization constraints. The demand tensor, the power consumption mapping model, and the optimization constraints are input into the cluster-level resource allocation optimization model, and the target resource allocation amount is output. Based on the target resource allocation amount of the training task, an elastic resource quota is generated and distributed to the original computing power scheduling platform as the resource access boundary for the training task. Based on the target resource allocation of the inference service, a cross-cluster routing weight is generated, and the cross-cluster routing weight is distributed to the traffic scheduling device to adjust the request distribution of the inference service among different intelligent computing clusters; wherein, the original computing power scheduling platform continues to perform node selection and resource binding for training tasks within the resource boundaries limited by the elastic resource quota.

2. The computational power scheduling method based on elastic resource quotas and routing weights according to claim 1, characterized in that, The construction of the training task requirements and inference service requirements into a requirement tensor includes: Based on historical task submission sequences, task queue status, GPU model preferences, number of GPUs requested for tasks, and task type tags, the training task requirements for different clusters and GPU models in future time periods are obtained; these training task requirements include: multi-GPU training task requirements. and the need for low-calorie training tasks If a training task requirement is marked as non-delayable or not allowed to be queued, the corresponding training task requirement will be included in the rigid load baseline; otherwise, the corresponding training task requirement will be included in the flexible adjustable training load. By analyzing the API gateway's historical query per second (QPS), user access regions, current service instance status, and historical load changes, we can obtain the inference service requirements for different model services in different future time periods. ; Based on training task requirements and inference service requirements, a demand tensor for the future scheduling window is generated according to time, cluster, GPU model, and task type. , Where T is the decision time window, D is the set of physical clusters of the intelligent computing center, G is the set of heterogeneous GPU hardware models, and C is the task type classification.

3. The computational power scheduling method based on elastic resource quotas and routing weights according to claim 2, characterized in that, The process of obtaining GPU resource usage as input to the computing power consumption mapping model and outputting the cluster-level total power through the computing power consumption mapping model includes: Obtain cluster-level total power for multiple historical time periods And GPU resource usage, which includes: the number of G-model GPUs in use. Average utilization of g-type GPU during time period t and average CPU utilization ; Data fitting based on the multiple historical time periods , and , so that: ;in, This represents the fitted power consumption slope of the g-model GPU. Indicates the cluster's base power consumption; This represents the power consumption coefficient corresponding to CPU utilization. Based on the power consumption parameters obtained from the fitting, the resource allocation of each GPU model is converted into the corresponding cluster-level power consumption; among which, the power consumption parameters include: the fitted power consumption slope, the base power consumption, and the power consumption coefficient.

4. The computing-electricity collaborative scheduling method based on elastic resource quotas and routing weights according to claim 1, characterized in that, The acquisition of environmental variables from the power supply side forms optimization constraints, including: The environmental variables obtained from the power sector include: spot electricity price forecast, time-of-use electricity price, renewable energy output forecast, green electricity availability, carbon emission factor, demand response invitation, ancillary service signals, power ceiling, power floor and grid ramping constraints. The optimization constraints include: upper power limit constraint and lower power limit constraint; When a peak shaving invitation is received, the target response quantity on the grid side is converted into the upper limit constraint of the target cluster power during the target time period; when a valley filling invitation or green electricity consumption demand is received, the target load increase is converted into the lower limit constraint of the target time period.

5. The computational power scheduling method based on elastic resource quotas and routing weights according to claim 1, characterized in that, The methods for obtaining the cluster-level resource allocation optimization model include: The demand tensor, power consumption mapping model and optimization constraints are used as inputs to the cluster-level resource allocation optimization model. An objective function is constructed based on electricity cost and penalty function. The cluster-level resource allocation optimization model is solved by the objective function and main constraints. The objective function is: ;in, To minimize overall cost, Indicates electricity cost. This indicates a penalty for queuing during training tasks. This indicates a penalty for cross-region routing in the inference service. This indicates a penalty for idle computing resources; , and These are the weight coefficients for the preset training task queuing penalty, inference service cross-region routing penalty, and resource idle penalty, respectively. The decision variables output by the cluster-level resource allocation optimization model are denoted as follows: , and ;in, This indicates the optimal number of g-type GPUs allocated to cluster d for multi-GPU training tasks during time period t, and determines the temporal distribution of the multi-GPU training load. This represents the optimal number of g-type GPUs allocated to cluster d for low-GPU training tasks during time period t. This variable determines the temporal distribution of the low-GPU training load. and Allocate resources to the target of the training task; This represents the optimal number of g-type GPUs allocated to cluster d for inference services during time period t, used to determine the spatial distribution of inference load. The amount of target resources allocated for the reasoning task.

6. The computational power scheduling method based on elastic resource quotas and routing weights according to claim 5, characterized in that, The methods for obtaining the electricity cost and penalty function include: The electricity cost: ;in, This represents the electricity price of cluster d during time period t; Indicates a time interval; The penalty functions include: a training task queuing penalty function, an inference service cross-region routing penalty function, and a computing resource idle penalty function; The training task queuing penalty function: ;in, This represents the penalty function caused by the backlog of training tasks on GPUs of type d and g in time period t. This indicates the preset weights for training task latency costs corresponding to different GPU models. This is used to prevent the optimizer from suppressing training tasks for a long time in order to save electricity, which would lead to an indefinite backlog of training tasks. The cross-region routing penalty function for the inference service: ;in, This indicates the network latency, user experience loss, and migration costs caused by cross-regional routing for inference services; This represents the preset routing penalty coefficient for cluster d area; The idle computing resources penalty function: ;in, , as well as They correspond to as , and The process quantity; This indicates the maximum capacity of the g-type GPUs in cluster d, used to prevent excessive resource restrictions that could lead to low cluster resource utilization.

7. The computational power scheduling method based on elastic resource quotas and routing weights according to claim 5, characterized in that, The main constraints include: capacity constraints, power safety constraints, training task queue conservation constraints, rigid load constraints, and inference service SLA constraints. The capacity constraint: Capacity constraints mean that, at any given time period t, any given cluster d, and any given GPU model g, the total amount of resources allocated to various tasks cannot exceed the capacity limit of the corresponding GPU model in cluster d. The power safety constraints: ;in, This represents the maximum allowable power supply for cluster d during time period t; if grid ramping constraints exist, additional constraints are added. Limit the rate of increase and decrease of power; among which, This represents the total power of cluster d in the previous scheduling period. This indicates the preset power ramp-up limit for cluster d; The rigid load constraint: ;in, This represents the number of g-type computing cards allocated to cluster d at time t for class c tasks; This represents a rigid load baseline; for non-delayable tasks, non-migratable services, or guaranteed inference instances, the optimization result must not fall below the rigid load baseline. The training task queue conservation constraint: ;in, This represents the backlog of training tasks corresponding to GPU model g in cluster d at the beginning of time period t; This indicates the new training demand during time period t; express and The total execution volume is used to prevent training tasks from being delayed indefinitely; The inference service SLA constraints are as follows: ;in, This represents the number of inference requests for prediction during time period t.

8. The computational power scheduling method based on elastic resource quotas and routing weights according to claim 3, characterized in that, The step of generating elastic resource quotas based on the target resource allocation amount of the training task includes: The elastic resource quota includes: multi-GPU training task ERQ and low-GPU training task ERQ; Retrieve the target resource allocation amount for multi-GPU training tasks from the cluster-level resource allocation optimization model. And the amount of resources allocated to the training task objectives. For multi-GPU training tasks, based on the target resource allocation and the power consumption parameters of the corresponding GPU model, the formula is used. The ERQ of the multi-GPU training task allocated to cluster d during time period t is calculated. For training tasks with limited GPUs, based on the target resource allocation and the power consumption parameters of the corresponding GPU model, the formula is used. The ERQ of the low-card training task allocated to cluster d during time period t is calculated. The elastic resource quota is distributed to the existing computing power scheduling platform; when the sum of the equivalent resource requirement of a newly arriving training task and the currently occupied training resources exceeds the ERQ corresponding to the current time period, the training task is kept in a pending scheduling state; when there is available margin in the elastic resource quota, the training task enters the normal scheduling process of the existing computing power scheduling platform.

9. The computational power scheduling method based on elastic resource quotas and routing weights according to claim 1, characterized in that, The step of generating cross-cluster routing weights based on the target resource allocation of the inference service includes: Retrieve the output of the cluster-level resource allocation optimization model Through formula The ideal QPS capacity of cluster d for inference services during the target time period was calculated; among which, This indicates the single-card throughput of the g-model GPU for this inference service; The target QPS capacity of each cluster is normalized to obtain the cross-cluster routing weight of the gateway: ;in, This represents the inference service traffic allocation ratio for cluster d, calculated based on the target QPS capacity of each cluster. ,and ; i represents a cluster, i=1,2,3,…,d.

10. A computing-powered collaborative scheduling system based on elastic resource quotas and routing weights, applied to the computing-powered collaborative scheduling method based on elastic resource quotas and routing weights as described in any one of claims 1-9, characterized in that, include: The module includes data acquisition and demand modeling, computing power consumption mapping, resource decision optimization, and strategy conversion and execution. The data acquisition and demand modeling module is used to acquire computing power-side operation data, power-side operation data, and business-side data from multiple intelligent computing clusters, and to construct cluster-level demand tensors based on training task requirements and inference service requirements, according to time, cluster, GPU model, and task type. The computing power mapping module is used to fit the power consumption parameters of different GPU models based on PD-level power data, the number and utilization of different GPU models, and CPU utilization, and to establish a computing power mapping relationship between GPU resource usage and cluster power consumption. The resource decision optimization module takes the demand tensor, the power consumption mapping relationship, and the power-side optimization parameters and constraints as inputs to construct a cluster-level resource allocation optimization model with the goal of minimizing the overall cost. The decision variables are output based on a cluster-level resource allocation optimization model. These decision variables include: the number of computing cards allocated to multi-GPU training tasks, few-GPU training tasks, and inference services for each time period t, each cluster d, and each GPU model g, respectively. , and ; The strategy conversion and execution module is used to convert the target resource allocation of training tasks into elastic resource quotas, and issue the elastic resource quotas through the original computing power scheduling platform interface as resource admission boundaries or concurrency boundaries for training tasks; and to calculate the target QPS capacity of each cluster based on the target resource allocation of inference services and the single-card throughput of different GPU models, further generate cross-cluster routing weights, and issue the cross-cluster routing weights through the traffic scheduling interface; after issuing elastic resource quotas and cross-cluster routing weights through the strategy package, the status is verified and marked as completed for intelligent computing cluster computing power collaborative scheduling strategy issuance.

Images