A cross-domain data center oriented workflow scheduling method and device

Abstract

The application relates to the technical field of cloud computing, and provides a workflow scheduling method and device for cross-domain data centers, which comprises the following steps: constructing operation state vectors of each data center in a cross-domain data center set in a same scheduling period; receiving a workflow set and constructing a directed acyclic graph of each workflow; obtaining execution time, transmission time, completion time, power cost and carbon emission of each task in the workflow set by each data center based on each directed acyclic graph and each operation state vector; calculating comprehensive ranking values of each workflow and determining a target workflow and a to-be-scheduled task in the target workflow; determining a target data center based on elastic available capacity, load rate and completion time, power cost and carbon emission of the to-be-scheduled task of each data center, and delivering the to-be-scheduled task. The application can avoid the overall performance decline caused by taking the lowest electricity cost, the lowest carbon emission or the most idle computing power as a single target.

Description

Technical Field

[0001] This application relates to the field of cloud computing, and in particular to a workflow scheduling method and apparatus for cross-domain data centers. Background Technology

[0002] With the rapid growth of AI training, inference services, scientific computing, video processing, and large-scale data analytics, geographically distributed data centers have become a crucial infrastructure for supporting complex workflows. Data centers in different regions exhibit significant differences in electricity prices, carbon emission intensity, renewable energy output, weather conditions, network links, and available computing resources. Furthermore, different subtasks within the same workflow are subject to dependencies, data transmission, and deadline constraints.

[0003] Therefore, task scheduling in cross-regional data centers is no longer a simple problem of computing power allocation, but a problem of coordinated optimization involving electricity costs, computing capacity, carbon emissions, and service quality. Existing scheduling methods usually use a single objective among task completion time, resource utilization, or electricity cost as the basis for optimization. They are difficult to make stable decisions in conflicting scenarios such as low electricity prices but slow network transmission, low carbon emission intensity but insufficient computing capacity, and sufficient computing power but data center power approaching its limit. This can easily lead to situations such as local low cost but overall timeout, and low-carbon migration causing transmission bottlenecks. Summary of the Invention

[0004] This application provides a workflow scheduling method and apparatus for cross-domain data centers, which can uniformly model electricity costs, computing capacity and carbon emissions, avoiding the overall performance degradation caused by taking only the lowest electricity cost, lowest carbon emissions or most idle computing power as a single objective.

[0005] In a first aspect, embodiments of this application provide a workflow scheduling method for cross-domain data centers, including: Construct the operational status vector of each data center in the cross-domain data center set in the same scheduling cycle. The operational status vector includes the data center's electricity price, carbon emission intensity, energy consumption coefficient per unit workload, energy use efficiency, and network connection status. Receive the workflow set and construct the directed acyclic graph of each workflow; based on each directed acyclic graph and each running state vector, obtain the execution time, transmission time, completion time, power cost and carbon emissions of each task in the workflow set for each data center; The overall ranking value of each workflow is calculated based on the average effective computing power and average transmission bandwidth of the cross-domain data center set, and the workflow with the smallest overall ranking value is selected as the target workflow. The tasks to be scheduled in the target workflow are determined based on execution time and transmission time. Calculate the elastic available capacity of each data center, determine the target data center based on the elastic available capacity, load rate, completion time of the task to be scheduled, power cost and carbon emissions of each data center, and issue the task to be scheduled.

[0006] Furthermore, the above-mentioned electricity costs and carbon emissions for each task in the workflow set of each data center are obtained based on each directed acyclic graph and each running state vector, including: The task energy consumption allocated to the data center is calculated based on the task's computational load, the unit workload energy consumption coefficient in the running state vector, and the energy usage efficiency. The electricity cost and carbon emissions of tasks allocated to the data center are calculated based on task energy consumption, electricity price in the operating state vector, and carbon emission intensity.

[0007] Furthermore, the above-mentioned comprehensive ranking value for each workflow, calculated based on the average effective computing power and average transmission bandwidth of the cross-domain data center set, includes: The normalized computational scale factor of the workflow is obtained based on the workload of the workflow, the computational workload of each task in the workflow, and the number of workflows to be scheduled in the current scheduling cycle. The estimated execution time and estimated transmission time for each task are obtained based on the computational load, data volume, average effective computing power of the cross-domain data center set, and average transmission bandwidth. The estimated earliest start time for each task is obtained based on the workflow topology, the estimated execution time of each task, and the estimated transmission time. The normalized relaxation time factor of the workflow is determined based on the estimated earliest start time of each task in the workflow and the overall deadline of the workflow; the comprehensive ranking value of the workflow is calculated based on the normalized relaxation time factor and the normalized computational scale factor.

[0008] Furthermore, the above-mentioned determination of the tasks to be scheduled in the target workflow based on execution time and transmission time includes: The estimated execution time of a task is obtained based on the execution time of the task in each data center. The values ​​are sorted upwards based on the expected execution time and transmission time of the task; The sub-deadline of a task is obtained based on its upward sorting value, estimated execution time, and the overall deadline of the target workflow. The priority value of a task is obtained based on its computational load, upward sorting value, and sub-deadline. The task with the highest priority value in the target workflow is then selected as the task to be scheduled.

[0009] Furthermore, the above-mentioned determination of target data centers based on the elastic availability capacity, load rate, completion time of tasks to be scheduled, power costs, and carbon emissions of each data center includes: Data centers with elastic available capacity greater than or equal to the computational load of the task to be scheduled and completion time less than or equal to the sub-deadline of the task to be scheduled are selected as candidate data centers. The comprehensive evaluation value of the candidate data centers is calculated based on their load rate, completion time of tasks to be scheduled, power costs, and carbon emissions. The candidate data center with the lowest overall evaluation value will be selected as the target data center.

[0010] Furthermore, the method also includes: Obtain the elastic load base capacity of future windows and individual data centers, and predict the forecasted electricity price and carbon emission intensity of each data center within the future window; The elastic available capacity of each data center is obtained based on the elastic load base capacity, predicted electricity price, and predicted carbon emission intensity, and a carbon-electricity synergistic elastic capacity curve is generated for each data center.

[0011] Furthermore, the method also includes: Update the elastic available capacity of the target data center based on the computational load of the tasks to be scheduled, and collect the actual operating status of the tasks to be scheduled; determine whether the deviation between the actual operating status and the predicted status exceeds the preset status threshold, or whether the deviation between the updated elastic available capacity and the carbon-electricity co-operation elastic capacity curve exceeds the preset capacity threshold. The predicted status includes the predicted electricity price and the predicted carbon emission intensity; if so, redetermine the target data center.

[0012] Secondly, another embodiment of this application provides a workflow scheduling apparatus for cross-domain data centers, comprising: The building module is used to construct the operating status vector of each data center in the cross-domain data center set in the same scheduling cycle. The operating status vector includes the data center's electricity price, carbon emission intensity, energy consumption coefficient per unit workload, energy use efficiency, and network connection status. The modeling module receives the workflow set and constructs the directed acyclic graphs (DAGs) for each workflow. Based on each DAG and each running state vector, it obtains the execution time, transmission time, completion time, power cost, and carbon emissions of each task in the workflow set for each data center. The sorting module is used to calculate the comprehensive sorting value of each workflow based on the average effective computing power and average transmission bandwidth of the cross-domain data center set, and to select the workflow with the smallest comprehensive sorting value as the target workflow; and to determine the tasks to be scheduled in the target workflow based on the execution time and transmission time. The scheduling module is used to calculate the elastic available capacity of each data center, determine the target data center based on the elastic available capacity, load rate, completion time of the task to be scheduled, power cost and carbon emissions of each data center, and issue the task to be scheduled.

[0013] In another aspect, this application provides a computer device including at least one connected processor, memory, and transceiver, wherein the memory is used to store program code, and the processor is used to call the program code in the memory to execute the methods described in the above aspects.

[0014] In another aspect, embodiments of this application provide a computer storage medium including instructions that, when executed on a computer, cause the computer to perform the methods described in the above aspects.

[0015] Compared to existing technologies, the workflow scheduling method for cross-domain data centers provided in this application has the following beneficial effects: First, this application constructs a data center operation state vector including multi-dimensional data such as electricity price, carbon emission intensity, energy consumption coefficient per unit workload, and energy efficiency, as well as a directed acyclic graph containing task dependency constraints in the workflow. Based on this, the execution time, transmission time, completion time, power cost, and carbon emissions of each task in any workflow are obtained for each data center. The target workflow and the optimal task to be scheduled are determined by the average effective computing power and average transmission bandwidth of each data center, as well as the execution time and transmission time of the task, thus reducing the risk of overall workflow timeout caused by local optima of a single task. Second, after determining the task to be scheduled, the target data center for processing the task is determined based on the elastic available capacity and load rate of the data center, as well as the completion time, power cost, and carbon emissions of the task to be scheduled. This realizes the unified inclusion of power cost, computing power capacity, and carbon emissions into the cross-domain data center workflow scheduling process, avoiding the overall performance degradation caused by only taking the lowest electricity cost, lowest carbon emissions, or most idle computing power as a single objective. Attached Figure Description

[0016] Figure 1 This is a flowchart of a workflow scheduling method for cross-domain data centers according to this application; Figure 2 This is a structural diagram of a workflow scheduling device for cross-domain data centers according to this application; Detailed Implementation

[0017] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.

[0018] Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The module divisions appearing in the embodiments of this application are merely logical divisions; in actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be ignored or not performed. In addition, the shown or discussed mutual couplings or direct couplings or communication connections may be through some interfaces, and the indirect couplings or communication connections between modules may be electrical or other similar forms, none of which are limited in the embodiments of this application. Moreover, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0019] With the rapid growth of AI training, inference services, scientific computing, video processing, and large-scale data analytics, geographically distributed data centers have become crucial infrastructure for handling complex workflows. Data centers in different regions exhibit significant differences in electricity prices, carbon emission intensity, renewable energy output, weather conditions, network links, and available computing resources. Furthermore, different sub-tasks within the same workflow face dependencies, data transmission, and deadline constraints. Therefore, task scheduling across geographically distributed data centers is no longer simply a matter of computing power allocation, but rather a collaborative optimization problem involving electricity costs, computing capacity, carbon emissions, and service quality. Existing scheduling methods typically use a single objective—task completion time, resource utilization, or electricity cost—as the optimization basis, making it difficult to make stable decisions in conflicting scenarios such as low electricity prices but slow network transmission, low carbon emission intensity but insufficient computing capacity, and sufficient computing power but data center power approaching its limit. While some methods take into account dynamic electricity prices or energy consumption, they do not adequately consider the directed acyclic graph dependencies of the workflow, subtask deadlines, rigid load protection, and flexible load migration across regions. This can easily lead to situations such as local low cost but overall timeout, low-carbon migration causing transmission bottlenecks, or flexible tasks crowding out online service resources.

[0020] Therefore, to solve the above problem, please refer to Figure 1 This application provides a workflow scheduling method for cross-domain data centers, applied to a data center management system, mainly including: Step S1: Construct the operating status vector of each data center in the cross-domain data center set in the same scheduling cycle. The operating status vector includes the data center's electricity price, carbon emission intensity, energy consumption coefficient per unit workload, energy use efficiency, and network connection status.

[0021] Specifically, the first step is to establish a cross-domain data center cluster. Each of the data centers This includes computing resources, storage resources, network links, and power supply and distribution facilities.

[0022] For any scheduling period t, the data collection data center Electricity price, carbon emission intensity, energy consumption coefficient per unit of work, PUE, and network connectivity status form an operating state vector:

[0023] in, For data centers The running state vector, Indicates data center The electricity price at time t Indicates data center Carbon emission intensity at time t Indicates data center Energy consumption coefficient per unit workload at time t Indicates data center Energy efficiency at time t This indicates the network connection status of the data center at any given time.

[0024] Step S2: Receive the workflow set and construct the directed acyclic graph of each workflow; based on each directed acyclic graph and each running state vector, obtain the execution time, transmission time, completion time, power cost and carbon emissions of each task in the workflow set for each data center.

[0025] Specifically, in receiving the workflow collection submitted by the user Then, for any of the workflows Model it as a directed acyclic graph ,in For a set of task nodes, This is the set of edges that the task depends on. If edges exist: This indicates the task. Must be in the task Execution can only begin after the data transfer is complete.

[0026] Task start time Must meet ,in This refers to the completion time of the precursor mission. To the task To the mission Data transmission time.

[0027] For the task If it is allocated to a data center Its execution time is expressed as:

[0028] in, Indicates the computational load of the task. Indicates data center The available effective computing power per unit time, if the task Precursor mission In data center Execute on, while the current task In data center If executed above, its transmission time is:

[0029] in, For the task To the mission The amount of data transmitted For data centers and Real-time bandwidth between tasks. The completion time is:

[0030] in, For the task The set of precursor tasks.

[0031] Furthermore, the above-mentioned electricity costs and carbon emissions for each task in the workflow set of each data center are obtained based on each directed acyclic graph and each running state vector, including: Step S21: Calculate the task energy consumption allocated to the data center based on the task's computational load, the unit workload energy consumption coefficient in the running state vector, and the energy usage efficiency.

[0032] Step S22: Calculate the power cost and carbon emissions of the task allocated to the data center based on the task energy consumption, electricity price in the running state vector, and carbon emission intensity.

[0033] Specifically, the task Assigned to data center Then, its task energy consumption is calculated as follows:

[0034] in, Indicates data center Energy consumption coefficient per unit workload at time t Indicates task The computational load, Energy consumption for the task.

[0035] Corresponding electricity cost for:

[0036] Corresponding carbon emission data for:

[0037] Step S3: Calculate the comprehensive ranking value of each workflow based on the average effective computing power and average transmission bandwidth of the cross-domain data center set, and select the workflow with the smallest comprehensive ranking value as the target workflow.

[0038] Specifically, the comprehensive ranking value of each workflow calculated based on the average effective computing power and average transmission bandwidth of the cross-domain data center set includes: Step S31: Obtain the normalized computation scale factor of the workflow based on the workload of the workflow, the computational workload of each task in the workflow, and the number of workflows to be scheduled in the current scheduling cycle.

[0039] Specifically, when there are multiple workflows to be scheduled within the same scheduling period, the workflow scheduling order will be determined based on the urgency of the overall deadline of the workflows and the computational scale.

[0040] For workflow Its normalized scaling factor is:

[0041] in, Indicates task The computational load, Workflow The task quantity in the figure, N represents the number of workflows to be scheduled in the current scheduling period.

[0042] Step S32: Based on the computational load, data volume, average effective computing power of the cross-domain data center set, and average transmission bandwidth of the task, obtain the estimated execution time and estimated transmission time of each task.

[0043] Workflow is estimated based on average effective computing power and average transmission bandwidth. The earliest completion time, specifically, calculating the average effective computing power of the cross-domain data center collection:

[0044] in, This represents the effective computing power per unit time for all M data centers. Indicates data center The effective computing power per unit time, where M represents the number of data centers.

[0045] For the task Its estimated execution time is:

[0046] Calculate the average transmission bandwidth based on the network bandwidth between cross-domain data centers. and the task To the mission The estimated transmission time is expressed as:

[0047] in, Indicates task To the mission The amount of data transmitted.

[0048] Step S33: Based on the workflow topology, the estimated execution time and estimated transmission time of each task, obtain the estimated earliest start time of each task.

[0049] Specifically, for the entry task in the workflow topology sequence, its estimated earliest start time is:

[0050] For non-entry-level tasks, the earliest estimated start time is:

[0051] Task The earliest estimated completion time is:

[0052] Workflow The earliest estimated completion time is:

[0053] Step S34: Determine the normalized relaxation time factor of the workflow based on the estimated earliest start time of each task in the workflow and the overall deadline of the workflow; calculate the comprehensive ranking value of the workflow based on the normalized relaxation time factor and the normalized computational scale factor.

[0054] Furthermore, calculate the relaxation time of the workflow:

[0055] in Workflow The overall deadline.

[0056] Furthermore, the normalized relaxation time factor of the workflow is calculated:

[0057] The overall sorting value is:

[0058] in, and The weights for the relaxation time factor and the computational scale factor are respectively, satisfying:

[0059] According to the comprehensive ranking value Workflows are sorted from smallest to largest, with priority given to workflows with shorter relaxation times. When relaxation times are close, workflows with larger computational scales are given priority. Therefore, the workflow with the smallest overall ranking value is selected as the target workflow for current scheduling.

[0060] Step S4: Determine the tasks to be scheduled in the target workflow based on the execution time and transmission time.

[0061] The above-mentioned determination of the tasks to be scheduled in the target workflow based on execution time and transmission time includes: Step S41: Obtain the estimated execution time of the task based on the execution time of the task in each data center; specifically, the median or average execution time of the task in each data center can be taken as the estimated execution time of the task. Alternatively, model or algorithmic predictions can be made based on the execution time of the task in each data center, and the prediction results can be used as the expected execution time of the task. .

[0062] Step S42: Obtain the upward sorting value based on the expected execution time and transmission time of the task:

[0063] in, For the task The set of successor tasks can be obtained through a directed acyclic graph.

[0064] Step S43: Obtain the sub-deadline of the task based on the task's upward sorting value, the estimated execution time, and the overall deadline of the target workflow.

[0065] Let the maximum upward sorting value of the entry task be:

[0066] in For workflow The set of entry tasks.

[0067] Then the task Sub-deadline It can be set to:

[0068] Step S44: Obtain the priority value of the task based on the computational load, upward sorting value, and sub-deadline, and select the task with the highest priority value in the target workflow as the task to be scheduled.

[0069] Specifically, a task scheduling sequence is generated based on task dependencies and critical path information. For workflows... Only tasks that meet the following conditions can enter the candidate scheduling set:

[0070] For tasks that enter the candidate scheduling set, they are sorted according to their priority values:

[0071] in, , and These are weighting coefficients, based on priority values. Tasks are selected in descending order of size as tasks to be scheduled and enter the resource allocation phase.

[0072] Step S5: Calculate the elastic available capacity of each data center, determine the target data center based on the elastic available capacity, load rate, completion time of the task to be scheduled, power cost and carbon emissions of each data center, and issue the task to be scheduled.

[0073] Specifically, this application divides data center load into rigid load and elastic load. Rigid load refers to online services that are non-migratable, non-delayable, or have strict requirements for low latency; elastic load refers to workflow tasks that are delayable or can be migrated across regions.

[0074] Furthermore, the method also includes: Step S501: Obtain the elastic load base capacity of the future window and each data center, and predict the predicted electricity price and predicted carbon emission intensity of each data center within the future window.

[0075] For data centers Its total physical computing power capacity is The capacity occupied by rigid loads is Then the data center The elastic load base capacity at time t is:

[0076] in, This is for safety redundancy capacity, used to avoid scheduling overload. If the calculation yields: Then let Then, based on the operating state vectors of each data center, the future window of each data center is predicted. Forecasted electricity prices within and predicted carbon emission intensity , where H is the prediction step size.

[0077] Step S502: Based on the elastic load base capacity, predicted electricity price and predicted carbon emission intensity, obtain the elastic available capacity of each data center and generate the carbon-electricity co-elastic capacity curve for each data center.

[0078] For data centers Calculate its capacity adjustment coefficient at time t. :

[0079] in, and The influence weights of electricity price and carbon emission intensity are respectively, and they satisfy:

[0080] in , , This represents the minimum predicted electricity price for all data centers within the current future window. Similarly.

[0081] , , This represents the minimum predicted carbon intensity for all data centers within the current future window. Similarly.

[0082] To avoid overly drastic capacity adjustment, the above capacity adjustment coefficients must satisfy:

[0083] in and These are preset lower and upper limits for capacity adjustment, used to limit the fluctuation range of the elastic capacity curve. The final result is the data center... Elastic available capacity at time t:

[0084] The carbon-electricity co-elastic capacity curve is constructed from the elastic available capacity at all times within the future window:

[0085] When the predicted electricity price or predicted carbon intensity of a data center is high at a certain moment, This reduces the available elastic workflow capacity at that moment; when predicted electricity prices and predicted carbon intensity are low, This increases the capacity of elastic workflows that can be received at that moment, thereby enabling peak shaving and valley filling and low-carbon, low-cost scheduling of cross-regional data center workflows in the time dimension.

[0086] Specifically, the above-mentioned determination of target data centers based on the elastic availability capacity, load rate, completion time of tasks to be scheduled, power costs, and carbon emissions of each data center includes: Step S51: Select data centers whose elastic available capacity is greater than or equal to the computational load of the task to be scheduled and whose completion time is less than or equal to the sub-deadline of the task to be scheduled as candidate data centers.

[0087] For tasks to be scheduled, a candidate data center set is selected from the cross-domain data center set:

[0088] in, This indicates that the data center has sufficient elastic load capacity available during the current scheduling cycle. This indicates that executing the task in this data center will not violate the sub-deadline. If the western set in the candidate data is empty, that is... If the conditions for low carbon or low electricity price are relaxed, data centers that can meet the sub-deadline constraints will be selected first; if no data center can meet the sub-deadline, the data center with the earliest completion time will be selected and the SLA risk mark will be recorded.

[0089] Step S52: Calculate the comprehensive evaluation value of the candidate data center based on its load rate, completion time of the scheduled tasks, power cost, and carbon emissions.

[0090] Calculate the comprehensive evaluation value of the candidate data centers. Specifically, for any candidate data center in the set... Computational tasks Assigned to The overall evaluation values ​​are as follows:

[0091] in , , , Let the weighting coefficients satisfy: . Represents the normalized electricity cost. Indicates normalized carbon emissions. Indicates the normalization completion time. The normalized load factor of a data center is calculated using the following formula:

[0092] in Indicates data center The used computing load at time t, and the other parameters with subscripts max or min are the maximum or minimum values ​​among the target parameters (electricity cost, carbon emissions, completion time) of all candidate data centers, which will not be elaborated here.

[0093] Step S53: Select the candidate data center with the smallest comprehensive evaluation value as the target data center.

[0094] Compared to existing technologies, the workflow scheduling method for cross-domain data centers provided in this application has the following beneficial effects: First, this application constructs a data center operation state vector including multi-dimensional data such as electricity price, carbon emission intensity, energy consumption coefficient per unit workload, and energy efficiency, as well as a directed acyclic graph containing task dependency constraints in the workflow. Based on this, the execution time, transmission time, completion time, power cost, and carbon emissions of each task in any workflow are obtained for each data center. The target workflow and the optimal task to be scheduled are determined by the average effective computing power and average transmission bandwidth of each data center, as well as the execution time and transmission time of the task, thus reducing the risk of overall workflow timeout caused by local optima of a single task. Second, after determining the task to be scheduled, the target data center for processing the task is determined based on the elastic available capacity and load rate of the data center, as well as the completion time, power cost, and carbon emissions of the task to be scheduled. This realizes the unified inclusion of power cost, computing power capacity, and carbon emissions into the cross-domain data center workflow scheduling process, avoiding the overall performance degradation caused by only taking the lowest electricity cost, lowest carbon emissions, or most idle computing power as a single objective.

[0095] Furthermore, data center power and computing power status exhibit significant temporal variations. Electricity prices and carbon emission intensity fluctuate over time, data center PUE is affected by environmental factors such as temperature, cross-regional link bandwidth varies with network congestion, and the rigid load generated by online services changes with business requests. If the scheduling system only performs static allocation at the moment a task arrives, without considering future power, carbon emission, and computing power status for rolling optimization, it will be difficult to fully utilize low electricity prices, low carbon emission intensity, and idle computing power windows. To address this issue, in some embodiments, the method further includes: Step S61: Update the elastic availability capacity of the target data center based on the computational workload of the tasks to be scheduled, and collect the actual running status of the tasks to be scheduled. Specifically, distribute the tasks to be scheduled to the target data center. Then, the local resource manager of the target data center allocates specific computing instances based on the local resource status and updates the elastic available capacity of the data center:

[0096] Simultaneously record the task's start time, estimated completion time, energy consumption, electricity cost, and carbon emissions:

[0097] Once the task is completed, the corresponding computing resources are released, and the schedulable status of subsequent tasks is updated.

[0098] Step S62: Determine whether the deviation between the actual operating state and the predicted state exceeds a preset state threshold, or whether the deviation between the updated elastic available capacity and the carbon-electricity co-operated elastic capacity curve exceeds a preset capacity threshold. The predicted state includes the predicted electricity price and the predicted carbon emission intensity. If so, the target data center is redefined. Specifically, after scheduling, the deviation between the actual operating state and the predicted state is continuously monitored. The predicted state includes the predicted electricity price, predicted carbon emission intensity, and the predicted elastic available capacity corresponding to the carbon-electricity co-operated elastic capacity curve obtained in the aforementioned steps at the current moment. If the deviation between the actual electricity price, the carbon emission intensity calculated through carbon emission inversion, or the updated actual elastic available capacity and the predicted value exceeds the corresponding threshold, a rescheduling judgment is triggered, i.e., returning to step S5:

[0099] in, and These are the electricity price deviation threshold and the carbon emission intensity deviation threshold, which are preset state thresholds. This is the threshold for the available capacity deviation.

[0100] The above embodiments improve the scheduling robustness of cross-regional data centers in dynamic environments by setting a rolling feedback optimization mechanism. When the deviation between the actual operating state and the predicted state exceeds a threshold, local rescheduling is performed on delayed, interruptible, or migrated tasks.

[0101] Please refer to Figure 2 Another embodiment of this application provides a workflow scheduling device for cross-domain data centers, applied in a data center management system, mainly including: Module 101 is used to construct the operating status vector of each data center in the cross-domain data center set in the same scheduling cycle. The operating status vector includes the data center's electricity price, carbon emission intensity, energy consumption coefficient per unit workload, energy use efficiency, and network connection status.

[0102] Modeling module 102 is used to receive the workflow set and construct the directed acyclic graph of each workflow; based on each directed acyclic graph and each running state vector, the execution time, transmission time, completion time, power cost and carbon emissions of each task in the workflow set for each data center are obtained.

[0103] The sorting module 103 is used to calculate the comprehensive sorting value of each workflow based on the average effective computing power and average transmission bandwidth of the cross-domain data center set, and to select the workflow with the smallest comprehensive sorting value as the target workflow; and to determine the tasks to be scheduled in the target workflow based on the execution time and transmission time.

[0104] The scheduling module 104 is used to calculate the elastic available capacity of each data center, determine the target data center based on the elastic available capacity, load rate, completion time of the task to be scheduled, power cost and carbon emissions of each data center, and issue the task to be scheduled.

[0105] Furthermore, the device also includes: The elastic capacity calculation module is used to obtain the elastic load base capacity of future windows and each data center, predict the predicted electricity price and predicted carbon emission intensity of each data center in the future window; obtain the elastic available capacity of each data center based on the elastic load base capacity, predicted electricity price and predicted carbon emission intensity, and generate the carbon-electricity coordinated elastic capacity curve of each data center.

[0106] Furthermore, the device also includes: The rolling optimization module is used to update the elastic available capacity of the target data center according to the computational load of the task to be scheduled, and collect the actual running status of the task to be scheduled; determine whether the deviation between the actual running status and the predicted status exceeds the preset status threshold, or whether the deviation between the updated elastic available capacity and the carbon-electricity co-operation elastic capacity curve exceeds the preset capacity threshold. The predicted status includes the predicted electricity price and the predicted carbon emission intensity; if so, return to the scheduling module 104.

[0107] The specific limitations of the workflow scheduling device for cross-domain data centers provided in this embodiment can be found in the embodiment of the workflow scheduling method for cross-domain data centers described above, and will not be repeated here. Each module in the aforementioned workflow scheduling device for cross-domain data centers can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0108] This application provides a computer device that may include a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it causes the processor to perform the steps of a workflow scheduling method for cross-domain data centers as described in any of the above embodiments. The working process, details, and technical effects of the computer device provided in this embodiment can be found in the above embodiments regarding a workflow scheduling method for cross-domain data centers, and will not be repeated here.

[0109] This application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of a workflow scheduling method for cross-domain data centers as described in any of the above embodiments. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or memory sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The working process, details, and technical effects of the computer-readable storage medium provided in this embodiment can be found in the embodiments of a workflow scheduling method for cross-domain data centers described above, and will not be repeated here.

[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0111] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A workflow scheduling method for cross-domain data centers, characterized in that, include: Construct an operational state vector for each data center in a cross-domain data center set during the same scheduling cycle. The operational state vector includes the data center's electricity price, carbon emission intensity, energy consumption coefficient per unit workload, energy efficiency, and network connection status. Receive a workflow set and construct a directed acyclic graph for each workflow; based on each directed acyclic graph and each running state vector, obtain the execution time, transmission time, completion time, power cost, and carbon emissions of each task in the workflow set for each data center; The comprehensive ranking value of each workflow is calculated based on the average effective computing power and average transmission bandwidth of the cross-domain data center set, and the workflow with the smallest comprehensive ranking value is selected as the target workflow. The tasks to be scheduled in the target workflow are determined based on execution time and transmission time. Calculate the elastic availability capacity of each of the data centers, determine the target data center based on the elastic availability capacity, load rate, completion time, power cost and carbon emissions of the task to be scheduled, and issue the task to be scheduled.

2. The workflow scheduling method for cross-domain data centers according to claim 1, characterized in that, The process of obtaining the power cost and carbon emissions of each data center for each task in the workflow set based on each of the directed acyclic graphs and each of the running state vectors includes: The task energy consumption allocated to the data center is calculated based on the computational workload of the task, the energy consumption coefficient per unit workload in the running state vector, and the energy usage efficiency. The power cost and carbon emissions of the task allocated to the data center are calculated based on the task energy consumption, the electricity price in the operating state vector, and the carbon emission intensity.

3. The workflow scheduling method for cross-domain data centers according to claim 1, characterized in that, The calculation of the comprehensive ranking value of each workflow based on the average effective computing power and average transmission bandwidth of the cross-domain data center set includes: The normalized computation scale factor of the workflow is obtained based on the workload of the workflow, the computational workload of each task in the workflow, and the number of workflows to be scheduled in the current scheduling cycle. The estimated execution time and estimated transmission time for each task are obtained based on the computational load, data volume, average effective computing power of the cross-domain data center set, and average transmission bandwidth. The estimated earliest start time of each task is obtained based on the topological order of the workflow, the estimated execution time of each task, and the estimated transmission time of each task. The normalized relaxation time factor of the workflow is determined based on the estimated earliest start time of each task in the workflow and the overall deadline of the workflow; the comprehensive ranking value of the workflow is calculated based on the normalized relaxation time factor and the normalized computational scale factor.

4. The workflow scheduling method for cross-domain data centers according to claim 3, characterized in that, The process of determining the tasks to be scheduled in the target workflow based on execution time and transmission time includes: The estimated execution time of a task is obtained based on the execution time of the task in each data center. The upward sorting values ​​are obtained based on the expected execution time and transmission time of the task; The sub-deadline of the task is obtained based on the task's upward sorting value, estimated execution time, and the overall deadline of the target workflow; The priority value of the task is obtained based on the computational load, upward sorting value, and sub-deadline. The task with the highest priority value in the target workflow is selected as the task to be scheduled.

5. The workflow scheduling method for cross-domain data centers according to claim 4, characterized in that, The determination of target data centers based on the elastic availability capacity, load rate, completion time, power cost, and carbon emissions of the tasks to be scheduled for each data center includes: Data centers with elastic available capacity greater than or equal to the computational load of the task to be scheduled and completion time less than or equal to the sub-deadline of the task to be scheduled are selected as candidate data centers. The comprehensive evaluation value of the candidate data center is calculated based on its load rate, completion time of the scheduled task, power cost, and carbon emissions. The candidate data center with the lowest comprehensive evaluation value is selected as the target data center.

6. The workflow scheduling method for cross-domain data centers according to claim 1, characterized in that, Also includes: Obtain the elastic load base capacity of each data center within the future window, predict the predicted electricity price and predicted carbon emission intensity of each data center within the future window; obtain the elastic available capacity of each data center based on the elastic load base capacity, predicted electricity price and predicted carbon emission intensity, and generate the carbon-electricity co-elastic capacity curve for each data center.

7. The workflow scheduling method for cross-domain data centers according to claim 6, characterized in that, Also includes: The elastic available capacity of the target data center is updated based on the computational workload of the task to be scheduled, and the actual running status of the task to be scheduled is collected. Determine whether the deviation between the actual operating state and the predicted state exceeds a preset state threshold, or whether the deviation between the updated elastic available capacity and the carbon-electricity synergistic elastic capacity curve exceeds a preset capacity threshold. The predicted state includes the predicted electricity price and the predicted carbon emission intensity. If so, the target data center is redefined.

8. A workflow scheduling device for cross-domain data centers, characterized in that, include: The construction module is used to construct the operating status vector of each data center in the cross-domain data center set in the same scheduling cycle. The operating status vector includes the data center's electricity price, carbon emission intensity, energy consumption coefficient per unit workload, energy use efficiency, and network connection status. The modeling module is used to receive a workflow set and construct a directed acyclic graph for each workflow; based on each directed acyclic graph and each running state vector, the execution time, transmission time, completion time, power cost and carbon emissions of each task in the workflow set for each data center are obtained; The sorting module is used to calculate the comprehensive sorting value of each workflow based on the average effective computing power and average transmission bandwidth of the cross-domain data center set, and to select the workflow with the smallest comprehensive sorting value as the target workflow. The tasks to be scheduled in the target workflow are determined based on execution time and transmission time. The scheduling module is used to calculate the elastic available capacity of each of the data centers, determine the target data center based on the elastic available capacity, load rate, completion time, power cost and carbon emissions of the task to be scheduled, and issue the task to be scheduled.

9. A computer device, characterized in that, It includes at least one processor, a memory, and a transceiver; wherein the memory is used to store program code, and the processor is used to call the program code stored in the memory to perform the method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, It includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-7.

Images