Task scheduling method and system, storage medium and computer program product

Abstract

The application provides a task scheduling method, system, storage medium and computer program product, and relates to cloud computing and big data task scheduling. The method comprises the following steps: receiving a first task; obtaining the CPU utilization of the first task, determining the first CPU utilization of each host after the first task is accepted; calculating the first power consumption of each host after the first task is accepted; selecting a host with first power consumption not greater than a preset power consumption threshold to obtain at least one candidate host; obtaining the current power consumption of each candidate host and determining the current power consumption range to which the current power consumption belongs; from the at least one candidate host, selecting a candidate host with the largest current power consumption range and the closest distance between the first power consumption and the upper limit value of the current power consumption range as a target host; and distributing the first task to the target host. The application can more accurately predict the power consumption of the host, thereby improving the rationality of task scheduling and improving the energy efficiency of the system.

Description

Technical Field

[0001] This application relates to the field of cloud computing and big data task scheduling, specifically to task scheduling methods, systems, storage media, and computer program products. Background Technology

[0002] With the rapid development of big data and cloud computing technologies, energy consumption has become an increasingly prominent issue. For example, statistics show that global data centers consume approximately 3-5% of total electricity, and this proportion continues to rise. High energy consumption not only increases operating costs but also puts enormous pressure on the environment. Therefore, how to improve energy efficiency while ensuring service quality has become a focus of common concern for both academia and industry.

[0003] Big data task scheduling is a key factor affecting data center energy efficiency. The characteristics of big data tasks, such as high concurrency, data intensity, and computational intensity, pose significant challenges to scheduling. Furthermore, different types of tasks have vastly different resource requirements, making energy optimization a complex multi-objective optimization problem—ensuring service level agreement (SLA) requirements are met. Existing solutions generally use simplified linear models to describe host (server) power consumption characteristics. Especially in heterogeneous computing environments, the power consumption characteristics of hosts (servers) with different hardware configurations vary significantly, and a single model cannot accurately describe the complex power consumption behavior. This lack of modeling accuracy directly affects the effectiveness of energy optimization, potentially causing scheduling decisions to deviate from the optimal solution. Therefore, existing big data task scheduling methods suffer from low and inaccurate scheduling efficiency and high energy consumption. Summary of the Invention

[0004] At least one embodiment of this application provides a task scheduling method, system, storage medium, and computer program product to solve the problems of low scheduling efficiency, inaccuracy, and high energy consumption in the prior art big data task scheduling.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a task scheduling method, executed by a task scheduling system, the method comprising:

[0007] Receive the first task to be assigned;

[0008] Obtain the CPU utilization of the first task, and determine the first CPU utilization of each host after accepting the first task based on the CPU utilization of the first task and the current CPU utilization of each host.

[0009] According to the first CPU utilization of each host after receiving the first task and a piecewise quadratic model between the power consumption of the host and the CPU utilization of the host obtained in advance, the first power consumption of each host after receiving the first task is calculated; a host whose first power consumption is not greater than a preset power consumption threshold is selected to obtain at least one candidate host;

[0010] The current power consumption of each candidate host is obtained, and a current power consumption bin to which the current power consumption belongs is determined, wherein the plurality of power consumption bins include a plurality of power consumption bins in ascending order, and each power consumption bin corresponds to a power consumption interval;

[0011] From the at least one candidate host, a candidate host whose current power consumption bin is the largest and whose first power consumption is closest to an upper limit value of a current power consumption interval corresponding to the current power consumption bin is selected as a target host;

[0012] The first task is allocated to the target host.

[0013] Optionally, the piecewise quadratic model between the power consumption of the host and the CPU utilization of the host includes:

[0014] In a case where the CPU utilization of the host is less than a preset power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host;

[0015] In a case where the CPU utilization of the host is greater than or equal to the power consumption saturation point, the power consumption of the host is a fixed value.

[0016] Optionally, according to the first CPU utilization of each host after receiving the first task and the piecewise quadratic model between the power consumption of the host and the CPU utilization of the host obtained in advance, the first power consumption of each host after receiving the first task is calculated, including:

[0017] The first power consumption of each host after receiving the first task is calculated according to the following formula:

[0018]

[0019] wherein, P1 is the first power consumption of the host after receiving the first task, u1 is the first CPU utilization of the host after receiving the first task, a1, a2, a3 and a4 are respectively a first fitting coefficient, a second fitting coefficient, a third fitting coefficient and a fourth fitting coefficient determined in advance, Pmax is the preset power consumption saturation point.

[0020] Optionally, before the first task to be allocated is received, the method further includes:

[0021] constructing a piecewise quadratic model between the power consumption of the host and the CPU utilization of the host, the piecewise quadratic model comprising: in the case that the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host being a quadratic function of the CPU utilization of the host, the quadratic function comprising a first fitting coefficient, a second fitting coefficient, and a third fitting coefficient to be fitted; in the case that the CPU utilization of the host is greater than or equal to the preset power consumption saturation point, the power consumption of the host being a fourth fitting coefficient to be fitted;

[0022] collecting the CPU utilization of the host and the power consumption of the host in the case that the host runs different loads;

[0023] fitting the piecewise quadratic model by using the collected CPU utilization of the host and the power consumption of the host, determining the values of the fitting coefficients, and obtaining the piecewise quadratic model between the power consumption of the host and the CPU utilization of the host.

[0024] Optionally, the method further comprises:

[0025] determining the maximum task power consumption according to the power consumption required by the host to perform various tasks;

[0026] determining a power consumption range, the minimum value of the power consumption range being 0 and the maximum value of the power consumption range being the power consumption threshold;

[0027] dividing the power consumption range into at least one power consumption interval with the maximum task power consumption as a division unit, starting from the power consumption of 0, until the remaining power consumption range is not greater than the maximum task power consumption, and taking the remaining power consumption range as a power consumption interval, to obtain a plurality of power consumption intervals corresponding to the plurality of power consumption gears.

[0028] Optionally, the method further comprises:

[0029] obtaining the current power consumption of each host and determining the high-power-consumption host and / or the low-power-consumption host, wherein the current power consumption of the high-power-consumption host is greater than a first threshold, and the current power consumption of the low-power-consumption host is less than or equal to a second threshold;

[0030] repeating the following steps until the power consumption of the high-power-consumption host is not greater than the first threshold: migrating the tasks on the high-power-consumption host to a normal-power-consumption host, wherein the power consumption of the normal-power-consumption host is not greater than the first threshold after receiving the tasks of the high-power-consumption host;

[0031] and / or,

[0032] repeating the following steps until all tasks of the low-power host are migrated and the low-power host is shut down: migrating tasks on the low-power host to a normal-power host, wherein the normal-power host has a power consumption no greater than the first threshold after receiving the tasks of the low-power host;

[0033] wherein the normal-power host is a remaining host other than the high-power host and the low-power host.

[0034] Optionally, the method further comprises:

[0035] The first threshold and the second threshold each have an initial value and are dynamically adjusted according to a preset threshold adjustment statistical period, wherein,

[0036] The first threshold of the current statistical period is calculated according to the following formula:

[0037] The second threshold of the current statistical period is calculated according to the following formula:

[0038] wherein, is an average value of the first threshold in a historical statistical period, is a standard deviation of the power consumption of each high-power host in the historical statistical period, is a first adjustment coefficient, and the first adjustment coefficient is positively correlated with the SLA requirement of the host; is an average value of the second threshold in a historical statistical period, is a standard deviation of the power consumption of each low-power host in the historical statistical period, is a second adjustment coefficient, and the second adjustment coefficient is positively correlated with the SLA requirement of the host.

[0039] Optionally, the method further comprises:

[0040] optimizing the current tasks of the task scheduling system according to a preset offline adjustment period, wherein the optimization specifically comprises:

[0041] determining all tasks currently executed by each host to obtain a task set when each offline adjustment period arrives;

[0042] obtaining predicted power consumptions of each task in the task set according to historical data of CPU utilization and power consumption of various tasks collected in advance;

[0043] ​​According to the predicted power consumptions of the tasks in the task set, the allocation of the tasks in the task set among the hosts is iteratively optimized with the goal of reducing the total power consumption of the task scheduling system, and an optimization result is obtained;

[0044] According to the optimization result, the tasks in the task set are allocated.

[0045] In a second aspect, the embodiments of the present application provide a task scheduling system, comprising:

[0046] An online scheduler configured to receive a first task to be allocated;

[0047] A power consumption modeler configured to obtain a CPU utilization of the first task, and determine a first CPU utilization of each host after the first task is accepted according to the CPU utilization of the first task and a current CPU utilization of each host; and calculate a first power consumption of each host after the first task is accepted according to the first CPU utilization of each host after the first task is accepted and a piecewise quadratic model between the power consumption of the host and the CPU utilization of the host obtained in advance.

[0048] The online scheduler is further configured to select a host whose first power consumption is not greater than a preset power consumption threshold, to obtain at least one candidate host; obtain a current power consumption of each candidate host, and determine a current power consumption bin to which the current power consumption belongs, wherein the plurality of power consumption bins comprise a plurality of power consumption bins in ascending order, and each power consumption bin corresponds to a power consumption interval; select, from the at least one candidate host, a candidate host whose current power consumption bin is the largest and whose first power consumption is closest to an upper limit of a current power consumption interval, as a target host, wherein the current power consumption interval is the power consumption interval corresponding to the current power consumption bin; and allocate the first task to the target host.

[0049] Optionally, the piecewise quadratic model between the power consumption of the host and the CPU utilization of the host comprises:

[0050] In a case where the CPU utilization of the host is less than a preset power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host.

[0051] In a case where the CPU utilization of the host is greater than or equal to the power consumption saturation point, the power consumption of the host is a fixed value.

[0052] Optionally, the power consumption modeler is further configured to calculate the first power consumption of each host after the first task is accepted according to the following formula:

[0053]

[0054] wherein, a first power consumption of the host after hosting the first task, a first CPU utilization of the host after hosting the first task, respectively, a first fitting coefficient, a second fitting coefficient, a third fitting coefficient and a fourth fitting coefficient, a preset power consumption saturation point.

[0055] Optionally, the power consumption modeler is further configured to:

[0056] construct a piecewise quadratic model between the power consumption of the host and the CPU utilization of the host, the piecewise quadratic model comprising: in a case where the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host being a quadratic function of the CPU utilization of the host, the quadratic function comprising the first fitting coefficient, the second fitting coefficient and the third fitting coefficient to be fitted; in a case where the CPU utilization of the host is greater than or equal to the preset power consumption saturation point, the power consumption of the host being a fourth fitting coefficient to be fitted;

[0057] collect the CPU utilization of the host and the power consumption of the host in a case where the host runs different loads;

[0058] fit the piecewise quadratic model using the collected CPU utilization of the host and the power consumption of the host, determine the values of the fitting coefficients, and obtain the piecewise quadratic model between the power consumption of the host and the CPU utilization of the host.

[0059] Optionally, the power consumption modeler further comprises:

[0060] a power consumption gear divider configured to:

[0061] determine a maximum task power consumption according to the power consumption required by the host to execute various tasks;

[0062] determine a power consumption range, the minimum value of the power consumption range being 0 and the maximum value of the power consumption range being the power consumption threshold;

[0063] divide the power consumption range into at least one power consumption interval starting from 0 with the maximum task power consumption as the division unit until the remaining power consumption range is not greater than the maximum task power consumption, and take the remaining power consumption range as a power consumption interval, to obtain a plurality of power consumption intervals corresponding to the plurality of power consumption gears.

[0064] Optionally, the power consumption modeler further comprises:

[0065] a high-power host identifier configured to obtain the current power consumption of each host and determine a high-power host, wherein the current power consumption of the high-power host is greater than a first threshold;

[0066] a low-power host identifier configured to obtain current power consumptions of the hosts and determine a low-power host, wherein the current power consumption of the low-power host is less than or equal to a second threshold value;

[0067] an offline scheduler configured to repeatedly perform the following processes until the power consumption of the high-power host is not greater than the first threshold value: migrating tasks on the high-power host to a normal-power host, wherein the power consumption of the normal-power host is not greater than the first threshold value after receiving the tasks of the high-power host; and / or repeatedly performing the following processes until all tasks of the low-power host are migrated: migrating tasks on the low-power host to a normal-power host, wherein the power consumption of the normal-power host is not greater than the first threshold value after receiving the tasks of the low-power host; and / or shutting down the low-power host;

[0068] wherein the normal-power host is a remaining host other than the high-power host and the low-power host.

[0069] Optionally, the first threshold value and the second threshold value each have an initial value; and the task scheduling system further comprises:

[0070] a threshold adjuster configured to dynamically adjust the first threshold value and the second threshold value according to a preset threshold adjustment statistical period, wherein,

[0071] the first threshold value of the current statistical period is calculated according to the following formula: ;

[0072] the second threshold value of the current statistical period is calculated according to the following formula: ;

[0073] wherein, is an average value of the first threshold value in a historical statistical period, is a standard deviation of the power consumptions of the high-power hosts in the historical statistical period, is a first adjustment coefficient, and the first adjustment coefficient is positively correlated with an SLA requirement of the host; is an average value of the second threshold value in the historical statistical period, is a standard deviation of the power consumptions of the low-power hosts in the historical statistical period, is a second adjustment coefficient, and the second adjustment coefficient is positively correlated with the SLA requirement of the host.

[0074] Optionally, the offline scheduler is further configured to optimize current tasks of the task scheduling system according to a preset offline adjustment period, wherein the optimization specifically includes:

[0075] At each offline adjustment period, all tasks currently executed by each host are determined to obtain a task set;

[0076] According to historical data of CPU utilization and power consumption of various tasks collected in advance, predicted power consumption of each task in the task set is obtained;

[0077] According to the predicted power consumption of each task in the task set, distribution of tasks in the task set among hosts is iteratively optimized with the goal of reducing total power consumption of the task scheduling system, and an optimization result is obtained;

[0078] According to the optimization result, tasks in the task set are distributed.

[0079] In a third aspect, an embodiment of the present application provides a task scheduling system, and the electronic device comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the processor executes the program or instruction, the electronic device executes the task scheduling method provided by the embodiments of the present application.

[0080] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a program. When the program is executed by a processor, the task scheduling method provided by the embodiments of the present application is implemented.

[0081] In a fifth aspect, an embodiment of the present application provides a computer program product comprising computer instructions. When the computer instructions are executed by a processor, the task scheduling method provided by the embodiments of the present application is implemented.

[0082] Compared with the prior art, the task scheduling method, device, equipment, medium and computer program product provided by the embodiments of the present application can more accurately predict the power consumption of the host by pre-establishing a segmented quadratic model between the power consumption of the host and the CPU utilization of the host. By selecting the candidate host with the maximum current power consumption range and the closest upper limit value of the first power consumption distance as the target host of the to-be-distributed task, the rationality of task distribution can be improved, the system energy consumption can be reduced, and the system energy efficiency can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0083] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments, and are not meant to limit the present application. Moreover, the same reference numerals are used throughout the accompanying drawings to represent same or similar components. In the drawings:

[0084] Figure 1 FIG. 1 is a schematic structural diagram of a task scheduling system according to an embodiment of the present application.

[0085] Figure 2 A flow chart of a task scheduling method according to an embodiment of the present application;

[0086] Figure 3 A fitting example diagram of CPU utilization and power consumption of a host;

[0087] Figure 4 An example diagram of task allocation according to an embodiment of the present application;

[0088] Figure 5 Another structural diagram of a task scheduling system according to an embodiment of the present application. DETAILED DESCRIPTION

[0089] The terms "first", "second", and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second" are generally of a kind and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in the present application means at least one of the connected objects. For example, "A or B" covers three scenarios, namely, scenario one: including A and not including B; scenario two: including B and not including A; scenario three: including A and B. The character " / " generally represents that the objects before and after are in an "or" relationship.

[0090] The term "indication" in the present application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). Among them, the direct indication can be understood as that the sender explicitly informs the receiver of specific information, operations to be performed or requested results, etc. in the sent indication; the indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or judges and determines the operation to be performed or the requested result according to the judgment result.

[0091] As described in the background, the existing scheme generally uses a simplified linear model to describe the server power consumption characteristics, ignoring the nonlinear characteristics of power consumption changes. In particular, in a heterogeneous computing environment, the power consumption characteristics of hosts (servers) with different hardware configurations differ significantly, and a single model is difficult to accurately describe the complex host (server) power consumption behavior. The lack of modeling accuracy directly affects the effectiveness of energy optimization, causing the scheduling decision to deviate from the optimal solution.

[0092] To solve the above problems, an embodiment of the present application provides a task scheduling method, which is executed by a task scheduling system. Figure 1A structural schematic diagram of a task scheduling system of an embodiment of the present application, the task scheduling system comprising five modules: a power consumption modeler, an online scheduler, an offline scheduler, a high-power host identifier (OHD), and a low-power host identifier (UHD). The task scheduling system performs scheduling of tasks in a host cluster composed of multiple hosts.

[0093] In the task scheduling system, the power consumption modeler uses an advanced algorithm to accurately capture the power consumption characteristics of servers under different load conditions, providing a reliable data basis for subsequent decision-making. The high-power host identifier and the low-power host identifier identify high-power hosts that may affect service quality and potential energy-saving opportunities by analyzing the load and power consumption state of servers in real time. Among them, the high-power host identifier focuses on servers close to the performance bottleneck, and adjusts resource allocation in a timely manner to ensure the satisfaction of service level agreement (SLA); while the low-power host identifier identifies servers with low utilization, providing opportunities for the system to integrate and shut down some hosts to save energy. The online task scheduler dynamically adjusts the task allocation strategy based on these real-time information, optimizing energy utilization while ensuring performance. At the same time, the offline task scheduler uses historical data and prediction models to perform global optimization when the system load is low, reorganizing task distribution to further improve long-term energy efficiency. Through close cooperation, the five modules form a closed-loop intelligent scheduling system, realizing the whole-process optimization from data collection, analysis to decision execution, and achieving the balance between performance and energy efficiency under dynamic load environment, providing strong support for the efficient operation of data centers.

[0094] The main functions of each module of the task scheduling system are described below.

[0095] 1. Power consumption modeler: uses a piecewise quadratic model to model the power consumption characteristics of servers under different loads, and updates and adjusts the model in real time to improve the accuracy of power consumption prediction, providing a reliable data basis for subsequent scheduling decisions. In this paper, the load is represented by CPU utilization.

[0096] 2. High-power host identifier and low-power host identifier: used to identify hosts with excessive power consumption and insufficient load, respectively. OHD identifies high-power hosts that may affect service quality by setting dynamic thresholds and combining multi-dimensional load indicators, ensuring that the service level agreement (SLA) of critical tasks is met. UHD uses a sliding time window and statistical analysis method to identify servers that are continuously in a low load state, providing decision-making basis for task integration and shutting down redundant servers, thereby achieving the goal of energy saving.

[0097] Specifically, the high-power host identifier is configured to acquire current power consumptions of the hosts and determine high-power hosts, wherein the current power consumption of each high-power host is greater than a first threshold.

[0098] The low-power host identifier is configured to acquire current power consumptions of the hosts and determine low-power hosts, wherein the current power consumption of each low-power host is less than or equal to a second threshold.

[0099] 3. Online scheduler and offline scheduler: The online scheduler and the offline scheduler are respectively responsible for real-time task allocation and periodic task integration. The online scheduler adopts a multi-objective optimization strategy to allocate tasks according to real-time task requirements and host load conditions, to ensure efficient execution of tasks while minimizing energy consumption. The offline scheduler performs statistical analysis based on historical data, and recombines and reallocates resources for tasks through a heuristic algorithm, to further improve resource utilization and energy-saving effect. The combination of the two achieves short-term and long-term energy efficiency optimization through dynamic adjustment and global optimization.

[0100] Specifically, the online scheduler is configured to receive a first task to be allocated.

[0101] The power consumption modeler is configured to acquire CPU utilization of the first task, and determine first CPU utilizations of the hosts after receiving the first task according to the CPU utilization of the first task and current CPU utilizations of the hosts; and calculate first power consumptions of the hosts after receiving the first task according to the first CPU utilizations of the hosts after receiving the first task and a piecewise quadratic model between power consumptions of the hosts and CPU utilizations of the hosts obtained in advance.

[0102] The online scheduler is further configured to select hosts with first power consumptions not greater than a preset power consumption threshold, to obtain at least one candidate host; acquire current power consumptions of the candidate hosts, and determine current power consumption ranges to which the current power consumptions belong, wherein the current power consumption ranges are included in multiple power consumption ranges in ascending order, and each power consumption range corresponds to a power consumption interval; select, from the at least one candidate host, a candidate host with the largest current power consumption range and the closest distance between the first power consumption and an upper limit of a current power consumption interval, as a target host, wherein the current power consumption interval is the power consumption interval corresponding to the current power consumption range; and allocate the first task to the target host.

[0103] The offline scheduler repeatedly performs the following processes until the power consumption of the high-power host is not greater than the first threshold value: migrating tasks on the high-power host to a normal-power host, wherein the power consumption of the normal-power host is not greater than the first threshold value after receiving the tasks of the high-power host; and / or repeatedly performing the following steps until the tasks of the low-power host are all migrated: migrating tasks on the low-power host to a normal-power host, wherein the power consumption of the normal-power host is not greater than the first threshold value after receiving the tasks of the low-power host; and shutting down the low-power host.

[0104] The normal-power host is a remaining host other than the high-power host and the low-power host. In the task scheduling system, the power consumption modeler is the cornerstone of the entire system and provides necessary data support for other components. The power consumption modeler accurately characterizes the power consumption characteristics of the host (server) under different loads (CPU utilization) through a piecewise quadratic model, and these data are directly input into the high-load and low-power host identifiers. Based on the data provided by the power consumption model, the two identifiers dynamically adjust the judgment threshold in combination with real-time monitoring information, thereby accurately identifying the overloaded and low-power hosts in the host cluster.

[0105] Specifically, the power consumption modeler is further configured to: construct a piecewise quadratic model between the power consumption of the host and the CPU utilization of the host, the piecewise quadratic model comprising: in a case where the CPU utilization of the host is less than the power saturation point, the power consumption of the host being a quadratic function of the CPU utilization of the host, the quadratic function comprising a first fitting coefficient, a second fitting coefficient, and a third fitting coefficient to be fitted; in a case where the CPU utilization of the host is greater than or equal to a preset power saturation point, the power consumption of the host being a fourth fitting coefficient to be fitted; collecting the CPU utilization of the host and the power consumption of the host in a case where the host runs different loads; fitting the piecewise quadratic model by using the collected CPU utilization of the host and the power consumption of the host, determining the values of the fitting coefficients, and obtaining the piecewise quadratic model between the power consumption of the host and the CPU utilization of the host.

[0106] The identification result is immediately passed to the online and offline task schedulers, forming a continuous decision chain. When the online scheduler receives a new task request, it will refer to the distribution of high and low power hosts to avoid assigning tasks to overloaded hosts, and will also give priority to those hosts with low load but not yet meeting the offline standard, in order to improve resource utilization.

[0107] The offline scheduler operates on a larger time scale, synthesizing power consumption model data, host load states, and historical task information to formulate a global optimization strategy. When enough low-power hosts are identified, the offline scheduler triggers a task consolidation process to migrate tasks from these hosts to other appropriately loaded hosts, thereby achieving energy saving goals.

[0108] The five modules are tightly coupled through a data sharing and feedback mechanism, forming an adaptive scheduling system. The power consumption model provides the basis for load identification, which in turn provides the basis for scheduling decisions. The scheduling results, in turn, affect the load state and power consumption characteristics of the hosts, thereby driving the continuous updating and optimization of the power consumption model. This closed-loop design ensures that the system can continuously optimize the balance between performance and energy efficiency in a dynamically changing environment.

[0109] According to at least one embodiment of the present application, the task scheduling system further comprises:

[0110] The power consumption range divider is configured to determine a maximum task power consumption based on the power consumption required by the hosts to execute various tasks, determine a power consumption range with a minimum value of 0 and a maximum value of the power threshold, and divide the power consumption range into at least one power consumption interval starting from 0 with the maximum task power consumption as the division unit until the remaining power consumption range is not greater than the maximum task power consumption, and take the remaining power consumption range as a power consumption interval, to obtain a plurality of power consumption intervals corresponding to the plurality of power consumption gears. Through the division of the power consumption gears, the power consumption scheduling system can fully utilize the current power consumption gear of the target host during task allocation, thereby improving the rationality of task scheduling.

[0111] According to at least one embodiment of the present application, the first threshold and the second threshold each have an initial value; and the task scheduling system further comprises a threshold adjuster. The threshold adjuster is configured to dynamically adjust the first threshold and the second threshold according to a preset threshold adjustment statistical period, wherein,

[0112] The first threshold T_H of the current statistical period is calculated according to the following formula: T_H = μ_H - ασ_H;

[0113] The second threshold T_L of the current statistical period is calculated according to the following formula: T_L = μ_L + γσ_L;

[0114] Wherein, μ H is the average value of the first threshold in a historical statistical period, σ H is the standard deviation of the power consumption of each high-power host in the historical statistical period, and α is the first adjustment coefficient, the value of which is positively correlated with the SLA requirement of the host; μ L is the average value of the second threshold in the historical statistical period, σ L is the standard deviation of the power consumption of each low-power host in the historical statistical period, and γ is the second adjustment coefficient, which is positively correlated with the SLA requirement of the host.

[0115] Through the above threshold adjuster, the embodiments of the present application can set reasonable thresholds, thereby ensuring the effect of subsequent task scheduling.

[0116] According to at least one embodiment of the present application, the offline scheduler is further configured to optimize the current tasks of the task scheduling system according to a preset offline adjustment period, wherein the optimization specifically includes:

[0117] Upon arrival of each offline adjustment period, all tasks currently executed by each host are determined to obtain a task set;

[0118] According to the historical data of CPU utilization and power consumption of various tasks collected in advance, the predicted power consumption of each task in the task set is obtained;

[0119] According to the predicted power consumption of each task in the task set, the distribution of the tasks in the task set among the hosts is iteratively optimized to reduce the total power consumption of the task scheduling system, and an optimization result is obtained;

[0120] According to the optimization result, the tasks in the task set are distributed.

[0121] Through the optimization scheduling performed by the offline scheduler according to the offline adjustment period, the rationality of task scheduling can be further improved.

[0122] The embodiments of the present application also provide a task scheduling method based on dynamic power consumption perception and load (CPU utilization) identification. Through accurate power consumption modeling and intelligent scheduling algorithm, the optimization of data center energy efficiency and the efficient scheduling of big data tasks are realized. Specifically, the key technical content of the method is as follows:

[0123] In the embodiment of the present application, after receiving a new task (e.g., a first task) to be allocated, the task scheduling system schedules among multiple hosts, determines a target host to execute the first task, and allocates the first task to the target host. The host can be a server. This task scheduling manner determines a host to execute a task after receiving the task, and thus can also be called an online scheduling method. This method is mainly executed by an online scheduler of the task scheduling system, and of course, involves cooperation among other modules (e.g., a power consumption modeler).

[0124] As shown in Figure 2 The method includes the following steps:

[0125] Step 11: receiving a first task to be allocated.

[0126] Step 12: obtaining a CPU utilization of the first task, and determining a first CPU utilization of each host after receiving the first task according to the CPU utilization of the first task and a current CPU utilization of each host.

[0127] Here, the embodiment of the present application pre-statistically obtains a CPU utilization of a task of each specification, and the CPU utilization is a statistical value. The CPU utilization of a task refers to a CPU utilization required by the task when executed by a host, i.e., a CPU resource occupied. The specification of a task can be represented by a process type and a memory occupation size, etc. Thus, for the first task in step 11, the CPU utilization of the first task can be determined according to the specification of the first task.

[0128] In step 12, the embodiment of the present application can calculate the CPU utilization of each host after allocating the first task to each host, respectively. Specifically, the current CPU utilization of each host is added to the CPU utilization of the first task to obtain the first CPU utilization of each host after receiving the first task.

[0129] Step 13: calculating a first power consumption of each host after receiving the first task according to the first CPU utilization of each host after receiving the first task and a piecewise quadratic model pre-obtained between the power consumption of the host and the CPU utilization of the host; and selecting a host with a first power consumption not greater than a preset power consumption threshold to obtain at least one candidate host.

[0130] Here, the embodiment of the present application pre-establishes a segmented quadratic model between the power consumption of the host and the CPU utilization of the host, which includes: in the case that the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host; in the case that the CPU utilization of the host is greater than or equal to a preset power consumption saturation point, the power consumption of the host is a fixed value. Thus, in step 13, the embodiment of the present application can calculate the first power consumption of each host after receiving the first task according to the following formula:

[0131] (Formula One)

[0132] wherein, is the first power consumption of the host after receiving the first task, is the first CPU utilization of the host after receiving the first task, are respectively a first fitting coefficient, a second fitting coefficient, a third fitting coefficient and a fourth fitting coefficient determined in advance, is a preset power consumption saturation point. In order to avoid the power consumption of the host being too high after receiving the first task, the embodiment of the present application pre-sets a power consumption threshold, if the calculated first power consumption of the host after receiving the first task is not greater than the power consumption threshold, the host is taken as a candidate host; if the power consumption threshold is exceeded, the host is not taken as a candidate host, and one or more candidate hosts can be selected. Considering some extreme cases, the first power consumption of all hosts may exceed the power consumption threshold, which indicates that the task quantity has exceeded the upper limit of the system processing capacity, which can be usually judged before receiving the task to avoid the above-mentioned situation.

[0133] Step 14, obtaining the current power consumption of each candidate host and determining the current power consumption range to which the current power consumption belongs in a plurality of power consumption ranges, the plurality of power consumption ranges including a plurality of power consumption ranges from small to large, and each power consumption range corresponding to a power consumption interval.

[0134] Here, the embodiment of the present application pre-divides a plurality of power consumption gears, and each power consumption gear corresponds to a power consumption interval. Specifically, the maximum task power consumption can be determined according to the power consumption required by the host to perform various tasks; then, a power consumption range is determined, for example, the minimum value of the power consumption range is 0, and the maximum value is the power consumption threshold; then, the power consumption range is divided into at least one power consumption interval, specifically, the maximum task power consumption is taken as the division unit, and the power consumption range is divided into at least one power consumption interval from 0, until the remaining power consumption range is not greater than the maximum task power consumption, and the remaining power consumption range is taken as a power consumption interval, to obtain a plurality of power consumption intervals. For example, assuming that the maximum power consumption unit is P0, the power consumption threshold is Pm, and assuming that the result of the upward rounding of Pm / P0 is n, then a total of n power gears are obtained, from small to large, in turn, (0, P0], (P0, 2P0], …, ((n-2)*P0, ((n-1)*P0], ((n-1)*P0, Pm]. In the embodiment of the present application, an index of each power consumption gear can be established, such as power consumption gear 1, power consumption gear 2, …. Among them, the power consumption gear can be represented by the index of the power consumption gear, and the larger the power consumption gear, the greater the power consumption. That is, when the index of the first power consumption gear is smaller than the index of the second power consumption gear, the power consumption in the first power consumption interval corresponding to the first power consumption gear is smaller than the power consumption in the second power consumption interval corresponding to the second power consumption gear. For example, the power consumption in the power consumption interval corresponding to the power consumption gear 1 is smaller than the power consumption in the power consumption interval corresponding to the power consumption gear 2.

[0135] Step 15, from the at least one candidate host, a candidate host with the maximum current power consumption gear and the first power consumption distance closest to the upper limit value of the current power consumption interval is selected as the target host, wherein the current power consumption interval is the power consumption interval corresponding to the current power consumption gear.

[0136] Here, when the target host is selected for the first task, the candidate host with the maximum current power consumption gear and the first power consumption distance closest to the upper limit value of the current power consumption gear is selected as the target host. If there is only one host with the maximum current power consumption gear, the host is taken as the target host; if there are multiple hosts with the maximum current power consumption gear, the candidate host with the first power consumption distance closest to the upper limit value of the current power consumption interval after the host accepts the first task can be selected as the target host. Through the above-mentioned manner, the first task can be preferentially assigned to the target host with a higher power consumption gear, and the current power consumption gear of the target host can be fully utilized.

[0137] Step 16, the first task is assigned to the target host.

[0138] Through the above steps, the embodiment of the application can more accurately predict the host power consumption by pre-establishing a segmented quadratic model between the power consumption of the host and the CPU utilization of the host, and can improve the rationality of task allocation, reduce system energy consumption, and improve system energy efficiency by selecting a candidate host with the maximum current power consumption range and the closest upper limit value of the first power consumption distance from the current power consumption interval as the target host of the task to be allocated.

[0139] First, the segmented quadratic model between the power consumption of the host and the CPU utilization of the host will be described.

[0140] The embodiment of the application can pre-construct a segmented quadratic model between the power consumption of the host and the CPU utilization of the host, which includes: in the case that the CPU utilization of the host is less than the power saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host, and the quadratic function includes a first fitting coefficient, a second fitting coefficient, and a third fitting coefficient to be fitted; in the case that the CPU utilization of the host is greater than or equal to a preset power saturation point, the power consumption of the host is a fourth fitting coefficient to be fitted. The formula expression of the specific model can refer to the above formula one, in which the first fitting coefficient, the second fitting coefficient, the third fitting coefficient, and the fourth fitting coefficient are to be fitted. The first fitting coefficient, the second fitting coefficient, the third fitting coefficient, and the fourth fitting coefficient are to be fitted, respectively.

[0141] Then, in the case that the host runs different loads, the CPU utilization of the host and the power consumption of the host are collected, so as to obtain a large amount of data of the CPU utilization of the host and the power consumption of the host.

[0142] Then, the segmented quadratic model is fitted by using the collected CPU utilization of the host and the power consumption of the host, the values of each fitting coefficient are determined, and thus the segmented quadratic model between the power consumption of the host and the CPU utilization of the host is obtained.

[0143] The embodiment of the application can realize the above modeling process through a host power consumption modeler. Taking a server (i.e. the host in the above) as an example, a specific modeling example is provided.

[0144] This example accurately models the power consumption characteristics of the server under different loads, providing basic data support for efficient task scheduling and energy efficiency optimization. Here, the design and implementation of the host power consumption modeler will be described in detail, including the laboratory test method, the verification process on a large-scale experimental data set, and the performance evaluation of the final model.

[0145] A. Laboratory test method

[0146] To establish an accurate power consumption model, first, a stress test is performed on the server in a laboratory environment to collect power consumption data under different load conditions. The experiment uses a server from a certain brand, equipped with two CPUs from a certain manufacturer (2.10 GHz) and 128 GB of RAM. The virtual machines in the experiment run common big data applications such as databases, etc. The specific experimental process is as follows:

[0147] 1. Set up the experimental environment: install the KVM virtualization platform on the test server and deploy a set of virtual machines. These virtual machines run different workloads, including compiling GNU gcc-11.2.0, processing Apache server requests, running MySQL benchmark tests, calculating trigonometric functions, and performing Linpack benchmark tests.

[0148] 2. Gradually increase the load: start with 6 virtual machines and add one virtual machine at a time until there are 16 virtual machines. Each experiment runs for 600 seconds, followed by a 300-second pause to prevent overheating.

[0149] 3. Data collection: collect the server's power consumption data in real time through the SNMP protocol, and collect CPU utilization and other load data using the Linux system's NET-SNMP-EXTEND-MIB module. The power consumption data is collected at a frequency of 20 times per second, and then averaged.

[0150] Through experimental data collection, the relationship curve between the host's power consumption and its CPU utilization is as shown in Figure 2 .

[0151] Figure 3 The blue points in the figure represent the CPU utilization and power consumption information collected in the laboratory environment, the orange line is the optimal fitting of the point set through the linear model, the green line represents the optimal fitting of the point set through the quadratic model, and the red line represents the fitting of the point set using the segmented quadratic model of the present example. From the distribution of points in Figure 2 , it can be intuitively known that the fitting accuracy of the linear model is poor, and using a simple quadratic model for fitting, the power consumption tends to decrease when the CPU utilization is high, which does not conform to the physical law. However, using the segmented quadratic model, the point distribution can be well fitted, and the power consumption tends to be stable when the host CPU utilization reaches a certain threshold. The mathematical expression of the segmented quadratic model is given in Equation 1 above.

[0152] B. Large-scale data set verification

[0153] To verify the effectiveness of the established model, tests were conducted on large-scale experimental datasets. The experimental data came from a large data computing platform in a real production environment, containing long-term running data of multiple servers. The data collection period was every 5-6 minutes, lasting 24 hours. During data processing, for power consumption data collection, a simple moving average (SMA) algorithm was used for smoothing to eliminate the influence of outliers. During model training and verification, the dataset was first divided into training and test sets according to time sequence, and then the model performance was evaluated through cross-validation. The specific steps are as follows:

[0154] 1. Dataset division: divide the entire dataset into parts in chronological order, and select one part as the test set each time, and the remaining part as the training set, and cycle times.

[0155] 2. Model training: in each cycle, use the training set data to fit the piecewise quadratic model and adjust the model parameters.

[0156] 3. Model testing: use the test set data to evaluate the prediction performance of the model, calculate the mean absolute error (MAE) and mean absolute percentage error (MAPE), and the specific calculation method is:

[0157]

[0158]

[0159] where and are the observed and predicted values, is the sample size.

[0160] The experimental results show that for the mainstream models of big data services, the average absolute error (MAE) of the model is less than 1W on average, and the average absolute percentage error (MAPE) is less than 3%.

[0161] Figure 2 The method shown can be performed by an online scheduler. As can be seen, in the embodiments of the present application, the online and offline schedulers optimize the resource utilization and energy efficiency of the data center through real-time and periodic task scheduling. Among them, the online scheduler adopts a "ladder" matching rule and combines a multi-objective optimization strategy to perform real-time task allocation, ensuring efficient resource utilization and energy efficiency optimization in the short term; the offline scheduler performs statistical analysis based on historical data, and further improves the overall energy efficiency of the data center through heuristic algorithms for global reorganization and resource reallocation of tasks.

[0162] The specific functions of the online scheduler will be described in more detail below in conjunction with the above method.

[0163] The core idea of the online scheduler is to adopt a "ladder" matching rule, and to perform real-time task allocation through a multi-objective optimization strategy. It divides the host power consumption into multiple gears, and according to the load (CPU utilization) of the task, it predicts the power consumption of each host after receiving the first task, and selects the host that can make the current power consumption gear the highest (but still within the safe range) to allocate the task. This can not only fully utilize the host resources and reduce the number of active hosts, but also reserve enough space for future large-scale tasks by reasonably setting the gears.

[0164] The ladder allocation rule specifically divides the host power consumption range into multiple power consumption gears, for example, the specifications of large data tasks are 1u, 2u, 4u, 8u, 16u, and 32u, and the maximum power consumption of the task is selected as a unit of a gear, for example, 32u a gear. When a new task (such as the first task) arrives, according to its CPU utilization, the power consumption of each host after receiving the first task is predicted , and the host that can make the power consumption gear the highest but still within the safe range is selected to receive the first task, which can not only fully utilize the host resources, but also reserve enough space for future large-scale tasks, avoiding the situation that large-scale tasks cannot be allocated due to "host fragmentation".

[0165] Figure 4 An example of allocating a newly arrived task (such as the first task) is provided, assuming that there are 5 hosts in the host cluster, and the power consumption threshold lines of each host are the same, and a part of the host resources cannot be utilized, otherwise it will risk generating a power hotspot. The safe space below the power consumption threshold line of each host is divided into 3 power consumption gears, from small to large, they are one-gear space, two-gear space and three-gear space. The current amount of resources utilized by each host is Figure 4 marked in blue-green. When a new task arrives, the power consumption of the host after receiving the first task can be calculated, and at this time it needs to be matched to the most suitable host. Host 4 (host-4) will reach the power hotspot area after joining the new task, so it is excluded. Host 5 (host-5) has the lowest current resource utilization gear, and will be used to save energy in the future, so it is also excluded. Among the remaining hosts 1, 2, 4, the new task is allocated to host 2 (host-2), which is closest to the upper limit of the current power consumption gear, so the task is allocated to host 2.

[0166] In the embodiments of the present application, the task scheduling system can further identify high and low power consumption hosts through the high and low power consumption host identifier, and schedule tasks of the high and low power consumption hosts through the offline task scheduler. This process relies on accurate power consumption data provided by the host power consumption modeler, and combines multi-dimensional power consumption indicators and a dynamic threshold adjustment mechanism to ensure efficient operation and energy efficiency optimization of the system under various load conditions.

[0167] Specifically, the task scheduling method further includes the following steps:

[0168] In step a, the current power consumption of each host is obtained, and a high power consumption host and / or a low power consumption host are determined, wherein the high power consumption host is a host with a current power consumption greater than a first threshold, and the low power consumption host is a host with a current power consumption less than or equal to a second threshold.

[0169] Here, the real-time power consumption data of each host can be collected by the power consumption modeler to monitor the power consumption state of each host.

[0170] In step b, the following steps are repeatedly performed until the power consumption of the high power consumption host is not greater than the first threshold: migrating tasks on the high power consumption host to a normal power consumption host, wherein after receiving the tasks of the high power consumption host, the power consumption of the normal power consumption host is not greater than the first threshold.

[0171] And / or,

[0172] The following steps are repeatedly performed until the tasks of the low power consumption host are all migrated: migrating tasks on the low power consumption host to a normal power consumption host, wherein after receiving the tasks of the low power consumption host, the power consumption of the normal power consumption host is not greater than the first threshold.

[0173] The normal power consumption host is a remaining host other than the high and low power consumption hosts.

[0174] Here, when migrating tasks of high and low power consumption hosts, the CPU utilization of the task to be migrated can be determined in a similar manner as described above, and then the CPU utilization of each normal power consumption host after the task to be migrated is migrated to the normal power consumption host is determined, and then the power consumption of the normal power consumption host after migration is calculated using Formula One. If it does not exceed the first threshold, the task can be migrated to the normal power consumption host. Details are not repeated here.

[0175] In this embodiment, the first threshold can be a pre-set high power consumption threshold, for example, 80% of the rated power consumption can be used as the threshold. The second threshold can also be a pre-set low power consumption threshold, for example, 20% of the rated power consumption can be used as the threshold. The above thresholds can be flexibly set based on experience, and this embodiment does not specifically limit them.

[0176] As another implementation method, the first and second thresholds can also be dynamically adjusted based on historical power consumption data, thereby ensuring efficient operation and energy efficiency optimization of the system under various power consumption conditions. The identification process of high-power and low-power hosts will be explained below.

[0177] The primary function of a high-power host identifier is to identify hosts whose power consumption is close to or exceeds their capacity limits, preventing service quality degradation and triggering corresponding scheduling operations. When a host's power consumption exceeds a certain threshold, it leads to a decrease in CPU frequency and processing power, resulting in longer task queuing times, increased task latency, and SLA breaches. The process of identifying high-power hosts can be divided into the following steps:

[0178] 1. Data Acquisition and Preprocessing: Monitor the current power consumption status of the host through real-time power consumption data provided by the host power consumption modeler.

[0179] 2. Based on historical data and the current system status, set a dynamic first threshold. This first threshold is not a fixed value, but is adjusted based on SLA requirements, the overall power consumption of the system, and historical power consumption trends. The first threshold has an initial value, for example, initially set to 70% of the rated power consumption. Subsequent adjustments are made dynamically according to a preset statistical period. This statistical period has a preset length. For example, the first threshold in the current statistical period... One dynamic adjustment formula is as follows:

[0180]

[0181] in, It is the average value of the first threshold over the historical statistical period. It is the standard deviation of power consumption of various high-power hosts within the historical statistical period. This is the first adjustment factor, and the value of the first adjustment factor is positively correlated with the SLA requirements of the host computer. The higher the SLA requirements, the better. The larger the value of , the better. The smaller the value, the lower the high load threshold line.

[0182] 3. High-power host identification: For each host, obtain its current power consumption. ,like , then the host is marked as a high-power host.

[0183] 4. Triggering scheduling operation: Once a high-power host is identified, the task scheduling system will perform resource reallocation or task migration according to the scheduling strategy to alleviate the pressure of the high-power host and ensure stable operation and service quality of the system. For example, part of the tasks can be migrated to the host of the normal-power host. In addition, the embodiments of the present application can also reduce the task load on the high-power host by dynamically adjusting the resource allocation strategy.

[0184] The main function of the low-power host identifier is to identify those hosts with low power consumption and capable of task integration, so as to put these hosts into sleep state or shut down to achieve the energy saving goal. The process of identifying low-power hosts can be divided into the following steps:

[0185] 1. Data collection and preprocessing: The current power consumption state of the host is monitored through the real-time power consumption data provided by the host power consumption modeler.

[0186] 2. According to the historical data and the current system state, a dynamic second threshold is set . The second threshold is not a fixed value, but is adjusted according to the SLA requirement level, the overall power consumption of the system and the historical power consumption trend. The second threshold has an initial value, for example, initially set to 20% of the rated power consumption. Subsequently, it is dynamically adjusted according to the statistical period. For example, a dynamic adjustment formula of the second threshold in the current statistical period is as follows:

[0187] ;

[0188] is the average value of the second threshold in the historical statistical period, is the standard deviation of the power consumption of each low-power host in the historical statistical period, is the second adjustment coefficient, which is positively correlated with the SLA requirement of the host. The higher the SLA requirement, the larger the value, and the larger the value of , i.e. the higher the low-load threshold line.

[0189] 3. Low-power host identification: For each host, its current power consumption is obtained, and if , the host is marked as a low-power host.

[0190] 4. Triggering a scheduling operation: Once the low-power host is identified, the task scheduling system will perform task migration according to the scheduling strategy, migrate the tasks on these low-power hosts to other hosts, so as to put the low-power hosts into sleep state or shutdown, and achieve the energy saving goal. For example, the tasks on the low-power host can be migrated to the normal-power host, so as to integrate resources and reduce energy consumption.

[0191] In the embodiments of the present application, the offline scheduler of the task scheduling system can optimize the resource utilization and energy efficiency of the data center through periodic task scheduling. The offline task scheduler performs statistical analysis based on historical data, globally recombines and reallocates resources for tasks through heuristic algorithms, and further improves the overall energy efficiency of the data center. The offline scheduler uses intelligent merging and migration strategies based on in-depth cognition of task load and power consumption characteristics, maximizes resource utilization, and minimizes the number of active nodes as much as possible.

[0192] Specifically, the embodiments of the present application can set an offline adjustment period, for example, set to every 12 hours or every 24 hours as an offline adjustment period. The task scheduling method of the embodiments of the present application also optimizes the current tasks of the task scheduling system according to the preset offline adjustment period, wherein the optimization specifically includes the following steps:

[0193] 1. When each offline adjustment period arrives, determine all tasks currently executed by each host to obtain a task set.

[0194] 2. According to the historical data of the load characteristics (such as CPU utilization) and power consumption of various tasks collected in advance, the predicted power consumption of each task in the task set is obtained.

[0195] Here, the historical data of the load characteristics and power consumption of various tasks are collected in advance, so that the corresponding relationship between the load characteristics and power consumption of various tasks can be established. In this way, the corresponding relationship and the CPU utilization of the task can be used to predict the power consumption of each task in the task set, thereby obtaining the predicted power consumption.

[0196] 3. According to the predicted power consumption of each task in the task set, the distribution of the tasks in the task set among the hosts is iteratively optimized to reduce the total power consumption of the task scheduling system, and an optimization result is obtained.

[0197] In the above offline scheduling process, metaheuristic algorithms can be used for global recombination of tasks and resource reallocation. Metaheuristic algorithms gradually approach the optimal solution through iterative optimization, improving resource utilization and energy efficiency. Taking the Adaptive Large Neighborhood Search (ALNS) algorithm as an example, this algorithm improves the current solution by dynamically selecting different neighborhood operations (such as insertion, removal, and exchange). ALNS adjusts the neighborhood selection strategy based on historical performance during the iteration process to improve search efficiency. The specific process includes:

[0198] A. Initial solution generation: generate an initial scheduling scheme based on historical data and load prediction.

[0199] B. Neighborhood operation selection: in each iteration, select the appropriate neighborhood operation based on heuristic information and historical performance.

[0200] C. Solution update: generate a new solution using the selected neighborhood operation, and decide whether to accept the new solution based on certain criteria (such as energy efficiency improvement).

[0201] D. Strategy adjustment: dynamically adjust the selection probability of neighborhood operations based on their success rate to adaptively optimize the quality of the solution.

[0202] In addition, other metaheuristic algorithms such as genetic algorithms, particle swarm optimization, and simulated annealing can be used according to specific needs.

[0203] 4. According to the optimization result, allocate tasks in the task set.

[0204] Through the above steps and mechanisms, the online scheduler and offline scheduler of the embodiments of the present application can optimize system performance and energy efficiency in the short term and long term, respectively. This process relies on accurate data provided by the power consumption modeler and ensures efficient operation and energy efficiency optimization of the system under various load conditions through dynamic adjustment and feedback mechanisms.

[0205] From the above, it can be seen that the above method of the embodiments of the present application solves one or more of the following problems:

[0206] (1) Improve the accuracy of power consumption modeling: through the unique piecewise quadratic model, the embodiments of the present application can significantly improve the accuracy of power consumption prediction, and the power consumption prediction error can be reduced to within 3%, providing a reliable data foundation for energy efficiency optimization.

[0207] (2) Strengthen the SLA guarantee level: the embodiment of the application realizes multi-dimensional load characteristic analysis and dynamic resource allocation strategy, and the target is to guarantee that more than 98% of tasks meet the SLA requirement while maximizing the energy efficiency of the data center. The system will dynamically adjust resource allocation according to the priority, resource demand and performance index of the task, ensure the performance of the key task is not affected, and avoid over-allocation of resources.

[0208] (3) Improve the off-line cooperation efficiency of the scheduling system: the embodiment of the application constructs a scheduling system with high dynamic adaptability, which can shorten the response time to milliseconds in the case of load mutation. At the same time, through the intelligent off-line task reorganization strategy, the system will continuously optimize resource allocation in long-term operation. At the same time, through the cooperation of online scheduling and offline scheduling, global optimization of resource allocation is realized, and according to the change of environmental characteristics, real-time optimization is achieved.

[0209] The embodiment of the application also provides a task scheduling system, as shown in Figure 5 The processor executes the program or instruction, so that the electronic device executes the task scheduling method provided by the embodiment of the application, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

[0210] The embodiment of the application also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to realize various processes of the above task scheduling method embodiment, and the same technical effect can be achieved. To avoid repetition, details are not repeated here. The computer readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0211] The embodiment of the application also provides a computer program product, which includes computer instructions. The computer instructions are executed by a processor to realize various processes of the above task scheduling method embodiment, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

[0212] It should be noted that, in the present document, the terms "comprises / comprising" or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0213] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, also can be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of software product, and the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a plurality of instructions to make a terminal (which can be a mobile phone, computer, server, air conditioner, or network equipment, etc.) execute the method described in each embodiment of the present application.

[0214] The embodiments of the present application are described above in conjunction with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative, not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims.

Claims

1. A task scheduling method, characterized in that, Executed by a task scheduling system, the method includes: Receive the first task to be assigned; Obtain the CPU utilization of the first task, and determine the first CPU utilization of each host after accepting the first task based on the CPU utilization of the first task and the current CPU utilization of each host. Based on the first CPU utilization of each host after accepting the first task, and the piecewise quadratic model between the power consumption of the host and the CPU utilization of the host obtained in advance, the first power consumption of each host after accepting the first task is calculated; the hosts whose first power consumption is not greater than the preset power consumption threshold are selected to obtain at least one candidate host. Obtain the current power consumption of each candidate host and determine the current power consumption level among multiple power consumption levels, wherein the multiple power consumption levels include multiple power consumption levels from small to large, and each power consumption level corresponds to a power consumption range. From the at least one candidate host, select the candidate host with the highest current power consumption level and the first power consumption level closest to the upper limit of the current power consumption range as the target host, wherein the current power consumption range is the power consumption range corresponding to the current power consumption level; The first task is assigned to the target host.

2. The method according to claim 1, characterized in that, The piecewise quadratic model relating the host's power consumption to its CPU utilization includes: When the CPU utilization of the host is less than a preset power saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host. When the CPU utilization of the host is greater than or equal to the power saturation point, the power consumption of the host is a fixed value.

3. The method according to claim 2, characterized in that, Based on the first CPU utilization of each host after accepting the first task, and the piecewise quadratic model of the relationship between the host's power consumption and its CPU utilization obtained in advance, the first power consumption of each host after accepting the first task is calculated, including: The initial power consumption of each host after accepting the first task is calculated using the following formula: in, To determine the initial power consumption of the host after the first task is completed. To determine the initial CPU utilization of the host after accepting the first task. These are the pre-determined first, second, third, and fourth fitting coefficients, respectively. This is the preset power saturation point.

4. The method according to claim 3, characterized in that, Before receiving the first task to be assigned, the method further includes: A piecewise quadratic model is constructed between the power consumption of the host and the CPU utilization of the host. The piecewise quadratic model includes: when the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host, and the quadratic function includes a first fitting coefficient, a second fitting coefficient, and a third fitting coefficient to be fitted; when the CPU utilization of the host is greater than or equal to the preset power consumption saturation point, the power consumption of the host is a fourth fitting coefficient to be fitted. The CPU utilization and power consumption of the host are collected under different load conditions. By using the collected CPU utilization and power consumption of the host, the piecewise quadratic model is fitted, and the values ​​of each fitting coefficient are determined to obtain a piecewise quadratic model between the power consumption and CPU utilization of the host.

5. The method according to claim 1, characterized in that, Also includes: The maximum task power consumption is determined based on the power consumption required by the host to perform various tasks; A power consumption range is defined, wherein the minimum value of the power consumption range is 0 and the maximum value is the power consumption threshold; Using the maximum task power consumption as the dividing unit, starting from power consumption of 0, the power consumption range is divided into at least one power consumption interval until the remaining power consumption range is no greater than the maximum task power consumption. The remaining power consumption range is then used as a power consumption interval, resulting in multiple power consumption intervals, each corresponding to one of the multiple power consumption levels.

6. The method according to claim 1, characterized in that, Also includes: Obtain the current power consumption of each host and identify high-power hosts and / or low-power hosts, wherein the current power consumption of the high-power host is greater than a first threshold and the current power consumption of the low-power host is less than or equal to a second threshold. Repeat the following steps until the power consumption of the high-power host is no greater than the first threshold: migrate the tasks on the high-power host to the normal-power host, wherein, after accepting the tasks of the high-power host, the power consumption of the normal-power host is no greater than the first threshold; And / or, Repeat the following steps until all tasks on the low-power host have been migrated, and then shut down the low-power host: migrate the tasks on the low-power host to the normal power host, wherein, after accepting the tasks on the low-power host, the power consumption of the normal power host is not greater than the first threshold. The normal power consumption host refers to the remaining hosts other than the high power consumption host and the low power consumption host.

7. The method according to claim 6, characterized in that, Also includes: Both the first threshold and the second threshold have an initial value, and are dynamically adjusted according to a preset threshold adjustment statistical period, wherein, The first threshold of the current statistical period It is calculated according to the following formula: ; The second threshold for the current statistical period It is calculated according to the following formula: ; in, It is the average value of the first threshold over the historical statistical period. It is the standard deviation of power consumption of various high-power hosts within the historical statistical period. It is the first adjustment factor, and the value of the first adjustment factor is positively correlated with the SLA requirements of the host; It is the average value of the second threshold over the historical statistical period. It is the standard deviation of power consumption of each low-power host within the historical statistical period. It is the second adjustment factor, which is positively correlated with the SLA requirements of the host.

8. The method according to claim 6, characterized in that, Also includes: According to a preset offline adjustment cycle, the current tasks of the task scheduling system are optimized, wherein the optimization specifically includes: At the end of each offline adjustment cycle, determine all tasks currently being executed on each host to obtain a task set; Based on historical data of CPU utilization and power consumption of various tasks collected in advance, the predicted power consumption of each task in the task set is obtained. Based on the predicted power consumption of each task in the task set, with the goal of reducing the total power consumption of the task scheduling system, the allocation of tasks in the task set among the hosts is iteratively optimized to obtain the optimization result. Based on the optimization results, tasks in the task set are allocated.

9. A task scheduling system, characterized in that, include: An online scheduler is used to receive the first task to be assigned; A power consumption modeler is used to obtain the CPU utilization of the first task, and determine the first CPU utilization of each host after accepting the first task based on the CPU utilization of the first task and the current CPU utilization of each host; and calculate the first power consumption of each host after accepting the first task based on the first CPU utilization of each host after accepting the first task and the piecewise quadratic model between the power consumption of the host and the CPU utilization of the host obtained in advance. The online scheduler is further configured to select hosts whose first power consumption is not greater than a preset power consumption threshold, thereby obtaining at least one candidate host; obtain the current power consumption of each candidate host, and determine the current power consumption level among multiple power consumption levels, wherein the multiple power consumption levels include multiple power consumption levels from smallest to largest, and each power consumption level corresponds to a power consumption range; select the candidate host with the largest current power consumption level and whose first power consumption is closest to the upper limit of the current power consumption range from the at least one candidate host, as the target host, wherein the current power consumption range is the power consumption range corresponding to the current power consumption level; and assign the first task to the target host.

10. The task scheduling system according to claim 9, characterized in that, The piecewise quadratic model relating the host's power consumption to its CPU utilization includes: When the CPU utilization of the host is less than a preset power saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host. When the CPU utilization of the host is greater than or equal to the power saturation point, the power consumption of the host is a fixed value.

11. The task scheduling system according to claim 10, characterized in that, The power consumption modeler is also used to calculate the first power consumption of each host after accepting the first task, according to the following formula: in, To determine the initial power consumption of the host after the first task is completed. To determine the initial CPU utilization of the host after accepting the first task. These are the pre-determined first, second, third, and fourth fitting coefficients, respectively. This is the preset power saturation point.

12. The task scheduling system according to claim 11, characterized in that, The power consumption modeler is also used for: A piecewise quadratic model is constructed between the power consumption of the host and the CPU utilization of the host. The piecewise quadratic model includes: when the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host, and the quadratic function includes a first fitting coefficient, a second fitting coefficient, and a third fitting coefficient to be fitted; when the CPU utilization of the host is greater than or equal to the preset power consumption saturation point, the power consumption of the host is a fourth fitting coefficient to be fitted. The CPU utilization and power consumption of the host are collected under different load conditions. By using the collected CPU utilization and power consumption of the host, the piecewise quadratic model is fitted, and the values ​​of each fitting coefficient are determined to obtain a piecewise quadratic model between the power consumption and CPU utilization of the host.

13. The task scheduling system according to claim 9, characterized in that, Also includes: Power consumption level divider, used for: The maximum task power consumption is determined based on the power consumption required by the host to perform various tasks; A power consumption range is defined, wherein the minimum value of the power consumption range is 0 and the maximum value is the power consumption threshold; Using the maximum task power consumption as the dividing unit, starting from power consumption of 0, the power consumption range is divided into at least one power consumption interval until the remaining power consumption range is no greater than the maximum task power consumption. The remaining power consumption range is then used as a power consumption interval, resulting in multiple power consumption intervals, each corresponding to one of the multiple power consumption levels.

14. The task scheduling system according to claim 9, characterized in that, Also includes: A high-power host identifier is used to obtain the current power consumption of each host and identify high-power hosts, wherein the current power consumption of the high-power host is greater than a first threshold. A low-power host identifier is used to obtain the current power consumption of each host and identify low-power hosts, wherein the current power consumption of the low-power host is less than or equal to a second threshold. An offline scheduler is configured to repeatedly perform the following processes until the power consumption of the high-power host is no greater than the first threshold: migrating tasks from the high-power host to a normal-power host, wherein, after accepting tasks from the high-power host, the power consumption of the normal-power host is no greater than the first threshold; and / or, repeatedly performing the following processes until all tasks from the low-power host have been migrated and the low-power host is shut down: migrating tasks from the low-power host to a normal-power host, wherein, after accepting tasks from the low-power host, the power consumption of the normal-power host is no greater than the first threshold; The normal power consumption host refers to the remaining hosts other than the high power consumption host and the low power consumption host.

15. The task scheduling system according to claim 14, characterized in that, Both the first threshold and the second threshold have an initial value; the task scheduling system further includes: A threshold adjuster is used to dynamically adjust the first threshold and the second threshold according to a preset statistical period, wherein... The first threshold of the current statistical period It is calculated according to the following formula: ; The second threshold for the current statistical period It is calculated according to the following formula: ; in, It is the average value of the first threshold over the historical statistical period. It is the standard deviation of power consumption of various high-power hosts within the historical statistical period. It is the first adjustment factor, and the value of the first adjustment factor is positively correlated with the SLA requirements of the host; It is the average value of the second threshold over the historical statistical period. It is the standard deviation of power consumption of each low-power host within the historical statistical period. It is the second adjustment factor, which is positively correlated with the SLA requirements of the host.

16. The task scheduling system according to claim 14, characterized in that, The offline scheduler is further configured to optimize the current tasks of the task scheduling system according to a preset offline adjustment period, wherein the optimization specifically includes: At the end of each offline adjustment cycle, determine all tasks currently being executed on each host to obtain a task set; Based on historical data of CPU utilization and power consumption of various tasks collected in advance, the predicted power consumption of each task in the task set is obtained. Based on the predicted power consumption of each task in the task set, with the goal of reducing the total power consumption of the task scheduling system, the allocation of tasks in the task set among the hosts is iteratively optimized to obtain the optimization result. Based on the optimization results, tasks in the task set are allocated.

17. A task scheduling system, characterized in that, include: Processor, memory, and programs or instructions stored in said memory and executable on said processor; When the processor executes the program or instructions, it implements the steps of the method as described in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 8.

19. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 8.

Images