Task scheduling method and system, storage medium and computer program product

Abstract

The invention provides a task scheduling method and system, a storage medium and a computer program product, and relates to cloud computing and big data task scheduling. The method comprises the following steps: receiving a first task; obtaining a CPU utilization rate of the first task, and determining a first CPU utilization rate of each host after receiving the first task; calculating first power consumption of each host after the first task is accepted; selecting hosts of which the first power consumption is not greater than a preset power consumption threshold to obtain at least one candidate host; obtaining the current power consumption of each candidate host, and determining the current power consumption gear of the current power consumption in the plurality of power consumption gears; selecting the candidate host with the maximum current power consumption gear and the nearest first power consumption to the upper limit value of the current power consumption gear from the at least one candidate host, and taking the selected candidate host as a target host; and distributing the first task to the target host. The host power consumption can be predicted more accurately, so that the reasonability of task scheduling is improved, and the energy efficiency of the system is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing and big data task scheduling, and specifically to a task scheduling method, system, storage medium, and computer program product. Background Art

[0002] With the rapid development of big data and cloud computing technologies, energy consumption has become increasingly prominent. For example, statistics show that data centers account for approximately 3-5% of total global electricity consumption, and this proportion continues to rise. High energy consumption not only increases operating costs but also places significant pressure on the environment. Therefore, improving energy efficiency while ensuring service quality has become a common focus in academia and industry.

[0003] Big data task scheduling is a key factor affecting data center energy efficiency. The characteristics of big data tasks, such as high concurrency, data density, and computational intensity, pose significant scheduling challenges. Furthermore, different types of tasks have vastly different resource requirements. Optimizing energy consumption while maintaining service level agreements (SLAs) presents a complex multi-objective optimization problem. Existing solutions generally use simplified linear models to describe host (server) power consumption characteristics. This is particularly true in heterogeneous computing environments, where hosts (servers) with different hardware configurations exhibit significant power consumption variations. A single model struggles to accurately describe complex power consumption behaviors. This lack of modeling accuracy directly impacts energy optimization, potentially causing scheduling decisions to deviate from the optimal solution. Consequently, existing big data task scheduling methods suffer from low scheduling efficiency, inaccuracy, and high energy consumption. Summary of the Invention

[0004] At least one embodiment of the present application provides a task scheduling method, system, storage medium, and computer program product for solving the problems of low scheduling efficiency, inaccuracy, and high energy consumption in the prior art big data task scheduling.

[0005] In order to solve the above technical problems, this application is implemented as follows: In a first aspect, an embodiment of the present application provides a task scheduling method, which is executed by a task scheduling system, and the method includes: receiving a first task to be assigned; Obtaining the CPU utilization of the first task, and determining a first CPU utilization of each host after accepting the first task based on the CPU utilization of the first task and the current CPU utilization of each host; Calculating, based on the first CPU utilization of each host after accepting the first task and a pre-obtained piecewise quadratic model between the power consumption of the host and the CPU utilization of the host, a first power consumption of each host after accepting the first task; selecting a host whose first power consumption is not greater than a preset power consumption threshold to obtain at least one candidate host; Obtaining the current power consumption of each candidate host and determining the current power consumption level among a plurality of power consumption levels, wherein the plurality of power consumption levels include a plurality of power consumption levels from small to large, each power consumption level corresponding to a power consumption interval; Selecting, from the at least one candidate host, a candidate host whose current power consumption level is the highest and whose first power consumption is closest to an upper limit of a current power consumption range as the target host, wherein the current power consumption range is a power consumption range corresponding to the current power consumption level; Allocate the first task to the target host.

[0006] Optionally, the segmented quadratic model between the power consumption of the host and the CPU utilization of the host includes: When the CPU utilization of the host is less than a preset power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host; When the CPU utilization of the host is greater than or equal to the power consumption saturation point, the power consumption of the host is a fixed value.

[0007] Optionally, calculating the first power consumption of each host after accepting the first task based on the first CPU utilization of each host after accepting the first task and a pre-obtained piecewise quadratic model between the power consumption of the host and the CPU utilization of the host includes: The first power consumption of each host after accepting the first task is calculated according to the following formula:

[0008] in, The first power consumption of the host after accepting the first task, is the first CPU utilization of the host after accepting the first task, are respectively a predetermined first fitting coefficient, a second fitting coefficient, a third fitting coefficient and a fourth fitting coefficient, is the preset power consumption saturation point.

[0009] Optionally, before receiving the first task to be assigned, the method further includes: Constructing a piecewise quadratic model between the power consumption of the host and the CPU utilization of the host, the piecewise quadratic model including: when the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host, the quadratic function including a first fitting coefficient to be fitted, a second fitting coefficient, and a third fitting coefficient; when the CPU utilization of the host is greater than or equal to a preset power consumption saturation point, the power consumption of the host is a fourth fitting coefficient to be fitted; When the host runs different loads, collecting the CPU utilization rate and power consumption of the host; The segmented quadratic model is fitted using the collected CPU utilization and power consumption of the host, and the values ​​of each fitting coefficient are determined to obtain a segmented quadratic model between the power consumption of the host and the CPU utilization of the host.

[0010] Optionally, also include: Determine the maximum task power consumption based on the power consumption required by the host to perform various tasks; Determine a power consumption range, where a minimum value of the power consumption range is 0 and a maximum value is the power consumption threshold; Taking the maximum task power consumption as the division unit, starting from the power consumption of 0, the power consumption range is divided into at least one power consumption interval until the remaining power consumption range is no greater than the maximum task power consumption, and the remaining power consumption range is taken as a power consumption interval to obtain multiple power consumption intervals, which correspond to the multiple power consumption gears respectively.

[0011] Optionally, also include: Obtaining current power consumption of each host and determining a high-power consumption host and / or a low-power consumption host, wherein the current power consumption of the high-power consumption host is greater than a first threshold, and the current power consumption of the low-power consumption host is less than or equal to a second threshold; Repeating the following steps until the power consumption of the high-power-consumption host is no greater than the first threshold: migrating tasks on the high-power-consumption host to a host with normal power consumption, wherein after accommodating the tasks of the high-power-consumption host, the power consumption of the host with normal power consumption is no greater than the first threshold; and / or, Repeat the following steps until all tasks of the low-power host are migrated, and shut down the low-power host: migrate tasks on the low-power host to a normal-power host, wherein after accepting the tasks of the low-power host, the power consumption of the normal-power host is not greater than the first threshold; The normal power consumption hosts are the remaining hosts except the high power consumption hosts and the low power consumption hosts.

[0012] Optionally, also include: The first threshold and the second threshold both have an initial value and are dynamically adjusted according to a preset threshold adjustment statistical period, wherein: The first threshold value of the current statistical period It is calculated according to the following formula: ; The second threshold of the current statistical period It is calculated according to the following formula: ; in, is the average value of the first threshold in the historical statistical period, is the standard deviation of the power consumption of each high-power-consuming host in the historical statistical period. is a first adjustment coefficient, the value of which is positively correlated with the SLA requirement of the host; is the average value of the second threshold in the historical statistical period, is the standard deviation of the power consumption of each low-power host in the historical statistical period, is a second adjustment coefficient, and the second adjustment coefficient is positively correlated with the SLA requirement of the host.

[0013] Optionally, also include: Optimizing the current task of the task scheduling system according to a preset offline adjustment period, wherein the optimization specifically includes: When each offline adjustment period arrives, all tasks currently executed by each host are determined to obtain a task set; Obtaining predicted power consumption of each task in the task set based on pre-collected historical data on CPU utilization and power consumption of various tasks; According to the predicted power consumption of each task in the task set, with the goal of reducing the total power consumption of the task scheduling system, iteratively optimizing the distribution of the tasks in the task set among the hosts to obtain an optimization result; The tasks in the task set are allocated according to the optimization result.

[0014] In a second aspect, an embodiment of the present application provides a task scheduling system, including: An online scheduler, configured to receive a first task to be assigned; a power consumption modeler, configured to obtain the CPU utilization of the first task, and determine a first CPU utilization of each host after accepting the first task based on the CPU utilization of the first task and the current CPU utilization of each host; and calculate a first power consumption of each host after accepting the first task based on the first CPU utilization of each host after accepting the first task and a pre-obtained piecewise quadratic model between the power consumption of the host and the CPU utilization of the host; The online scheduler is further used to select a host whose first power consumption is not greater than a preset power consumption threshold to obtain at least one candidate host; obtain the current power consumption of each candidate host, and determine the current power consumption level to which the current power consumption belongs among multiple power consumption levels, wherein the multiple power consumption levels include multiple power consumption levels from small to large, and each power consumption level corresponds to a power consumption interval; from the at least one candidate host, select the candidate host whose current power consumption level is the largest and whose first power consumption is closest to the upper limit of the current power consumption interval as the target host, wherein the current power consumption interval is the power consumption interval corresponding to the current power consumption level; and assign the first task to the target host.

[0015] Optionally, the segmented quadratic model between the power consumption of the host and the CPU utilization of the host includes: When the CPU utilization of the host is less than a preset power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host; When the CPU utilization of the host is greater than or equal to the power consumption saturation point, the power consumption of the host is a fixed value.

[0016] Optionally, the power consumption modeler is further configured to calculate a first power consumption of each host after accepting the first task according to the following formula:

[0017] in, The first power consumption of the host after accepting the first task, is the first CPU utilization of the host after accepting the first task, are respectively a predetermined first fitting coefficient, a second fitting coefficient, a third fitting coefficient and a fourth fitting coefficient, is the preset power consumption saturation point.

[0018] Optionally, the power consumption modeler is further configured to: Constructing a piecewise quadratic model between the power consumption of the host and the CPU utilization of the host, the piecewise quadratic model including: when the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host, the quadratic function including a first fitting coefficient to be fitted, a second fitting coefficient, and a third fitting coefficient; when the CPU utilization of the host is greater than or equal to a preset power consumption saturation point, the power consumption of the host is a fourth fitting coefficient to be fitted; When the host runs different loads, collecting the CPU utilization rate and power consumption of the host; The segmented quadratic model is fitted using the collected CPU utilization and power consumption of the host, and the values ​​of each fitting coefficient are determined to obtain a segmented quadratic model between the power consumption of the host and the CPU utilization of the host.

[0019] Optionally, also include: Power consumption level divider for: Determine the maximum task power consumption based on the power consumption required by the host to perform various tasks; Determine a power consumption range, where a minimum value of the power consumption range is 0 and a maximum value is the power consumption threshold; Taking the maximum task power consumption as the division unit, starting from the power consumption of 0, the power consumption range is divided into at least one power consumption interval until the remaining power consumption range is no greater than the maximum task power consumption, and the remaining power consumption range is taken as a power consumption interval to obtain multiple power consumption intervals, which correspond to the multiple power consumption gears respectively.

[0020] Optionally, also include: a high-power-consumption host identifier, configured to obtain current power consumption of each host and identify a high-power-consumption host, wherein the current power consumption of the high-power-consumption host is greater than a first threshold; a low-power host identifier, configured to obtain current power consumption of each host and determine a low-power host, wherein the current power consumption of the low-power host is less than or equal to a second threshold; The offline scheduler is configured to repeatedly perform the following processing until the power consumption of the high-power consumption host is no more than the first threshold: migrating tasks on the high-power consumption host to a host with normal power consumption, wherein after accepting the tasks of the high-power consumption host, the power consumption of the host with normal power consumption is no more than the first threshold; and / or repeatedly performing the following processing until all tasks of the low-power consumption host are migrated, and shutting down the low-power consumption host: migrating tasks on the low-power consumption host to a host with normal power consumption, wherein after accepting the tasks of the low-power consumption host, the power consumption of the host with normal power consumption is no more than the first threshold; The normal power consumption hosts are the remaining hosts except the high power consumption hosts and the low power consumption hosts.

[0021] Optionally, the first threshold and the second threshold both have an initial value; and the task scheduling system further includes: A threshold adjuster is used to dynamically adjust the first threshold and the second threshold according to a preset threshold adjustment statistical period, wherein: The first threshold value of the current statistical period It is calculated according to the following formula: ; The second threshold of the current statistical period It is calculated according to the following formula: ; in, is the average value of the first threshold in the historical statistical period, is the standard deviation of the power consumption of each high-power-consuming host in the historical statistical period. is a first adjustment coefficient, the value of which is positively correlated with the SLA requirement of the host; is the average value of the second threshold in the historical statistical period, is the standard deviation of the power consumption of each low-power host in the historical statistical period, is a second adjustment coefficient, and the second adjustment coefficient is positively correlated with the SLA requirement of the host.

[0022] Optionally, the offline scheduler is further configured to optimize the current task of the task scheduling system according to a preset offline adjustment period, wherein the optimization specifically includes: When each offline adjustment period arrives, all tasks currently executed by each host are determined to obtain a task set; Obtaining predicted power consumption of each task in the task set based on pre-collected historical data on CPU utilization and power consumption of various tasks; Iteratively optimizing the distribution of tasks in the task set among the hosts based on the predicted power consumption of each task in the task set and aiming at reducing the total power consumption of the task scheduling system to obtain an optimization result; The tasks in the task set are allocated according to the optimization result.

[0023] In the third aspect, an embodiment of the present application provides a task scheduling system, which electronic device includes: a processor, a memory, and a program or instruction stored in the memory and runnable on the processor. When the processor executes the program or instruction, the electronic device executes the task scheduling method provided by the embodiment of the present application.

[0024] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, the task scheduling method provided in the embodiment of the application is implemented.

[0025] In a fifth aspect, an embodiment of the present application provides a computer program product, including computer instructions, which, when executed by a processor, implement the task scheduling method provided in the embodiment of the application.

[0026] Compared with the prior art, the task scheduling method, apparatus, equipment, medium and computer program product provided in the embodiments of the present application can more accurately predict the host power consumption by pre-establishing a segmented quadratic model between the host's power consumption and the host's CPU utilization. By selecting the candidate host with the largest current power consumption level and the first power consumption closest to the upper limit of the current power consumption range as the target host for the task to be assigned, the rationality of task allocation can be improved, system energy consumption can be reduced, and system energy efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings: Figure 1 A structural diagram of a task scheduling system according to an embodiment of the present application; Figure 2 This is a flowchart of a task scheduling method according to an embodiment of the present application; Figure 3 This is a sample diagram of the host's CPU utilization and power consumption. Figure 4 An example diagram of task allocation for an embodiment of the present application; Figure 5 This is another structural diagram of the task scheduling system according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in this application represents at least one of the connected objects. For example, "A or B" covers three options, namely, Option 1: including A but not including B; Option 2: including B but not including A; Option 3: including both A and B. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0029] The term "indication" in this application can be either a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, the required operation, or the requested result, etc. in the instruction sent; an indirect indication can be understood as the receiver determining the corresponding information based on the instruction sent by the sender, or making a judgment and determining the required operation or request result based on the judgment result.

[0030] As mentioned in the background, existing solutions generally use simplified linear models to describe server power consumption characteristics, ignoring the nonlinear nature of power consumption variations. In heterogeneous computing environments, hosts (servers) with different hardware configurations exhibit significant power consumption variations, making it difficult for a single model to accurately describe complex host (server) power consumption behavior. This lack of modeling accuracy directly impacts energy optimization, potentially causing scheduling decisions to deviate from the optimal solution.

[0031] To solve the above problems, an embodiment of the present application provides a task scheduling method, which is executed by a task scheduling system. Figure 1 This is a schematic diagram of the structure of a task scheduling system according to an embodiment of the present application. The task scheduling system includes five modules: a power consumption modeler, an online scheduler, an offline scheduler, a high-power host identifier (OHD), and a low-power host identifier (UHD). The task scheduling system schedules tasks within a host cluster consisting of multiple hosts.

[0032] In this task scheduling system, the power consumption modeler employs advanced algorithms to accurately capture the power consumption characteristics of servers under varying load conditions, providing a reliable data foundation for subsequent decision-making. The high-power host identifier and low-power host identifier analyze server load and power consumption in real time, identifying high-power hosts that could impact service quality and potential energy-saving opportunities, respectively. The high-power host identifier focuses on servers approaching performance bottlenecks, ensuring service level agreements (SLAs) are met through timely resource allocation adjustments. The low-power host identifier identifies servers with low utilization, providing opportunities for system consolidation and power-down to conserve energy. Based on this real-time information, the online task scheduler dynamically adjusts task allocation strategies to optimize energy utilization while ensuring performance. Meanwhile, the offline task scheduler leverages historical data and predictive models to perform global optimization when system load is low, reorganizing task distribution and further improving long-term energy efficiency. These five modules work closely together to form a closed-loop intelligent scheduling system, optimizing the entire process from data collection and analysis to decision execution. This system can achieve a balance between performance and energy efficiency under dynamic load conditions, providing strong support for efficient data center operations.

[0033] The main functions of each module of the task scheduling system are explained below.

[0034] 1. Power Consumption Modeler: This uses a segmented quadratic model to model the power consumption characteristics of servers under different loads. Through real-time data updates and model adjustments, it improves the accuracy of power consumption predictions and provides a reliable data foundation for subsequent scheduling decisions. In this article, CPU utilization is used to represent load.

[0035] 2. High-Power Host Identifier and Low-Power Host Identifier: These are used to identify hosts with overloaded and underloaded power consumption, respectively. OHD uses dynamic thresholds and combines multi-dimensional load metrics to promptly identify high-power hosts that could impact service quality, ensuring that service level agreements (SLAs) for critical missions are met. UHD utilizes sliding time windows and statistical analysis to identify servers with consistently low loads, providing a basis for decision-making on task consolidation and the shutdown of redundant servers, ultimately achieving energy savings.

[0036] Specifically, the high-power-consumption host identifier is used to obtain the current power consumption of each host and determine the high-power-consumption host, wherein the current power consumption of the high-power-consumption host is greater than a first threshold; The low-power host identifier is configured to obtain current power consumption of each host and determine a low-power host, wherein the current power consumption of the low-power host is less than or equal to a second threshold.

[0037] 3. Online Scheduler and Offline Scheduler: The online scheduler and offline scheduler are responsible for real-time task allocation and periodic task integration, respectively. The online scheduler uses a multi-objective optimization strategy to allocate tasks based on real-time task requirements and host load, ensuring efficient task execution while minimizing energy consumption. The offline scheduler performs statistical analysis based on historical data and uses heuristic algorithms to globally reorganize tasks and reallocate resources, further improving resource utilization and energy savings. The combination of the two, through dynamic adjustment and global optimization, achieves short-term and long-term energy efficiency optimization.

[0038] Specifically, the online scheduler is configured to receive a first task to be assigned; a power consumption modeler, configured to obtain the CPU utilization of the first task, and determine a first CPU utilization of each host after accepting the first task based on the CPU utilization of the first task and the current CPU utilization of each host; and calculate a first power consumption of each host after accepting the first task based on the first CPU utilization of each host after accepting the first task and a pre-obtained piecewise quadratic model between the power consumption of the host and the CPU utilization of the host; The online scheduler is further used to select a host whose first power consumption is not greater than a preset power consumption threshold to obtain at least one candidate host; obtain the current power consumption of each candidate host, and determine the current power consumption level to which the current power consumption belongs among multiple power consumption levels, wherein the multiple power consumption levels include multiple power consumption levels from small to large, and each power consumption level corresponds to a power consumption interval; from the at least one candidate host, select the candidate host whose current power consumption level is the largest and whose first power consumption is closest to the upper limit of the current power consumption interval as the target host, wherein the current power consumption interval is the power consumption interval corresponding to the current power consumption level; and assign the first task to the target host.

[0039] The offline scheduler, when performing offline scheduling, repeatedly performs the following processing until the power consumption of the high-power-consumption host is no more than the first threshold: migrating tasks on the high-power-consumption host to a normal-power-consumption host, wherein after accepting the tasks of the high-power-consumption host, the power consumption of the normal-power-consumption host is no more than the first threshold; and / or repeatedly performs the following steps until all tasks of the low-power-consumption host are migrated, and shuts down the low-power-consumption host: migrating tasks on the low-power-consumption host to a normal-power-consumption host, wherein after accepting the tasks of the low-power-consumption host, the power consumption of the normal-power-consumption host is no more than the first threshold; The normal power consumption hosts are the remaining hosts excluding the high-power and low-power hosts. In the aforementioned task scheduling system, the power consumption modeler is the cornerstone of the entire system, providing essential data support for other components. The power consumption modeler accurately characterizes the power consumption characteristics of hosts (servers) under varying loads (CPU utilization) using a segmented quadratic model. This data is directly input into the high-load and low-power host identifiers. Based on the data provided by the power consumption model and combined with real-time monitoring information, the two identifiers dynamically adjust their judgment thresholds to accurately identify overloaded and low-power hosts in the host cluster.

[0040] Specifically, the power consumption modeler is also used to: construct a segmented quadratic model between the power consumption of the host and the CPU utilization of the host, the segmented quadratic model including: when the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host, and the quadratic function includes a first fitting coefficient, a second fitting coefficient, and a third fitting coefficient to be fitted; when the CPU utilization of the host is greater than or equal to the preset power consumption saturation point, the power consumption of the host is a fourth fitting coefficient to be fitted; when the host runs different loads, the CPU utilization of the host and the power consumption of the host are collected; the segmented quadratic model is fitted using the collected CPU utilization of the host and the power consumption of the host, the values ​​of each fitting coefficient are determined, and the segmented quadratic model between the power consumption of the host and the CPU utilization of the host is obtained.

[0041] The identification results are then passed to the online and offline task schedulers, forming a continuous decision chain. When the online scheduler receives a new task request, it considers the distribution of high- and low-power hosts to avoid assigning tasks to overloaded hosts. It also prioritizes hosts with lower loads but not yet reaching the offline threshold to improve resource utilization.

[0042] The offline scheduler operates on a larger timescale, comprehensively analyzing power consumption model data, host load status, and historical task information to formulate a global optimization strategy. When enough low-power hosts are identified, the offline scheduler triggers a task consolidation process, migrating tasks from these hosts to other hosts with appropriate loads, thereby achieving energy savings.

[0043] These five modules are tightly coupled through data sharing and feedback mechanisms, forming an adaptive scheduling system. The power consumption model provides the basis for load identification, which in turn informs scheduling decisions. The scheduling results, in turn, influence the host's load status and power consumption characteristics, driving the continuous updating and optimization of the power consumption model. This closed-loop design ensures the system can continuously optimize the balance between performance and energy efficiency in a dynamically changing environment.

[0044] According to at least one embodiment of the present application, the task scheduling system further includes: The power consumption level divider is used to determine the maximum task power consumption based on the power consumption required by the host to perform various tasks; determine a power consumption range, wherein the minimum value of the power consumption range is 0 and the maximum value is the power consumption threshold; using the maximum task power consumption as the division unit, starting from the power consumption of 0, the power consumption range is divided into at least one power consumption interval until the remaining power consumption range is no greater than the maximum task power consumption, and the remaining power consumption range is used as a power consumption interval to obtain multiple power consumption intervals corresponding to the multiple power consumption levels. By dividing the power consumption levels, the power consumption scheduling system can make full use of the current power consumption level of the target host when allocating tasks to improve the rationality of task scheduling.

[0045] According to at least one embodiment of the present application, the first threshold and the second threshold both have an initial value; the task scheduling system further includes a threshold adjuster. The threshold adjuster is used to dynamically adjust the first threshold and the second threshold according to a preset threshold adjustment statistical period, wherein: The first threshold T_H of the current statistical period is calculated according to the following formula: T_H=μ_H-ασ_H; The second threshold T_L of the current statistical period is calculated according to the following formula: T_L=μ_L+γσ_L; Among them, μ_H is the average value of the first threshold within the historical statistical period, σ_H is the standard deviation of the power consumption of each high-power-consumption host within the historical statistical period, α is the first adjustment coefficient, and the value of the first adjustment coefficient is positively correlated with the SLA requirement of the host; μ_L is the average value of the second threshold within the historical statistical period, σ_L is the standard deviation of the power consumption of each low-power-consumption host within the historical statistical period, γ is the second adjustment coefficient, and the second adjustment coefficient is positively correlated with the SLA requirement of the host.

[0046] Through the above threshold adjuster, the embodiment of the present application can set a reasonable threshold, thereby ensuring the effect of subsequent task scheduling.

[0047] According to at least one embodiment of the present application, the offline scheduler is further configured to optimize the current task of the task scheduling system according to a preset offline adjustment period, wherein the optimization specifically includes: When each offline adjustment period arrives, all tasks currently executed by each host are determined to obtain a task set; Obtaining predicted power consumption of each task in the task set based on pre-collected historical data on CPU utilization and power consumption of various tasks; Iteratively optimizing the distribution of tasks in the task set among the hosts based on the predicted power consumption of each task in the task set, with the goal of reducing the total power consumption of the task scheduling system, to obtain an optimization result; The tasks in the task set are allocated according to the optimization result.

[0048] By the offline scheduler performing optimized scheduling according to the offline adjustment period, the rationality of task scheduling can be further improved.

[0049] This embodiment of the application also provides a task scheduling method based on dynamic power consumption awareness and load (CPU utilization) identification. Through accurate power consumption modeling and intelligent scheduling algorithms, it optimizes data center energy efficiency and efficiently schedules big data tasks. Specifically, the key technical contents of the method are as follows: In an embodiment of the present application, upon receiving a new task to be assigned (e.g., a first task), the task scheduling system will schedule the task among multiple hosts, determine a target host for executing the first task, and assign the first task to the target host. The host may specifically be a server. This task scheduling method determines the host to execute the task after receiving the task, and can therefore also be referred to as an online scheduling method. This method is primarily executed by the online scheduler of the task scheduling system, and of course, it also involves cooperation between other modules (e.g., a power consumption modeler).

[0050] like Figure 2 As shown, the method includes the following steps: Step 11: Receive the first task to be assigned.

[0051] Step 12: Obtain the CPU utilization of the first task, and determine the first CPU utilization of each host after accepting the first task based on the CPU utilization of the first task and the current CPU utilization of each host.

[0052] Here, the embodiment of the present application pre-calculates the CPU utilization of tasks of different specifications, and the CPU utilization is a statistical value. The CPU utilization of a task refers to the CPU utilization required when the task is executed by the host, that is, the CPU resources occupied. The specifications of the task can generally be represented by indicators such as process type and memory usage. Thus, for the first task in step 11, the CPU utilization of the first task can be determined based on the specifications of the first task.

[0053] In step 12, the embodiment of the present application can respectively calculate the CPU utilization of each host after the first task is assigned to each host. Specifically, the current CPU utilization of each host is added to the CPU utilization of the first task to obtain the first CPU utilization of each host after accepting the first task.

[0054] Step 13, calculate the first power consumption of each host after accepting the first task based on the first CPU utilization of each host after accepting the first task and the pre-obtained segmented quadratic model between the power consumption of the host and the CPU utilization of the host; select the host whose first power consumption is not greater than the preset power consumption threshold to obtain at least one candidate host.

[0055] Here, the embodiment of the present application pre-establishes a segmented quadratic model between the host's power consumption and the host's CPU utilization. The segmented quadratic model includes: when the host's CPU utilization is less than the power consumption saturation point, the host's power consumption is a quadratic function of the host's CPU utilization; when the host's CPU utilization is greater than or equal to the preset power consumption saturation point, the host's power consumption is a fixed value. Thus, in step 13, the embodiment of the present application can calculate the first power consumption of each host after accepting the first task according to the following formula: (Formula 1) in, The first power consumption of the host after accepting the first task, is the first CPU utilization of the host after accepting the first task, are respectively a predetermined first fitting coefficient, a second fitting coefficient, a third fitting coefficient and a fourth fitting coefficient, is a preset power consumption saturation point. To avoid excessive power consumption of the host after accepting the first task, the embodiment of the present application pre-sets a power consumption threshold. If the calculated first power consumption of the host after accepting the first task is not greater than the power consumption threshold, the host is considered a candidate host; if it exceeds the power consumption threshold, it is not considered a candidate host, and one or more candidate hosts can be selected. Taking into account some extreme cases, the first power consumption of all hosts may exceed the power consumption threshold, which means that the task volume has exceeded the upper limit of the system processing capacity. Usually, a judgment can be made before accepting the task to avoid the above situation.

[0056] Step 14: obtain the current power consumption of each candidate host and determine the current power consumption level among multiple power consumption levels. The multiple power consumption levels include multiple power consumption levels from small to large, and each power consumption level corresponds to a power consumption interval.

[0057] Here, the embodiment of the present application pre-divides multiple power consumption levels, and each power consumption level corresponds to a power consumption interval. Specifically, the maximum task power consumption can be determined based on the power consumption required by the host to perform various tasks; then, a power consumption range is determined, for example, the minimum value of the power consumption range is 0, and the maximum value is the power consumption threshold; then, the power consumption range is divided into at least one power consumption interval, specifically, with the maximum task power consumption as the division unit, starting from the power consumption of 0, the power consumption range is divided into at least one power consumption interval, until the remaining power consumption range is no greater than the maximum task power consumption, and the remaining power consumption range is taken as a power consumption interval, to obtain multiple power consumption intervals. For example, assuming that the maximum power consumption unit is P0, the power consumption threshold is Pm, and assuming that the result of rounding up Pm / P0 is n, a total of n power levels are obtained, from small to large: (0, P0], (P0, 2P0], ..., ((n-2)*P0, ((n-1)*P0], ((n-1)*P0, Pm]. In an embodiment of the present application, a level index can be established for each power consumption level, such as power consumption level 1, power consumption level 2, .... The power consumption level can be represented by the index of the power consumption level, and a larger power consumption level has a larger power consumption. A larger power consumption level means that the level index of the power consumption level is larger. That is, when the index of the first power consumption level is less than the index of the second power consumption level, the power consumption in the first power consumption interval corresponding to the first power consumption level is less than the power consumption in the second power consumption interval corresponding to the second power consumption level. For example, the power consumption in the power consumption interval corresponding to power consumption level 1 is less than the power consumption in the power consumption interval corresponding to power consumption level 2.

[0058] Step 15: Select, from the at least one candidate host, a candidate host whose current power consumption level is the highest and whose first power consumption is closest to the upper limit of the current power consumption range as the target host, wherein the current power consumption range is the power consumption range corresponding to the current power consumption level.

[0059] Here, when selecting a target host for the first task, the embodiment of the present application selects the candidate host whose current power consumption level is the highest and whose first power consumption is closest to the upper limit of the current power consumption level as the target host. If there is only one host with the highest current power consumption level, then that host will be selected as the target host; if there are multiple hosts with the highest current power consumption level, then the candidate host whose first power consumption is closest to the upper limit of the current power consumption range after accepting the first task can be selected as the target host. In the above manner, the first task can be preferentially assigned to a target host with a higher power consumption level, and the current power consumption level of the target host can be fully utilized.

[0060] Step 16: Allocate the first task to the target host.

[0061] Through the above steps, the embodiment of the present application can more accurately predict the host power consumption by pre-establishing a segmented quadratic model between the host's power consumption and the host's CPU utilization. By selecting the candidate host with the largest current power consumption level and the first power consumption closest to the upper limit of the current power consumption range as the target host for the task to be assigned, the rationality of task allocation can be improved, system energy consumption can be reduced, and system energy efficiency can be improved.

[0062] The following first describes how to establish a piecewise quadratic model between the power consumption of a host and the CPU utilization of the host.

[0063] The embodiment of the present application can pre-construct a segmented quadratic model between the power consumption of the host and the CPU utilization of the host. The segmented quadratic model includes: when the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host, and the quadratic function includes a first fitting coefficient, a second fitting coefficient, and a third fitting coefficient to be fitted; when the CPU utilization of the host is greater than or equal to the preset power consumption saturation point, the power consumption of the host is a fourth fitting coefficient to be fitted. The formula expression of the specific model can refer to the above formula 1, where are the first fitting coefficient, the second fitting coefficient, the third fitting coefficient and the fourth fitting coefficient to be fitted respectively.

[0064] Then, when the host runs different loads, the CPU utilization rate of the host and the power consumption of the host are collected, thereby obtaining a large amount of data on the CPU utilization rate of the host and the power consumption of the host.

[0065] Then, the segmented quadratic model is fitted using the collected CPU utilization and power consumption of the host, and the values ​​of the fitting coefficients are determined, thereby obtaining a segmented quadratic model between the power consumption and CPU utilization of the host.

[0066] In the embodiment of the present application, the above modeling process can be implemented by a host power consumption modeler. A specific modeling example is provided below using a server (ie, the host mentioned above) as an example.

[0067] This example accurately models the power consumption characteristics of servers under varying loads, providing fundamental data support for efficient task scheduling and energy efficiency optimization. This article details the design and implementation of a host power consumption modeler, including laboratory testing methods, validation on a large-scale experimental dataset, and performance evaluation of the final model.

[0068] A. Laboratory Test Methods To establish an accurate power consumption model, we first subjected the server to stress testing in a lab environment, collecting power consumption data under various load conditions. The experiment utilized a server of a certain brand, equipped with two 2.10GHz CPUs from a certain manufacturer and 128GB of RAM. The virtual machines in the experiment ran common big data applications, such as databases. The specific experimental process is as follows: 1. Setting up the experimental environment: Install the KVM virtualization platform on the test server and deploy a set of virtual machines (VMs). These VMs run different workloads, including compiling GNU gcc-11.2.0, processing Apache server requests, running MySQL benchmarks, calculating trigonometric functions, and executing the Linpack benchmark.

[0069] 2. Gradually increase the load: Start with 6 VMs and add one VM at a time until the number reaches 16. Each experiment runs for 600 seconds and then pauses for 300 seconds to prevent overheating.

[0070] 3. Data Collection: Real-time server power consumption data is collected using the SNMP protocol. Load data, such as CPU utilization, is collected using the Linux system's NET-SNMP-EXTEND-MIB module. Power consumption data is collected 20 times per second and then averaged.

[0071] By collecting data through experiments, we can get the relationship curve between host power consumption and CPU utilization rate as follows: Figure 2 shown.

[0072] Figure 3 The blue dots represent the corresponding information of CPU utilization and power consumption collected in the laboratory environment. The orange line represents the optimal fit of the point set using the linear model, the green line represents the optimal fit of the point set using the quadratic model, and the red line represents the fit of the point set using the piecewise quadratic model used in this example. Figure 2 The distribution of midpoints shows intuitively that the linear model has poor fitting accuracy. Using a simple quadratic model, power consumption decreases when CPU utilization is high, which is inconsistent with physical laws. However, a piecewise quadratic model fits the distribution of points well and stabilizes power consumption after the host CPU utilization reaches a certain threshold. For the mathematical expression of the piecewise quadratic model, refer to Formula 1 above.

[0073] B. Large-scale dataset verification To verify the effectiveness of the established model, a large-scale experimental data set was tested. The experimental data comes from a big data computing platform in an actual production environment and contains long-term operating data from multiple servers. The data collection cycle is every 5-6 minutes for 24 hours. During data processing, the power consumption data is smoothed using the simple moving average (SMA) algorithm to eliminate the impact of outliers. During model training and validation, the dataset is first divided into training and test sets according to the time series, and then the model performance is evaluated using cross-validation. The specific steps are as follows: 1. Dataset division: Divide the entire dataset into Each time, a part of the set is selected as the test set, and the rest is selected as the training set. Second-rate.

[0074] 2. Model training: In each cycle, the piecewise quadratic model is fitted using the training set data and the model parameters are adjusted.

[0075] 3. Model testing: Use the test set data to evaluate the model's predictive performance and calculate the mean absolute error (MAE) and mean absolute percentage error (MAPE). The specific calculation method is:

[0076]

[0077] in, and are the observed and predicted values, respectively. is the sample size.

[0078] Experimental results show that for mainstream models of big data services, the model's mean absolute error (MAE) is less than 10,000 on average, and the mean absolute percentage error (MAPE) is less than 3%.

[0079] Figure 2 The method shown can be executed by an online scheduler. As can be seen, in the embodiments of the present application, the online and offline schedulers optimize the resource utilization and energy efficiency of the data center through real-time and periodic task scheduling. Among them, the online scheduler adopts a "step-by-step" matching rule and combines a multi-objective optimization strategy to perform real-time task allocation to ensure efficient resource utilization and energy efficiency optimization in the short term; the offline scheduler performs statistical analysis based on historical data and uses a heuristic algorithm to globally reorganize tasks and reallocate resources, further improving the overall energy efficiency of the data center.

[0080] The following is a more detailed description of the specific functions of the online scheduler in combination with the above method.

[0081] The core concept of the online scheduler is to use a "step-by-step" matching rule and a multi-objective optimization strategy to allocate real-time tasks. It divides host power consumption into multiple levels, predicts each host's power consumption after accepting the first task based on the task's load (CPU utilization), and selects the host that maximizes the current power consumption (while still within a safe range) for task allocation. This not only fully utilizes host resources and reduces the number of active hosts, but also, through the appropriate level setting, leaves sufficient space for future large-scale tasks.

[0082] Specifically, the step-by-step allocation rule divides the power consumption range of the host into multiple power consumption levels. For example, the specifications of large data tasks are 1u, 2u, 4u, 8u, 16u, and 32u, and the maximum power consumption of the task is selected as a unit of a level, for example, 32u is a level. When a new task (such as the first task) arrives, the power consumption of each host after accepting the first task is predicted based on its CPU utilization. , select the host that can achieve the highest power consumption but still within a safe range to accept the first task. This allows the host resources to be fully utilized on the one hand, and on the other hand, it can leave enough space for future "large-scale tasks" to avoid the situation where large-scale tasks cannot be issued due to the generation of "host fragmentation".

[0083] Figure 4 An example diagram for allocating newly arrived tasks (such as the first task) is provided. Assume that there are 5 hosts in the host cluster, and each host has the same power consumption threshold line. Among them, some host resources cannot be used, otherwise there will be a risk of power consumption hotspots. The safe space below the power consumption threshold line of each host is divided into 3 power consumption levels, from small to large, namely level 1, level 2, and level 3. The currently used resources of each host are Figure 4 When a new task arrives, the host consumes the power after accepting the first task. This can be calculated, so the most suitable host needs to be matched. Host 4 (host-4) will reach a power consumption hotspot after adding the new task, so it is excluded. Host 5 (host-5) currently has the lowest resource utilization level and is considered a potential future energy-saving host, so it is also excluded. Of the remaining hosts 1, 2, and 4, the new task is assigned to host 2 (host-2), as it is closest to the upper limit of the current power consumption level.

[0084] In this embodiment of the present application, the task scheduling system can also identify high- and low-power hosts using the high- and low-power host identifiers, and schedule tasks for the high- and low-power hosts using the offline task scheduler. This process relies on accurate power consumption data provided by the host power modeler, combined with multi-dimensional power consumption indicators and a dynamic threshold adjustment mechanism to ensure efficient operation and energy efficiency optimization of the system under various load conditions.

[0085] Specifically, the task scheduling method further includes the following steps: Step a, obtain the current power consumption of each host and determine the high-power consumption host and / or the low-power consumption host, wherein the high-power consumption host is a host whose current power consumption is greater than a first threshold, and the low-power consumption host is a host whose current power consumption is less than or equal to a second threshold.

[0086] Here, the real-time power consumption data of each host can be collected through the power consumption modeler to monitor the power consumption status of each host.

[0087] Step b, repeatedly performing the following steps until the power consumption of the high-power-consumption host is no more than the first threshold: migrating tasks on the high-power-consumption host to a host with normal power consumption, wherein after accepting the tasks of the high-power-consumption host, the power consumption of the host with normal power consumption is no more than the first threshold; and / or, Repeat the following steps until all tasks of the low-power host are migrated, and shut down the low-power host: migrate tasks on the low-power host to a normal-power host, wherein after accepting the tasks of the low-power host, the power consumption of the normal-power host is not greater than the first threshold; The normal power consumption hosts are the remaining hosts except the high power consumption hosts and the low power consumption hosts.

[0088] Here, when migrating tasks between high-power and low-power hosts, the CPU utilization of the task to be migrated can be determined using a method similar to the above. Furthermore, based on the current CPU utilization of each normal-power host, the CPU utilization of the task to be migrated after it is migrated to the normal-power host is determined. Then, the power consumption of the normal-power host after migration is calculated using Formula 1. If the power consumption does not exceed the first threshold, the task can be migrated to the normal-power host. This will not be repeated here.

[0089] In the embodiment of the present application, the first threshold may be a pre-set high power consumption threshold, for example, 80% of the rated power consumption may be used as the threshold, and the second threshold may be a pre-set low power consumption threshold, for example, 20% of the rated power consumption may be used as the threshold. The above thresholds can be flexibly set based on experience and are not specifically limited in the embodiment of the present application.

[0090] As another implementation, the first threshold and the second threshold can also be dynamically adjusted based on historical power consumption data, thereby ensuring efficient operation and energy efficiency optimization of the system under various power consumption conditions. The following describes the identification process of high-power and low-power hosts.

[0091] The main function of the high-power host identifier is to identify hosts whose power consumption approaches or exceeds their capacity limit, preventing service quality degradation and triggering appropriate scheduling actions. When a host's power consumption exceeds a certain threshold, the CPU frequency will be reduced, processing power will be reduced, and this will lead to longer task queues, increased task delays, and SLA violations. The process of identifying high-power hosts can be divided into the following steps: 1. Data collection and preprocessing: The host power consumption modeler provides real-time power consumption data to monitor the current power consumption status of the host.

[0092] 2. Set a dynamic first threshold based on historical data and current system status The first threshold is not a fixed value, but is adjusted according to the SLA requirement level, the overall power consumption of the system and the historical power consumption change trend. The first threshold has an initial value, for example, it is initially set to 70% of the rated power consumption. It is subsequently dynamically adjusted according to the statistical period of the preset threshold adjustment. The statistical period has a preset length. For example, the first threshold of the current statistical period A dynamic adjustment formula is as follows:

[0093] in, is the average value of the first threshold in the historical statistical period, is the standard deviation of the power consumption of each high-power-consuming host in the historical statistical period. is the first adjustment coefficient. The value of the first adjustment coefficient is positively correlated with the SLA requirement of the host. The higher the SLA requirement, The larger the value of The smaller the value is, the lower the high load threshold line is.

[0094] 3. Identify high-power hosts: For each host, obtain its current power consumption ,like , the host is marked as a high power consumption host.

[0095] 4. Triggering Scheduling Operations: Once a high-power host is identified, the task scheduling system will reallocate resources or migrate tasks based on the scheduling policy to alleviate pressure on the high-power host and ensure stable system operation and service quality. For example, some tasks can be migrated to hosts with normal power consumption. Furthermore, embodiments of the present application can also reduce the task load on high-power hosts by dynamically adjusting resource allocation policies.

[0096] The main function of the low-power host identifier is to identify hosts with low power consumption and suitable for task consolidation, so that these hosts can be put into sleep mode or shut down to achieve energy saving goals. The process of identifying low-power hosts can be divided into the following steps: 1. Data collection and preprocessing: The host power consumption modeler provides real-time power consumption data to monitor the current power consumption status of the host.

[0097] 2. Set a dynamic second threshold based on historical data and current system status The second threshold is not a fixed value, but is adjusted according to the SLA requirement level, the overall power consumption of the system and the historical power consumption change trend. The second threshold has an initial value, for example, it is initially set to 20% of the rated power consumption. It is subsequently dynamically adjusted according to the statistical period. For example, the second threshold of the current statistical period is A dynamic adjustment formula is as follows: ; is the average value of the second threshold in the historical statistical period, is the standard deviation of the power consumption of each low-power host in the historical statistical period, is the second adjustment coefficient, which is positively correlated with the SLA requirement of the host. The higher the SLA requirement, The larger the value, the The larger the value, the higher the low load threshold line.

[0098] 3. Identify low power consumption: For each host, obtain its current power consumption ,like , the host is marked as a low-power host.

[0099] 4. Triggering Scheduling: Once low-power hosts are identified, the task scheduling system migrates tasks from these hosts to other hosts based on the scheduling policy. This allows the low-power hosts to be put to sleep or shut down, achieving energy savings. For example, tasks can be migrated from a low-power host to a normal-power host, consolidating resources and reducing energy consumption.

[0100] In an embodiment of the present application, the offline scheduler of the task scheduling system can optimize the resource utilization and energy efficiency of the data center through periodic task scheduling. The offline task scheduler performs statistical analysis based on historical data and uses heuristic algorithms to globally reorganize tasks and reallocate resources, further improving the overall energy efficiency of the data center. Through in-depth understanding of task load and power consumption characteristics, the offline scheduler adopts intelligent merging and migration strategies to maximize resource utilization while minimizing the number of active nodes.

[0101] Specifically, the embodiment of the present application can set an offline adjustment period, for example, set it to every 12 hours or every 24 hours as an offline adjustment period. The task scheduling method of the embodiment of the present application also optimizes the current task of the task scheduling system according to the preset offline adjustment period, wherein the optimization specifically includes the following steps: 1. When each offline adjustment cycle arrives, determine all tasks currently executed by each host and obtain the task set.

[0102] 2. According to the previously collected load characteristics (such as CPU utilization) and historical power consumption data of various tasks, the predicted power consumption of each task in the task set is obtained.

[0103] Here, the historical data of the load characteristics and power consumption of various tasks are collected in advance, so that a correspondence between the load characteristics and power consumption of various tasks can be established. In this way, the power consumption of each task in the task set can be predicted by using the correspondence and the CPU utilization of the task, thereby obtaining the predicted power consumption.

[0104] 3. Based on the predicted power consumption of each task in the task set and with the goal of reducing the total power consumption of the task scheduling system, iteratively optimize the distribution of the tasks in the task set among the hosts to obtain an optimization result.

[0105] During the offline scheduling process, metaheuristic algorithms can be used to globally reorganize tasks and reallocate resources. Metaheuristic algorithms use iterative optimization to gradually approach the optimal solution, improving resource utilization and energy efficiency. For example, the Adaptive Large Neighborhood Search (ALNS) algorithm dynamically selects different neighborhood operations (such as insertion, removal, and swapping) to improve the current solution. During the iteration process, ALNS adjusts the neighborhood selection strategy based on historical performance to improve search efficiency. The specific process includes: A. Initial solution generation: Generate an initial scheduling plan based on historical data and load forecast.

[0106] B. Neighborhood Operation Selection: In each iteration, an appropriate neighborhood operation is selected based on heuristic information and historical performance.

[0107] C. Solution update: Generate a new solution using the selected neighborhood operation and decide whether to accept the new solution based on certain criteria (such as energy efficiency improvement).

[0108] D. Strategy Adjustment: Dynamically adjust the selection probability of the neighborhood operations based on their success rates to adaptively optimize the quality of the solution.

[0109] In addition, other metaheuristic algorithms, such as genetic algorithms, particle swarm optimization, simulated annealing, etc., can also be used and selected according to specific needs.

[0110] 4. Allocate tasks in the task set according to the optimization result.

[0111] Through the above steps and mechanisms, the online scheduler and offline scheduler of the embodiment of the present application can optimize system performance and energy efficiency in the short and long term, respectively. This process relies on accurate data provided by the power consumption modeler and, through dynamic adjustment and feedback mechanisms, ensures efficient operation and energy efficiency optimization under various load conditions.

[0112] From the above description, it can be seen that the above methods of the embodiments of the present application solve one or more of the following problems: (1) Improving the accuracy of power consumption modeling: Through the innovative piecewise quadratic model, the embodiments of the present application can significantly improve the accuracy of power consumption prediction, and are expected to reduce the power consumption prediction error to within 3%, providing a reliable data basis for energy efficiency optimization.

[0113] (2) Enhanced SLA assurance: This embodiment of the application utilizes multi-dimensional load characteristic analysis and dynamic resource allocation strategies to maximize data center energy efficiency while ensuring that more than 98% of tasks meet SLA requirements. The system dynamically adjusts resource allocation based on task priority, resource requirements, and performance metrics to ensure that the performance of critical tasks is not affected while avoiding over-allocation of resources.

[0114] (3) Improving the offline coordination efficiency of the scheduling system: The embodiments of the present application construct a scheduling system with highly dynamic adaptability, which can shorten the response time to milliseconds in the event of sudden load changes. At the same time, through intelligent offline task reorganization strategies, the system will continuously optimize resource allocation in the long run. At the same time, through the coordination of online and offline scheduling, global optimal allocation of resources is achieved, and "real-time" optimization is achieved according to changes in environmental characteristics.

[0115] The present application also provides a task scheduling system. Figure 5As shown, the electronic device includes: a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the processor executes the program or instruction, the electronic device executes the task scheduling method provided in the embodiment of the present application and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0116] The present application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described task scheduling method embodiment and achieves the same technical effects. To avoid repetition, the details are not described here. The computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0117] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned task scheduling method embodiment are implemented and can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0118] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0119] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of this application.

[0120] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A task scheduling method, characterized in that: Executed by a task scheduling system, the method includes: receiving a first task to be assigned; Obtaining the CPU utilization of the first task, and determining a first CPU utilization of each host after accepting the first task based on the CPU utilization of the first task and the current CPU utilization of each host; Calculating, based on the first CPU utilization of each host after accepting the first task and a pre-obtained piecewise quadratic model between the power consumption of the host and the CPU utilization of the host, a first power consumption of each host after accepting the first task; selecting a host whose first power consumption is not greater than a preset power consumption threshold to obtain at least one candidate host; Obtaining the current power consumption of each candidate host and determining the current power consumption level among a plurality of power consumption levels, wherein the plurality of power consumption levels include a plurality of power consumption levels from small to large, each power consumption level corresponding to a power consumption interval; Selecting, from the at least one candidate host, a candidate host whose current power consumption level is the highest and whose first power consumption is closest to an upper limit of a current power consumption range as the target host, wherein the current power consumption range is a power consumption range corresponding to the current power consumption level; Allocate the first task to the target host.

2. The method according to claim 1, characterized in that The piecewise quadratic model between the power consumption of the host and the CPU utilization of the host includes: When the CPU utilization of the host is less than a preset power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host; When the CPU utilization of the host is greater than or equal to the power consumption saturation point, the power consumption of the host is a fixed value.

3. The method according to claim 2, characterized in that Calculating the first power consumption of each host after accepting the first task based on the first CPU utilization of each host after accepting the first task and a pre-obtained piecewise quadratic model between the power consumption of the host and the CPU utilization of the host includes: The first power consumption of each host after accepting the first task is calculated according to the following formula: in, The first power consumption of the host after accepting the first task, is the first CPU utilization of the host after accepting the first task, are respectively a predetermined first fitting coefficient, a second fitting coefficient, a third fitting coefficient and a fourth fitting coefficient, is the preset power consumption saturation point.

4. The method according to claim 3, characterized in that Before receiving the first task to be assigned, the method further includes: Constructing a piecewise quadratic model between the power consumption of the host and the CPU utilization of the host, the piecewise quadratic model including: when the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host, the quadratic function including a first fitting coefficient to be fitted, a second fitting coefficient, and a third fitting coefficient; when the CPU utilization of the host is greater than or equal to a preset power consumption saturation point, the power consumption of the host is a fourth fitting coefficient to be fitted; When the host runs different loads, collecting the CPU utilization rate and power consumption of the host; The segmented quadratic model is fitted using the collected CPU utilization and power consumption of the host, and the values ​​of each fitting coefficient are determined to obtain a segmented quadratic model between the power consumption of the host and the CPU utilization of the host.

5. The method according to claim 1, characterized in that Also includes: Determine the maximum task power consumption based on the power consumption required by the host to perform various tasks; Determine a power consumption range, where a minimum value of the power consumption range is 0 and a maximum value is the power consumption threshold; Taking the maximum task power consumption as the division unit, starting from the power consumption of 0, the power consumption range is divided into at least one power consumption interval until the remaining power consumption range is no greater than the maximum task power consumption, and the remaining power consumption range is taken as a power consumption interval to obtain multiple power consumption intervals, which correspond to the multiple power consumption gears respectively.

6. The method according to claim 1, characterized in that Also includes: Obtaining current power consumption of each host and determining a high-power consumption host and / or a low-power consumption host, wherein the current power consumption of the high-power consumption host is greater than a first threshold, and the current power consumption of the low-power consumption host is less than or equal to a second threshold; Repeating the following steps until the power consumption of the high-power-consumption host is no greater than the first threshold: migrating tasks on the high-power-consumption host to a host with normal power consumption, wherein after accommodating the tasks of the high-power-consumption host, the power consumption of the host with normal power consumption is no greater than the first threshold; and / or, Repeat the following steps until all tasks of the low-power host are migrated, and shut down the low-power host: migrate tasks on the low-power host to a normal-power host, wherein after accepting the tasks of the low-power host, the power consumption of the normal-power host is not greater than the first threshold; The normal power consumption hosts are the remaining hosts except the high power consumption hosts and the low power consumption hosts.

7. The method according to claim 6, characterized in that Also includes: The first threshold and the second threshold both have an initial value and are dynamically adjusted according to a preset threshold adjustment statistical period, wherein: The first threshold value of the current statistical period It is calculated according to the following formula: ; The second threshold of the current statistical period It is calculated according to the following formula: ; in, is the average value of the first threshold in the historical statistical period, is the standard deviation of the power consumption of each high-power-consuming host in the historical statistical period. is a first adjustment coefficient, the value of which is positively correlated with the SLA requirement of the host; is the average value of the second threshold in the historical statistical period, is the standard deviation of the power consumption of each low-power host in the historical statistical period, is a second adjustment coefficient, and the second adjustment coefficient is positively correlated with the SLA requirement of the host.

8. The method according to claim 6, characterized in that Also includes: Optimizing the current task of the task scheduling system according to a preset offline adjustment period, wherein the optimization specifically includes: When each offline adjustment period arrives, all tasks currently executed by each host are determined to obtain a task set; Obtaining predicted power consumption of each task in the task set based on pre-collected historical data on CPU utilization and power consumption of various tasks; According to the predicted power consumption of each task in the task set, with the goal of reducing the total power consumption of the task scheduling system, iteratively optimizing the distribution of the tasks in the task set among the hosts to obtain an optimization result; The tasks in the task set are allocated according to the optimization result.

9. A task scheduling system, characterized in that: include: An online scheduler, configured to receive a first task to be assigned; a power consumption modeler, configured to obtain the CPU utilization of the first task, and determine a first CPU utilization of each host after accepting the first task based on the CPU utilization of the first task and the current CPU utilization of each host; and calculate a first power consumption of each host after accepting the first task based on the first CPU utilization of each host after accepting the first task and a pre-obtained piecewise quadratic model between the power consumption of the host and the CPU utilization of the host; The online scheduler is further used to select a host whose first power consumption is not greater than a preset power consumption threshold to obtain at least one candidate host; obtain the current power consumption of each candidate host, and determine the current power consumption level to which the current power consumption belongs among multiple power consumption levels, wherein the multiple power consumption levels include multiple power consumption levels from small to large, and each power consumption level corresponds to a power consumption interval; from the at least one candidate host, select the candidate host whose current power consumption level is the largest and whose first power consumption is closest to the upper limit of the current power consumption interval as the target host, wherein the current power consumption interval is the power consumption interval corresponding to the current power consumption level; and assign the first task to the target host.

10. The task scheduling system according to claim 9, characterized in that: The piecewise quadratic model between the power consumption of the host and the CPU utilization of the host includes: When the CPU utilization of the host is less than a preset power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host; When the CPU utilization of the host is greater than or equal to the power consumption saturation point, the power consumption of the host is a fixed value.

11. The task scheduling system according to claim 10, characterized in that: The power consumption modeler is further configured to calculate a first power consumption of each host after accepting the first task according to the following formula: in, The first power consumption of the host after accepting the first task, is the first CPU utilization of the host after accepting the first task, are respectively a predetermined first fitting coefficient, a second fitting coefficient, a third fitting coefficient and a fourth fitting coefficient, is the preset power consumption saturation point.

12. The task scheduling system according to claim 11, characterized in that: The power consumption modeler is further configured to: Constructing a piecewise quadratic model between the power consumption of the host and the CPU utilization of the host, the piecewise quadratic model including: when the CPU utilization of the host is less than the power consumption saturation point, the power consumption of the host is a quadratic function of the CPU utilization of the host, the quadratic function including a first fitting coefficient to be fitted, a second fitting coefficient, and a third fitting coefficient; when the CPU utilization of the host is greater than or equal to a preset power consumption saturation point, the power consumption of the host is a fourth fitting coefficient to be fitted; When the host runs different loads, collecting the CPU utilization rate and power consumption of the host; The segmented quadratic model is fitted using the collected CPU utilization and power consumption of the host, and the values ​​of each fitting coefficient are determined to obtain a segmented quadratic model between the power consumption of the host and the CPU utilization of the host.

13. The task scheduling system according to claim 9, characterized in that: Also includes: Power consumption level divider for: Determine the maximum task power consumption based on the power consumption required by the host to perform various tasks; Determine a power consumption range, where a minimum value of the power consumption range is 0 and a maximum value is the power consumption threshold; Taking the maximum task power consumption as the division unit, starting from the power consumption of 0, the power consumption range is divided into at least one power consumption interval until the remaining power consumption range is no greater than the maximum task power consumption, and the remaining power consumption range is taken as a power consumption interval to obtain multiple power consumption intervals, which correspond to the multiple power consumption gears respectively.

14. The task scheduling system according to claim 9, characterized in that: Also includes: a high-power-consumption host identifier, configured to obtain current power consumption of each host and identify a high-power-consumption host, wherein the current power consumption of the high-power-consumption host is greater than a first threshold; a low-power host identifier, configured to obtain current power consumption of each host and determine a low-power host, wherein the current power consumption of the low-power host is less than or equal to a second threshold; The offline scheduler is configured to repeatedly perform the following processing until the power consumption of the high-power consumption host is no more than the first threshold: migrating tasks on the high-power consumption host to a host with normal power consumption, wherein after accepting the tasks of the high-power consumption host, the power consumption of the host with normal power consumption is no more than the first threshold; and / or repeatedly performing the following processing until all tasks of the low-power consumption host are migrated, and shutting down the low-power consumption host: migrating tasks on the low-power consumption host to a host with normal power consumption, wherein after accepting the tasks of the low-power consumption host, the power consumption of the host with normal power consumption is no more than the first threshold; The normal power consumption hosts are the remaining hosts except the high power consumption hosts and the low power consumption hosts.

15. The task scheduling system according to claim 14, characterized in that: The first threshold and the second threshold both have an initial value; the task scheduling system further includes: A threshold adjuster is used to dynamically adjust the first threshold and the second threshold according to a preset threshold adjustment statistical period, wherein: The first threshold value of the current statistical period It is calculated according to the following formula: ; The second threshold of the current statistical period It is calculated according to the following formula: ; in, is the average value of the first threshold in the historical statistical period, is the standard deviation of the power consumption of each high-power-consuming host in the historical statistical period. is a first adjustment coefficient, the value of which is positively correlated with the SLA requirement of the host; is the average value of the second threshold in the historical statistical period, is the standard deviation of the power consumption of each low-power host in the historical statistical period, is a second adjustment coefficient, and the second adjustment coefficient is positively correlated with the SLA requirement of the host.

16. The task scheduling system according to claim 14, characterized in that: The offline scheduler is further configured to optimize the current task of the task scheduling system according to a preset offline adjustment period, wherein the optimization specifically includes: When each offline adjustment period arrives, all tasks currently executed by each host are determined to obtain a task set; Obtaining predicted power consumption of each task in the task set based on pre-collected historical data on CPU utilization and power consumption of various tasks; Iteratively optimizing the distribution of tasks in the task set among the hosts based on the predicted power consumption of each task in the task set, with the goal of reducing the total power consumption of the task scheduling system, to obtain an optimization result; The tasks in the task set are allocated according to the optimization result.

17. A task scheduling system, characterized in that: include: a processor, a memory, and programs or instructions stored in the memory and executable on the processor; When the processor executes the program or instruction, the steps of the method according to any one of claims 1 to 8 are implemented.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

19. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images