Task scheduling method and device, electronic equipment and storage medium

Abstract

The invention provides a task scheduling method and device, electronic equipment and a storage medium, and belongs to the technical field of cloud computing. Compared with the prior art, the method has the advantages that the historical scheduling information is utilized, so that the to-be-processed task can be matched with the virtual machine type more accurately; resource waste in the task processing process can be reduced, and the task processing efficiency is improved. By monitoring the resource use condition of each cloud host and calculating the resource utilization rate based on the monitoring data, the load state of each host can be known in real time, and when the resource utilization rate of a source cloud host is within a relatively low first threshold range, a task and a virtual machine in which the task runs can be dynamically migrated to a target cloud host with a relatively high resource utilization rate; the migration strategy is beneficial to balancing the resource load of the cloud data center and improving the overall resource utilization rate; through virtual machine resource migration, the energy consumption of the whole cloud data center is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of cloud service technology, and in particular to a task scheduling method and device, electronic device and storage medium. Background Art

[0002] With the rapid development of cloud computing technology, cloud data centers (CDCs) have become key infrastructures supporting big data processing, high-performance computing, and various online services. CDCs integrate a large number of physical hosts, storage, network resources, and use virtualization technology to achieve unified management and on-demand allocation of resources, greatly improving the flexibility and scalability of computing resources. However, with the rapid increase in the scale and number of CDCs, their energy consumption and resource utilization efficiency issues have become increasingly prominent, becoming a bottleneck restricting the further development of cloud computing.

[0003] Currently, the main challenges facing CDCs are low resource utilization and excessive energy consumption. Since the utilization of computing resources is usually only between 10% and 50%, even if the host is idle, it will consume about 50% of additional energy, resulting in a large amount of energy waste. This not only increases operating costs, but also has a negative impact on the environment. Therefore, improving resource utilization and reducing energy consumption have become urgent issues for CDCs. Summary of the invention

[0004] The present disclosure provides a task scheduling method and device, electronic device and storage medium, which are mainly aimed at solving the problems of excessive energy consumption and low resource utilization in current cloud data centers.

[0005] According to a first aspect of the present disclosure, a method for task scheduling is provided, comprising:

[0006] Match the pending tasks uploaded by users with the virtual machine types according to historical scheduling information;

[0007] Invoking a target virtual machine of the virtual machine type to process the task to be processed;

[0008] Acquire monitoring data of each cloud host running a virtual machine, and calculate resource utilization of each cloud host based on the monitoring data;

[0009] If the resource utilization of the source cloud host processing the task to be processed is within a first threshold range, the target virtual machine is migrated to a target cloud host whose resource utilization is within a second threshold range.

[0010] In some embodiments, matching the pending tasks uploaded by the user with the virtual machine type according to the historical scheduling information includes:

[0011] Analyze the pending tasks uploaded by the user to determine the task type of the pending tasks;

[0012] Based on the historical scheduling information, determining whether there is a historical task type that is the same as the task type of the task to be processed, and determining a virtual machine type corresponding to the historical task type;

[0013] When it is determined that the historical task type exists, the pending tasks are merged into a task merging queue, and the pending tasks in the task merging queue are merged to obtain a task merging queue.

[0014] In some embodiments, the calling of a target virtual machine of the virtual machine type to perform task processing on the to-be-processed task includes:

[0015] Calculating the matching degree between each virtual machine in the virtual machine type corresponding to the historical task type and the task to be processed, and determining the virtual machine with the highest matching degree as the target virtual machine;

[0016] The target virtual machine is called to process the tasks in the task merging queue.

[0017] In some embodiments, the method further comprises:

[0018] When it is determined that the historical task type does not exist, a task type and a corresponding virtual machine type corresponding to the task to be processed are created, and the task to be processed is allocated to the newly created target virtual machine for task processing.

[0019] In some embodiments, before merging the pending tasks in the task merging queue, the method further includes:

[0020] According to the deadline of each task in the task merging queue, judging whether the task merging queue can be completed within the preset deadline, wherein the deadline of each task in the task merging queue is sorted in descending order;

[0021] If the task cannot be completed within the preset deadline, the computing resources of the target virtual machine for processing the task to be processed are expanded.

[0022] In some embodiments, obtaining monitoring data of each cloud host running a virtual machine and calculating resource utilization of each cloud host based on the monitoring data includes:

[0023] Based on the monitoring data, calculating the power of each cloud host and the total energy consumption of each cloud host;

[0024] The power-frequency ratio of each cloud host is calculated according to the task execution information of the task to be processed and the total energy consumption of each cloud host.

[0025] In some embodiments, if the resource utilization of the source cloud host processing the to-be-processed task is lower than a first threshold range, migrating the target virtual machine to a target cloud host having a resource utilization higher than a second threshold range includes:

[0026] Calculating a migration cost of the target virtual machine to the target cloud host;

[0027] Determine whether to migrate the target virtual machine to the target cloud host according to the migration cost and the host frequency of the source cloud host.

[0028] According to a second aspect of the present disclosure, a task scheduling device is provided, comprising:

[0029] A matching unit, used to match the pending tasks uploaded by the user with the virtual machine type according to the historical scheduling information;

[0030] A processing unit, used for calling a target virtual machine of the virtual machine type to process the task to be processed;

[0031] A computing unit, configured to obtain monitoring data of each cloud host running a virtual machine, and to calculate resource utilization of each cloud host based on the monitoring data;

[0032] The migration unit is used to migrate the target virtual machine to a target cloud host whose resource utilization is within a second threshold range when the resource utilization of the source cloud host processing the task to be processed is within a first threshold range.

[0033] In some embodiments, the matching unit comprises:

[0034] An analysis module, used to analyze the pending tasks uploaded by the user and determine the task type of the pending tasks;

[0035] A first determination module is used to determine whether there is a historical task type that is the same as the task type of the task to be processed based on the historical scheduling information, and determine the virtual machine type corresponding to the historical task type;

[0036] The merging module is used to merge the pending tasks into a task merging queue when it is determined that the historical task type exists, and to merge the pending tasks in the task merging queue to obtain a task merging queue.

[0037] In some embodiments, the processing unit includes:

[0038] A first calculation module, used to calculate the matching degree between each virtual machine in the virtual machine type corresponding to the historical task type and the task to be processed, and determine the virtual machine with the highest matching degree as the target virtual machine;

[0039] The processing module is used to call the target virtual machine to perform task processing on the tasks in the task merging queue.

[0040] In some embodiments, the apparatus further comprises:

[0041] The creation unit is used to create a task type and a corresponding virtual machine type corresponding to the task to be processed when it is determined that the historical task type does not exist, and allocate the task to be processed to the newly created target virtual machine for task processing.

[0042] In some embodiments, the apparatus further comprises:

[0043] A judging unit, used for judging whether the task merging queue can be completed within a preset deadline before merging the pending tasks in the task merging queue, according to the deadline of each task in the task merging queue, wherein the deadline of each task in the task merging queue is sorted in descending order;

[0044] The expansion unit is used to expand the computing resources of the target virtual machine for processing the task to be processed when the task cannot be completed within a preset deadline.

[0045] In some embodiments, the computing unit comprises:

[0046] A second calculation module, used to calculate the power of each cloud host and the total energy consumption of each cloud host based on the monitoring data;

[0047] The third calculation module is used to calculate the power-frequency ratio of each cloud host according to the task execution information of the task to be processed and the total energy consumption of each cloud host.

[0048] In some embodiments, the migration unit comprises:

[0049] A fourth calculation module, used to calculate the migration cost of migrating the target virtual machine to the target cloud host;

[0050] The second determination module is used to determine whether to migrate the target virtual machine to the target cloud host according to the migration cost and the host frequency of the source cloud host.

[0051] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0052] at least one processor; and

[0053] a memory communicatively connected to the at least one processor; wherein,

[0054] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0055] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0056] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0057] The present disclosure provides a method and apparatus, an electronic device and a storage medium for task scheduling, which match pending tasks uploaded by users with virtual machine types according to historical scheduling information; call a target virtual machine in the virtual machine type to process the pending tasks; obtain monitoring data of each cloud host running the virtual machine, and calculate the resource utilization of each cloud host based on the monitoring data; if the resource utilization of the source cloud host processing the pending tasks is within a first threshold range, migrate the target virtual machine to a target cloud host whose resource utilization is within a second threshold range. Compared with the related art, the embodiment of the present disclosure can more accurately match the task to be processed with the virtual machine type by utilizing historical scheduling information; it can reduce resource waste in the task processing process and improve the efficiency of task processing; by monitoring the resource usage of each cloud host and calculating the resource utilization based on the monitoring data, it can understand the load status of each host in real time, and when the resource utilization of the source cloud host is within a lower first threshold range, it can dynamically migrate the task and the virtual machine running it to the target cloud host with higher resource utilization; this migration strategy helps to balance the resource load of the cloud data center and improve the overall resource utilization; through the migration of virtual machine resources, the energy consumption of the entire cloud data center is reduced.

[0058] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0060] Figure 1A flowchart of a task scheduling method provided by an embodiment of the present disclosure;

[0061] Figure 2 A flowchart of another task scheduling method provided by an embodiment of the present disclosure;

[0062] Figure 3 A system structure diagram of a task scheduling system provided by an embodiment of the present disclosure;

[0063] Figure 4 A schematic diagram of the structure of a task scheduling device provided in an embodiment of the present disclosure;

[0064] Figure 5 A schematic diagram of the structure of another task scheduling device provided by an embodiment of the present disclosure;

[0065] Figure 6 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0066] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0067] CDCs (cloud data centers) not only face the challenge of excessive energy consumption, but also need to deal with the problem of low resource utilization. This problem leads to a lot of energy waste, because even if the host is completely idle, it will consume about 50% of additional energy. The utilization of computing resources is usually only between 10% and 50%. To meet this challenge, improving resource utilization becomes a key strategy. One potential approach is to save energy by reducing the number of running hosts, for example, by dynamically arranging virtual machines (VMs) on the hosts. However, the implementation of this strategy needs to take into account the actual situation in CDCs, that is, most applications are heterogeneous applications submitted dynamically by customers, and these applications have clear deadlines. Therefore, when implementing energy-saving strategies, it is necessary to ensure that there are a sufficient number of active hosts in the CDC at any time so that all incoming or upcoming tasks can be successfully executed before the deadline. Otherwise, the failure of task execution will not only further increase energy consumption but also reduce its service quality.

[0068] In order to ensure service quality while saving energy, CDC needs to adopt a comprehensive strategy that takes into account resource utilization issues and can flexibly respond to heterogeneous tasks and task deadline requirements dynamically submitted by users. The present disclosure provides a task scheduling scheme to achieve high energy efficiency and optimize CDC's task guarantee rate, average response time and resource utilization. Tasks are classified using a task classification method and then mapped to the most suitable virtual machine based on the requirements of the task and the capacity of the host. Task classification based on Bayesian classifiers and historical scheduling records allows the search for task types and virtual machine types. The task merging strategy is to merge tasks of similar types and plan the most suitable VM until resource conditions are met. Therefore, the scheduler can dynamically expand and integrate CDC's resources based on resource requirements.

[0069] The following describes the task scheduling method and apparatus, electronic device, and storage medium of the embodiments of the present disclosure with reference to the accompanying drawings.

[0070] Figure 1 A flowchart of a task scheduling method provided in an embodiment of the present disclosure.

[0071] like Figure 1 As shown, the method comprises the following steps:

[0072] Step 101 : Match the pending tasks uploaded by the user with the virtual machine type according to the historical scheduling information.

[0073] In the embodiments of the present disclosure, the historical scheduling information is a historical scheduling record (HSR) generated in the CDC based on previous scheduling information; that is, the HSR is a detailed record of task scheduling, resource allocation, and execution results generated by the cloud data center when executing tasks in the past. This information includes, but is not limited to, the arrival time, execution time, deadline, required resource type and quantity, virtual machine (VM) type of task execution, and the results of task execution. The main function of the historical scheduling record is to provide data support for current task scheduling decisions. By analyzing the historical scheduling records, the best matching relationship between different types of tasks and virtual machines can be obtained, as well as how to maximize resource utilization and energy efficiency while meeting task requirements.

[0074] Analyze the correspondence between task types and virtual machine types in historical scheduling records to find out which types of tasks are best suited for execution on which types of virtual machines. This helps to quickly select the appropriate virtual machine type based on the task type in subsequent task scheduling, thereby reducing resource waste and improving task execution efficiency. Evaluate the rationality of resource allocation in historical scheduling records. For example, for tasks with high resource requirements, whether enough resources are allocated to ensure that they can be completed on time; for tasks with low resource requirements, whether resources are over-allocated, resulting in resource waste. By analyzing these records, you can optimize resource allocation strategies and achieve efficient resource utilization.

[0075] Analyze the matching of task execution time and deadline in historical scheduling records. This helps to understand how the system handled task deadlines in the past and whether there were situations where tasks were not completed on time due to insufficient resources or other reasons. With this information, the task scheduling strategy can be adjusted to better meet the real-time needs of users. Use historical scheduling records to evaluate the energy efficiency and resource utilization of the system. By comparing the energy consumption and resource usage under different task scheduling strategies, the optimal scheduling strategy can be found to achieve higher energy efficiency and resource utilization.

[0076] Based on the task execution data and performance in the historical scheduling record (HSR), the system can intelligently match the pending tasks uploaded by users with the most appropriate virtual machine (VM) type. This matching process needs to fully consider the characteristics of the task, such as computationally intensive or memory intensive, as well as key factors such as the task deadline requirements and resource requirements. By using advanced algorithms such as Bayesian classifiers to conduct in-depth analysis and learning of historical scheduling information, the system can identify the optimal mapping relationship between different task types and virtual machine types, thereby ensuring that each task can be assigned to the virtual machine that can meet its resource requirements and has the highest energy efficiency. Such a matching strategy not only helps to minimize the average response time of tasks, but also significantly improves the resource utilization and energy efficiency of the cloud data center (CDC), providing users with more efficient and reliable cloud computing services.

[0077] Step 102: calling a target virtual machine of the virtual machine type to process the task to be processed.

[0078] In the embodiment of the present disclosure, in the process of calling the target virtual machine in the virtual machine type to process the task to be processed, the system will first select a most matching target virtual machine from the preset virtual machine types according to the specific requirements of the task to be processed, such as computing resources, memory requirements, network bandwidth, and disk storage. This selection process relies on the previously accumulated historical scheduling records (HSR), which contain detailed information in past task scheduling, such as task type, virtual machine configuration, resource utilization, and task execution results. Through in-depth analysis of historical scheduling information, the system can learn the best matching mode between different types of tasks and virtual machine configurations, so that when faced with new tasks to be processed, it can more accurately select the most suitable virtual machine type.

[0079] Once the target virtual machine is determined, the system will trigger the corresponding scheduling mechanism to assign the pending tasks to the target virtual machine of that type for execution. During the task processing, the system will continuously monitor the resource usage of the virtual machine and the execution progress of the task to ensure that the task can be completed efficiently within the specified deadline. At the same time, the system will also dynamically adjust the virtual machine according to the real-time situation, such as by migrating virtual machines, expanding or reducing computing resources, etc., to further optimize resource allocation and improve energy efficiency.

[0080] Step 103, obtaining monitoring data of each cloud host running a virtual machine, and calculating resource utilization of each cloud host based on the monitoring data.

[0081] In an embodiment of the present disclosure, in the process of obtaining the monitoring data of each cloud host running a virtual machine and calculating the resource utilization of each cloud host based on the monitoring data, key performance indicators will first be collected in real time through the resource monitor deployed on each cloud host, including but not limited to CPU usage, memory occupancy, network input and output rates, disk read and write speeds, etc. Subsequently, these data will be summarized for comprehensive analysis. Based on these detailed monitoring data, the system calculates the current resource utilization of each cloud host through an algorithm model. This calculation process takes into account the actual resource consumption of all virtual machines assigned to the cloud host and the tasks they carry. In this way, the resource utilization status of each cloud host can be accurately evaluated, providing strong data support for subsequent resource scheduling and optimization decisions.

[0082] Step 104: If the resource utilization of the source cloud host processing the task to be processed is within a first threshold range, the target virtual machine is migrated to a target cloud host whose resource utilization is within a second threshold range.

[0083] In an embodiment of the present disclosure, if the system detects that the resource utilization of the source cloud host processing the task to be processed is continuously within a preset first threshold range, this usually means that the source cloud host may have a low resource utilization rate, that is, its current load is light, but the resources are not fully utilized. At this time, in order to ensure the efficient execution of the task and the overall resource optimization of the cloud data center (CDC) and avoid energy waste, the system will automatically trigger the virtual machine (VM) migration mechanism. This mechanism aims to achieve dynamic balance of resources and maximize energy efficiency through intelligent scheduling. The mechanism will identify the target cloud host whose resource utilization is within a more reasonable second threshold range. This second threshold range is carefully set, representing that the cloud host is in a state that can both meet the task requirements and maintain a high resource utilization rate. Specifically, the target cloud host should have good processing power, sufficient remaining resources, and appropriate energy efficiency performance to ensure that the migrated virtual machine can obtain a better execution environment and higher quality of service (QoS).

[0084] After the target cloud host is determined, the system will automatically perform the virtual machine migration operation. During this process, the real-time migration technology will be fully utilized to ensure that the downtime during the migration process is close to zero, thereby minimizing the impact on users. After the migration is completed, the load of the source cloud host will be released, and the target cloud host will be able to take on more tasks, thereby achieving effective redistribution of resources and improving overall utilization.

[0085] In addition, the system will continuously monitor the resource utilization of all cloud hosts and dynamically adjust and optimize according to actual conditions to ensure that the cloud data center always remains efficient, energy-saving and stable. Through this intelligent virtual machine migration mechanism, not only can the resource utilization be improved, but also the energy consumption of the cloud data center can be optimized.

[0086] The present disclosure provides a method for task scheduling, which matches the pending tasks uploaded by the user with the virtual machine type according to historical scheduling information; calls the target virtual machine in the virtual machine type to process the pending tasks; obtains the monitoring data of each cloud host running the virtual machine, and calculates the resource utilization of each cloud host based on the monitoring data; if the resource utilization of the source cloud host processing the pending tasks is within the first threshold range, the target virtual machine is migrated to the target cloud host whose resource utilization is within the second threshold range. Compared with the related art, the embodiment of the present disclosure can more accurately match the pending tasks with the virtual machine type by utilizing the historical scheduling information; can reduce the waste of resources in the task processing process and improve the efficiency of task processing; by monitoring the resource usage of each cloud host and calculating the resource utilization based on the monitoring data, the load status of each host can be understood in real time, and when the resource utilization of the source cloud host is within the lower first threshold range, the task and the virtual machine running on it can be dynamically migrated to the target cloud host with higher resource utilization; this migration strategy helps to balance the resource load of the cloud data center and improve the overall resource utilization; through the migration of virtual machine resources, the energy consumption of the entire cloud data center can be reduced.

[0087] In order to clearly illustrate the embodiment of the present disclosure, this embodiment provides a flowchart of another task scheduling method.

[0088] like Figure 2 As shown, the method comprises the following steps:

[0089] Step 201: Analyze the pending tasks uploaded by the user to determine the task type of the pending tasks.

[0090] Step 202: Based on the historical scheduling information, determine whether there is a historical task type that is the same as the task type of the task to be processed, and determine the virtual machine type corresponding to the historical task type.

[0091] Specifically, in steps 201 to 202, Figure 3 It is a system structure diagram of a task scheduling system. After receiving the pending tasks uploaded by the user, a comprehensive analysis is performed on the pending tasks uploaded by the user. This step aims to deeply understand the characteristics and requirements of the tasks, so as to determine the specific task type of the pending tasks. This classification process is crucial for subsequent virtual machine allocation because it can help the system better understand the type and quantity of resources required for the task. Based on the rich historical scheduling information stored in advance, detailed retrieval and comparison are performed. The goal of this link is to determine whether there are historical task types that are exactly the same or highly similar to the task type of the current pending task. The historical scheduling information contains a large amount of task execution records and virtual machine usage data.

[0092] If it can find historical task types that match the task to be processed, it will further determine the virtual machine types corresponding to these historical task types. This step is based on the resource usage patterns and performance characteristics of the historical tasks during execution, and aims to find the most suitable virtual machine type for the current task to be processed to ensure efficient execution of tasks and optimal utilization of resources.

[0093] Step 203: when it is determined that the historical task type exists, the to-be-processed tasks are merged into a task merging queue, and the to-be-processed tasks in the task merging queue are merged to obtain a task merging queue.

[0094] Step 204, based on the deadline of each task in the task merge queue, determine whether the task merge queue can be completed within the preset deadline, wherein the deadline of each task in the task merge queue is sorted in descending order; if it cannot be completed within the preset deadline, expand the computing resources of the target virtual machine for processing the task to be processed.

[0095] Specifically, in steps 203 to 204, when it is determined that there is a historical task type that is the same as the task type of the task to be processed, the system will add the task to be processed to the task merging queue. This queue is specifically used to store similar tasks that can be merged for more efficient resource allocation and scheduling. Subsequently, the system will merge the tasks to be processed in the task merging queue. This step merges the tasks that can be merged by analyzing factors such as the resource requirements and deadlines of the tasks, thereby obtaining an optimized task merging queue. This process not only improves the efficiency of task processing, but also helps to maximize the use of virtual machine resources, reduce energy consumption, and improve the overall performance of the cloud data center.

[0096] According to the preset deadline of each task in the task merging queue, the system will judge one by one to determine whether all tasks in the task merging queue can be successfully completed within the preset deadline required by each task. In order to ensure the priority and efficiency of task processing, the tasks in the task merging queue will be sorted in descending order of their deadlines, that is, the tasks with the earliest deadlines will be processed first.

[0097] In the process of making judgments, the system will comprehensively consider factors such as currently available computing resources, network bandwidth, storage capacity, and the specific resource requirements of each task and the dependencies between them. If the system determines that under the current resource conditions, all tasks in the task merging queue cannot be completed within their respective preset deadlines, then the system will take corresponding measures to expand the computing resources of the target virtual machine that processes the pending tasks.

[0098] This expansion process may include but is not limited to increasing the number of CPU cores of the virtual machine, increasing memory capacity, expanding disk storage space, etc., to ensure that the task can be processed in a timely manner within the specified deadline. Through such dynamic resource adjustment and task scheduling strategies, the system can flexibly respond to various task requirements, maximize resource utilization and task execution efficiency, and ensure the overall performance and stability of the cloud data center.

[0099] Specifically, define two task sequences of the same type and The same VM type. Where k represents the task number, m = {m1, m2, ...m n} represents the queue length from one task to another of the same type, which is not necessarily equal. A sequence of the same task type can be represented as

[0100] The system saves all tasks generated by the task observer in a merge queue. The task observer classifies tasks according to the HSR and then maps them to the most suitable VM. The merge queue is generated by the task observer with task type and VM type. The merge queue is sorted in descending order by deadline.

[0101] make is the execution time of a task series of the same type of tasks. and Merge, then merge the execution time of the tasks k ' is given as Because the task Must be on the deadline Before the task is completed, the deadline k ' will become For example, most applications in cloud environments are dynamically submitted by end users and assigned deadlines to ensure their timeliness requirements in many fields, including scientific visualization, real-time signal processing, etc. Assume is a series of tasks of task type 1 in a merge queue (MQ), and are VM series of the same type. Because MQ is sorted in descending order of deadline, the deadline Very close. First, if the task Can be scheduled to a virtual machine And there is a deadline It checks for the same task type. if Can be Before completing its execution, it checks for another successor of the same type if Can be arranged in This process will continue until the same task type is completed. Could not be completed before It will expand resources and re-plan. Unable to If the task is completed within the specified time, resources will be expanded and it will be checked whether scheduling the virtual machine is feasible. If so, the scheduling will be executed, otherwise the task will be rejected.

[0102] Step 205 , calculating the matching degree between each virtual machine in the virtual machine type corresponding to the historical task type and the task to be processed, and determining the virtual machine with the highest matching degree as the target virtual machine.

[0103] Specifically in step 205, by traversing the historical task type database, for each historical task type that is similar or related to the task to be processed, the corresponding virtual machine type information is extracted. This information includes but is not limited to the configuration of the virtual machine (such as CPU, memory, network bandwidth and disk storage, etc.), historical performance data and energy consumption performance in past task execution. For each extracted virtual machine type, a matching algorithm is used to calculate the matching degree between the virtual machine and the task to be processed. This matching algorithm will be based on multiple factors, including but not limited to whether the resource configuration of the virtual machine can meet the requirements of the task to be processed, the efficiency and energy consumption ratio of the virtual machine when performing similar tasks in the past, and the similarity between the virtual machine and the task to be processed in terms of task type, resource requirements, etc.

[0104] In the process of calculating the matching degree, a Bayesian classifier is used to classify tasks according to the historical scheduling record (HSR). The best VM type is determined to map with the most suitable VM, and the historical data set is obtained from the HSR to classify the tasks and generate the VM type. Let H Τ is the historical task set extracted from HSR in CDC.

[0105] make Represents the historical task type count, then the ratio It can be expressed as:

[0106]

[0107] Based on the resource model and task model, the matching degree can be expressed as:

[0108] here

[0109] Assumptions

[0110]

[0111] According to the above formula, we can get the result: 0.98>0.64>0.1. Note that if is the most perfect match, so Compare and With a better match, by using the Bayesian filter, we can calculate Τ k belong The probability is:

[0112]

[0113] Then k The decision function is:

[0114]

[0115] Step 206: calling the target virtual machine to process the tasks in the task merging queue.

[0116] Specifically, in step 206, after determining the target virtual machine with the highest matching degree, the system will call the target virtual machine to process the tasks in the task merging queue one by one. This process involves allocating and executing the tasks in the task merging queue according to the established order and the processing capacity of the target virtual machine.

[0117] Specifically, the system will first sort the tasks in the task merging queue according to priority, deadline or other relevant factors, and then assign these tasks to the target virtual machine for processing. After receiving the task, the target virtual machine will execute the corresponding task processing logic according to its configuration and performance characteristics, including computing, storage, network transmission and other operations. During the processing, the system will monitor the execution progress and resource usage of the task in real time to ensure that the task can be completed on time and maximize resource utilization efficiency.

[0118] At the same time, in order to further improve processing efficiency and resource utilization, the system may also adopt some optimization strategies, such as task merging, virtual machine migration, etc. These strategies can be dynamically adjusted according to actual conditions and task requirements to achieve the best processing effect.

[0119] Step 207: When it is determined that the historical task type does not exist, create a task type and a corresponding virtual machine type corresponding to the task to be processed, and assign the task to be processed to the newly created target virtual machine for task processing.

[0120] Specifically, in step 207, during the in-depth analysis and processing of the task, if the system determines that there is no historical task type that matches the task to be processed, this means that the corresponding historical scheduling record cannot be found in the historical scheduling information. In this case, the system will create a completely new task type based on the characteristics and requirements of the task to be processed. This new task type will accurately reflect the unique attributes and requirements of the task to be processed. Subsequently, in order to support the execution of this new task type, the system will further create a corresponding virtual machine type. This virtual machine type will be specially designed and optimized to ensure that all tasks of the new task type can be processed efficiently and accurately.

[0121] After the new task type and virtual machine type are created, the system will assign the pending tasks to the newly created target virtual machine for task processing. This process will ensure that the pending tasks can be executed in the virtual machine environment that best suits their characteristics and requirements, thereby maximizing the efficiency and accuracy of task processing.

[0122] Step 208: Calculate the power of each cloud host and the total energy consumption of each cloud host based on the monitoring data.

[0123] Step 209: Calculate the power-frequency ratio of each cloud host according to the task execution information of the task to be processed and the total energy consumption of each cloud host.

[0124] Specifically, in steps 208 to 209, based on the collected comprehensive monitoring data, the real-time power of each cloud host in the cloud data center can be accurately calculated. This calculation process covers various energy-consuming components involved in the operation of the cloud host, including but not limited to CPU, memory, disk storage, and network equipment. Furthermore, in order to evaluate the overall energy efficiency of the cloud host, it is necessary to calculate the total energy consumption of each cloud host. The energy efficiency of the cloud host is deeply analyzed by combining the task execution information of the task to be processed. The task execution information includes but is not limited to the arrival time, execution time, required resources (such as CPU, memory, etc.) and priority of the task. By combining this information with the total energy consumption of the cloud host, a more comprehensive understanding of the energy consumption performance of the cloud host when processing different tasks can be obtained.

[0125] On this basis, the power-frequency ratio of each cloud host is further calculated, which is an important indicator that reflects the balance between the energy efficiency and performance of the cloud host. The power-frequency ratio is usually defined as the ratio of the power consumption of the cloud host when processing tasks to its CPU frequency. By calculating the power-frequency ratio, the energy efficiency of the cloud host at a given performance level can be intuitively evaluated, thus providing strong data support for optimizing the energy utilization of cloud data centers.

[0126] Specifically, CDC consists of an infinite host set H = {H1, H2, ..., Hi}, providing a physical infrastructure for creating virtualized resources to meet user needs. A running host collection is Hi is characterized by H i =((f i ,v i ),c i ,m i ), where (f i ,vi)={(f i 1 ,vi 1 ),…,(f i max ,θi max )} is the host H i Discrete pair of frequency and voltage, c i Represents host H i CPU power, m i Indicates host H i The memory capacity used for computing. The virtual machine VM running on the host can be modeled as here Indicates the type of virtual machine VM, and It represents the VM demand characteristics and is modeled as here and Represents the parameters of the virtual machine's processor, memory, network bandwidth, and disk storage.

[0127] A task is a medium or container that expresses or holds user requirements. As can be seen, there are two main types of tasks. The first is a computational task that requires high CPU and low memory, and the second is a task that requires low CPU and high memory. Here, we define a set of tasks as Τ = {Τ1, Τ2, Τ3, …, Τ k} which includes a set of parameters here as well as They represent the required computing power, main memory, network bandwidth, and disk storage space for the kth given processing task. Similarly, H Τ , Represent historical task records, task types, and type counts, respectively. In a CDC environment, the service provider obtains independent real-time tasks submitted by end users within a random time range, and then the system accepts the service without knowing the complexity of the computing infrastructure. Independent real-time tasks can be modeled as here They represent the task arrival time, task workload (i.e., total computing units), and task deadline, respectively. The cloud provider specifies the task parameters when the user submits a task to it.

[0128] Generally speaking, the following components consume energy, including the host, electrical and network components, cooling systems, and storage devices. The main source of CDC energy consumption is the host, and the host's energy consumption is mainly determined by the CPU, RAM, and disk storage. i The energy consumed can be divided into static power (P static ) and dynamic power (P dymamic ), where an idle host consumes about 70% of the energy consumed by a host running at full CPU speed. The total consumed power is the sum of static and dynamic power as follows:

[0129] P total =P static +P dynamic

[0130] This paper mainly studies the dynamic power P of the CPU dynamic , which is proportional to the square of the power supply voltage and frequency f i d is proportional to. Here α i is a proportionality coefficient, H i P dynamic It can be expressed as follows:

[0131]

[0132] Let γ i Is an idle host The power ratio, f i max Host H i The maximum frequency and P i max Host H i Maximum power consumption. H i The proportionality coefficient can be expressed as:

[0133]

[0134] The dynamic power can be obtained from the above two formulas:

[0135]

[0136] Therefore, the host H i The power consumption can be expressed as:

[0137]

[0138] here, Indicates the running status of the host at time t. If it is 1, it means the host is in running state, and if it is 0, it means the host is in shutdown state. iFrom start time To end time The total energy consumption is It can be approximated as follows:

[0139]

[0140] Therefore, the total energy consumption of the physical host can be expressed as follows:

[0141]

[0142] Assume ξ is a task Τ k On host H i Use frequency f i d The computational size, then, the execution time It can be expressed as ξ and f i d Ratio, i.e. Therefore H i The total energy can be expressed as follows:

[0143]

[0144] After optimization:

[0145]

[0146] Based on the above equation, H i The optimized frequency can be calculated as:

[0147]

[0148] Host H i Optimized frequency (f i o ) is expressed as a discrete frequency set as: The maximum power P i max Formal definition.

[0149]

[0150] f i o =|f i d |,if f i 1 ≤f i d ≤f i max ,

[0151] Here | f id | indicates that more than f i 1 frequency and close to f i max .

[0152] f i 0 =f i max ,if f i d ≥f i max

[0153] Power-to-frequency ratio It can be calculated as follows:

[0154]

[0155] The above expression shows that the host The larger the value, the higher the energy efficiency, in which case the system will schedule the execution of real-time heterogeneous tasks when the workload is reduced. This, in turn, will lead to shutting down the Lower host.

[0156] Furthermore, the present disclosure also provides an optimization method, which specifically considers four optimization objectives to minimize the total energy consumption and average response time, and maximize the task guarantee ratio and utilization rate. and For the task k On the host H i On the virtual machine V ij The start and end time of For the task k The execution time of task Τ k In the virtual machine V ijk Completion time It can be expressed as:

[0157]

[0158] Parameter X kij Used to indicate k In H i Virtual Machine V ij If task Τ k Mapped to virtual machine V ij Up, X kij The value is 1, otherwise, the value is 0;

[0159]

[0160] According to the above analysis, the optimization problem can be expressed as:

[0161] Maximum mission assurance rate:

[0162] Minimum total energy consumption:

[0163] Minimum response time: (waiting time + execution time + migration time)

[0164] Maximum resource utilization:

[0165] constraint:

[0166] (a)

[0167] (b)

[0168] (c)

[0169] (d)

[0170] (e)

[0171] Step 210: Calculate the migration cost of the target virtual machine to the target cloud host.

[0172] Step 211: Determine whether to migrate the target virtual machine to the target cloud host according to the migration cost and the host frequency of the source cloud host.

[0173] Specifically in steps 210 to 211, migration is widely used in data centers to optimize host-level resource utilization and implement cloud resource management. There are basically two types of migration, offline migration: it refers to moving a suspended VM from one host, and live migration: it refers to moving a running VM from one host to another.

[0174] In this disclosure, only live migration is considered for two main reasons: First, it is more efficient from a performance perspective because it is able to transfer virtual machines between hosts with close to zero downtime. Second, it is the most adopted type of VM migration in modern VM managers. The length of a live migration depends on the total CPU size, memory size, disk storage used by the VM, and the available network bandwidth. During a live migration, the CPU state context of the virtual machine will be switched from the source host to the target host. Therefore, there will be a small amount of data to be transferred and it represents the lowest limit to minimize the migration time. Similarly, the memory state of the VM also needs to be transferred to the target host. This information may be larger than the CPU state, and includes the memory state of the operating system and all running processes in the VM.

[0175] The virtual machine migration time is a very important factor affecting energy consumption. Therefore, the time must be considered before making any migration-related decisions. The total migration time depends on the number of VMs to be migrated and the size of the VMs. However, the total number of virtual machines to be migrated and the size of each virtual machine cannot be known in advance except in a probabilistic sense. The time cost of migrating a single VM can be expressed by the statistics of the CPU content size, RAM size and storage size of this VM and the available bandwidth at the time of migration. It is important to emphasize that the RAM size is the main parameter affecting the VM migration time. The migration time is given in the equation in the following formula.

[0176] The virtual machine migration time can be expressed as:

[0177]

[0178] here It is a virtual machine VMV j The bandwidth of the migration between the source and destination hosts, measured in bytes per second.

[0179] The total VM migration time can be calculated as:

[0180]

[0181] Estimating total VM migration time The values ​​of are statistically independent for all VMs, where V Mig Indicates the number of migrated virtual machines.

[0182] The energy consumed by virtual machine migration can be calculated as follows:

[0183]

[0184] Among them, P Mig The unit power consumption of the migrated VM.

[0185] Cost E is the energy consumption per unit, cost P Indicates a fixed penalty value for SLA violation. For the target host The proportion of k tasks served, λ i Is the target host The average request rate.

[0186] Therefore, the migration cost can be expressed as follows:

[0187]

[0188] As an achievable method of the present disclosure, when determining whether to perform virtual machine migration, the following method may be used but is not limited to: i th is the host’s threshold frequency. If the host’s resource utilization is very low (the host’s frequency (f i d ) is less than or equal to the threshold frequency (f i th ) and is greater than the optimal frequency f i o , that is, f i o <f i d <f i th ), and then a migration mechanism can be used to migrate running virtual machines from hosts with lower utilization to hosts with higher utilization (the frequency of the host (f i d ) is greater than or equal to the threshold frequency (f i th ) and is less than or equal to the maximum frequency (f i max ), that is, f i th <f i d <f i max ).

[0189] As a supplementary explanation of the present disclosure, this embodiment further provides an implementation method of the present disclosure from the perspective of an algorithm, which is as follows:

[0190] Task scheduling and resource management are important means to optimize CDC energy consumption to ensure task time requirements. Scheduling strategies are used to dynamically schedule real-time tasks. The scheduling process handles various interruptions during system operation, including the arrival of new tasks or tasks that have just been completed. In this case, the workload of CDC can become large and will lead to vertical expansion situations that integrate available physical and virtual resources according to task requirements.

[0191] The main method of energy-saving dynamic scheduling EDS is proposed in Algorithm 2, which has four methods. MARKVM is used to calculate various tasks and virtual machine types. Similarly, TASKMERGE is used to merge tasks of the same type. When the system dynamically scales up and down computing resources, SCALEUPRESOURCES and CONSOLIDATION methods are called multiple times.

[0192] When a user submits a task k, the energy-efficient dynamic scheduling scheme (EDS) collects the information in Algorithm 2, including the arrival time and deadline Then, these tasks are added to the task queue of the entire system in descending order of deadline, and the MARKVM method is called (Algorithm 1). Algorithm 1 is based on historical scheduling records HSR and Bayesian classifier. It mainly consists of two parts. The first part is to classify the upcoming tasks, and the second part is to mark the virtual machines. In Algorithm 1, the information of the tasks is collected from the historical planning records, and then the number of task types is calculated. The task observer checks whether the task type of a given task is available. If the task type is found, it will mark the task type and VM type, otherwise it will mark Τ k As new and And update HSR.

[0193] In Algorithm 2, the virtual machine is based on the power frequency ratio To sort, if All tasks The type will be sorted as Merge Queue (MQ). After this step, the method TASKMERGING will be called.

[0194] In the task merging method, in Algorithm 3, tasks of similar types can be merged and scheduled to VMs of the same VM type. Merge as many tasks of the same type as possible and assign them to VMs that can meet the resource requirements. If the execution time of a task is and Can be on the deadline To meet resource requirements, two tasks and The tasks are merged and scheduled to the VM. Then, the new execution time and deadline of the tasks are merged. will become Otherwise, resources are scaled up proportionally and execution continues with the next steps.

[0195] When scaling up resources in Algorithm 4, all active hosts are sorted according to If the active host can satisfy the task's requirements, Type of virtual machine V ij Will create and update HSR. Otherwise, Descending (Power off host list) (No running host list) Start the new host, then the new Type of virtual machine V ijA schedule record HSR will be created and updated.

[0196] If the arrival rate of real-time tasks on the terminal decreases, the workload of CDC will naturally decrease. Higher than t Idle , the integration method will be applied to minimize the number of running virtual machines and hosts, as shown in Algorithm 5: If V ij on V ij is deleted (if V ij of becomes idle, will be shut down (. In addition, the virtual machine can be removed from the host with lower utilization (f i d ≥f i o And f i d ≤f i th ) to the host with higher utilization (f i d >f i th And f i d ≤f i max ) and the idle host H i Idle (f i d <f i o ) will be turned off to reduce the power consumption of the host in CDC.

[0197] Example: Suppose the user assigns their task k Submitted to the cloud, these tasks have different deadlines and heterogeneity. The scheduling scheme arranges these tasks into a task queue in descending order of deadlines. The task set in the task queue is Τ = {Τ1, Τ2, Τ3, ..., Τ k}. The task observer checks similar information in historical scheduling records and virtual machine data. For example, task arrival rate, deadline, execution time, virtual machine information (CPU, memory, network bandwidth, disk storage) and resource requirements (CPU, memory, network bandwidth, disk storage). As mentioned above, task classification is performed based on the Bayesian classifier. If historical scheduling records are found, it is decided to sort these tasks in the merged queue by task type. If the task observer cannot find any historical scheduling information that matches the task, it is decided to create a new virtual machine, mark it with a new task type and virtual machine type, and update the HSR. Assume that the task type of task Τ1 is 1 and virtual machine type 1, the task type of task Τ2 is 3 and virtual machine type 3, the task type of task Τ3 is 1 and virtual machine type 1, the task type of task Τ4 is 1 and virtual machine type 1, and so on. Then, the scheduler selects (Task Τ1 of type 1) merge them.

[0198] The scheduler follows Sort all virtual machines of type 1 in descending order. Then, check V1 1 Whether the resource can be accommodated and scheduled within the deadline If V1 1 Can accommodate The scheduler will see the next task of the same task type 1. It will also check V1 again 1 Can the two tasks be accommodated? If both tasks can be accommodated, merge the two tasks and merge them. and Merged tasks The deadline becomes The deadline of V1 (i.e. the shortest deadline of the two). 1 If both tasks cannot be accommodated at the same time, the scheduler will expand resources and check V1 again. 1 Can both tasks be accommodated at the same time? If so, follow and The deadline rules are merged into V1 1 Otherwise, only the task is scheduled. To V1 1 to be processed.

[0199] It should be noted that the embodiments of the present disclosure may include multiple steps. For the convenience of description, these steps are numbered, but these numbers do not limit the execution time slots or execution order between the steps; these steps can be implemented in any order, and the embodiments of the present disclosure do not limit this.

[0200] Corresponding to the above-mentioned task scheduling method, the present invention also provides a task scheduling device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0201] Figure 4 A schematic diagram of a task scheduling device provided by an embodiment of the present disclosure is shown in FIG. Figure 4 As shown, including:

[0202] A matching unit 31 is used to match the to-be-processed tasks uploaded by the user with the virtual machine type according to the historical scheduling information;

[0203] A processing unit 32, configured to call a target virtual machine of the virtual machine type to process the task to be processed;

[0204] The calculation unit 33 is used to obtain monitoring data of each cloud host running a virtual machine, and calculate the resource utilization rate of each cloud host based on the monitoring data;

[0205] The migration unit 34 is configured to migrate the target virtual machine to a target cloud host whose resource utilization is within a second threshold range when the resource utilization of the source cloud host processing the task to be processed is within a first threshold range.

[0206] The present disclosure provides a task scheduling device, which matches the pending tasks uploaded by the user with the virtual machine type according to the historical scheduling information; calls the target virtual machine in the virtual machine type to process the pending tasks; obtains the monitoring data of each cloud host running the virtual machine, and calculates the resource utilization of each cloud host based on the monitoring data; if the resource utilization of the source cloud host processing the pending tasks is within the first threshold range, the target virtual machine is migrated to the target cloud host whose resource utilization is within the second threshold range. Compared with the related art, the embodiment of the present disclosure can more accurately match the pending tasks with the virtual machine type by utilizing the historical scheduling information; can reduce the resource waste in the task processing process and improve the efficiency of task processing; by monitoring the resource usage of each cloud host and calculating the resource utilization based on the monitoring data, the load status of each host can be understood in real time, and when the resource utilization of the source cloud host is within the lower first threshold range, the task and the virtual machine running on it can be dynamically migrated to the target cloud host with higher resource utilization; this migration strategy helps to balance the resource load of the cloud data center and improve the overall resource utilization; through the migration of virtual machine resources, the energy consumption of the entire cloud data center can be reduced.

[0207] Furthermore, in a possible implementation of this embodiment, as Figure 5As shown, the matching unit 31 includes:

[0208] The analysis module 311 is used to analyze the pending tasks uploaded by the user and determine the task type of the pending tasks;

[0209] A first determination module 312 is used to determine whether there is a historical task type that is the same as the task type of the task to be processed based on the historical scheduling information, and determine the virtual machine type corresponding to the historical task type;

[0210] The merging module 313 is used to merge the pending tasks into a task merging queue when it is determined that the historical task type exists, and to merge the pending tasks in the task merging queue to obtain a task merging queue.

[0211] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the processing unit 32 includes:

[0212] A first calculation module 321 is used to calculate the matching degree between each virtual machine in the virtual machine type corresponding to the historical task type and the task to be processed, and determine the virtual machine with the highest matching degree as the target virtual machine;

[0213] The processing module 322 is used to call the target virtual machine to perform task processing on the tasks in the task merging queue.

[0214] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the device also includes:

[0215] The creation unit 35 is used to create a task type and a corresponding virtual machine type corresponding to the task to be processed when it is determined that the historical task type does not exist, and allocate the task to be processed to the newly created target virtual machine for task processing.

[0216] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the device also includes:

[0217] A judging unit 36 ​​is used to judge whether the task merge queue can be completed within a preset deadline before merging the pending tasks in the task merge queue, according to the deadline of each task in the task merge queue, wherein the deadline of each task in the task merge queue is sorted in descending order;

[0218] The expansion unit 37 is used to expand the computing resources of the target virtual machine for processing the task to be processed when the task cannot be completed within the preset deadline.

[0219] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the calculation unit 33 includes:

[0220] A second calculation module 331 is used to calculate the power of each cloud host and the total energy consumption of each cloud host based on the monitoring data;

[0221] The third calculation module 332 is used to calculate the power-frequency ratio of each cloud host according to the task execution information of the task to be processed and the total energy consumption of each cloud host.

[0222] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the migration unit 34 includes:

[0223] The fourth calculation module 341 is used to calculate the migration cost of migrating the target virtual machine to the target cloud host;

[0224] The second determination module 342 is used to determine whether to migrate the target virtual machine to the target cloud host according to the migration cost and the host frequency of the source cloud host.

[0225] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.

[0226] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0227] Figure 6 A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0228] like Figure 6As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 402 or a computer program loaded from a storage unit 408 to a RAM (Random Access Memory) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O (Input / Output) interface 405 is also connected to the bus 404.

[0229] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0230] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as a method for task scheduling. For example, in some embodiments, the method for task scheduling may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute the aforementioned task scheduling method in any other appropriate manner (for example, by means of firmware).

[0231] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0232] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0233] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0234] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0235] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0236] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0237] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0238] The various numerical numbers such as first and second involved in the present disclosure are only for the convenience of description and are not used to limit the scope of the embodiments of the present disclosure, but also indicate the order of precedence.

[0239] At least one in the present disclosure may also be described as one or more, and a plurality may be two, three, four or more, which is not limited in the present disclosure. In the embodiments of the present disclosure, for a technical feature, the technical features in the technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D", etc., and there is no order of precedence or size between the technical features described by the "first", "second", "third", "A", "B", "C" and "D".

[0240] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0241] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A task scheduling method, characterized in that: include: Match the pending tasks uploaded by users with the virtual machine types according to historical scheduling information; Invoking a target virtual machine of the virtual machine type to process the task to be processed; Acquire monitoring data of each cloud host running a virtual machine, and calculate resource utilization of each cloud host based on the monitoring data; If the resource utilization of the source cloud host processing the task to be processed is within a first threshold range, the target virtual machine is migrated to a target cloud host whose resource utilization is within a second threshold range.

2. The method according to claim 1, characterized in that The matching of the pending tasks uploaded by the user with the virtual machine types according to the historical scheduling information includes: Analyze the pending tasks uploaded by the user to determine the task type of the pending tasks; Based on the historical scheduling information, determining whether there is a historical task type that is the same as the task type of the task to be processed, and determining a virtual machine type corresponding to the historical task type; When it is determined that the historical task type exists, the pending tasks are merged into a task merging queue, and the pending tasks in the task merging queue are merged to obtain a task merging queue.

3. The method according to claim 1, characterized in that The calling of a target virtual machine of the virtual machine type to process the task to be processed includes: Calculating the matching degree between each virtual machine in the virtual machine type corresponding to the historical task type and the task to be processed, and determining the virtual machine with the highest matching degree as the target virtual machine; The target virtual machine is called to process the tasks in the task merging queue.

4. The method according to claim 2, characterized in that: The method further comprises: When it is determined that the historical task type does not exist, a task type and a corresponding virtual machine type corresponding to the task to be processed are created, and the task to be processed is allocated to the newly created target virtual machine for task processing.

5. The method according to claim 2, characterized in that: Before merging the pending tasks in the task merging queue, the method further includes: According to the deadline of each task in the task merging queue, judging whether the task merging queue can be completed within the preset deadline, wherein the deadline of each task in the task merging queue is sorted in descending order; If the task cannot be completed within the preset deadline, the computing resources of the target virtual machine for processing the task to be processed are expanded.

6. The method according to claim 1, characterized in that The acquiring monitoring data of each cloud host running the virtual machine and calculating the resource utilization rate of each cloud host based on the monitoring data includes: Based on the monitoring data, calculating the power of each cloud host and the total energy consumption of each cloud host; The power-frequency ratio of each cloud host is calculated according to the task execution information of the task to be processed and the total energy consumption of each cloud host.

7. The method according to claim 6, characterized in that If the resource utilization rate of the source cloud host processing the task to be processed is lower than the first threshold range, migrating the target virtual machine to a target cloud host having a resource utilization rate higher than the second threshold range includes: Calculating a migration cost of the target virtual machine to the target cloud host; Determine whether to migrate the target virtual machine to the target cloud host according to the migration cost and the host frequency of the source cloud host.

8. A task scheduling device, characterized in that: include: A matching unit, used to match the pending tasks uploaded by the user with the virtual machine type according to the historical scheduling information; A processing unit, used for calling a target virtual machine of the virtual machine type to process the task to be processed; A computing unit, configured to obtain monitoring data of each cloud host running a virtual machine, and to calculate resource utilization of each cloud host based on the monitoring data; The migration unit is used to migrate the target virtual machine to a target cloud host whose resource utilization is within a second threshold range when the resource utilization of the source cloud host processing the task to be processed is within a first threshold range.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

11. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images