Cloud computing task scheduling method and device, equipment, medium and program product

Abstract

The invention provides a cloud computing task scheduling method and device, equipment, a medium and a program product, and the method comprises the steps: carrying out the load rating of computing resources in a cloud resource pool, obtaining the load grade of each computing resource, and constructing a resource load pyramid model according to the load grade of each computing resource, the computing resources with low load levels in the resource load pyramid model are under the computing resources with high load levels; obtaining a plurality of tasks received by the cloud computing platform, performing priority division on the plurality of tasks, and arranging the plurality of tasks in a descending order according to a priority order to obtain a task priority queue; and carrying out resource allocation according to the load level of the computing resources in the resource load pyramid model and the priority sequence in the task priority queue. According to the method, the characteristics of task priority, resource load capacity and the cloud resource pool are integrated, and effective management and efficient utilization of cloud computing resources are realized through construction and dynamic adjustment of the pyramid model.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of cloud computing technology, and in particular to a cloud computing task scheduling method, device, equipment, medium and program product. Background Art

[0002] Cloud computing is a new computing model that provides efficient and convenient computing services for enterprises and individuals with its elastic scalability and pay-as-you-go features. However, the advantages of cloud computing cannot be realized without efficient task scheduling technology. Task scheduling is a key link in the management and utilization of cloud computing resources, and it directly affects the performance and stability of cloud computing systems.

[0003] However, with the widespread application and popularization of cloud computing, the management and scheduling of its core cloud resource pool are facing more and more challenges. First, the computing resources in the cloud resource pool are of various types and have different performances. How to effectively manage and utilize these resources to meet the needs of different tasks is an urgent problem to be solved. Secondly, the amount of tasks in the cloud computing environment is huge and changes dynamically, and the task types are also diverse. How to schedule resources according to the characteristics of different tasks to ensure the efficient execution of tasks is also an important research direction in the field of cloud computing.

[0004] Regarding the task scheduling problem in cloud computing, although the existing task scheduling methods have their own advantages, there are still many shortcomings, including: priority-based task scheduling methods, although they can ensure that key tasks are processed in a timely manner, may lead to unbalanced resource utilization and unfairness between tasks; load balancing technology, although it can balance the load of computing nodes, it is difficult to adapt to the dynamic changes in task volume and differences in node performance in the cloud computing environment; prediction technology, although it can predict future resource requirements and task load conditions based on historical data and current environmental information, the accuracy of the prediction is affected by many factors, and it is difficult to ensure the optimality of scheduling decisions. Summary of the invention

[0005] The embodiments of the present invention provide a cloud computing task scheduling method, apparatus, device, medium and program product to solve the existing problem of being unable to effectively manage and efficiently utilize cloud computing resources.

[0006] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0007] In a first aspect, an embodiment of the present invention provides a cloud computing task scheduling method, comprising:

[0008] Performing load rating on computing resources in the cloud resource pool to obtain a load level of each computing resource, and constructing a resource load pyramid model according to the load level of each computing resource, wherein computing resources with a low load level are located below computing resources with a high load level in the resource load pyramid model;

[0009] Acquire multiple tasks received by the cloud computing platform, prioritize the multiple tasks, and arrange the multiple tasks in descending order of priority to obtain a task priority queue;

[0010] Resource allocation is performed according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

[0011] Optionally, the load rating of the computing resources in the cloud resource pool to obtain different load levels includes:

[0012] Partitioning and integrating the scattered computing resources in the cloud resource pool to obtain a computing resource set;

[0013] Obtain static indicator parameters and dynamic indicator parameters of the computing resources in the computing resource set; wherein the static indicator parameters include at least one of the following: CPU utilization, storage capacity, network bandwidth, memory occupancy, disk I / O; the dynamic indicator parameters include at least one of the following: resource request volume, resource service capability, resource service intensity;

[0014] The load capacity value of the computing resource is calculated based on the static indicator parameter and the dynamic indicator parameter, and the load level of the computing resource is obtained according to the load capacity value.

[0015] Optionally, constructing a resource load pyramid model according to the load level includes:

[0016] Determining the initial number of layers of the resource load pyramid model according to the load levels of the computing resources in the computing resource set;

[0017] Inputting the load level and the initial number of layers into a pre-trained clustering model, the clustering model outputting a computing resource set of each layer according to the distribution of load capacity values ​​and business requirements;

[0018] Determine whether the load level in the computing resource set of each level meets the preset load balancing threshold, and if the load level in the computing resource set of each level meets the preset load balancing threshold, directly obtain the final level division;

[0019] If the load level in the computing resource set of each level does not meet the preset load balancing threshold, the computing resource set of each level is readjusted to obtain the final level division;

[0020] The resource load pyramid model is obtained according to the final level division.

[0021] Optionally, the acquiring of tasks received by the cloud computing platform and prioritizing the tasks includes:

[0022] Performing demand analysis on the tasks received by the cloud computing platform, and prioritizing the tasks according to task priority consideration parameters, wherein the task priority consideration parameters include at least one of the following: task type, task importance, system load, and resource demand;

[0023] Arrange the tasks in descending order of priority to obtain a task priority queue.

[0024] Optionally, the step of acquiring tasks received by the cloud computing platform, prioritizing the tasks, and arranging the tasks in descending order of priority to obtain a task priority queue may further include:

[0025] A high priority ratio of tasks is set for the task priority queue, and the task priority queue is divided into a low priority team and a high priority team based on the high priority ratio of tasks, wherein the difference between the proportion of the high priority team in the task priority team and the proportion of the high load level in the resource load pyramid model satisfies a preset ratio threshold.

[0026] Optionally, allocating resources according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue includes:

[0027] A top-down load level reduction task resource allocation strategy is adopted for the high priority team in the task priority queue;

[0028] A bottom-up load-level incremental task resource allocation strategy is adopted for the low-priority queue in the task priority queue.

[0029] Optionally, also include:

[0030] Monitoring and recording the task execution effect based on the resource load pyramid model, and forming historical data according to the recorded task execution effect;

[0031] The historical data is analyzed to optimize the process of allocating resources according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

[0032] In a second aspect, an embodiment of the present invention provides a cloud computing task scheduling device, including:

[0033] A first processing module is used to load-rate the computing resources in the cloud resource pool to obtain the load level of each computing resource, and to construct a resource load pyramid model according to the load level of each computing resource, wherein computing resources with low load levels are located below computing resources with high load levels in the resource load pyramid model;

[0034] The second processing module is used to obtain multiple tasks received by the cloud computing platform, prioritize the multiple tasks, and arrange the multiple tasks in descending order of priority to obtain a task priority queue;

[0035] The resource allocation module is used to allocate resources according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

[0036] Optionally, the first processing module includes:

[0037] An integration submodule, used to partition and integrate the scattered computing resources in the cloud resource pool to obtain a computing resource set;

[0038] An acquisition submodule is used to acquire static indicator parameters and dynamic indicator parameters of the computing resources in the computing resource set; wherein the static indicator parameters include at least one of the following: CPU utilization, storage capacity, network bandwidth, memory occupancy, disk I / O; the dynamic indicator parameters include at least one of the following: resource request quantity, resource service capability, resource service intensity;

[0039] The first calculation submodule is used to calculate the load capacity value of the computing resource based on the static indicator parameter and the dynamic indicator parameter, and obtain the load level of the computing resource according to the load capacity value.

[0040] Optionally, the first processing module includes:

[0041] A second calculation submodule, used to determine the initial number of layers of the resource load pyramid model according to the load level of the computing resources in the computing resource set;

[0042] A first processing submodule, configured to input the load level and the initial number of layers into a pre-trained clustering model, wherein the clustering model outputs a computing resource set of each layer according to the distribution of load capacity values ​​and business requirements;

[0043] The second processing submodule is used to determine whether the load level in the computing resource set of each level meets the preset load balancing threshold. If the load level in the computing resource set of each level meets the preset load balancing threshold, the final level division is directly obtained; if the load level in the computing resource set of each level does not meet the preset load balancing threshold, the computing resource set of each level is readjusted to obtain the final level division; the resource load pyramid model is obtained according to the final level division.

[0044] Optionally, the second processing module includes:

[0045] A third processing submodule is used to perform demand analysis processing on the tasks received by the cloud computing platform, and prioritize the tasks according to task priority consideration parameters, wherein the task priority consideration parameters include at least one of the following: task type, task importance, system load, and resource demand;

[0046] The fourth processing submodule is used to arrange the tasks in descending order of priority to obtain a task priority queue.

[0047] Optionally, also include:

[0048] The optimization module is used to set a high-priority task ratio for the task priority queue, and divide the task priority queue into a low-priority team and a high-priority team based on the high-priority task ratio, wherein the difference between the proportion of the high-priority team in the task priority queue and the proportion of the high load level in the resource load pyramid model meets a preset proportion threshold.

[0049] Optionally, the resource allocation module includes:

[0050] A first resource allocation submodule, configured to adopt a top-down load level reduction task resource allocation strategy for a high priority queue in the task priority queue;

[0051] The second resource allocation submodule is used to adopt a bottom-up load level increasing task resource allocation strategy for the low priority queue in the task priority queue.

[0052] Optionally, also include:

[0053] A monitoring module, used to monitor and record the task execution effect based on the resource load pyramid model, and form historical data according to the recorded task execution effect;

[0054] The analysis module is used to analyze the historical data and optimize the resource allocation process according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

[0055] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps in the cloud computing task scheduling method described in any one of the first aspects.

[0056] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps in the cloud computing task scheduling method as described in any one of the first aspects are implemented.

[0057] In a fifth aspect, an embodiment of the present invention provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps in the cloud computing task scheduling method as described in any one of the first aspects.

[0058] In the present invention, the computing resources in the cloud resource pool are load rated to obtain the load level of each computing resource, and a resource load pyramid model is constructed according to the load level of each computing resource, wherein computing resources with low load levels are located below computing resources with high load levels in the resource load pyramid model; multiple tasks received by the cloud computing platform are obtained, and the multiple tasks are prioritized, and the multiple tasks are arranged in descending order according to the priority order to obtain a task priority queue; resources are allocated according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue. Taking into account the characteristics of task priority, resource load capacity and cloud resource pool, the effective management and efficient utilization of cloud computing resources are realized through the construction and dynamic adjustment of the pyramid model, and the existing problem of being unable to effectively manage and efficiently utilize cloud computing resources is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0060] Figure 1 is a flow chart of a cloud computing task scheduling method provided by an embodiment of the present invention;

[0061] Figure 2 It is a schematic diagram of a resource load pyramid model of a cloud computing task scheduling method provided by an embodiment of the present invention;

[0062] Figure 3It is a schematic diagram of a hierarchical determination code of a cloud computing task scheduling method provided by an embodiment of the present invention;

[0063] Figure 4 It is a clustering code schematic diagram of a cloud computing task scheduling method provided by an embodiment of the present invention;

[0064] Figure 5 It is a schematic diagram of resource allocation of a cloud computing task scheduling method provided by an embodiment of the present invention;

[0065] Figure 6 is a flow chart of another cloud computing task scheduling method provided by an embodiment of the present invention;

[0066] Figure 7 It is a structural diagram of a cloud computing task scheduling device provided by an embodiment of the present invention;

[0067] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] Please refer to Figure 1 The embodiment of the present invention provides a cloud computing task scheduling method, which is applied to a cloud computing platform, and includes:

[0070] Step 11: Load rating the computing resources in the cloud resource pool to obtain the load level of each computing resource, and construct a resource load pyramid model according to the load level of each computing resource, wherein computing resources with low load levels are located below computing resources with high load levels in the resource load pyramid model;

[0071] In the embodiment of the present invention, the computing resources in the cloud resource pool are firstly rated for load, and the resources are divided into different load levels; and a resource load pyramid model is introduced, such as Figure 2 As shown, a resource load pyramid model is established according to the load level, wherein computing resources with weak load capacity are located at the bottom of the pyramid, and computing resources with stronger load capacity are located at higher levels, that is, computing resources with low load levels are located below computing resources with high load levels in the resource load pyramid model;

[0072] In the embodiment of the present invention, optionally, the load rating of the computing resources in the cloud resource pool to obtain different load levels includes:

[0073] Partitioning and integrating the scattered computing resources in the cloud resource pool to obtain a computing resource set;

[0074] Obtain static indicator parameters and dynamic indicator parameters of the computing resources in the computing resource set; wherein the static indicator parameters include at least one of the following: CPU utilization, storage capacity, network bandwidth, memory occupancy, disk I / O; the dynamic indicator parameters include at least one of the following: resource request volume, resource service capability, resource service intensity;

[0075] The load capacity value of the computing resource is calculated based on the static indicator parameter and the dynamic indicator parameter, and the load level of the computing resource is obtained according to the load capacity value.

[0076] In the embodiment of the present invention, a method combining static evaluation and dynamic evaluation is used to evaluate resource load, thereby further improving the efficiency and scientificity of cloud computing task scheduling;

[0077] Specifically, the scattered computing resources in the cloud resource pool are partitioned and integrated, wherein the computing resources after partition integration are some computing resources that are not suitable for further splitting and division physically or logically, so as to avoid the possibility of failures caused by executing different tasks. 1 ,U 2 ....U m}; Any computing resource set U j (1≤j≤m) contains n computing resources, that is,

[0078] The resource request amount r j , which is the average number of services received by the resource per unit time. The resource request amount is Then U j The resource request amount is:

[0079]

[0080] The resource service capability h j , which is the average number of service requests completed by resources per unit time; where h j The larger the value, the stronger the resource service capability, and the higher the resource price, that is, the resource price is proportional to h. The resource service capacity is Then U j The resource service capabilities are:

[0081]

[0082] The resource service intensity q j , which is the ratio of the average time to complete a service request to the average time interval between service requests, and:

[0083]

[0084] Among them, C j For the parallel service capability of resources, if the resource node The parallel service capacity is Resource U j Parallel service resource capabilities The resource service intensity reflects the relative relationship between the resource load pressure and its available capacity. When the resource service intensity approaches 1, it indicates that the resource working pressure is approaching full load. Then U j The resource intensity is:

[0085]

[0086] Based on the above dynamic indicator parameters, the computing resource load capacity value is obtained. Given a computing resource set U j There are p static evaluation performance indicators, and their resource request amount Resource service capabilities Calculate resource service intensity Then the computing resource set U j Load capacity value L j for:

[0087]

[0088] Among them, δ represents the weight of the resource intensity of the computing resource set; p kj Represents the kth static evaluation performance indicator of the computing resource set, ω kj is the weight corresponding to the static evaluation performance indicator.

[0089] In the embodiment of the present invention, optionally, constructing a resource load pyramid model according to the load level includes:

[0090] Determining the initial number of layers of the resource load pyramid model according to the load levels of the computing resources in the computing resource set;

[0091] Inputting the load level and the initial number of layers into a pre-trained clustering model, the clustering model outputting a computing resource set of each layer according to the distribution of load capacity values ​​and business requirements;

[0092] Determine whether the load level in the computing resource set of each level meets the preset load balancing threshold, and if the load level in the computing resource set of each level meets the preset load balancing threshold, directly obtain the final level division;

[0093] If the load level in the computing resource set of each level does not meet the preset load balancing threshold, the computing resource set of each level is readjusted to obtain the final level division;

[0094] The resource load pyramid model is obtained according to the final level division.

[0095] In the embodiment of the present invention, specifically, the collected computing resource set U j , and its corresponding load capacity value L j (1≤j≤m), and remove abnormal or obsolete data;

[0096] Determine the initial number of pyramid layers M. Specifically, there are two methods to choose from:

[0097] 1. Directly set based on business needs and management granularity. For example, if the business needs are fine management, more levels may be required; on the contrary, if the management is relatively extensive, the number of levels can be relatively small;

[0098] 2. Taking the K-means clustering algorithm as an example, clustering effect evaluation indicators such as Silhouette Score or Davies-Bouldin Index (DBI) are used to determine the optimal number of levels.

[0099] In the embodiment of the present invention, taking the silhouette coefficient as the evaluation index to determine the optimal number of levels as an example, the pseudo code is as follows: Figure 3 As shown, according to the distribution of load capacity values ​​and business requirements, the preprocessed load capacity value and the optimal level are used as input, and a suitable clustering algorithm is used to output the computing resource set of each level. Taking the K-means clustering algorithm as an example, the resources are divided into M clusters so that the resources in each cluster have similar load capacity. The pseudo code is as follows Figure 4 As shown; then check whether the resource capabilities within each level are balanced. If not, readjust the level boundaries;

[0100] Specifically, the standard deviation of the load capacity of the computing resource set in each layer is calculated. If the standard deviation is too large, consider merging the two layers with the smallest standard deviation or splitting the layer with the largest standard deviation. The calculation formula is as follows:

[0101]

[0102] Where σ is the standard deviation of the computing resource set in this level; N is the number of computing resource sets in this level; L i is the load capacity value of each computing resource set in the level; finally, according to the final level division, the structural information of the pyramid model is output, including the final number of levels M * , the computing resource set of each layer and other data.

[0103] In an embodiment of the present invention, a resource load pyramid model is proposed to be introduced into the cloud resource pool to rate and stratify the load of computing resources. Resources are divided into different levels according to factors such as performance and availability, so that resources can be managed and scheduled more finely, ensuring that high-performance resources are used to perform critical tasks, while lower-performance resources are used to perform non-critical tasks, thereby maximizing resource utilization and avoiding resource waste.

[0104] Step 12: Acquire multiple tasks received by the cloud computing platform, prioritize the multiple tasks, and arrange the multiple tasks in descending order of priority to obtain a task priority queue;

[0105] In the embodiment of the present invention, optionally, the acquiring of tasks received by the cloud computing platform and prioritizing the tasks includes:

[0106] Performing demand analysis on the tasks received by the cloud computing platform, and prioritizing the tasks according to task priority consideration parameters, wherein the task priority consideration parameters include at least one of the following: task type, task importance, system load, and resource demand;

[0107] Arrange the tasks in descending order of priority to obtain a task priority queue.

[0108] In the embodiment of the present invention, it is first necessary to clarify the considerations of task priority, among which, task type (TaskTyep, Tt): batch processing, interactive, real-time tasks, etc., and assign different weights, among which real-time tasks need to be set with higher priority; task importance (Task Importance, Ti), based on business needs, user feedback or the impact of tasks on the overall performance of the system to evaluate, high-importance tasks should be given higher priority; system load (System Load, Sl) is based on key indicators such as CPU utilization, memory utilization, network bandwidth utilization; resource requirements (ResourceRequirement, Rr) evaluate the CPU cycles, memory size, storage space, etc. required for the task. The weighted sum method is used to calculate the priority of the task, that is, each factor has a weight (Weight, W), and after normalization (Normalization, N), it is compared on the same scale, then the formula of task priority (Task Priority, TP) is:

[0109] TP=W Tt ×N Tt +W Ti ×N Ti +W Sl ×N Sl +W Rr ×N Rr ;

[0110] Where: W Tt , W Ti , W Sl and W Rr are the weights of task type, task importance, system load and resource requirements; N Tt 、N Ti 、N Sl and N Rr It is the normalized value of the corresponding factor, usually mapped to the range of [0,1]. Specifically, appropriate factors can be added or subtracted according to the actual application situation to add to the consideration of task priority, and the weight of each factor can be adjusted according to the actual application scenario. By establishing a priority queue for the cloud computing tasks to be executed and dividing them according to actual needs, fair scheduling of tasks is achieved, avoiding the situation where high-priority tasks occupy resources for a long time and low-priority tasks cannot be executed. By reasonably setting the queue size and execution strategy, the resource requirements between tasks of different priorities are balanced, and the overall performance of the cloud resource pool is improved.

[0111] In the embodiment of the present invention, optionally, the step of acquiring tasks received by the cloud computing platform, prioritizing the tasks, and arranging the tasks in descending order of priority to obtain a task priority queue further includes:

[0112] A high priority ratio of tasks is set for the task priority queue, and the task priority queue is divided into a low priority team and a high priority team based on the high priority ratio of tasks, wherein the difference between the proportion of the high priority team in the task priority team and the proportion of the high load level in the resource load pyramid model satisfies a preset ratio threshold.

[0113] In the embodiment of the present invention, based on the task priority queue, a reasonable high priority ratio T is set to divide the queue into a low priority queue and a high priority queue.

[0114] Specifically, according to the current task priority queue and the hierarchical information of the resource load pyramid, it is assumed that the first k layers (k <M * ) are resources with strong load capacity, then the proportion P of these resource sets in the overall computing resource set can be expressed as:

[0115]

[0116] Among them, n i is the number of computing resources in the i-th layer; a reasonable high priority ratio T can be approximately equal to P, that is, T = P, so that resources with strong load capacity can handle more high priority tasks.

[0117] If the original number of task priority queues is G, which is the number of priority queues, accumulate them from the front to the back and generate new queues until the number of new queues is G. 1 Meet G 1 =G×T, then the new queue generated is the high priority queue;

[0118] The remaining tasks to be executed in the original task priority queue are kept in their original order and a low priority queue is generated, the number of which is G 2 Meet G 2 =GG 1 .

[0119] In the embodiment of the present invention, the task queue is divided into a low-priority queue and a high-priority queue, and the divided high-priority queue and the low-priority queue are executed simultaneously, which can ensure that high-priority tasks are processed first and low-priority tasks are given the opportunity to be executed, thereby achieving fair scheduling of tasks and avoiding the situation where high-priority tasks occupy resources for a long time and low-priority tasks cannot be executed. In subsequent resource allocation, a top-down reduction-type task resource allocation strategy is adopted for the high-priority team, and a bottom-up incremental task resource allocation strategy is adopted for the low-priority team, thereby improving the execution efficiency of tasks. By reasonably setting the queue size and execution strategy, the resource requirements between tasks of different priorities can be balanced, thereby improving the overall performance of the cloud resource pool.

[0120] Step 13: Allocate resources according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

[0121] In the embodiment of the present invention, please refer to Figure 5 , in determining the high-quality team G 1 When allocating task resources, we start from the highest level of the resource load pyramid, and then 1 The tasks in the high priority team calculate resources layer by layer from top to bottom. When initially allocating resources, the tasks in the high priority team are given priority to resources with stronger load capacity to ensure the execution rate of their tasks and to start execution quickly. Then, a reduction-type dynamic adjustment is performed. That is, if it is found that the utilization rate of a computing resource is continuously too low, that is, the resource is idle for a long time or is not fully utilized, it can be considered to reduce and release the computing resource, and when the next task similar to the past is executed, the next layer of computing resources with slightly weaker load capacity is allocated to the task. After determining the low priority team G 2 When allocating task resources, we start from the lowest level of the pyramid, which is G 2 The tasks in the task list are allocated computing resources layer by layer from bottom to top. During the initial allocation, the low-priority team is given priority to use resources with weaker load capacity to reserve more high-performance resources for the high-priority team. Then, incremental dynamic adjustment is performed. That is, if the execution speed of a task is found to be too low, more computing resources of the same layer can be allocated to it to improve the execution efficiency. If the execution speed of the task still cannot meet the requirements after adding resources of the same layer, some higher-level computing resources with stronger load capacity are allocated to it.

[0122] In the embodiment of the present invention, by setting respective resource allocation strategies for the two divided queues, resources can be allocated according to the characteristics and requirements of the tasks in the queues. For the high-priority queue, high-performance resources with strong load capacity can be allocated to ensure its fast execution; and for the low-priority queue, an appropriate amount of resources with weaker load capacity can be allocated to meet its basic needs. This differentiated resource allocation strategy can better meet the needs of different tasks and improve flexibility and rationality.

[0123] In an embodiment of the present invention, the computing resources in the cloud resource pool are load rated to obtain the load level of each computing resource, and a resource load pyramid model is constructed according to the load level of each computing resource, wherein computing resources with low load levels are located below computing resources with high load levels in the resource load pyramid model; multiple tasks received by the cloud computing platform are obtained, and the multiple tasks are prioritized, and the multiple tasks are arranged in descending order according to the priority order to obtain a task priority queue; resources are allocated according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue. Taking into account the characteristics of task priority, resource load capacity and cloud resource pool, the effective management and efficient utilization of cloud computing resources are realized through the construction and dynamic adjustment of the pyramid model, and the existing problem of being unable to effectively manage and efficiently utilize cloud computing resources is solved.

[0124] In the embodiment of the present invention, optionally, performing resource allocation according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue includes:

[0125] A top-down load level reduction task resource allocation strategy is adopted for the high priority team in the task priority queue;

[0126] A bottom-up load-level incremental task resource allocation strategy is adopted for the low-priority queue in the task priority queue.

[0127] In an embodiment of the present invention, resources are allocated according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue, and a resource concession waiting strategy is introduced, that is, when the computing resources in the resource load pyramid are less or the pyramid levels are less, the high priority team and the low priority team can call the same computing resource at the same time during the simultaneous execution. When this happens, a resource concession or waiting strategy is adopted, wherein the concession strategy is to give up allocating the computing resource to the task in the low priority team, allocate it to the high priority team task, and find other idle computing resources at the same level or adjacent levels to allocate to the low priority team task; the waiting strategy is to temporarily suspend the low priority team task if there are no other idle resources that meet the conditions, wait for the computing resource to be released, or other computing resources that meet the requirements to be released, and then allocate computing resources to it to execute the task.

[0128] In the embodiment of the present invention, a resource concession strategy is introduced, which can not only ensure the priority occupation right of high-priority tasks for resources under normal circumstances, but also allow low-priority tasks to temporarily occupy the resources of high-priority tasks under certain special circumstances, thereby ensuring the overall stability and availability of the system, so that system resources can be fully utilized without affecting the execution of high-priority tasks, thereby improving the fault tolerance and robustness of the system.

[0129] In the embodiment of the present invention, optionally, the method further includes:

[0130] Monitoring and recording the task execution effect based on the resource load pyramid model, and forming historical data according to the recorded task execution effect;

[0131] The historical data is analyzed to optimize the process of allocating resources according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

[0132] In the embodiment of the present invention, monitoring and recording the task execution effect based on the resource load pyramid model, and forming historical data according to the recorded task execution effect specifically includes:

[0133] Collection and recording of data, including task execution data, resource usage data, log information, etc.;

[0134] Clean the collected data to remove duplicate, erroneous or invalid data; organize the data so that it can be aggregated and queried according to dimensions such as task sequence, resource level, and time;

[0135] The historical data is analyzed to optimize the resource allocation process according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue, specifically including:

[0136] Analyze the historical data generated above, including task execution efficiency analysis, resource utilization analysis, task and resource matching analysis, and time series analysis;

[0137] According to the above analysis, the strategy is optimized and adjusted, for example: Optimize the resource allocation strategy: Based on the historical data analysis results, adjust the resource allocation strategy, such as optimizing the high priority team G 1 The top-down reduction allocation strategy and low priority team G 2A bottom-up incremental allocation strategy is adopted to achieve high accuracy in the initial resource allocation and reduce the time required for subsequent adjustments; task scheduling optimization: according to the execution sequence and effect of historical tasks, the task scheduling algorithm is optimized to reduce the time required for the next allocation and scheduling of similar tasks; feedback and iteration of results: the optimized strategy is applied to the actual environment and its effect is continuously monitored; the execution effect of the new strategy is regularly evaluated, and compared with the old strategy to identify possible problems and room for improvement; finally, the evaluation results are fed back to the strategy optimization link for iterative optimization to form a closed-loop optimization process.

[0138] Please refer to Figure 6 In the embodiment of the present invention, the overall process of cloud computing task scheduling includes:

[0139] Load rating and pyramid model establishment for computing resources in the cloud resource pool, that is, load rating of computing resources in the cloud resource pool is carried out to obtain the load level of each computing resource, and a resource load pyramid model is constructed according to the load level of each computing resource; by proposing a resource load pyramid model and introducing it into the load rating and stratification of computing resources in the cloud resource pool, resources are divided into different levels according to factors such as performance and availability, so that resources can be managed and scheduled more finely, ensuring that high-performance resources are used to perform critical tasks, while lower-performance resources are used to perform non-critical tasks, maximizing resource utilization and avoiding resource waste;

[0140] Priority division and task priority queue setting, that is, obtaining multiple tasks received by the cloud computing platform, and prioritizing the multiple tasks, and arranging the multiple tasks in descending order of priority to obtain a task priority queue; through task priority division, it can ensure that high-priority tasks are given priority and low-priority tasks are given the opportunity to be executed, thus achieving fair scheduling of tasks, balancing resource requirements between tasks of different priorities, and improving the overall performance of the cloud resource pool;

[0141] Reasonable high priority ratio setting and task queue division, that is, setting the task high priority ratio for the task priority queue, dividing the task priority queue into a low priority team and a high priority team based on the task high priority ratio, wherein the difference between the proportion of the high priority team in the task priority team and the proportion of the high load level in the resource load pyramid model meets the preset ratio threshold; by dividing the task queue into a low priority team and a high priority team, executing the divided high priority queue and the low priority queue at the same time, it can ensure that high priority tasks are given priority and low priority tasks are given the opportunity to be executed, thus realizing fair scheduling of tasks, avoiding the situation where high priority tasks occupy resources for a long time and low priority tasks cannot be executed, balancing the resource requirements between tasks of different priorities, and improving the overall performance of the cloud resource pool;

[0142] Task resource allocation strategy formulation and execution, that is, resource allocation is performed according to the load level of computing resources in the resource load pyramid model and the priority order in the task priority queue; by setting respective resource allocation strategies for the two divided queues, resources can be allocated according to the characteristics and requirements of the tasks in the queues. For high-priority queues, high-performance resources with strong load capacity can be allocated to ensure their fast execution; for low-priority queues, appropriate resources with weaker load capacity can be allocated to meet their basic needs. This differentiated resource allocation strategy can better meet the needs of different tasks and improve flexibility and rationality;

[0143] Historical data recording and optimization, i.e. monitoring and recording the task execution effects based on the resource load pyramid model, and forming historical data according to the recorded task execution effects; analyzing the historical data, and optimizing the resource allocation process according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue, thereby improving the accuracy of the model.

[0144] In the embodiment of the present invention, by dividing priorities and allocating resources in a pyramid model, it is ensured that high-priority tasks are processed in a timely and efficient manner, while avoiding long waiting times for low-priority tasks due to insufficient resources. Such a strategy helps to improve the resource utilization efficiency and task execution efficiency of the entire cloud platform and cloud resource pool, and reduce resource waste; and for critical tasks that require rapid response, by giving priority to allocating computing resources with strong load capacity, the task execution time can be significantly reduced, and the experience of important users can be improved; at the same time, for non-critical tasks, reasonable resource allocation can also be used to ensure that they are executed, and they will not be shelved for a long time due to insufficient resources, thereby improving user evaluation; and only optimization is performed on the algorithm to reduce unnecessary resource waste, thereby reducing operating costs; in addition, due to the improvement of task execution efficiency, the additional costs caused by task delays or failures can also be reduced.

[0145] Please refer to Figure 7 , an embodiment of the present invention provides a cloud computing task scheduling device, comprising:

[0146] The first processing module 71 is used to load rate the computing resources in the cloud resource pool, obtain the load level of each computing resource, and construct a resource load pyramid model according to the load level of each computing resource, wherein computing resources with low load levels are located below computing resources with high load levels in the resource load pyramid model;

[0147] The second processing module 72 is used to obtain multiple tasks received by the cloud computing platform, prioritize the multiple tasks, and arrange the multiple tasks in descending order of priority to obtain a task priority queue;

[0148] The resource allocation module 73 is used to allocate resources according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

[0149] In the embodiment of the present invention, optionally, the first processing module includes:

[0150] An integration submodule, used to partition and integrate the scattered computing resources in the cloud resource pool to obtain a computing resource set;

[0151] An acquisition submodule is used to acquire static indicator parameters and dynamic indicator parameters of the computing resources in the computing resource set; wherein the static indicator parameters include at least one of the following: CPU utilization, storage capacity, network bandwidth, memory occupancy, disk I / O; the dynamic indicator parameters include at least one of the following: resource request quantity, resource service capability, resource service intensity;

[0152] The first calculation submodule is used to calculate the load capacity value of the computing resource based on the static indicator parameter and the dynamic indicator parameter, and obtain the load level of the computing resource according to the load capacity value.

[0153] In the embodiment of the present invention, optionally, the first processing module includes:

[0154] A second calculation submodule, used to determine the initial number of layers of the resource load pyramid model according to the load level of the computing resources in the computing resource set;

[0155] A first processing submodule, configured to input the load level and the initial number of layers into a pre-trained clustering model, wherein the clustering model outputs a computing resource set of each layer according to the distribution of load capacity values ​​and business requirements;

[0156] The second processing submodule is used to determine whether the load level in the computing resource set of each level meets the preset load balancing threshold. If the load level in the computing resource set of each level meets the preset load balancing threshold, the final level division is directly obtained; if the load level in the computing resource set of each level does not meet the preset load balancing threshold, the computing resource set of each level is readjusted to obtain the final level division; the resource load pyramid model is obtained according to the final level division.

[0157] In the embodiment of the present invention, optionally, the second processing module includes:

[0158] A third processing submodule is used to perform demand analysis processing on the tasks received by the cloud computing platform, and prioritize the tasks according to task priority consideration parameters, wherein the task priority consideration parameters include at least one of the following: task type, task importance, system load, and resource demand;

[0159] The fourth processing submodule is used to arrange the tasks in descending order of priority to obtain a task priority queue.

[0160] In the embodiment of the present invention, optionally, the method further includes:

[0161] The optimization module is used to set a high-priority task ratio for the task priority queue, and divide the task priority queue into a low-priority team and a high-priority team based on the high-priority task ratio, wherein the difference between the proportion of the high-priority team in the task priority queue and the proportion of the high load level in the resource load pyramid model meets a preset proportion threshold.

[0162] In the embodiment of the present invention, optionally, the resource allocation module includes:

[0163] A first resource allocation submodule, configured to adopt a top-down load level reduction task resource allocation strategy for a high priority queue in the task priority queue;

[0164] The second resource allocation submodule is used to adopt a bottom-up load level increasing task resource allocation strategy for the low priority queue in the task priority queue.

[0165] In the embodiment of the present invention, optionally, the method further includes:

[0166] A monitoring module, used to monitor and record the task execution effect based on the resource load pyramid model, and form historical data according to the recorded task execution effect;

[0167] The analysis module is used to analyze the historical data and optimize the resource allocation process according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

[0168] The cloud computing task scheduling device provided by the embodiment of the present invention can realize Figure 1 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0169] An embodiment of the present invention provides an electronic device 80, see Figure 8 As shown, Figure 8 This is a principle block diagram of an electronic device 80 according to an embodiment of the present invention, including a processor 81, a memory 82, and a program or instruction stored in the memory 82 and executable on the processor 81. When the program or instruction is executed by the processor, the steps in any one of the cloud computing task scheduling methods of the present invention are implemented.

[0170] An embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of an embodiment of a cloud computing task scheduling method as described above is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0171] The embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the method embodiment shown in the above figure are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0172] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0173] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0174] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0175] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0176] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0177] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A cloud computing task scheduling method, characterized in that: include: Performing load rating on computing resources in the cloud resource pool to obtain a load level of each computing resource, and constructing a resource load pyramid model according to the load level of each computing resource, wherein computing resources with a low load level are located below computing resources with a high load level in the resource load pyramid model; Acquire multiple tasks received by the cloud computing platform, prioritize the multiple tasks, and arrange the multiple tasks in descending order of priority to obtain a task priority queue; Resource allocation is performed according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

2. The cloud computing task scheduling method according to claim 1, characterized in that: The computing resources in the cloud resource pool are rated for load to obtain different load levels, including: Partitioning and integrating the scattered computing resources in the cloud resource pool to obtain a computing resource set; Obtain static indicator parameters and dynamic indicator parameters of the computing resources in the computing resource set; wherein the static indicator parameters include at least one of the following: CPU utilization, storage capacity, network bandwidth, memory occupancy, disk I / O; the dynamic indicator parameters include at least one of the following: resource request volume, resource service capability, resource service intensity; The load capacity value of the computing resource is calculated based on the static indicator parameter and the dynamic indicator parameter, and the load level of the computing resource is obtained according to the load capacity value.

3. The cloud computing task scheduling method according to claim 2, characterized in that: The step of constructing a resource load pyramid model according to the load level includes: Determining the initial number of layers of the resource load pyramid model according to the load levels of the computing resources in the computing resource set; Inputting the load level and the initial number of layers into a pre-trained clustering model, the clustering model outputting a computing resource set of each layer according to the distribution of load capacity values ​​and business requirements; Determine whether the load level in the computing resource set of each level meets the preset load balancing threshold, and if the load level in the computing resource set of each level meets the preset load balancing threshold, directly obtain the final level division; If the load level in the computing resource set of each level does not meet the preset load balancing threshold, the computing resource set of each level is readjusted to obtain the final level division; The resource load pyramid model is obtained according to the final level division.

4. The cloud computing task scheduling method according to claim 1, characterized in that: The step of obtaining tasks received by the cloud computing platform and prioritizing the tasks includes: Performing demand analysis on the tasks received by the cloud computing platform, and prioritizing the tasks according to task priority consideration parameters, wherein the task priority consideration parameters include at least one of the following: task type, task importance, system load, and resource demand; Arrange the tasks in descending order of priority to obtain a task priority queue.

5. The cloud computing task scheduling method according to claim 1, characterized in that: The step of obtaining tasks received by the cloud computing platform, prioritizing the tasks, and arranging the tasks in descending order of priority to obtain a task priority queue further includes: A high-priority task ratio is set for the task priority queue, and the task priority queue is divided into a low-priority team and a high-priority team based on the high-priority task ratio, wherein the difference between the proportion of the high-priority team in the task priority team and the proportion of the high load level in the resource load pyramid model satisfies a preset ratio threshold.

6. The cloud computing task scheduling method according to claim 5, characterized in that: The resource allocation according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue includes: A top-down load level reduction task resource allocation strategy is adopted for the high priority team in the task priority queue; A bottom-up load-level incremental task resource allocation strategy is adopted for the low-priority queue in the task priority queue.

7. The cloud computing task scheduling method according to claim 1, characterized in that: Also includes: Monitoring and recording the task execution effect based on the resource load pyramid model, and forming historical data according to the recorded task execution effect; The historical data is analyzed to optimize the process of allocating resources according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

8. A cloud computing task scheduling device, characterized in that: include: A first processing module is used to load rate the computing resources in the cloud resource pool to obtain the load level of each computing resource, and to construct a resource load pyramid model according to the load level of each computing resource, wherein computing resources with low load levels are located below computing resources with high load levels in the resource load pyramid model; The second processing module is used to obtain multiple tasks received by the cloud computing platform, prioritize the multiple tasks, and arrange the multiple tasks in descending order of priority to obtain a task priority queue; The resource allocation module is used to allocate resources according to the load level of the computing resources in the resource load pyramid model and the priority order in the task priority queue.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps in the cloud computing task scheduling method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps in the cloud computing task scheduling method according to any one of claims 1 to 7 are implemented.

11. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps in the cloud computing task scheduling method as described in any one of claims 1 to 7.

Images