Machine learning-based adaptive scheduling method for cloud computing tasks

Abstract

The present invention relates to the field of cloud computing resource management, and in particular to a machine learning-based adaptive scheduling method for cloud computing tasks. The method comprises: S1, carrying out virtual machine resource modeling; S2, classifying virtual machine resources by means of a GA-FCM algorithm, dividing the virtual machine resources into calculation resources, storage resources and network resources, and using an FCM algorithm improved on the basis of a genetic algorithm to complete clustering; S3, constructing task features; S4, calculating Pearson correlation coefficients of feature vectors and the clustering centers of three types of resources; S5, carrying out task scheduling and allocation; and S6, performing monitoring, and adjusting cloud computing tasks. The present invention can consider virtual machine resources for task execution from three aspects of calculation, storage and network: for the virtual machine resources for task execution, the utilization rates of the three types of resources of calculation, storage, and network need to be in a non-alarm state both before and after task execution, and at the same time, a time load for task execution needs to be minimized, and a time overhead for task execution is reduced.

Description

Machine Learning-Based Adaptive Scheduling Method for Cloud Computing Tasks

[0001] This application claims priority to Chinese patent application No. CN202410509515.8, filed on April 26, 2024, entitled "Adaptive Scheduling Method for Cloud Computing Tasks Based on Machine Learning", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention belongs to the field of cloud computing resource management, and in particular relates to an adaptive scheduling method for cloud computing tasks based on machine learning. Background Technology

[0003] Resource scheduling, as one of the core technologies of cloud computing, significantly impacts the overall system's operational efficiency and service performance through its rational scheduling strategy. A well-designed resource scheduling strategy can reduce the runtime of cloud computing tasks, improve system reliability and utilization, ensure the secure and stable operation of the system platform, provide customers with a superior user experience, and ultimately increase customer satisfaction.

[0004] In a cloud computing environment, there are abundant and diverse resources, including computing resources, network resources, and storage resources. Different tasks have different requirements for these resources. Some tasks that require a large amount of computation will prioritize high-performance computing resources, while some tasks that require network interaction have lower performance requirements for computing resources but higher requirements for sufficient bandwidth resources.

[0005] Most existing resource scheduling strategies are task-oriented, selecting suitable virtual machine resources from the cloud platform's resource pool based on the specific needs of the business system. However, due to the large scale and heterogeneity of cloud computing server clusters, matching suitable virtual machine resources to the task queue when the task volume is high and the number of resources in the cloud pool is large can lead to frequent access requests and significant overall time overhead. This results in low cloud platform resource scheduling efficiency and poses a significant challenge to cloud platform resource management, allocation, and performance stability. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the purpose of the invention is to provide a machine learning-based adaptive scheduling method for cloud computing tasks. This method comprehensively considers the usage of virtual resources and underlying host resources (in non-alarm states) and time load to determine the virtual resources for executing the current task, thereby realizing adaptive scheduling of cloud computing tasks.

[0007] In a first aspect, the present invention proposes a machine learning-based adaptive scheduling method for cloud computing tasks, comprising:

[0008] S1, perform virtual machine resource modeling, based on the host cluster of compute node components, and obtain multiple virtual machine resources through virtualization processing;

[0009] S2 classifies virtual machine resources using the GA-FCM algorithm, dividing them into computing resources, storage resources, and network resources, and then uses the FCM algorithm, which is improved by combining genetic algorithms, to complete the clustering.

[0010] S3 builds characteristics for each independent cloud computing task;

[0011] S4, calculate the feature vectors of each cloud computing task and the Pearson correlation coefficients of the cluster centers of the three types of resources;

[0012] S5 schedules and allocates independent cloud computing tasks;

[0013] S6 monitors virtual machine and host machine resources and adjusts cloud computing tasks based on resource utilization.

[0014] Furthermore, in the step of virtual machine resource modeling, the host cluster consisting of H computing nodes virtualizes e virtual machine resources; using a one-dimensional vector Od j Od represents the j-th virtual machine resource, j = 1, 2, ..., e. j ={num j ,cal j ,stor j ,bw j}, where num j ,num j =1,2,…,c indicates that j virtual machine resources are located on the num-th computing server; cal j This represents the computing power of the j-th virtual machine resource; stor j bw represents the storage capacity of the j-th virtual machine resource; j This represents the network performance of j virtual machine resources.

[0015] Furthermore, in S1, the steps for modeling virtual machine resources include: constructing computing power, storage power, and network capabilities, wherein the network capabilities are described by the bandwidth configured in the virtual machine resources.

[0016] Furthermore, for the num-th computing server, num = 1, ..., H, its total computing power cal_num tol For: cal_num tol =ser_n*ser_cor*ser_fre*ser_th r

[0017] Where ser_n is the number of CPUs in the num-th computing server; ser_cor is the number of CPU cores in the num-th computing server; ser_fre is the clock speed of the num-th computing server's CPU; ser_thr is the number of threads in the num-th computing server's CPU; and the num-th computing server can provide n... num If there are 10 vCPUs, then the computing power (calc) of the j-th virtual machine resource located on the num-th compute server is... j for:

[0018] Where, n num_j This represents the number of vCPUs of the j-th virtual machine resource on the num-th computing server.

[0019] Furthermore, the IOPS of the cloud disk mounted on the j-th virtual machine resource is denoted as stor_cd. j The storage capacity of the j-th virtual machine resource is stor j For: stor j =min(stor_cd) j ,stor_sd j );

[0020] Among them, stor_cd j The IOPS of the cloud disk mounted for the j-th virtual machine resource is related to the cloud disk capacity; stor_sd j The upper limit of IOPS for the disk type corresponding to the cloud disk mounted for the j-th virtual machine resource is related to the underlying technology.

[0021] Furthermore, the virtual machine resources are classified into computing resources, storage resources, and network resources using the GA-FCM algorithm. The steps include: modeling the problem, initializing the parameters of the genetic algorithm, performing iterative evolution based on the genetic algorithm, simulating binary crossover, performing polynomial mutation, evaluating the fitness values ​​of all individuals in the corresponding population at the current iteration number, calculating the fitness function, determining the iteration termination condition, and obtaining the optimal initial cluster center.

[0022] Furthermore, the step of using the FCM algorithm improved by combining genetic algorithm to complete clustering includes setting smoothing parameters, calculating the membership matrix, updating the cluster centers, determining the iteration termination condition, and after the stacking is completed, obtaining the cluster centers of various resources and virtual machine resources, and dividing each resource into resource clusters according to the maximum membership value corresponding to each cluster center in the membership matrix.

[0023] Furthermore, in S1, the step of scheduling and allocating independent cloud computing tasks includes setting alarm parameters, determining alarm thresholds for each type of resource, selecting cloud host resources that can participate in task scheduling, calculating the completion time of each cloud computing task, calculating the time load of each virtual machine resource, recalculating the computer resource utilization rate, storage resource utilization rate, and bandwidth resource utilization rate of the cloud host when carrying cloud computing tasks, and scheduling the independent cloud computing tasks.

[0024] Furthermore, the cloud host resources capable of task scheduling are traversed, and cloud hosts whose resource values ​​for all types of cloud hosts are less than the corresponding alarm thresholds are retained. The time load is compared, and independent cloud computing tasks are assigned to the cloud hosts with the lowest time load for execution.

[0025] Furthermore, in the step of monitoring virtual machine resources and host machine resources and adjusting cloud computing tasks based on resource utilization, non-core tasks with low resource utilization are prioritized for reallocation during resource adjustment. If an alarm still occurs, core tasks with high usage frequency need to be scheduled during idle time.

[0026] The beneficial effects of this invention are as follows:

[0027] The machine learning-based adaptive scheduling method for cloud computing tasks described in this invention starts from the virtual resources of the cloud platform, takes into account the characteristics of the resource pool itself, and models the performance of computing, storage, and network in a comprehensive manner. Users can select appropriate types of cloud resources according to their needs and preferences, thereby improving resource utilization and reducing the time overhead of resource scheduling.

[0028] This invention establishes virtual resource distribution characteristics from three dimensions: computing, network, and storage. It incorporates factors such as server CPU frequency, overcapacity, and system overhead to establish a virtual resource computing capacity model; it also considers the differences in technology among different cloud vendors and the limitations of cloud host specifications to establish a virtual resource storage capacity model; and it establishes a network capacity model based on bandwidth. Based on these three resource characteristic models, the GA-FCM algorithm is introduced to classify virtual resources into three categories: computing resources, storage resources, and network resources. Simultaneously, tasks to be scheduled are abstracted into three dimensions: computing, storage, and network. Similarity is used to determine the resource category to which the current task belongs, enabling the current strategy to adaptively schedule cloud computing tasks.

[0029] Task scheduling considers not only task completion time but also the potential impact of scheduling strategies on the overall virtual machine cluster. Taking into account the task-handling capacity of multiple virtual machines, to improve system platform stability, the scheduling allocation considers virtual machine resources in three aspects—computing, storage, and network—based on established resource distribution characteristics. Before and after task execution, the utilization rates of computing, storage, and network resources of the virtual machines executing tasks must be in a non-alarm state, while minimizing the time load of task execution to reduce execution time overhead. This ensures that the scheduling strategy not only guarantees rapid task completion but also ensures the effectiveness and security of the virtual machine cluster. Attached Figure Description

[0030] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. It is obvious that the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings.

[0031] Figure 1 is a first flowchart of the cloud computing task adaptive scheduling method based on machine learning provided in an embodiment of the present invention;

[0032] Figure 2 is a second flowchart of the cloud computing task adaptive scheduling method based on machine learning provided in an embodiment of the present invention;

[0033] Figure 3 is an architecture diagram of the machine learning-based adaptive scheduling method for cloud computing tasks provided in an embodiment of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts disclosed in this invention.

[0036] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The terms "installed," "connected," and "linked" should be interpreted broadly; for example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0037] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of methods and systems consistent with some aspects of the invention as detailed in the appended claims.

[0038] This invention proposes a machine learning-based adaptive scheduling method for cloud computing tasks. It addresses the problem that due to the large scale and heterogeneity of cloud computing server clusters, when the task volume is large and the number of resources in the cloud pool is large, matching appropriate virtual machine resources to the task queue will lead to frequent access requests, resulting in high overall time overhead and low resource scheduling efficiency of the cloud platform.

[0039] This invention provides a machine learning-based adaptive scheduling method for cloud computing tasks, comprising:

[0040] S1, perform virtual machine resource modeling, based on the host cluster of compute node components, and obtain multiple virtual machine resources through virtualization processing;

[0041] S2 classifies virtual machine resources using the GA-FCM algorithm, dividing them into computing resources, storage resources, and network resources, and then uses the FCM algorithm, which is improved by combining genetic algorithms, to complete the clustering.

[0042] S3 builds characteristics for each independent cloud computing task;

[0043] S4, calculate the feature vectors of each cloud computing task and the Pearson correlation coefficients of the cluster centers of the three types of resources;

[0044] S5 schedules and allocates independent cloud computing tasks;

[0045] S6 monitors virtual machine and host machine resources and adjusts cloud computing tasks based on resource utilization.

[0046] In this embodiment of the invention, as shown in Figures 1, 2, and 3, a host cluster composed of computing nodes virtualizes e virtual machine resources. A one-dimensional vector Od is used. j Od represents the j-th virtual machine resource, j = 1, 2, ..., e. j ={num j ,cal j ,stor j ,bw j}, where num j ,num j =1,2,…,c indicates that j virtual machine resources are located on the num-th computing server; cal j This represents the computing power of the j-th virtual machine resource; stor j bw represents the storage capacity of the j-th virtual machine resource; j This represents the network performance of j virtual machine resources.

[0047] Step 1: Virtual machine resource modeling.

[0048] Step 1.1, Calculation ability (cal) j For the num-th computing server, num = 1, ..., H, its total computing power cal_num tol For: cal_num tol =ser_n*ser_cor*ser_fre*ser_thr

[0049] Where ser_n is the number of CPUs in the num-th computing server; ser_cor is the number of CPU cores in the num-th computing server; ser_fre is the clock speed of the num-th computing server's CPU; and ser_thr is the number of threads in the num-th computing server's CPU. Considering over-provisioning and system overhead, let the num-th computing server provide n... num If there are 10 vCPUs, then the computing power (cal) of the 1st virtual machine resource located on the num-th compute server is... j for:

[0050] Where, n num_j This represents the number of vCPUs of the j-th virtual machine resource on the num-th computing server.

[0051] Step 1.2, Storage Capacity (Store) j IOPS is a primary metric for measuring disk performance, and this invention uses IOPS to describe the storage capacity of virtual machine resources. IOPS can be considered as the number of reads and writes per second. The IOPS performance of cloud disks mounted on virtual machine resources is affected by capacity, and different cloud providers use different calculation parameters. This invention denotes the IOPS of the cloud disk mounted on the j-th virtual machine resource as `stor_cd`. j The storage performance of the j-th virtual machine resource is also limited by the IOPS limit of the disk type corresponding to the cloud disk to which the current virtual machine resource is mounted, which is also limited by the storage_sd limit. j The impact is that the storage capacity of the i-th virtual machine resource is stored. j For: stor j =min(stor_cd) j ,stor_sd j );

[0052] Among them, stor_cd j The IOPS of the cloud disk mounted for the j-th virtual machine resource is related to the cloud disk capacity; stor_sd j The upper limit of IOPS for the disk type corresponding to the cloud disk mounted for the j-th virtual machine resource is related to the underlying technology.

[0053] Step 1.3, Network Capabilities (bw) j This invention uses the bandwidth configured in the virtual machine resources to describe its network performance.

[0054] Step 2: Clustering of virtual machine resources.

[0055] Virtual machine resources are classified into computing resources, storage resources, and network resources using the GA-FCM algorithm, and clustering is completed using the FCM algorithm improved by combining genetic algorithm.

[0056] Step 2.1: Use the GA algorithm to obtain the initial cluster centers for the three types of resources: computing, storage, and network.

[0057] Step 2.1.1, Problem Modeling:

[0058] In the calculation, the label number of the virtual machine resource is removed before calculation, and the feature vector of the virtual machine resource is denoted as... The feature vectors of the three cluster centers are Encode the three cluster centers into a real-number encoded genetic string x = [cal c1 ,stor c1 ,bw c1 ,cal c2 ,stor c2,bw c2 ,cal c3 ,stor c3 ,bw c3 There are 9 variables in total. The fitness function for the genetic algorithm is defined as follows:

[0059] Where J(U,C) is the objective function of the FCM algorithm:

[0060] Among them, u ij Let c represent the membership degree of the j-th, j=1,…,e virtual machine resource to the i-th, i=1,…,c cluster center. In this invention, virtual machine resources are divided into three categories, hence c=3. i Let represent the i-th cluster center. m is the weighting exponent, m≥1. ||·||2 denotes the vector 2-norm.

[0061] Step 2.1.2: Initialize the genetic algorithm parameters:

[0062] Given a population size P, a maximum number of generations T, a current iteration number t = 1, a crossover probability β, and a mutation probability η, calculate all... The maximum and minimum values ​​of each dimension of the virtual machine resource vector constitute the upper and lower bounds of the genetic code. P is then randomly generated within the range of these upper and lower bounds. (0) Given an initial number of individuals, calculate the fitness function f of the population. m .

[0063] Step 2.1.3, Iterative Evolution:

[0064] In the t-th iteration of evolution, the previous generation population P... (t-1) The individual with the highest fitness, x j(max) Unconditionally replicated into the next generation of the population, the remaining individuals undergo random genetic variation, forming a new population P. (t) For population P (t-1) Individual x j(k) , x j(k) =[cal c1 j(k) ,stor c1 j(k) ,bw c1 j(k) ,cal c2 j(k) ,stor c2 j(k) ,bw c2 j(k) ,cal c3 j(k) ,stor c3 j(k) ,bwc3 j(k) For a given set of k = 1, ..., P, and j ≠ j(max), perform genetic mutation by following these steps:

[0065] Step 2.1.3.1: Simulate binary crossover

[0066] According to probability rand β From population P (t-1) Select individuals to mate. Randomly select two parent individuals x. j(i1) x j(i2) Two offspring individuals x are generated using a crossover operator that simulates single-point binary crossover. j(igen1) x j(igen2) :

[0067] Where χ is composed of the crossover probability β and the random parameter rand β Decide:

[0068] Offspring individual x j(igen1) and x j(igen2) Replace the parent individual in the new population P (t′) Individuals that do not mate directly enter the new population P. (t') .

[0069] Step 2.1.3.2, Polynomial Mutation. According to the probability rand... η For the new population P (t′) Individual x j(igen) Perform a mutation operation to obtain a new mutated individual xj(igen_new)': xj(igen_new)'=x j(igen) '+δ×(x max '-x min ')

[0070] in,

[0071] x max Population P (t′) The individual in the equation is composed of the maximum values ​​of each dimension, x. max =max[P (t') ];x min 'Represents population P (t′) The individual x is formed by the minimum value of each dimension of the individual. min =min[P (t') The mutated new individual xj(igen_new)' replaces the original individual and enters the new population P. (t') Individuals that have not undergone mutation directly enter the new population, P (t′)The population is the set of individuals in the population after the t-th inheritance.

[0072] Step 2.1.3.3: Evaluate the population P at the current iteration number t. (t′) The fitness values ​​of all individuals. Calculate the fitness function:

[0073] Step 2.1.3.4, Iteration Termination Condition Judgment: Determine whether the maximum number of iterations T has been reached. If t < T, then t = t + 1 and return to step 2.1.3.

[0074] Otherwise, terminate the loop and return the individual x with the highest fitness. f_max : x f_max =[cal c1 max ,stor c1 max ,bw c1 max ,cal c2 max ,stor c2 max ,bw c2 max ,cal c3 max ,stor c3 max ,bw c3 max ].

[0075] Extracting every three dimensions from the 9-dimensional vector of this individual, resulting in three 3-dimensional vectors, these represent the optimal initial cluster centers for the three types of virtual resources found by the GA algorithm, denoted as . have

[0076] Step 2.2: Based on the optimal initial cluster centers obtained in Step 2.1 Cluster virtual machine resources using the FCM algorithm.

[0077] Step 2.2.1: Set the smoothing parameter m. Set the maximum number of FCM iterations T. fcm The current iteration number t = 1. Set the iteration threshold t = 1. v0, respectively, serves as the initial cluster centers for the three classes in the FCM clustering algorithm. 1 v0 2 v0 3 The process is iterated, and the cluster center of the t-th iteration is denoted as v. t i , i = 1, ..., c.

[0078] Step 2.2.2: Calculate the membership matrix. In the t-th iteration, calculate the virtual machine resources. With the i-th, i=1,…,c resource cluster center v t-1 i membership degree u ij :

[0079] Where, d 2 (x,y) represents the Euclidean distance between vectors x and y.

[0080] Step 2.2.3: Update cluster centers. Update the cluster centers v in the t-th iteration. t i :

[0081] Step 2.2.4, v t =[v t 1 ,v t 2 ,v t 3 If ||v t+1 -v t ||≤ε or the maximum number of iterations T is reached fcm If the iteration fails, the iteration terminates; otherwise, t = t + 1, and the process returns to step 2.2.2.

[0082] Step 2.2.5: After iteration, the cluster centers v of various resources are obtained. f 1 v f 2 v f 3 and e virtual machine resources Membership matrix u of various resource types ij Each resource is assigned a membership matrix u. ij Resource clusters are defined based on the maximum membership value of each cluster center; that is, for the j-th resource... Find max{u 1j ,u 2j ,u 3j}, and assign the j-th resource to this cluster.

[0083] Step 3: For each independent task M i Establish features: namely M i ={cal_T i ,stor_T i ,bw_T i}. Where cal_T i Indicates task Mi The computational requirements; stor_T i Indicates storage requirements; bw_T i This indicates bandwidth requirements.

[0084] Step 4: Calculate M for each task i Feature vectors and three types of resource cluster centers v f i Pearson correlation coefficient for (i = 1, 2, 3):

[0085] Where E(·) represents the expectation. The cluster center with the highest correlation coefficient is obtained, and the task is assigned to the resource pool corresponding to that cluster.

[0086] Step 5: For the independent cloud computing task M i Perform scheduling and allocation.

[0087] Step 5.1, Alarm Parameter Settings. Set the computing resource utilization alarm threshold ω. c Storage resource utilization alarm threshold ω s Bandwidth resource utilization alarm threshold ω b .

[0088] Step 5.2: Select cloud server resources capable of task scheduling. Based on Step 4, select M... i Assigned to virtual machine resources of type np, and within those resources, select k virtual machines capable of task scheduling according to the following requirements. np Taiyun Server:

[0089] ① Able to satisfy task M i The computing, storage, and bandwidth requirements;

[0090] ② The cloud server is in a non-alarm state, that is, the computing resource utilization rate is less than ω. c Storage resource utilization is less than ω s Bandwidth resource utilization is less than ω b .

[0091] Step 5.3: Calculate the completion time of the cloud computing task. The number of tasks that can be scheduled (k) is... np Taiwan Cloud Server, if task M i Scheduled to the kth np Taiyun ECS l , l=1,…,k np Execute on the cloud server ECS. l The execution time of the existing task is t. l According to ECS l Resource configuration details, record the l-th cloud server ECS l Processing task Mi The time required is Since each virtual machine executes tasks in parallel, the completion time of a single virtual machine is determined by the last task to finish. Therefore, when the l-th cloud host ECS... l Execute task M i At that time, its maximum task completion time t max(l) for:

[0092] Step 5.4: Calculate the virtual machine resource time load. If task M... i Dispatch to the l-th cloud server ECS l l=1,…,k np If executed on the above, then the l-th cloud host will carry task M. i Impact of time load on system cluster (LOD) l for:

[0093] sequentially for k np The cloud server carries task M i The time load is calculated to obtain the time load matrix lod = [lod1, ..., lod]. l ,…,lod np ].

[0094] Step 5.5: Recalculate the l-th cloud server (ECS) l Carrying task M i The utilization rates of computing resources, storage resources, and bandwidth resources at that time are calculated. For ease of explanation, the utilization rate of the l-th cloud server (ECS) will be recalculated. l The computing resource utilization rate is denoted as ω. 1(l) Storage resource utilization is denoted as ω. 2(l) Bandwidth resource utilization is denoted as ω. 3(l) .

[0095] Step 5.6: For the independent cloud computing task M i Scheduling is performed. After resource scheduling, all resources must be in a non-alarm state, and the overall platform time load must be minimized. Based on this, task M is determined to be executed. i Virtual machine:

[0096] Step 5.6.1: Traverse the k selected in Step 5.2 that can be scheduled. np Taiyun server, retain ω 1(l) <ω c And ω 2(l) <ω s And ω 3(l) <ω b Cloud servers.

[0097] Step 5.6.2: For cloud hosts that meet the conditions in Step 5.6.1, compare their time load and assign independent cloud computing tasks M... i The task is assigned to the cloud server with the lowest time load for execution.

[0098] Step Six: Data Monitoring. Monitor the usage of virtual machine resources and underlying host machine resources in real time. When an alarm occurs (the utilization of any one of the resources—computing, storage, or bandwidth—exceeds the alarm threshold set in Step 5.1, indicating an alarm status), the cloud computing tasks must be adjusted in real time. During resource adjustment, prioritize non-core tasks with low resource utilization and reallocate them according to Step Five. If an alarm status still occurs, schedule high-frequency core tasks during idle time.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A machine learning-based adaptive scheduling method for cloud computing tasks, characterized in that, include: S1, perform virtual machine resource modeling, based on the host cluster of compute node components, and obtain multiple virtual machine resources through virtualization processing; S2 classifies virtual machine resources using the GA-FCM algorithm, dividing them into computing resources, storage resources, and network resources, and then uses the FCM algorithm, which is improved by combining genetic algorithms, to complete the clustering. S3 builds characteristics for each independent cloud computing task; S4, calculate the feature vectors of each cloud computing task and the Pearson correlation coefficients of the cluster centers of the three types of resources; S5 schedules and allocates independent cloud computing tasks; S6 monitors virtual machine and host machine resources and adjusts cloud computing tasks based on resource utilization.

2. The adaptive scheduling method for cloud computing tasks based on machine learning according to claim 1, characterized in that, In the steps of virtual machine resource modeling, a host cluster consisting of H compute nodes virtualizes e virtual machine resources; a one-dimensional vector Od is used. j Od represents the j-th virtual machine resource, j = 1, 2, ..., e. j ={num j ,cal j ,stor j ,bw j }, where num j ,num j =1,2,...,c indicates that j virtual machine resources are located on the num-th computing server; cal j This represents the computing power of the j-th virtual machine resource; stor j bw represents the storage capacity of the j-th virtual machine resource; j This represents the network performance of j virtual machine resources.

3. The adaptive scheduling method for cloud computing tasks based on machine learning according to claim 2, characterized in that, In S1, the steps for modeling virtual machine resources include: constructing computing power, storage power, and network capabilities, where network capabilities are described by the bandwidth configured in the virtual machine resources.

4. The adaptive scheduling method for cloud computing tasks based on machine learning according to claim 3, characterized in that, For the num-th computing server, num = 1, ..., H, its total computing power cal_num tol For: cal_num tol =ser_n*ser_cor*ser_fre*ser_thr Where ser_n is the number of CPUs in the num-th computing server; ser_cor is the number of CPU cores in the num-th computing server; ser_fre is the clock speed of the num-th computing server's CPU; ser_thr is the number of threads in the num-th computing server's CPU; and the num-th computing server can provide n... num If there are 10 vCPUs, then the computing power (calc) of the j-th virtual machine resource located on the num-th compute server is... j for: Where, n num_j This represents the number of vCPUs of the j-th virtual machine resource on the num-th computing server.

5. The adaptive scheduling method for cloud computing tasks based on machine learning according to claim 3, characterized in that: The storage capacity of the j-th virtual machine resource is stor j for: great j =min(large_cd j ,large_sd j ); Among them, stor_cd j The IOPS of the cloud disk mounted for the j-th virtual machine resource is related to the cloud disk capacity; stor_sd j The upper limit of IOPS for the disk type corresponding to the cloud disk mounted for the j-th virtual machine resource is related to the underlying technology.

6. The adaptive scheduling method for cloud computing tasks based on machine learning according to claim 1, characterized in that, The steps of classifying virtual machine resources using the GA-FCM algorithm into computing resources, storage resources, and network resources include: problem modeling, parameter initialization of the genetic algorithm, iterative evolution based on the genetic algorithm, simulation of binary crossover, polynomial mutation, evaluation of the fitness values ​​of all individuals in the corresponding population at the current iteration number, calculation of the fitness function, determination of the iteration termination condition, and obtaining the optimal initial cluster center.

7. The adaptive scheduling method for cloud computing tasks based on machine learning according to claim 6, characterized in that, The steps of using the FCM algorithm improved by combining genetic algorithm to complete clustering include setting smoothing parameters, calculating the membership matrix, updating the cluster centers, determining the iteration termination condition, and after the iteration is completed, obtaining the cluster centers of various resources and virtual machine resources, and dividing each resource into resource clusters according to the maximum membership value corresponding to each cluster center in the membership matrix.

8. The adaptive scheduling method for cloud computing tasks based on machine learning according to claim 1, characterized in that, In S1, the step of scheduling and allocating independent cloud computing tasks includes setting alarm parameters, determining alarm thresholds for each type of resource, selecting cloud host resources that can participate in task scheduling, calculating the completion time of each cloud computing task, calculating the time load of each virtual machine resource, recalculating the computer resource utilization rate, storage resource utilization rate, and bandwidth resource utilization rate of the cloud host when carrying cloud computing tasks, and scheduling the independent cloud computing tasks.

9. The adaptive scheduling method for cloud computing tasks based on machine learning according to claim 8, characterized in that, Iterate through the cloud host resources that can perform task scheduling, retain cloud hosts whose resource values ​​for all types of cloud hosts are below the corresponding alarm thresholds, compare the time load, and assign independent cloud computing tasks to the cloud hosts with the lowest time loads for execution.

10. The adaptive scheduling method for cloud computing tasks based on machine learning according to claim 1, characterized in that, In the process of monitoring virtual machine and host resources and adjusting cloud computing tasks based on resource utilization, non-core tasks with low resource utilization should be prioritized for reallocation during resource adjustment. If an alarm still occurs, core tasks with high usage frequency should be scheduled during idle time.

Images