Adaptive scheduling method for cloud computing tasks based on machine learning

Abstract

The present invention relates to a method for adaptive scheduling of cloud computing tasks based on machine learning, which belongs to the field of cloud computing resource management. The method includes S1, modeling virtual machine resources; S2, classifying virtual machine resources by GA-FCM algorithm, dividing virtual machine resources into computing resources, storage resources and network resources, and completing clustering by using FCM algorithm improved by genetic algorithm; S3, constructing task features; S4, calculating feature vectors and Pearson correlation coefficients of cluster centers of three types of resources; S5, performing task scheduling and allocation; S6, monitoring and adjusting cloud computing tasks. The present invention can consider the virtual machine resources for executing tasks from three aspects: computing, storage and network: the utilization rates of the three types of resources of computing, storage and network of the virtual machine resources for executing tasks need to be in a non-alarm state before and after executing the tasks, and at the same time, it is also necessary to meet the minimum time load of executing the tasks and reduce the time overhead of task execution.

Description

Technical Field

[0001] The present invention belongs to the field of cloud computing resource management, and in particular relates to a cloud computing task adaptive scheduling method based on machine learning. Background Art

[0002] Resource scheduling is one of the core technologies of cloud computing. The rationality of the scheduling strategy will affect the operating efficiency and service performance of the entire system. A reasonable resource scheduling strategy can reduce the running time of cloud computing tasks, improve the reliability and utilization of the system, ensure the safe and stable operation of the system platform, bring customers a good user experience, and improve customer satisfaction.

[0003] In the cloud computing environment, there are a variety of resources, including computing resources, network resources, and storage resources. Different tasks have different requirements for these resources. Some tasks that require a lot of computing will give priority to computing resources with better performance, while some tasks that require network interaction do not have high requirements for the performance of computing resources, but have high requirements for the adequacy of bandwidth resources.

[0004] Most of the existing resource scheduling strategies start from the task and select the virtual machine resources suitable for executing the current task from the resource pool of the cloud platform according to the needs of the specific business system. Due to the large-scale and heterogeneous nature of cloud computing server clusters, when the task volume is large and the number of resources in the cloud pool is large, matching the task queue with the appropriate virtual machine resources will result in frequent access requests and a large overall time overhead, making the cloud platform resource scheduling efficiency low and also bringing huge challenges to the cloud platform resource management allocation and performance stability. Summary of the invention

[0005] In view of the above shortcomings of the prior art, the purpose of the invention is to provide a method for adaptive scheduling of cloud computing tasks based on machine learning, which comprehensively considers the usage of virtual resources and underlying host resources (in non-alarm state) and time load, determines the virtual resources for executing the current task, and realizes adaptive scheduling of cloud computing tasks.

[0006] In a first aspect of the present invention, a method for adaptively scheduling cloud computing tasks based on machine learning is proposed, comprising:

[0007] S1, virtual machine resource modeling, based on the computing node component host cluster, multiple virtual machine resources are obtained through virtualization processing;

[0008] S2, classify virtual machine resources through GA-FCM algorithm, divide virtual machine resources into computing resources, storage resources and network resources, and complete clustering using FCM algorithm improved by genetic algorithm;

[0009] S3, builds features for each independent cloud computing task;

[0010] S4, calculate the Pearson correlation coefficient of the feature vector of each cloud computing task and the cluster centers of the three types of resources;

[0011] S5, scheduling and allocating independent cloud computing tasks;

[0012] S6, monitor the virtual machine resources and host machine resources, and adjust the cloud computing tasks according to the resource utilization rate.

[0013] Furthermore, in the step of modeling virtual machine resources, a host cluster consisting of H computing nodes virtualizes e virtual machine resources; a one-dimensional vector Od j represents the jth virtual machine resource, j=1,2,…,e, Od j ={num j ,cal j ,stor j ,bw j}, where num j , num j =1,2,…,c means that j virtual machine resources are located on the nymth computing server; cal j represents the computing capacity of the jth virtual machine resources; j represents the storage capacity of the jth virtual machine resources; bw j Represents the network performance of j virtual machine resources.

[0014] Furthermore, in S1, the step of modeling virtual machine resources includes: building computing capacity, storage capacity and network capacity, wherein the network capacity is described by the bandwidth configured by the virtual machine resources.

[0015] Furthermore, for the num, nym = 1, ..., H computing servers, their total computing power cal_num tol for:

[0016] cal_num tol =ser_n*ser_cor*ser_fre*ser_thr

[0017] Among them, ser_n is the number of CPUs in the num-th computing server; ser_cor is the number of cores in the num-th computing server CPU; ser_fre is the main frequency of the num-th computing server CPU; ser_thr is the number of threads in the num-th computing server CPU. The num-th computing server can provide n num vcpus, then the computing capacity of the jth virtual machine resource on the numth computing server is ccal j for:

[0018]

[0019] Among them, n nym_j Indicates the number of vCPUs of the j-th virtual machine resources on the num-th computing server.

[0020] Furthermore, the IOPS of the cloud disk mounted by the jth virtual machine resource is recorded as stor_cd j , the storage capacity of the jth virtual machine resource is stor j for:

[0021] Stor j =min(stor_cd j ,stor_sd j );

[0022] Among them, stor_cd j The IOPS of the cloud disk mounted to the jth virtual machine resource is related to the cloud disk capacity; stor_sd j The IOPS upper limit of the disk type corresponding to the cloud disk mounted to the j-th virtual machine resource is related to the underlying technology.

[0023] Furthermore, the virtual machine resources are classified by the GA-FCM algorithm, and the steps of dividing the virtual machine resources into computing resources, storage resources and network resources include: problem modeling, initializing parameters of the genetic algorithm, iterative evolution based on the genetic algorithm, simulating binary crossover, performing polynomial mutation, evaluating the fitness values ​​of all individuals in the corresponding population under the current number of iterations, calculating the fitness function, judging the iteration termination condition, and obtaining the optimal initialized cluster center.

[0024] Furthermore, the step of completing clustering using the FCM algorithm improved by combining the genetic algorithm includes setting smoothing parameters, calculating the membership matrix, updating the cluster center, and determining the iteration termination condition. After the stacking is completed, the cluster centers of various types of resources and virtual machine resources are obtained, and each resource is divided into resource clusters according to the maximum membership value corresponding to each cluster center in the membership matrix.

[0025] Furthermore, in S1, the step of scheduling and allocating independent cloud computing tasks includes setting alarm parameters, determining the alarm threshold of each resource, selecting cloud host resources that can participate in task scheduling, calculating the completion time of each cloud computing task, and calculating the time load of each virtual machine resource, recalculating the computer resource utilization rate, storage resource utilization rate and bandwidth resource utilization rate of the cloud host when carrying cloud computing tasks, and scheduling independent cloud computing tasks.

[0026] Furthermore, the cloud host resources that can perform task scheduling are traversed, and cloud hosts whose various cloud host resources are less than the corresponding alarm thresholds are retained. The time loads are compared, and independent cloud computing tasks are divided and executed on the cloud host with the smallest time load.

[0027] Furthermore, the virtual machine resources and host machine resources are monitored, and in the step of adjusting the cloud computing tasks according to the resource utilization rate, when adjusting the resources, non-core tasks with low resource utilization are given priority and the cloud computing tasks are reallocated. If the alarm state still occurs, the core tasks with high usage frequency need to be scheduled during idle time.

[0028] The beneficial effects of the present invention are as follows:

[0029] The method for adaptive scheduling of cloud computing tasks based on machine learning described in the present invention starts from the virtual resources of the cloud platform, takes the characteristics of the resource pool itself into consideration, and comprehensively considers the performance of computing, storage, and network to perform modeling. Users can choose the appropriate type of cloud resources according to their needs and preferences, thereby improving resource utilization and reducing the time overhead of resource scheduling.

[0030] The present invention establishes the characteristics of virtual resource distribution from the three dimensions of computing, network, and storage, takes into account the server CPU main frequency, over-allocation ratio, system loss, etc., and establishes a virtual resource computing capacity model; comprehensively considers the different technologies of different cloud vendors and the limitations of cloud host specifications, and establishes a virtual resource storage capacity model; and establishes a network capacity model based on bandwidth. Based on the three resource feature models, the GA-FCM algorithm is introduced to divide virtual resources into three categories: computing resources, storage resources, and network resources. At the same time, the tasks to be scheduled are abstracted into three dimensions of computing, storage, and network, and the resource category to which the current task to be scheduled belongs is judged by similarity, so that the current strategy can adaptively schedule cloud computing tasks.

[0031] When scheduling tasks, not only the task completion time is considered, but also the impact that the scheduling strategy may have on the overall virtual machine cluster. Taking into account the ability of multiple virtual machines to carry tasks, when scheduling and allocating tasks, in order to improve the stability of the system platform, the virtual machine resources for executing tasks are considered from three aspects: computing, storage, and network. Before and after the execution of the task, the utilization rate of the three types of resources, computing, storage, and network, of the virtual machine resources for executing the task must be in a non-alarm state. At the same time, the time load of the task must be minimized to reduce the time overhead of the task execution. This allows the scheduling strategy to not only ensure the rapid completion of the task, but also ensure the effectiveness and security of the virtual machine cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are only used to illustrate specific embodiments and are not considered to limit the present invention. In the entire drawings, the same reference symbols represent the same components. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0033] Figure 1 A first flow chart of a method for adaptively scheduling cloud computing tasks based on machine learning provided in an embodiment of the present invention;

[0034] Figure 2 A second flow chart of the method for adaptive scheduling of cloud computing tasks based on machine learning provided in an embodiment of the present invention;

[0035] Figure 3 An architectural diagram of a method for adaptively scheduling cloud computing tasks based on machine learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present invention.

[0037] Furthermore, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts disclosed in the present invention.

[0038] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0039] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of methods and systems consistent with some aspects of the present invention as detailed in the appended claims.

[0040] The present invention proposes a method for adaptive scheduling of cloud computing tasks based on machine learning, which solves the problem that due to the large-scale and heterogeneous nature of cloud computing server clusters, when the task volume is large and the number of resources in the cloud pool is large, matching appropriate virtual machine resources for the task queue will result in frequent access requests, large overall time overhead, and low cloud platform resource scheduling efficiency.

[0041] Method Embodiment

[0042] The present invention provides a method for adaptively scheduling cloud computing tasks based on machine learning, comprising:

[0043] S1, virtual machine resource modeling, based on the computing node component host cluster, multiple virtual machine resources are obtained through virtualization processing;

[0044] S2, classify virtual machine resources through GA-FCM algorithm, divide virtual machine resources into computing resources, storage resources and network resources, and complete clustering using FCM algorithm improved by genetic algorithm;

[0045] S3, builds features for each independent cloud computing task;

[0046] S4, calculate the Pearson correlation coefficient of the feature vector of each cloud computing task and the cluster centers of the three types of resources;

[0047] S5, scheduling and allocating independent cloud computing tasks;

[0048] S6, monitor the virtual machine resources and host machine resources, and adjust the cloud computing tasks according to the resource utilization rate.

[0049] In the embodiment of the present invention, Figure 1 , Figure 2 and Figure 3 As shown in the figure, the host cluster consisting of H computing nodes virtualizes e virtual machine resources. j represents the jth virtual machine resource, j=1,2,…,e, Od j ={num j ,cal j ,stor j ,bwj}, where num j , num j =1,2,…,c means j virtual machine resources are located on the numth computing server; cal j represents the computing capacity of the jth virtual machine resources; j represents the storage capacity of the jth virtual machine resources; bw j Represents the network performance of j virtual machine resources.

[0050] Step 1: Virtual Machine Resource Modeling

[0051] Step 1.1, calculation capacity cal j :For the numth, num=1,…,H computing servers, their total computing power cal_num tol for:

[0052] cal_num tol =ser_n*ser_cor*ser_fre*ser_thr

[0053] Among them, ser_n is the number of CPUs in the numth computing server; ser_cor is the number of cores in the numth computing server CPU; ser_fre is the main frequency of the numth computing server CPU; ser_thr is the number of threads in the numth computing server CPU. Considering the over-provisioning ratio and system loss, the numth computing server can provide n num vcpus, then the computing capacity of the jth virtual machine resource on the numth computing server is cal j for:

[0054]

[0055] Among them, n num_j Indicates the number of vCPUs of the j-th virtual machine resources on the num-th computing server.

[0056] Step 1.2: Storage capacity j IOPS is the main indicator for measuring disk performance. This paper uses IOPS to describe the storage capacity of virtual machine resources. IOPS can be regarded as the number of reads and writes per second. The IOPS performance of the cloud disk mounted by the virtual machine resources is affected by the capacity, and the calculation parameters of different cloud vendors are different. This paper records the IOPS of the cloud disk mounted by the jth virtual machine resource as stor_cd j The storage performance of the jth virtual machine resource is also affected by the IOPS upper limit stor_sd corresponding to the disk type of the cloud disk mounted by the current virtual machine resource. j The storage capacity of the j-th virtual machine resources isj for:

[0057] Stor j =min(stor_cd j ,stor_sd j );

[0058] Among them, stor_cd j The IOPS of the cloud disk mounted to the jth virtual machine resource is related to the cloud disk capacity; stor_sd j The IOPS limit of the disk type corresponding to the cloud disk mounted to the j-th virtual machine resource, which is related to the underlying technology

[0059] Step 1.3: Network capability bw j : The present invention uses the bandwidth of virtual machine resource configuration to describe its network performance.

[0060] Step 2: Clustering virtual machine resources

[0061] The GA-FCM algorithm is used to classify virtual machine resources, which are divided into computing resources, storage resources and network resources. The clustering is completed using the FCM algorithm improved by combining genetic algorithm.

[0062] Step 2.1: Use the GA algorithm to obtain the initial clustering centers of computing, storage, and network resources

[0063] Step 2.1.1, Problem Modeling:

[0064] In the calculation, the label number of the virtual machine resource is removed first, and the characteristic vector of the virtual machine resource is recorded as The eigenvectors of the three cluster centers are Encode the three cluster centers into real-number coded genetic gene strings x = [cal c1 ,stor c1 ,bw c1 ,cal c2 ,stor c2 ,bw c2 ,cal c3 ,stor c3 ,bw c3 ], a total of 9 variables. Set up the genetic algorithm fitness function:

[0065]

[0066] Among them, J(U,C) is the objective function of the FCM algorithm:

[0067]

[0068] Among them, u ij represents the membership of the jth, j=1,…,eth virtual machine resource to the ith, i=1,…,cth cluster center. In the present invention, virtual machine resources are divided into three categories, so c=3. i represents the i-th cluster center. m is the weighted index, m≥1. ‖·‖2 represents the vector 2-norm.

[0069] Step 2.1.2, initialize genetic algorithm parameters:

[0070] Set the population size P, the maximum number of generations T, the current number of iterations t = 1, the crossover probability β and the mutation probability η. Calculate all The maximum and minimum values ​​of each dimension of j=1,…,e virtual machine resource vectors constitute the upper and lower limits of the genetic gene, and P is randomly generated within the range of the upper and lower limits. (0) Initial individuals, calculate the fitness function f of the population m .

[0071] Step 2.1.3, iterative evolution:

[0072] In the tth iteration evolution, the previous generation population P (t-1) The individual x with the largest fitness j(max) Unconditionally replicated to the next generation population, the remaining individuals undergo random genetic mutations to form a new population P (t) For population P (t-1) Individual x in j(k) ,

[0073] x j(k) =[cal c1 j(k) ,stor c1 j(k) ,bw c1 j(k) ,cal c2 j(k) ,stor c2 j(k) ,bw c2 j(k) ,cal c3 j(k) ,stor c3 j(k) ,bw c3 j(k) ]k=1,…,P, and j≠j(max), perform genetic mutation and perform the following steps:

[0074] Step 2.1.3.1. Simulate binary crossover

[0075] According to the probability randβ From the population P (t-1) Randomly select two parent individuals x j(i1) 、x j(i2) , using a crossover operator that simulates a single-point binary crossover to generate two offspring individuals x j(igen1) 、x j(igen2) :

[0076]

[0077] where χ is determined by the crossover probability β and the random parameter rand β Decide:

[0078]

[0079] Offspring individual x j(igen1) and x j(igen2) Replace the parent individual and enter the new population P (t′) , individuals without mating directly enter the new population P (t′) .

[0080] Step 2.1.3.2: Polynomial mutation. According to the probability rand η , for the new population P t′ Individual x in j(igen) ′ performs mutation operation to obtain the new individual x j(igen_new) ′:

[0081] x j(igen_new) '=x j(igen) '+δ×(x max '-x min ')

[0082] in,

[0083]

[0084] x max Represents the population P (t′) The individuals in each dimension are the maximum values ​​of the individuals in the max '=max[P (t ' ) ]; x min ' represents the population P (t′) The individuals in each dimension are composed of the minimum values ​​of individuals, x min '=min[P (t ' ) ]. The new individual x after mutation j(igen_new) 'Replace the original individual and enter the new population P (t) , individuals that have not undergone mutations directly enter the new population P (t) , population P (t)That is the set of individuals in the population after the tth inheritance.

[0085] Step 2.1.3.3: Evaluate the corresponding population P under the current number of iterations t (t) The fitness value of all individuals in . Calculate the fitness function:

[0086]

[0087] Step 2.1.3.4, iterative termination condition judgment: judge whether the maximum number of iterations T is reached. If t<T is satisfied, then t=t+1 and return to step 2.1.3.

[0088] Otherwise, the loop ends and returns the individual x with the largest fitness. f_max :

[0089] x f_max =[cal c1 max ,stor c1 max ,bw c1 max ,cal c2 max ,stor c2 max ,bw c2 max ,cal c3 max ,stor c3 max ,bw c3 max ].

[0090] Every 3 dimensions of the individual's 9-dimensional vector are extracted as a vector, for a total of three 3-dimensional vectors, which are the optimal initial clustering centers of the three types of virtual resources searched by the GA algorithm.

[0091] Recorded as have

[0092]

[0093] Step 2.2: Based on the optimal initial cluster center obtained in step 2.1 The FCM algorithm is used to cluster virtual machine resources.

[0094] Step 2.2.1. Set the smoothing parameter m. Set the maximum number of FCM iterations T fcm , the current iteration number t = 1. Set the iteration threshold ε. As the three types of initial clustering centers of the FCM clustering algorithm Iterate and record the cluster center of the tth iteration as v t i , i=1,L,c.

[0095] Step 2.2.2, calculate the membership matrix. In the tth iteration, calculate the virtual machine resources and the i-th, i=1,L,c-class resource cluster center v t-1 i The membership degree u ij :

[0096]

[0097] Among them, d 2 (x,y) represents the Euclidean distance between vector x and vector y.

[0098] Step 2.2.3, Update the cluster center. Update the cluster center vx of the tth iteration i :

[0099]

[0100] Step 2.2.4, v t =[v t 1 ,v t 2 ,v t 3 ], if ||v t+1 -v t ||≤ε or the maximum number of iterations T is reached fcm , the iteration terminates, otherwise, t=t+1, and returns to step 2.2.2.

[0101] Step 2.2.5: After the iteration is completed, the cluster center v of each resource is obtained f 1 , v f 2 , v f 3 , and e virtual machine resources j=1,L,e and the membership matrix u of each type of resource ij . Each resource is classified according to the membership matrix u ij The maximum membership value corresponding to each cluster center is used to divide the resource clusters, that is, for the jth resource Find max{u 1j ,u 2j ,u 3j}, divide the j-th resource into the cluster.

[0102] Step 3: For each independent task M i, establish features: namely M i ={cal_T i ,stor_T i ,bw_T i}. Where cal_T i Represents task M i Computational requirements; stor_T i Indicates storage requirements; bw_T i Indicates bandwidth requirements.

[0103] Step 4: Calculate each task M i The characteristic vectors and the cluster centers v of three types of resources f i Pearson correlation coefficient for (i=1,2,3):

[0104]

[0105] Where E(·) represents the expectation. The cluster center with the largest correlation coefficient is obtained, and the task is divided into the resource pool corresponding to the cluster.

[0106] Step 5: For independent cloud computing tasks M i Perform scheduling and allocation.

[0107] Step 5.1: Set alarm parameters. Set the alarm threshold for computing resource usage ω c , storage resource usage alarm threshold ω s , bandwidth resource usage alarm threshold ω b .

[0108] Step 5.2: Select cloud host resources that can perform task scheduling. i Allocate to the npth class of virtual machine resources. Among the npth class of virtual machine resources, select k capable of task scheduling according to the following requirements np Cloud Host:

[0109] ① Able to meet task M i Computing, storage, and bandwidth requirements;

[0110] ② The cloud host is in a non-alarm state, that is, the computing resource usage is less than ω c , storage resource utilization is less than ω s , bandwidth resource utilization is less than ω b .

[0111] Step 5.3: Calculate the completion time of the cloud computing task. np If the task M i Dispatched to the first cloud host ECS l ,l=1,L,knp Execute on the cloud host ECS l The execution time of the existing task is t l , according to ECS l Resource configuration of the first cloud host ECS l Processing Task M i The time required is Since each virtual machine executes tasks in parallel, the completion time of a single virtual machine is determined by the last completed task. l Execute Task M i When the maximum task completion time is t max(l) for:

[0112]

[0113] Step 5.4: Calculate the virtual machine resource time load. i Go to the first cloud host ECS l ,l=1,L,k np Then the lth cloud host carries task M i Impact of time load on system cluster lod l for:

[0114]

[0115] k np The cloud host carries the task M i The time load is calculated to obtain the time load matrix lod = [lod1, L, lod l ,L,lod np ].

[0116] Step 5.5: Recalculate the ECS of the first cloud host l Carrying Task M i The computing resource utilization, storage resource utilization, and bandwidth resource utilization at that time. For the sake of convenience, we will recalculate the lth cloud host ECS l The computing resource utilization rate is denoted as ω 1(l) , storage resource utilization is denoted as ω 2(l) , bandwidth resource utilization is recorded as ω 3(l) .

[0117] Step 5.6: For independent cloud computing tasks M i After resource scheduling, each resource should be in a non-alarm state, and the time load of the entire platform should be minimal. Based on this, the execution task M is determined. i Virtual Machine:

[0118] Step 5.6.1: Traverse the k nodes selected in step 5.2 that can be scheduled np Cloud host, reserved 1(l) <ω c And ω 2(l) <ω s And ω 3(l) <ω b cloud host.

[0119] Step 5.6.2: For cloud hosts that meet the conditions in step 5.6.1, compare their time loads and assign independent cloud computing tasks M i Assign tasks to the cloud host with the smallest time load.

[0120] Step 6: Data monitoring. Monitor the usage of virtual machine resources and underlying host resources in real time. When an alarm occurs (computing, storage, bandwidth, any resource usage exceeds the alarm threshold set in step 5.1, which means that the current resource is considered to be in an alarm state), adjust the cloud computing task in real time. When adjusting resources, give priority to non-core tasks with low resource utilization, and reallocate the cloud computing task according to step 5. If the alarm state still occurs, it is necessary to schedule the core tasks with high usage frequency during idle time.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the above embodiments, or replace some of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or replacements that can be easily thought of by those skilled in the art within the technical scope disclosed in the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for adaptive scheduling of cloud computing tasks based on machine learning, characterized in that: include: S1, virtual machine resource modeling, based on the computing node component host cluster, multiple virtual machine resources are obtained through virtualization processing; S2, classify virtual machine resources through GA-FCM algorithm, divide virtual machine resources into computing resources, storage resources and network resources, and complete clustering using FCM algorithm improved by genetic algorithm; S3, builds features for each independent cloud computing task; S4, calculate the Pearson correlation coefficient of the feature vector of each cloud computing task and the cluster centers of the three types of resources; S5, scheduling and allocating independent cloud computing tasks; S6, monitor the virtual machine resources and host machine resources, and adjust the cloud computing tasks according to the resource utilization rate.

2. The method for adaptive scheduling of cloud computing tasks based on machine learning according to claim 1, characterized in that: In the step of virtual machine resource modeling, a host cluster consisting of H computing nodes virtualizes e virtual machine resources; a one-dimensional vector Od j represents the jth virtual machine resource, j=1,2,…,e, Od j ={num j ,cal j ,stor j ,bw j }, where num j , num j =1,2,…,c means j virtual machine resources are located on the numth computing server; cal j represents the computing capacity of the jth virtual machine resources; j represents the storage capacity of the jth virtual machine resources; bw j Represents the network performance of j virtual machine resources.

3. The method for adaptive scheduling of cloud computing tasks based on machine learning according to claim 2, characterized in that: In S1, the steps of modeling virtual machine resources include: building computing capacity, storage capacity and network capacity, wherein the network capacity is described by the bandwidth configured by the virtual machine resources.

4. The method for adaptive scheduling of cloud computing tasks based on machine learning according to claim 3 is characterized in that: For the numth, num=1,…,H computing servers, their total computing power cal_num tol for: cal_num tol =ser_n*ser_cor*ser_fre*ser_thr Among them, ser_n is the number of CPUs in the num-th computing server; ser_cor is the number of cores in the num-th computing server CPU; ser_fre is the main frequency of the num-th computing server CPU; ser_thr is the number of threads in the num-th computing server CPU. The num-th computing server can provide n num vcpus, then the computing capacity of the jth virtual machine resource on the numth computing server is cal j for: Among them, n num_j Indicates the number of vCPUs of the j-th virtual machine resources on the num-th computing server.

5. The method for adaptive scheduling of cloud computing tasks based on machine learning according to claim 3, characterized in that: The storage capacity of the j-th virtual machine resource is stor j for: great j =min(large_cd j ,large_sd j ); Among them, stor_cd j The IOPS of the cloud disk mounted to the jth virtual machine resource is related to the cloud disk capacity; stor_sd j The IOPS upper limit of the disk type corresponding to the cloud disk mounted to the j-th virtual machine resource is related to the underlying technology.

6. The method for adaptive scheduling of cloud computing tasks based on machine learning according to claim 1, characterized in that: The GA-FCM algorithm is used to classify virtual machine resources, and the steps of dividing virtual machine resources into computing resources, storage resources and network resources include: problem modeling, initializing parameters of the genetic algorithm, iterative evolution based on the genetic algorithm, simulating binary crossover, performing polynomial mutation, evaluating the fitness values ​​of all individuals in the corresponding population under the current number of iterations, calculating the fitness function, judging the iteration termination condition, and obtaining the optimal initialized cluster center.

7. The method for adaptive scheduling of cloud computing tasks based on machine learning according to claim 6, characterized in that: The steps of completing clustering by using the FCM algorithm improved by combining with the genetic algorithm include setting smoothing parameters, calculating the membership matrix, updating the cluster center, and determining the iteration termination condition. After the stacking is completed, the cluster centers of various types of resources and virtual machine resources are obtained, and each resource is divided into resource clusters according to the maximum membership value corresponding to each cluster center in the membership matrix.

8. The method for adaptive scheduling of cloud computing tasks based on machine learning according to claim 1, characterized in that: In S1, the step of scheduling and allocating independent cloud computing tasks includes setting alarm parameters, determining the alarm threshold of each resource, selecting cloud host resources that can participate in task scheduling, calculating the completion time of each cloud computing task, and calculating the time load of each virtual machine resource, recalculating the computer resource utilization rate, storage resource utilization rate and bandwidth resource utilization rate of the cloud host when carrying cloud computing tasks, and scheduling independent cloud computing tasks.

9. The method for adaptive scheduling of cloud computing tasks based on machine learning according to claim 8, characterized in that: Traverse the cloud host resources that can perform task scheduling, retain the cloud hosts whose various cloud host resources are less than the corresponding alarm threshold, compare the time load, and divide the independent cloud computing tasks into the cloud hosts with the smallest time load for execution.

10. The method for adaptive scheduling of cloud computing tasks based on machine learning according to claim 1, characterized in that: Monitor virtual machine resources and host machine resources, and adjust cloud computing tasks according to resource utilization. When adjusting resources, give priority to non-core tasks with low resource utilization and reallocate the cloud computing tasks. If the alarm state still occurs, it is necessary to schedule the core tasks with high usage frequency during idle time.

Images