Cloud resource scheduling method based on intelligent cooperative control

Abstract

The invention discloses a cloud resource scheduling method based on intelligent cooperative control, and belongs to the technical field of resource scheduling, and the scheduling method comprises the following specific steps: Q1, analyzing a cloud resource use condition and predicting a resource demand trend; q2, classifying available resources according to a prediction result and matching the available resources with tasks; q3, evaluating each scheduling scheme and determining the priority of the scheme; according to the method, idle and waste of resources can be reduced to the greatest extent, the resource utilization rate can be improved, the overall scheduling efficiency of cloud resources can be optimized, efficient operation can be kept in a complex and uncertain environment, a more reliable decision basis is provided for a system, the intelligent level of scheduling decision is improved, and the service life of the system is prolonged. According to the method, the scheduling strategy can be ensured to meet different target requirements, meanwhile, the overall optimality is kept, good robustness is kept, the efficiency and the response speed of the algorithm are improved, and a better scheduling priority can be found in a shorter time.

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling, and particularly to a cloud resource scheduling method based on intelligent collaborative control. Background Art

[0002] As an important supporting technology in today's information society, the efficient management and scheduling of infrastructure resources in cloud computing are the core issues to ensure cloud service quality. With the wide application of cloud computing technology, the efficient scheduling of cloud resources has become a key issue to improve system performance and resource utilization. In a multi-tenant and multi-task cloud environment, the dynamic nature, heterogeneity of resources, and uncertainty of demands make traditional scheduling methods difficult to adapt to complex and changing scenarios. In recent years, with the rapid development of intelligent technologies, intelligent optimization methods such as deep learning, evolutionary algorithms, and Monte Carlo tree search have shown great potential in the field of cloud resource scheduling. These methods provide innovative solutions for the optimization of resource scheduling through collaborative control and intelligent decision-making, and can allocate and adjust resources in real time and accurately in complex environments. However, the current cloud resource scheduling methods still face various challenges, such as the accuracy of resource prediction, the efficient matching of tasks and resources, and the dynamic changes during the scheduling process. Therefore, it is particularly important to invent a cloud resource scheduling method based on intelligent collaborative control.

[0003] After retrieval, Chinese Patent No. CN115640128A discloses a cloud resource scheduling method in a hybrid cloud mode. Although this invention improves the accuracy of resource scheduling, effectively preventing path errors in resource scheduling and increasing the cost of resource scheduling, it cannot minimize the idle and waste of resources, reduce resource utilization, and cannot operate efficiently in complex and uncertain environments; in addition, the existing cloud resource scheduling methods cannot maintain overall optimality while meeting different target requirements, and the efficiency and response speed of the algorithms are low, which is not conducive to finding better scheduling priorities in a shorter time; for this reason, we propose a cloud resource scheduling method based on intelligent collaborative control. Summary of the Invention

[0004] The purpose of the present invention is to solve the defects existing in the prior art, and to propose a cloud resource scheduling method based on intelligent collaborative control.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A cloud resource scheduling method based on intelligent collaborative control, and the specific steps of this scheduling method are as follows:

[0007] Q1: Analyze the usage of cloud resources and predict the trend of resource requirements;

[0008] Q2: Classify the available resources according to the prediction results and match them with the tasks;

[0009] Q3: Evaluate each scheduling scheme and determine the scheme priority;

[0010] Q4: Statistically analyze the execution time and cost range of the scheme and conduct simulation scheduling tests;

[0011] Q5: Real-time execute the scheduling scheme and monitor the usage status of resources;

[0012] Q6: Dynamically adjust the resource allocation strategy and analyze and record the resource usage.

[0013] As a further solution of the present invention, the specific steps of predicting the resource demand trend in Q1 are as follows:

[0014] S1.1: Collect historical cloud resource usage data, remove outliers and missing values from each group of data, determine the usage amounts of various resources based on the historical cloud resource usage data, and process the usage amounts of various resources to a unified numerical range through a normalization method;

[0015] S1.2: Divide the historical cloud resource usage data into multiple groups of sliding windows according to a preset time interval, use the resource usage records in each group of windows as input features, and then divide the generated input features into a training set and a test set;

[0016] S1.3: Input the input features in the training set into a group of pre-constructed LSTM models, and through the forward propagation algorithm, generate prediction results corresponding to each input feature through the forget gate, input gate, and output gate of each recursive layer in the LSTM model, and measure the gap value between the predicted value and the true value through the mean square error loss function;

[0017] S1.4: Back-propagate the gap value from the output layer of the LSTM model through the backpropagation algorithm, calculate the gradient of the gap value for each layer, and then update each network parameter based on the gradient information of each layer through the Adam optimizer. After training is completed, use the test set to verify and evaluate the model, judge the prediction accuracy of the model, and repeat the iterative optimization of the LSTM model until the prediction accuracy reaches the preset threshold and then stop;

[0018] S1.5: Input the current cloud resource usage data into the trained LSTM model, and then the LSTM model obtains the resource demand at the next moment through forward propagation, and real-time monitors the deviation between the actual resource usage and the predicted value. If the deviation value is within the preset range, use the new data to retrain and adjust the LSTM model.

[0019] As a further solution of the present invention, the specific steps of classifying the available resources in Q2 are as follows:

[0020] S2.1: Extract the characteristic data of available resources and tasks, represent each resource and task as a set of vectors respectively, and the dimensions of the two sets of vectors are the number of resource characteristics and the number of characteristics required by the tasks respectively. According to the characteristics of resources and tasks, construct a set of resource-task adaptation matrices;

[0021] S2.2: Calculate the Euclidean distance of each resource and task characteristic in the matrix to obtain the adaptation degree between each group of resources and tasks, construct resource classification matrices at different levels, and through matrix hierarchical screening and optimization, select the resource groups matching each task respectively;

[0022] S2.3: Continuously adjust the allocation of resources and tasks in an iterative manner and generate multiple groups of allocation plans. Then calculate the total cost of each allocation plan, output the generated resource and task allocation plans, and the total cost of each allocation plan.

[0023] As a further solution of the present invention, the resource characteristic data in S2.1 specifically includes the performance indicators of CPU, memory, bandwidth, and storage; the task characteristic data specifically includes the indicators of computational complexity, data transmission requirements, and latency sensitivity.

[0024] As a further solution of the present invention, the specific steps for determining the scheme priority in Q3 are as follows:

[0025] S3.1: Collect each group of resource scheduling schemes generated by iteration, integrate the collected groups of resource scheduling schemes into an initial population, and represent each group of resource scheduling schemes in the form of population individuals. Then calculate the fitness value of each resource scheduling scheme according to the scheme execution time, resource utilization rate, and total cost;

[0026] S3.2: According to the fitness value of each scheme, select multiple groups of individuals from the initial population by roulette wheel selection to enter the next-generation population. Then randomly select individuals in pairs, and by selecting a crossover point in each individual, exchange the gene sequences after the crossover point to generate two new individuals;

[0027] S3.3: Randomly select one group or multiple groups of individuals, and randomly select a task in their scheduling schemes, allocate it to different resources, and generate a new population of resource scheduling schemes through crossover and mutation operations. Then combine the new population with the high-quality individuals in the old population to form the next-generation population;

[0028] S3.4: Repeat the iterative update of the generated population until the preset maximum number of iterations is reached, output the optimal resource scheduling scheme, and determine the execution priority of each task according to this scheme.

[0029] As a further solution of the present invention, the specific steps of the statistical analysis described in Q4 are as follows:

[0030] S4.1: Analyze the output optimal resource scheduling scheme, obtain the initial state at the start of the resource scheduling scheme, use this initial state as the root node, and use the state after the tasks in the resource scheduling scheme are scheduled and assigned to the corresponding resources as the child nodes. Construct a corresponding exploration tree based on the generated root node and child nodes;

[0031] S4.2: Calculate the UCB selection value of each node. Starting from the root node, select the node with the largest UCB selection value and conduct the next-level search. Search layer by layer downward until reaching an incompletely expanded child node. Collect the state corresponding to the current node and the unscheduled tasks in this state;

[0032] S4.3: Collect the state information after the unscheduled tasks are assigned to the corresponding resources and add it as a new child node to the exploration tree. Starting from the newly expanded node, randomly select the remaining tasks and assign them to the available resources in turn until all tasks are scheduled, and calculate the total execution time and cost under this simulation path;

[0033] S4.4: After the simulation is completed, starting from the newly expanded node, backtrack upward with the simulation results and return them to each node on the same path in the search tree, and update the statistical information of each node;

[0034] S4.5: Repeat the processes of selection, expansion, simulation, and backtracking until reaching the preset number of simulation times or when the change value in consecutive simulations converges within the preset range, terminate the exploration. Starting from the root node, select the child node with the highest average reward value and output the task plan execution time and cost range of the corresponding scheme.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] 1. The cloud resource scheduling method based on intelligent collaborative control analyzes the output optimal resource scheduling scheme, takes the initial state at the start of the resource scheduling scheme as the root node, and takes the state after the tasks in the resource scheduling scheme are scheduled and assigned to the corresponding resources as the child nodes. An exploration tree is constructed based on the generated root node and child nodes. The UCB selection value of each node is calculated. Starting from the root node, the node with the largest UCB selection value is selected and searched layer by layer downward until an unexpanded child node is reached. Then, this node is expanded. Starting from the newly expanded node, the remaining tasks are randomly selected and assigned to the available resources in turn until all tasks are scheduled. The total execution time and cost under this simulation path are calculated. After the simulation is completed, the simulation results start from the newly expanded node and are traced back upward to return to each node on the same path in the search tree, and the statistical information of each node is updated. The selection, expansion, simulation, and backtracking are repeated until the preset number of simulations is reached or the change value in consecutive simulations converges to the preset range, at which point the exploration is terminated. Starting from the root node, the child node with the highest average reward value is selected, and the task plan execution time and cost range of the corresponding scheme are output. This can minimize the idle and waste of resources, help improve resource utilization, optimize the overall scheduling efficiency of cloud resources, be able to operate efficiently in a complex and uncertain environment, provide a more reliable decision-making basis for the system, and enhance the intelligent level of scheduling decisions.

[0037] 2. The cloud resource scheduling method based on intelligent collaborative control collects each group of resource scheduling schemes generated iteratively, integrates the collected groups of resource scheduling schemes into an initial population, and represents each group of resource scheduling schemes in the form of population individuals. Then, the fitness value of each resource scheduling scheme is calculated based on the scheme execution time, resource utilization rate, and total cost. According to the fitness value of each scheme, multiple groups of individuals are selected from the initial population by roulette wheel selection to enter the next generation population. Then, two individuals are randomly selected as a pair, and by selecting a crossover point in each individual and exchanging the gene sequences after this crossover point, two new individuals are generated. One or more groups of individuals are randomly selected, and a task is randomly selected in their scheduling schemes and assigned to different resources. After crossover and mutation operations, a new population of resource scheduling schemes is generated, and the new population is combined with the high-quality individuals in the old population to form the next generation population. The generated population is iteratively updated repeatedly until the preset maximum number of iterations is reached, and the optimal resource scheduling scheme is output. At the same time, the execution priority of each task is determined according to this scheme, which can ensure that the scheduling strategy maintains the overall optimality while meeting different target requirements, has good robustness, improves the efficiency and response speed of the algorithm, and helps to find a better scheduling priority in a shorter time. Description of the Drawings

[0038] The accompanying drawings are used to provide a further understanding of the present invention and form a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention.

[0039] Figure 1 It is a flowchart of the cloud resource scheduling method based on intelligent collaborative control proposed by the present invention. Detailed implementation manners

[0040] Embodiment 1

[0041] Refer to Figure 1 , for the cloud resource scheduling method based on intelligent collaborative control, the specific steps of the scheduling method are as follows:

[0042] Analyze the cloud resource usage situation and predict the resource demand trend.

[0043] Specifically, collect historical cloud resource usage data, remove the outliers and missing values in each group of data, determine the usage amounts of various resources based on the historical cloud resource usage data, process the usage amounts of various resources to a unified numerical range through a normalization method, divide the historical cloud resource usage data into multiple groups of sliding windows according to a preset time interval, use the resource usage records in each group of windows as input features, then divide the generated input features into a training set and a test set, input the input features in the training set into a group of pre-constructed LSTM models, and through the forward propagation algorithm, generate prediction results corresponding to each input feature through the forget gate, input gate, and output gate of each recursive layer in the LSTM model, and measure the gap value between the predicted value and the true value through the mean square error loss function. Reverse-transmit the gap value from the output layer of the LSTM model through the backpropagation algorithm, and calculate the gradient of the gap value for each layer. Then, based on the gradient information of each layer, update each network parameter through the Adam optimizer. After the training is completed, use the test set to verify and evaluate the model, judge the prediction accuracy of the model, repeat the iterative optimization of the LSTM model until the prediction accuracy reaches a preset threshold and then stop. Input the current cloud resource usage data into the trained LSTM model, and then the LSTM model obtains the resource demand for the next moment through forward propagation, and real-time monitors the deviation between the actual resource usage and the predicted value. If the deviation value is within the preset range, re-train and adjust the LSTM model with new data.

[0044] Classify the available resources according to the prediction results and match them with the tasks.

[0045] Specifically, extract the characteristic data of available resources and tasks, represent each resource and task as a group of vectors respectively, and the dimensions of the two groups of vectors are the number of resource characteristics and the number of characteristics required by the tasks respectively. According to the characteristics of resources and tasks, construct a group of resource-task adaptation matrices, calculate the Euclidean distances of each resource and task characteristic in the matrix to obtain the adaptation degrees between each group of resources and tasks, construct resource classification matrices at different levels, and through matrix hierarchical screening and optimization, select the resource groups matching each task respectively. Continuously adjust the allocation of resources and tasks in an iterative manner and generate multiple groups of allocation schemes. Then calculate the total cost of each allocation scheme, output the generated allocation scheme of resources and tasks, and the total cost of each allocation scheme.

[0046] In this embodiment, the resource characteristic data specifically includes performance indicators such as CPU, memory, bandwidth, and storage; the task characteristic data specifically includes indicators such as computational complexity, data transmission requirements, and latency sensitivity.

[0047] Evaluate each scheduling scheme and determine the scheme priority.

[0048] Specifically, collect each group of resource scheduling schemes generated by iteration, integrate the collected groups of resource scheduling schemes into an initial population, and represent each group of resource scheduling schemes in the form of population individuals. Then calculate the fitness value of each resource scheduling scheme according to the scheme execution time, resource utilization rate, and total cost. According to the fitness value of each scheme, select multiple groups of individuals from the initial population to enter the next generation population by roulette wheel selection. Then randomly select individuals in pairs, and by selecting a crossover point in each individual, exchange the gene sequences after the crossover point to generate two new individuals. Randomly select one or more groups of individuals, and randomly select a task in their scheduling schemes and allocate it to different resources. After crossover and mutation operations, generate a new population of resource scheduling schemes, and combine the high-quality individuals in the new population and the old population to form the next generation population. Repeat the iterative update of the generated population until the preset maximum number of iterations is reached, and output the optimal resource scheduling scheme. At the same time, determine the execution priority of each task according to this scheme.

[0049] Embodiment 2

[0050] Refer to Figure 1 , a cloud resource scheduling method based on intelligent collaborative control, the specific steps of the scheduling method are as follows:

[0051] Statistically analyze the scheme execution time and cost interval and conduct simulation scheduling tests.

[0052] Specifically, analyze the optimal resource scheduling scheme of the output, obtain the initial state at the start of the resource scheduling scheme, and use this initial state as the root node. Use the state after the tasks in the resource scheduling scheme are scheduled and assigned to the corresponding resources as the child nodes. Construct the corresponding exploration tree based on the generated root node and child nodes, calculate the UCB selection value of each node. Starting from the root node, select the node with the largest UCB selection value and conduct the next-level search, searching layer by layer downwards until reaching an incompletely expanded child node. At this time, collect the state corresponding to the current node and the unscheduled tasks in this state, collect the state information after the unscheduled tasks are assigned to the corresponding resources, and add it as a new child node to the exploration tree. Starting from the newly expanded node, randomly select the remaining tasks and assign them to the available resources in sequence until all tasks are scheduled, and calculate the total execution time and cost under this simulated path. After the simulation is completed, trace back upwards from the newly expanded node of the simulation result and return it to each node on the same path in the search tree, and update the statistical information of each node. Repeat the processes of selection, expansion, simulation, and backtracking until reaching the preset number of simulations or the change value in consecutive simulations converges within the preset range, then terminate the exploration. Starting from the root node, select the child node with the highest average reward value and output the task plan execution time and cost interval of the corresponding scheme. Execute the scheduling scheme in real time and monitor the usage status of resources. Dynamically adjust the resource allocation strategy and analyze and record the resource usage situation.

Claims

1. A cloud resource scheduling method based on intelligent collaborative control, characterized in that: The specific steps of the scheduling method are as follows: Q1: Analyze cloud resource usage and predict resource demand trends; Q2: Classify available resources and match them to tasks based on the prediction results; Q3: Evaluate each scheduling plan and determine the priority of the plan; Q4: Statistical analysis of solution execution time and cost range and simulation scheduling test; Q5: Execute scheduling plans and monitor resource usage in real time; Q6: Dynamically adjust resource allocation strategies and analyze and record resource usage.

2. The cloud resource scheduling method based on intelligent collaborative control according to claim 1 is characterized in that: The specific steps for predicting resource demand trends described in Q1 are as follows: S1.1: Collect historical cloud resource usage data, remove outliers and missing values ​​in each group of data, determine the usage of each type of resource based on the historical cloud resource usage data, and normalize the usage of each type of resource to a unified numerical range; S1.2: Divide the historical cloud resource usage data into multiple sliding windows according to the preset time interval, and use the resource usage records in each window as input features, and then divide the generated input features into a training set and a test set; S1.3: Input the input features in the training set into a set of pre-built LSTM models, and generate the prediction results corresponding to each input feature through the forward propagation algorithm, the forget gate, input gate and output gate of each recursive layer in the LSTM model, and measure the difference between the predicted value and the true value through the mean square error loss function; S1.4: The gap value is transmitted backward from the output layer of the LSTM model through the back-propagation algorithm, and the gradient of the gap value for each layer is calculated. Then, based on the gradient information of each layer, the network parameters are updated through the Adam optimizer. After the training is completed, the model is verified and evaluated using the test set to determine the prediction accuracy of the model. The LSTM model is repeatedly iterated and optimized until the prediction accuracy reaches the preset threshold. S1.5: Input the current cloud resource usage data into the trained LSTM model. The LSTM model then obtains the resource demand at the next moment through forward propagation, and monitors the deviation between the actual resource usage and the predicted value in real time. If the deviation value is within the preset range, the LSTM model is retrained and adjusted using new data.

3. The cloud resource scheduling method based on intelligent collaborative control according to claim 2 is characterized in that: The specific steps for classifying available resources as described in Q2 are as follows: S2.1: Extract the feature data of available resources and tasks, represent each resource and task as a set of vectors, and the dimensions of the two sets of vectors are the number of resource features and the number of features required by the task, respectively. According to the characteristics of resources and tasks, construct a set of resource-task adaptation matrices; S2.2: Calculate the Euclidean distance between each resource and task feature in the matrix to obtain the degree of fit between each group of resources and tasks, construct resource classification matrices at different levels, and select the resource groups that match each task through matrix hierarchical screening and optimization; S2.3: Continuously adjust the allocation of resources and tasks in an iterative manner, and generate multiple sets of allocation plans. Then calculate the total cost of each allocation plan, output the generated resource and task allocation plan, and the total cost of each allocation plan.

4. The cloud resource scheduling method based on intelligent collaborative control according to claim 3 is characterized in that: The resource characteristic data described in S2.1 specifically include CPU, memory, bandwidth and storage performance indicators; the task characteristic data specifically include computational complexity, data transmission requirements and delay sensitivity indicators.

5. The cloud resource scheduling method based on intelligent collaborative control according to claim 3 is characterized in that: The specific steps for determining the priority of the solutions described in Q3 are as follows: S3.1: Collect the iteratively generated resource scheduling plans, integrate the collected resource scheduling plans into an initial population, and represent each resource scheduling plan in the form of a population individual. Then, calculate the fitness value of each resource scheduling plan based on the plan execution time, resource utilization rate and total cost; S3.2: According to the fitness value of each scheme, multiple groups of individuals are selected from the initial population by roulette wheel selection to enter the next generation population, and then two groups of individuals are randomly selected, and a crossover point is selected in each individual, and the gene sequence after the crossover point is exchanged to generate two new groups of individuals; S3.3: Randomly select one or more groups of individuals, and randomly select a task in their scheduling plan, assign it to different resources, generate a new resource scheduling plan population through crossover and mutation operations, and combine the new population with the high-quality individuals in the old population to form the next generation population; S3.4: Repeat the iterative update of the generated population until the preset maximum number of iterations is reached, and output the optimal resource scheduling plan, and determine the execution priority of each task based on the plan.

6. The cloud resource scheduling method based on intelligent collaborative control according to claim 5 is characterized in that: The specific steps of the statistical analysis described in Q4 are as follows: S4.1: Analyze the output optimal resource scheduling solution, obtain the initial state of the resource scheduling solution at the beginning, and use the initial state as the root node, and the state after the tasks in the resource scheduling solution are scheduled and assigned to the corresponding resources as the child node, and construct a corresponding exploration tree according to the generated root node and child nodes; S4.2: Calculate the UCB selection value of each node, start from the root node, select the node with the largest UCB selection value, and perform the next level search, searching down layer by layer until reaching the child node that is not fully expanded, collect the state corresponding to the current node, and the unscheduled tasks in this state; S4.3: Collect the status information of the unscheduled tasks after they are assigned to the corresponding resources, and add them as new child nodes to the exploration tree. Starting from the expanded new node, randomly select the remaining tasks and assign them to the available resources in turn until all tasks are scheduled, and calculate the total execution time and cost under the simulation path; S4.4: After the simulation is completed, the simulation results are traced back from the newly expanded node to each node on the same path in the search tree, and the statistical information of each node is updated; S4.5: Repeat the selection, expansion, simulation and backtracking until the preset number of simulations is reached or the change value in the continuous simulation converges to the preset range, then terminate the exploration, start from the root node, select the child node with the highest average reward value, and output the task plan execution time and cost range of the corresponding plan.

Images