Cloud computing task scheduling method and system based on two-stage adaptive search

Abstract

The invention discloses a cloud computing task scheduling algorithm and system based on two-stage adaptive search. The cloud computing task scheduling algorithm comprises a task receiving module, a virtual machine management module, a scheduling model building module, a two-stage adaptive scheduling algorithm module and a result output module. The scheduling algorithm module completes population aggregation through a preference perception distance strategy, and when the population centroid variance reaches a threshold value, second-stage search is carried out by adopting a preference region classification strategy; meanwhile, a double-layer coding mode and a task balance mapping strategy are combined to generate a filial generation population, and multi-target collaborative optimization is achieved. The method is superior to an existing algorithm in indexes such as completion time, lease cost, energy consumption and load balancing index under the medium and large-scale cloud environment, interests of multiple parties can be effectively balanced, and a new scheme is provided for task scheduling in the complex cloud environment.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and specifically to a cloud computing task scheduling method and system based on two-stage adaptive search. Background Technology

[0002] Cloud computing integrates multiple technologies such as parallel computing and distributed computing, leveraging the internet and virtual machine technology to provide elastic computing resources, large-scale data storage, and efficient data processing services. It is widely used in scenarios such as deep learning model training and large-scale intelligent inference. As the cloud computing user base continues to expand, the task scheduling pressure on cloud data centers is increasing, while the conflict of interest between users and cloud service providers is becoming more and more apparent.

[0003] Users are more concerned with task completion time and rental costs, while cloud service providers focus on load balancing, resource utilization, and energy consumption control. Most existing scheduling algorithms are designed from a single perspective, making it difficult to consider the interests of all parties. Traditional heuristic algorithms, based on static task and resource information, cannot adapt to the dynamic changes in the cloud computing environment; while multi-objective optimization algorithms can obtain globally non-dominated solutions, the selection of solutions lacks specificity, making it difficult to achieve a "win-win" situation for both users and cloud service providers. Therefore, there is an urgent need for an efficient task scheduling method and system that can balance the interests of all parties and adapt to cloud environments of different scales. Summary of the Invention

[0004] This application provides a cloud computing task scheduling method and system based on two-stage adaptive search, aiming to solve the problems of increased conflict of interest between users and cloud service providers and difficulty in balancing multiple objectives in existing cloud computing task scheduling.

[0005] According to the first aspect, one embodiment provides a cloud computing task scheduling system based on two-stage adaptive search, including a task receiving module, a virtual machine management module, a scheduling model construction module, a two-stage adaptive scheduling algorithm module, and a result output module; The task receiving module is used to receive cloud tasks submitted by users, collect and store the attribute information of each task, including task byte length, input size and output size. The virtual machine management module is used to manage the virtual machine cluster in the cloud platform, record the configuration attributes of each virtual machine, including the number of CPUs, CPU processing power, bandwidth, memory and unit time rental cost, and monitor the running status of the virtual machine in real time. The scheduling model construction module is used to establish a multi-objective scheduling model based on the preferences of users and cloud service providers. The optimization objectives of the multi-objective scheduling model include total task completion time, total rental cost, total energy consumption, and server load balancing index. The two-stage adaptive scheduling algorithm module is used to call a preset two-stage adaptive search scheduling algorithm to solve the multi-objective scheduling model and output the optimal task scheduling scheme. The result output module is used to distribute the optimal task scheduling scheme to the virtual machine cluster, control the virtual machines to execute the corresponding tasks, and provide feedback on the task execution status to the user.

[0006] In some embodiments, in the scheduling model construction module, the total task completion time Makespan is determined by the maximum task completion time of all virtual machines, and the calculation formula is as follows: ; Among them, VT j The task completion time of the j-th virtual machine is determined by the sum of the completion times of all tasks allocated to that virtual machine; the total rental cost (Cost) is the sum of the transmission cost and computing cost of all virtual machines; the total energy consumption (EnergyCost) is the sum of the energy consumption of all virtual machines in active and idle states; the load balancing index (Load) is calculated by the variance of the task execution time of each virtual machine and the average execution time of all virtual machines.

[0007] According to the second aspect, one embodiment provides a cloud computing task scheduling method based on two-stage adaptive search, comprising the following steps: Step 1, Initialization settings: An initial population P is generated using a two-layer encoding method. The first layer of the two-layer encoding represents the order in which tasks are added to the scheduling center, and the second layer represents the mapping relationship between tasks and virtual machines. At the same time, a set of uniformly distributed reference vectors is generated. Step 2, Offspring Population Generation: The initial population P is used to generate the offspring population O by combining crossover mutation and task balancing mapping strategies; the crossover mutation adopts the index-based crossover method, simulated binary crossover, and polynomial mutation; the task balancing mapping strategy balances the load by sorting virtual machines according to task completion time and exchanging tasks on different virtual machines. Step 3, Two-stage adaptive search: Merge the parent population P and the offspring population O to obtain a new population P', and perform a two-stage adaptive environment selection based on the new population P' to obtain the next generation parent population CP; Step 4, Termination Judgment: If the maximum number of iterations has been reached, output the non-dominated solution as the optimal task scheduling scheme; otherwise, return to step 2 to continue iterating.

[0008] In some embodiments, the specific process of the two-stage adaptive environment selection in step 3 includes: Step 3.1: Calculate the angle between an individual in the population CP and the reference vector, associate the individual with the sub-region corresponding to the reference vector with the smallest angle, and perform non-dominated sorting on the population, retaining the non-dominated sorting level of the individual. Step 3.2: Calculate the adaptive transformation conditions. These conditions are determined by the population centroid variance (Variance) and the number of iterations. The population centroid is calculated from the average perceived distance of individuals in the population, and the variance (Variance) measures the degree of population aggregation. The comprehensive stage transformation strategy proceeds to the first stage if the following equation is satisfied; otherwise, it proceeds to the second stage: ; Where g is the current iteration number, G max η is the maximum number of iterations, η is the two-stage transformation parameter, m is the number of objective functions, and ∂ is the transformation threshold; Step 3.3 First stage search: In each sub-region, retain the individual with the smallest number of non-dominated layers. If there are multiple individuals, select the individual with the smallest penalty-based boundary intersection PBI value. After pre-screening, retain the individual with the smallest preference perception distance to form the population. Step 3.4 Second stage search: Divide the target space into several spatial levels, pre-screen individuals and classify them into preferred regions based on reference points, prioritize retaining individuals in regions that balance the interests of users and cloud service providers, and supplement the population to the set size.

[0009] In some embodiments, the preference-perceived distance ED(x) is calculated using the following formula: ,in Let Euclidean distance be the distance between the individual and the reference point. This represents the Euclidean distance between the individual and the current spatial origin.

[0010] In some embodiments, the preference region classification in step 3.4 divides the target space into four regions, which correspond to the region that balances the interests of both parties, the region that protects the interests of users, the region that meets the needs of cloud service providers, and the region that cannot meet the needs of either party. The region to which an individual belongs is determined based on the individual's average spatial level gp, the user's average spatial level gu, and the service provider's average spatial level gc.

[0011] In some embodiments, the task load balancing mapping strategy first sorts the virtual machines by their task completion time in ascending order and records the virtual machine index number K. i Determine the minimum task value z for all virtual machines, then swap some tasks on virtual machines with fewer tasks with corresponding tasks on virtual machines with more tasks, until all virtual machines have completed the task balancing adjustment.

[0012] In some embodiments, the two-stage transformation parameter η is set to 0.7, the transformation threshold ∂ is set to 0.025, the population size is set to 100, and the maximum number of iterations is set to 100.

[0013] Based on the above embodiments, the present invention has the following beneficial effects compared with the prior art: The multi-objective collaborative optimization yields significant results: the constructed scheduling model simultaneously considers four objectives—total task completion time, total lease cost, total energy consumption, and load balancing index—overcoming the limitations of single-objective algorithms. In medium-scale cloud environments, completion time is reduced by approximately 29% compared to NSGA-Ⅲ, and lease cost is reduced by approximately 37.5% compared to RVEA; in large-scale cloud environments, energy consumption is reduced by approximately 28.2% compared to GrEA, and load balancing index is reduced by approximately 34.8% compared to MCT-PSO.

[0014] Strong adaptability: The two-stage adaptive search strategy can dynamically adjust the search method according to the population evolution state, adapting to different scale cloud environments. Whether it is a small-scale scenario with 80 tasks or a large-scale scenario with 1000 tasks, it can quickly converge and obtain high-quality solutions.

[0015] High practicality: The dual-layer encoding method and task balancing mapping strategy adopted are in line with the actual needs of cloud computing task scheduling. The attribute settings of virtual machines and tasks conform to the configuration of real cloud environments. The scheduling scheme output by the algorithm can be directly applied to actual cloud platforms, effectively improving resource utilization efficiency and achieving a win-win situation for users and cloud service providers. Attached Figure Description

[0016] Figure 1 This is a flowchart of the cloud computing task scheduling method of the present invention. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0018] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0019] This application provides a cloud computing task scheduling system based on two-stage adaptive search, including a task receiving module, a virtual machine management module, a scheduling model construction module, a two-stage adaptive scheduling algorithm module, and a result output module; The task receiving module is used to receive cloud tasks submitted by users, collect and store the attribute information of each task, including task byte length, input size and output size. The virtual machine management module is used to manage the virtual machine cluster in the cloud platform, record the configuration attributes of each virtual machine, including the number of CPUs, CPU processing power, bandwidth, memory and unit time rental cost, and monitor the running status of the virtual machine in real time. The scheduling model construction module is used to establish a multi-objective scheduling model based on the preferences of users and cloud service providers. The optimization objectives of the multi-objective scheduling model include total task completion time, total rental cost, total energy consumption, and server load balancing index. The two-stage adaptive scheduling algorithm module is used to call a preset two-stage adaptive search scheduling algorithm to solve the multi-objective scheduling model and output the optimal task scheduling scheme. The result output module is used to distribute the optimal task scheduling scheme to the virtual machine cluster, control the virtual machines to execute the corresponding tasks, and provide feedback on the task execution status to the user.

[0020] In some embodiments, in the scheduling model construction module, the total task completion time Makespan is determined by the maximum task completion time of all virtual machines, and the calculation formula is as follows: ; Among them, VT j The task completion time of the j-th virtual machine is determined by the sum of the completion times of all tasks allocated to that virtual machine; the total rental cost (Cost) is the sum of the transmission cost and computing cost of all virtual machines; the total energy consumption (EnergyCost) is the sum of the energy consumption of all virtual machines in active and idle states; the load balancing index (Load) is calculated by the variance of the task execution time of each virtual machine and the average execution time of all virtual machines.

[0021] refer to Figure 1This application provides a cloud computing task scheduling method based on two-stage adaptive search, including the following steps: Step 1, Initialization settings: An initial population P is generated using a two-layer encoding method. The first layer of the two-layer encoding represents the order in which tasks are added to the scheduling center, and the second layer represents the mapping relationship between tasks and virtual machines. At the same time, a set of uniformly distributed reference vectors is generated. Step 2, Offspring Population Generation: The initial population P is used to generate the offspring population O by combining crossover mutation and task balancing mapping strategies; the crossover mutation adopts the index-based crossover method, simulated binary crossover, and polynomial mutation; the task balancing mapping strategy balances the load by sorting virtual machines according to task completion time and exchanging tasks on different virtual machines. Step 3, Two-stage adaptive search: Merge the parent population P and the offspring population O to obtain a new population P', and perform a two-stage adaptive environment selection based on the new population P' to obtain the next generation parent population CP; Step 4, Termination Judgment: If the maximum number of iterations has been reached, output the non-dominated solution as the optimal task scheduling scheme; otherwise, return to step 2 to continue iterating.

[0022] In some embodiments, the specific process of the two-stage adaptive environment selection in step 3 includes: Step 3.1: Calculate the angle between an individual in the population CP and the reference vector, associate the individual with the sub-region corresponding to the reference vector with the smallest angle, and perform non-dominated sorting on the population, retaining the non-dominated sorting level of the individual. Step 3.2: Calculate the adaptive transformation conditions. These conditions are determined by the population centroid variance (Variance) and the number of iterations. The population centroid is calculated from the average perceived distance of individuals in the population, and the variance (Variance) measures the degree of population aggregation. The comprehensive stage transformation strategy proceeds to the first stage if the following equation is satisfied; otherwise, it proceeds to the second stage: ; Where g is the current iteration number, G max η is the maximum number of iterations, η is the two-stage transformation parameter, m is the number of objective functions, and ∂ is the transformation threshold; Step 3.3 First stage search: In each sub-region, retain the individual with the smallest number of non-dominated layers. If there are multiple individuals, select the individual with the smallest penalty-based boundary intersection PBI value. After pre-screening, retain the individual with the smallest preference perception distance to form the population. Step 3.4 Second stage search: Divide the target space into several spatial levels, pre-screen individuals and classify them into preferred regions based on reference points, prioritize retaining individuals in regions that balance the interests of users and cloud service providers, and supplement the population to the set size.

[0023] In some embodiments, the preference-perceived distance ED(x) is calculated using the following formula: ,in Let Euclidean distance be the distance between the individual and the reference point. This represents the Euclidean distance between the individual and the current spatial origin.

[0024] In some embodiments, the preference region classification in step 3.4 divides the target space into four regions, which correspond to the region that balances the interests of both parties, the region that protects the interests of users, the region that meets the needs of cloud service providers, and the region that cannot meet the needs of either party. The region to which an individual belongs is determined based on the individual's average spatial level gp, the user's average spatial level gu, and the service provider's average spatial level gc.

[0025] In some embodiments, the task load balancing mapping strategy first sorts the virtual machines by their task completion time in ascending order and records the virtual machine index number K. i Determine the minimum task value z for all virtual machines, then swap some tasks on virtual machines with fewer tasks with corresponding tasks on virtual machines with more tasks, until all virtual machines have completed the task balancing adjustment.

[0026] In some embodiments, the two-stage transformation parameter η is set to 0.7, the transformation threshold ∂ is set to 0.025, the population size is set to 100, and the maximum number of iterations is set to 100.

[0027] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0028] The present invention will be further described in detail below with reference to specific embodiments.

[0029] (1) System deployment The cloud computing task scheduling system of this invention was deployed on a Windows 11 operating system. The hardware configuration consisted of a 13th-generation Intel® Core™ i9-13900HX CPU and 32GB of Samsung DDR5 5600MHz memory. The experimental environment was built based on the CloudSim cloud computing simulation platform. In the virtual machine cluster, virtual machine attributes were configured from low to high according to preset steps, with the number of CPUs ranging from [1,50] and the CPU processing power ranging from [300,1800] to meet the computing needs of different tasks. Task attributes were dynamically adjusted according to the scale, with the number of tasks covering three gradients: [80, 300, 1000].

[0030] (2) Algorithm execution steps In the above deployment environment, a cloud computing task scheduling method based on two-stage adaptive search is executed: The first step involves the task receiving module receiving 300 cloud tasks and recording the byte length, input size, and output size of each task; the virtual machine management module synchronously updates the real-time status of 15 virtual machines. The second step involves the scheduling model construction module calculating the total task completion time, total rental cost, total energy consumption, and load balancing index according to the formula, and establishing a multi-objective scheduling model. The third step involves initializing the population size to 100 using the two-stage adaptive scheduling algorithm module, generating a uniform reference vector, and obtaining the initial population through double-layer encoding. The offspring population is then generated using methods such as OBX crossover and simulated binary crossover, combined with a task balancing mapping strategy. The fourth step is to calculate the centroid variance of the population. When the transformation condition is met, the search phase is switched. After multiple iterations, the optimal scheduling scheme is output. Fifth, the results output module distributes the solution to the virtual machine cluster, and the virtual machines begin to execute according to the assigned tasks, while providing real-time feedback on the execution progress to the user.

[0031] (3) Effect verification Comparative experiments have verified that the algorithm in this embodiment has a completion time of 10496, a rental cost of 5862, an energy consumption of 96373, and a load index of 3007. All indicators are superior to those of the comparative algorithms such as NSGA-Ⅱ, RVEA, and FCFS, which fully demonstrates the effectiveness and superiority of the present invention.

[0032] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A cloud computing task scheduling system based on two-stage adaptive search, characterized in that, It includes a task receiving module, a virtual machine management module, a scheduling model construction module, a two-stage adaptive scheduling algorithm module, and a result output module; The task receiving module is used to receive cloud tasks submitted by users, collect and store the attribute information of each task, including task byte length, input size and output size. The virtual machine management module is used to manage the virtual machine cluster in the cloud platform, record the configuration attributes of each virtual machine, including the number of CPUs, CPU processing power, bandwidth, memory and unit time rental cost, and monitor the running status of the virtual machine in real time. The scheduling model construction module is used to establish a multi-objective scheduling model based on the preferences of users and cloud service providers. The optimization objectives of the multi-objective scheduling model include total task completion time, total rental cost, total energy consumption, and server load balancing index. The two-stage adaptive scheduling algorithm module is used to call a preset two-stage adaptive search scheduling algorithm to solve the multi-objective scheduling model and output the optimal task scheduling scheme. The result output module is used to distribute the optimal task scheduling scheme to the virtual machine cluster, control the virtual machines to execute the corresponding tasks, and provide feedback on the task execution status to the user.

2. The system according to claim 1, characterized in that, In the scheduling model construction module, the total task completion time Makespan is determined by the maximum task completion time of all virtual machines, and the calculation formula is as follows: ； Among them, VT j The task completion time of the j-th virtual machine is determined by the sum of the completion times of all tasks allocated to that virtual machine; the total rental cost (Cost) is the sum of the transmission cost and computing cost of all virtual machines; the total energy consumption (Energy Cost) is the sum of the energy consumption of all virtual machines in active and idle states; the load balancing index (Load) is calculated by the variance of the task execution time of each virtual machine and the average execution time of all virtual machines.

3. A cloud computing task scheduling method based on two-stage adaptive search, applied to the cloud computing task scheduling system described in any one of claims 1-2, characterized in that, Includes the following steps: Step 1, Initialization settings: An initial population P is generated using a two-layer encoding method. The first layer of the two-layer encoding represents the order in which tasks are added to the scheduling center, and the second layer represents the mapping relationship between tasks and virtual machines. At the same time, a set of uniformly distributed reference vectors is generated. Step 2, Offspring Population Generation: The initial population P is used to generate the offspring population O by combining crossover mutation and task balancing mapping strategies; the crossover mutation adopts the index-based crossover method, simulated binary crossover, and polynomial mutation; the task balancing mapping strategy balances the load by sorting virtual machines according to task completion time and exchanging tasks on different virtual machines. Step 3, Two-stage adaptive search: Merge the parent population P and the offspring population O to obtain a new population P', and perform a two-stage adaptive environment selection based on the new population P' to obtain the next generation parent population CP; Step 4, Termination Judgment: If the maximum number of iterations has been reached, output the non-dominated solution as the optimal task scheduling scheme; otherwise, return to step 2 to continue iterating.

4. The method according to claim 3, characterized in that, The specific process of the two-stage adaptive environment selection described in step 3 includes: Step 3.1: Calculate the angle between an individual in the population CP and the reference vector, associate the individual with the sub-region corresponding to the reference vector with the smallest angle, and perform non-dominated sorting on the population, retaining the non-dominated sorting level of the individual. Step 3.2: Calculate the adaptive transformation conditions. These conditions are determined by the population centroid variance (Variance) and the number of iterations. The population centroid is calculated from the average perceived distance of individuals in the population, and the variance (Variance) measures the degree of population aggregation. The comprehensive stage transformation strategy proceeds to the first stage if the following equation is satisfied; otherwise, it proceeds to the second stage: ； Where g is the current iteration number, G max η is the maximum number of iterations, η is the two-stage transformation parameter, m is the number of objective functions, and ∂ is the transformation threshold; Step 3.3 First stage search: In each sub-region, retain the individual with the smallest number of non-dominated layers. If there are multiple individuals, select the individual with the smallest penalty-based boundary intersection PBI value. After pre-screening, retain the individual with the smallest preference perception distance to form the population. Step 3.4 Second stage search: Divide the target space into several spatial levels, pre-screen individuals and classify them into preferred regions based on reference points, prioritize retaining individuals in regions that balance the interests of users and cloud service providers, and supplement the population to the set size.

5. The method according to claim 4, characterized in that, The formula for calculating the preference-perceived distance ED(x) is as follows: ,in Let Euclidean distance be the distance between the individual and the reference point. This represents the Euclidean distance between the individual and the current spatial origin.

6. The method according to claim 4, characterized in that, In step 3.4, the preference region classification divides the target space into four regions, corresponding to the region that balances the interests of both parties, the region that protects the interests of users, the region that meets the needs of cloud service providers, and the region that cannot meet the needs of either party. The region to which an individual belongs is determined based on the individual's average spatial level gp, the user's average spatial level gu, and the service provider's average spatial level gc.

7. The method according to claim 3, characterized in that, In the task balancing mapping strategy, the virtual machines are first sorted in ascending order by task completion time, and the virtual machine index number K is recorded. i Determine the minimum task value z for all virtual machines, then swap some tasks on virtual machines with fewer tasks with corresponding tasks on virtual machines with more tasks, until all virtual machines have completed the task balancing adjustment.

8. The method according to claim 3, characterized in that, The two-stage transformation parameter η is set to 0.7, the transformation threshold ∂ is set to 0.025, the population size is set to 100, and the maximum number of iterations is set to 100.

Images