Cost and time aware task scheduling method, device and equipment and storage medium

By formalizing the task scheduling problem into Markov decision-making process and using the deep Q network model, combining the reward function of completion time and cost penalty items, the intelligence and adaptability of task scheduling in a multi-cloud environment is achieved, and the task is solved The issue of completion time and cost control improves the efficiency and economicality of task scheduling.

CN119088519AActive Publication Date: 2024-12-06SOUTH CHINA UNIV OF TECH

Patent Information

Application Number
CN202411176662.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-12-06
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

In multi-cloud environments, existing task scheduling methods are difficult to shorten task completion time and control costs at the same time, resulting in waste of resources and computing needs difficult to support.

Method used

Formulate task scheduling problems into Markov decision-making process (MDP), and train them using the deep Q network (DQN) model. Through intelligent and adaptive scheduling strategies, optimize tasks allocation to virtual machine instances, combined with completion time rewards and the reward function of the cost penalty item to achieve optimal scheduling.

Benefits of technology

It significantly shortens the task completion time, reduces execution costs, improves the overall efficiency and economic benefits of task scheduling, and adapts to changes in the dynamic cloud environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088519A_ABST
    Figure CN119088519A_ABST
Patent Text Reader

Abstract

The invention relates to a cost and time aware task scheduling method, device and equipment and a storage medium, the method is applied to a multi-cloud environment comprising a plurality of cloud service providers (CSPs), each CSP comprises a plurality of virtual machine instances, and the method comprises the steps of obtaining to-be-scheduled tasks, and sorting the to-be-scheduled tasks; obtaining to-be-scheduled task information and multi-cloud environment information, and determining a state space, an action space and a reward function according to the to-be-scheduled task information and the multi-cloud environment information; the reward function comprises a completion time reward and a cost penalty term; a task scheduling problem forms a Markov decision process MDP, and a deep Q network DQN model is used for training; and determining an optimal scheduling strategy of a task scheduling problem by utilizing the trained DQN model, and allocating a corresponding virtual machine instance to a to-be-scheduled task according to the optimal scheduling strategy. A reward function of the DQN model comprises a completion time reward and a cost penalty term, so that a scheduling strategy made by the model can give consideration to the completion time and execution cost of a task at the same time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of task scheduling, and more specifically, to a cost- and time-aware task scheduling method, device, equipment and storage medium. Background Art

[0002] As a revolutionary computing model, cloud computing has become a research hotspot in the field of global information technology with its ultra-large-scale service capabilities. With the continuous development of cloud computing technology, the requirements for data processing speed, computing performance and storage capacity have risen sharply, directly promoting the rapid growth of the number of data centers around the world, but at the same time it has also triggered an exponential increase in infrastructure energy consumption.

[0003] In a multi-cloud environment, as enterprises deploy their applications and data across multiple cloud service providers, task scheduling methods face unprecedented challenges and opportunities. In the early days, when faced with a surge in data volume, enterprises often increased the number of physical facilities to alleviate the pressure brought by data growth. Although this method can temporarily alleviate the pressure, in the long run, it not only leads to a huge waste of physical resources, but also makes it difficult to effectively support the continued growth of computing needs brought by massive data.

[0004] Therefore, how to implement a scheduling method in a multi-cloud environment that can significantly shorten task completion time and effectively control costs has become a key issue that technical personnel in the current cloud computing field need to solve urgently. Summary of the invention

[0005] The present invention aims to overcome at least one defect (shortcoming) of the above-mentioned prior art and provide a cost-and-time-aware task scheduling method, apparatus, device and storage medium for solving the problem of how to implement a scheduling method in a multi-cloud environment that can significantly shorten the task completion time and effectively control the cost.

[0006] According to a first aspect of the present application, a cost- and time-aware task scheduling method is provided, the method being applied to a multi-cloud environment, the multi-cloud environment comprising at least two cloud service providers CSP, each of the cloud service providers CSP comprising a plurality of virtual machine instances, the method comprising: Obtaining tasks to be scheduled, and sorting the tasks to be scheduled; Obtaining information about tasks to be scheduled and information about a multi-cloud environment, and determining a state space, an action space, and a reward function according to the information about tasks to be scheduled and information about a multi-cloud environment; the reward function includes a completion time reward and a cost penalty item; The task scheduling problem is formalized into a Markov decision process (MDP) and trained using the deep Q network (DQN) model. The trained DQN model is used to determine the optimal scheduling strategy for the task scheduling problem, and a corresponding virtual machine instance is allocated to the task to be scheduled according to the optimal scheduling strategy.

[0007] By formalizing the task scheduling problem in a multi-cloud environment as a Markov decision process (MDP) and using the deep Q network (DQN) model for training and optimization, the intelligent and adaptive task scheduling is achieved. Through continuous learning, the DQN model can effectively capture and respond to dynamic changes in the cloud environment to obtain a scheduling strategy that takes into account both the completion time efficiency and the execution cost, thereby improving the overall efficiency and economic benefits of task scheduling.

[0008] Optionally, a plurality of the virtual machine instances constitute a virtual machine instance set , the virtual machine instance set It is expressed as: in, Indicates Cloud service providers CSP, Indicates virtual machine instances, Indicates The first virtual machine instances, Represents a collection of virtual machine instances in a multi-cloud environment; Assume that there are tasks to be scheduled The tasks to be scheduled are expressed as: in, Indicates Tasks to be scheduled; Basic characteristics of each of the virtual machine instances It is expressed as: in, express The computing power of express The number of virtual CPUs, express The memory size, express The unit time rental cost; Task exist Execution time The calculation formula is as follows: in, Indicates the task size; Task The maximum completion time is expressed as: in, Indicates the task The longest completion time; Task exist The cost of executing above is expressed as: in, Indicates The billing mechanism of a cloud service provider CSP, Indicates the task exist The cost of execution; Task The total cost of execution is expressed as: in, Indicates the task Is it in If = 1 indicates a task exist If =0 means task Not Available Execute on.

[0009] Optionally, the state space includes a virtual machine instance state and a task state, and each state information in the state space Defined as: in, Representing the state space Any status information in Represents the current status information of the task. =0 means waiting for scheduling, =1 means scheduling is in progress. =2 means the scheduling has been completed; Represents a collection of virtual machine instances The available virtual machine instances in Indicates the completion time of the task in its current state; Indicates the operating cost of the VM instance in the current state.

[0010] Optionally, the action space includes a set of all executable actions of the agent of the DQN model in any state, and the action represents selecting an action from the action space and executing it. The action space is represented as follows: in, Represents the action space; for action , indicating that you choose to assign the current task to and execute; Indicates that waiting is performed under full load or other abnormal conditions.

[0011] Optionally, the reward function calculation formula is as follows: in, Represents the completion time reward, represents the cost penalty term.

[0012] Completion time bonus The calculation formula is as follows: in, is the weight coefficient of the completion time optimization objective, The value of is a positive number; and Respectively represent the status and status The maximum completion time under The cost penalty calculation formula is as follows: in, is the weight coefficient of the cost-reward optimization objective, The value of is a positive number.

[0013] Optionally, the DQN model includes an evaluation network and a target network, and the training using the deep Q network DQN model includes the following steps: Initialize the evaluation network using random network parameters ω; Get the current status , according to the current state use Strategy selection action , and perform actions ; Execute an action After that, get the immediate reward of environmental feedback and the next state ; The current state , Actions performed , get instant rewards , and the next state Forming Tuples And stored as experience samples in the experience playback module; wherein the instant reward Calculated by the reward function; When the data volume of the experience samples reaches a preset value, randomly selecting experience samples from the experience playback module to train the evaluation network; Calculating a predicted Q value based on the selected experience sample using the evaluation network, and calculating a target Q value based on the selected experience sample using the target network; Calculate the loss function according to the predicted Q value and the target Q value, and update the network parameter ω using the gradient descent method; Iterate the training evaluation network. After the training is completed, save the final training network parameters ω; During the iterative training and evaluation of the network, the network parameters of the network are evaluated every preset number of steps. Copy to the target network.

[0014] By introducing the separation of the evaluation network and the target network and regular weight synchronization, combined with the experience replay mechanism, the DQN model effectively improves the stability and convergence speed of training and enhances the model's adaptability to complex environments.

[0015] Optionally, the current state use Strategy selection action The following steps are involved: Set an exploration rate , initialized to 1; Each time a decision is made, a random number in the range [0, 1] is generated ,if , then the utilization decision is executed, that is, the action with the highest Q value in the current state is selected for execution; if , then continue to execute the exploration decision, that is, randomly select an action to execute; Gradually reduce the exploration rate at a preset interval The value of .

[0016] use The strategy selects actions. In the initial stage, more exploration of the strategy is done to gain more experience, and the Q value of each action is estimated more and more accurately. With time decay, exploration can be gradually reduced after accumulating a lot of experience, and the existing Q value can be used more to optimize the selection of actions.

[0017] According to a second aspect of the present application, a cost- and time-aware task scheduling device is provided, the device being applied to a multi-cloud environment, the multi-cloud environment comprising at least two cloud service providers CSP, each of the cloud service providers CSP comprising a plurality of virtual machine instances, the device comprising: A task submission module is used to submit tasks to be scheduled and sort the tasks to be scheduled; An information acquisition module is used to acquire information about tasks to be scheduled and multi-cloud environment information, and determine a state space, an action space, and a reward function according to the information about tasks to be scheduled and multi-cloud environment information; the reward function includes a completion time reward and a cost penalty item; DQN model training module, which is used to formalize the task scheduling problem into a Markov decision process (MDP) and train it using the deep Q network (DQN) model; The resource allocation module is used to determine the optimal scheduling strategy for the task scheduling problem by using the trained DQN model, and allocate corresponding virtual machine instances to the tasks to be scheduled according to the optimal scheduling strategy.

[0018] According to a third aspect of the present application, an electronic device is provided, including: a memory for storing one or more computer programs; The processor implements the cost- and time-aware task scheduling method described in the first aspect above when the one or more computer programs are executed by the processor.

[0019] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the cost- and time-aware task scheduling method described in the first aspect when executed.

[0020] Based on any of the above aspects, the cost- and time-aware task scheduling method, device, electronic device, and computer storage medium provided in the embodiments of the present application can automatically learn and adapt to changes in the environment by formalizing the task scheduling problem as a Markov decision process and using the DQN model for training to find the optimal adjustment strategy, so as to flexibly respond to different task scheduling requirements, making the task scheduling method more reliable and efficient in practical applications. The reward function of the DQN model includes a completion time reward and a cost penalty term, so that the scheduling strategy made by the model can take into account both the completion time and execution cost of the task. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic application scenario diagram of the cost- and time-aware task scheduling method provided in this embodiment.

[0023] Figure 2 A flowchart of the cost- and time-aware task scheduling method provided in this embodiment.

[0024] Figure 3 Schematic diagram of the DQN model framework provided in this embodiment.

[0025] Figure 4 A schematic diagram of the overall structural framework of the cost- and time-aware task scheduling method provided in this embodiment.

[0026] Figure 5 This embodiment provides a comparison chart of the cost and completion time experimental results of different data sets.

[0027] Figure 6 A schematic diagram of the structure of a cost- and time-aware task scheduling device provided in this embodiment.

[0028] Figure 7 A schematic diagram of the structure of a cost- and time-aware task scheduling electronic device provided in this embodiment. DETAILED DESCRIPTION

[0029] The drawings of this application are only used for illustrative purposes and should not be construed as limiting the present application. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Task scheduling in cloud computing is a key link to ensure the reasonable allocation of tasks and resources, including the allocation of tasks to virtual machines (VMs) and the allocation of task sequences within VMs. At present, the research points on cloud computing resource scheduling at home and abroad are different, but each has its own limitations. For example, heuristic algorithms and static optimization algorithms often find it difficult to achieve satisfactory results in reducing the completion time and cost of task scheduling in the face of increasingly complex and dynamic multi-cloud environments.

[0033] This embodiment provides a technical solution that can solve the above-mentioned problem. The specific implementation methods of this application are described in detail below in conjunction with the accompanying drawings.

[0034] For example, a schematic diagram of an application scenario of a cost and time-aware task scheduling method provided in an embodiment of the present application is provided. Figure 1 As shown, the application scenario at least includes a server 100 and a terminal 200 that can communicate with the server 100, the server 100 has an image processing function; the terminal device 200 has a streaming media playback function.

[0035] It is understandable that the server 100 can be an independent electronic device or a cluster of multiple electronic devices; the terminal 200 can be a smart phone terminal, a personal computer, a tablet computer, a car terminal, etc., but is not limited thereto.

[0036] In one practicable manner, the server 100 and the terminal 200 may respectively execute the cost-and-time-aware task scheduling method provided in the embodiment of the present application, or, optionally, the cost-and-time-aware task scheduling method provided in the embodiment of the present application is partially executed in the server 100 and partially executed in the terminal 200.

[0037] like Figure 2 As shown, this embodiment provides a cost- and time-aware task scheduling method, which is applied to a multi-cloud environment and may include the following steps: S1. Obtain tasks to be scheduled, and sort the tasks to be scheduled.

[0038] The multi-cloud environment includes at least two cloud service providers CSP, and each cloud service provider CSP includes several virtual machine instances. Specifically, in a multi-cloud environment, the resources of all cloud service providers CSP jointly form a unified resource pool, which shields the heterogeneity of underlying resources, communication protocols, and technical architecture, and provides a standardized resource directory and a unified communication interface to cloud service users. All virtual machine instances in the resource pool constitute a virtual machine instance set. , a collection of virtual machine instances It is expressed as: in, Indicates Cloud service providers CSP, Indicates VMs, Indicates The first VMs, Represents a collection of virtual machine instances in a multi-cloud environment.

[0039] Assuming there are i tasks to be scheduled, the tasks to be scheduled are expressed as: in, Indicates Tasks to be scheduled.

[0040] Basic characteristics of each VM It is expressed as: in, express The computing power of express The number of virtual CPUs, express The memory size, express The unit time rental cost.

[0041] In a multi-cloud environment, if the tasks submitted by users are assigned to appropriate resources in the resource pool for execution, it can ensure the efficient use of multi-cloud resources while meeting the user's quality of service (QoS) requirements. Each cloud service provider CSP is independent in a multi-cloud environment and provides virtual machine (VM) instances of different types and prices. VMs of the same cloud service provider CSP communicate with each other through a virtual communication network, while VMs of different cloud service providers CSP communicate with each other through a commercial communication network. Therefore, the communication speed between VMs of the same cloud service provider CSP is generally higher than the communication speed between VMs of different cloud service providers CSP.

[0042] Any task exist Execution time The calculation formula is as follows: in, Indicates the task size.

[0043] Task The maximum completion time is: in, Indicates the task The longest completion time.

[0044] In a multi-cloud environment, the billing mechanisms of VMs of different cloud service providers (CSPs) may be different. For example, Amazon EC2 and Alibaba ECS are billed by the hour, while Microsoft Azure is billed by the minute, and Google Compute Engine is billed by the second.

[0045] set up The billing mechanism is , then the task exist The cost of executing above is expressed as: in, Indicates The billing mechanism of a cloud service provider CSP, Indicates the task exist The cost of execution.

[0046] Task The total cost of execution is expressed as: in, Indicates the task Is it in If = 1 indicates a task exist If =0 means task Not Available Execute on.

[0047] S2. Obtain information about tasks to be scheduled and information about the multi-cloud environment, and determine a state space, an action space, and a reward function according to the information about tasks to be scheduled and the multi-cloud environment; the reward function includes a completion time reward and a cost penalty item.

[0048] In this embodiment, the state space Includes VM instance status and task status. State space Each status information in Defined as: in, Representing the state space Any status information in Indicates the current status information of the task. =0 means waiting for scheduling, =1 means scheduling is in progress. =2 means the scheduling has been completed; Represents a collection of virtual machine instances The available virtual machine instances in Indicates the completion time under the current status; Indicates the current state of the virtual machine operating costs.

[0049] Assume that there are three cloud service providers (CSPs) in a multi-cloud environment, and each cloud service provider (CSP) provides three different configurations. And each are available, then Contains 9 types , that is, 9 types of available resource information.

[0050] In this embodiment, the action space includes the set of all executable actions of the agent in a specific state. The action represents the decision-making choice of the agent in different states. The agent can select and execute an action from the action space according to the current state. The action space is defined as follows: in represents the action space, Indicates that the current task can be assigned to ; Indicates that waiting is performed under full load or other abnormal conditions.

[0051] In order to measure the impact of each scheduling decision on the overall completion time and cost, the reward function of this embodiment includes a completion time reward and a cost penalty. The reward function calculation formula is as follows: in, Represents the completion time reward, represents the cost penalty term.

[0052] Specifically, completion time reward The calculation formula is as follows: in, is the weight coefficient of the completion time optimization objective, The value of is a positive number; and Respectively represent the status and status The maximum completion time.

[0053] The cost penalty calculation formula is as follows: in, is the weight coefficient of the cost-reward optimization objective, The value of is a positive number.

[0054] S3. Formalize the task scheduling problem into a Markov decision process (MDP) and use the deep Q network (DQN) model for training.

[0055] The task scheduling problem is formalized as a Markov decision process (MDP), and the MDP formalization of the task scheduling process is: in, represents the state space; A represents the virtual machine instance that can be assigned to the task in the action space; is the state transition function , Indicates in status Next action Then transfer to the new state probability; represents the reward function; represents the discount factor; It is a formal representation of the MDP of the task scheduling process.

[0056] This embodiment uses the DQN (Deep Q-Network) algorithm in deep reinforcement learning to optimize the task scheduling strategy. The DQN algorithm includes two networks with the same structure: the evaluation network and the target network.

[0057] like Figure 3 , Figure 4 As shown, training using the deep Q network DQN model includes the following steps: S10, initializing the evaluation network using a random network parameter ω; S11. Get current status , according to the current state use Strategy selection action , and perform actions ; Since there is no historical experience to be used in the initial stage of task scheduling, in order to maintain a balance between exploration and utilization, this embodiment adopts ( ) strategy to select actions. Exploration is to explore new actions to discover potential high-return behaviors, while exploitation is to use the current known information to maximize the immediate return. Under the strategy, the agent uses probability Randomly select an action to explore, and 1- The probability of selecting the action with the highest Q value in the current state to be utilized. As the learning process proceeds, The value can be gradually reduced so that it tends to use known information after gaining more experience. In the specific implementation, the experience replay module includes a memory space with a capacity of RM, which is used to store the data obtained from each exploration. When the number of samples in RM reaches the set value, a batch of data is taken from RM each time to train the DQN model.

[0058] Specifically, according to the current status use Strategy selection action The following steps are involved: S110, set an exploration rate , initialized to 1.

[0059] In the initial stage, the exploration rate will be at its maximum, which means that the learning strategy will randomly select actions for a lot of exploration.

[0060] S111. Each time a decision is made, a random operator is generated. ,if , then the utilization decision is executed, that is, the action with the highest Q value in the current state is selected for execution; if , then continue to execute the exploration decision, that is, randomly select an action to execute.

[0061] S112: The state of the environmental feedback after executing the action and rewards right The value is updated.

[0062] As training progresses, the exploration rate is gradually reduced at preset intervals. The value of makes the agent use strategies more in the later stages of learning. In this embodiment, the exploration rate Greater than 0 and less than or equal to 1.

[0063] S12. Execute action After that, get the immediate reward of environmental feedback and the next state ; S13, the current state , Actions performed , get instant rewards , and the next state Forming Tuples And stored as experience samples in the experience playback module; wherein the instant reward Calculated by the reward function; S14, when the data amount of the experience samples reaches a preset value, randomly selecting experience samples from the experience playback module to train the evaluation network; The deep Q network is trained through experience replay and priority experience replay technology to store and replay past experience to improve the training efficiency and stability of the model. Finally, the task scheduling strategy is iteratively optimized until it converges to the globally optimal cost and completion time balance solution.

[0064] S15, using the evaluation network to calculate the predicted Q value based on the selected experience sample, and at the same time, using the target network to calculate the target Q value based on the selected experience sample; S16, calculating the loss function according to the predicted Q value and the target Q value, and updating the network parameter ω using the gradient descent method; The loss function is calculated as follows: in, is the discount factor, is the reward function, and Represent the current state and the next state respectively, ( ) and( ) represent the current action and the next action respectively, and Represent the evaluation network parameters and target network parameters respectively, Represents expectations.

[0065] S17, iteratively train the evaluation network, and after the training is completed, save the network parameters ω obtained by the final training; During the iterative training and evaluation of the network, the network parameters of the network will be evaluated every preset number of steps. Copy to the target network.

[0066] The task scheduling network parameters can be continuously corrected by continuous training of the task scheduling network. , until the task scheduling network can learn a strategy that effectively reduces the completion time and cost of task scheduling.

[0067] After the training phase is completed, save the trained network parameters , so that the parameter can be used to schedule the task later.

[0068] In this embodiment, the following formula is used to update value: in, is the current state, For the current action, Indicates in status Take action The expected return (i.e., Q value) is is the learning rate, Indicates in status Take action After receiving the instant reward, is the discount factor, Indicates that in the next state All possible actions The largest value.

[0069] S4. Determine the optimal scheduling strategy for the task scheduling problem using the trained DQN model, and assign corresponding virtual machine instances to the tasks to be scheduled according to the optimal scheduling strategy.

[0070] The cost-and-time-aware task scheduling method of this embodiment was tested. During the test, this embodiment used the Google Cluster Trace dataset and datasets of different sizes to simulate the workload of the task scheduling system in a multi-cloud environment. These datasets are released by Google and contain task tracking data of a Borg unit within 7 hours. The dataset includes tasks executed on a single machine, which consume memory and one or more cores, and multiple tasks can belong to a single job (such as mapper and reducer). In order to enhance the authenticity of the dataset and verify the effectiveness of the cost-and-time-aware task scheduling method of this embodiment, a screening and extraction process was used in the test process, and task datasets of different sizes were selected for testing.

[0071] This example uses three cloud service providers CSP: Amazon EC2, Microsoft Azure, and AlibabaCloud EC2. Each cloud service provider CSP configures three virtual machine instance types with different performance for selection. In order to simulate the real situation, the computing power of these virtual machine instances is converted into MIPS (million instructions per second) using official documents. In this example, the bandwidth within the virtual communication network is 100Mbps / s, while the communication bandwidth between different clouds is 20Mbps / s. In the specific implementation, the network learning rate of the algorithm is Set to 0.01, discount factor Set to 0.95, and the capacity of the experience replay module is set to , the batch size is set to 16. During the model training process, the parameters of the target network are updated every time 30 transformation tuples are stored in the experience replay module.

[0072] like Figure 5 As shown, the test results show that the cost-and-time-aware task scheduling method of this embodiment can achieve shorter completion time and lower cost on various data sets. On the Large data set, the completion time of the algorithm is reduced by 60.59% compared with the RR algorithm, which significantly improves the efficiency and effect of task scheduling. In addition, the task scheduling method of this embodiment can achieve the lowest rental cost on all four data sets. For example, in the case of 1,000 scheduling tasks, the task scheduling method of this embodiment saves 39.87% of the rental cost compared with the GA algorithm. As the number of tasks increases, the performance of the task scheduling method of this embodiment in cost optimization continues to improve, and it performs outstandingly on the Google Cluster Data data set.

[0073] The technical solution of the cost- and time-aware task scheduling method provided in this application is described below in conjunction with a specific embodiment.

[0074] like Figure 6 As shown, the embodiment of the present application further provides a cost and time-aware task scheduling device 610, which is applied to a multi-cloud environment, wherein the multi-cloud environment includes at least two cloud service providers CSP, and each of the cloud service providers CSP includes a plurality of virtual machine instances. Optionally, the cost and time-aware task scheduling device 610 may include: The task submission module 611 is used to submit the tasks to be scheduled and sort the tasks to be scheduled; An information acquisition module 612 is used to acquire information about tasks to be scheduled and information about multi-cloud environments, and determine a state space, an action space, and a reward function according to the information about tasks to be scheduled and information about multi-cloud environments; the reward function includes a completion time reward and a cost penalty item; A DQN model training module 613 is used to formalize the task scheduling problem into a Markov decision process MDP and train it using a deep Q network DQN model; The resource allocation module 614 is used to determine the optimal scheduling strategy for the task scheduling problem using the trained DQN model, and allocate corresponding virtual machine instances to the tasks to be scheduled according to the optimal scheduling strategy.

[0075] It can be understood that the above-mentioned device embodiments and the above-mentioned method embodiments can correspond to each other, and similar descriptions of the device embodiments can refer to the method embodiments. To avoid repetition, it will not be repeated here. A cost-and-time-aware task scheduling device provided in an embodiment of the present application can execute a cost-and-time-aware task scheduling method provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method. The functional modules of the cost-and-time-aware task scheduling device can be implemented in the form of hardware, can be implemented by instructions in the form of software, and can also be implemented by a combination of hardware and software modules.

[0076] Specifically, each step of the method embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the cost and time-aware task scheduling method in combination with the embodiment of the present application can be directly embodied as a hardware coding processor for execution, or a combination of hardware and software modules in the coding processor for execution. Optionally, the software module can be located in a random access memory, a read-only memory, a programmable read-only memory, a flash memory, an electrically erasable programmable memory, a register, or other storage media. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in combination with its hardware.

[0077] The present application embodiment provides an electronic device 710, whose structure is as follows: Figure 7 The electronic device 710 may be Figure 1 The server 100 or the terminal 200 is shown.

[0078] like Figure 7 As shown, the electronic device 710 includes a memory 711, a processor 712, a communication module 713 and an input / output interface 714, etc. Optionally, the memory 711, the processor 712, the communication module 713 and the input / output interface 714 can be connected and communicated through a bus 715.

[0079] The memory 711 is used to store one or more computer programs and transmit the code of the computer program to the processor 712; when the one or more computer programs are executed by the processor 711, the cost and time-aware task scheduling method in the embodiment of the present application is implemented.

[0080] Optionally, the electronic device 710 can be connected to a network via a communication module 713 to communicate with other devices, such as a terminal or a server, through the network to achieve data interaction. The electronic device 710 can be various forms of digital computers, such as desktop computers, servers, workstations, mainframe computers or other types of computers. The electronic device 710 can also be various forms of mobile terminals, such as smart phones, tablet computers, wearable devices (such as helmets, glasses, watches, etc.) and other similar mobile terminals.

[0081] Optionally, the electronic device 710 can be connected to the required input / output devices, such as a keyboard, a display device, etc., through the input / output interface 714. The electronic device 710 itself can have a display device, and can also be connected to other display devices through the input / output interface 714. Optionally, a storage device, such as a hard disk, can also be connected through the input / output interface 714, so that data in the electronic device 710 can be stored in the storage device, or data in the storage device can be read, and data in the storage device can also be stored in the memory 711. It can be understood that the input / output interface 714 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 714 can be a component of the electronic device 710, or it can be an external device connected to the electronic device 710 when needed.

[0082] Optionally, the memory 711 may be a volatile memory and / or a non-volatile memory, the volatile memory may be a random access memory, etc., and the non-volatile memory may be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory or a flash memory, etc.

[0083] Optionally, the computer program stored in the processor 711 may be divided into one or more modules, which are stored in the memory 711 and executed by the processor 712 to complete the method provided in the embodiment. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the computer program instruction segments are used to describe the execution process of the computer program in the electronic device 710.

[0084] Optionally, the processor 712 may be various general and / or special processing components with processing and computing capabilities. Some examples of the processor 712 include, but are not limited to, a central processing unit, a graphics processing unit, a digital signal processor, various special artificial intelligence computing chips, various processors running machine learning model algorithms, and may also be any appropriate controller, microcontroller, processor, etc. The processor 712 executes various methods and processes of this embodiment, illustratively, such as a cost and time-aware task scheduling method of an embodiment of the present application.

[0085] Optionally, the bus 715 may include a path for transmitting information. The bus 715 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. According to different functions, the bus 715 may be divided into an address bus, a data bus, a control bus, etc.

[0086] In an optional implementation, the embodiment of the present application further provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. Part or all of the computer program can be loaded and / or installed on the memory 711 of the electronic device 710. When the computer program is executed by the processor 712, one or more steps of a cost and time-aware task scheduling method of the embodiment of the present application can be performed.

[0087] Optionally, the computer-readable storage medium may be a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, or the like.

[0088] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A cost- and time-aware task scheduling method, the method being applied to a multi-cloud environment, the multi-cloud environment comprising at least two cloud service providers CSP, each of the cloud service providers CSP comprising a number of virtual machine instances, characterized in that: The method comprises: Obtaining tasks to be scheduled, and sorting the tasks to be scheduled; Obtaining information about tasks to be scheduled and information about a multi-cloud environment, and determining a state space, an action space, and a reward function according to the information about tasks to be scheduled and information about a multi-cloud environment; the reward function includes a completion time reward and a cost penalty item; The task scheduling problem is formalized into a Markov decision process (MDP) and trained using the deep Q network (DQN) model. The trained DQN model is used to determine the optimal scheduling strategy for the task scheduling problem, and a corresponding virtual machine instance is allocated to the task to be scheduled according to the optimal scheduling strategy.

2. A cost and time-aware task scheduling method according to claim 1, characterized in that: A plurality of said virtual machine instances constitute a virtual machine instance set , the virtual machine instance set It is expressed as: in, Indicates Cloud service providers CSP, Indicates virtual machine instances, Indicates The first virtual machine instances, Represents a collection of virtual machine instances in a multi-cloud environment; Assume that there are tasks to be scheduled The tasks to be scheduled are expressed as: in, Indicates Tasks to be scheduled; Basic characteristics of each of the virtual machine instances It is expressed as: in, express The computing power of express The number of virtual CPUs, express The memory size, express The unit time rental cost; Task exist Execution time The calculation formula is as follows: in, Indicates the task size; Task The maximum completion time is expressed as: in, Indicates the task The longest completion time; Task exist The cost of executing above is expressed as: in, Indicates The billing mechanism of a cloud service provider CSP, Indicates the task exist The cost of execution; Task The total cost of execution is expressed as: in, Indicates the task Is it in If = 1 indicates a task exist If = 0 means task Not Available Execute on.

3. A cost and time-aware task scheduling method according to claim 2, characterized in that: The state space includes the virtual machine instance state and the task state. Each state information in the state space Defined as: in, Representing the state space Any status information in Indicates the current status information of the task. =0 means waiting for scheduling, =1 means scheduling is in progress. =2 means the scheduling has been completed; Represents a collection of virtual machine instances The available virtual machine instances in Indicates the completion time of the task in its current state; Indicates the operating cost of the VM instance in the current state.

4. A cost and time-aware task scheduling method according to claim 2, characterized in that: The action space includes a set of all executable actions of the agent of the DQN model in any state, and the action represents selecting an action from the action space and executing it. The action space is represented as follows: in, Represents the action space; for action , indicating that you choose to assign the current task to and execute; Indicates that waiting is performed under full load or other abnormal conditions.

5. A cost and time-aware task scheduling method according to claim 2, characterized in that: The reward function calculation formula is as follows: in, Represents the completion time reward, Represents the cost penalty term. Completion time bonus The calculation formula is as follows: in, is the weight coefficient of the completion time optimization objective, The value of is a positive number; and Respectively represent the status and status The maximum completion time under The cost penalty calculation formula is as follows: in, is the weight coefficient of the cost-reward optimization objective, The value of is a positive number.

6. A cost and time-aware task scheduling method according to claim 2, characterized in that: The DQN model includes an evaluation network and a target network, and the training using the deep Q network DQN model includes the following steps: Initialize the evaluation network using random network parameters ω; Get the current status , according to the current state use Strategy selection action , and perform actions ; Execute an action After that, get instant rewards for multi-cloud environment feedback and the next state ; The current state , Actions performed , get instant rewards , and the next state Forming Tuples And stored as experience samples in the experience playback module; wherein the instant reward Calculated by the reward function; When the data volume of the experience samples reaches a preset value, randomly selecting experience samples from the experience playback module to train the evaluation network; Calculating a predicted Q value based on the selected experience sample using the evaluation network, and calculating a target Q value based on the selected experience sample using the target network; Calculate the loss function according to the predicted Q value and the target Q value, and update the network parameter ω using the gradient descent method; Iterate the training evaluation network. After the training is completed, save the final training network parameters ω; During the iterative training and evaluation of the network, the network parameters of the network are evaluated every preset number of steps. Copy to the target network.

7. A cost and time-aware task scheduling method according to claim 6, characterized in that: According to the current status use Strategy selection action The following steps are involved: Set an exploration rate , initialized to 1; Each time a decision is made, a random number in the range [0, 1] is generated ,if , then the utilization decision is executed, that is, the action with the highest Q value in the current state is selected for execution; if , then continue to execute the exploration decision, that is, randomly select an action to execute; Gradually reduce the exploration rate at a preset interval The value of .

8. A cost- and time-aware task scheduling device, the device being applied to a multi-cloud environment, the multi-cloud environment comprising at least two cloud service providers CSP, each of the cloud service providers CSP comprising a number of virtual machine instances, characterized in that: The device comprises: A task submission module is used to submit tasks to be scheduled and sort the tasks to be scheduled; An information acquisition module is used to acquire information about tasks to be scheduled and multi-cloud environment information, and determine a state space, an action space, and a reward function according to the information about tasks to be scheduled and multi-cloud environment information; the reward function includes a completion time reward and a cost penalty item; DQN model training module, which is used to formalize the task scheduling problem into a Markov decision process (MDP) and train it using the deep Q network (DQN) model; The resource allocation module is used to determine the optimal scheduling strategy for the task scheduling problem by using the trained DQN model, and allocate corresponding virtual machine instances to the tasks to be scheduled according to the optimal scheduling strategy.

9. An electronic device, characterized in that: include: a memory for storing one or more computer programs; A processor, when the one or more computer programs are executed by the processor, implements the cost and time-aware task scheduling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the cost- and time-aware task scheduling method according to any one of claims 1 to 7 when executed.

Citation Information

Patent Citations

  • Cloud order dynamic receiving and scheduling method based on deep reinforcement learning

    CN113935586A

  • Distribution method for task scheduling of DQN algorithm based on Lyapunov optimization

    CN117376141A

  • Engineering project file management optimization method based on file workflow and reinforcement learning

    CN118134412A

Cited By

  • Two-stage imaging satellite task scheduling method, storage medium and computer equipment

    CN119831287A

  • Staff task dynamic scheduling control method and system based on virtual digital identity

    CN120806571A

  • Employee task dynamic scheduling control method and system based on virtual digital identity

    CN120806571B