Cost and time-aware task scheduling method, device, equipment and storage medium

Abstract

The application relates to a cost and time-aware task scheduling method and device, equipment and a storage medium. The method is applied to a multi-cloud environment including a plurality of cloud service providers (CSPs), each of which includes a plurality of virtual machine instances. The method comprises the following steps: obtaining a to-be-scheduled task, and sorting the to-be-scheduled task; obtaining to-be-scheduled task information and multi-cloud environment information, determining a state space, an action space and a reward function according to the to-be-scheduled task information and the multi-cloud environment information; the reward function includes a completion time reward and a cost penalty term; formalizing a task scheduling problem as a Markov decision process (MDP), and training by using a deep Q network (DQN) model; determining an optimal scheduling strategy of the task scheduling problem by using the trained DQN model, and allocating corresponding virtual machine instances to the to-be-scheduled task according to the optimal scheduling strategy. The reward function of the DQN model includes the completion time reward and the cost penalty term, so that the scheduling strategy made by the model can simultaneously consider the completion time and the execution cost of the task.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of task scheduling, and more particularly, to a cost and time-aware task scheduling method, device, equipment and storage medium. BACKGROUND

[0002] As a revolutionary computing model, cloud computing has become a research hotspot in the global information technology field due to its super large-scale service capability. With the continuous development of cloud computing technology, the requirements for data processing speed, computing efficiency and storage capacity have increased dramatically, directly driving the rapid growth of the number of global data centers, but also causing exponential growth in infrastructure energy consumption.

[0003] In a multi-cloud environment, as enterprises deploy their applications and data across multiple cloud service providers, task scheduling methods face unprecedented challenges and opportunities. In the early days, in the face of the rapid increase in data volume, enterprises often increase the number of physical devices to alleviate the pressure brought about by data growth. This method can temporarily alleviate the pressure, but in the long run, it not only leads to a huge waste of physical resources, but also is difficult to effectively support the growing computing needs brought about by massive data.

[0004] Therefore, how to achieve a scheduling method that can significantly shorten the task completion time and effectively control the cost in a multi-cloud environment has become a key problem that needs to be solved by technical personnel in the field of cloud computing. SUMMARY

[0005] The present application aims to overcome at least one of the above-mentioned defects (shortcomings) of the prior art, and provides a cost and time-aware task scheduling method, device, equipment and storage medium, which solves the problem of how to achieve a scheduling method that can significantly shorten the task completion time and effectively control the cost in a multi-cloud environment.

[0006] According to a first aspect of the present application, a cost and time-aware task scheduling method is provided, which is applied to a multi-cloud environment including at least two cloud service providers CSP, each of which includes a plurality of virtual machine instances, and the method comprises:

[0007] Obtaining a task to be scheduled, and sorting the task to be scheduled;

[0008] Obtaining task information to be scheduled and multi-cloud environment information, and determining a state space, an action space and a reward function according to the task information to be scheduled and the multi-cloud environment information; the reward function includes a completion time reward and a cost penalty term;

[0009] Formalizing the task scheduling problem as a Markov decision process MDP, and training using a deep Q network DQN model;

[0010] The optimal scheduling strategy for the task scheduling problem is determined using the trained DQN model, and a corresponding virtual machine instance is allocated to the task to be scheduled according to the optimal scheduling strategy.

[0011] By formalizing the task scheduling problem as a Markov Decision Process (MDP) in a multi-cloud environment and training and optimizing it using a Deep Q-Network (DQN) model, intelligent and adaptive task scheduling is achieved. Through continuous learning, the DQN model can effectively capture and respond to dynamic changes in the cloud environment, thereby obtaining a scheduling strategy that considers both completion time efficiency and execution cost, improving the overall performance and economic benefits of task scheduling.

[0012] Optionally, multiple virtual machine instances constitute a virtual machine instance set. The virtual machine instance set Represented as:

[0013]

[0014] in, Indicates the first Cloud service provider CSP, Indicates the first One virtual machine instance, Indicates the first The first cloud service provider CSP One virtual machine instance, Represents a collection of virtual machine instances in a multi-cloud environment;

[0015] Suppose there are tasks to be scheduled One, the tasks to be scheduled are represented as:

[0016]

[0017] in, Indicates the first One task awaiting scheduling;

[0018] Basic characteristics of each virtual machine instance Represented as:

[0019]

[0020] in, express computing power express The number of virtual CPUs, express memory size, express The unit time rental cost;

[0021] task In the time of execution of a task is calculated as follows:

[0022]

[0023] wherein, T denotes the size of a task ;

[0024] The longest completion time of a task is denoted as:

[0025]

[0026] wherein, T denotes the longest completion time of a task ;

[0027] The cost of executing a task on is denoted as:

[0028]

[0029] wherein, CSP denotes the billing mechanism of the th cloud service provider CSP

[0030] The total cost of executing a task is denoted as:

[0031]

[0032] wherein, I denotes whether a task is executed on , if I = 1 denotes that a task is executed on , if I = 0 denotes that a task is not executed on ;

[0033] Optionally, the state space comprises virtual machine instance states and task states, each state information in the state space is defined as: s

[0034]

[0035] wherein, s S denotes any state information in the state space S , STcurrent state information of a task, ST =0 means waiting for scheduling, ST =1 means being scheduled, ST =2 means completed scheduling; ASR represent a set of virtual machine instances available in the virtual machine instance pool; M represent the completion time of the task in the current state; CA represent the running cost of the virtual machine instance in the current state.

[0036] Optionally, the action space includes a set of all executable actions of the agent of the DQN model in any state, the action represents selecting an action from the action space and executing, and the action space represents as follows:

[0037]

[0038] wherein, A represents the action space; for action , it represents selecting to assign the current task to and executing; hold represents waiting in a full load state or other abnormal conditions.

[0039] Optionally, the reward function is calculated as follows:

[0040]

[0041] wherein, represents the completion time reward, PC represents the cost penalty term;

[0042] The calculation formula of the completion time reward is as follows:

[0043]

[0044] wherein, is the weight coefficient of the completion time optimization target, the value of is positive; and respectively represent the longest completion time in state and state s ;

[0045] The calculation formula of the cost penalty term is as follows:

[0046]

[0047] wherein, is the weight coefficient of the cost reward optimization target, the value of is positive.

[0048] Optionally, the DQN model comprises an evaluation network and a target network, and the training using the deep Q network (DQN) model comprises the following steps:

[0049] initializing the evaluation network using random network parameters ω;

[0050] obtaining a current state s , according to the current state s adopting a policy to select an action , and performing the action ;

[0051] after performing the action , obtaining an immediate reward r and a next state feedback from the multi-cloud environment;

[0052] forming a tuple s of the current state , the performed action r , the obtained immediate reward , and the next state and storing it as an experience sample into an experience replay module; wherein the immediate reward r is calculated by the reward function;

[0053] when the amount of data of the experience samples reaches a preset value, randomly selecting experience samples from the experience replay module to train the evaluation network;

[0054] calculating a predicted Q value based on the selected experience samples using the evaluation network, and calculating a target Q value according to the selected experience samples using the target network;

[0055] calculating a loss function according to the predicted Q value and the target Q value, and updating the network parameters ω using a gradient descent method;

[0056] iteratively training the evaluation network, and saving the network parameters ω obtained after the final training;

[0057] wherein during the iterative training of the evaluation network, the network parameters ω of the evaluation network are copied to the target network every preset number of steps.

[0058] By introducing the separation of the evaluation network and the target network and the regular weight synchronization, and combining the experience replay mechanism, the DQN model effectively improves the stability and convergence speed of the training, and enhances the adaptability of the model to complex environments.

[0059] Optionally, the current state s is​ policy selection action comprising the following steps:

[0060] setting an exploration rate initialized to 1;

[0061] at each decision, randomly generating a random number in the range [0, 1] if , performing an exploitation decision, i.e. selecting the action with the highest Q value in the current state to perform; if , performing an exploration decision, i.e. randomly selecting an action to perform;

[0062] every pre-set time, gradually reducing the value of the exploration rate .

[0063] using the policy selection action, more exploration of the policy is performed in the initial stage to obtain more experience, and the Q value estimates of each action become more and more accurate. The exploration rate decays over time, which can gradually reduce exploration after accumulating a large amount of experience, and more use of existing Q values to optimize the selection of actions.

[0064] According to a second aspect of the present application, a cost and time-aware task scheduling device is provided, which is applied to a multi-cloud environment, the multi-cloud environment including at least two cloud service providers CSPs, each of the cloud service providers CSPs including a plurality of virtual machine instances, and the device including:

[0065] a task submission module configured to submit a to-be-scheduled task and sort the to-be-scheduled task;

[0066] an information acquisition module configured to acquire to-be-scheduled task information and multi-cloud environment information, determine a state space, an action space and a reward function according to the to-be-scheduled task information and the multi-cloud environment information; the reward function including a completion time reward and a cost penalty term;

[0067] a DQN model training module configured to formalize a task scheduling problem as a Markov decision process MDP and train using a deep Q network DQN model;

[0068] a resource allocation module configured to determine an optimal scheduling strategy of the task scheduling problem using the trained DQN model, and allocate corresponding virtual machine instances to the to-be-scheduled task according to the optimal scheduling strategy.

[0069] According to a third aspect of the present application, an electronic device is provided, including:

[0070] a memory configured to store one or more computer programs;

[0071] The processor, when executing the one or more computer programs, implements the cost and time-aware task scheduling method of the first aspect.

[0072] According to a fourth aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for causing a processor to implement the cost and time-aware task scheduling method of the first aspect when executed.

[0073] Based on any one of the above aspects, the cost and time-aware task scheduling method, device, electronic device and computer storage medium provided by the embodiments of the present application can automatically learn and adapt to changes in the environment by formulating the task scheduling problem as a Markov decision process and using the DQN model to train and find the optimal adjustment strategy, thereby flexibly coping with different task scheduling requirements, making the task scheduling method more reliable and efficient in actual application. The reward function of the DQN model includes a completion time reward and a cost penalty term, so that the scheduling strategy made by the model can take into account both the completion time and the execution cost of the task. BRIEF DESCRIPTION OF DRAWINGS

[0074] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0075] Figure 1 An exemplary application scenario diagram of the cost and time-aware task scheduling method provided by the present embodiment.

[0076] Figure 2 A flowchart of the cost and time-aware task scheduling method provided by the present embodiment.

[0077] Figure 3 A DQN model framework diagram provided by the present embodiment.

[0078] Figure 4 An overall structure framework diagram of the cost and time-aware task scheduling method provided by the present embodiment.

[0079] Figure 5 A comparison diagram of the cost and completion time experimental results of different data sets provided by the present embodiment.

[0080] Figure 6 A structural diagram of the cost and time-aware task scheduling device provided by the present embodiment.

[0081] Figure 7 The embodiment provides a structural schematic diagram of the cost and time-aware task scheduling electronic device. DETAILED DESCRIPTION

[0082] The drawings of the application are only used for illustrative description and cannot be understood as limitation to the application. In order to better illustrate the following embodiments, some components in the drawings are omitted, enlarged or reduced, and the size of the actual product is not represented. It is understandable for those skilled in the art that some well-known structures and their descriptions in the drawings can be omitted.

[0083] In order for those skilled in the art to better understand the scheme of the application, the technical scheme in the embodiments of the application will be clearly and completely described below in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the application.

[0084] It should be noted that the terms "first", "second" and the like in the specification and claims of the application and the above-described drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0085] Task scheduling of cloud computing is a key link to ensure reasonable allocation of tasks and resources, including allocation of tasks to virtual machines (VMs) and allocation of task sequences within virtual machines. At present, research points of cloud computing resource scheduling at home and abroad are different, but each has its limitations. For example, heuristic algorithms and static optimization algorithms often have difficulty in achieving satisfactory results in reducing task scheduling completion time and cost when facing increasingly complex and dynamic multi-cloud environments.

[0086] The embodiment provides a technical solution to solve the above problems, and the specific embodiments of the application will be described in detail below in combination with the drawings.

[0087] Exemplarily, a cost and time-aware task scheduling method provided by the embodiment of the application is applied to a scene. As shown in FIG. 1, the cost and time-aware task scheduling method provided by the embodiment of the application is applied to a multi-cloud environment. Figure 1As shown, the application scenario at least includes a server 100 and a terminal 200 which can communicate with the server 100, the server 100 has an image processing function; the terminal device 200 has a streaming media playing function.

[0088] It can be understood that the server 100 can be an independent electronic device, or a cluster composed of multiple electronic devices; the terminal 200 can be a smart phone terminal, a personal computer, a tablet computer, a vehicle-mounted terminal, etc., but is not limited thereto.

[0089] In an implementable manner, the server 100 and the terminal 200 can respectively execute the cost and time-aware task scheduling method provided by the embodiments of the present application, or alternatively, the cost and time-aware task scheduling method provided by the embodiments of the present application is partially executed in the server 100 and partially executed in the terminal 200.

[0090] As shown in the Figure 2 , the embodiments provide a cost and time-aware task scheduling method, the method is applied to a multi-cloud environment, and the method can include the following steps:

[0091] S1, obtaining a to-be-scheduled task, and sorting the to-be-scheduled task.

[0092] The multi-cloud environment includes at least two cloud service providers CSPs, and each cloud service provider CSP includes a plurality of virtual machine instances. Specifically, in the multi-cloud environment, the resources of all cloud service providers CSPs form a unified resource pool together, which masks the heterogeneity in aspects of underlying resources, communication protocols, and technical architectures, and provides a standardized resource directory and a unified communication interface for cloud service users. All virtual machine instances in the resource pool form a virtual machine instance set , which is represented as:

[0093]

[0094] wherein, represents the i-th cloud service provider CSP, represents the i-th VM, represents the i-th VM on the i-th cloud service provider CSP, represents the i-th VM on the i-th cloud service provider CSP, represents the i-th VM on the i-th cloud service provider CSP, represents the i-th VM on the i-th cloud service provider CSP, represents the virtual machine instance set in the multi-cloud environment. Suppose there are i to-be-scheduled tasks, the to-be-scheduled tasks are represented as:

[0095]

[0096] ​​

[0097] wherein, denotes the th task to be scheduled.

[0098] Basic characteristics of each VM are denoted as:

[0099]

[0100] wherein, denotes the computing power of, denotes the number of virtual CPUs of, denotes the memory size of, denotes the unit time rental cost of.

[0101] In a multi-cloud environment, if the tasks submitted by users are allocated to appropriate resources in the resource pool for execution, the efficient use of multi-cloud resources can be ensured, while meeting the quality of service (QoS) requirements of users. Each cloud service provider CSP is independent in the multi-cloud environment, and provides virtual machine (VM) instances of different types and prices. The VMs of the same cloud service provider CSP communicate through a virtual communication network, while the VMs of different cloud service providers CSP communicate through a commercial communication network, so the communication speed between the VMs of the same cloud service provider CSP is generally higher than that between the VMs of different cloud service providers CSP.

[0102] The execution time of any one task on is calculated as follows:

[0103]

[0104] wherein, denotes the size of the task ;

[0105] The longest completion time of the task is:

[0106]

[0107] wherein, denotes the longest completion time of the task .

[0108] ​In a multi-cloud environment, the billing mechanism of VMs of different cloud service providers (CSPs) can be different. For example, Amazon EC2 and Alibaba ECS are billed by the hour, while Microsoft Azure is billed by the minute, and Google Compute Engine is billed by the second.

[0109] Suppose the billing mechanism of is , the cost of task executed on is represented as:

[0110]

[0111] wherein, represents the billing mechanism of the th cloud service provider (CSP).

[0112] The total cost of task execution is represented as:

[0113]

[0114] wherein, represents whether task is executed on , if = 1 represents that task is executed on , if = 0 represents that task is not executed on .

[0115] S2, obtaining task information to be scheduled and multi-cloud environment information, determining state space, action space and reward function according to the task information to be scheduled and the multi-cloud environment information; the reward function includes a completion time reward and a cost penalty term.

[0116] In this embodiment, the state space S includes VM instance states and task states. Each state information S in the state space s is defined as:

[0117]

[0118] wherein, s represents any state information in the state space S , s ST represents the current state information of the task, ST =0 represents waiting for scheduling, ST =1 represents being scheduled, ST= 2 means the scheduling is completed; ASR represents a set of virtual machine instances available in the cloud environment; M represents the completion time of the task in the current state of the virtual machine instance; CA represents the running cost of the virtual machine instance in the current state.

[0119] Suppose there are 3 cloud service providers (CSPs) in the multi-cloud environment, each of which provides 3 different configurations of virtual machine instances. and each is available, then ASR contains 9 kinds , that is, 9 kinds of available resource information.

[0120] In this embodiment, the action space includes a set of all executable actions of the agent in a specific state. The action represents the decision choice of the agent in different states, and the agent can select and execute an action from the action space according to the current state. The action space is defined as follows:

[0121]

[0122] wherein A represents the action space, represents that the current task can be assigned to ; hold represents waiting in a full load state or other abnormal conditions.

[0123] In order to measure the influence of each scheduling decision on the overall completion time and cost, the reward function of this embodiment includes a completion time reward and a cost penalty term. The reward function calculation formula is as follows:

[0124]

[0125] wherein, represents the completion time reward, PC represents the cost penalty term;

[0126] Specifically, the calculation formula of the completion time reward is as follows:

[0127]

[0128] wherein, is the weight coefficient of the completion time optimization target, the value of and respectively represent the longest completion time in the state and the state s .

[0129] The calculation formula of the cost penalty term is as follows:

[0130]

[0131] wherein, is a weight coefficient of the cost reward optimization target, the value of is positive.

[0132] S3, formalize the task scheduling problem into a Markov decision process (MDP), and train using a deep Q network (DQN) model.

[0133] Formalize the task scheduling problem into a Markov decision process (MDP), and the MDP formalization of the task scheduling process is as follows:

[0134] wherein, S represents a state space; A represents a virtual machine instance that can be allocated to a task in an action space; P is a state transition function , represents a probability of transitioning to a new state s after performing an action in a state ; R represents a reward function; represents a discount factor; MDP formalization of the task scheduling process.

[0135] This embodiment uses a DQN (Deep Q-Network) algorithm in deep reinforcement learning to optimize the task scheduling strategy. The DQN algorithm includes two networks with the same structure: an evaluation network (Evaluation Network) and a target network (Target Network).

[0136] As shown in Figure 3 , Figure 4 training using a deep Q network (DQN) model includes the following steps:

[0137] S10, initialize the evaluation network using random network parameters ω;

[0138] S11, obtain a current state s , select an action s according to the current state using a policy, and perform the action ;

[0139] Since there is no historical experience to utilize in the initial stage of task scheduling, in order to maintain a balance between exploration and utilization, this embodiment uses an ( A greedy strategy selects actions. Exploration involves seeking new actions to discover potentially high-reward behaviors, while exploitation involves utilizing currently known information to maximize immediate rewards. Under the policy, the agent uses probability Randomly select an action to explore, and use 1- The probability is used to select the action with the highest Q value in the current state for exploitation. As the learning process progresses, The value can be gradually decreased so that it gradually tends to utilize known information as more experience is gained. In specific implementation, the experience replay module includes a memory space with a capacity of RM to store the data obtained from each exploration. When the number of samples in RM reaches a set value, a batch of data is taken from RM each time to train the DQN model.

[0140] Specifically, based on the current state s use Strategy selection action Includes the following steps:

[0141] S110, Set an exploration rate It is initialized to 1.

[0142] In the initial stage, the exploration rate will be at its maximum, meaning that the learning strategy will randomly select actions to explore a large number of times.

[0143] S111. At each decision-making stage, a random operator is generated. ,if If so, then the action with the highest Q value in the current state is selected and executed; if If the decision is not made, the exploration decision will continue, which means randomly selecting an action to perform.

[0144] S112. Based on the environmental feedback status after performing this action. and rewards r Update the Q value.

[0145] As training progresses, the exploration rate is gradually reduced at preset intervals. The value of allows the agent to utilize the strategy more extensively in the later stages of learning. In this embodiment, the exploration rate... Greater than 0 and less than or equal to 1.

[0146] S12, Execute the action Afterwards, receive immediate rewards for environmental feedback. r and the next state ;

[0147] S13, Change the current state s, the action performed , the immediate reward obtained r , and the next state forming a tuple and storing the tuple as an experience sample to an experience replay module; wherein the immediate reward r is calculated by the reward function;

[0148] S14, when the data amount of the experience samples reaches a preset value, randomly selecting experience samples from the experience replay module to train the evaluation network;

[0149] By experience replay and priority experience replay technology, the deep Q network is trained, the past experience is stored and replayed, the training efficiency and stability of the model are improved, and finally the task scheduling strategy is iteratively optimized until the globally optimal cost and completion time balance scheme is converged.

[0150] S15, calculating a predicted Q value based on the selected experience samples by using the evaluation network, and calculating a target Q value according to the selected experience samples by using the target network;

[0151] S16, calculating a loss function according to the predicted Q value and the target Q value, and updating the network parameters ω by using the gradient descent method;

[0152] The calculation formula of the loss function is as follows:

[0153]

[0154] wherein, is a discount factor, r is a reward function, s and represent the current state and the next state respectively, and represent the current action and the next action respectively, ω and represent the evaluation network parameters and the target network parameters respectively, E represents an expectation.

[0155] S17, iteratively training the evaluation network, and saving the network parameters ω obtained after the training ends;

[0156] During the iterative training of the evaluation network, the network parameters ω of the evaluation network are copied to the target network every preset number of steps.

[0157] By continuously training the task scheduling network, the task scheduling network parameters ω can be continuously corrected until the task scheduling network can learn a strategy to effectively reduce the completion time and cost of task scheduling.

[0158] After the training phase ends, the network parameters ω obtained by training are saved, so as to subsequently schedule the task by using the parameters.

[0159] In this embodiment, the Q value is updated by the following formula:

[0160]

[0161] wherein, is the current state, is the current action, represents the expected return (i.e., the Q value) of taking action in state , is the learning rate, represents the immediate reward obtained after taking action in state , is the discount factor, represents the maximum Q value among all possible actions in the next state .

[0162] S4, determining an optimal scheduling strategy of the task scheduling problem by using the trained DQN model, and allocating a corresponding virtual machine instance to the task to be scheduled according to the optimal scheduling strategy.

[0163] The cost and time-aware task scheduling method of this embodiment is tested. In the testing process, this embodiment uses Google Cluster Trace dataset and different sizes of datasets to simulate the workloads of the task scheduling system in a multi-cloud environment. These datasets are published by Google and contain the task tracking data of a Borg unit within 7 hours. The dataset includes tasks executed on a single machine, which consume memory and one or more cores, and multiple tasks can belong to a single job (e.g., mapper and reducer). In order to enhance the authenticity of the dataset and verify the effectiveness of the cost and time-aware task scheduling method of this embodiment, a screening and extraction process is adopted in the testing process to select different sizes of task datasets for testing.

[0164] Three cloud service providers (CSPs) are selected in this example: Amazon EC2, Microsoft Azure, and AlibabaCloud EC2, each of which is configured with three different performance virtual machine instance types for selection. In order to simulate real conditions, the computing power of these virtual machine instances is converted into MIPS (million instructions per second) using official documents. In this example, the bandwidth within the virtual communication network is 100 Mbps / s, while the communication bandwidth between different clouds is 20 Mbps / s. In actual implementation, the network learning rate of the algorithm is set to 0.01, the discount factor is set to 0.95, the capacity of the experience replay module is set to 10000, and the batch size is set to 16. During model training, the parameters of the target network are updated every time 30 transition tuples are stored in the experience replay module.

[0165] As shown in Figure 5 , the test results show that the cost and time-aware task scheduling method of the embodiment can achieve shorter completion time and lower cost on various data sets. On the Large data set, the completion time of the algorithm is reduced by 60.59% compared to the RR algorithm, significantly improving the efficiency and effectiveness of task scheduling. In addition, the task scheduling method of the embodiment can achieve the lowest rental cost on four data sets. For example, in the case of 1000 scheduling tasks, the task scheduling method of the embodiment saves 39.87% of the rental cost compared to the GA algorithm. As the number of tasks increases, the performance of the task scheduling method of the embodiment in cost optimization continues to improve, with outstanding performance on the Google Cluster Data data set.

[0166] The technical solution of the cost and time-aware task scheduling method provided by the present application will be described below in conjunction with a specific embodiment.

[0167] As shown in Figure 6 , the present embodiment also provides a cost and time-aware task scheduling device 610, which is applied to a multi-cloud environment including at least two cloud service providers (CSPs), each of which includes a plurality of virtual machine instances. Optionally, the cost and time-aware task scheduling device 610 can include:

[0168] a task submission module 611 for submitting a task to be scheduled and sorting the task to be scheduled;

[0169] ​​​The information obtaining module 612 is configured to obtain task information to be scheduled and multi-cloud environment information, determine a state space, an action space and a reward function according to the task information to be scheduled and the multi-cloud environment information, and the reward function includes a completion time reward and a cost penalty term.

[0170] The DQN model training module 613 is configured to formalize the task scheduling problem as a Markov decision process (MDP) and train a deep Q network (DQN) model.

[0171] The resource allocation module 614 is configured to determine an optimal scheduling strategy of the task scheduling problem by using the trained DQN model, and allocate a corresponding virtual machine instance to the task to be scheduled according to the optimal scheduling strategy.

[0172] It can be understood that the above-mentioned device embodiments and the above-mentioned method embodiments can correspond to each other, and similar descriptions of the device embodiments can be referred to the method embodiments. To avoid repetition, it will not be described here. The cost and time-aware task scheduling device provided in the embodiments of the present application can execute the cost and time-aware task scheduling method provided in any embodiment of the present application, and has the corresponding function modules and beneficial effects of executing the method. The function modules of the cost and time-aware task scheduling device can be realized by hardware, can be realized by software instructions, and can also be realized by a combination of hardware and software modules.

[0173] Specifically, the steps of the method embodiments of the present application can be completed by integrated logic circuits of hardware in the processor and / or software instructions. The steps of the cost and time-aware task scheduling method in the embodiments of the present application can be directly embodied as hardware coding processor execution completion, or executed by a combination of hardware and software modules in the coding processor. Alternatively, the software module can be located in a random memory, a read-only memory, a programmable read-only memory, a flash memory, an electrically erasable programmable memory, a register, and the like storage medium. The storage medium is located in the memory, and the processor reads the information in the memory, and combines the hardware to complete the steps in the above-mentioned method embodiments.

[0174] The electronic device 710 provided in the embodiments of the present application has the structure as shown in Figure 7 The electronic device 710 can be the server 100 or the terminal 200 shown in the embodiments of the present application. Figure 1

[0175] As shown in Figure 7 The electronic device 710 includes a memory 711, a processor 712, a communication module 713, an input / output interface 714, and the like. Optionally, the memory 711, the processor 712, the communication module 713 and the input / output interface 714 can be connected and communicated through the bus 715.​

[0176] The memory 711 is configured to store one or more computer programs and transmit codes of the computer programs to the processor 712; when the one or more computer programs are executed by the processor 712, the cost and time-aware task scheduling method in the embodiments of the present application is implemented.

[0177] Optionally, the electronic device 710 can be connected to a network through the communication module 713, so as to communicate with other devices such as terminals or servers through the network, and realize the interaction of data. The electronic device 710 can be various forms of digital computers, such as desktop computers, servers, workstations, mainframe computers or other types of computers. The electronic device 710 can also be various forms of mobile terminals, such as smart phones, tablet computers, wearable devices (such as helmets, glasses, watches, etc.) and other similar mobile terminals.

[0178] Optionally, the electronic device 710 can connect the required input / output devices such as keyboards, display devices, etc. through the input / output interface 714, and the electronic device 710 itself can have a display device, and can also be externally connected to other display devices through the input / output interface 714. Optionally, the storage device such as hard disk, etc. can be connected through the input / output interface 714, so that the data in the electronic device 710 can be stored in the storage device, or the data in the storage device can be read, and the data in the storage device can also be stored in the memory 711. It can be understood that the input / output interface 714 can be a wired interface or a wireless interface. According to different actual application scenarios, the devices connected with the input / output interface 714 can be a component of the electronic device 710, or can be an external device connected with the electronic device 710 when needed.

[0179] Optionally, the memory 711 can be a volatile memory and / or a non-volatile memory, the volatile memory can be a random access memory, etc., and the non-volatile memory can be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory or a flash memory, etc.

[0180] Optionally, the computer program stored in the processor 712 can be divided into one or more modules, the one or more modules are stored in the memory 711 and executed by the processor 712 to complete the method provided by the embodiments. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, and the computer program instruction segments are used to describe the execution process of the computer program in the electronic device 710.

[0181] Optionally, the processor 712 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 712 include, but are not limited to, a central processing unit, a graphics processing unit, a digital signal processor, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, and can also be any appropriate controller, microcontroller, processor, etc. The processor 712 executes various methods and processes of the embodiments, for example, a cost and time-aware task scheduling method of the embodiments.

[0182] Optionally, the bus 715 can include a path for transmitting information. The bus 715 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. According to different functions, the bus 715 can be divided into an address bus, a data bus, a control bus, etc.

[0183] In an optional implementation, the embodiments also provide a computer storage medium having a computer program stored thereon, which enables a computer to execute the method of the method embodiments when executed by the computer. Part or all of the computer program can be loaded and / or installed on the memory 711 of the electronic device 710. When the computer program is executed by the processor 712, one or more steps of a cost and time-aware task scheduling method of the embodiments can be executed.

[0184] Optionally, the computer-readable storage medium can be a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, etc.

[0185] Obviously, the above-described embodiments of the present application are only examples for clearly illustrating the technical solutions of the present application, and are not intended to limit the specific embodiments of the present application. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application should be included in the protection scope of the claims of the present application.

Claims

1. A cost and time-aware task scheduling method, said method being applied to a multi-cloud environment, said multi-cloud environment comprising at least 2 cloud service providers, CSPs, each of said cloud service providers, CSPs, comprising a number of virtual machine instances, characterized in that, The method comprises: obtaining a to-be-scheduled task, and sorting the to-be-scheduled task; obtaining to-be-scheduled task information and multi-cloud environment information, determining a state space, an action space and a reward function according to the to-be-scheduled task information and the multi-cloud environment information; the reward function comprises a completion time reward and a cost penalty term; formalizing a task scheduling problem as a Markov decision process (MDP), and training by using a deep Q network (DQN) model; determining an optimal scheduling strategy of the task scheduling problem by using the trained DQN model, and allocating a corresponding virtual machine instance to the to-be-scheduled task according to the optimal scheduling strategy; A plurality of the virtual machine instances form a virtual machine instance set The virtual machine instance set is expressed as: wherein, represents a first cloud service provider, CSP, represents a first virtual machine instance, represents a first virtual machine instance on a first cloud service provider, CSP, represents a set of virtual machine instances in a multi-cloud environment; Supposing that there are tasks to be scheduled, the tasks to be scheduled are represented as: wherein, represents the i th task to be scheduled; Basic characteristics of each of the virtual machine instances is represented as: wherein, represents the computing power of represents the number of virtual CPUs of represents the memory size of represents the unit time rental cost of Task In the time of execution on the computer is calculated as follows: wherein, indicates the size of the task ; Task The longest completion time of the task is expressed as: wherein, represents the longest completion time of a task . Task The cost of performing the task on the is represented as: wherein, represents the billing mechanism of the cloud service provider CSP. Task The total cost of execution is expressed as: wherein, indicates a task is executed on , if = 1 indicates that a task is executed on , if = 0 indicates that a task is not executed on ; the reward function calculation formula is as follows: wherein, represents a completion time reward, PC represents a cost penalty term; The completion time reward The calculation formula is as follows: wherein, is a weight coefficient for the makespan optimization objective, is a positive number; and represent the longest makespan in state and state s respectively. the cost penalty term calculation formula is as follows: wherein is a weight coefficient for the cost reward optimization objective, is a positive number.

2. The cost and time aware task scheduling method of claim 1, wherein, The state space includes virtual machine instance states and task states, each state information in the state space s is defined as: wherein, s represents any state information in the state space S ST represents the current state information of the task, ST =0 represents waiting for scheduling, ST =1 represents being scheduled, ST =2 represents having completed scheduling; ASR represents the virtual machine instance available in the virtual machine instance set M represents the completion time under the current state of the task; CA represents the running cost of the virtual machine instance under the current state.​​ 3. The cost and time aware task scheduling method of claim 1, wherein, the action space comprises a set of all executable actions of an agent of the DQN model in any state, the action represents selecting and executing an action from the action space, and the action space is represented as follows: wherein, A represents the action space; for the action , represents the selection of assigning the current task to and executing; hold represents performing a wait in case of full load or other exceptional situations.

4. The cost and time aware task scheduling method of claim 1, wherein, the DQN model comprises an evaluation network and a target network, and the training by using the deep Q network (DQN) model comprises the following steps: initializing the evaluation network using random network parameters ω; acquire a current state s , according to the current state s adopt a strategy to select an action , and execute the action ; Performing an action After, acquiring an immediate reward from the multi-cloud environment feedback r And a next state ; the current state s , the action performed , the immediate reward obtained r , and the next state form a tuple and are stored as an experience sample to an experience replay module; wherein the immediate reward r is computed by the reward function when the data amount of the experience samples reaches a preset value, randomly selecting experience samples from the experience replay module to train the evaluation network; calculating a predicted Q value based on the selected experience samples by using the evaluation network, and calculating a target Q value according to the selected experience samples by using the target network; calculating a loss function according to the predicted Q value and the target Q value, and updating the network parameters ω by using a gradient descent method; iteratively training the evaluation network, and saving the network parameters ω finally obtained after training; In the iterative training and evaluation of the network, the network parameters of the evaluation network are copied to the target network every preset number of steps to the target network.

5. The cost and time aware task scheduling method of claim 4, wherein, The current state s Adopting Strategy selection action Including the following steps: Set an exploration rate initialized to 1; At each decision, a random number in the range [0, 1] is generated randomly If , then a exploitation decision is performed, i.e. the action with the highest Q-value in the current state is selected for execution; if , then an exploration decision is performed, i.e. a random action is selected for execution; Every preset time, the exploration rate is gradually reduced the value of the parameter.

6. A cost and time-aware task scheduling apparatus, said apparatus is applied to a multi-cloud environment, said multi-cloud environment comprises at least 2 cloud service providers (CSPs), each of said cloud service providers (CSPs) comprises a number of virtual machine instances, characterized in that, The device comprises: a task submission module configured to submit a to-be-scheduled task and sort the to-be-scheduled task; an information acquisition module configured to acquire to-be-scheduled task information and multi-cloud environment information, and determine a state space, an action space and a reward function according to the to-be-scheduled task information and the multi-cloud environment information; the reward function comprises a completion time reward and a cost penalty term; a DQN model training module configured to formalize a task scheduling problem as a Markov decision process (MDP), and train by using a deep Q network (DQN) model; a resource allocation module configured to determine an optimal scheduling strategy of the task scheduling problem by using the trained DQN model, and allocate a corresponding virtual machine instance to the to-be-scheduled task according to the optimal scheduling strategy; A plurality of the virtual machine instances form a virtual machine instance set The virtual machine instance set is expressed as: wherein, represents a first cloud service provider CSP, represents a first virtual machine instance, represents a first virtual machine instance on a first cloud service provider CSP, represents a set of virtual machine instances in a multi-cloud environment; Supposing that there are tasks to be scheduled, the tasks to be scheduled are represented as: wherein, represents the i th task to be scheduled; Basic characteristics of each of the virtual machine instances is represented as: wherein, represents the computing power of represents the number of virtual CPUs of represents the memory size of represents the unit time rental cost of Task In the time of execution on the computer is calculated as follows: wherein indicates the size of the task ; Task The longest completion time of the task is expressed as: wherein, represents the longest completion time of a task ; Task The cost of performing the task on the is represented as: wherein, represents the billing mechanism of the cloud service provider CSP. Task The total cost of execution is expressed as: in, Indicates task Is it in If executed above, = 1 indicates a task exist If executed above, = 0 indicates a task Not here Execute above; the reward function calculation formula is as follows: wherein, represents a completion time reward, PC represents a cost penalty term; The completion time reward The calculation formula is as follows: wherein, is a weight coefficient for the makespan optimization objective, is a positive number; and represent the longest makespan in state and state s respectively. the cost penalty term calculation formula is as follows: wherein is a weight coefficient for the cost reward optimization objective, is a positive number.

7. An electronic device, comprising: comprises: a memory configured to store one or more computer programs; a processor configured to implement the cost and time-aware task scheduling method according to any one of claims 1-5 when the one or more computer programs are executed by the processor. 8.A computer readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to implement the cost and time-aware task scheduling method according to any one of claims 1-5 when the computer instructions are executed by the processor.

Images