Multi-target task scheduling method, device and equipment based on cloud system, medium and product

Abstract

The present disclosure provides a multi-target edge task scheduling method, device, equipment, medium and product, which can be applied to the technical field of data scheduling. The method comprises: constructing an edge cloud system model, the edge cloud system model comprising an edge computing architecture and a task scheduling strategy, the edge cloud system model being used to execute a current task, the optimization objective being to minimize the failure rate of the task, the energy consumption of the task and the time delay of the task, constructing a task scheduling model, the task scheduling model being constructed based on a deep reinforcement learning model, and executing the task based on the task scheduling strategy output by the task scheduling model by using the edge computing architecture, which can improve the task execution efficiency.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of task scheduling, and particularly relates to a multi-objective edge task scheduling method, device, equipment, medium and product. BACKGROUND

[0002] At present, deep reinforcement learning (DRL) in edge computing task scheduling method has a wide range of applications in edge cloud computing scenarios.

[0003] Deep reinforcement learning is an artificial intelligence learning method that simulates the process of continuous learning and trial and error to achieve self-improvement and optimization. The basic process includes observing the state of the environment, taking action, obtaining reward, and updating the policy according to the reward. Deep reinforcement learning has good adaptability and scalability in solving task scheduling problems in edge cloud environments: DRL models can adapt to dynamic environments, allowing them to handle changes in task quantity, computing node state, and edge resources, effectively managing large-scale edge computing networks and making effective decisions in uncertain states. DRL algorithms can learn to optimize complex goals involving multiple conflicting goals, and have good results in edge computing task scheduling problems that require simultaneous consideration of multiple goals. It can simultaneously learn from experience obtained from interaction with the environment and online learning, and this real-time learning capability makes DRL algorithms very suitable for solving edge computing task scheduling problems where the environment is constantly changing. Among them, Deep-Q-Network (DQN) combines deep neural networks (DNNs) with Q-learning algorithms, uses neural networks to represent and learn Q functions, i.e. action value functions, which represent the quality of an agent performing a certain action in a certain state. Experience replay technology is used to store the experiences of the agent in a replay buffer and randomly sample samples for training, and a target network is introduced to stabilize the training process. During the training process, the agent explores and utilizes the policy balance to gradually update the weights of the neural network to approximate the optimal policy. DQN is widely used to solve problems with a large number of states and action spaces, and can find the optimal task allocation strategy in a dynamic edge environment.

[0004] ​For multi-objective reinforcement learning (MORL), Deng Shi-quan et al. proposed a multi-objective task offloading algorithm based on DQN, which considered the sequence relationship between tasks, established a multi-objective task offloading model with the optimization goal of minimizing task delay and energy consumption, and used the complete trajectory generated in one round instead of the transition sample of one time step in the original DQN as the minimum unit of the experience pool to improve the original DQN algorithm. Different fixed weights are used for different objectives, and the simulation results show that this algorithm is superior to the original DQN algorithm and the multi-objective evolutionary algorithm based on decomposition. Tajmajer et al. use a separate DQN to control the agent's behavior for a specific goal, and introduce a decision value that allows the agent to change the priority of a specific goal after learning while maintaining the overall performance of the agent. The algorithm is applied to simulate the behavior of an automatic vacuum cleaner, allowing the vacuum cleaner to automatically adjust the priority of cleaning dust and avoiding collision with the wall, and experiments show that compared to assigning the same priority to different optimization goals, the overall performance of the agent guided by this algorithm is better.

[0005] Task scheduling in edge computing is an online NP-hard problem, which is suitable for deep reinforcement learning. For mobile edge computing (MEC) environment, Quan et al. proposed a task offloading scheduling and resource allocation algorithm for multiple users and multiple tasks, aiming to minimize system delay and transmission energy consumption, and local energy consumption. Simulation results show that the proposed method has superior performance in reducing delay and local energy consumption. Zheng et al. proposed a DQN workload scheduling method that balances workload, reduces service time and task failure rate, combines deep neural networks with Q-learning, and solves the complexity and high dimensionality of workload scheduling problems, mainly reducing service time and improving virtual machine (VM) utilization. Tang et al. proposed a distributed algorithm combining long short-term memory (LSTM) and dueling deep Q-network (Dueling-DQN) to solve the task offloading problem in mobile edge computing systems, allowing each device to determine its offloading decision without knowing the task model and offloading decision of other devices, improving the compatibility of mobile edge computing systems.

[0006] The following difficulties exist in task scheduling in edge cloud environment:

[0007] 1. The hardware and software structures of the numerous devices in the edge cloud computing platform are different. On the one hand, the heterogeneity of devices and the limitation of resources will lead to different execution efficiencies of tasks on different devices. On the other hand, the compatibility of computing nodes will lead to poor portability of tasks. If a task needs to access a certain hardware resource, and such hardware resource does not exist in some devices, the task can not be executed on these devices, which makes the scheduling of tasks between different devices difficult.

[0008] 2. The edge cloud network conditions are complex and low in stability. First, edge computing nodes are usually distributed, and due to the mobility of devices, network access is unstable, and devices exist in the cases of accessing and moving out of the network. The dynamicity of computing resources increases the complexity of resource scheduling due to the randomness of task arrival and the difference in workloads between application programs. On the other hand, network bandwidth fluctuation leads to changes in transmission rate between devices, differences between task execution time and expectation, and the need for rescheduling.

[0009] Existing researches on the task scheduling problem in the edge cloud environment mainly focus on mobile devices and mobile networks, consider the multi-user MEC offloading scene, and aim to minimize energy consumption or delay. There is a lack of research on maximizing the reliability of tasks in highly dynamic environments and for the above multi-objective optimization. SUMMARY

[0010] In view of the above problems, the present disclosure provides a multi-objective edge task scheduling method, device, equipment, medium and program product, which trains an intelligent agent using multi-objective reinforcement learning, and dynamically selects the weight of the optimization target according to the environment. Considering the different computing capabilities of edge computing nodes, task delay, task execution success rate and device energy consumption and other factors are used as optimization targets, and a variety of test scenarios are randomly generated to evaluate the performance of the algorithm. In terms of task delay, execution success rate and device energy consumption, the algorithm is better than the ordinary DQN algorithm and Double-DQN algorithm.

[0011] According to a first aspect of the present disclosure, a multi-objective edge task scheduling method is provided, comprising:

[0012] An edge cloud system model is constructed, which includes an edge computing architecture and a task scheduling strategy, and is used to execute a current task;

[0013] A task scheduling model is constructed with the optimization targets of minimizing the failure rate of the task, the energy consumption of the task and the time delay of the task, and the task scheduling model is constructed based on a deep reinforcement learning model;

[0014] The task is executed based on the task scheduling strategy output by the task scheduling model using the edge computing architecture.

[0015] According to an embodiment of the present disclosure, the edge computing architecture comprises a local device, an edge server and a data center cloud host;

[0016] When the computing resources and / or computing capability of the local device do not meet preset requirements, the task is scheduled to an edge server or a data center cloud host adjacent to the local device for processing;

[0017] When the task is processed by the local device, a preset local computing model is used to calculate the time delay of the task and the energy consumption of the task;

[0018] When the task is processed by the edge server, a preset edge computing model is used to calculate the time delay of the task and the energy consumption of the task;

[0019] When the task is processed by the data center cloud host, a preset computing offloading model is used to calculate the time delay of the task and the energy consumption of the task.

[0020] According to an embodiment of the present disclosure, the time delay of the task is minimized :

[0021]

[0022] wherein the transmission time delay and the execution time delay of the i-th task are respectively , the attribute of the i-th task is , denotes the input data size of the i-th task, denotes the output data size of the i-th task, denotes the total number of CPU cycles required to complete the i-th task, denotes the time delay constraint of the i-th task, and the task scheduling decision vector is , ∈{0,1}, the value of the scheduling target in the task scheduling decision vector is 1, and the value of the non-scheduling target is 0, and N is the number of edge servers, respectively denote the time delay of the i-th task at the local device, the N-th edge server and the data center cloud host, represents the computing capability of the n-th device, and the unit is the number of CPU cycles per second;

[0023] is the time constraint for the successful execution of the i-th task, and the sum of the transmission time delay and the processing time delay of the i-th task is less than the time delay constraint of the task, which is considered as successful execution, otherwise it is considered as task execution failure;

[0024] performing the ith task at the local device, performing the ith task at the edge server, or performing the ith task at the data center cloud host, denotes that the ith task is performed at the local device, denotes that the ith task is performed at the edge server, denotes that the ith task is performed at the data center cloud host;

[0025] C3 is the sum of the computing power required for selecting the local computing model and the edge computing model, which does not exceed the total computing power of the local device or the edge server, and the computing power of each edge server is ;

[0026] Minimize the energy consumption E of the task: wherein, denotes the energy consumption of the ith task performed on the local terminal, the edge server, and the data center cloud host, respectively.

[0027] According to an embodiment of the present disclosure, the method comprises:

[0028] Based on the Markov decision process, the deep reinforcement learning model is constructed, and the Markov decision process comprises a learning state space S, an action space A, and a reward function R(t);

[0029] wherein, , represents the state of the network at time t, represents the state of the edge server, comprises the computing power of the edge server and the data center cloud host at time t and the connectivity of the edge server, wherein represents the computing power of the edge server at the current time, , represents the state of the node, when the node is working normally, and when the node is disconnected, represents the information of the task that needs to be scheduled at time t , and the node is a device in the edge cloud system model;

[0030] Action space , denotes that the ith task is calculated at the local device, denotes the operation of scheduling the ith task to one of the adjacent edge servers, , denotes the operation of scheduling the ith task to the data center cloud host;

[0031] reward function ;

[0032]

[0033]

[0034] is a delay reward function at the t th moment, E(t) is an energy consumption reward function at the t th moment, when the constraint indicates that the i th task can be completed within the maximum delay constraint, and the reward is defined as the difference between the maximum delay constraint and the actual execution time, indicates that the sum of the transmission delay and the execution delay of the i th task in the unstable environment of the edge cloud system model is measured according to the average delay, indicates the delay constraint corresponding to the i th task, is the initial energy consumption, E max is the maximum energy consumption, E i is the energy consumption of executing the i th task, is a constant.

[0035] According to an embodiment of the present disclosure, the method further comprises:

[0036] When setting the reward function of the deep reinforcement learning model, the rewards of the optimization objectives of the failure rate of the task, the energy consumption of the task and the delay of the task are normalized, so that the reward values of the two different optimization objectives are located in the same interval.

[0037] According to an embodiment of the present disclosure, the method further comprises:

[0038] When the delay reward function of the task approaches the minimum value, the weight of the optimization objective of the delay of the task is increased;

[0039] When the energy consumption reward function is in a negative value, the weight of the optimization objective of the energy consumption of the task is increased.

[0040] A second aspect of the present disclosure provides a multi-objective edge task scheduling device, comprising:

[0041] A first construction module is configured to construct an edge cloud system model, wherein the edge cloud system model comprises an edge computing architecture and a task scheduling strategy, and the edge cloud system model is used to execute a current task;

[0042] A second construction module is configured to construct a task scheduling model with the optimization objectives of minimizing the failure rate of the task, the energy consumption of the task and the delay of the task, wherein the task scheduling model is constructed based on a deep reinforcement learning model.

[0043] An execution module is configured to execute the task based on the task scheduling strategy output by the task scheduling model using the edge computing architecture.

[0044] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0045] A fourth aspect of the present disclosure further provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above method.

[0046] A fifth aspect of the present disclosure further provides a computer program product comprising a computer program that, when executed by a processor, implements the above method.

[0047] According to the multi-target edge task scheduling method, device, equipment, medium and program product provided by the present disclosure, on the one hand, the characteristics of the edge cloud environment are comprehensively considered, the dynamic nature of the computing nodes in the edge cloud environment, the heterogeneity of the edge computing resources and other characteristics are quantitatively represented in the model, and the task scheduling problem is converted into a multi-target optimization problem for task delay and energy consumption based on the characteristics of the node usage state and the to-be-scheduled task. On the other hand, the DQN model is established, and the dynamic weight adjustment is performed on the q value calculated after the reward is normalized, and the operation is performed on the immediate feedback of each step, so that the system allocates different importance to multiple optimization targets in different states, so that the system can preferentially meet the targets with high importance, and the situation of negative reward, i.e. task scheduling failure, occurs less, thereby improving the success rate of task scheduling. BRIEF DESCRIPTION OF DRAWINGS

[0048] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure taken in conjunction with the accompanying drawings, in which:

[0049] Figure 1 A schematic diagram of an edge computing architecture according to an embodiment of the present disclosure is schematically shown;

[0050] Figure 2 A flowchart of a multi-target edge task scheduling method according to an embodiment of the present disclosure is schematically shown;

[0051] Figure 3 A schematic diagram of a deep reinforcement learning model according to an embodiment of the present disclosure is schematically shown;

[0052] Figure 4 A schematic diagram of the completion rate of three task scheduling methods according to an embodiment of the present disclosure is schematically shown;

[0053] Figure 5 A schematic diagram of latency of tasks of three task scheduling methods according to embodiments of the present disclosure is shown;

[0054] Figure 6 A schematic diagram of energy consumption of tasks of three task scheduling methods according to embodiments of the present disclosure is shown;

[0055] Figure 7 A block diagram of a multi-objective edge task scheduling apparatus according to embodiments of the present disclosure is shown;

[0056] Figure 8 A block diagram of an electronic device suitable for implementing a multi-objective edge task scheduling method according to embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0057] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It is to be understood, however, that the description is merely exemplary of the present disclosure, and is not intended to limit the scope of the present disclosure. In the following detailed description of the embodiments of the present disclosure, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it would be apparent to one skilled in the art that the present disclosure can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present disclosure.

[0058] The terms used herein are merely used to describe specific embodiments, and are not intended to limit the present disclosure. The terms "include", "comprise" and the like used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0059] All terms used herein, including technical and scientific terms, have the same meanings as those generally understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the present description, and should not be interpreted in an idealized or excessively formal manner.

[0060] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should generally be interpreted to include at least one of A, B, and C, etc. (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, a system having B alone, a system having C alone, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having A, B, and C together, etc.).

[0061] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information comply with relevant laws and regulations, necessary security measures are taken, and the public order and good customs are not violated.

[0062] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information comply with relevant laws and regulations, necessary security measures are taken, and the public order and good customs are not violated.

[0063] Edge cloud is a cloud computing platform based on the core of cloud computing technology and the capability of edge computing, which is built on edge infrastructure, and places computing resources (such as servers and storage) in a location closer to the device that generates data to ensure faster data processing, reduce latency, and improve overall efficiency of data processing in various applications, with characteristics such as distributed architecture, scalability and elasticity, security and privacy, and heterogeneous computing, as shown in Figure 1

[0064] In this scenario, there are a large number of mobile computing nodes in the edge cloud environment. Due to the dynamic nature of nodes, network congestion and interference, etc., the network topology changes and the link quality changes. In order to maintain high reliability in this case, the system must be able to adapt to the frequent changes of network topology, handle intermittent connections, and ensure data transmission and communication between nodes. Given the optimization requirements of the dynamic nature of computing nodes, the heterogeneity of edge computing resources, and the geographical dispersion of processors, new challenges are formed for resource allocation and task scheduling optimization, so it is necessary to use a reasonable task scheduling strategy to ensure the effectiveness of resource use and the service experience of cloud applications. In edge computing task scheduling, the usual goals of task schedulers are to minimize service latency, maximize resource utilization, improve user experience quality, and reduce energy consumption, etc. This requires considering many factors, including the type of task, the location of data, the resource limitations of edge devices, and network bandwidth, etc.

[0065] The present application sets the failure rate of the task, the energy consumption of the edge computing architecture and the task latency as the optimization target, further normalizes the Q value to the reward of each round, replaces the fixed weight with the dynamic weight, and based on the Multi-Objective Deep Q Network (MODQN) algorithm, calculates the factor that has the greatest impact on the success of the current task through the constraint condition, and improves the priority of the factor (latency or energy consumption) through the dynamic weight, so that the model can minimize the task failure rate and the total cost of task execution under the constraints of network conditions, task latency and computing resources.

[0066] Figure 2 ​A flowchart of a multi-target edge task scheduling method according to an embodiment of the present disclosure is shown schematically.

[0067] As shown in Figure 2 , the multi-target edge task scheduling method of this embodiment includes operations S210-S230.

[0068] In operation S210, an edge cloud system model is constructed, which includes an edge computing architecture and a task scheduling strategy, and is used to execute a current task.

[0069] In operation S220, a task scheduling model is constructed based on a deep reinforcement learning model, with the optimization goal of minimizing the failure rate, energy consumption and time delay of the task.

[0070] In operation S230, the task is executed based on the task scheduling strategy output by the task scheduling model using the edge computing architecture.

[0071] In the present disclosure, the edge computing architecture includes a local device, an edge server and a data center cloud host. Generally, a task is usually executed on a local device to minimize transmission delay. When the local device cannot provide sufficient computing resources or computing power, the corresponding task should be scheduled to a nearby edge server or data center cloud host for processing. The computing power of the local device is ; the number of edge servers is N, the computing power of the nth edge server is , the failure probability of the nth edge server is ; the number of data center cloud hosts is 1, and the computing power is F.

[0072] Task scheduling strategy: assuming that tasks are indivisible and independent of each other, they are offloaded as a whole to an edge server or a data center cloud host for execution. The attributes of each task are represented by a four-tuple: , where represents the input data size of the ith task, represents the output data size (bits) of the ith task, represents the total number of CPU cycles required to complete the ith task, represents the time delay constraint (maximum allowable time delay) of the ith task. The task scheduling decision vector is , ∈{0,1}, the value of the scheduling target in the task scheduling decision vector is 1, and the rest is 0, represents that the task is executed on the local device, represents that the task is executed on the edge server, , represents that the task is processed on the data cloud center.

[0073] Parameter Meaning System bandwidth Number of edge servers Transmission power of edge devices D Distance between devices Channel gain of device n in the wireless channel Complex Gaussian channel noise power Local device computing power ， Computing power of the nth device, in units of CPU cycles per second ， Damage probability of the nth device Data center cloud host computing power ， Task input data size (bits) ， Task output data size (bits) ， Total number of CPU cycles required ， latency constraints

[0074] In the present disclosure, in the case where the task is processed by the local device, the time delay of the task is calculated by using the preset local computing model. If the calculation is performed on the local mobile device, the transmission time delay can be ignored, and the task execution time delay mainly includes the local CPU processing time delay. If the node state changes when the task is executed, the task needs to be rescheduled. The time delay of the i th task on the local device is calculated by using the local computing model , the energy consumption of the i th task executed on the local device . Wherein, represents the energy consumption of each CPU cycle of the i th task executed on the local device: .

[0075] In the present disclosure, in the case where the task is processed by the edge server, the time delay of the task is calculated by using the preset edge computing model. The task is uploaded to the surrounding edge server for calculation, and the time delay of the i th task on the edge server includes the data transmission time delay and the task processing time delay: .

[0076] Assuming that the wireless channel is symmetric, according to the Shannon formula, the transmission bandwidth between the edge servers is as follows, wherein is the system bandwidth, is the transmission power of the edge server, which can be not considered in the application layer and regarded as a constant, is the channel gain between the edge servers, which is related to the distance between the devices, is the noise power:

[0077]

[0078] According to the transmission bandwidth, the transmission time delay of the i th task is: , and the execution time delay of the i th task is: .

[0079] is the running power of the edge server, and the energy consumption of executing the i th task is .

[0080] At the same time, when the node state changes, the task needs to be redistributed, and the time delay cost of the redistribution is the sum of the time required for the task data to be transmitted from the device to the redistributed node and the execution time on the redistributed node:

[0081] ,

[0082] The time delay of the ith task at the edge server is: .

[0083] In the present disclosure, the time delay of the task is calculated by using the preset computing offloading model when the task is processed by the data center cloud host. The input data needs to be uploaded to the data center cloud host through the wireless access network, and the computing task completed by the data center cloud host by allocating part of the computing resources, and finally the execution result is returned to the computing device by the data center cloud host.

[0084] The transmission time delay of the data transmission between the device and the data center cloud host, that is, the transmission time delay of the ith task is: The execution time delay of the ith task is: .

[0085] The transmission energy consumption of the ith task to the data center cloud host is: .

[0086] The execution energy consumption of the ith task is: .

[0087] The time delay of the ith task is: .

[0088] The energy consumption of the ith task is: .

[0089] In an embodiment of the present disclosure, according to the time delay of the task and the energy consumption of the task when the local device executes the task, the time delay of the task and the energy consumption of the task when the edge server executes the task, and the time delay of the task and the energy consumption of the task when the data center cloud host executes the task, the optimization objective of the present disclosure to the task scheduling problem is to minimize the failure rate of the task, the energy consumption of the edge computing architecture and the time delay of the task. The multi-objective optimization problem is expressed as follows:

[0090] Minimize the task processing time delay :

[0091]

[0092] Wherein, represents the constraints of the task execution conditions and parameters. is the time constraint for the successful execution of the task, and the sum of the transmission time delay and the processing time delay of each task is less than the time delay constraint of the task. In order to execute successfully, otherwise, it is considered as task execution failure, which needs to be rescheduled; is the task executed at the local device, executed at the edge server or executed at the data center cloud host, indicates that the task is executed at the local device, indicates that the task is executed at the edge server, represents that the task is executed on the data center cloud host; C3 is the sum of the computing power required by the tasks selected by the local computing model and the edge computing model, which does not exceed the total computing power of the local device or the edge server.

[0093] Minimize the energy consumption E of the task: .

[0094] Above, the task scheduling process is constructed as Markov decision processes (MDP), an agent with multiple objectives is set and rewards are obtained according to the objectives, the rewards in different units of measurement are normalized and a dynamic weight setting is used to control the influence of the weight of each optimization objective on the agent. Based on the Markov decision process, a deep reinforcement learning model is constructed, which includes a state space S, an action space A and a reward function R(t).

[0095] Establish a learning state space S:

[0096] ,

[0097] wherein, represents the state of the network at time t, represents the state of the edge server, includes the computing power and edge server connectivity of the edge server and the data center cloud host at time t, wherein represents the computing power of the edge server at the current time, , represents the state of the node, when the node is working normally, when the node is disconnected, represents the information of the task that needs to be scheduled at time t , the node is a device in the edge cloud system model, such as a local device, an edge server and a data center cloud host.

[0098] Establish an action space A:

[0099] The action is the decision made by the agent, which determines the transfer rule between states. The agent gives the corresponding action according to the current state of the environment, and when the task arrives, it needs to select a suitable server to execute the task, so the action space includes all edge devices and data center cloud hosts, which represents the allocation of tasks on each device at time t: ; for a given task, the operation space will be represented as a set of discrete choices, each choice corresponds to assigning the task to a specific server, wherein, represents local computing of the task, an operation representing assigning the task to one of the nearby edge servers, an operation representing assigning the task to a data center cloud host.

[0100] The reward function R(t):

[0101] The reward function is the standard used by the agent to evaluate the goodness of the action, which is used to describe the immediate reward from one state to another after taking action, so the designed reward function needs to be closely related to the optimization goal. First of all, the optimization goal needs to be determined, and then the positive reward or negative reward is judged according to the goal.

[0102] In the edge cloud environment, it is mainly due to unreasonable task allocation, resulting in long task waiting execution or transmission time, or task failure due to network interruption and node damage. The optimization goal is to minimize the task processing delay and energy consumption under the constraint of ensuring the success of task execution. First of all, based on the local computing model, the edge computing model and the computing offloading model, the total cost of each task is obtained, including the task delay and the task energy consumption:

[0103]

[0104]

[0105]

[0106] When the execution decision satisfies the constraint , it means that the ith task can be completed within the maximum delay constraint, and the experimental reward is defined as the difference between the maximum delay constraint and the actual execution time, denotes the delay of the ith task, denotes the delay constraint corresponding to the ith task. The size of the reward is measured with reference to the average value, and the reward for energy consumption is the difference between the maximum energy consumption and the current task energy consumption and is normalized, The initial energy consumption is 0, E max is the maximum energy consumption, E i is the energy consumption of executing the ith task. If is not satisfied, the task completion time exceeds its deadline, which means that the decision is unacceptable, and the reward is defined as a negative value, which is usually set to -5.

[0107] wherein, specifically, denotes the sum of the transmission delay and the execution delay of the ith task in the unstable environment of the edge cloud system model according to the average value of the delay. When the ith task is executed on the local device, the transmission delay is approximately

[0108] The above establishes an agent with multiple goals, which receives rewards related to these goals, and each goal corresponds to a Q function. To improve the adaptability of the algorithm to the environment, the more important goal in a specific state needs to be dynamically selected, and adaptive weight adjustment is used instead of fixed weights to weight the rewards of different goals.

[0109] The present disclosure normalizes the rewards of different optimization goals when setting the reward function of the deep reinforcement learning model, so that the reward values of two different optimization goals are in the same interval. The operation is performed for the immediate feedback of each step, which reduces the reward value difference between different goals and improves the perception of numerical changes of the algorithm.

[0110] In an optional embodiment, the multi-goal dynamic weight adjustment method can dynamically select the more important reward in a specific state. Each weight independently corresponds to the current state of DQN, and the weight , represents the weight of the optimization goal . When the state of the agent approaches one of the optimization goals, the weight corresponding to that goal will be increased, which will make the agent closer to or farther away from that state. If the agent has multiple goals, it will pay more attention to the goal that is already very close to completion. Under the constraint of the task success execution condition, the agent has two goals: to reduce the time delay of the task and to reduce the energy consumption of the task, and the two normalization functions of the corresponding rewards are , the time delay function value is positive, and the reward is negative when the current task energy consumption is higher than the average task energy consumption in the system, otherwise it is positive. Since the direct addition of the two may cancel out and result in invalid choices, the importance of the optimization goal should be adjusted for different states, and the rewards corresponding to the goals are and

[0111] In the edge cloud environment, when the task time delay reward function approaches the minimum value, the task completion time approaches the maximum time delay constraint. At this time, the weight of the time delay reward function is increased, so that the agent pays more attention to the reward and punishment brought by the task completion time delay. When the energy consumption reward function E(t) is negative, the optimization goal weight of the energy consumption required for task transmission and execution should be increased so that the reward pays more attention to the impact of energy consumption on task execution. The weight independently corresponds to the current state of DQN, assuming is the output of DQN related to task execution time delay, and is the output of DQN related to energy consumption. A weight value reward is created and provided to the agent performing the action. The weight value is defined as a function of the state, and the policy The value of the state s relative to the decision reward of a particular goal, a dynamic weight value is defined as follows:

[0112]

[0113]

[0114] According to the above definition, the defined dynamic weight value will provide high value around the vicinity of the reward extremum and low value in states far from the reward extremum. Thus, the weight will provide the importance of the current state s and the importance of following a policy in the case of different optimization goals, where the policy is the policy provided by the Q-function for a particular goal. The reward weight of an action changes as a function of the distance from the reward function critical value, scaling the weight to the range where is a sigmoid function, is an approximation of the mean of and is an approximation of the standard deviation of .

[0115] As follows:

[0116]

[0117]

[0118] The is added as a weight to the q-values in the different goal DQNs, and the weighted q-values are added to obtain as the output.

[0119] In an optional embodiment, a loss function for the deep reinforcement learning model is constructed.

[0120] DQN utilizes a target network and a training network to stabilize the overall network performance, which have the same structure but different parameters. The online network uses parameters and is updated according to minimizing a loss function. The target network uses parameters and is copied from the online network once every fixed number of steps. DQN uses a deep convolutional network to compute the action value function of the online network, denoted as The action value function of the target network can be denoted as:

[0121]

[0122] where Rt denotes the reward received after action ​is a discount factor, representing the consideration of future rewards by the current reward function. The action a with the maximum q value is selected to be performed, causing the state to transition to . The loss function is defined as the mean squared error between the online network and the target network's value function, and the L2 norm loss, i.e., squared loss, is selected, and the update of the parameters is as follows:

[0123]

[0124] The present application adopts stochastic gradient descent (SGD) to update the parameters of the deep reinforcement learning neural network. Then in the mini-batch stochastic gradient descent, the gradient is the partial derivative of the loss function:

[0125]

[0126] First, a certain experience pool is stored, and then a small batch of batch is sampled from the experience pool and put into the network. In order to perform experience replay, the experience of each time step t of the agent is stored in the replay experience pool. During the training process, some experience data is randomly selected from the replay experience pool, and the parameters of the network are updated by stochastic gradient descent. The advantage of the experience replay mechanism is that it can break the data correlation and make the neural network update more efficient.

[0127] The weights can be updated using TD-learning :

[0128]

[0129] The loss function is defined as follows:

[0130]

[0131] The algorithm schematic diagram is shown in Figure 3 . ①: initialize the experience pool, initialize the Q function environment state, and input the network; ②: calculate the Q function corresponding to the action; ③: calculate the weight corresponding to each optimization target according to the current state and the policy; ④: multiply the Q value vector calculated by the neural network of different optimization targets with the weight value and add them to obtain a Q value vector; ⑤: select the action with the maximum value function according to the greedy policy; ⑥: execute the action and observe the reward; ⑦: update the environment state; ⑧: store the four-tuple in the experience pool; ⑨: uniformly and randomly sample a four-tuple from the experience pool and input it into the network; ⑩: determine whether the round is over, and if so, perform a gradient descent algorithm, and update the parameters of the target network every 200 steps.

[0132] This invention uses a simulation environment built on the PyTorch network framework to compare the scheduling processes of three deep reinforcement learning tasks: Multi-Objective Dynamic Weighted DQN, Baseline DQN, and Double-DQN. The weights for energy consumption and latency in the Baseline DQN are fixed at 0.5. The advantages of existing methods in terms of task completion rate, average task completion time, and node energy consumption are demonstrated. The three task scheduling algorithms are tested in the simulation environment using the same task. The task execution success rate, task completion latency, and cumulative task energy consumption are as follows: Figures 4-6 As shown, the multi-objective dynamic weighted DQN algorithm has better performance.

[0133] The edge cloud environment consists of one data center cloud host, 10 edge servers, and edge mobile devices. The initial number of edge mobile devices is 100, with a maximum of 200. The number of tasks increases with the number of mobile devices. Due to the heterogeneity of the devices, the task generation rate varies, and the task generation rate for each device follows a set rule. The Poisson distribution is such that the computing power of each edge server follows a [1, 5] GHz / sec. Uniform distribution of intervals, probability of node disconnection The local device's computing power is 0.6 GHz, and the edge device's operating power is set to... The number of tasks generated is initialized to 0, and the task data size is in kbit. It follows a uniform distribution between (300, 500). Transmission bandwidth between mobile devices and the edge cloud. The bandwidth is set to 9Mbps, with a transmission bandwidth of 20Mbps between the edge cloud and the data center cloud host. The network consists of two fully connected layers, each with 32 neurons, using the ReLU activation function and the Adam optimizer. Discount factor. The value is 0.99, and the number of training epochs is Episode=1000. The parameters of the greedy strategy are reduced from 1 to 0.01 at each time step with a decay rate of 0.995.

[0134] Figure 4 The diagram illustrates the completion rates of three task scheduling methods according to embodiments of this disclosure. When the number of mobile devices increases from 100 to 200, the horizontal axis represents the number of devices, and the vertical axis represents the task completion rate of different algorithms. As the number of devices and tasks gradually increases, the task completion rate of the algorithm proposed in this invention is higher than that of the benchmark DQN and Double-DQN algorithms.

[0135] Figure 5The diagram illustrates the task latency of three task scheduling methods according to embodiments of the present disclosure. When the number of mobile devices increases from 100 to 200, the cumulative task latency T under the three different algorithms is shown below. Figure 5 As shown. The algorithm proposed in this invention has a lower task latency than the benchmark DQN and Double-DQN algorithms.

[0136] Figure 6 The diagram illustrates the energy consumption of tasks according to three task scheduling methods based on embodiments of the present disclosure. The cumulative energy consumption E of the tasks under the three different task numbers is shown below, as the number of mobile devices increases from 100 to 200. Figure 6 As shown. The algorithm proposed in this invention has a lower cumulative energy consumption than the benchmark DQN and Double-DQN algorithms.

[0137] Based on the above-described multi-objective edge task scheduling method, this disclosure also provides a multi-objective edge task scheduling device. The following will be combined with... Figure 7 The device is described in detail.

[0138] Figure 7 A schematic block diagram of a multi-target edge task scheduling apparatus according to an embodiment of the present disclosure is shown.

[0139] like Figure 7 As shown, the multi-target edge task scheduling device 700 of this embodiment includes a first construction module 710, a second construction module 720, and an execution module 730.

[0140] The first construction module 710 is used to construct an edge cloud system model, which includes an edge computing architecture and a task scheduling strategy. This edge cloud system model is used to execute the current task. In one embodiment, the first construction module 710 can be used to execute the operation S210 described above, which will not be repeated here.

[0141] The second construction module 720 is used to construct a task scheduling model with the optimization objectives of minimizing the failure rate, energy consumption, and latency of the task. This task scheduling model is constructed based on a deep reinforcement learning model. In one embodiment, the second construction module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0142] The execution module 730 is used to execute the task based on the task scheduling strategy output by the task scheduling model using the edge computing architecture. In one embodiment, the execution module 730 can be used to execute the operation S230 described above, which will not be repeated here.

[0143] According to embodiments of this disclosure, the edge computing architecture includes local devices, edge servers, and data center cloud hosts;

[0144] When the computing resources and / or computing power of the local device do not meet the preset requirements, the task will be scheduled to an edge server or data center cloud host adjacent to the local device for processing.

[0145] When the task is processed by the local device, the latency and energy consumption of the task are calculated using a pre-built local computing model.

[0146] When the task is processed by the edge server, the latency and energy consumption of the task are calculated using a pre-built edge computing model.

[0147] When the task is processed by the cloud host in the data center, the latency and energy consumption of the task are calculated using a pre-built computation offloading model.

[0148] According to embodiments of this disclosure, the latency of the task is minimized. :

[0149]

[0150] Where the number of tasks is R, and the attributes of each task are... , This indicates the size of the input data for this task. This indicates the size of the data output by the task. Indicates completion Total number of CPU cycles required express Given the time delay constraint, the task scheduling decision vector is: , For each element ∈ {0, 1}, the value of the scheduling target in the task scheduling decision vector is 1, and the value of anything other than the scheduling target is 0. This indicates that the scheduling target is a local device. This indicates that the scheduling target is an edge server. This indicates that the scheduling target is the data cloud center host, and N is the number of edge servers;

[0151] For each task, there is a time constraint for successful execution. If the sum of the transmission delay and processing delay is less than the delay constraint of the task, the task is considered to have succeeded; otherwise, the task is considered to have failed.

[0152] Whether the task is executed on the local device, on the edge server, or on the cloud host in the data center, This indicates that the task is executed on the local device. This indicates that the task was executed on the edge server. indicates that the task is executed on the data center cloud host;

[0153] C3 is the sum of the computing power required for selecting the task of the local computing model and the edge computing model, which does not exceed the total computing power of the local device or the edge server, and the computing power of each edge server is .

[0154] According to an embodiment of the present disclosure, the method comprises:

[0155] Based on the Markov decision process, the deep reinforcement learning model is constructed, and the Markov decision process comprises a learning state space S, an action space A and a reward function R(t);

[0156] wherein, , represents the state of the network at time t, represents the state of the edge server, comprises the computing power of the edge server and the data center cloud host at time t and the connectivity of the edge server, wherein represents the computing power of the edge server at the current time, , represents the state of the node, when the node is working normally, and when the node is disconnected, indicates the information of the task that needs to be scheduled at time t , and the node is a device in the edge cloud system model;

[0157] action space , indicates that the task adopts the local computing model, indicates the operation of scheduling the task to one of the adjacent edge servers, indicates the operation of scheduling the task to the data center cloud host;

[0158] reward function ;

[0159]

[0160]

[0161] is the delay reward function of the task, E(t) is the energy consumption reward function of the task, when the constraint is satisfied, it indicates that the task can be completed within the maximum delay constraint, and the reward is defined as the difference between the maximum delay constraint measured by the reference average value and the actual execution time, a time delay of the task, a task a corresponding time delay constraint, an initial energy consumption of 0, if not satisfied if the task completion time exceeds its deadline, indicating that the decision is unacceptable, the reward is defined as a negative value, is a constant.

[0162] According to an embodiment of the present disclosure, the method further includes:

[0163] When setting the reward function of the deep reinforcement learning model, the rewards of the optimization targets of the failure rate of the task, the energy consumption of the task and the time delay of the task are normalized, so that the reward values of the two different optimization targets are located in the same interval.

[0164] According to an embodiment of the present disclosure, the method further includes:

[0165] When the time delay reward function of the task approaches a minimum value, the weight of the optimization target of the time delay of the task is increased;

[0166] When the energy consumption reward function is in a negative value, the weight of the optimization target of the energy consumption of the task is increased.

[0167] According to an embodiment of the present disclosure, any of the first construction module 710, the second construction module 720 and the execution module 730 can be combined in one module for implementation, or any of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of the modules can be combined with at least part of the functions of other modules, and implemented in one module. According to an embodiment of the present disclosure, at least one of the first construction module 710, the second construction module 720 and the execution module 730 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware that can be integrated or packaged with a circuit, or any one of software, hardware and firmware or any appropriate combination of any of them. Alternatively, at least one of the first construction module 710, the second construction module 720 and the execution module 730 can be at least partially implemented as a computer program module that can perform corresponding functions when the computer program module is run.

[0168] Figure 8 The block diagram of an electronic device suitable for implementing the multi-target edge task scheduling method according to an embodiment of the present disclosure is schematically shown.

[0169] As Figure 8As shown, the electronic device 800 according to embodiments of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 802 or a program loaded into a random access memory (RAM) 803 from a storage section 808. The processor 801 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chip set, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 801 can also include an on-board memory for cache use. The processor 801 can include a single processing unit or multiple processing units for executing different actions of the method processes according to embodiments of the present disclosure.

[0170] In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. The processor 801 performs various operations of the method processes according to embodiments of the present disclosure by executing the programs in the ROM 802 and / or the RAM 803. Note that the programs can also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 can also perform various operations of the method processes according to embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0171] According to embodiments of the present disclosure, the electronic device 800 can further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 can further include one or more of the following components connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as necessary. A removable recording medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 810 as necessary, so that a computer program read therefrom is installed into the storage section 808 as necessary.

[0172] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, which when executed, implement the method according to embodiments of the present disclosure.

[0173] According to an embodiment of the present disclosure, the computer readable storage medium can be a nonvolatile computer readable storage medium, for example, can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In this disclosure, a computer readable storage medium can be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer readable storage medium can include the ROM 802 and / or the RAM 803 described above and / or one or more memory other than the ROM 802 and the RAM 803.

[0174] Embodiments of the present disclosure also include a computer program product that includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the methods provided by the embodiments of the present disclosure.

[0175] The above-described functions defined in the system / device of the embodiments of the present disclosure are performed when the computer program is executed by the processor 801. According to an embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0176] In one embodiment, the computer program can rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of a signal on a network medium and installed and downloaded through the communication part 809 and / or installed from the detachable medium 811. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0177] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809 and / or installed from the detachable medium 811. When the computer program is executed by the processor 801, the above-described functions defined in the system of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0178] According to embodiments of the present disclosure, program code of the computer programs provided by embodiments of the present disclosure can be written in any combination of one or more programming languages, and specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming language, and / or assembly / machine language. Programming languages include, but are not limited to, such as Java, C++, python, “C” language or similar programming languages. Program code can be executed entirely on a user computing device, partially on a user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected to the Internet through an Internet service provider).

[0179] The flow diagrams and the block diagrams in the drawings are illustrations of possible architectures, functions, and operations for systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0180] Those skilled in the art can understand that the features described in various embodiments of the present disclosure and / or claims can be combined or / and integrated, even if such combinations or integrations are not explicitly described in the present disclosure. In particular, the features described in various embodiments of the present disclosure and / or claims can be combined and / or integrated in various combinations, without departing from the spirit and teachings of the present disclosure. All such combinations and / or integrations are within the scope of the present disclosure.

[0181] The above describes embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims

1. A multi-objective edge task scheduling method, comprising: Construct an edge cloud system model, which includes an edge computing architecture and a task scheduling strategy, and the edge cloud system model is used to execute the current task; With the optimization objectives of minimizing the failure rate, energy consumption, and latency of the task, a task scheduling model is constructed, which is based on a deep reinforcement learning model. The task is executed using the edge computing architecture and the task scheduling strategy output by the task scheduling model. Minimize the latency of the task : The transmission latency and execution latency of the i-th task in the unstable edge cloud environment are respectively... The attributes of the i-th task , This represents the size of the input data for the i-th task. This indicates the size of the data output by the i-th task. This represents the total number of CPU cycles required to complete the i-th task. Let represent the latency constraint of the i-th task, and let the task scheduling decision vector be . ], ∈{0, 1}, where the value of the scheduling target in the task scheduling decision vector is 1, and the value of anything other than the scheduling target is 0, and N is the number of edge servers. Let represent the latency of the i-th task on the local device, the N-th edge server, and the cloud host in the data center, respectively. This represents the computing power of the nth device, expressed in CPU cycles per second. This indicates that the task is executed on the local device. This indicates that the task is executed on the nth edge server. , This indicates that the task is executed on a cloud host in the data center, and R is the number of tasks; The time constraint for the successful execution of the i-th task is defined as follows: if the sum of the transmission delay and processing delay of the i-th task is less than the time constraint of the task, the task is considered to have been successfully executed; otherwise, the task is considered to have failed. The i-th task is executed on the local device, on the edge server, or on the cloud host in the data center. C3 ensures that the sum of the computing power required to assign tasks to the nth device using both the local and edge computing models does not exceed the total computing power of either the local device or the edge server. The computing power of each edge server is... ; Minimize the energy consumption E of the task: ,in, These represent the energy consumption of the i-th task executed on the local terminal, edge server, and data center cloud host, respectively.

2. The multi-target edge task scheduling method according to claim 1, wherein the devices in the edge computing architecture include local devices, edge servers, and data center cloud hosts; When the computing resources and / or computing power of the local device do not meet the preset requirements, the task will be scheduled to an edge server or data center cloud host adjacent to the local device for processing. When the task is processed by the local device, the latency and energy consumption of the task are calculated using a preset local computing model. When the task is processed by the edge server, the latency and energy consumption of the task are calculated using a preset edge computing model. When the task is processed by the cloud host in the data center, the latency and energy consumption of the task are calculated using a preset computation offloading model.

3. The multi-target edge task scheduling method according to claim 1, the method comprising: The deep reinforcement learning model is constructed based on the Markov decision process, which includes learning the state space S, the action space A, and the reward function R(t). in, , This represents the state of the network at time t. Let be the state of the edge cloud system model at time t. This represents the state of the edge server. This includes the computing power of the edge server at time t and the connectivity of the edge server. ,in This represents the computing power of the edge server at the current moment. , , representing the state of the node, when The time node is working normally, when The node disconnects. This represents information about tasks that need to be scheduled at time t. The node is a device in the edge cloud system model. The probability of the nth device failing. , This represents the size of the task input data at time t. This represents the size of the data output by the task at time t. This represents the total number of CPU cycles required to complete the task at time t. This represents the time delay constraint of the task at time t; Action space The action space is a scheduling decision vector. =1 indicates that the i-th task uses the local computing model. This represents the operation of scheduling the i-th task to one of the adjacent edge servers. , =1 indicates that the operation of scheduling the i-th task to the cloud host in the data center is performed; reward function ; Let E(t) be the time delay reward function at time t, and E(t) be the energy consumption reward function at time t. When the constraints are satisfied... This indicates that the i-th task can be completed within the maximum latency constraint, and the latency reward is defined as the difference between the maximum latency constraint and the actual execution time. This represents the sum of transmission latency and execution latency of the i-th task in an unstable edge cloud environment, measured by the average latency. This represents the latency constraint corresponding to the i-th task. For initial energy consumption, E max For maximum energy consumption, E i The energy consumption for performing the i-th task, R is a constant, and R is the number of tasks.

4. The multi-target edge task scheduling method according to claim 1, further comprising: When setting the reward function of the deep reinforcement learning model, the optimization objectives of the failure rate of the task, the energy consumption of the task, and the latency reward of the task are normalized so that the reward values ​​of the two different optimization objectives are in the same range.

5. The multi-target edge task scheduling method according to claim 1, further comprising: When the latency reward function of the task approaches its minimum value, the weight of the task latency as the optimization objective is increased. When the energy consumption reward function of the task is negative, the weight of the task's energy consumption as the optimization objective is increased.

6. A multi-target edge task scheduling device, comprising: The first construction module is used to construct an edge cloud system model, which includes an edge computing architecture and a task scheduling strategy. The edge cloud system model is used to execute the current task. The second construction module is used to construct a task scheduling model with the optimization objectives of minimizing the failure rate, energy consumption and latency of the task. The task scheduling model is constructed based on a deep reinforcement learning model. The execution module is used to execute the task based on the task scheduling strategy output by the task scheduling model using the edge computing architecture. Minimize the latency of the task : The transmission latency and execution latency of the i-th task in the unstable edge cloud environment are respectively... The attributes of the i-th task , This represents the size of the input data for the i-th task. This indicates the size of the data output by the i-th task. This represents the total number of CPU cycles required to complete the i-th task. Let represent the latency constraint of the i-th task, and let the task scheduling decision vector be . , ∈{0, 1}, where the value of the scheduling target in the task scheduling decision vector is 1, and the value of anything other than the scheduling target is 0, and N is the number of edge servers. Let represent the latency of the i-th task on the local device, the N-th edge server, and the cloud host in the data center, respectively. This represents the computing power of the nth device, expressed in CPU cycles per second. This indicates that the task is executed on the local device. This indicates that the task is executed on the nth edge server. , This indicates that the task is executed on a cloud host in the data center, and R is the number of tasks; The time constraint for the successful execution of the i-th task is defined as follows: if the sum of the transmission delay and processing delay of the i-th task is less than the time constraint of the task, the task is considered to have been successfully executed; otherwise, the task is considered to have failed. The i-th task is executed on the local device, on the edge server, or on the cloud host in the data center. C3 ensures that the sum of the computing power required to assign tasks to the nth device using both the local and edge computing models does not exceed the total computing power of either the local device or the edge server. The computing power of each edge server is... ; Minimize the energy consumption E of the task: ,in, These represent the energy consumption of the i-th task executed on the local terminal, edge server, and data center cloud host, respectively.

7. An electronic device, comprising: One or more processors; as well as Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 5.

9. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images