Dependent subtask-oriented mobile edge computing task scheduling system and method

Abstract

The invention provides a dependency subtask-oriented mobile edge computing task scheduling system and method, and relates to the technical field of mobile edge computing, the system comprises an Internet application module, a DAG application analysis module, a network training module, an unloading scheduling module, a computing unloading module, a local processing module and a result output module; according to the method, firstly, in a mobile edge computing environment, the scheduling and computing unloading problem of a DAG dependent task is expressed as a Markov decision process (MDP), and application delay is minimized as a target; on the basis, a distributed actor-commentator (DA-C) network is constructed and is used for performing optimal decisions, including an unloading task, a terminal equipment CPU frequency, a transmission distance and the like, under a dynamic network condition.

Description

Technical Field

[0001] The present invention relates to the field of mobile edge computing technology, and in particular to a mobile edge computing task scheduling system and method for dependent subtasks. Background Art

[0002] Task offloading algorithms are a core component of edge computing architecture, directly affecting network performance and user experience. To address the problem of task offloading in mobile edge computing (MEC) scenarios, researchers have recently proposed several techniques based on deep reinforcement learning (DRL) in edge computing environments. This method integrates deep neural networks into reinforcement learning to construct an action-value function (Q-Network), which is called Deep Q Network (DQN). The research results of Ale et al. show that compared with traditional reinforcement learning algorithms or greedy algorithms, the DQN-based network model can complete more tasks within the same time limit while reducing energy consumption. In the research of Jun et al., the recurrent neural network (RNN) and dueling dual deep Q network (DDDQN) were used to improve the model and achieve better optimization results.

[0003] The above research proves that in the MEC scenario, deep reinforcement learning technology can better achieve task offloading. However, in real scenarios, many mobile applications are composed of multiple dependent subtasks, which should be processed according to their logical priority. Specifically, the input of each dependent task must be the result output of one or more of its previous tasks. Only when all its dependent tasks are completed, an application is considered complete. This feature has become an important consideration in the current MEC research at home and abroad. Researchers have proposed a way to model dependent tasks as a directed acyclic graph (DAG) and it has been widely adopted. The IoT application model that meets the DAG structure will generate additional constraints in the task offloading process, which are parts that previous studies cannot meet.

[0004] In the context of dependent subtasks, domestic and foreign research has proposed a variety of algorithms to achieve computational offloading, such as heuristic algorithms, deep reinforcement learning, and other neural network models. They have also optimized multiple aspects of the application, such as shortening latency, reducing energy consumption, the trade-off between latency and energy consumption, and increasing the number of completed tasks. Many works have made significant contributions in this field.

[0005] In the above research, there are many ways to "solve" the dependency constraints of DAG tasks. For example, based on the idea of ​​solving the standard DAG task scheduling problem (such as multi-core CPU task scheduling), set priorities for tasks and use this to pre-calculate the pre-scheduling sequence for DAG tasks. When applied to the MEC environment, it is necessary to consider the dynamic variability of the environment (such as changes in server channel conditions) and the additional transmission delay caused by offloading to different locations (local or different servers). In the current research on priority pre-scheduling, an algorithm is used to calculate the delay of all possible offloading decisions for each subtask and then take the average value. This algorithm reduces the impact of extreme values ​​by calculating the average processing power of the current temporal server. However, for large-scale DAG tasks in a multi-server MEC environment, the average value needs to calculate all possible offloading decisions, which brings a large computational overhead; and the average value cannot accurately reflect the actual priority of the task, especially when the offloading decision has a large impact on the delay, resulting in inaccurate pre-scheduling. The pre-scheduling algorithm converts the DAG graph structure into a task sequence by priority sorting, but the temporal feature modeling of the sequence still needs to rely on an efficient sequence learning model. Recently, many studies have added recurrent neural networks (RNNs), especially its variant, long short-term memory networks (LSTMs) to train sequence prediction. These algorithms can accurately identify highly nonlinear patterns between different input features, thereby significantly accelerating the learning speed of the model. However, RNNs have the problem of gradient vanishing or gradient exploding. LSTM solves this problem, but its long-term memory is still limited. When the sequence length increases with the number of subtasks, information may still be forgotten during the transmission process.

[0006] In addition, a special optimization idea has been proposed in recent domestic and foreign research on vehicular edge computing (VEC): VEC is a specific application of MEC in the real connected car network, in which the calculation offloading of vehicle tasks can not only be selected to be offloaded to the edge server through the end-to-edge (D2E) method, but also through the end-to-end (D2D) method, where the vehicle-mounted device transmits data to other devices (in this study, idle devices can also be used as offloading nodes) to achieve calculation offloading; Similar to this optimization idea: Zhou et al. proposed an external dependency (ED) optimization method: some subtasks in the DAG task that can share intermediate resources (in this study, it is necessary to determine in advance which tasks from different users should share their intermediate data) are divided into external dependency nodes. Once such tasks complete the calculation, the intermediate data can be shared with other nodes that have not yet completed the calculation, thereby accelerating the task offloading process and reducing latency. The commonality of the above studies is that parameters must be manually set in advance based on the original system. For example, in D2D mode, user devices that can be used as offloading nodes are pre-set; in ED mode, nodes that can be used as shared nodes are pre-set. However, when the MEC environment changes dynamically or needs to be applied to another MEC environment, setting up and optimizing requires additional time, and the scalability is poor.

[0007] In summary, the optimization problem of mobile edge computing task scheduling for dependent subtasks is an important research direction to improve network performance and user experience. Although a variety of algorithms and optimization schemes have been proposed, in practical applications, existing methods still have several limitations, such as the complexity of parameter setting, difficulty in neural network training, and insufficient system scalability. Summary of the invention

[0008] In view of the shortcomings of the prior art, the present invention provides a mobile edge computing task scheduling system and method for dependent subtasks, which is dedicated to solving the problem of how to achieve efficient scheduling and unloading of IoT applications in a mobile edge computing environment for IoT applications that are presented in the form of dependent subtasks and have high computing power requirements.

[0009] On the one hand, a mobile edge computing task scheduling system for dependent subtasks includes: an Internet of Things application module, a DAG application parsing module, a network training module, an offloading scheduling module, a computing offloading module, a local processing module, and a result output module.

[0010] The IoT application module is used to read IoT applications from the MEC environment to obtain raw data.

[0011] The DAG application parsing module receives the original data of the Internet of Things application module as input, and parses the original data into a DAG structure to obtain parsed data; the parsed data specifically includes node information, edge dependencies, and an adjacency matrix of the DAG application, thereby obtaining the number of subtasks of the Internet of Things application, the dependencies between subtasks, and the computing resources required for each subtask node.

[0012] The network training module is used to receive the parsed data from the DAG application parsing module and upload it to the MEC host as training data, train the distributed actor-critic DA-C network built into the MEC host, and deploy the trained network to the offloading scheduling module.

[0013] The distributed actor-critic DA-C network is specifically: The existing actor-critic network is improved and a distributed asynchronous optimization framework is designed. The Actor network is trained in parallel on multiple cores, and then each Actor network periodically sends local experience trajectories to the Critic network. The Critic network updates the target strategy π based on the trajectory sets received from different Actor networks and the past trajectories stored in the experience replay buffer queue. Each time the Critic network's strategy is updated, the Actor network uses the updated strategy π to update the local strategy μ. Firstly, the scheduling and computation offloading problem of DAG dependent tasks in the mobile edge computing environment is modeled as a Markov decision process MDP, with the goal of minimizing application time delay. The core tuples of MDP include system state, action space, and reward function, which are defined as follows: the system state includes DAG task characteristics and MEC network environment; the DAG task characteristics are the DAG structure data received from the DAG application parsing module, which includes the computing demand and transmission data volume of each node, as well as the dependency relationship; the MEC network environment reflects the computing power, delay and bandwidth capacity of the edge nodes; the action space is the offloading strategy selection for each task node, i.e. local execution or offloading to a specified server; the reward function is the negative increment of time delay.

[0014] The Actor network adopts an encoder-decoder architecture based on the attention mechanism, namely, a seq2seq model. In order to process the DAG graph structure data of the topological structure, the priority of each node task is recursively calculated through a rank function, and then processed into an execution order sequence through priority sorting. The encoder extracts features through the execution order sequence and uses the attention mechanism to explicitly model the dependency relationship between task nodes. When gradually generating the unloading strategy, the decoder dynamically combines the execution status information of its predecessor task and finally outputs the optimal action probability function that conforms to the current environment. The Critic network shares the network layers and some weights except the output layer with the Actor network, and outputs the state value estimation function through an independent fully connected layer.

[0015] The seq2seq model includes an encoder and a decoder, both of which are implemented by recurrent neural networks; the input of the encoder is the task embedding sequence, and the output of the decoder is the offloading strategy for each subtask ,in a n Indicates n An offloading strategy for each task; an attention mechanism is added, which allows the decoder to focus on different parts of the source sequence, i.e., the input sequence of the encoder, at each step of output generation, and encode the input sequence into a vector with a fixed dimension; After receiving the DA-C network trained by the network training module, the offloading scheduling module receives the DAG structure data transmitted by the DAG application parsing module and uses the policy network reasoning to make an offloading strategy.

[0016] The offloading strategy includes offloading to the MEC server for execution and local execution; After the offloading scheduling module makes an offloading strategy, if the calculation offloading module chooses to offload to the MEC server for execution, the calculation offloading module is responsible for the offloading process; After the unloading scheduling module makes the unloading strategy, if the unloading scheduling module chooses to execute the unloading strategy locally, the local processing module is responsible for local processing.

[0017] The result output module is used to receive the result output data of the local processing module and the calculation offloading module, including calculation time delay, transmission time delay and application execution time data.

[0018] A method for scheduling mobile edge computing tasks for dependent subtasks is implemented by the aforementioned mobile edge computing task scheduling system for dependent subtasks, and includes the following steps:

[0019] Step 1: Data reading and parsing stage; After the system is initialized, the IoT application module reads the IoT application from the MEC environment as raw data.

[0020] Step 2: The DAG application parsing module receives the raw data from the IoT application module and reads the IoT application into a DAG structure through the graph structure construction algorithm, thereby obtaining the number of subtasks of the IoT application, the dependencies between the subtasks, and the computing resources required for each subtask node; The graph structure construction algorithm is as follows: the node information and edge dependencies in the task configuration information file are sequentially constructed into a DAG graph data format, and finally saved as an adjacency matrix.

[0021] Step 3: Network training phase; After receiving the DAG structure data parsed by the DAG application parsing module, the network training module first starts the training process: uploading the DAG structure data to the MEC host, training based on the distributed actor-critic DA-C network until the network converges and reaches the expected performance indicator of reducing the overall application runtime delay. The trained network will be deployed to the offload scheduling module.

[0022] Step 4: Uninstall strategy phase; Step 4.1: The offloading scheduling module receives the policy network trained by the network training module and infers the offloading policy. On the other hand, it continuously receives the data of the new DAG application parsing module, which contains the latest task information. Step 4.2: Combining the data from the policy network and the new DAG application parsing module, the offloading scheduling module calculates the probability distribution of the offloading strategy through the policy network, and then makes an offloading strategy for each subtask based on the probability distribution, that is, deciding whether the subtask should be offloaded to the MEC server for execution or chosen to be executed locally.

[0023] Step 5: Task execution phase; If the offloading scheduling module determines that the subtask is selected to be offloaded to the MEC server for execution, the computing offloading module is responsible for execution; according to the decision of the offloading scheduling module, the corresponding task data is offloaded to the MEC server, and the computing task is completed using the server's resources; If the offload scheduling module determines that the subtask is to be executed locally, the local processing module is responsible for execution; the computing task is completed by mobilizing the computing resources of the local device.

[0024] Step 6: Result output stage; Step 6.1: After completing the task processing, the local processing module transmits the result data to the result output module; After the computation offloading module completes the task processing on the MEC server, it transmits the computation time delay, transmission time delay, and application execution time data to the result output module; Step 6.2: The result output module summarizes and organizes the result data from the local processing module and the calculation offloading module.

[0025] The beneficial effects of adopting the above technical solution are: The present invention provides a design and implementation of a mobile edge computing task scheduling algorithm for dependent subtasks, which has the following beneficial effects: 1: Considering the dynamic network conditions, the scheduling of dependent subtasks based on DAG and the computation offloading of IoT devices in MEC environment are proposed as an optimization problem to minimize the execution time of IoT applications. For DAG dependent subtasks, this paper proposes a pre-scheduling technique to define the execution order of related tasks in each DAG-based IoT application. Then the complex MEC environment and optimization problem are transformed into a Markov decision process (MDP); 2: A distributed actor-critic algorithm (DA-C) is proposed to solve the computational offloading problem and achieve rapid adaptation to dynamic offloading scenarios. Thanks to the distributed network structure, DA-C also has high sample efficiency for new learning tasks, allowing user devices to use their own data to run the training process even when computing resources are limited. In addition, it also uses an experience replay buffer, which greatly improves the sampling efficiency and breaks the strong correlation between experience trajectories; 3: The unloading strategy process is combined with the pre-scheduling technology to convert it into a sequence prediction process. Compared with the average pre-scheduling algorithm in previous studies, this invention dynamically learns the optimal scheduling sequence through the model, which improves the overall computing performance of the model and the accuracy of the scheduling sequence. A seq2seq neural network with an attention mechanism is designed to train the unloading and scheduling strategies, solving the problem of information bottleneck in long sequence processing in traditional LSTM networks.

[0026] 4: Simulation experiments are performed using randomly generated DAGs to evaluate the performance of the proposed algorithm, which covers a wide range of topologies, number of tasks, and transmission rates. It is compared with other benchmarks. The results show that compared with other methods, the algorithm can achieve the lowest latency and effectively improve the application utility under dynamic network conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the system structure of the present invention; Figure 2 This is a diagram of the calculation offloading structure in the MEC environment of the present invention; Figure 3This is a structural diagram of a seq2seq neural network according to an embodiment of the present invention; Figure 4 A distributed actor-critic algorithm framework for an embodiment of the present invention; Figure 5 This is a DAG example diagram of an embodiment of the present invention; Figure 6 This is a diagram of the evaluation results of the topology structure 1 according to an embodiment of the present invention; Figure 7 This is a diagram of the evaluation results of topology structure 2 according to an embodiment of the present invention; Figure 8 This is a diagram of the evaluation results of the topology structure 3 of the embodiment of the present invention; Fig. 9 This is a diagram of the evaluation results when the number of tasks n=20 in the embodiment of the present invention; Fig.10 This is a diagram of the evaluation results when the number of tasks n=30 in the embodiment of the present invention; Fig.11 This is a diagram of the evaluation results when the number of tasks n=40 in the embodiment of the present invention; Fig.12 In the embodiment of the present invention, when the uplink and downlink transmission rates are Evaluation result diagram when Fig.13 In the embodiment of the present invention, when the uplink and downlink transmission rates are Evaluation result diagram when Fig.14 In the embodiment of the present invention, when the uplink and downlink transmission rates are Evaluation result diagram when . DETAILED DESCRIPTION

[0028] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0029] On the one hand, a mobile edge computing task scheduling system for dependent subtasks, e.g. Figure 1 As shown, it includes: Internet of Things application module, DAG application analysis module, network training module, offloading scheduling module, computing offloading module, local processing module and result output module.

[0030] The IoT application module is used to read IoT applications from the MEC environment to obtain raw data as input for network training module training or offloading scheduling module calculation.

[0031] The DAG application parsing module receives the original data of the Internet of Things application module as input, and parses the original data into a DAG structure to obtain parsed data, specifically including node information, edge dependencies, and adjacency matrix of the DAG application, so as to obtain the number of subtasks of the Internet of Things application, the dependencies between subtasks, and the computing resources required for each subtask node.

[0032] The network training module is used to receive the DAG structure data from the DAG application parsing module, upload it to the MEC host as training data, train the distributed actor-critic DA-C network built into the MEC host, and deploy the trained network to the offloading scheduling module.

[0033] The Distributed Actor-Critic DA-C network is as follows Figure 4 As shown, specifically: The Actor-Critic network is a powerful and widely used algorithmic framework in the field of reinforcement learning. It combines the "Actor" part based on policy gradients with the "Critic" part based on value function evaluation. The two work together to efficiently learn the optimal strategy to maximize the cumulative reward. However, the asynchronous update mechanism of this architecture may lead to instability in the learning strategy. In addition, a large amount of data interaction and sample utilization may also limit the efficiency of training. The distributed algorithm proposed in the present invention is an improvement to reduce this shortcoming.

[0034] In order to improve the training efficiency and stability, the existing actor-critic network was improved and a distributed asynchronous optimization framework was designed. The Actor network was trained in parallel with multiple cores, and then each Actor network periodically sent local experience trajectories to the Critic network. The Critic network updated the target strategy π based on the trajectory sets received from different Actor networks and the past trajectories stored in the experience replay buffer queue. After each update of the Critic network's strategy, the Actor network used its updated strategy π to update its local strategy μ. Because multiple Actors interact with the environment in a distributed and parallel manner, the algorithm can explore data sources in multiple environments, which enhances stability. The experience replay buffer can store past experience and extract it through a priority sampling mechanism during training, which further breaks the strong correlation between experience trajectories and improves the efficiency of sample use.

[0035] Firstly, the scheduling and computation offloading problem of DAG dependent tasks in the mobile edge computing environment is modeled as a Markov decision process MDP, with the goal of minimizing the application time delay. The core tuples of MDP include system state, action space, and reward function, which are defined as follows: the system state includes DAG task characteristics and MEC network environment; the DAG task characteristics are the DAG structure data received from the DAG application parsing module, which includes the computational requirements and transmission data volume of each node, as well as the dependency relationship; the MEC network environment reflects the computing power, delay and bandwidth capacity of the edge nodes; the action space is the offloading strategy selection for each task node, i.e. local execution or offloading to a specified server; the reward function is the negative increment of time delay; The Actor network adopts an encoder-decoder architecture based on the attention mechanism, namely, a seq2seq model. In order to process the DAG graph structure data of the topological structure, the priority of each node task is recursively calculated through a rank function, and then processed into an execution order sequence through priority sorting. The encoder extracts features through the execution sequence and uses the attention mechanism to explicitly model the dependencies between task nodes. When the decoder gradually generates decision steps, it dynamically combines the execution status information of its predecessor tasks and finally outputs the optimal action probability function that conforms to the current environment; The Critic network and the Actor network share network layers and some weights except the output layer, and output the state value estimation function through an independent fully connected layer; this network sharing design can reduce computational overhead and ensure the consistency of the strategy and value function in understanding the environment.

[0036] The seq2seq model includes an encoder and a decoder, both of which are implemented by recurrent neural networks; the input of the encoder is the task embedding sequence, and the output of the decoder is the scheduling decision of each subtask ,In order to improve the performance, an attention mechanism is added, which allows the decoder to focus on different parts of the source sequence, i.e., the input sequence of the encoder, at each step of output generation. Therefore, it can effectively alleviate the information loss problem caused by the original seq2seq neural network and encode the input sequence into a vector with a fixed dimension; After receiving the DA-C network trained by the network training module, the offloading scheduling module receives the DAG structure data transmitted by the DAG application parsing module and uses the policy network reasoning to make an offloading strategy; The offloading strategy includes offloading to the MEC server for execution and local execution; After the offloading scheduling module makes an offloading strategy, if the calculation offloading module chooses to offload to the MEC server for execution, the calculation offloading module is responsible for the offloading process; After the unloading scheduling module makes the unloading strategy, if the unloading scheduling module chooses to execute the unloading strategy locally, the local processing module is responsible for local processing. The result output module is used to receive the result output data of the local processing module and the calculation offloading module, including calculation time delay, transmission time delay and application execution time data; In order to better describe the specific parameters and processes, Figure 2 An example of computation offloading in a mobile edge computing (MEC) environment is given. This example considers a real-world IoT application: face recognition, which consists of multiple subtasks and dependencies such as slicing, detection, and feature merging. The user device (UD) will offload these tasks according to the offloading scheduling module, so that some tasks can be run locally on the UD, while other tasks can be offloaded to the MEC host through the wireless channel.

[0037] In general, MEC hosts can meet the computing resources required by UD. The computing power of each MEC host (i.e., the number of CPU cores multiplied by the clock speed of each core) is expressed as , as shown in Table 1, you can view the introduction of all parameters.

[0038] Table 1: Meaning of all parameter symbols in the embodiments of the present invention:

[0039] In terms of form, in this embodiment, the IoT application is modeled as , , where the vertex set Represents a set of subtasks, a directed edge set Represents the dependency relationship between tasks. Each directed edge is specifically represented as , corresponding to the task and The dependence between for The direct parent task of the task. It is stipulated that under the dependency constraint, the child task cannot be executed before its parent task is completed. In the specific implementation, the task configuration information file (JSON format) of the IoT application is read, and the IoT application is read as a graph data structure through the graph structure construction algorithm. The algorithm steps are: the node information and edge dependencies in the task configuration information file are sequentially constructed into a DAG graph data format, and finally saved as an adjacency matrix for subsequent operation and processing.

[0040] In computing offloading, computing tasks can be executed locally on the UD or offloaded to the MEC host. When selecting offloading execution, there are three steps: First, UD sends a message to the MEC host through the wireless channel. Next, the MEC host runs the received task. Finally, The running results are returned to UD. The delay of each step is related to the task profile and MEC system status.

[0041] here, The task summary includes the CPU cycles required to run the task, the amount of data sent by the task and the amount of data to be received In addition, the MEC system status also includes the transmission rate of the wireless uplink channel and the transmission rate of the downlink channel Therefore, the task The delay of sending data , runtime latency on the MEC host and receiving delay It can be calculated as: ; (1) If the task If it is run on a local terminal, there is only the runtime delay item on the terminal, that is, In summary, the time delay of the task offloading process includes local processing, uplink, downlink and remote processing delays.

[0042] For DAG: In this embodiment, the scheduling plan is expressed as The number of vertices , which is equal to the number of subtasks, Represents the task The tasks are scheduled in the order of the scheduling plan, where all parent tasks are scheduled before their child tasks; tasks on the uplink wireless channel, MEC host, downlink wireless channel, and terminal are scheduled in the order of the scheduling plan. The completion time is expressed as , , and ; Express the available time of resources as: , , and The availability of a resource depends on the time it takes to perform the task. The completion time of the previously scheduled task. If the previously scheduled task did not use the resource, this item is set to 0.

[0043] If the task is offloaded to the MEC host, then only when the parent task is completed and the uplink channel is available, Therefore, the completion time on the upstream channel is: ;

[0044] ; (2) Similarly, The completion time on the MEC and the completion time of the downlink transmission are given by: ;

[0045] ; ;

[0046] ; (3) if If you choose to schedule locally, its start time depends on the completion time of its parent node and the available time of the local UD, expressed as follows: ;

[0047] ; (4) In general, the overall goal of the task is to find an effective offloading plan for the DAG to obtain the minimum total delay while satisfying the constraints of the above formulas (1), (2), (3), and (4). The rank function form is as follows: ; (5) in Indicates exiting the task set (i.e., a node without subtasks).

[0048] In order to enable the deep reinforcement learning algorithm to solve the problem of computational offloading, Markov decision process (MDP) modeling was performed in this embodiment; First, the learning task Represented as a multi-tuple ,in. represents the state space (state), represents the action space (action), represents the reward function (reward), represents the state transition probability matrix, represents the discount factor. Strategy Indicates status Select action in case In this embodiment, according to the strategy The trajectory sampled from the environment is defined as ,in , is the time step Instant rewards.

[0049] definition is a vector of policy parameters, and the state value function can be expressed as: ;

[0050] in is based on The goal of reinforcement learning is to find an optimal parameterized strategy The expected value of the total reward maximum.

[0051] Specifically, in this embodiment, all learning tasks in the MEC environment are defined as , where each task is represented by , the state, action and reward functions of the MDP are defined as follows: State: When uninstalling the task When running tasks, the latency depends on the task characteristics (such as required CPU frequency, data volume), DAG topology logic, wireless transmission rate, and the availability of MEC resources. According to formulas (1), (2), (3), and (4), the status of MEC resources is related to The task is unloaded according to the previously scheduled task's unloading strategy. The state is defined as a combination of DAG and partial uninstallation plan: ; (6) in includes a task embedding sequence, and It's the first mission To convert the DAG into a task embedding sequence, the tasks are first sorted and indexed in descending order according to the rank value of each task, defined as: ; (7) in Indicates the task The delay from the start of unloading to the end of execution, the detailed calculation formula of the three items is shown in formula (1). express The set of direct subtasks of . Each task is converted into an embedding sequence consisting of three elements: 1) embedding the vector of the current task index and the normalized task profile, 2) a vector containing the indices of the immediate parent tasks, 3) Contains the index of the immediate subtask.

[0052] Action: The scheduling of each task is two-dimensional, so the action space is defined as ;

[0053] 0 means execution on UD, and 1, 2...n means offloading to server n.

[0054] Reward function: The goal is to minimize the total delay given by formula (5) To achieve this goal, this embodiment defines the reward function as the estimated negative increment of the delay after making the task offloading decision. As shown below: ; (8) Based on the above definition, this embodiment will schedule The strategy is expressed as For a DAG with n child nodes, let the strategy In the given graph Uninstall plan in case of Therefore, we can Apply the probability chain rule to obtain: , then the optimal action probability function is as follows: ; (9) Approximating the optimal parameterization strategy through deep neural networks , the task offloading problem can be solved.

[0055] The seq2seq neural network is a method that can efficiently learn the above strategy. Figure 3 The design of the seq2seq neural network is shown in Figure 1: It can be divided into two parts: encoder and decoder, both of which are implemented by recurrent neural networks. The input of the encoder is the task embedding sequence, while the output of the decoder is the scheduling decision for each subtask. ; To improve performance, an attention mechanism is added in this embodiment. It allows the decoder to focus on different parts of the source sequence (i.e., the encoder's input sequence) at each step of output generation, thus effectively alleviating the information loss problem caused by the original seq2seq neural network, which encodes the input sequence into a vector with a fixed dimension.

[0056] Formally, this embodiment defines the encoder and decoder functions as and , specifically the Long Short-Term Memory Network (LSTM) is used.

[0057] At each step of the encoding, the encoder The output is obtained by: ; (10) After encoding all input task embeddings, the output vector is defined as At each decoding step, the output of the decoder is defined as: ; (11) in The decoding step The context vector is calculated as the weighted sum of the encoder’s outputs: ; (12) The output of each encoder Weight The calculation method is as follows: ; (13) The addition of the attention mechanism requires a similarity score to measure the matching degree between the output of each encoder layer and the input of the decoder. In this embodiment, the similarity function is defined as is a trainable feedforward neural network, and two fully connected layers are used to make the output of the decoder Separation into policy functions and value function .

[0058] Since the policy and value functions share most parameters (i.e., encoder and decoder) for extracting common features of the DAG (e.g., graph structure and task summary), training the policy can speed up the training of the value function, and vice versa. is generated by sampling the strategy. Once trained, the DAG’s subtask offloading strategy sequence can be inferred through a seq2seq neural network, where the action Depend on generate.

[0059] The entire DA-C network algorithm process is as follows. In this embodiment, Algorithm 1 and Algorithm 2 are used to represent the roles of the Actor network and the Critic network respectively. The detailed analysis of the algorithm is as follows: Algorithm 1 shows how the Actor network performs the application's uninstallation decision and generates experience trajectories. Each Actor network performs the following steps: At the initial stage of each trajectory, the Actor passes the Critic's strategy Update your local policy .

[0060] When the Actor starts to make an uninstallation strategy (that is, when flag-init= ), it will accept the request from the application queue ( ) , then the pre-scheduled "decision" sequence is obtained through the seq2seq neural network, and the Actor changes flag-init to , indicating that in the subsequent steps, the Actor only needs to obtain the current state of the environment according to (line 4) and no longer uninstall other applications. After obtaining the decision sequence, for each subtask in the sequence, according to the current state of the environment and , choose to unload to a suitable location for processing, so as to further calculate the reward and the next environment state .

[0061] Then, create a four-tuple experience And store it in the experience batch buffer. When all subtasks of the task are offloaded, the overall latency can be calculated. In addition, set flag-init= This allows the Actor to continue to obtain tasks from the task queue and continue processing.

[0062] After N steps, each actor forwards its experience batch buffer to the Critic (line 15). The Critic periodically updates its policy π through it, noting that these experience sequences are obtained by multiple actors.

[0063] Algorithm 1:

[0064] Algorithm 2 shows the role of the Critic network. Critic continuously receives and stores the experience trajectories sent by the Actor experience batch buffer, and updates the main buffer and replay buffer until the training batch reaches the set value (TBS). Critic and Actor will The optimizer adjusts the model parameters to minimize the loss function values ​​of the Actor and Critic and optimize the current target strategy. The loss function of the Actor and Critic is calculated as follows: Actor's loss function: ; (14) Critic’s loss function: ; (15) The calculation parameter is the four-element experience tuple in the buffer. , the parameters of the network are , is the policy function, is the value function, Representation strategy The state distribution below.

[0065] Algorithm 2:

[0066] Strategy in Critic After the update, the Actor will also update its local policy before performing an application uninstall By repeating the above steps, the Actor will converge to a better strategy through training, or stop training early when the predetermined number of training times is reached.

[0067] In this example, a simulation experiment is designed to evaluate the performance of DA-C in dynamic scenarios: DA-C is implemented using Tensorflow. Both the encoder and decoder of the seq2seq neural network are set as two-layer dynamic long short-term memory (LSTM) networks with 256 hidden units in each layer. In addition, layer normalization is added to both the encoder and decoder.

[0068] The parameters in the network are set as follows: The learning rates of “Actor” and “Critic” are both set to 5×10 -4 , optimization function loss coefficient The value is set to 0.5 and the discount factor is 0.99. The results are shown in Table 2.

[0069] Table 2: Network parameter meaning table:

[0070] In this example, a cellular network is considered, where the data transmission rate varies with the location of the terminal. The CPU clock frequency of the UD is set to 1 GHz. Each virtual machine of the MEC host has 4 cores, and the CPU clock speed of each core is 2.5 GHz.

[0071] The application is modeled as a DAG structure, using various topologies and task profiles. In order to simulate heterogeneous DAGs, a synthetic DAG generator is implemented in this embodiment. There are four parameters n, fat, density, and ccr that control the topology and task profile of the generated DAG, where n represents the number of tasks, fat controls the width and height of the DAG, density determines the number of edges between two layers of the DAG, and ccr represents the ratio of the communication and computational costs of the task. Figure 5 The figure shows the DAG generated from the low fat and density to the fat and density example.

[0072] The results of the three experiments are as follows Figure 6-Figure 14 As shown: The first experiment Figure 6-Figure 8 The following simulations simulate the scenarios of DAG tasks with different topological structures. Figure 9-11 The following simulations show different number of tasks. Figure 12-14 The following simulations show different dynamic transfer rate scenarios. In all scenarios, the data size of each task ranges from 5KB to 50KB; the CPU frequency ranges from 10 7 to 10 8 In addition, this embodiment also makes a comparison with the greedy algorithm.

[0073] Result analysis: In the first experiment, this embodiment generates DAG sets with different topological structures to simulate the scenario where users have different preferences for IoT applications. Each dataset contains 100 DAG sets with the same and These are two key parameters that affect the DAG graph structure. ,set up Group , Group , 25 data sets were generated in different combinations, 20 sets were selected and input into the network for training.

[0074] Afterwards, three DAG topologies that were not included in the training set were randomly selected as test data sets to verify the effect of the DAC network. The parameters of the three topologies are: , .

[0075] Although the number of DAG tasks in this experiment is the same, due to different DAG topology designs, for example, a larger fat parameter will result in fewer DAG task layers and more nodes per layer; a larger density parameter will result in more edges between layers, which increases the dependency. These parameters will increase the complexity of the DAG topology, thereby increasing the difficulty of DA-C training. From the experimental results Figure 6 , Figure 7 and Figure 8 It can be seen that DA-C can reduce time delay and can quickly adapt to different topologies to achieve convergence. Different topologies will affect the number of rounds of DA-C model training to reach convergence. For example, topology 2 has fewer dependencies, so it can converge quickly, while topology 3 has more dependencies and converges relatively slowly.

[0076] The purpose of the second experiment is to show the impact of different task numbers n on the algorithm performance. At this time, 6 groups of task numbers will be selected The data set is used as the training set, and the As a validation set. At this point, the topological logic of the DAG task still maintains diversity, because Experiment 1 has verified that D-AC can adapt to different topological structures.

[0077] From the experimental results Fig. 9 , Fig.10 and Fig.11 It can be seen that the algorithm training has reached convergence and has completed the optimization of latency very well. As the number of tasks increases, the final latency of the DA-C model training when it reaches convergence will also increase. For example, when the number of tasks n=20, the latency at convergence is about 1000ms, and when the number of tasks n=40, the latency at convergence is about 1900ms. The difference in latency between the two is in line with the positive correlation with the number of tasks.

[0078] In the third experiment, different transmission rates were selected for comparison. , the combinations of transmission rates are , unit: Mbps.

[0079] From the experimental results Fig.12 , Fig.13 and Fig.14 It can be seen that as an important part of the delay calculation, the faster the transmission rate, the faster the network training can be achieved, so as to reach convergence faster. When the transmission rate is When , the convergence delay is about 650ms, and the difference between the two delays is in line with the negative correlation with the transmission rate.

[0080] All the above experiments also selected the greedy algorithm as the benchmark algorithm. The results show that the use of the DA-C model can obtain lower latency and enhance the adaptability to the environment through training. The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also should cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. A mobile edge computing task scheduling system for dependent subtasks, characterized in that: include: IoT application module, DAG application analysis module, network training module, offloading scheduling module, computing offloading module, local processing module and result output module; The IoT application module is used to read IoT applications from the MEC environment to obtain raw data; The DAG application parsing module receives the original data of the IoT application module as input, and parses the original data into a DAG structure to obtain parsed data; the parsed data specifically includes node information, edge dependencies, and an adjacency matrix of the DAG application, thereby obtaining the number of subtasks of the IoT application, the dependencies between subtasks, and the computing resources required for each subtask node; The network training module is used to receive the parsed data from the DAG application parsing module and upload it to the MEC host as training data, train the distributed actor-critic DA-C network built into the MEC host, and deploy the trained network to the offloading scheduling module; After receiving the DA-C network trained by the network training module, the offloading scheduling module receives the DAG structure data transmitted by the DAG application parsing module and uses the policy network reasoning to make an offloading strategy; The offloading strategy includes offloading to the MEC server for execution and local execution; After the offloading scheduling module makes an offloading strategy, if the calculation offloading module chooses to offload to the MEC server for execution, the calculation offloading module is responsible for the offloading process; After the unloading scheduling module makes the unloading strategy, if the unloading scheduling module chooses to execute the unloading strategy locally, the local processing module is responsible for local processing. The result output module is used to receive the result output data of the local processing module and the calculation offloading module, including calculation time delay, transmission time delay and application execution time data.

2. A mobile edge computing task scheduling system for dependent subtasks according to claim 1, characterized in that: The distributed actor-critic DA-C network is specifically: The existing actor-critic network is improved and a distributed asynchronous optimization framework is designed. The Actor network is trained in parallel on multiple cores, and then each Actor network periodically sends local experience trajectories to the Critic network. The Critic network updates the target strategy based on the trajectory sets received from different Actor networks and the past trajectories stored in the experience replay buffer queue. Each time the Critic network's strategy is updated, the Actor network uses the updated strategy to update the local strategy. Firstly, the scheduling and computation offloading problem of DAG dependent tasks in the mobile edge computing environment is modeled as a Markov decision process MDP, with the goal of minimizing application time delay. The core tuples of MDP include system state, action space, and reward function, which are defined as follows: the system state includes DAG task characteristics and MEC network environment; the DAG task characteristics are the DAG structure data received from the DAG application parsing module, which includes the computing demand and transmission data volume of each node, as well as the dependency relationship; the MEC network environment reflects the computing power, delay and bandwidth capacity of the edge nodes; the action space is the offloading strategy selection for each task node, i.e. local execution or offloading to a specified server; the reward function is the negative increment of time delay.

3. A mobile edge computing task scheduling system for dependent subtasks according to claim 2, characterized in that: The Actor network adopts an encoder-decoder architecture based on the attention mechanism, namely, a seq2seq model implementation; in order to process the DAG graph structure data of the topological structure, the priority of each node task is recursively calculated through a rank function, and then processed into an execution order sequence through priority sorting; wherein the encoder extracts features through the execution order sequence, and uses the attention mechanism to explicitly model the dependency relationship between task nodes; when the decoder gradually generates the unloading strategy, it dynamically combines the execution status information of its predecessor task, and finally outputs the optimal action probability function that conforms to the current environment; the Critic network shares the network layers and some weights except the output layer with the Actor network, and outputs the state value estimation function through an independent fully connected layer; The seq2seq model includes an encoder and a decoder, both of which are implemented by recurrent neural networks; the input of the encoder is the task embedding sequence, and the output of the decoder is the offloading strategy for each subtask, where a n Indicates n The offloading strategy of each task; An attention mechanism is added, which allows the decoder to focus on different parts of the source sequence, i.e., the input sequence of the encoder, at each step of output generation, and encodes the input sequence into a vector with a fixed dimension.

4. A method for scheduling mobile edge computing tasks for dependent subtasks, implemented by a mobile edge computing task scheduling system for dependent subtasks as claimed in claim 1, characterized in that: The following steps are involved: Step 1: Data reading and parsing stage; After the system is initialized, the IoT application module reads the IoT application from the MEC environment as raw data; Step 2: The DAG application parsing module receives the raw data from the IoT application module and reads the IoT application into a DAG structure through the graph structure construction algorithm, thereby obtaining the number of subtasks of the IoT application, the dependencies between the subtasks, and the computing resources required for each subtask node; The graph structure construction algorithm is as follows: constructing the DAG graph data format in sequence with the node information and edge dependencies in the task configuration information file, and finally saving it as an adjacency matrix; Step 3: Network training phase; After receiving the DAG structure data parsed by the DAG application parsing module, the network training module first starts the training process: the DAG structure data is uploaded to the MEC host, and training is performed based on the distributed actor-critic DA-C network until the network converges and reaches the expected performance indicator of reducing the overall application runtime delay. The trained network will be deployed to the offload scheduling module; Step 4: Uninstall strategy phase; Step 5: Task execution phase; If the offloading scheduling module determines that the subtask is selected to be offloaded to the MEC server for execution, the computing offloading module is responsible for execution; According to the decision of the offloading scheduling module, the corresponding task data is offloaded to the MEC server, and the computing task is completed using the server's resources; If the offload scheduling module determines that the subtask is to be executed locally, the local processing module will be responsible for execution; Complete computing tasks by mobilizing the computing resources of local devices; Step 6: Result output stage; Step 6.1: After completing the task processing, the local processing module transmits the result data to the result output module; After the computation offloading module completes the task processing on the MEC server, it transmits the computation time delay, transmission time delay, and application execution time data to the result output module; Step 6.2: The result output module summarizes and organizes the result data from the local processing module and the calculation offloading module.

5. A method for scheduling mobile edge computing tasks for dependent subtasks according to claim 4, characterized in that: The step 4 specifically comprises the following steps: Step 4.1: The offloading scheduling module receives the policy network trained by the network training module and infers the offloading policy. On the other hand, it continuously receives the data of the new DAG application parsing module, which contains the latest task information. Step 4.2: Combining the data from the policy network and the new DAG application parsing module, the offloading scheduling module calculates the probability distribution of the offloading strategy through the policy network, and then makes an offloading strategy for each subtask based on the probability distribution, that is, deciding whether the subtask should be offloaded to the MEC server for execution or chosen to be executed locally.

Images