Dependent task unloading method, terminal equipment and storage medium

Abstract

The invention discloses a dependent task unloading method, a terminal device and a storage medium, and provides a multi-dependent task double-path dynamic unloading and scheduling scheme with two state embedding layers based on the dependent task unloading method, and characteristic vectors of a plurality of dependent tasks are captured by using a double-path GRU, so that the dynamic unloading and scheduling of the dependent tasks is realized. Remodeling the dependency task into a directed acyclic graph according to the feature vector, performing feature fusion of the multi-heterogeneous dependency task to obtain an optimal scheduling path of the multi-heterogeneous dependency task, training an unloading model according to the obtained optimal scheduling path, and making an optimal unloading decision; in consideration of the limitation of local edge model training, a new edge-cloud cooperative unloading framework is provided, and a trained unloading scheduling model in a heterogeneous edge server is uploaded to a cloud for weighted aggregation updating so as to improve the generalization ability of the model. The method has the characteristics of improving the calculation efficiency to more accurately and quickly process tasks and multi-task low-overhead processing.

Description

Technical Field

[0001] The present invention relates to the field of edge computing technology, and specifically to a dependent task offloading method, a terminal device and a storage medium. Background Art

[0002] In the field of Internet of Things, edge computing technology is expected to be properly applied in smart cities and show good performance under full deployment and development. At present, the tasks of many applications are mostly composed of subtasks that are dependent on each other. Usually, such tasks can be modeled as directed acyclic graphs (DAGs) for offloading scheduling. When the amount of task data generated by mobile devices is large, it is necessary to balance the energy consumption, latency and other indicators of task processing to achieve the comprehensive optimal situation and distribute data processing tasks to edge nodes or cloud networks. This task distribution method is called task offloading of mobile edge computing. The offloading of dependent tasks can be divided into the following three categories: local execution, offloading to the edge, and offloading to the cloud; However, these existing methods only optimize the planning problem of the model. Conventional task offloading methods cannot make scheduling decisions for tasks that are dependent on each other, and cannot perform adaptive task offloading for heterogeneous multi-dependent tasks at the same time. This leads to excessive reliance on expert experience in the learning process of the intelligent agent, and a lack of self-learning ability. It is impossible to perceive the overall situation of the complex system more comprehensively, resulting in insufficient accuracy in the decision-making, which further leads to the problem of being unable to perform adaptive task offloading according to the scene in real time in the existing edge computing scenario. Therefore, it is necessary to design a dependent task offloading method, terminal device and storage medium that improves computing efficiency to process tasks more accurately and quickly and improve the adaptability of the task offloading algorithm. Summary of the invention

[0003] The purpose of the present invention is to provide a dependent task offloading method, a terminal device and a storage medium to solve the problems raised in the above background technology.

[0004] In order to solve the above technical problem, the present invention provides the following technical solution: a method for offloading dependent tasks, comprising the following steps: Step S1: Initialize the model parameters of the Actor network and the Critic network, use the current system environment state vector as part of the algorithm input, and find the centrality node; Step S2: Perform heterogeneous feature fusion on heterogeneous DAG tasks and use it as the input of the algorithm to obtain an action set; Step S3: Randomly select a scheduling action from the obtained action set Next hop path selection as task scheduling at time t , through the Actor network and the state at the current time t Get the predicted action set, Indicates that the Actor network parameters are ,state The output probability when the next action is a, X represents the action space; For the i-th action randomly selected from X, the Actor network takes the state of the environment as input and outputs the probability distribution of executing each action in a given state. The Critic network evaluates the value of the action in a given state and is used to estimate the quality of the action selected by the policy network. The Actor-Critic uses the policy gradient method to update the policy network. During the working process, the Actor network calculates the state through forward propagation. Perform each action The probability distribution of , execute the selected offloading action in the MEC environment and obtain the corresponding reward value and the system state at the next moment. The Critic network has the same structural information as the Actor network and calculates the state through forward propagation. The value function is estimated under , the output advantage value of the Actor-Critic network is used as the input of the loss function, the gradient of the loss function is calculated, the parameters of the Actor network are adjusted using the gradient, and the parameters of the Critic network are soft-updated at the same time. After the soft update is completed, the target network obtained is the current task offloading model, and the predicted action set is obtained using the current task offloading model; the action that the Actor-Critic network needs to obtain is a two-dimensional action, which is achieved by obtaining the characteristic state of the DAG task using the gating unit of GRU: ; ; In order to obtain the next-hop scheduling path and the offloading decision of the current scheduling node, the offloading decision is executed in the edge computing system to obtain the reward value of the action, and then the network model is updated; Step S4: Record the number of iterations, upload the locally updated model to the cloud for aggregation and fusion, obtain the update of the global model, and distribute the update weight according to the proportion of task data of mobile devices within the service range of the base station; Step S5: Repeat steps S3 and S4 until the exit node of the DAG task is scheduled; Step S6: When local resources can process the task, local execution is performed to calculate the task offloading delay ; When the local resource cannot process the task, execute steps S7-S8; Step S7: Process the task by offloading the task to the edge base station; Step S8: Calculate the data upload rate offloaded to the edge base station ; and the delay of offloading the processing task to the edge base station includes the transmission delay , calculate the delay , receiving delay ; Step S9: After the task is uninstalled, you will receive the corresponding reward ; Step S10: Set the current time t state ,action ,award and the latest status Collect into the experience collector; Step S11: Calculate the neural network advantage estimate using the value in the experience collector And calculate the loss gradient; Step S12: Update Actor network parameters ; Step S13: Replace the Actor network and Critic network of step S3 with the updated Actor network and Critic network respectively, and start the next round of calculation; Step S14: The edge base station runs multiple training iterations locally to update the model parameters After completing local training, the cloud server collects the model parameters updated in r rounds of iterations and updates the global model .

[0005] According to the above technical solution, in step S1, it also includes: By utilizing the powerful computing power of the central cloud server, we can process the parts that need to be processed together in the smart terminal in the task-dependent edge computing architecture due to the large amount of tasks that exceed its computing power. We can establish a task offloading model with mobile edge computing as the system structure. The task offloading model is divided into two parts: local execution and task offloading. Divide multiple edge networks. The edge servers in each edge network can solve most of the computing and communication needs in the edge network. For tasks that cannot be handled by smart devices in the edge network, the edge servers provide a large amount of computing power support. The edge servers will select the optimal strategy based on the requests sent by smart terminals, the computing resource allocation in the network, and the node bandwidth; At the same time, the dependent task offloading method based on the Actor-Critic model can more reasonably allocate the computing resources of the edge base station; the offloading scheduling model trained on the heterogeneous BS is regularly uploaded to the cloud for weighted average aggregation and then sent to update each model.

[0006] According to the above technical solution, in step S2, it also includes: The state space of the algorithm used includes the MEC system environment state and the characteristic state of the DAG task, namely MEC's ​​environmental information includes not only information data related to the edge server but also information related to the user device. In the system's decision-making process, if the task is offloaded to the edge server, the edge server's status, channel gain, bandwidth and other data are considered. The user device's computing power and status and other data information affect the system's decision. The edge server information and user device information together constitute the key reference data information when the task is scheduled for decision-making. A multi-layer perceptron (MLP) layer is used to learn the characteristic status of the edge server and user device and use it as embedded information for the system environment. In each time step, there is only one task for scheduling decision. By observing the status information in the current MEC system environment, the user equipment's offload scheduler will perform offload scheduling locally according to the determined offload scheduling order. The action space is divided into two parts, namely, the scheduling path selection action and the offload decision selection action. In each time step, a task will be scheduled and the decision will be executed in the MEC environment, at which time the system state will change; A joint scheduling scheme for heterogeneous multi-DAG tasks based on centrality measurement is proposed to achieve optimal offloading decisions while finding the best scheduling path for tasks. The global information of DAG tasks is used to find the central nodes in all tasks and record them as key decision points for priority scheduling allocation. Secondly, the scheduling characteristics of heterogeneous DAG tasks and MEC environment characteristics are combined to ensure that the scheduling path selected in each time slot is the current optimal solution.

[0007] According to the above technical solution, in step S2, it also includes: An improved GRU gating mechanism is adopted, and a dual-path scheduling offloading strategy network model is proposed to generate the optimal scheduling strategy in a heterogeneous multi-DAG task environment. The feature vector of task information is recorded as In the feature extraction process, the feature vector is updated through the input gate, forget gate and output gate operations, which can effectively capture the context information. The process includes: (1) The input gate determines whether to accept new input information at the current time step and integrates it into the current candidate feature state: ; (2) The forget gate determines which feature information of the previous time step is forgotten at the current time step: ; (3) Update the current feature status: ,in, is a new candidate feature state, which can be calculate; (4) The final characteristic state of the system It is the feature representation of the entire DAG task, which contains the context information and important features in the sequence, through which the scheduling path can be generated.

[0008] According to the above technical solution, in step S4, it includes: In the MEC system, each edge base station has its own task data for mobile device applications within its responsible area. These data are locally trained on the edge base station to obtain a local scheduling model. These locally updated models are uploaded to the cloud for aggregation and fusion to obtain an update of the global model. The local models on different edge base stations are weighted averaged according to the model weighted average fusion to obtain a global model. The global model is distributed to each edge base station as the initial parameter for the next round of training. The advantage of the model weighted average fusion is that it can make full use of the local knowledge on each edge base station to obtain a more representative and generalized global model. At the same time, by setting different weights, different base stations can be given priority, thereby improving the efficiency and performance of learning. The weights are allocated by the proportion of mobile device task data within the base station service range. Using the data volume as the weight for model fusion can make fuller use of the data on each device or node, and achieve more effective model fusion and global model update.

[0009] According to the above technical solution, in step S4, it also includes: (1) Edge base station model parameter update: The edge base station runs multiple training iterations locally to update the model parameters; ; (2) Global model aggregation update: After completing local training, the cloud server collects the model parameters updated in r rounds of iterations and updates the global model; ; in, represents the weight of the i-th edge base station; in particular, the weight parameter always satisfies ,The weight parameter can be calculated by the amount of mobile device data within the service range of each edge base station: ; (3) Sending global model parameters: Downloading the trained global model parameters to the edge base station to continue local model training; During this training and updating process, all the original data is kept in the local edge base station. This process improves the generalization ability of the model while effectively protecting the privacy of the data.

[0010] According to the above technical solution, the terminal device includes: One or more processors and memory: wherein one or more programs are stored in the memory, and when the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method.

[0011] According to the above technical solution, the multiple processors include: Local servers, edge servers.

[0012] According to the above technical solution, the storage medium includes: a computer-readable storage medium storing a computer program, and the computer program executes the steps of the method when executed by a processor.

[0013] Compared with the prior art, the beneficial effects achieved by the present invention are: 1. First, the present invention adopts a dual-path dependent task offloading method based on edge computing to more reasonably allocate the computing resources of edge base stations. The present invention adopts a gating unit similar to GRU to remodel the extracted scheduling features, realize the scheduling-offloading dual-path decision-making mechanism, and use the Actor-Critic network model to continuously update the gradient training, so as to achieve the optimal offloading decision while obtaining the optimal scheduling path of the DAG task; 2. Secondly, the present invention adopts an aggregate update method of heterogeneous offloading scheduling models to improve the generalization ability and stability of the model. The offloading scheduling models trained on heterogeneous BSs are regularly uploaded to the cloud for weighted average aggregation and then sent to update their respective models to make up for the problem of insufficient data of a single model, and finally obtain better performance and prediction ability; 3. Finally, the present invention can not only make full use of the computing resources of smart terminals and edge clouds to fully meet the processing requirements of dependent tasks, but also use a method with lower overhead to offload tasks, thereby improving the adaptability of the task offloading method. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 Schematic diagram of the TMHD algorithm for dependent tasks based on reinforcement learning in the first embodiment of the present invention; Figure 2 Schematic diagram of the convergence of TMHD at different learning rates in Example 1 of the present invention; Figure 3 This is a schematic diagram showing the impact of the effectiveness of the TMHD scheduling sequence in the first embodiment of the present invention; Figure 4This is a schematic diagram of the impact of TMHD cloud aggregation effectiveness in the first embodiment of the present invention; Figure 5 It is a schematic diagram of the convergence performance of TMHD processing heterogeneous DAG in the first embodiment of the present invention; Figure 6 Schematic diagram of the effect of the number of DAG task nodes on the delay of the task offloading algorithm in the first embodiment of the present invention; Figure 7 Schematic diagram of the effect of transmission rate on the delay of the task offloading algorithm in the first embodiment of the present invention; Figure 8 A schematic diagram of the composition of a terminal device and a storage medium provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Embodiment 1: Figure 1 A schematic diagram of a TMHD algorithm for dependent tasks based on reinforcement learning provided in Embodiment 1 of the present invention is provided. This embodiment can be applied to the scenario of dual-path dependent task offloading of edge computing. The method can be executed by a terminal device and a storage medium provided in this embodiment. The method specifically includes the following steps: Step S1: Initialize the model parameters of the Actor network and the Critic network, use the current system environment state vector as part of the algorithm input, and find the centrality node; Step S2: Perform heterogeneous feature fusion on heterogeneous DAG tasks and use it as the input of the algorithm to obtain an action set; Step S3: Randomly select a scheduling action from the constructed scheduling action set Next hop path selection as task scheduling at time t , through the Actor network and the state at the current time t Get the predicted action set, Indicates that the Actor network parameters are ,state The output probability when the next action is a, X represents the action space; For the i-th action randomly selected from X, the Actor network takes the state of the environment as input and outputs the probability distribution of executing each action in a given state. The Critic network evaluates the value of the action in a given state and is used to estimate the quality of the action selected by the policy network. The Actor-Critic uses the policy gradient method to update the policy network. During the working process, the Actor network calculates the state through forward propagation. The probability distribution of executing each action under In particular, in the MEC environment, the selected offloading action is executed and the corresponding reward value and the system state at the next moment are obtained. The Critic network has the same structural information as the Actor network and calculates the state through forward propagation. The value function is estimated under , the output advantage value of the Actor-Critic network is used as the input of the loss function, the gradient of the loss function is calculated, the parameters of the Actor network are adjusted using the gradient, and the parameters of the Critic network are soft-updated at the same time. After the soft update is completed, the target network obtained is the current task offloading model, and the predicted action set is obtained using the current task offloading model; the action that the Actor-Critic network needs to obtain is a two-dimensional action, which can be achieved by obtaining the characteristic state of the DAG task using a gating unit similar to GRU: ; ; In order to obtain the next-hop scheduling path and the offloading decision of the current scheduling node, the reward value of the action can be obtained by executing the offloading decision in the edge computing system, and then the network model can be updated.

[0017] Step S5: Repeat steps S3 and S4 until the exit node of the DAG task is scheduled; Step S6: When local resources can process the task, local execution is performed to calculate the task offloading delay ; Step S7: Otherwise, the task is processed by task offloading, and the task is offloaded to the edge base station; Step S8: Calculate the data upload rate offloaded to the edge base station ; and the delay of offloading the processing task to the edge base station includes the transmission delay , calculate the delay , receiving delay ; Step S9: After the task is uninstalled, you will receive the corresponding reward ; Step S10: Set the current time t state ,action ,award and the latest status Collect into the experience collector; Step S11: Calculate the neural network advantage estimate using the value in the experience collector And calculate the loss gradient; Step S12: Update network parameters ; Step S13: Replace the Actor network and Critic network of step S3 with the updated Actor network and Critic network respectively, and start the next round of calculation; Step S14: The edge base station runs multiple training iterations locally to update the model parameters After completing local training, the cloud server collects the model parameters updated in r rounds of iterations and updates the global model .

[0018] In this embodiment, the powerful computing power of the central cloud server is used to process the parts that need to be jointly processed in the intelligent terminal in the task-dependent edge computing architecture due to excessive task volume exceeding its computing power, and a task offloading model with mobile edge computing as the system structure is established. The task offloading model is divided into two parts, namely local execution and task offloading; For example, the local execution part of the task offloading model: if the current task is executed locally, the transmission overhead of the task is not considered, only the overhead generated by the computing task is considered. Indicates the computing resources required to process the task. The latency of local execution based on the CPU processing power of a mobile device is as follows: ; Taking into account the dependencies between tasks, the end execution time of the task is: ; in Representation Task The direct predecessor task; Exemplarily, the task offloading part in the task offloading model includes the TMHD algorithm as follows: There are a large number of intelligent terminals in the mobile edge computing architecture. When the computing power of the intelligent terminal exceeds the resources required for the task, the intelligent terminal will initiate a request to the edge cloud to jointly solve the task. When the task is offloaded to the edge base station, the task processing includes transmission time, calculation time and reception time. The task transmission start time is : ; in, is the waiting time of the task in the transmission queue, Likewise, the transfer time of a task can be calculated as: ; in is the transmission rate of the task. represents the channel bandwidth, is the noise power, is the channel gain between mobile device n and edge base station s, P is the transmit power of the mobile device, and the final transmission completion time can be calculated as: ; The start execution time of a task must be after it reaches the target server and its predecessor task completes execution, that is, ; The time required for task execution can be , in summary, the final completion time of the task can be calculated as ; The task processing result reception time will only be available when the subsequent task of the task needs to be processed locally. That is, the task reception completion time only needs to be satisfied after the task is completed, which can be expressed as, ; The transmission time can be calculated as ; In summary, the final completion time of a task can be expressed as the longest completion time of all tasks without successor nodes, that is, ; Where E represents the set of tasks without successor nodes.

[0019] In this embodiment, multiple edge networks are divided, and the edge servers in each edge network can solve most of the computing and communication needs in the edge network. For tasks that cannot be handled by smart devices in the edge network, the edge servers can provide a large amount of computing power support. The edge servers will select the optimal strategy based on the requests sent by the smart terminals and the computing resource allocation and node bandwidth in the network to solve the large amount of overhead added by smart devices in the traditional architecture to communicate through cloud servers; Exemplarily, the mobile device makes a choice based on its own system load and existing computing resources, which is generally divided into local execution operation and task offloading operation. When the local environment is sufficient to meet the needs of task operation, task offloading will not be performed and all tasks will be executed locally. When the computing and communication resources required by the task cannot be met locally, the task can be offloaded to the edge for execution;

[0020] Exemplarily, the embodiment of the present invention adopts the dependent task offloading method based on the Actor-Critic model to more reasonably allocate the computing resources of the edge base station. In order to avoid the overestimation problem caused by the local optimal solution in the Actor-Critic decision, the present invention adopts the offloading scheduling model trained on the heterogeneous BS to upload it to the cloud regularly for weighted average aggregation and then send down to update each model to make up for the problem of insufficient data of a single model, and finally obtain better performance and prediction ability. For example, reinforcement learning has several major elements: agent, state, action, and reward. In a mobile edge computing environment, the edge base station records the information of smart devices within the current edge network and the operation status of the server. The state of the environment includes the current state of the edge network and the characteristic state of the DAG task. The order of task scheduling and the candidate action set are obtained through these two types of states. The rewards obtained after executing the action will be recorded to solve the tasks that need to be processed by the smart devices in the environment in the best way.

[0021] In this embodiment, the state space of the algorithm used in the present invention includes the MEC system environment state and the characteristic state of the DAG task, namely ,,MEC environmental information includes not only information data related to edge servers but also relevant information of user devices. In the system re-decision process, if the task is offloaded to the edge server, then the edge server status, channel gain, bandwidth and other data should be considered. In addition, the computing power and status of the user device and other data information will also affect the system's decision. The edge server information and the user device information together constitute the key reference data information for task decision scheduling. The present invention uses a multi-layer perceptron (MLP) layer to learn the characteristic status of the edge server and user device and use it as embedded information of the system environment; For example, in each time step t, there is only one task for scheduling decision. By observing the status information in the current MEC system environment, the user equipment's offload scheduler will perform offload scheduling locally according to the determined offload scheduling order. The action space is divided into two parts, namely, the scheduling path selection action and the offload decision selection action. The action space of the scheduling path selection is The selection space of unloading decision actions is .

[0022] For example, at each time step t, a scheduling decision is made for a task , the decision is executed in the MEC environment, and the state of the system is Transformed into The optimization goal of the present invention is to minimize the average processing time of all user equipment tasks. The reward at time t is defined as the negative increment of the delay of all user devices after each time slot scheduling in the MEC system, which can be expressed as: ; Heterogeneous multi-DAG joint scheduling, in order to avoid the resource idle problem in the Local and BS processors caused by the irrationality of the task scheduling order of heterogeneous DAG, and find the optimal scheduling path for heterogeneous DAG tasks, the present invention proposes a joint scheduling scheme for heterogeneous multi-DAG tasks based on centrality measurement, which can achieve optimal offloading decisions while finding the optimal scheduling path for tasks, and use the global information of DAG tasks to find the centrality nodes in all tasks, and record them as key decision points for priority scheduling allocation. Secondly, by combining the scheduling characteristics of heterogeneous DAG tasks and MEC environment characteristics, it is ensured that the scheduling path selected in each time slot is the current optimal solution.

[0023] For example, given an application's DAG task ,in Represents a set of tasks, It represents the dependency between tasks and uses the central metric algorithm to measure the degree to which a node is linked by other important nodes. The basic principle is to define a random walk model on a directed graph to describe the behavior of a random walker randomly visiting each node along the directed graph. Under certain conditions, the probability of visiting each node in the extreme case converges to a stable distribution. At this time, the stable probability value of each node is its central value, which represents the importance of the node. By finding important nodes and prioritizing their scheduling and decision-making, the processing process of the entire DAG task can be accelerated. Suppose a random walk model on a graph, that is, a first-order Markov chain, whose transfer matrix is , the transition probability from one node to all the nodes connected to it is equal. Suppose we consider another completely random walk model, the elements of its transfer matrix are all , that is to say, the transition probability from any node to any node is , the linear combination of the two transfer matrices constitutes a new transfer matrix, on which a new Markov chain can be defined, which satisfies: ; in, is the damping factor, is a matrix of all 1s, yes dimensional vector, All components are 1 dimensional vector, M represents the general centrality of the directed graph. By calculating the centrality of the nodes in the DAG, we can sort the node tasks in the DAG according to their importance, and then help find the best scheduling order for the tasks.

[0024] For example, use Represents the characteristics of the task, then the characteristics of the entire DAG task of a mobile user can be expressed as: ; Second, using the adjacency matrix To represent the graph The neighbor relationship between tasks, then the state characteristics of the task can be represented as a 2D feature matrix consisting of feature states and adjacent states. In the process of processing heterogeneous multi-DAG tasks, a relationship matrix is ​​generated Perform multi-DAG task scheduling. Specifically, the system stores an adjacency matrix that expresses the DAG task dependencies in its state space. When scheduling each subtask, first set the row corresponding to the directed edge with the task as the out-degree in the adjacency matrix to 0, and then take all unscheduled tasks with in-degree 0 in the adjacency matrix as the current scheduling candidate set of the system.

[0025] In this embodiment, the present invention adopts an improved GRU gating mechanism, proposes a dual-path scheduling offloading strategy network model, generates the best scheduling strategy in a heterogeneous multi-DAG task environment, and records the feature vector of task information as In the feature extraction process, the feature vector is updated through the input gate, forget gate and output gate operations, which can effectively capture the context information. The process includes: (1) The input gate determines whether to accept new input information at the current time step and integrates it into the current candidate feature state: ; (2) The forget gate determines which feature information of the previous time step is forgotten at the current time step: ; (3) Update the current feature status: ,in, is a new candidate feature state, which can be calculate; (4) The final characteristic state of the system It is the feature representation of the entire DAG task, which contains the context information and important features in the sequence, through which the scheduling path can be generated.

[0026] Exemplarily, the last feature vector of GRU can be regarded as the feature representation of the entire DAG graph. These features contain the dependencies between nodes and important scheduling information. By utilizing these features, the generation of scheduling paths and unloading actions can be further performed through the policy network. Different from the traditional GRU policy network, the present invention takes into account the current state of the task scheduling object. and the status of servers and mobile devices in the MEC environment , so that it can make unloading decisions while finding the best scheduling path, and make next-hop path selection and current task unloading decisions according to the current task feature vector and state vector respectively. The process is as follows: ; ; in, represents the probability distribution of all candidate scheduling objects at the next moment, represents the probability distribution of the current object's uninstall decision, is a vector set , which respectively represent the previous scheduling object, the current scheduling object and the exit task of the system. Represents the status information of the current MEC system server and information related to the task data. The present invention selects the candidate task object with the highest probability for scheduling and then moves to the next state.

[0027] In this embodiment, the unloading scheduling model and update based on DRL: The Actor-Critic algorithm is an integrated algorithm that combines policy iteration and value iteration. It is usually used to solve reinforcement learning problems in continuous action space and high-dimensional state space. The algorithm uses the state of the environment as input by the Actor network and outputs the probability distribution of executing each action in a given state. The Critic network evaluates the value of the action in a given state and is used to estimate the quality of the action selected by the policy network. The Actor-Critic uses the policy gradient method to update the policy network. During the scheduling training process, the training goal is to find the best scheduling strategy, which can maximize the total discounted reward, that is, .in, is the discount factor, which affects how much importance the reinforcement learning algorithm places on future rewards.

[0028] For example, the input of the Actor network includes the status information of the user device, the status information of the edge base station, the characteristic status information of the DAG task, etc. During the working process, the Actor network calculates the state through forward propagation. The probability distribution of executing each action under ,in, Indicates in status Next select action The probability of executing the selected offloading action in the MEC environment and obtaining the corresponding reward value and the system state at the next moment. The Critic network has the same structural information as the Actor network and calculates the state through forward propagation. The value function estimation under this condition uses tanh as the activation function to convert the state of the system at this time As the input of the Critic network, the output is the value of the current state; For example, in the Actor-Critic policy network, we define the loss function of the Actor network through the policy gradient method, and update the Actor network with the goal of maximizing the cumulative reward. The goal of the Critic network is to evaluate the quality of the strategy to help the Actor network improve the strategy. The update of the Critic network is usually updated by minimizing the Temporal Difference (TD) error using the Temporal Difference (TD) method. Its function can be expressed as: ; ; ; ; in, Used to weigh the difference between bias and variance, is the reward of the present moment, is the discount factor, is the state value at the next moment, is the state value at the current moment, we can use each state TD (Temporal Difference) error The square of the current value function is used to measure The error is calculated and the parameters of the Critic network are updated with the error. Finally, the minimum value between the clipped target and the uncut target is taken as the target value of the policy function. In this embodiment, in the MEC system, each edge base station has task data of mobile device applications within its responsible area. These data are locally trained on the edge base station to obtain a local scheduling model. The present invention uploads these locally updated models to the cloud for aggregation and fusion, thereby obtaining an update of the global model. The local models on different edge base stations are weighted averaged according to the model weighted average fusion to obtain a global model; and the global model is distributed to each edge base station as the initial parameter for the next round of training. The advantage of the model weighted average fusion is that it can make full use of the local knowledge on each edge base station to obtain a more representative and generalized global model. At the same time, by setting different weights, it is also possible to give priority to different base stations and improve the efficiency and performance of learning. The weight allocation is considered based on the proportion of mobile device task data within the base station service range. This is because the location of mobile users in the MEC environment is constantly changing, so the amount of data in the areas served by different base stations may be different. A base station with a large amount of data within its service range means that it has more samples and feature information, and can provide more comprehensive and representative data. Conversely, devices with less data may lead to instability in model training or insufficient generalization ability due to insufficient data. Therefore, using data volume as a weight for model fusion can make more full use of the data on each device or node, thereby achieving more effective model fusion and global model update; Exemplarily, the operation process of the cloud model aggregation module formed above includes: (1) Edge base station model parameter update: The edge base station runs multiple training iterations locally to update the model parameters; ; (2) Global model aggregation update: After completing local training, the cloud server collects the model parameters updated in r rounds of iterations and updates the global model; ; in, represents the weight of the i-th edge base station; in particular, the weight parameter always satisfies The weight parameter can be calculated by the amount of mobile device data within the service range of each edge base station: ; (3) Sending global model parameters: Downloading the trained global model parameters to the edge base station to continue local model training; During this training and updating process, all the original data is kept in the local edge base station. This process improves the generalization ability of the model while effectively protecting the privacy of the data.

[0029] Embodiment 2: Embodiment 2 of the present invention provides a terminal device and a storage medium. Figure 8 A schematic diagram of the composition of a terminal device and a storage medium provided in Embodiment 2 of the present invention is shown in FIG. Figure 8 As shown, the terminal device includes: one or more processors; a memory on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the above method of the present invention.

[0030] In some embodiments of the present invention, the plurality of processors include: Local servers, used to offload tasks when local resources are sufficient; Edge servers are used to offload tasks when local resources are insufficient.

[0031] In an embodiment of the present invention, the storage medium includes: A computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0032] Example 3: Example 3 of the present invention analyzes the experimental operation results formed by the above-mentioned Example 1 and Example 2: In this embodiment, the performance of the experimental platform used in the embodiment of the present invention is: Intel (R) Xeon (R) Gold6330 CPU @ 2.00GHz, processor basic frequency 2.00GHz, memory 32*4GB, software environment is Ubuntu18.04.6, Python3.7; In order to verify the processing capability of TMHD for heterogeneous DAG tasks with different numbers of nodes, four types of heterogeneous DAG datasets with different topologies were generated, namely DAG100-15, DAG100-20, DAG100-25, and DAG100-30; the number of nodes of DAG tasks in each dataset was 15, 20, 25, and 30, respectively. In order to verify the effectiveness of TMHD cloud-based aggregation updates, four types of datasets with different numbers of DAG tasks were generated, namely DAG100, DAG200, DAG300, and DAG400. Each dataset contains 100, 200, 300, and 400 heterogeneous DAG tasks, respectively, and the number of nodes in each DAG task is fixed at 20. In order to study the impact of dependent task offloading performance in the mobile edge computing architecture, we use the impact of learning rate on TMHD algorithm latency, the effectiveness of scheduling order, the effectiveness of cloud aggregation, the convergence performance of TMHD on heterogeneous DAGs, the impact of the number of DAG task nodes on TMHD algorithm latency, and the impact of transmission rate on the latency of different structure-dependent task offloading algorithms as key indicators for performance evaluation in this experiment on four data sets. Through experimental tests, we compared the task processing latency of the TMHD algorithm of the present invention with that of Local, Edge, Random, Greedy, HEFT, Round-Robin and other algorithms under different environment settings; In this embodiment, Figure 2 The variation curves of the average task completion delay of TMHD under different learning rates are compared. Considering that the optimal learning rates for data sets of different sizes are different, we verify the corresponding optimal learning rates on the data sets DAG100, DAG200, DAG300, and DAG400. According to the experimental results, the task processing efficiency of the data set DAG100 increases as the learning rate increases from 1.6E-5 to 2E-5, but if the learning rate continues to increase, the processing efficiency will decrease. Similarly, the task processing efficiency of the data set DAG400 increases gradually as the learning rate increases to 1.1E-5. If the learning rate continues to increase, the processing efficiency will decrease. In summary, As mentioned above, the larger the data set, the smaller the optimal learning rate for task offloading processing. In addition, when a larger learning rate is used, overfitting will occur, which will greatly increase the average task processing time. This is because the larger the data set, the more information it contains. The training process may require more iterations to achieve the best performance. Therefore, through this experiment, we clearly understand the optimal learning rate size corresponding to each data set. The optimal learning rate corresponding to DAG100 is 2E-5, DAG200 is 1.7E-5, DAG300 is 1.5E-5, and DAG400 is 1.1E-5. All the experiments described below are tested under the corresponding optimal learning rate size.

[0033] In this embodiment, Figure 3 The effectiveness of the scheduling order arrangement of the present invention is demonstrated; TMHD will learn the scheduling characteristics of DAG tasks according to the dependencies of DAG tasks, and finally obtain the optimal scheduling path for dependent tasks. In this process, we embed the relevant information of DAG tasks into the input features of tasks, and use the dual-path GRU strategy network to extract the features of tasks and obtain the optimal scheduling path, which ultimately affects the response processing delay of tasks. We verify the effectiveness of TMHD on data sets DAG100, DAG200, DAG300, and DAG400. We compare TMHD with the method of scheduling and sorting using the DRL algorithm, and use other dependent task processing methods as benchmarks to compare the average delay of task processing. This experiment verifies that for different DAGs, G task dataset, TMHD can find an optimal task scheduling sequence for each heterogeneous DAG task to effectively reduce the average processing delay of the task. Compared with the existing dependent task processing methods, TMHD has the lowest average task processing delay. Specifically, compared with the method of using DRL for task scheduling and sorting, the average task processing time of TMHD on 4 different datasets is reduced by 30.52ms, 24.34ms, 26.35ms, and 454.34ms, respectively. Compared with the existing scheduling method that prioritizes tasks according to task completion time, the average task processing time of TMHD on 4 different datasets is reduced by 18.68ms, 71.79ms, 70.99ms, and 64.32ms, respectively.

[0034] In this embodiment, Figure 4 The effectiveness of the cloud-based aggregation of the present invention is demonstrated; TMHD will regularly upload the offload scheduling model trained on the edge to the cloud for aggregation and update, so as to improve the generalization ability of a single scheduling model and avoid model training to the local optimum. In order to evaluate the effectiveness of this method, we use four heterogeneous BS trained models for weighted aggregation and update, where the data on each BS are DAG100, DAG200, DAG300, and DAG400, respectively. We will compare the performance of TMHD with the method without cloud-based model aggregation and update, and analyze the average task processing delay and convergence speed of each individual BS model after aggregation. According to the experimental results, compared with the method without cloud-based model aggregation and update, TMHD can accelerate the convergence speed of each heterogeneous BS model while reducing the average processing time of tasks. Among them, the average task processing time of TMHD on 4 different data sets was reduced by 171.43ms, 105.3ms, 72.79ms, and 95.74ms, respectively.

[0035] In this embodiment, Figure 5The convergence performance of TMH in task processing in heterogeneous DAG environments was studied. We used the datasets DAG100-15, DAG100-20, DAG100-25, and DAG100-30 to verify the policy performance of TMHD in processing heterogeneous DAG tasks, and recorded the policy loss changes during the TMHD training process, and demonstrated the training process of TMHD. According to the experimental results, the loss of TMHD dropped sharply in the first 400 rounds of training, and began to stabilize in a lower range after 600 rounds of training, which shows that TMHD has good convergence performance.

[0036] In this embodiment, Figure 6 The effect of the TMHD method on task processing performance under different numbers of DAG task nodes is studied. In this experiment, we use the data sets DAG100-15, DAG100-20, DAG100-25, and DAG100-30 to verify the performance of TMHD. Compared with the existing dependent task processing algorithms, the average task time of TMHD on 4 different data sets is reduced by 29.24ms, 25.82ms, 61.83ms, and 97.29ms, respectively. In summary, for different heterogeneous DAG tasks, TMHD always maintains a lower average task processing delay than other algorithms. It can be seen that TMHD can better adapt to changes in the number of task nodes and ensure that computationally intensive program tasks can be completed within their effective time.

[0037] In this embodiment, Figure 7 We evaluated the task processing performance of TMHD under different transmission rates between the end and the edge on the datasets DAG100-15, DAG100-20, DAG100-25, and DAG100-30. In the experiment, we controlled the transmission bandwidth of the task from 7Mbps to 10Mbps to conduct 4 comparative experiments to evaluate the performance of TMHD. The evaluation index is also the average processing delay of the task. The experimental results show that the faster the transmission rate between the end and the edge, the shorter the time to complete the task processing. Moreover, for different DAG task topologies, TMHD always has lower processing delays than other methods. Compared with existing methods, the average task processing delay of TMHD on 4 different datasets is reduced by 19.48ms, 29.09ms, 61.74ms, and 81.6ms on average.

[0038] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0039] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for offloading dependent tasks, characterized in that: The method specifically comprises the following steps: Step S1: Initialize the model parameters of the Actor network and the Critic network, use the current system environment state vector as part of the algorithm input, and find the centrality node; Step S2: Perform heterogeneous feature fusion on heterogeneous DAG tasks and use it as the input of the algorithm to obtain an action set; Step S3: Randomly select a scheduling action from the obtained action set Next hop path selection as task scheduling at time t , through the Actor network and the state at the current time t Get the predicted action set, Indicates that the Actor network parameters are ,state The output probability when the next action is a, X represents the action space; For the i-th action randomly selected from X, the Actor network takes the state of the environment as input and outputs the probability distribution of executing each action in a given state. The Critic network evaluates the value of the action in a given state and is used to estimate the quality of the action selected by the policy network. The Actor-Critic uses the policy gradient method to update the policy network. During the working process, the Actor network calculates the state through forward propagation. Perform each action The probability distribution of , execute the selected offloading action in the MEC environment and obtain the corresponding reward value and the system state at the next moment. The Critic network has the same structural information as the Actor network and calculates the state through forward propagation. The value function is estimated under , the output advantage value of the Actor-Critic network is used as the input of the loss function, the gradient of the loss function is calculated, the parameters of the Actor network are adjusted using the gradient, and the parameters of the Critic network are soft-updated at the same time. After the soft update is completed, the target network obtained is the current task offloading model, and the predicted action set is obtained using the current task offloading model; the action that the Actor-Critic network needs to obtain is a two-dimensional action, which is achieved by using the gating unit of GRU to obtain the characteristic state of the DAG task: ； ； In order to obtain the next-hop scheduling path and the offloading decision of the current scheduling node, the offloading decision is executed in the edge computing system to obtain the reward value of the action, and then the network model is updated; Step S4: Record the number of iterations, upload the locally updated model to the cloud for aggregation and fusion, obtain the update of the global model, and distribute the update weight according to the proportion of task data of mobile devices within the service range of the base station; Step S5: Repeat steps S3 and S4 until the exit node of the DAG task is scheduled; Step S6: When local resources can process the task, local execution is performed to calculate the task offloading delay ; When the local resources cannot process the task, execute steps S7-S8; Step S7: Process the task by performing task offloading, and offload the task to the edge base station; Step S8: Calculate the data upload rate offloaded to the edge base station ; and the delay of offloading the processing task to the edge base station includes the transmission delay , calculate the delay , receiving delay ; Step S9: After the task is uninstalled, you will receive the corresponding reward ; Step S10: Set the current time t state ,action ,award and the latest status Collect into the experience collector; Step S11: Calculate the neural network advantage estimate using the value in the experience collector And calculate the loss gradient; Step S12: Update Actor network parameters ; Step S13: Replace the Actor network and Critic network of step S3 with the updated Actor network and Critic network respectively, and start the next round of calculation; Step S14: The edge base station runs multiple training iterations locally to update the model parameters After completing local training, the cloud server collects the model parameters updated in r rounds of iterations and updates the global model .

2. A method for offloading dependent tasks according to claim 1, characterized in that: In the step S1, it also includes: Using the computing power of the central cloud server, we can process the parts that need to be processed together in the intelligent terminal in the task-dependent edge computing architecture due to the large amount of tasks that exceed its computing power. We can establish a task offloading model with mobile edge computing as the system structure. The task offloading model is divided into two parts: local execution and task offloading. Divide multiple edge networks. The edge servers in each edge network solve most of the computing and communication needs in the edge network. For tasks that cannot be handled by smart devices in the edge network, the edge servers provide computing power support. The edge servers will select the optimal strategy based on the requests sent by smart terminals, the computing resource allocation in the network, and the node bandwidth; At the same time, the dependent task offloading method based on the Actor-Critic model is used to allocate the computing resources of the edge base station; the offloading scheduling model trained on the heterogeneous BS is regularly uploaded to the cloud for weighted average aggregation and then sent to update each model.

3. A method for unloading dependent tasks according to claim 2, characterized in that: In the step S2, it includes: The state space of the algorithm used includes the MEC system environment state and the characteristic state of the DAG task, namely MEC's ​​environmental information includes not only information data related to the edge server but also relevant information about the user device. In the system's decision-making process, if the task is offloaded to the edge server, the edge server's status, channel gain, bandwidth and other data are considered. The computing power and status data of the user device affect the system's decision. The edge server information and the user device information together constitute the key reference data information for task decision scheduling. A multi-layer perceptron (MLP) layer is used to learn the characteristic status of the edge server and user device and use it as embedded information for the system environment. In each time step, there is only one task for scheduling decision. By observing the status information in the current MEC system environment, the user equipment's offload scheduler performs offload scheduling locally according to the determined offload scheduling order. The action space is divided into two parts, namely, the scheduling path selection action and the offload decision selection action. In each time step, a task is scheduled and the decision is executed in the MEC environment, at which time the system state changes; A joint scheduling scheme for heterogeneous multi-DAG tasks based on centrality measurement can make optimal offloading decisions while finding the best scheduling path for tasks. The global information of DAG tasks is used to find the central nodes in all tasks and record them as key decision points for priority scheduling allocation. Secondly, the scheduling characteristics of heterogeneous DAG tasks and MEC environment characteristics are combined to ensure that the scheduling path selected in each time slot is the current optimal solution.

4. A method for offloading dependent tasks according to claim 3, characterized in that: In the step S2, it also includes: The improved GRU gating mechanism and a dual-path scheduling offloading strategy network model are used to generate the optimal scheduling strategy in a heterogeneous multi-DAG task environment. The feature vector of task information is recorded as In the feature extraction process, the feature vector is updated through the input gate, forget gate and output gate operations, which can effectively capture the context information. The process includes: (1) The input gate determines whether to accept new input information at the current time step and integrates it into the current candidate feature state: ; (2) The forget gate determines which feature information of the previous time step is forgotten at the current time step: ; (3) Update the current feature status: ,in, is a new candidate feature state, which can be calculate; (4) Final characteristic state of the system It is the feature representation of the entire DAG task, which contains the context information and important features in the sequence, and the scheduling path is generated through these features.

5. A method for offloading dependent tasks according to claim 4, characterized in that: In the step S4, it includes: In the MEC system, each edge base station has its own task data for mobile device applications within its responsible area. These data are locally trained on the edge base station to obtain a local offload scheduling model. These locally updated models are uploaded to the cloud for aggregation and fusion to obtain the update of the global model. The local models on different edge base stations are weighted averaged according to the model weighted average fusion to obtain the global model; and the global model is distributed to each edge base station as the initial parameter for the next round of training. The model weighted average fusion can make full use of the local knowledge on each edge base station to obtain the global model. At the same time, by setting different weights, different base stations can be given priority, thereby improving the efficiency and performance of learning. The weights are allocated based on the proportion of mobile device task data within the base station service range, and the data volume is used as the weight for model fusion. The data on each device or node is used to achieve model fusion and global model update.

6. A method for offloading dependent tasks according to claim 5, characterized in that: In the step S4, it also includes: (1) Edge base station model parameter update: The edge base station runs multiple training iterations locally to update the model parameters; ； (2) Global model aggregation update: After completing local training, the cloud server collects The model parameters updated in the round iteration and the global model are updated; ； in, represents the weight of the i-th edge base station; in particular, the weight parameter always satisfies , the weight parameter can be calculated by the amount of mobile device data within the service range of each edge base station: ; (3) Sending global model parameters: Downloading the trained global model parameters to the edge base station to continue local model training; During this training and updating process, all the original data is kept in the local edge base station. This process improves the generalization ability of the model while protecting the privacy of the data.

7. A terminal device, characterized in that: The terminal device comprises: One or more processors and a memory: wherein one or more programs are stored on the memory, and when the one or more programs are executed by the one or more processors, the one or more processors execute the steps of the method described in claim 1 above.

8. A terminal device according to claim 7, characterized in that: The multiple processors include: a local server and an edge server.

9. A storage medium, characterized in that: The storage medium comprises: a computer-readable storage medium storing a computer program, wherein the computer program executes the steps of the method according to claim 1 when executed by a processor.

Images