Method for re-scheduling edge service tasks based on HG-Trans

Abstract

The application discloses a method for rescheduling of edge service tasks based on HG-Trans, comprising the following steps: S1, modeling a dynamic task scheduling problem as a Markov decision process, and converting a scheduling state into a heterogeneous graph structure; S2, inputting the heterogeneous graph structure into an HG-Trans network to obtain an estimated state value and a probability distribution of all actions; and S3, inputting the estimated state value and the probability distribution of all actions into a decision network to output a task scheduling strategy. The method uses a heterogeneous graph neural network-based edge fault scene task rescheduling strategy to comprehensively improve the task scheduling performance of an edge computing system in a fault scene. The method solves the problem that existing scheduling methods rely on local information or simple rules, lack an efficient fault recovery mechanism, and cannot quickly adjust a task execution path after a fault occurs, reduces scheduling delay, and realizes global optimal allocation of resources.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of task management, and particularly relates to a method for rescheduling of edge service tasks based on HG-Trans. BACKGROUND

[0002] With the rapid development of Internet of Things (IoT) and Mobile Edge Computing (MEC) technologies, more and more computing tasks are transferred from centralized cloud processing to edge devices close to data sources to shorten response time, reduce data transmission delay and improve computing efficiency. However, edge computing environments face many challenges due to their inherent distributed and heterogeneous characteristics, especially in terms of device failures and uncertainties. Compared with traditional centralized computing architecture, edge nodes are often resource-constrained devices such as low-power servers, IoT sensors or smart terminals, which are vulnerable to hardware damage, power consumption limitations, unstable network connections and other problems. Whether it is an edge device that suddenly goes offline or a communication delay between nodes due to network congestion, it will affect the tasks being executed. If not handled in a timely manner, it may result in task loss, data inconsistency, reduced service quality, and even affect the stability of the entire system. Therefore, in the event of a failure, how to effectively recover interrupted tasks and minimize the recovery cost becomes a key issue to ensure the reliability and performance of edge computing systems.

[0003] Directed acyclic graph (DAG) is a common task representation form, which is widely used in workflow management, data processing and computing task allocation. DAG task scheduling aims to assign tasks to appropriate computing nodes according to the dependency relationship and resource constraints of tasks. The DAG task scheduling problem in distributed heterogeneous computing environment has been widely studied. H. Topcuoglu proposed HEFT algorithm, which assigns each subtask to the processor with the least execution time. Q. Shen et al. proposed DTOSC for DAG task offloading and service caching in vehicle edge computing. Y. Liu developed MAMTS, which assigns different tasks to different computing topologies according to their priorities. The above methods rely on static scheduling, which determines the scheduling scheme in advance before the task starts execution, and does not consider the changes at runtime, so it is difficult to deal with dynamic failures. Dynamic scheduling method adjusts according to the actual situation at runtime, which has better adaptability. J. Yan proposed an actor-critic DRL to learn the optimal DAG subtask allocation of access points. J. Wang proposed a DAG task offloading method based on meta-reinforcement learning. X. Wei developed a DRL-based joint optimization of UAV trajectory planning and DAG task scheduling algorithm. In order to effectively extract the dependency relationship between subtasks, H. Lee used graph convolutional network (GCN) for DAG task scheduling. J. Chen proposed a DAG task offloading algorithm named ACE, which uses GCN to capture the topological information of DAG subtasks. The above researches do not consider the influence of fault events on task interruption in the dynamic scheduling process. For the fault problem, Lee and Gil proposed a checkpoint and replication based on clustering heuristic (CRCH) to schedule jobs and tolerate faults in cloud framework. Wu proposed an improved workflow scheduling and fault tolerance process integrated scheduling mechanism. Malik completes the fault scheduling and detection mechanism through the dynamic standby replication (LSR) strategy. The above researches provide many valuable methods for task scheduling and fault recovery problem, but do not consider how to extract information state features in the scheduling process under fault conditions, effectively integrate task information and scheduling information and quickly process interrupted tasks. Under this background, it is necessary to develop a rescheduling method that can perceive the task dependency relationship and environment state when a fault occurs and migrate the failed task to other edge devices for execution with the least cost. SUMMARY

[0004] In view of the above problems in the prior art, the rescheduling method for edge service tasks based on HG-Trans provided by the present application solves the problem that the existing scheduling method relies on local information or simple rules, lacks efficient fault recovery mechanism, and cannot quickly adjust the task execution path after a fault occurs.

[0005] In order to achieve the above object, the technical scheme adopted by the present application is as follows: a method for re-scheduling of edge service tasks based on HG-Trans, comprising the following steps:

[0006] S1, modeling the dynamic task scheduling problem as a Markov decision process, and converting the scheduling state into a heterogeneous graph structure;

[0007] S2, inputting the heterogeneous graph structure into the HG-Trans network to obtain the estimated state value and the probability distribution of all actions;

[0008] S3, inputting the estimated state value and the probability distribution of all actions into the decision network to output the task scheduling strategy.

[0009] Further, in S1, the method for modeling the dynamic task scheduling problem as a Markov decision process is as follows:

[0010] (1) setting the state: the state at any time is composed of the comprehensive reflection of all task arrangements and computing resource states at the corresponding time;

[0011] (2) setting the action: the action at any time is defined as a T-S pair, T is a task node, and S is a server node;

[0012] (3) setting the transition function: the transition function represents the probability of the action a transitioning from the current state s to the next state s', and the expression of the transition function P a (s,s′) is as follows:

[0013] P a (s,s′)=P(s t+1 =s′|s t =s,a t =a)

[0014] In the formula, s t is the state at time t, s t+1 is the state at time t+1, and a t is the action at time t;

[0015] (4) setting the reward: for any time, the expression of the reward function R(Makespan) is as follows:

[0016]

[0017] In the formula, R(·) is the reward function, Makespan is the maximum time span for task completion, Makespan(HEFT) is the maximum time for HEFT completion, HEFT is the Heterogeneous Earliest Finish Time algorithm, which is the most commonly used heuristic algorithm in task scheduling.

[0018] Furthermore: In S1, the heterogeneous graph structure H t =(T,S,ε) t ), where ε t For each TS pair, there is an arc set, with each task node corresponding to a server node.

[0019] Further: S2 includes the following sub-steps:

[0020] S21. Input the heterogeneous graph structure into the multi-layer TransformerConv, update the node features, and generate node embeddings.

[0021] S22. Embed the nodes into the input multilayer perceptron to generate the probability distribution of estimated state values ​​and actions.

[0022] Further: In S21, the method for updating the features of each node using a single-layer TransformerConv includes the following sub-steps:

[0023] S211. Input the heterogeneous graph structure into a multi-layer TransformerConv, and perform weighted aggregation of nodes and their neighboring nodes based on the Transformer's self-attention mechanism, calculating the first-order neighborhood of node i. Attention coefficient A between nodes j in ij ;

[0024]

[0025] In the formula, Q i For the query vector, K j Let d be the key vector. k Let T be the dimension of the key vector, and T be the matrix transpose.

[0026]

[0027] S212. Normalize the attention coefficients within the neighborhood using the softmax function, and calculate the relationship between node i and its first-order neighborhood. Attention weights α between nodes j in the array ij ;

[0028]

[0029] S213. The information of neighboring nodes is weighted and aggregated using attention weights to update the node's features. The updated node features h... i The specific expression is:

[0030]

[0031] In the formula, σ is the nonlinear activation function, and V jThe feature representation of the node j.

[0032] Further, in the S22, the expression for generating the estimated state value V is specifically:

[0033]

[0034] where N is the total number of nodes, W v is a projection matrix, W v ∈R d×1 , and R is a matrix.

[0035] Further, in the S22, the method for generating the probability distribution of the action includes the following steps:

[0036] S221, aggregating the node embeddings of the available tasks to generate a batch matrix H T ;

[0037] H T = [h1, h2,..., h M ] T

[0038] where h M is the node embedding of the Mth task.

[0039] S222, mapping the batch matrix into a one-dimensional vector space to generate a score for each task, and normalizing the score through a fully connected layer Softmax function to obtain the probability distribution of the action, where the expression of the probability π l of the lth action is specifically:

[0040]

[0041] where o l is the lth component in the Softmax function output vector o.

[0042] Further, the S3 is specifically:

[0043] The policy gradient theorem optimizes the policy network, establishes a target function of the policy network according to the estimated state value and the probability distribution of all actions, and minimizes the target function to search for the optimal scheduling action;

[0044] where the expression of the target function is specifically:

[0045]

[0046] where H is the entropy function of the probability distribution, β is a hyperparameter controlling the influence of entropy regularization, and A(s, a) is an advantage function measuring the advantage of the action a in the state s over the average value, which is calculated by (st a t r t s t+1 ) and an empirical estimate of the current estimate of the state value V, π(a|s) is a policy function, is the gradient of the policy network parameters θ.

[0047] The beneficial effects of the present application are: the present application provides a task rescheduling method for edge service based on HG-Trans, which uses an edge fault scenario task rescheduling strategy based on a heterogeneous graph neural network to comprehensively improve the task scheduling performance of an edge computing system in a fault scenario. The existing scheduling method relies on local information or simple rules, lacks an efficient fault recovery mechanism, and cannot quickly adjust the task execution path after a fault occurs. The problem is solved, which reduces the scheduling delay while realizing the global optimal allocation of resources. Compared with the prior art, the following effects are achieved:

[0048] (1) The task rescheduling problem is modeled as a Markov decision process, combined with a reinforcement learning strategy, which can dynamically adapt to node failures in an edge computing scenario and adjust the scheduling strategy in real time.

[0049] (2) A fault tolerance mechanism is introduced in the state modeling, which automatically identifies fault nodes and adaptively adjusts interrupted tasks through the state representation of the heterogeneous graph structure, thereby improving the success rate of task rescheduling execution.

[0050] (3) The heterogeneous graph embedding method based on Transformer can efficiently capture the complex relationship between tasks and computing resources, optimize the scheduling strategy, reduce the scheduling delay, and realize the global optimal allocation of resources. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 is a flowchart of the task rescheduling method for edge service based on HG-Trans of the present application.

[0052] Figure 2 is a framework of the edge fault scenario task rescheduling strategy based on a heterogeneous graph neural network.

[0053] Figure 3 is a HG-Trans network structure diagram. DETAILED DESCRIPTION

[0054] The specific embodiments of the present application are described below to facilitate understanding of the present application by those skilled in the art, but it should be clear that the present application is not limited to the scope of the specific embodiments. It is obvious to those skilled in the art that all the inventions utilizing the concept of the present application are within the scope of the present application as defined and limited by the appended claims.

[0055] As Figure 1 shown, in one embodiment of the application, the method for rescheduling edge service tasks based on HG-Trans includes the following steps:

[0056] S1, model the dynamic task scheduling problem as a Markov decision process, and convert the scheduling state into a heterogeneous graph structure;

[0057] S2, input the heterogeneous graph structure into the HG-Trans network to obtain the estimated state value and the probability distribution of all actions;

[0058] S3, input the estimated state value and the probability distribution of all actions into the decision network to output the task scheduling strategy.

[0059] In this embodiment, the method of the application uses an edge fault scenario task rescheduling strategy based on a heterogeneous graph neural network, and the framework is as shown in Figure 2 The application builds an edge simulation environment based on an edge network model and a resource fault model, and establishes a task computing model according to the edge network architecture. Task rescheduling is a continuous decision-making process, which assigns tasks to executable computing resources at each state by iteratively taking scheduling actions until all tasks are scheduled. In each iteration, first, the scheduling state is converted into a heterogeneous graph structure, and then the heterogeneous graph structure with a two-stage embedding process is input into the HG-Trans network, which is applied to the processing of heterogeneous graph convolution, and the feature embeddings of operations and nodes are extracted. The decision network uses these feature embeddings to generate an action probability distribution and samples a scheduling action from it.

[0060] In S1, the method for modeling the dynamic task scheduling problem as a Markov decision process is as follows:

[0061] (1) Set state: the comprehensive reflection of all task arrangements and computing resource states at any time constitutes the state at the corresponding time;

[0062] The comprehensive reflection of all task arrangements and computing resource states at t constitutes the state s t The information represented in s t is approximated as information about running tasks, ready tasks, and their descendants. The state of the server resources is represented by a vector that contains the type of each computing resource and its estimated availability time.

[0063] (2) Set action: define the action at any time as a T-S pair, T is a task node, and S is a server node;

[0064] In the action definition process, task selection and resource allocation are taken as a composite decision. At time t, the action a t ∈At is defined as a T-S pair. When the server resource is available, the action is to select the task running on this computing resource. When the server resource is unavailable due to failure, the RL agent quickly responds to the task executing on the failed server to re-enter it into the pending queue, and clears all relevant computing resource allocation information of the task on the failed resource.

[0065] (3) Set the transfer function: the transfer function represents the probability of the action a from the current state s to the next state s', and the transfer function P a (s,s') is specifically expressed as:

[0066] P a (s,s') = P(s t+1 =s'|s t =s,a t =a)

[0067] In the formula, s t is the state at time t, s t+1 is the state at time t+1, and a t is the action at time t. The present application distinguishes two different states by the topological structure and characteristics of the heterogeneous graph.

[0068] (4) Set the reward: for any time, the expression of the reward function r(Makespan) is specifically:

[0069]

[0070] In the formula, R(·) is the reward function, Makespan is the maximum time span of task completion, Makespan(HEFT) is the maximum time of HEFT completion, and HEFT is the earliest completion time algorithm of heterogeneity, which is the most commonly used heuristic algorithm in task scheduling.

[0071] In the embodiment, the reward provides feedback information for the RL agent on how it performs in the optimization target. Since the HEFT algorithm has stability and applicability in the scheduling problem, the reward function is defined by normalizing the current scheduling Makespan and the Makespan of the HEFT baseline algorithm. The RL agent performs operations according to the accessed state and the current strategy, and gradually adjusts the strategy to optimize the objective function, i.e. minimizes the Makespan, by interacting with the problem to be solved.

[0072] The present application models the dynamic task scheduling problem as a Markov decision process (MDP). At time t, the agent observes the current system state s t and makes a decision a tThat is, at the current time T(t), the unexecuted tasks are assigned to the available computing resources, and then the environment moves to t+1, and the above process is iterated continuously until all tasks are scheduled.

[0073] In the S1, the heterogeneous graph structure H t =(T,S,ε t ), wherein the arc set of T-S pair is ε t , and each task node corresponds to a server node.

[0074] The S2 includes the following steps:

[0075] S21, input the heterogeneous graph structure into the multi-layer TransformerConv, update the features of the nodes, and generate node embeddings;

[0076] S22, input the node embedding into the multi-layer perceptron, generate the estimated state value and the probability distribution of the action.

[0077] In the embodiment, the HG-Trans network is as shown in Figure 3 The network of HG-Trans is used to process the graph topology and the node features.

[0078] In the S21, the method for updating the features of each node by the single-layer TransformerConv includes the following steps:

[0079] S211, input the heterogeneous graph structure into the multi-layer TransformerConv, and perform weighted aggregation on the node and its neighbor nodes based on the self-attention mechanism of the Transformer, calculate the attention coefficient A ij between the node i and the node j in the first-order neighborhood of the node i;

[0080]

[0081] In the formula, Q i is a query vector, K j is a key vector, d k is the dimension of the key vector, and T is the matrix transpose,

[0082]

[0083] S212, normalize the attention coefficient in the neighborhood by using the softmax function, and calculate the attention weight a ij between the node i and the node j in the first-order neighborhood of the node i;

[0084]

[0085] S213, aggregate the information of the neighbor nodes by weighting with the attention weight, update the feature of the node, and the updated feature of the node h i The expression of the estimated state value V is specifically:

[0086]

[0087] In the formula, sigma is a nonlinear activation function, V j is the feature representation of node j.

[0088] In the S22, the expression of the estimated state value V is specifically:

[0089]

[0090] In the formula, N is the total number of nodes, W v is a projection matrix, W v ∈R d×1 , and R is a matrix.

[0091] In the embodiment, the mean pooling is used to aggregate all node embeddings, and a one-dimensional projection is used to estimate the state value V, is the global feature vector after mean pooling. The neural network structure of the improved “actor-critic” algorithm (A2C) is used for learning of the RL agent, and the value function V is improved by minimizing the mean square error of the Bellman function.

[0092] The “actor-critic” algorithm (A2C) is a reinforcement learning algorithm based on a policy function and a value function, which includes two networks. One is the policy network, which outputs an action a according to the current state s and takes an action. The second is the value network, which is responsible for evaluating the goodness of the current action a. In the decision optimization, two neural networks in A2C are improved. One is to improve the estimation of the value network by minimizing the mean square error of the Bellman function. The second is to convert the maximization problem of cumulative rewards into the combined optimization of policy network gradient and advantage function by the policy gradient theorem, and add the entropy of the policy to the objective function minimized by the policy network.

[0093] In the S22, the method for generating the probability distribution of the action includes the following steps:

[0094] S221, aggregate the node embeddings of the available tasks to generate a batch matrix H T ;

[0095] H T =[h1,h2,...,h M ] T

[0096] In the formula, hM embedding of the node for the Mth task;

[0097] S222, mapping the batch matrix into a one-dimensional vector space, generating a score of each task, and normalizing the score through a Softmax function of a full connection layer to obtain a probability distribution of the action, wherein the probability of the lth action is l The expression of the formula is specifically as follows:

[0098]

[0099] In the formula, o l is the lth component in the Softmax function output vector o.

[0100] The S3 is specifically as follows:

[0101] The policy gradient theorem optimizes the policy network, establishes a target function of the policy network according to the estimated state value and the probability distribution of all actions, and minimizes the target function to search for an optimal scheduling action, wherein the policy network is a policy function π θ (a t |s t ), θ is a policy network weight parameter, and an action probability distribution is output under the state s at a given time t, which represents the probability of the agent taking a certain action under each state.

[0102] The expression of the target function is specifically as follows:

[0103]

[0104] In the formula, H is an entropy function of the probability distribution, β is a hyperparameter for controlling the influence of entropy regularization, and A(s, a) is an advantage function for measuring that the action a is better than the average value under the state s, which is obtained through (s t ,a t ,r t ,s t+1 ) and the current estimated value experience estimation of the estimated state value V, and π(a|s) is a policy function, is the gradient of the policy network parameter θ.

[0105] In the embodiment, the task scheduling is optimized by minimizing the target function, the action is selected according to the current policy, and the environment is interacted until a terminal state is reached. Secondly, at each time step t, the experience of the agent is collected, and the network weight is corrected by using the stochastic gradient descent to continuously improve the policy. Through the related experiment, the optimal scheduling policy can be effectively found, and the efficient rescheduling and resource reallocation of the task are realized.

[0106] The application has the beneficial effects that the application provides a task rescheduling method for edge service based on HG-Trans, uses a task rescheduling strategy for edge fault scenarios based on a heterogeneous graph neural network to comprehensively improve the task scheduling performance of an edge computing system in a fault scenario, solves the problem that existing scheduling methods rely on local information or simple rules, lack an efficient fault recovery mechanism, and cannot quickly adjust the task execution path after a fault occurs, reduces scheduling delay while achieving global optimal allocation of resources, and has the following effects compared with the prior art:

[0107] (1) The task rescheduling problem is modeled as a Markov decision process, and combined with a reinforcement learning strategy, the node faults in the edge computing scenario can be dynamically adapted, and the scheduling strategy can be adjusted in real time.

[0108] (2) A fault tolerance mechanism is introduced in state modeling, the fault nodes are automatically identified through the state representation of the heterogeneous graph structure, and the interrupted tasks are adaptively adjusted, so that the execution success rate of task rescheduling is improved.

[0109] (3) The heterogeneous graph embedding method based on the Transformer can efficiently capture the complex relationship between tasks and computing resources, optimize the scheduling strategy, reduce the scheduling delay, and achieve global optimal allocation of resources.

[0110] In the description of the application, it should be understood that the orientations or positional relationships indicated by the terms "center", "thickness", "upper", "lower", "horizontal", "top", "bottom", "inner", "outer", "radial" and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features. Therefore, the features limited by "first", "second", "third" can explicitly or implicitly include one or more of the features.

Claims

1. A method for rescheduling of edge service tasks based on HG-Trans, characterized in that, Comprise the following steps: S1, modeling the dynamic task scheduling problem as a Markov decision process, converting the scheduling state into a heterogeneous graph structure; In S1, the method of modeling the dynamic task scheduling problem as a Markov decision process is specifically: (1) Set state: the comprehensive reflection of all task arrangements and computing resource states at any time constitutes the state at the corresponding time; (2) Set action: define the action at any time as To, For task node, S For server node; (3) Set the transfer function: The transfer function represents the action. a From the current state s Transition to the next state The probability, the transition function The specific expression is: wherein is t the state at time is t the state at time is t the action at time (4) Set the reward: for any moment, the reward function The expression of which is specifically: wherein, is the reward function, Makespan is the maximum time span for task completion, is the HEFT maximum time for completion, HEFT is the Heterogeneous Earliest Time (HET) algorithm, which is the most commonly used heuristic algorithm in task scheduling. In the S1, the heterogeneous graph structure wherein, is a set of arcs, each task node corresponding to a server node; S2, input the heterogeneous graph structure into the HG-Trans network to obtain the estimated state value and the probability distribution of all actions; S3, input the estimated state value and the probability distribution of all actions into the decision network to output the task scheduling strategy; The S3 is specifically: The strategy gradient theorem optimizes the probability distribution input into the decision network, establishes the objective function of the policy network according to the estimated state value and the probability distribution of all actions, minimizes the objective function to search for the optimal scheduling action; Wherein, the expression of the objective function is specifically: where is an entropy function of the probability distribution, is a hyperparameter that controls the influence of entropy regularization, is a measure of the action a at state s is a reward function that is greater than average at state and estimates state values V is an empirical estimate of the current estimate of the value is a policy function, is the gradient of the policy network parameters .

2. The method of claim 1, wherein the HG-Trans based edge service task rescheduling method is characterized by, The S2 comprises the following steps: S21, input the heterogeneous graph structure into the multi-layer TransformerConv to update the features of the nodes and generate node embeddings; S22, input the node embedding into the multi-layer perceptron to generate the estimated state value and the probability distribution of the action.

3. The method of claim 2, wherein the HG-Trans based edge service task rescheduling method is characterized by, In S21, the method of updating the features of each node by the multi-layer TransformerConv comprises the following steps: S211, input the heterogeneous graph structure into the multi-layer TransformerConv, weight and aggregate the nodes and their neighbor nodes based on the self-attention mechanism of the Transformer, calculate the node i with its first-order neighborhood in the node j between the attention coefficients A ij ; wherein, Q i is a query vector, K j is a key vector, d k is a dimension of the key vector, T is a matrix transpose, ; S212, normalizing the attention coefficients within the neighborhood with a softmax function, calculating the node i and its first order neighborhood of nodes j between them ; S213, aggregate the information of the neighbor nodes by attention weight, update the feature of the node, the updated feature of the node The expression is specifically: where σ is a non-linear activation function, is a feature representation of a node j .

4. The method of claim 2, wherein the HG-Trans based edge service task rescheduling method is characterized by, In the S22, the estimated state value V is generated. The expression of the estimated state value is specifically where N is the total number of nodes, W v is a projection matrix, , R is a matrix.

5. The method of claim 4, wherein the HG-Trans-based edge service task rescheduling method is characterized by, In S22, the method of generating the probability distribution of the action comprises the following steps: S221, aggregating the node embeddings of the available tasks to generate a batch matrix ; In the formula, is the node embedding for the M th task. S222, map the batch matrix into a one-dimensional vector space, generate the score of each task, and normalize the score through the Softmax function of the full connection layer to obtain the probability distribution of the action, wherein the expression of the probability of the i-th action is specifically: l ​​ In the formula, o l is the Softmax function output vector o th component of l th component of

Images