Method and system for unloading tasks with dependency relationship in mobile edge computing

Abstract

The invention provides a method and system for unloading tasks with a dependency relationship in mobile edge computing, and relates to the technical field of edge computing, task data of a user equipment layer is obtained, a task graph is generated based on the task data, and task embedding vectors are extracted; modeling a task unloading problem as a Markov decision process, and outputting a scheduling strategy of subtasks based on the constructed sequence-to-sequence neural network; unloading and executing each sub-task in the task graph according to the scheduling strategy, and evaluating the performance of the scheduling strategy to obtain an execution result of each sub-task and a corresponding reward function value; training the sequence-to-sequence neural network by using a soft actor commentator algorithm to obtain a trained sequence-to-sequence neural network; and inputting the embedded vector of the task to be unloaded into the trained sequence-to-sequence neural network to obtain an unloading scheduling strategy result of each subtask and a corresponding performance evaluation result, thereby improving the robustness and performance of the unloading strategy in an uncertain environment.

Description

Technical Field

[0001] This invention belongs to the field of edge computing technology, and in particular relates to a method and system for unloading dependent tasks in mobile edge computing. Background Technology

[0002] With the continuous development of mobile internet, the Internet of Things (IoT), and 5G mobile communication technology, computationally intensive applications and applications with high real-time requirements have been widely used in modern society, such as augmented reality / virtual reality and facial recognition. These applications have been widely deployed on mobile devices such as smartphones and tablets. Although the processing power of mobile devices has improved significantly in recent years, running these computationally intensive applications independently still faces considerable difficulties due to hardware limitations. To better handle these internet applications, the edge computing paradigm has emerged. Unlike the deployment method of mobile cloud computing, edge computing deploys computing and storage resources closer to user devices, thereby reducing the number of data forwardings during task transmission.

[0003] Task offloading is a key technology in edge computing, referring to the technique of migrating computing tasks to edge computing servers for auxiliary execution. Users can choose to execute some or all of their computing tasks on the edge server; this is known as the offloading process. However, the latency and energy consumption of the offloading process are affected by many factors, such as the underlying communication technology in the edge computing infrastructure, propagation channel conditions, mobility, and the number of users. Although edge computing architectures tend to provide low latency and low energy consumption, improper configuration of these factors can lead to poor performance. Therefore, designing a highly efficient offloading algorithm in edge computing networks to achieve optimal decision-making for task offloading and reasonable allocation of system resources has become a critical issue in the current edge computing field.

[0004] Existing research on task offloading faces two main challenges. First, most studies treat tasks as a whole, neglecting the dependencies between them. However, in real-world applications, a task often consists of multiple subtasks with complex dependencies. Ignoring these dependencies not only fails to accurately depict the task's execution flow but can also lead to inappropriate offloading strategies. Therefore, accurately modeling and rationally scheduling the dependencies between tasks is crucial when designing offloading strategies. Second, regarding offloading strategies, most existing research uses heuristic or game-theoretic algorithms. While heuristic algorithms demonstrate good problem-solving capabilities for computational offloading, they still suffer from local optima, convergence dependence on expert knowledge, and poor dynamic adaptability. These issues limit their effectiveness in complex dynamic environments. Game-theoretic resource allocation is suitable for instantaneous decisions in static or dynamic scenarios. However, in the dynamically changing scenarios of mobile IoT, calculating the current optimal solution at every moment would impose a significant computational burden on the system and might not necessarily yield a long-term optimal strategy. Summary of the Invention

[0005] To address the shortcomings of existing task offloading methods—namely, most studies treat tasks as a whole while ignoring the dependencies between subtasks, leading to unreasonable offloading strategies, and heuristic or game-theoretic methods are prone to getting trapped in local optima in dynamic environments and incurring high computational costs—this invention proposes a task offloading method with dependencies in mobile edge computing.

[0006] On the one hand, the present invention provides a method for offloading dependent tasks in mobile edge computing, including a training phase and an inference execution phase;

[0007] The training phase includes the following steps:

[0008] Obtain task data from the user device layer, generate a task graph based on the task data, and extract task embedding vectors based on the task graph;

[0009] The task unloading problem is modeled as a Markov decision process, defining the state space, action space, and reward function.

[0010] Construct a sequence-to-sequence neural network, taking task embedding vectors and state space as input, and outputting actions based on the sequence-to-sequence neural network, i.e., the scheduling strategy of sub-tasks;

[0011] The task graph is unloaded and executed according to the scheduling strategy. The performance of the scheduling strategy is evaluated, and the execution results of each subtask and the corresponding reward function value are obtained.

[0012] The sequence-to-sequence neural network is trained using the soft actor critic algorithm to obtain a trained sequence-to-sequence neural network;

[0013] The inference execution phase includes:

[0014] The embedding vectors of the tasks to be unloaded are input into the trained sequence neural network to obtain the unloading scheduling strategy results and corresponding performance evaluation results for each subtask.

[0015] Furthermore, the task data includes the number of mobile terminal devices in the user equipment layer, the number of subtasks executed by each mobile terminal device, subtask attribute information, and the predecessor-successor relationship between each subtask.

[0016] The predecessor-successor relationship includes: several subtasks executed before the current subtask are the predecessor subtasks of the current subtask, and several subtasks executed after the current subtask are the successor subtasks of the current subtask;

[0017] The specific method for generating task graphs based on task data is as follows:

[0018] Calculate the priority of each subtask in each mobile terminal device, sort all subtasks in descending order of priority, generate a scheduling priority list, and generate a task graph based on the scheduling priority list. The task graph is a directed acyclic graph, including a vertex set and a directed edge set. Vertices in the vertex set represent subtasks, and directed edges in the directed edge set represent dependency constraints between two subtasks. Specifically:

[0019] vertex and vertex Directed edges between Subtasks Must be in subtask Complete before starting, i.e., subtask For subtasks The successor subtask;

[0020] The starting point of the task graph is the entry subtask, and the ending point is the exit subtask. The entry subtask is a subtask in the task graph that has no predecessor task, and the exit subtask is a subtask in the task graph that has no successor node.

[0021] In the scheduling priority list, the subtask with the highest scheduling priority is selected as the entry subtask, and the subtask with the lowest scheduling priority is selected as the exit subtask.

[0022] Furthermore, the specific method for extracting task embedding vectors based on the task graph is as follows:

[0023] The subtask attribute information is encoded into a configuration information vector. The subtask attribute information includes the subtask index, the execution time of the subtask on the mobile terminal device, the time of the subtask being uploaded to the edge layer, the execution time of the subtask on the edge layer, and the time of data download to the mobile terminal device after the subtask is executed on the edge layer.

[0024] Based on the task graph, the predecessor subtasks of the subtasks are extracted, and the indexes of the predecessor subtasks are encoded to obtain the predecessor subtask index vector.

[0025] The successor subtasks of the subtasks are extracted based on the task graph, and the indices of the successor subtasks are encoded to obtain the successor subtask index vector.

[0026] Furthermore, the specific methods for setting the state space, action space, and reward function are as follows:

[0027] The state space is defined as a combination of a task graph and a partial offload plan, where the partial offload plan is the scheduling result of the predecessor subtask of the current subtask when scheduling the current subtask.

[0028] The action space is set as the unloading decision value for the subtask, which is used to determine whether the subtask is executed on the mobile terminal device or unloaded to the edge layer for execution.

[0029] A reward function is constructed based on the latency increment and energy consumption increment during subtask execution.

[0030] Furthermore, a sequence-to-sequence neural network is constructed, including a policy neural subnetwork and a Q-value neural subnetwork;

[0031] The policy neural subnetwork is used to generate scheduling policies for subtasks; the Q-value neural subnetwork is used to generate corresponding Q-values ​​based on the scheduling policies.

[0032] The sequence-to-sequence neural network includes an encoder, a decoder, a policy neural subnetwork output layer, and a Q-value neural subnetwork output layer. The policy neural subnetwork and the Q-value neural subnetwork are mainly composed of the encoder and decoder, and the policy neural subnetwork output layer and the Q-value neural subnetwork output layer are respectively connected at the decoder end. The policy neural subnetwork output layer includes a fully connected layer and a softmax layer, and the Q-value neural subnetwork output layer includes a multilayer perceptron.

[0033] The specific method for outputting actions based on a sequence-to-sequence neural network, using task embedding vectors and state space as input, is as follows:

[0034] The task embedding vector is input into the encoder, which then generates a context vector based on the task embedding vector.

[0035] The context vector is input into the decoder, and the hidden state of the subtask is generated through several decoding steps;

[0036] The hidden states of the subtasks output from several decoding steps are input into the fully connected layer, and the softmax function is used to transform the scheduling strategy for generating subtasks.

[0037] The hidden state of the subtask and the scheduling policy of the subtask are input into the multilayer perceptron to generate the Q value corresponding to the scheduling policy of the subtask.

[0038] Furthermore, the specific method for evaluating the performance of the scheduling strategy and obtaining the execution results of each subtask and the corresponding reward function value is as follows:

[0039] According to the scheduling strategy of the subtasks, each subtask in the task graph is scheduled in sequence. The subtasks are executed collaboratively by the mobile terminal device and the edge layer. When the unloading decision value of the subtask is 1, the subtask is unloaded to the edge layer for processing. When the unloading decision value is 0, the subtask is executed by the mobile terminal device.

[0040] Obtain subtask performance data as an indicator to evaluate the performance of the scheduling strategy, including subtask execution latency and subtask execution energy consumption;

[0041] Subtask execution latency includes the data transmission time and execution time of the subtask;

[0042] Subtask execution energy consumption includes the energy consumption generated by mobile terminal devices at the user device layer and the edge layer during subtask execution;

[0043] Once all subtasks are completed, the edge layer sends the corresponding results back to the mobile terminal device in the user device layer, integrates the results of each subtask, and obtains the execution results of each subtask and the corresponding reward function value.

[0044] Furthermore, the specific method for training the sequence-to-sequence neural network using the soft actor critic algorithm is as follows:

[0045] S1 Initializes the neural subnetwork parameters Strategy temperature parameters And the experience replay pool R; set a corresponding target Q-value subnetwork for the Q-value neural subnetwork, the structure of the target Q-value subnetwork is consistent with the Q-value neural subnetwork, and the parameters of the target Q-value subnetwork are synchronized from the Q-value neural subnetwork through soft update; initialize the parameters of the two sets of Q-value neural subnetworks. and And set the target subnetwork parameters to and ;

[0046] S2 samples the unloading decision value of each subtask in the task graph from the scheduling policy based on the current state of the subtask.

[0047] S3 executes the subtask based on the unloading decision value, and obtains the status after the subtask is completed and the reward function value of the subtask;

[0048] S4 constructs an experience quadruple based on the current state of the subtask, the unload decision value of the subtask, the state after the subtask is completed, and the reward function value of the subtask, and stores it in the experience replay pool.

[0049] S5 randomly samples a small batch of samples from the experience replay pool as training data;

[0050] S6 uses the soft actor critic algorithm to minimize the mean square error of the Q-value neural subnetwork using the Bellman objective, updates the parameters of the Q-value neural subnetwork, and updates the parameters of the target Q-value subnetwork according to the soft update strategy;

[0051] S7 uses the soft actor critic algorithm to update the policy neural network parameters for the policy objective. The strategy objective is to maximize the balance between expected return and strategy entropy;

[0052] Repeat steps S2 to S7 until the subtask is completed. Exit the loop and continue processing the next subtask in the scheduling priority list. After making unloading decisions for all subtasks in the scheduling priority list, you will obtain the trained sequence-to-sequence neural network.

[0053] On the other hand, the present invention also provides a task offloading system with dependencies in mobile edge computing, which includes at least a user device layer and an edge layer;

[0054] The user equipment layer includes several user equipments. Each user equipment includes an offload scheduler module and a local processing unit. The offload scheduler module is used to generate a task graph based on the subtasks in the user equipment and upload it to the edge layer. It also makes offload decisions for the subtasks in the user equipment, including local execution and edge layer execution. The local processing unit is used to execute the subtasks whose offload decision is to be executed locally and calculate the latency and energy consumption of the subtask execution.

[0055] The edge layer comprises multiple edge servers, each of which includes a task graph pool, an offloaded trainer module, and a virtual computing resource pool.

[0056] The task graph pool is used to receive and cache task graphs uploaded by user devices, providing support for unified scheduling and resource allocation at the edge.

[0057] The offloading trainer module is used to obtain the task execution data of subtasks in the user device. Based on the task graph and subtask task execution data in the task graph pool, the soft actor critic algorithm is used to train and optimize the task offloading model to obtain the trained task offloading model. The parameters of the trained task offloading model are sent to each user device in the user device layer for offloading strategy generation in the subsequent inference stage.

[0058] The virtual computing resource pool includes several virtual machines, which are used to execute subtasks from different user devices that are offloaded to the edge layer in parallel. After execution, the latency and energy consumption of the subtask are calculated, and the execution results, latency and energy consumption of the subtask are sent to the corresponding user devices.

[0059] Furthermore, the system also includes an optional cloud layer, which comprises a cloud server cluster;

[0060] The cloud layer provides computing power support when the virtual computing resource pool at the edge layer is resource-constrained or when the offloading trainer module needs to optimize the task offloading model for a long time.

[0061] Furthermore, the overall operation flow of the system is as follows:

[0062] The user equipment layer's offload scheduler module generates a task graph based on the subtasks in the user equipment and sends the task graph to the task graph pool in the edge layer.

[0063] The offloading trainer module of the edge layer is used to obtain the task execution data of subtasks in the user device. Based on the task graph and subtask task execution data in the task graph pool, the soft actor critic algorithm is used to train and optimize the task offloading model to obtain the trained task offloading model. The parameters of the optimized task offloading model are sent to the user device in the user device layer.

[0064] The user equipment layer loads the parameters of the optimized task unloading model into the user equipment, and uses the unloading scheduler module to reason about the task graph to generate unloading decisions for each subtask.

[0065] For subtasks whose offloading decision is to be executed at the edge layer, the subtasks are offloaded to the virtual computing resource pool at the edge layer for execution. For subtasks whose offloading decision is to be executed locally, the subtasks are executed based on the local processing unit of the user device, and the latency and energy consumption of the subtask execution are calculated.

[0066] After all subtasks in the virtual computing resource pool have been executed, the execution results of the subtasks are returned to the corresponding user devices. The user devices then upload the latency and energy consumption of all subtasks to the offloading trainer module in the edge layer.

[0067] The edge layer optimizes and updates the offloading decision model by utilizing the latency and energy consumption of subtask execution, and sends the optimization results to each user device in the user device layer.

[0068] The beneficial effects of adopting the above technical solution are as follows: The task offloading method with dependencies in mobile edge computing provided by this invention effectively represents the execution order and constraint relationships between tasks by modeling the dependencies between tasks as a directed acyclic graph (DAG). To effectively adapt to dynamic scenarios, this invention provides a new offloading scheme that integrates deep reinforcement learning training and inference programs into the mobile edge computing system, thereby achieving dynamic and adaptive offloading decisions. Specifically, this invention formalizes the task offloading problem as a Markov decision process and uses a sequence-to-sequence neural network to approximate its policy function and value function. Based on this, a soft actor-critic algorithm is further introduced to efficiently train the neural network, thereby improving the robustness and performance of the offloading strategy in uncertain environments. Attached Figure Description

[0069] Figure 1 A schematic diagram of a directed acyclic graph provided in Embodiment 1 of the present invention;

[0070] Figure 2 A schematic diagram of the sequence-to-sequence neural network structure provided in Embodiment 1 of the present invention;

[0071] Figure 3 A schematic diagram of the task unloading system provided in Embodiment 2 of the present invention. Detailed Implementation

[0072] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0073] Example 1:

[0074] A method for offloading dependent tasks in mobile edge computing, comprising a training phase and an inference execution phase;

[0075] The training phase includes the following steps:

[0076] Step 1: Obtain task data from the user device layer, generate a task graph based on the task data, and extract task embedding vectors based on the task graph;

[0077] Task data includes the number of mobile terminal devices in the user device layer, the number of subtasks executed by each mobile terminal device, subtask attribute information, and the predecessor-successor relationship between each subtask;

[0078] Precedence / afterference relationships include: in subtasks The previously executed subtasks are called subtasks. The preceding sub-task, in the sub-task The subsequent execution of several subtasks constitutes a subtask. The successor subtask;

[0079] Step 1.1: Calculate the priority of each subtask in each mobile terminal device. priority As shown in the formula below:

[0080]

[0081] in, For subtasks The average delay is calculated. For subtasks The set of successor subtasks Select the subtask with the highest priority value. As a subtask The path of influence;

[0082] Influence paths reflect the dependencies between tasks, i.e., subtasks. The priority of a task is affected not only by its own average computation latency, but also by the highest priority subtask among its successors. The impact of this is significant. By selecting the impact path, we can prioritize the scheduling of subtasks on the critical path as early as possible, thereby effectively reducing the total time for generating the task graph.

[0083] Sort all subtasks in descending order of priority to generate a scheduling priority list, and then generate a task graph based on the scheduling priority list. The task graph is a directed acyclic graph, such as... Figure 1 As shown, where, For the set of vertices, Let be a set of directed edges, and let the vertex set contain the first vertex. vertices Indicates the first Sub-tasks ,vertex and vertex Directed edges between Subtasks Kazuko's Mission The dependency constraints between them are specifically: subtasks Must be in subtask Complete before starting, i.e., subtask For subtasks The successor subtask;

[0084] Task Map The starting point is the entry subtask, and the ending point is the exit subtask; the entry subtask refers to a subtask that has no predecessor task in the task graph, and the exit subtask refers to a subtask that has no successor node in the task graph.

[0085] In the scheduling priority list, the subtask with the highest scheduling priority is selected as the entry subtask, and the subtask with the lowest scheduling priority is selected as the exit subtask.

[0086] Step 1.2: Encode each subtask into a task embedding vector, including a configuration information vector, a predecessor subtask index vector, and a successor subtask index vector;

[0087] The subtask attribute information is encoded into a configuration information vector. The subtask attribute information includes the subtask index, the subtask execution time, the time when the subtask is uploaded to the edge layer, the execution time of the subtask on the edge layer, and the time when the data of the subtask is downloaded to the mobile terminal device after execution on the edge layer.

[0088] Based on the task graph, the predecessor subtasks of the subtasks are extracted, and the indexes of the predecessor subtasks are encoded to obtain the predecessor subtask index vector.

[0089] The successor subtasks of the subtasks are extracted based on the task graph, and the indices of the successor subtasks are encoded to obtain the successor subtask index vector.

[0090] Step 2: Model the task unloading problem as a Markov decision process, defining the state space, action space, and reward function;

[0091] Schedule subtasks At that time, the state of the edge layer depends on the subtask. Partial uninstallation plan, i.e., subtasks The scheduling results of the predecessor subtasks. Therefore, the state space is defined. The combination of the task graph and a portion of the uninstallation plan is shown in the following formula:

[0092]

[0093] in, For subtasks Partial uninstallation plan, For subtasks The status reflects the task dependency topology and the unloading status of completed subtasks;

[0094] This embodiment defines the action space based on whether the subtask is unloaded to the user device layer or the edge layer. As shown in the formula below:

[0095]

[0096] in, For the first Sub-tasks The unloading decision value, To subtask Unload to the edge layer. Subtasks Runs at the user equipment layer;

[0097] In existing task offloading strategies, the reward function primarily considers latency and energy consumption as key factors for optimizing service quality. However, this method is computationally intensive and fails to capture real-time changes in latency and energy consumption. To address this issue, this embodiment introduces latency increments. and energy consumption increment The formula accurately represents the changes in actual latency and energy consumption during the execution phase of each subtask, as shown below:

[0098]

[0099] in, For the first Sub-tasks;

[0100] Delay increment This represents the change in delay and energy consumption increment between two adjacent subtasks during the unloading process. This represents the change in energy consumption between two adjacent subtasks during the unloading process, and the delay increment. and energy consumption increment It plays a crucial role in quickly identifying changes in system performance and efficiency, and serves as a core indicator in the optimization and evaluation process.

[0101] The reward function penalizes the incremental increase in latency and energy consumption at each step. By normalizing these values ​​with the overall local latency and energy consumption, a balance is ensured between penalizing inefficiency and evaluating system performance. The reward function is constructed as shown in the following formula:

[0102]

[0103] in, and These are the weighting coefficients. This represents the total local latency within the current time period, serving as the baseline value for latency normalization. This represents the total local energy consumption within the current time period, serving as the baseline value for energy consumption normalization. The increase in latency is normalized relative to the overall local latency. Normalizing the increase in energy consumption relative to the total local energy consumption reflects the system's efficiency and responsiveness, respectively.

[0104] Step 3: Construct a sequence-to-sequence neural network, including a policy neural subnetwork and a Q-value neural subnetwork; input the task embedding vector into the sequence neural network, and use the policy neural subnetwork and the Q-value neural subnetwork to generate the task offloading scheduling policy and the Q-value corresponding to the scheduling policy, respectively;

[0105] Sequence-to-sequence neural networks are used to output the probability distribution of unloading actions based on the task embedding vector, and also to evaluate the value of each unloading action, providing a basis for subsequent reinforcement learning; the structure of a sequence-to-sequence neural network is as follows: Figure 2 As shown, it includes an encoder, a decoder, a policy neural network output layer, and a Q-value neural network output layer. The policy neural network and the Q-value neural network are based on the encoder and decoder, respectively, and the policy neural network output layer and the Q-value neural network output layer are connected at the decoder end. The policy neural network output layer includes a fully connected layer and a softmax layer, and the Q-value neural network output layer includes a multilayer perceptron.

[0106] The encoder is used to extract features from the task embedding vector to obtain the context vector. The encoder captures both forward and backward dependency information between tasks based on a bidirectional gated recurrent unit structure. The decoder is used to generate an unloading decision sequence based on the context vector output by the encoder. The decoder adopts a gated recurrent unit structure with an attention mechanism to enhance the ability to model the dependencies of key tasks.

[0107] The policy neural subnetwork and the Q-value neural subnetwork share the encoder and decoder structures to maintain consistency in the representation of state features.

[0108] When the decoder generates a scheduling policy for task unloading, it uses the decoder's hidden state vector as the state representation of the task. The probability distribution of unloading actions is obtained through a fully connected layer and a Softmax function, thereby generating a scheduling policy to guide the dynamic unloading scheduling of tasks.

[0109] The hidden state of the decoder is concatenated with the unloading action in the scheduling policy and then input into the multilayer perceptron to obtain the Q value corresponding to the scheduling policy. This Q value is used to evaluate the expected return of the current scheduling policy and provides a valuable basis for optimization in the reinforcement learning stage.

[0110] In summary, sequence-to-sequence neural networks serve as the overall structural framework, with policy neural networks and Q-value neural networks as their functional subnetworks, jointly realizing the generation of strategies and value evaluation for task offloading.

[0111] Step 3.1: Input the task embedding vector into the encoder, and use the encoder to generate a context vector based on the task embedding vector;

[0112] Embedding the task vector Input the encoder to obtain the encoded hidden state sequence. For the first The task embedding vectors of each subtask are input into the decoder along with the encoded hidden state sequence. Based on the attention mechanism, the sequence is processed in the 1st... Each encoding step generates a context vector for each subtask, as shown in the following formula:

[0113]

[0114]

[0115] in, For encoder functions, For the first The output of the encoding step is the first... The hidden state of each sub-task For the first Task embedding vectors of each subtask For encoder parameters, For the first The output of the decoding step is the first... The context vector of each subtask For the first The decoding step is for the first... Attention weights for each hidden state are used to measure the task embedding vector. In the Each decoding step generates a context vector. The contribution size in the code; the context vector represents the weighted summary of the encoder's embedding information for all tasks, serving as the decoder's first... Each decoding step generates important inputs for the unloading strategy;

[0116] Step 3.2: Input the context vector into the decoder, and after the first... Each decoding step generates the hidden state of the subtask;

[0117] Decoder network in the The output of the decoding step is the first... Hidden state of individual tasks As shown in the formula below:

[0118]

[0119] in, For decoder functions, For the first The output of the decoding step is the first... The hidden state of each sub-task For the first The first step generated by the decoding process The unloading decision value for each subtask. These are the decoder network parameters;

[0120] For the The first sub-task The context vector generated by each encoding step Hidden state from the previous encoding step Unloading decision value of the previous coding step Together they determine the current hidden state of the decoder This affects the unloading decision value generated in the current decoding step. Therefore, the context vector is one of the key inputs for adjusting the scheduling strategy, ensuring that the decoder can make dynamic decisions based on the dependencies of the task sequence.

[0121] Step 3.3: Place the first The output of the decoding step is the first... Hidden state of individual tasks Input a fully connected layer and use the softmax function to transform it to generate the th... Scheduling strategy for individual tasks ;

[0122] The first The output of the decoding step is the first... Hidden state of individual tasks and the Scheduling strategy for individual tasks Input the multilayer perceptron and generate the first... Scheduling strategy for individual tasks The corresponding Q value ;

[0123] Step 4: Unload and execute each subtask in the task graph according to the scheduling strategy, evaluate the task unloading performance, and obtain the execution results of each subtask and the performance data of the task graph.

[0124] After generating the scheduling strategy for subtasks, the mobile terminal device and the edge layer collaboratively execute the subtasks to complete the actual unloading and computation operations of each subtask in the application task graph. The specific method is as follows:

[0125] According to the scheduling strategy of the subtask, each subtask in the task graph is scheduled in sequence. When the unloading decision value of the subtask is 1, the subtask is unloaded to the edge layer for processing. When the unloading decision value is 0, the subtask is executed by the mobile terminal device.

[0126] To accurately evaluate task unloading performance, the performance data of each subtask is recorded during task unloading and execution, serving as an indicator for evaluating the performance of the scheduling strategy, including subtask execution latency and subtask execution energy consumption.

[0127] Subtask execution latency includes the data transmission time and execution time of the subtask;

[0128] Subtask execution energy consumption includes the energy consumption generated by the mobile terminal device at the user device layer and the edge layer during subtask execution. As shown in the formula below:

[0129]

[0130] in, This refers to the transmission power consumption generated when mobile terminal devices in the user equipment layer offload subtask data to the edge layer. Computational energy consumption generated during the execution of subtasks for mobile terminal devices in the user equipment layer or the edge layer;

[0131] Transmission power consumption Based on the subtask data size and wireless communication bandwidth estimation, calculate the energy consumption. It was calculated based on the equipment power model and actual operating time.

[0132] Once all subtasks are completed, the edge layer sends the corresponding results back to the mobile terminal device in the user device layer, integrates the results of each subtask, and obtains the execution results of each subtask and the corresponding reward function value.

[0133] By implementing this step, the system not only achieves effective execution of the scheduling strategy, but also obtains key performance data during the task graph execution process, providing a data foundation for subsequent training.

[0134] Step 5: Train the sequence-to-sequence neural network using the soft actor critic algorithm to obtain the trained sequence-to-sequence neural network;

[0135] The soft actor critic algorithm is used to train the policy neural network and Q-value neural network, changing the action space of the soft actor critic algorithm from continuous to discrete to adapt to the unloading action of the task. By maximizing the balance between system reward and entropy, the model can improve the accuracy of unloading decisions while maintaining a high degree of exploratoryness.

[0136] Step 5.1: Initialize the parameters of the policy neural subnetwork Strategy temperature parameters and experience replay pool R;

[0137] To ensure the stability of Q-value estimation, a corresponding target Q-value subnetwork is set up for the Q-value neural subnetwork. The structure of the target Q-value subnetwork is consistent with that of the Q-value neural subnetwork, and the parameters of the target Q-value subnetwork are synchronized from those of the Q-value neural subnetwork through soft updates. The parameters of both sets of Q-value neural subnetworks are initialized. and And set the target Q-value subnetwork parameters to and ;

[0138] Step 5.2: For each subtask in the task graph, perform the following process:

[0139] (1) Based on subtasks Current state From the scheduling strategy Mid-sampling action That is, the unloading decision value;

[0140] (2) Based on the unloading decision value Execute subtasks Obtain subtask Completed state Kazuko's Mission reward function value ;

[0141] (3) Constructing experience quadruples And store it in the experience replay pool R;

[0142] (4) Randomly sample a small batch of samples from the experience replay pool R as training data;

[0143] (5) Use the Bellman objective of the soft actor critic algorithm to minimize the mean square error of the Q-value neural subnetwork and update the parameters of the Q-value neural subnetwork. and Update the target Q-value subnetwork parameters according to the soft update strategy. ;

[0144] (6) Update the policy neural network parameters using the policy objective of the soft actor critic algorithm. The strategy objective is to maximize the balance between expected return and strategy entropy;

[0145] (7) Repeat (1) to (6) until the subtask is completed, exit the loop and continue to process the next subtask in the scheduling priority list. After completing the unloading decision for all subtasks in the scheduling priority list, the trained sequence-to-sequence neural network is obtained.

[0146] The inference execution phase includes:

[0147] The embedding vectors of the tasks to be unloaded are input into a trained sequence neural network to obtain the unloading scheduling strategy results and corresponding performance evaluation results for each subtask, namely the estimated values ​​of execution latency and energy consumption of each subtask, thereby realizing localized unloading strategy reasoning.

[0148] Compared with existing technologies, in evaluations with transmission rates ranging from 3 to 15 Mbps, the task offloading method proposed in this embodiment exhibits good adaptability, dynamically adjusting the offloading strategy according to network conditions, and simultaneously achieving low latency and low energy consumption. In multi-objective optimization scenarios (such as latency-energy tradeoffs), the task offloading method proposed in this embodiment can flexibly adjust strategy preferences, obtaining a better latency-energy tradeoff solution than DDQNTO, HEFT, etc., demonstrating strong strategy generalization ability.

[0149] Example 2:

[0150] A task offloading system with dependencies in mobile edge computing, such as Figure 3 As shown, it includes at least a user equipment layer and an edge layer;

[0151] The user equipment layer includes several user equipments. Each user equipment includes an offload scheduler module and a local processing unit. The offload scheduler module is used to generate a task graph based on the subtasks in the user equipment and upload it to the edge layer. It also makes offload decisions for the subtasks in the user equipment, including local execution and edge layer execution. The local processing unit is used to execute the subtasks whose offload decision is to be executed locally and calculate the latency and energy consumption of the subtask execution.

[0152] In this embodiment, the user device can be a smartphone, tablet, wearable device, or in-vehicle terminal, etc. Each user device runs multiple high-performance applications, which include computationally intensive, low-latency-requirement, or complex task dependencies subtasks. To improve application execution efficiency, each user terminal device includes an offload scheduler module. The offload scheduler module dynamically makes offload decisions based on the current application state, the mobile terminal device's resource status, network status, and subtask information; that is, it determines which subtasks should be executed locally on the mobile terminal device and which should be offloaded to the edge layer for execution. The offload scheduler module can integrate a deep reinforcement learning inference engine to achieve autonomous and efficient decision support for complex task graphs.

[0153] The edge layer comprises multiple edge servers, each of which includes a task graph pool, an offloaded trainer module, and a virtual computing resource pool.

[0154] The task graph pool is used to receive and cache task graphs uploaded by user devices, providing support for unified scheduling and resource allocation at the edge.

[0155] The offloading trainer module is used to obtain the task execution data of subtasks in the user device. Based on the task graph and subtask task execution data in the task graph pool, the soft actor critic algorithm is used to train and optimize the task offloading model to obtain the trained task offloading model. The parameters of the trained task offloading model are sent to each user device in the user device layer for offloading strategy generation in the subsequent inference stage.

[0156] The virtual computing resource pool includes several virtual machines, which are used to execute subtasks from different user devices that are offloaded to the edge layer in parallel. After execution, the latency and energy consumption of the subtask are calculated, and the execution results, latency and energy consumption of the subtask are sent to the corresponding user devices.

[0157] In this embodiment, the virtual computing resource pool has high availability and low latency computing capabilities. The edge layer, through the collaboration of the task graph pool, the offloading trainer module and the virtual computing resource pool, realizes a unified closed loop of task execution, model optimization and data management. It can continuously update the task offloading model based on the task execution data fed back by the user device, so that the task offloading model maintains adaptability and efficiency.

[0158] In mobile edge computing, the task offloading system with dependencies provided in this embodiment also includes an optional cloud layer, which includes a cloud server cluster;

[0159] The cloud layer provides computing power support when the virtual computing resource pool at the edge layer is resource-constrained or when the offloading trainer module needs to optimize the task offloading model for a long time.

[0160] In this embodiment, the cloud layer does not participate in real-time task scheduling, but only interacts with the edge layer for data or model parameters when necessary, for updating the global model and maintaining consistency. When edge layer computing resources are limited or cannot meet the needs of specific computing tasks, some computing tasks in the edge layer are relayed to cloud layer processes for processing, thereby realizing a multi-layered collaborative task offloading architecture.

[0161] The overall operation flow of the task offloading system with dependencies in mobile edge computing provided in this embodiment is as follows:

[0162] The user equipment layer's offload scheduler module generates a task graph based on the subtasks in the user equipment and sends the task graph to the task graph pool in the edge layer.

[0163] The offloading trainer module of the edge layer is used to obtain the task execution data of subtasks in the user device. Based on the task graph and subtask task execution data in the task graph pool, the soft actor critic algorithm is used to train and optimize the task offloading model to obtain the trained task offloading model. The parameters of the optimized task offloading model are sent to the user device in the user device layer.

[0164] The user equipment layer loads the parameters of the optimized task unloading model into the user equipment, and uses the unloading scheduler module to reason about the task graph to generate unloading decisions for each subtask.

[0165] For subtasks whose offloading decision is to be executed at the edge layer, the subtasks are offloaded to the virtual computing resource pool at the edge layer for execution. For subtasks whose offloading decision is to be executed locally, the subtasks are executed based on the local processing unit of the user device, and the latency and energy consumption of the subtask execution are calculated.

[0166] After all subtasks in the virtual computing resource pool have been executed, the execution results of the subtasks are returned to the corresponding user devices. The user devices then upload the latency and energy consumption of all subtasks to the offloading trainer module in the edge layer.

[0167] The edge layer utilizes the latency and energy consumption of subtask execution to optimize and update the offloading decision model, and sends the optimization results to each user device in the user device layer, realizing closed-loop collaboration between training and inference.

[0168] Example 3:

[0169] This embodiment proposes an electronic device, including: one or more processors, and a memory for storing instructions, which, when executed by the one or more processors, cause the one or more processors to perform the task offloading method with dependencies in mobile edge computing.

[0170] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the task offloading method with dependencies in mobile edge computing as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0171] The processor is used to execute all or part of the steps in the task offloading method with dependencies in mobile edge computing as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0172] The processor may be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the task offloading method with dependencies in mobile edge computing described in the above embodiments.

[0173] Example 4:

[0174] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0175] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the task offloading method with dependencies in mobile edge computing as described in the various embodiments of this application.

[0176] The aforementioned storage media include: flash memory, hard disks, multimedia cards, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disks, optical discs, servers, APP (Application) app stores, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the task offloading method with dependencies in mobile edge computing described above.

[0177] Example 5:

[0178] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the task offloading method with dependencies in mobile edge computing.

[0179] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0180] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0181] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this disclosure and its equivalents, then the intent of this disclosure also includes these modifications and variations.

Claims

1. A method for unloading dependent tasks in mobile edge computing, characterized in that, Includes the training phase and the inference execution phase; The training phase includes the following steps: Obtain task data from the user device layer, generate a task graph based on the task data, and extract task embedding vectors based on the task graph; The task unloading problem is modeled as a Markov decision process, defining the state space, action space, and reward function. Construct a sequence-to-sequence neural network, taking task embedding vectors and state space as input, and outputting actions based on the sequence-to-sequence neural network, i.e., the scheduling strategy of sub-tasks; The task graph is unloaded and executed according to the scheduling strategy. The performance of the scheduling strategy is evaluated, and the execution results of each subtask and the corresponding reward function value are obtained. The sequence-to-sequence neural network is trained using the soft actor critic algorithm to obtain a trained sequence-to-sequence neural network; The inference execution phase includes: The embedding vectors of the tasks to be unloaded are input into the trained sequence neural network to obtain the unloading scheduling strategy results and corresponding performance evaluation results for each subtask.

2. The method for offloading dependent tasks in mobile edge computing according to claim 1, characterized in that, The task data includes the number of mobile terminal devices in the user device layer, the number of subtasks executed by each mobile terminal device, subtask attribute information, and the predecessor-successor relationship between each subtask. The predecessor-successor relationship includes: several subtasks executed before the current subtask are the predecessor subtasks of the current subtask, and several subtasks executed after the current subtask are the successor subtasks of the current subtask; The specific method for generating task graphs based on task data is as follows: Calculate the priority of each subtask in each mobile terminal device, sort all subtasks in descending order of priority, generate a scheduling priority list, and generate a task graph based on the scheduling priority list. The task graph is a directed acyclic graph, including a vertex set and a directed edge set. Vertices in the vertex set represent subtasks, and directed edges in the directed edge set represent dependency constraints between two subtasks. Specifically: vertex and vertex Directed edges between Subtasks Must be in subtask Complete before starting, i.e., subtask For subtasks The successor subtask; The starting point of the task graph is the entry subtask, and the ending point is the exit subtask. The entry subtask is a subtask in the task graph that has no predecessor task, and the exit subtask is a subtask in the task graph that has no successor node. In the scheduling priority list, the subtask with the highest scheduling priority is selected as the entry subtask, and the subtask with the lowest scheduling priority is selected as the exit subtask.

3. A method for unloading dependent tasks in mobile edge computing according to claim 2, characterized in that, The specific method for extracting task embedding vectors based on task graphs is as follows: The subtask attribute information is encoded into a configuration information vector. The subtask attribute information includes the subtask index, the execution time of the subtask on the mobile terminal device, the time of the subtask being uploaded to the edge layer, the execution time of the subtask on the edge layer, and the time of data download to the mobile terminal device after the subtask is executed on the edge layer. Based on the task graph, the predecessor subtasks of the subtasks are extracted, and the indexes of the predecessor subtasks are encoded to obtain the predecessor subtask index vector. The successor subtasks of the subtasks are extracted based on the task graph, and the indices of the successor subtasks are encoded to obtain the successor subtask index vector.

4. A method for unloading dependent tasks in mobile edge computing according to claim 1, characterized in that, The specific methods for setting the state space, action space, and reward function are as follows: The state space is defined as a combination of a task graph and a partial offload plan, where the partial offload plan is the scheduling result of the predecessor subtask of the current subtask when scheduling the current subtask. The action space is set as the unloading decision value for the subtask, which is used to determine whether the subtask is executed on the mobile terminal device or unloaded to the edge layer for execution. A reward function is constructed based on the latency increment and energy consumption increment during subtask execution.

5. A method for unloading dependent tasks in mobile edge computing according to claim 4, characterized in that, Construct a sequence-to-sequence neural network, including a policy neural subnetwork and a Q-value neural subnetwork; The policy neural subnetwork is used to generate scheduling policies for subtasks; the Q-value neural subnetwork is used to generate corresponding Q-values ​​based on the scheduling policies. Sequence-to-sequence neural networks include an encoder, a decoder, fully connected layers, a softmax function, and a multilayer perceptron. The policy neural subnetwork and the Q-value neural subnetwork share the encoder and decoder. The specific method for outputting actions based on a sequence-to-sequence neural network, using task embedding vectors and state space as input, is as follows: The task embedding vector is input into the encoder, which then generates a context vector based on the task embedding vector. The context vector is input into the decoder, and the hidden state of the subtask is generated through several decoding steps; The hidden states of the subtasks output from several decoding steps are input into the fully connected layer, and the softmax function is used to transform the scheduling strategy for generating subtasks. The hidden state of the subtask and the scheduling policy of the subtask are input into the multilayer perceptron to generate the Q value corresponding to the scheduling policy of the subtask.

6. A method for offloading dependent tasks in mobile edge computing according to claim 5, characterized in that, The specific method for evaluating the performance of the scheduling strategy and obtaining the execution results of each subtask and the corresponding reward function value is as follows: According to the scheduling strategy of the subtasks, each subtask in the task graph is scheduled in sequence. The subtasks are executed collaboratively by the mobile terminal device and the edge layer. When the unloading decision value of the subtask is 1, the subtask is unloaded to the edge layer for processing. When the unloading decision value is 0, the subtask is executed by the mobile terminal device. Obtain subtask performance data as an indicator to evaluate the performance of the scheduling strategy, including subtask execution latency and subtask execution energy consumption; Subtask execution latency includes the data transmission time and execution time of the subtask; Subtask execution energy consumption includes the energy consumption generated by mobile terminal devices at the user device layer and the edge layer during subtask execution; Once all subtasks are completed, the edge layer sends the corresponding results back to the mobile terminal device in the user device layer, integrates the results of each subtask, and obtains the execution results of each subtask and the corresponding reward function value.

7. A method for unloading dependent tasks in mobile edge computing according to claim 6, characterized in that, The specific method for training a sequence-to-sequence neural network using the soft actor critic algorithm is as follows: S1 Initializes the neural subnetwork parameters Strategy temperature parameters And the experience replay pool R; set a corresponding target Q-value subnetwork for the Q-value neural subnetwork, the structure of the target Q-value subnetwork is consistent with the Q-value neural subnetwork, and the parameters of the target Q-value subnetwork are synchronized from the Q-value neural subnetwork through soft update; initialize the parameters of the two sets of Q-value neural subnetworks. and And set the target subnetwork parameters to and ; S2 samples the unloading decision value of each subtask in the task graph from the scheduling policy based on the current state of the subtask. S3 executes the subtask based on the unloading decision value, and obtains the status after the subtask is completed and the reward function value of the subtask; S4 constructs an experience quadruple based on the current state of the subtask, the unload decision value of the subtask, the state after the subtask is completed, and the reward function value of the subtask, and stores it in the experience replay pool. S5 randomly samples a small batch of samples from the experience replay pool as training data; S6 uses the soft actor critic algorithm to minimize the mean square error of the Q-value neural subnetwork using the Bellman objective, updates the parameters of the Q-value neural subnetwork, and updates the parameters of the target Q-value subnetwork according to the soft update strategy; S7 uses the soft actor critic algorithm to update the policy neural network parameters for the policy objective. The strategy objective is to maximize the balance between expected return and strategy entropy; Repeat steps S2 to S7 until the subtask is completed. Exit the loop and continue processing the next subtask in the scheduling priority list. After making unloading decisions for all subtasks in the scheduling priority list, you will obtain the trained sequence-to-sequence neural network.

8. A task offloading system with dependencies in mobile edge computing, performing task offloading based on the method of claim 1, characterized in that, It includes at least the user equipment layer and the edge layer; The user equipment layer includes several user equipments. Each user equipment includes an offload scheduler module and a local processing unit. The offload scheduler module is used to generate a task graph based on the subtasks in the user equipment and upload it to the edge layer. It also makes offload decisions for the subtasks in the user equipment, including local execution and edge layer execution. The local processing unit is used to execute the subtasks whose offload decision is to be executed locally and calculate the latency and energy consumption of the subtask execution. The edge layer comprises multiple edge servers, each of which includes a task graph pool, an offloaded trainer module, and a virtual computing resource pool. The task graph pool is used to receive and cache task graphs uploaded by user devices, providing support for unified scheduling and resource allocation at the edge. The offloading trainer module is used to obtain the task execution data of subtasks in the user device. Based on the task graph and subtask task execution data in the task graph pool, the soft actor critic algorithm is used to train and optimize the task offloading model to obtain the trained task offloading model. The parameters of the trained task offloading model are sent to each user device in the user device layer for offloading strategy generation in the subsequent inference stage. The virtual computing resource pool includes several virtual machines, which are used to execute subtasks from different user devices that are offloaded to the edge layer in parallel. After execution, the latency and energy consumption of the subtask are calculated, and the execution results, latency and energy consumption of the subtask are sent to the corresponding user devices.

9. A task offloading system with dependencies in mobile edge computing according to claim 8, characterized in that, The system also includes an optional cloud layer, which comprises a cloud server cluster; The cloud layer provides computing power support when the virtual computing resource pool at the edge layer is resource-constrained or when the offloading trainer module needs to optimize the task offloading model for a long time.

10. A task offloading system with dependencies in mobile edge computing according to claim 9, characterized in that, The overall operation flow of the system is as follows: The user equipment layer's offload scheduler module generates a task graph based on the subtasks in the user equipment and sends the task graph to the task graph pool in the edge layer. The offloading trainer module of the edge layer is used to obtain the task execution data of subtasks in the user device. Based on the task graph and subtask task execution data in the task graph pool, the soft actor critic algorithm is used to train and optimize the task offloading model to obtain the trained task offloading model. The parameters of the optimized task offloading model are sent to the user device in the user device layer. The user equipment layer loads the parameters of the optimized task unloading model into the user equipment, and uses the unloading scheduler module to reason about the task graph to generate unloading decisions for each subtask. For subtasks whose offloading decision is to be executed at the edge layer, the subtasks are offloaded to the virtual computing resource pool at the edge layer for execution. For subtasks whose offloading decision is to be executed locally, the subtasks are executed based on the local processing unit of the user device, and the latency and energy consumption of the subtask execution are calculated. After all subtasks in the virtual computing resource pool have been executed, the execution results of the subtasks are returned to the corresponding user devices. The user devices then upload the latency and energy consumption of all subtasks to the offloading trainer module in the edge layer. The edge layer optimizes and updates the offloading decision model by utilizing the latency and energy consumption of subtask execution, and sends the optimization results to each user device in the user device layer.

Images