Multi-user multi-outlet DNN reasoning partition and task scheduling method

Abstract

A multi-user multi-exit DNN reasoning partitioning and task scheduling method includes the steps that trained multi-exit DNN models are deployed on each user device and an edge server, and in a time slot, tasks reach the user devices, meet Poisson distribution and are stored in task queues of the user devices; taking the strategy feasibility interval as a constraint, taking an exit point, a partition point and a scheduling decision of the task as optimization variables, and establishing an optimization problem P taking maximization of the total completion rate and the reasoning accuracy of task reasoning in a time slot as a target; decoupling the problem into a server scheduling optimization problem P1, establishing a graph model of a server task, and solving the problem P1 by using a double-greedy algorithm; and simplifying the problem into P2, establishing a Markov decision process, and solving an optimal strategy by using a multi-agent reinforcement learning algorithm. The method is suitable for multi-user equipment to carry out multi-outlet DNN reasoning partition, and the task completion rate and reasoning accuracy within a period of time can be effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of edge intelligence technology, and is a method for partitioning and scheduling multi-exit deep neural network (DNN) reasoning in a collaborative edge server in a multi-user scenario. The method establishes a server task model by predicting and calculating data for complex tasks, and formulates near-optimal exit, partition, and scheduling decisions, thereby maximizing the task completion rate and reasoning accuracy. Background Art

[0002] In order to meet the challenges of applying DNN in resource-constrained environments, researchers have begun to explore in depth how to offload some DNN reasoning tasks to edge servers, thereby reducing the amount of computation on edge devices, improving network reasoning efficiency and resource utilization, and enabling DNN to adapt to resource-constrained edge devices more efficiently. In edge intelligence environments, intelligently splitting and coordinating DNN reasoning tasks between local devices and edge devices can enable DNN models to be flexibly deployed according to different computing devices and network environments, thereby improving the overall efficiency of the reasoning process. At the same time, as an effective optimization strategy, the early exit mechanism can set multiple intermediate exits in the network architecture, and terminate some reasoning tasks in advance based on the complexity of the input tasks while maintaining an acceptable reasoning accuracy, thereby reducing unnecessary computation and reducing the task load transmitted to the edge server.

[0003] Most existing studies consider the optimization of the reasoning task structure. However, with the surge in DNN computing needs, especially in multi-user and multi-task scenarios, many tasks will be offloaded to edge servers. At this time, the server's scheduling of these tasks is crucial. Due to the complexity of the tasks, the performance of traditional scheduling algorithms needs to be improved, considering that the tasks will have time constraints and other conditions. Therefore, reasonable server scheduling combined with the optimization of the task's own structure can effectively improve the reasoning performance of the task. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention includes a variety of optimization techniques for DNN reasoning, and proposes an optimization method for partitioning multi-export DNN reasoning tasks and server task scheduling in a multi-user environment to maximize the task completion rate and reasoning accuracy. First, the present invention considers a system scenario consisting of multiple user devices and a single edge server, adopts a partition model and a multi-export architecture model, and combines server task scheduling to model DNN task reasoning, which is expressed as an optimization problem that maximizes the completion rate and reasoning accuracy of all tasks. Then, the present invention proposes a multi-agent reinforcement learning (Multi-Agent Proximal Policy Optimizationbased on Double Greedy Algorithm, DGMAPPO) algorithm based on double greedy to solve the optimization problem. In the solution process, a heuristic server task scheduling algorithm, the double greedy algorithm, is first proposed. According to the relevant prediction data of the tasks in the task set, a task model is established, and the task urgency and execution time are considered to optimize the server scheduling decision; then the optimization problem is simplified, and MAPPO is used to solve the optimal split point and the optimal network exit to maximize the task performance.

[0005] In order to achieve the above process, the present invention provides the following technical solutions:

[0006] A multi-user multi-exit DNN reasoning partitioning and task scheduling method is proposed. A trained multi-exit DNN model is deployed on each user device and edge server. Within a time slot, the tasks arrive at the user device to satisfy the Poisson distribution and are stored in the task queue of the user device. Taking the strategy feasibility interval as a constraint, the task exit point, partition point and scheduling decision as optimization variables, an optimization problem P is established with the goal of maximizing the total completion rate and reasoning accuracy of task reasoning within a time slot. The problem is decoupled into a server scheduling optimization problem P1, a graphical model of server tasks is established, and a doubly greedy algorithm is used to solve problem P1. The problem is then simplified to P2, a Markov decision process is established, and a multi-agent reinforcement learning algorithm is used to solve the optimal strategy.

[0007] Further, the method comprises the following steps:

[0008] Step 1: The system consists of N user equipments UE. Represents a UE set, and a capacity of M server The DNN model is deployed on each user device and edge server to perform DNN task reasoning. The task reasoning is divided into two parts. The first half of the division point is calculated locally by the user device, and the second half is offloaded to the server s for calculation.

[0009] Step 2: The deployed DNN model is a model with multiple export branches. Each export is a classifier consisting of only one fully connected layer and a softmax layer. L represents the total number of layers of the DNN model, and the K export set is represented by EXIT = {exit1, exit2, ..., exit ε ...,exit K} indicates that a threshold is set at each exit. If the maximum confidence value of the softmax layer output of the task at the exit is greater than the threshold, the task can choose this exit to exit reasoning early and return the prediction result.

[0010] Step 3: In time slot t, the number of tasks arriving at the user device satisfies the Poisson distribution, and the number of tasks arriving at user device i is expressed as M i (t), the jth task of user device i is denoted as task ij ={d ij ,τ ij ,dl ij}, where d ij is the size of the task input data, τ ij is the time when the task arrives at the device, dl ij is the task deadline, this M i (t) tasks are stored in the device’s task queue according to their deadlines. The exit selection strategy for each task is determined by x ij ∈{exit1,exit2,...,exit K} indicates that the partition selection strategy is determined by y ij ∈{1,2,...,L} represents;

[0011] Step 4: Each user device receives M i After (t) tasks, determine which tasks need to be transferred to server s to execute the partitioned tasks; if the task exit point is before the partition point, that is, x ij ≤y ij , the user device directly gives the task reasoning result and exits without transmitting it to server s for offloading calculation; otherwise, the part after the task partition point is offloaded to server s for further task scheduling processing;

[0012] Step 5: The task set that N user devices need to transmit to server s for scheduling and execution is called S server ={ta1,ta2,...,ta i ,...} means, among which, Represented as task ta i The time to arrive at server s, the time when server s starts to execute the task, the time required for server s to execute the task and the deadline of the task; the scheduling decision of the server is determined by the start execution time set It indicates that, Indicates that the task was not completed within its deadline;

[0013] Step 6: Calculate the task completion rate during the calculation process: The task completion status is divided into the number of completions on N user devices and the number of scheduling completions on server s; if the completion time of the task execution does not exceed its deadline, it is considered to be completed, otherwise it is considered to be unfinished; the number of tasks completed by the user device is calculated by IC local Indicates that the number of tasks completed by server s is represented by IC edge The completion rate of the task is:

[0014] Step 7: Calculate the task reasoning accuracy during the calculation process: Indicates task task ij Select exit ε The accuracy of the task, combined with the time limit of the task and its accuracy, the sum of the reasoning accuracy of all tasks is: in, is the emergency coefficient after the task time limit is converted, and σ is a parameter;

[0015] Step 8: Taking the task export selection strategy, partition selection strategy and server scheduling decision as optimization variables, maximizing the task completion rate and reasoning accuracy within time slot t as the optimization goal, an optimization problem is established. Its mathematical model is:

[0016]

[0017]

[0018] Constraint ① is the range constraint for export selection, constraint ② is the range constraint for partition points, constraints ③④ are the interval constraints for server scheduling decisions, and constraint ⑤ is the server task capacity constraint;

[0019] Step 9: Decouple the server scheduling strategy SE in problem P. The mathematical model is:

[0020]

[0021] The constraints are ③④⑤;

[0022] Step 10: Build a graph model of tasks in P1. Each task is abstracted as a square. The start and end of the square represent the time when the task arrives at the server. ij Its deadline dl ij ; The predicted execution time of the task on the server is a small shaded area in the square, and then we move this small shaded area inside the square to calculate the start execution time of each task;

[0023] Step 11: After establishing the graph model of server tasks, use the double greedy algorithm to solve: First, for the task set S server Medium deadline dl ij The smallest urgent task is processed first, represented as task obj , realize the first greedy; then, move the task obj The execution time of the grid is before the deadline, that is, its shaded area moves to the rightmost end of the grid, and its latest execution start time is calculated; use Q server Indicates that the task queue to be executed has arrived on the server. server The tasks are ranked by their predicted execution time Sort from small to large and insert them in order to be able to be in task obj The task that is completed before the latest start time is executed to achieve the second greedy; otherwise, execute task task obj ;

[0024] Step 12: Server Queue Q server Each time a task is inserted and executed, the queue Q is updated. server task; use τc to represent the completion time of the last executed task, and recalculate the queue Q server The start time of the task Then calculate whether the task can be completed before its deadline. If not, set its start execution time to ∞ and remove it from the task queue and collection, and then add the task to the queue Q. server Re-click the task in Sort by

[0025] Step 13: The task execution selection of each round of servers is processed according to steps 11 and 12 until the set S server All tasks are processed and the scheduling decision is obtained;

[0026] Step 14: Next, simplify the optimization problem P, and the mathematical model is:

[0027]

[0028] The constraints are ①②;

[0029] Step 15: Use the multi-agent reinforcement learning algorithm MAPPO to solve the optimal strategy to maximize the completion rate and reasoning accuracy. Model the problem P2 as a Markov decision process with an agent set of I = {1, 2, ..., N}, and use o i (t) = {M i(t),lc i (t),d i (t),τ i (t),dl i (t),c i (t)} represents the observation value of agent i in the tth time slot, where M i (t) is the number of tasks in time slot t, lc i (t) is the time when device i completed the last task in the previous time slot, d i (t) is the input data size set of task i, τ i (t) is the arrival time set of tasks, dl i (t) is the set of task deadlines, c i (t) is the computing power of device i; i (t) = {{x ij (t),y ij (t)}} represents the action of agent i in the tth time slot; the global reward based on all agent actions in the tth time slot is expressed as r(t) = αIC(t) + ACC(t);

[0030] Step 16: Use s(t) to represent the global state of the agent set in the tth time slot, and agent i obtains the local observation result o from s(t) i (t) and input into their respective actor networks, taking and executing action a i (t);

[0031] Step 17: The server obtains the action decisions of all agents Execute step 13 to get the scheduling decision and calculate the global reward r(t). The environment feeds back the global reward r(t) and the next state s(t+1) of the agent set at the current time slot t, and then converts Stored in a shared experience pool, all agents learn from the shared experience pool, evaluate and update their respective strategies by learning a global critic network;

[0032] Step 18: The trained strategy network outputs the optimal decision of task export and partition, and the double greedy algorithm gives the optimal scheduling decision.

[0033] The beneficial effects of the present invention are mainly manifested in that, for the scenario of multi-user devices and a single server, it not only considers the optimization of the DNN reasoning task's own structure in multiple exit points and partition directions, but also proposes an efficient server scheduling algorithm to perform reasonable scheduling optimization for complex tasks with time limits. The present invention can assist the server in formulating efficient scheduling strategies for tasks with urgent requirements to maximize the task completion rate and reasoning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A user-server collaboration model for DNN inference tasks.

[0035] Figure 2 The structural model for multi-export DNN partitioning.

[0036] Figure 3 The task model for the server.

[0037] Figure 4 Flowchart of the multi-agent reinforcement learning algorithm based on the server scheduling algorithm. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings.

[0039] Reference Figure 1 , Figure 2 , Figure 3 and Figure 4 , a multi-user multi-export DNN reasoning partitioning and task scheduling method, comprising the following steps:

[0040] Step 1: Figure 1 The scenario model of the system consists of N user equipment UE, represents the UE set, and a capacity of The system consists of edge servers s. A trained DNN model is deployed on each user device and edge server to perform DNN task reasoning. Task reasoning is divided into two parts. The first half of the division point is calculated locally by the user device, and the second half is offloaded to server s for calculation.

[0041] Step 2: The deployed DNN model is a model with multiple export branches. Each export is a classifier consisting of only one fully connected layer and a softmax layer, such as Figure 2 As shown in the structural model, L represents the total number of layers of the DNN model, and the K exit sets are represented by EXIT = {exit1, exit2, ..., exit ε ...,exit K}. A threshold is set at each exit. If the maximum confidence value of the softmax layer output of the task at the exit is greater than the threshold, the task can choose this exit to exit the reasoning early and return the prediction result;

[0042] Step 3: In time slot t, the number of tasks arriving at the user device satisfies the Poisson distribution, and the number of tasks arriving at user device i is expressed as M i (t). The jth task of user device i is denoted as task ij ={d ij ,τ ij ,dl ij}, where d ij is the size of the task input data, τ ij is the time when the task arrives at the device, dl ij is the task deadline, this M i (t) tasks are stored in the device’s task queue according to their deadlines. The exit selection strategy for each task is determined by x ij ∈{exit1,exit2,...,exit K} indicates that the partition selection strategy is determined by y ij ∈{1,2,...,L} represents;

[0043] Step 4: Each user device receives M i After (t) tasks, determine which tasks need to be transferred to server s to execute the partitioned tasks. If the task exit point is before the partition point, that is, x ij ≤y ij , the user device directly gives the task reasoning result and exits without transmitting it to server s for offloading calculation; otherwise, the part after the task partition point is offloaded to server s for further task scheduling processing;

[0044] Step 5: The task set that N user devices need to transmit to server s for scheduling and execution is called S server ={ta1,ta2,...,ta i ,...} means, among which, Represented as task ta i The time to arrive at server s, the time when server s starts to execute the task, the time required for server s to execute the task and the deadline of the task. The scheduling decision of the server is determined by the start execution time set It indicates that, Indicates that the task was not completed within its deadline;

[0045] Step 6: Calculate the task completion rate during the calculation process: The task completion status is divided into the number of completions on N user devices and the number of scheduling completions on server s. If the completion time of the task execution does not exceed its deadline, it is considered to be completed, otherwise it is considered to be unfinished. The number of tasks completed by the user device is calculated by IC local Indicates that the number of tasks completed by server s is represented by IC edge The completion rate of the task is:

[0046] Step 7: Calculate the task reasoning accuracy during the calculation process: Indicates task task ij Select exit εThe accuracy of the task, combined with the time limit of the task and its accuracy, the sum of the reasoning accuracy of all tasks is: in, is the emergency coefficient after the task time limit is converted, and σ is a parameter;

[0047] Step 8: Taking the task export selection strategy, partition selection strategy and server scheduling decision as optimization variables, maximizing the task completion rate and reasoning accuracy within time slot t as the optimization goal, an optimization problem is established. Its mathematical model is:

[0048]

[0049]

[0050] Constraint ① is the range constraint for export selection, constraint ② is the range constraint for partition points, constraints ③④ are the interval constraints for server scheduling decisions, and constraint ⑤ is the server task capacity constraint;

[0051] Step 9: Decouple the server scheduling strategy SE in problem P. The mathematical model is:

[0052]

[0053] The constraints are ③④⑤;

[0054] Step 10: Create a graph model for the tasks in P1, such as Figure 3 As shown, each task is abstracted as a square, and the start and end of the square represent the time when the task arrives at the server as ij Its deadline dl ij ; The predicted execution time of the task on the server is a small shaded area in the square, and then we move this small shaded area inside the square to calculate the start execution time of each task;

[0055] Step 11: After establishing the graph model of server tasks, use the double greedy algorithm to solve: First, for the task set S server Medium deadline dl ij The smallest urgent task is processed first, represented as task obj , realize the first greedy; then, move the task obj The execution time of the grid is before the deadline, that is, the shaded area moves to the rightmost end of the grid, and the latest execution start time is calculated. server Indicates that the task queue to be executed has arrived on the server. server The tasks are ranked by their predicted execution time Sort from small to large and insert them in order to be able to be in task objThe task that is completed before the latest start time is executed to achieve the second greedy; otherwise, execute task task obj ;

[0056] Step 12: Server Queue Q server Each time a task is inserted and executed, the queue Q is updated. server The task is to use τc to represent the completion time of the last executed task, and recalculate the queue Q server The start time of the task Then calculate whether the task can be completed before its deadline. If not, set its start execution time to ∞ and remove it from the task queue and collection, and then add the task to the queue Q. server Re-click the task in Sort by

[0057] Step 13: The task execution selection of each round of servers is processed according to steps 11 and 12 until the set S server Once all tasks are processed, the scheduling decision can be obtained;

[0058] Step 14: Next, simplify the optimization problem P, and the mathematical model is:

[0059]

[0060] The constraints are ①②;

[0061] Step 15: Use the multi-agent reinforcement learning algorithm MAPPO to solve the optimal strategy to maximize the completion rate and reasoning accuracy. Model the problem P2 as a Markov decision process with an agent set of I = {1, 2, ..., N}, and use o i (t) = {M i (t),lc i (t),d i (t),τ i (t),dl i (t),c i (t)} represents the observation value of agent i in the tth time slot, where M i (t) is the number of tasks in time slot t, lc i (t) is the time when device i completed the last task in the previous time slot, d i (t) is the input data size set of task i, τ i (t) is the arrival time set of tasks, dl i (t) is the set of task deadlines, c i (t) is the computing power of device i; i (t) = {{x ij (t),y ij(t)}} represents the action of agent i in the tth time slot; the global reward based on all agent actions in the tth time slot is expressed as r(t) = αIC(t) + ACC(t);

[0062] Step 16: Use s(t) to represent the global state of the agent set in the tth time slot, and agent i obtains the local observation result o from s(t) i (t) and input into their respective actor networks, taking and executing action a i (t);

[0063] Step 17: The server obtains the action decisions of all agents Execute step 13 to get the scheduling decision and calculate the global reward r(t). The environment feeds back the global reward r(t) and the next state s(t+1) of the agent set at the current time slot t, and then converts All agents learn from the shared experience pool and evaluate and update their own strategies by learning a global critic network. The process is as follows: Figure 4 As shown;

[0064] Step 18: The trained strategy network outputs the optimal decision of task export and partition, and the double greedy algorithm gives the optimal scheduling decision.

[0065] The solution of this embodiment is aimed at the scenario of multi-user devices and a single server. It not only considers the optimization of the DNN reasoning task's own structure in multiple exit points and partition directions, but also proposes an efficient server scheduling algorithm to perform reasonable scheduling optimization for complex tasks with time limits. The present invention can assist the server in formulating efficient scheduling strategies for tasks with urgent requirements to maximize the task completion rate and reasoning accuracy.

[0066] For example Figure 1 The scenario diagram of multiple user devices cooperating with a single edge server to process tasks is shown to illustrate a specific embodiment of the present invention.

[0067] First, there are N user devices and an edge server in the system. A trained multi-export DNN model is deployed on each user device and edge server to perform DNN task reasoning. Within a time slot, the tasks arrive at the user device to meet the Poisson distribution and are stored in the task queue of the user device. Task reasoning is divided into two parts. The first half of the division point is calculated locally by the user device, and the second half is offloaded to the edge server for calculation.

[0068] Secondly, after each user device receives the task, it determines which tasks need to be transmitted to the server for execution: if the task exit point is selected before the partition point, the user device directly gives the task reasoning result and exits early without transmitting it to the server for offloading calculation; if the task exit point is selected after the partition point, the user device calculates the part of the task before the partition point, and the part after the partition point to the exit point is offloaded to the server for further task scheduling processing.

[0069] Thirdly, taking the strategy feasibility interval as the constraint, the task exit point, partition point and scheduling decision as the optimization variables, the optimization problem P is established with maximizing the total completion rate and reasoning accuracy of task reasoning within a time slot as the optimization goal.

[0070] Then, the problem is decoupled into the server scheduling optimization problem P1, a graphical model of server tasks is established, and a double greedy algorithm is used to perform task scheduling selection for server iterations.

[0071] Finally, the problem is simplified to P2, a Markov decision process is established, and a multi-agent reinforcement learning algorithm is used to solve the optimal strategy.

[0072] The present invention aims at the scenario of multi-user devices and a single server. By optimizing the DNN reasoning task's own structure in multiple exit points and partition directions, it can assist the server in formulating efficient scheduling strategies for tasks with urgent requirements to maximize the task completion rate and reasoning accuracy.

[0073] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A multi-user multi-export DNN reasoning partitioning and task scheduling method, characterized in that: A trained multi-export DNN model is deployed on each user device and edge server. Within a time slot, tasks arrive at the user device in a Poisson distribution and are stored in the task queue of the user device. Taking the strategy feasibility interval as the constraint, the task exit point, partition point and scheduling decision as the optimization variables, an optimization problem P is established with the goal of maximizing the total completion rate and reasoning accuracy of task reasoning within a time slot; the problem is decoupled into the server scheduling optimization problem P1, a graphical model of server tasks is established, and a doubly greedy algorithm is used to solve problem P1; the problem is then simplified to P2, a Markov decision process is established, and a multi-agent reinforcement learning algorithm is used to solve the optimal strategy.

2. A multi-user multi-export DNN reasoning partitioning and task scheduling method as claimed in claim 1, characterized in that: The method comprises the following steps: Step 1: The system consists of N user equipments UE. represents the UE set, and a capacity of The system consists of edge servers s. A trained DNN model is deployed on each user device and edge server to perform DNN task reasoning. Task reasoning is divided into two parts. The first half of the division point is calculated locally by the user device, and the second half is offloaded to server s for calculation. Step 2: The deployed DNN model is a model with multiple exit branches. Each exit is a classifier consisting of only one fully connected layer and a softmax layer. L represents the total number of layers of the DNN model, and the K exit set is represented by EXIT = {exit1, exit2, ..., exit ε ..., exit K } indicates that a threshold is set at each exit. If the maximum confidence value of the softmax layer output of the task at the exit is greater than the threshold, the task can choose this exit to exit reasoning early and return the prediction result. Step 3: In time slot t, the number of tasks arriving at the user device satisfies the Poisson distribution, and the number of tasks arriving at user device i is expressed as M i (t), the jth task of user device i is denoted as taski j ={d ij , τ ij ,dl ij }, where d ij is the size of the task input data, τ ij is the time when the task arrives at the device, dl ij is the task deadline, this M i (t) tasks are stored in the device’s task queue according to their deadlines; the exit selection strategy for each task is determined by x ij ∈{exit1,exit2,...,exit K } indicates that the partition selection strategy is determined by y ij ∈{1, 2, ..., L} represents; Step 4: Each user device receives M i After (t) tasks, determine which tasks need to be transferred to server s to execute the partitioned tasks; if the task exit point is before the partition point, that is, x ij ≤y ij , the user device directly gives the task reasoning result and exits without transmitting it to server s for offloading calculation; otherwise, the part after the task partition point is offloaded to server s for further task scheduling processing; Step 5: The task set that N user devices need to transmit to server s for scheduling and execution is called S server ={ta1, ta2, ..., ta i , ...} means, among which, Represented as task ta i The time to arrive at server s, the time when server s starts to execute the task, the time required for server s to execute the task and the deadline of the task. The scheduling decision of the server is determined by the start execution time set It indicates that, Indicates that the task was not completed within its deadline; Step 6: Calculate the task completion rate during the calculation process: The task completion status is divided into the number of completions on N user devices and the number of scheduling completions on server s; if the completion time of the task execution does not exceed its deadline, it is considered to be completed, otherwise it is considered to be unfinished; the number of tasks completed by the user device is calculated by IC local Indicates that the number of tasks completed by server s is represented by IC edge Indicates; then the completion rate of the task is: Step 7: Calculate the task reasoning accuracy during the calculation process: Indicates task task ij Select exit ε The accuracy of the task, combined with the time limit of the task and its accuracy, the sum of the reasoning accuracy of all tasks is: in, is the emergency coefficient after the task time limit is converted, and σ is a parameter; Step 8: Taking the task export selection strategy, partition selection strategy and server scheduling decision as optimization variables, maximizing the task completion rate and reasoning accuracy within time slot t as the optimization goal, an optimization problem is established. Its mathematical model is: Constraint ① is the range constraint for export selection, constraint ② is the range constraint for partition points, constraints ③④ are the interval constraints for server scheduling decisions, and constraint ⑤ is the server task capacity constraint; Step 9: Decouple the server scheduling strategy SE in problem P. The mathematical model is: The constraints are ③④⑤; Step 10: Build a graph model of tasks in P1. Each task is abstracted as a square. The start and end of the square represent the time when the task arrives at the server. ij Its deadline dl ij ; The predicted execution time of the task on the server is a small shaded area in the square, and then we move this small shaded area inside the square to calculate the start execution time of each task; Step 11: After establishing the graph model of server tasks, use the double greedy algorithm to solve: First, for the task set S server Medium deadline dl ij The smallest urgent task is processed first, represented as task obj , realize the first greedy; then, move the task obj The execution time of the grid is before the deadline, that is, its shaded area moves to the rightmost end of the grid, and its latest execution start time is calculated; use Q server Indicates that the task queue to be executed has arrived on the server. server The tasks are ranked by their predicted execution time Sort from small to large and insert them in order to be able to be in task obj The task that is completed before the latest start time is executed to achieve the second greedy; otherwise, execute task task obj ; Step 12: Server Queue Q server Each time a task is inserted and executed, the queue Q is updated. server task; use τc to represent the completion time of the last executed task, and recalculate the queue Q server The start time of the task Then calculate whether the task can be completed before its deadline. If not, set its start execution time to ∞ and remove it from the task queue and collection, and then add the task to the queue Q. server Re-press the task in Sort by Step 13: The task execution selection of each round of servers is processed according to steps 11 and 12 until the set S server Once all tasks are processed, the scheduling decision can be obtained; Step 14: Next, simplify the optimization problem P, and the mathematical model is: The constraints are ①②; Step 15: Use the multi-agent reinforcement learning algorithm MAPPO to solve the optimal strategy to maximize the completion rate and reasoning accuracy. Model the problem P2 as a Markov decision process with an agent set of I = {1, 2, ..., N}, and use o i (t) = {M i (t), lc i (t), d i (t), τ i (t), dl i (t), c i (t)} represents the observation value of agent i in the tth time slot, where M i (t) is the number of tasks in time slot t, lc i (t) is the time when device i completed the last task in the previous time slot, d i (t) is the input data size set of task i, τ i (t) is the arrival time set of tasks, dl i (t) is the set of task deadlines, c i (t) is the computing power of device i; i (t) = {{x ij (t), y ij (t)}} represents the action of agent i in the tth time slot; the global reward based on all agent actions in the tth time slot is expressed as r(t) = αIC(t) + ACC(t); Step 16: Use s(t) to represent the global state of the agent set in the tth time slot, and agent i obtains the local observation result o from s(t) i (t) and input into their respective actor networks, taking and executing action a i (t); Step 17: The server obtains the action decisions of all agents Execute step 13 to get the scheduling decision and calculate the global reward r(t); the environment feeds back the global reward r(t) and the next state s(t+1) of the current time slot t to the agent set, and then converts s(t+1)) is stored in the shared experience pool, and all agents learn from the shared experience pool, evaluate and update their respective strategies by learning a global critic network; Step 18: The trained strategy network outputs the optimal decision of task export and partition, and the double greedy algorithm gives the optimal scheduling decision.

Images