Multi-uav assisted task graph computation offloading method and device with graph structure enhancement

Abstract

The application relates to a multi-unmanned aerial vehicle assisted task graph calculation unloading method and device with a graph structure enhancement. The method comprises the following steps: constructing a multi-unmanned aerial vehicle assisted mobile edge calculation system; modeling a calculation unloading problem as a cooperative multi-agent semi-Markov decision process; constructing a graph structure enhanced MAPPO collaborative strategy learning model, including local graph observation construction, a graph structure enhanced Actor network and a centralized graph Critic network; generating a local graph observation at each decision time; the Actor network outputs a hybrid action of subtask selection, execution position and power control; after the action is executed, a total system cost function is calculated, and a regret type shared reward is obtained based on a cost difference of adjacent decision steps; the Critic network receives a shared graph state to evaluate a global value; and the network is iteratively updated by combining the shared reward with a value function loss of a PPO pruning target. The method can effectively reduce the total system time delay overhead and stabilize convergence.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) communication technology, and in particular to a graph-enhanced multi-UAV collaborative mission graph calculation and unloading method and apparatus. Background Technology

[0002] With the development of mobile edge computing and UAV communication technologies, utilizing UAVs as aerial edge computing nodes to provide computational offloading services to ground users has become an important means of improving task processing efficiency. For complex edge computing applications such as target recognition, gene detection, and augmented reality, these applications are often not composed of a single independent task, but rather multiple subtasks with dependencies, which can typically be modeled using a directed acyclic graph (DAG). In the task graph, subsequent subtasks only have the conditions to execute after the current subtask is completed. The task completion process depends not only on communication transmission and computing resources, but also on the task graph topology and the results of previous executions. Furthermore, as the task arrival process, link status, and resource occupancy constantly change, traditional methods of statically modeling computational offloading in task graphs are no longer directly applicable. Therefore, constructing a dynamic computational offloading model oriented towards task graphs has become a challenge.

[0003] Currently, some work has attempted to introduce deep reinforcement learning into the task graph dynamic unloading problem to improve its adaptability to dynamic environments. However, most existing studies generally use independent vectors to describe the state of the task unloading model, which makes it difficult to fully characterize the topological dependencies and task progression order of the task graph, thus limiting the utilization of the task graph structural features by computing unloading strategies. Summary of the Invention

[0004] Based on this, it is necessary to provide a graph-enhanced multi-UAV assisted mission graph calculation and unloading method and apparatus that can fully utilize the mission graph topology information, achieve efficient multi-task collaborative execution, and reduce the total system latency overhead, in order to address the above-mentioned technical problems.

[0005] A graph-enhanced multi-UAV assisted mission graph computation and unloading method, the method comprising: Construct a mobile edge computing system assisted by multiple drones; the mobile edge computing system assisted by multiple drones includes multiple drones providing computing offloading services and multiple ground users; the ground users randomly generate task graph tasks; The multi-UAV assisted mission graph computation unloading problem is modeled as a multi-agent semi-Markov decision process. For multi-agent semi-Markov decision processes, a graph-enhanced MAPPO algorithm model is constructed. The graph-enhanced MAPPO algorithm model includes a local graph observation construction module, a graph-enhanced Actor network, and a centralized graph Critic network. At each decision-making moment, each ground user agent generates local graph observations based on its current active task graph and communication computing status through the local graph observation construction module; The Actor network takes local graph observations as input and outputs hybrid actions; the hybrid actions include subtask selection, execution location decision-making, and power control. After performing the mixed action, the total system cost function is calculated based on the system state change, and the regret-type shared reward is calculated based on the difference in the total system cost of adjacent decision steps. During the training phase, the Critic network receives the shared graph state, which is composed of graphs of all active tasks, and outputs a system state value assessment. The Actor and Critic networks are iteratively updated using the regret-based shared reward computational advantage estimation and the PPO pruning objective and value function loss to obtain an optimized computational offloading strategy.

[0006] A graph-enhanced multi-UAV assisted mission graph computation and offloading apparatus, the apparatus comprising: The system building module is used to build a multi-UAV assisted mobile edge computing system; the multi-UAV assisted mobile edge computing system includes multiple UAVs providing computing offloading services and multiple ground users; the ground users randomly generate task graph tasks; The modeling and network construction module is used to model the multi-UAV assisted task graph computational unloading problem as a multi-agent semi-Markov decision process. For the multi-agent semi-Markov decision process, a graph-enhanced MAPPO algorithm model is constructed. The graph-enhanced MAPPO algorithm model includes: a local graph observation construction module, used to construct local graph observations based on the current active task graph and communication computation state of each ground user agent; a graph-enhanced Actor network module, used to take local graph observations as input and output hybrid actions, including sub-task selection, execution location decision, and power control; and a centralized graph Critic network module, used to receive the shared graph state composed of all active task graphs during the training phase and output a system state value assessment. The reward calculation module is used to calculate the total system cost function based on the system state change after the execution of mixed actions, and to calculate the regret-type shared reward based on the difference of the total system cost of adjacent decision steps. The iterative update module is used to estimate the advantage by utilizing regret-type shared reward calculation and to iteratively update the Actor network and Critic network by using PPO pruning objective and value function loss to obtain an optimized computational offloading strategy.

[0007] The aforementioned graph-enhanced multi-UAV assisted task graph computation offloading method and apparatus, in this application, constructs a multi-UAV assisted mobile edge computing system and models it as a multi-agent semi-Markov decision process. It then constructs a MAPPO algorithm model comprising local graph observation construction, a graph-enhanced Actor network, and a centralized graph Critic network. At each decision time, local graph observations preserving task topology dependencies are generated. Based on these observations, the Actor network outputs a hybrid action of subtask selection, execution location, and power control. Subsequently, the total system cost function, including latency and energy default, is calculated, and a regret-based shared reward is obtained based on the cost difference between adjacent decision steps. The Critic network evaluates the global value using the shared graph state, and finally, the strategy is iteratively updated using the PPO algorithm. This process fully exploits the topological information of the task graph, achieves joint optimization of computation offloading decisions and transmit power control, effectively reduces the total system latency overhead, and achieves stable convergence. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating a graph-enhanced multi-UAV assisted mission graph computation and unloading method in one embodiment. Figure 2 This is a schematic diagram of a multi-UAV assisted mobile edge computing system in one embodiment; Figure 3 This is a schematic diagram of the internal computation process of a graph-enhanced MAPPO collaborative strategy learning model in one embodiment. Figure 4 This is a schematic diagram comparing the rewards obtained by this application and different algorithms with respect to the number of training steps in one embodiment; Figure 5 This is a schematic diagram comparing the total latency overhead of this application and different algorithms with respect to the task graph size in one embodiment; Figure 6 This is a schematic diagram comparing the total latency overhead of this application and different algorithms with respect to the number of users in one embodiment; Figure 7 This is a structural block diagram of a graph-enhanced multi-UAV assisted mission graph calculation and unloading device in one embodiment. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0010] In one embodiment, such as Figure 1 As shown, a graph-enhanced multi-UAV assisted mission graph computation and offloading method is provided, including: Step 102: Construct a mobile edge computing system assisted by multiple drones; the mobile edge computing system assisted by multiple drones includes multiple drones providing computing offloading services and multiple ground users; the ground users randomly generate task graph tasks.

[0011] Modeling a task graph-based multi-UAV assisted mobile edge computing system as Figure 2 The proposed system is composed of ground users and The system consists of drones with computational unloading capabilities, with ground users and drones each employing a cluster... and This indicates that ground users are randomly distributed within the target service area. The position is represented as Maximum transmission power is Drones, serving as aerial access nodes and edge computing nodes, are deployed at fixed altitudes. To provide communication and computing services to ground users, the spatial location of the UAV m is defined as follows: The drone platform is equipped with a directional antenna, and its ground coverage range is determined by the hovering altitude and beamwidth. Only ground users within the effective coverage area can establish a stable uplink with it and offload their tasks.

[0012] For any ground user, their business application consists of multiple interrelated subtasks, denoted as... To describe the dependencies between subtasks, Model as a directed acyclic graph , where nodes Indicates the first Sub-tasks ,in, This indicates the amount of input data for this subtask. Indicates the number of CPU cycles required to complete the subtask; directed edge Subtasks Execution depends on subtasks The output result. (Note: The original text appears to be incomplete and contains several typos. A more accurate translation would require the For subtasks The set of direct predecessors is such that subsequent successor tasks can only enter the execution state after all the preceding subtasks have been completed.

[0013] This application employs a binary offloading approach to model the computational offloading process of the task map generated by ground users. For users Executable subtasks Its unloading decision is denoted as .when When, it indicates that the subtask is executed by the user's local processor; when When the time slot is specified, it indicates that the subtask is offloaded to the drone edge server for execution via the uplink. Each subtask can only be executed locally or offloaded to a drone for execution, eliminating the issue of multiple drones being associated. It should be noted that whether a task can actually begin execution depends not only on the wireless transmission conditions and computing resource availability of the current time slot, but also on whether all its preceding subtasks have been completed.

[0014] (1) Local computing model When subtask When local computation is selected, it is executed by the corresponding user's local processor. The corresponding local computation time is...

[0015] It should be noted that subtasks An execution process can only begin when all its direct predecessor subtasks have been completed and the local processor is idle. Let... Subtasks In users The earliest available time on the local processor, which is also affected by the preceding task. Whether local processor usage is used and whether local computation is completed have an impact.

[0016] Among them, the boundary conditions are taken , Clearly, subtasks The completion time of all direct precursor sub-tasks is .therefore, The local completion time can be represented as

[0017] (2) Edge computing model When subtask When the data is unloaded for processing by the drone, it first needs to go through an uplink transmission phase. As can be seen from the aforementioned communication model, if the subtask... Unloaded onto drone Then its unloading delay is

[0018] Since users typically configure only one uplink sending interface, multiple unload subtasks for the same user cannot upload simultaneously. Subtasks The earliest available time for the corresponding uplink offloading channel is then:

[0019] Among them, the boundary conditions are taken , Considering task dependency constraints, the unloading process of a subtask can only begin after all its direct predecessor subtasks have been completed. Therefore, its upload completion time can be written as...

[0020] Once the subtask is uploaded, it will be executed on the corresponding drone's edge server. If the subtask... Assigned to drones Then its edge computing latency is

[0021] set up Indicates drone For users subtasks The earliest available time to provide edge computing services is as follows:

[0022] Among them, the boundary conditions are taken , Therefore, if the subtask Unloaded onto drone Then, the completion time of its edge execution on the drone can be expressed as

[0023] Since each subtask is assigned to at most one drone, its edge execution completion time can be further written as

[0024] In summary, subtasks The actual completion time is determined by its execution location. If executed locally, the completion time is... If executed on the drone side, the completion time is... Therefore, the user The entire application The completion time can be defined as

[0025] Considering that the task graph tasks generated by ground users are affected by precursor constraints, wireless transmission processes, computational offloading methods, and local energy constraints during computation and execution, the task completion latency is closely related to the task offloading strategy and communication resource allocation. Therefore, this application aims to minimize the total system latency by jointly optimizing the user's task offloading decision and transmit power.

[0026] First, define the energy consumption model on the user side. For users... subtasks If it is executed locally, the energy consumption of local computation can be expressed as:

[0027] in, For users The effective switching capacitor coefficient of the local processor.

[0028] If subtask If the data is unloaded to a drone for execution, then its unloading and transmission energy consumption is...

[0029] in, Subtasks Should I unload it onto the drone? .

[0030] Therefore, users The total energy consumption can be written as

[0031] Based on the model above, it can be seen that users The entire application completion time is The initial time when its task arrives at the system is Then the user Application completion delay is defined as

[0032] Therefore, the total delay of the entire system can be expressed as:

[0033] Step 104: Model the multi-UAV assisted mission graph computation unloading problem as a multi-agent semi-Markov decision process.

[0034] To ensure efficient computation of task graph tasks under limited energy, this application jointly computes the offloading decision and the user transmit power establishment system's total delay minimization problem. Considering that task graph tasks are not updated according to fixed-length discrete time slots during the decision-making process, but rather evolve continuously along the time axis after each decision, with events such as task transmission, task execution, and the arrival of new tasks, the time interval between two adjacent decisions is not fixed but is jointly determined by the current state, the selected action, and the event evolution process. Based on this characteristic, this application describes this problem as a cooperative multi-agent semi-Markov decision process, which is represented as follows: in, For state space, and Representing users respectively The observation space and action space, Represents the state transition kernel. Represents the reward function, This indicates the dwell time between two consecutive decisions.

[0035] (1) State and Observation: System status It is necessary to simultaneously characterize task dependencies, resource consumption, and dynamic changes in the communication and computing environment. Combining the event-driven scheduling process of this application, in the first... At the next decision point, the global state of the system can be represented as:

[0036] in, This indicates the task graph status of all currently active tasks. This indicates the resource usage status of the local processor, the drone edge processor, and the uplink. This indicates the communication status between the user and the drone, including the link quality and transmission rate. This represents the event evolution information consisting of events such as task arrival, transmission completion, and computation completion. These components collectively determine the system's evolution from the current decision moment to the next decision moment, thus constituting the global state in SMDP.

[0037] For users At the moment of decision-making Local observations are denoted as

[0038] in, Indicates user exist The corresponding DAG task at that time, The node feature matrix, The mask for currently schedulable nodes. A vector describing the user's local environment state, including information such as the user's positional relationship with the drone, available computing resources, and transmission power.

[0039] (2) Action space For any user intelligent agent Its actions involve mission selection, execution location selection, and transmit power control, and can be represented as follows:

[0040] in, This indicates the selected subtask node. Indicates the execution method. Indicates local execution. This indicates that the process will be unloaded onto the drone. This represents the normalized transmit power ratio and only applies in offload mode.

[0041] (3) State transition Given the current state and joint actions Subsequently, the environment completes state transitions through an event-driven mechanism. The system's transition relationships can be described by the following transition kernel:

[0042] in, Indicates that given the current state and joint actions Under these conditions, the system transitions to the next state. and experience the length of stay The joint transition probability; Indicates the probability of the corresponding event occurring; This represents the event-driven process of environmental evolution. This represents exogenous random factors such as task arrival, task graph generation, and channel changes.

[0043] (4) Reward function The objective of this application is to minimize the completion latency of DAG tasks while meeting user energy constraints. For state Time user The energy already consumed Its actual power capacity This is a pre-set energy warning threshold. Further, record... In the state Complete the user The minimum additional energy consumption estimate required for the remaining unfinished tasks. Here, "minimum" refers to the energy dimension; that is, ignoring latency advantages and disadvantages, it estimates the minimum energy cost necessary to complete the remaining tasks from the perspective of minimizing energy consumption. Based on this, users can be defined. In state The default rate under the warning is

[0044] in, The above formula means that if the sum of the user's current energy consumption and the minimum energy required to complete the remaining tasks has not exceeded the warning threshold, then the user is considered not to have triggered the warning risk. In this case, the warning default ratio is taken as... Otherwise, the proportion increases as the excess range increases.

[0045] Similarly, to describe the actual extent of budget overruns already incurred by users, the hard default ratio is defined as...

[0046] Based on this, this application will state The total system cost is defined as follows:

[0047] in, It is the estimated completion time delay of the task. , and These represent the weighting coefficients for the delay term, the early warning default term, and the actual default term, respectively.

[0048] Based on the aforementioned cost function, this application employs a regret-based shared reward design. Specifically, the shared reward for the $k$-th decision step is defined as...

[0049] When a joint decision can reduce the total system cost more quickly... The reward is relatively large; conversely, if the decision worsens subsequent delays or budget risks, the reward is reduced.

[0050] Step 106: For the multi-agent semi-Markov decision process, construct a graph-enhanced MAPPO algorithm model; the graph-enhanced MAPPO algorithm model includes a local graph observation construction module, a graph-enhanced Actor network, and a centralized graph Critic network.

[0051] To solve the time delay minimization problem under energy consumption constraints, this application proposes a MAPPO algorithm based on graph structure enhancement, such as... Figure 3As shown, the proposed algorithm framework consists of local graph observation construction, a graph-enhanced Actor network, a centralized graph Critic network, and MAPPO parameter updates. This algorithm is designed for multi-user, multi-UAV collaborative scheduling scenarios, employing a basic paradigm of centralized training and decentralized execution. For each user agent, a local graph observation is first constructed based on its current active task graph and communication / computation state, and a structured representation is extracted using a graph encoder. Secondly, the Actor outputs hybrid actions based on this, including sub-task selection, execution location decision, and power control. Correspondingly, the Critic receives a shared graph state composed of all active task graphs during the training phase and evaluates the state value of the entire system. Finally, the Actor and Critic complete iterative updates under the combined effect of PPO pruning objectives and value function loss. This algorithm is designed for multi-user, multi-UAV collaborative scheduling scenarios, employing a basic paradigm of centralized training and decentralized execution. During the execution phase, each user independently makes decisions based only on its own local observations; during the training phase, the centralized Critic estimates the global value of the system using the shared graph state, thereby mitigating the non-stationarity in multi-agent environments.

[0052] Step 108: At each decision moment, each ground user agent generates local graph observations through the local graph observation construction module based on its current active task graph and communication computing status.

[0053] The local graph observation construction module uses a graph neural network to encode the task dependencies in the task graph, extracting node-level, task-level, and global-level feature representations while preserving the task topology information. First, the basic features of subtasks are described by combining their execution state and context information. These basic features include the subtask's local cost information, dependent task release state, unloading decision, execution progress, uplink latency, and the busy / idle status of the local processor and the UAV-side processor. Then, the basic feature information of nodes is preprocessed using a multilayer perceptron to obtain the initial embedding of the nodes. The basic features and embedded features of the nodes are concatenated, and then transformed using a nonlinear mapping function to aggregate all nodes in the task graph to obtain the task-level representation. Finally, in a multi-task concurrent scenario, the representations of all currently active tasks are aggregated again to form a system-level global representation. This local graph observation preserves the topology of the task graph, providing rich structured information for subsequent policy decisions. Its specific implementation process is as follows: First, the basic features of subtasks are described by combining their execution state and context information, and defined as... This vector simultaneously reflects information such as the local cost of the subtask, the release status of dependent tasks, the unloading decision, the execution progress, the uplink latency, and the busy / idle status of the local processor and the UAV-side processor. Secondly, the basic feature information of the nodes is preprocessed using a multilayer perceptron to obtain the initial embedding of the nodes. The basic features of the nodes are then concatenated with the embedded features to obtain... Then, a transformation is performed using a nonlinear mapping function to aggregate all nodes in the task graph, resulting in a task-level representation. Finally, in a multi-task concurrent scenario, the representations of all currently active tasks are aggregated again to form a system-level global representation. .

[0054] Step 110: The Actor network takes local graph observations as input and outputs hybrid actions; the hybrid actions include subtask selection, execution location decision and power control.

[0055] For each user agent, the Actor network first encodes the graph structure of its local task graph, extracting node-level representations, task-level representations, and system-level structural information. Then, for any candidate node, its original input features, node embedding, graph-level representation of its task graph, and system-level global representation are concatenated to form the final decision features. Through this construction, the policy input not only reflects the execution state of the current subtask itself, but also its structural position in the entire DAG, the overall progress of its task, and the global competitive situation in the current system. For the hybrid action space, the Actor network uses a multi-head output structure for unified modeling: the subtask selection head outputs selection scores for all candidate nodes that satisfy the schedulable mask, and obtains the selection probability through softmax; the execution position selection head outputs the Bernoulli distribution probability of local execution or offloading execution; and the power control head outputs Beta distribution parameters to generate continuous power actions in the interval [0,1]. The policy decomposition of a single agent is the product of the above three parts, enabling different decision-making stages to share the same graph structure representation. This improves the policy network's ability to jointly model task dependencies and resource states. The specific implementation process is as follows: For the user agent, the Actor first encodes the graph structure of its local task graph, extracting node-level representations, task-level representations, and system-level structural information. Based on this, for any candidate node... The original input features, node embeddings, graph-level representations of the task, and system-level global representations are concatenated to form the final decision features.

[0056] in, Represents a node The task graph to which it belongs. Through this construction, the policy input not only reflects the execution status of the current subtask itself, but also its structural position in the entire DAG, the overall progress of the tasks to which it belongs, and the global competitive situation in the current system.

[0057] For the mixed action space, the Actor employs a multi-head output structure for unified modeling. First, the subtask selection head outputs a selection score for all candidate nodes that satisfy the ready mask.

[0058] The selection probabilities of candidate subtasks are obtained by combining the schedulable mask with softmax. Secondly, for selected nodes, the probability distribution of local execution or offloading execution is output by the execution position selection header.

[0059] in, For the Sigmoid function, This indicates the subtask execution location selection, with values ​​of 0 and 1 corresponding to local execution and offloading to the drone for execution, respectively. Finally, the power control head outputs the distribution parameters corresponding to the continuous power actions. Combining subtask selection, execution location selection, and power control, the actions of a single user agent can be written as follows:

[0060] in, Indicates the selected subtask node; This indicates the transmit power ratio when the unloading option is selected.

[0061] To accommodate the semantic features of different action components, subtask selection is modeled using a masked discrete classification distribution, execution location selection is modeled using a Bernoulli distribution, and power control is modeled using a Beta distribution for intervals. The continuous actions of the agent are modeled. Therefore, the policy of a single agent can be decomposed into...

[0062] This decomposition method preserves the semantic structure of the hybrid action space and enables different decision-making stages to share the same graph structure representation, thereby improving the policy network's ability to jointly model task dependencies and resource states.

[0063] Step 112: After performing the mixed action, calculate the total system cost function based on the system state change, and calculate the regret-type shared reward based on the difference in the total system cost of adjacent decision steps.

[0064] Based on the total system cost function, the regret-sharing reward for the k-th decision step is defined as the difference in the total system cost between adjacent decision steps: The reward is larger when a joint decision reduces the total system cost more quickly; conversely, the reward decreases if the decision worsens subsequent latency or budget risk. This reward design effectively guides agents to minimize task completion latency while satisfying energy constraints.

[0065] Step 114: During the training phase, the Critic network receives the shared graph state, which is composed of graphs of all active tasks, and outputs a system state value assessment.

[0066] Unlike local actors, Critic receives the shared graph state during the training phase—a shared graph composed of all active task graphs—and uses a graph structure encoder to uniformly process the multi-DAG composite graph, obtaining node-level, task-level, and global-level structural representations. Further, it combines numerical information such as global location, link status, and system load to construct the value evaluation input for the k-th decision step. ,in and These represent the results of node embedding and graph embedding after pooling, respectively. This represents the global structural representation of the shared graph. This represents the numerical observations corresponding to the shared graph states. Finally, Critic outputs the state value corresponding to each shared state. , These are the parameters of the value network, used for advantage function estimation.

[0067] Step 116: Utilize regret-based shared reward calculation advantage estimation and employ PPO pruning objective and value function loss to iteratively update the Actor network and Critic network to obtain an optimized computational offloading strategy.

[0068] The PPO algorithm is used for policy updates. The policy update magnitude is limited by pruning the objective function, and joint optimization is performed by combining the value function loss. After multiple iterations of training, the model can learn the optimal computation offloading and power control strategies under different task graph structures, user numbers, and resource contention conditions, and perform computation offloading according to the offloading strategy.

[0069] In the aforementioned graph-enhanced multi-UAV assisted task graph computation offloading method, firstly, a multi-UAV assisted mobile edge computing system is constructed, and the problem is modeled as a multi-agent semi-Markov decision process, laying the model foundation for dynamic, event-driven task graph scheduling. Secondly, an innovative MAPPO algorithm model is constructed, comprising a local graph observation construction module, a graph-enhanced Actor network, and a centralized graph Critic network. The local graph observation construction module uses a graph neural network to encode the task graph, extracting node-level, task-level, and global-level structural features, enabling each agent to perceive the dependencies between subtasks rather than solely relying on vector states, thereby solving the problem of... The problem of "inability to characterize topological dependencies" is addressed. The graph-enhanced Actor network takes local graph observations as input and outputs a hybrid action including subtask selection, execution location, and power control, achieving joint decision-making under dependency constraints. The centralized graph Critic network receives the shared graph state formed by stitching together the graphs of all active tasks during the training phase, using global structural information to evaluate state value, effectively mitigating non-stationarity in multi-agent environments. Simultaneously, by calculating the total system cost function including time delay and energy default, and the regret-based shared reward of the cost difference between adjacent decision steps, multi-objective optimization is transformed into a stable training signal. Finally, iterative updates are performed using the PPO pruning objective and value function loss. These methods work together to fully exploit the topological information of the task graph, achieving joint optimization of computational offloading decisions and transmit power control, thereby generating better scheduling strategies in dynamic environments.

[0070] In simulations, the algorithm proposed in this application achieves lower total latency overhead than existing benchmark schemes under different task graph sizes and number of users, and has higher training returns and more stable convergence, effectively solving the problem of limited scheduling performance caused by ignoring graph structure features in the background technology. In one embodiment, generating local graph observations through the local graph observation construction module includes: A graph neural network is used to encode the task dependencies in the task graph, and node-level features, task-level features, and system-level global features are extracted. The initial embedding of node-level features is obtained by preprocessing the basic feature vectors of subtasks using a multilayer perceptron. The basic feature vectors include the local cost information of the subtask, the release status of dependent tasks, the unloading decision, the execution progress, the uplink waiting time, and the busy / idle status of the local processor and the UAV-side processor. The task-level features are obtained by aggregating the node features of all nodes in the task graph. The system-level global features are obtained by aggregating the task-level features of all currently active tasks.

[0071] Specifically, this embodiment employs a graph neural network to encode the dependencies in the task graph. First, a basic feature vector is constructed for each subtask. This vector includes the subtask's local cost information (input data volume, required CPU cycles, etc.), dependency release status (whether the predecessor has completed), current unloading decision, execution progress (not started / in progress / completed), uplink waiting time, and the busy / idle status of the local processor and the UAV-side processor. Then, a multilayer perceptron maps the basic features to initial node embeddings. The basic features and embeddings are then concatenated and subjected to a nonlinear transformation to obtain node-level features. By aggregating the features of all nodes in the task graph, a task-level representation is obtained. Finally, in a multi-task concurrent scenario, the graph-level representations of all active tasks are aggregated again to form a system-level global representation. This embodiment can completely preserve the structural information of the task graph in the observation, enabling the subsequent Actor network to perceive the topological dependencies between subtasks and thus make more reasonable scheduling decisions.

[0072] In one embodiment, the Actor network takes the local graph observation as input and outputs a hybrid action, including: The Actor network first performs graph structure encoding on the local task graph, extracting node-level representations, task-level representations, and system-level structural information; For any candidate node, the original input features of the candidate node, the node embedding, the graph-level representation of the task graph to which it belongs, and the system-level global representation are concatenated to form a state feature vector; the Actor network adopts a multi-head output structure, which outputs the subtask selection probability, the execution position selection probability, and the power control parameters respectively.

[0073] Specifically, the Actor network first uses a graph encoder to structurally encode the local task graph, extracting node-level, task-level, and system-level structural information. For each candidate subtask node, its original input features, node embedding, graph-level representation of its task graph, and system-level global representation are concatenated into a high-dimensional state feature vector. This feature vector simultaneously characterizes the node's own state, its position in the DAG, the overall progress of its task, and the global competitive situation. Next, the Actor network adopts a multi-head output structure: the subtask selection head outputs the scores of candidate nodes (normalized to probabilities by softmax); the execution position selection head outputs a Bernoulli distribution to determine local execution or unloading; and the power control head outputs Beta distribution parameters for sampling continuous power ratios. Through feature concatenation and multi-head output, joint decision-making on discrete actions (subtask selection, execution position) and continuous actions (emission power) is achieved, and different decision-making stages share the same graph structure representation, improving the policy network's ability to jointly model task dependencies and resource states.

[0074] In one embodiment, the subtask selection probability is obtained by outputting selection scores for all candidate nodes that satisfy the schedulable mask and then normalizing them using softmax; the execution location selection probability is modeled using a Bernoulli distribution, outputting a binary distribution indicating local execution or offloading to a UAV for execution; the power control parameters are calculated using a Beta distribution for intervals. Modeling the continuous power action on the surface.

[0075] Specifically, subtask selection employs a masked discrete classification distribution. Only candidate nodes that satisfy the schedulable mask (i.e., all predecessors have completed) are considered, and the mask ensures the legality of the action. Execution location selection uses a Bernoulli distribution, with a value of 0 indicating local execution and 1 indicating offloading to the UAV for execution. Power control uses a Beta distribution, with its domain being... The Beta distribution is suitable for modeling continuous proportional values; compared to the Gaussian distribution, it can flexibly simulate different preferences. Using a probability distribution that matches the action semantics makes the policy network easier to train and converge, while a masking mechanism prevents the generation of illegal actions.

[0076] In one embodiment, the hybrid action is represented as ,in, Indicates the selected subtask node; This indicates the transmit power ratio when the unloading operation is selected. This indicates the execution location of the subtask; 0 indicates local execution, and 1 indicates execution offloaded to the drone.

[0077] Specifically, the above representation clearly distinguishes the three decision dimensions, facilitating subsequent policy decomposition and reward calculation. It clarifies the structure of the action space, providing a unified mathematical foundation for the multi-head output design and policy probability decomposition of Actor networks.

[0078] In one embodiment, the total system cost function is:

[0079] in, It is the estimated completion time delay of the task. , and These represent the weighting coefficients for the delay term, the early warning default term, and the actual default term, respectively. To provide early warning of default rates, The percentage of hard defaults. User serial number For user collection; The regret-type shared reward is defined as the first Shared rewards for each decision step .

[0080] Specifically, multi-objective optimization (minimizing time delay and satisfying energy constraints) is transformed into a single cost function, and a reward signal is constructed through the cost difference, thus avoiding the sparse reward problem and effectively guiding multi-agent collaborative learning.

[0081] In one embodiment, the early warning default ratio is:

[0082] in, This means that if the sum of the user's current energy consumption and the minimum energy required to complete the remaining tasks has not exceeded the warning threshold, then the user is considered not to have triggered a warning risk. In this case, the warning default ratio is taken as... Otherwise, the proportion of defaults under warning increases with the magnitude of the exceedance. For state Time user The energy already consumed Its actual power capacity For the pre-set energy warning threshold, In the state Complete the user Minimum additional energy required for the remaining unfinished tasks.

[0083] Specifically, by providing early warnings of default events, agents can be guided to adopt energy-saving strategies before energy is truly depleted, thus avoiding hard defaults and meeting long-term energy constraints.

[0084] In one embodiment, the state transition of the cooperative multi-agent semi-Markov decision process is achieved by an event-driven mechanism; the time interval between two adjacent decisions is not fixed and is determined by the current state, the selected action, and the event evolution process; the events include task arrival, transmission completion, and computation completion.

[0085] Specifically, the time interval between two adjacent decisions is not fixed, but is determined by the current state, the selected action, and events occurring in the system. Events include: the arrival of a new task, the completion of a subtask's transmission, and the completion of a subtask's computation. When an event occurs, the system state changes abruptly, triggering the next decision. Compared to discrete-time models with fixed time slots, event-driven models more closely reflect the uncertainties of real-world systems, reducing redundant decisions and improving learning efficiency.

[0086] In one embodiment, the specific process by which the Critic network receives the shared graph state includes: The graphs of all currently active tasks are stitched together into a shared graph. This shared graph is then processed uniformly using a graph structure encoder to obtain node-level, task-level, and global-level structural representations. Combining global location, link status, and system load information, the first... The value assessment input for each decision step is:

[0087] in, and These represent the results of node embedding and graph embedding after pooling, respectively. This represents the global structural representation of the shared graph. This represents the numerical observations corresponding to the shared graph state.

[0088] Specifically, during the training phase, the Critic network receives the shared graph state, which is composed of graphs from all active tasks. First, the stitched graph is uniformly processed by a graph structure encoder to obtain node-level embeddings and graph-level embeddings. Then, pooling is used to obtain the pooled results of the node and graph embeddings. Next, the global structural representation of the shared graph and numerical observations (global location, link state, system load, etc.) are combined to construct the value assessment input. Finally, the value network outputs the state value. The Critic network can accurately estimate the state value using global graph structure information and system-level numerical information, providing Actors with a low-variance value benchmark, thereby mitigating the non-stationarity problem in multi-agent environments and accelerating convergence.

[0089] In a specific embodiment, Figure 4 The study demonstrates the changing trends of training rewards for different algorithms as the number of training steps increases. It can be observed that the graph-enhanced MAPPO ultimately achieves the highest reward and maintains a relatively stable convergence state in the later stages of training. The graph-enhanced MAA2C ​​is second, and while its final performance is better than the two MLP-based methods, it is still slightly lower than the proposed method. Furthermore, the performance of the two heuristic methods significantly lags behind the learning-based methods. HEFT consistently outperforms RR, but both methods maintain low overall rewards that do not change significantly with the number of training steps.

[0090] Figure 5The total latency of different algorithms under different DAG sizes is shown. As the DAG size increases, the total latency of all algorithms increases. This is because with an increase in the number of task nodes, the system needs to process more subtasks, the predecessor constraints become more complex, and the coupling between task waiting, offloading, and edge execution also strengthens, thus further increasing the overall scheduling difficulty. In this process, the graph-enhanced MAPPO consistently maintains the lowest total latency, demonstrating the best scalability; the graph-enhanced MAA2C ​​is second; the two MLP-based methods generally lag behind their corresponding graph-structured versions; HEFT and RR show more significant latency increases, indicating that heuristic methods have difficulty maintaining stable performance when the task size increases.

[0091] Figure 6 The total latency overhead of different algorithms under varying user numbers is shown. As the number of users increases, the total latency of all algorithms increases. This is because, with the number of drones and system bandwidth remaining constant, more users accessing the system leads to stronger task contention and resource consumption, further burdening task transmission, edge execution, and queuing processes, thus increasing the total system latency. The results show that the graph-enhanced MAPPO consistently maintains the lowest total latency overhead across different user scales, demonstrating good adaptability to user scale. When the number of users is small, the performance difference between the various learning-based methods is relatively limited; however, as the number of users continues to increase, the gap between heuristic and learning-based methods gradually widens, with HEFT and RR showing more significant latency increases, indicating that fixed-rule methods struggle to maintain optimal scheduling performance under intensified multi-user contention.

[0092] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0093] In one embodiment, such as Figure 7 As shown, a graph-enhanced multi-UAV assisted mission graph computation and offloading device is provided, comprising: System building module 702 is used to build a mobile edge computing system assisted by multiple drones; the system includes multiple drones providing computing offloading services and multiple ground users; the ground users randomly generate task graphs and tasks; Modeling and Network Building Module 704 is used to model the multi-UAV assisted task graph computational unloading problem as a multi-agent semi-Markov decision process. For the multi-agent semi-Markov decision process, a graph-enhanced MAPPO algorithm model is constructed. The graph-enhanced MAPPO algorithm model includes: a local graph observation construction module, used to construct local graph observations based on the current active task graph and communication computation status of each ground user agent; a graph-enhanced Actor network module, used to take local graph observations as input and output hybrid actions, including sub-task selection, execution location decision, and power control; and a centralized graph Critic network module, used to receive the shared graph state composed of all active task graphs during the training phase and output a system state value assessment. The reward calculation module 706 is used to calculate the total system cost function based on the system state change after the execution of the mixed action, and to calculate the regret-type shared reward based on the difference of the total system cost of adjacent decision steps. The iterative update module 708 is used to estimate the advantage by utilizing regret-type shared reward calculation and to iteratively update the Actor network and Critic network by using PPO pruning objective and value function loss to obtain an optimized computational offloading strategy.

[0094] The system construction module initializes the deployment locations of multiple UAVs and ground users, task graph generation parameters, communication and computing resources, providing a foundational environment for subsequent decision-making. The modeling and network building module transforms the physical problem into a cooperative multi-agent semi-Markov decision process and establishes a graph-enhanced MAPPO network. The local graph observation construction module extracts structured features from the task graph, the Actor network module outputs mixed actions based on local observations, and the Critic network module evaluates value using the global shared state during the training phase. The reward calculation module calculates the total cost function and regret-based shared reward based on system state changes, providing supervisory signals for policy updates. The iterative update module updates the Actor and Critic network parameters using the PPO algorithm. These modules work collaboratively to achieve effective utilization of the task graph's structural features and joint optimization of computational offloading and power control, thereby reducing the overall system latency and satisfying energy constraints.

[0095] Specific limitations regarding the graph-enhanced multi-UAV assisted mission graph computation unloading device can be found in the above-described limitations regarding the graph-enhanced multi-UAV assisted mission graph computation unloading method, and will not be repeated here. Each module in the aforementioned graph-enhanced multi-UAV assisted mission graph computation unloading device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0096] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A graph-enhanced multi-UAV collaborative mission graph calculation and unloading method, characterized in that, The method includes: Construct a mobile edge computing system assisted by multiple drones; the mobile edge computing system assisted by multiple drones includes multiple drones providing computing offloading services and multiple ground users; the ground users randomly generate task graph tasks; The multi-UAV assisted mission graph computation unloading problem is modeled as a multi-agent semi-Markov decision process. For the multi-agent semi-Markov decision process, a graph-enhanced MAPPO algorithm model is constructed; the graph-enhanced MAPPO algorithm model includes a local graph observation construction module, a graph-enhanced Actor network, and a centralized graph Critic network; At each decision-making moment, each ground user agent generates local graph observations through the local graph observation construction module based on its current active task graph and communication computing status. The Actor network takes the local graph observations as input and outputs hybrid actions; the hybrid actions include subtask selection, execution location decision and power control. After performing the hybrid action, the total system cost function is calculated based on the system state change, and the regret-type shared reward is calculated based on the difference in the total system cost of adjacent decision steps. During the training phase, the Critic network receives a shared graph state composed of graphs stitched together from all active task graphs and outputs a system state value assessment. The Actor and Critic networks are iteratively updated using the regret-based shared reward computational advantage estimation and the PPO pruning objective and value function loss to obtain an optimized computational offloading strategy.

2. The method according to claim 1, characterized in that, Generating local map observations through the local map observation construction module includes: A graph neural network is used to encode the task dependencies in the task graph, and node-level features, task-level features, and system-level global features are extracted. The initial embedding of node-level features is obtained by preprocessing the basic feature vectors of subtasks using a multilayer perceptron. The basic feature vectors include the local cost information of the subtask, the release status of dependent tasks, the unloading decision, the execution progress, the uplink waiting time, and the busy / idle status of the local processor and the UAV-side processor. The task-level features are obtained by aggregating the node features of all nodes in the task graph. The system-level global features are obtained by aggregating the task-level features of all currently active tasks.

3. The method according to claim 1, characterized in that, The Actor network takes the local graph observations as input and outputs mixed actions, including: The Actor network first performs graph structure encoding on the local task graph, extracting node-level representations, task-level representations, and system-level structural information; For any candidate node, the original input features of the candidate node, the node embedding, the graph-level representation of the task graph to which it belongs, and the system-level global representation are concatenated to form a state feature vector; the Actor network adopts a multi-head output structure, which outputs the subtask selection probability, the execution position selection probability, and the power control parameters respectively.

4. The method according to claim 3, characterized in that, The subtask selection probability is obtained by outputting selection scores for all candidate nodes that satisfy the schedulable mask and then normalizing them using softmax; the execution location selection probability is modeled using a Bernoulli distribution, outputting a binary distribution indicating local execution or offloading to the UAV for execution; the power control parameters are calculated using a Beta distribution for intervals. Modeling the continuous power action on the surface.

5. The method according to claim 1, characterized in that, The hybrid action is represented as ,in, Indicates the selected subtask node; This indicates the transmit power ratio when the unloading operation is selected. This indicates the execution location of the subtask; 0 indicates local execution, and 1 indicates execution offloaded to the drone.

6. The method according to claim 1, characterized in that, The total cost function of the system is: in, It is the estimated completion time delay of the task. , and These represent the weighting coefficients for the delay term, the early warning default term, and the actual default term, respectively. To provide early warning of default rates, The percentage of hard defaults. User serial number For user collection; The regret-type shared reward is defined as the first Shared rewards for each decision step .

7. The method according to claim 6, characterized in that, The pre-warning default rate is: in, This means that if the sum of the user's current energy consumption and the minimum energy required to complete the remaining tasks has not exceeded the warning threshold, then the user is considered not to have triggered a warning risk. In this case, the warning default ratio is taken as... Otherwise, the proportion of defaults under warning increases with the magnitude of the exceedance. For state Time user The energy already consumed Its actual power capacity For the pre-set energy warning threshold, In the state Complete the user Minimum additional energy required for the remaining unfinished tasks.

8. The method according to claim 6, characterized in that, The state transitions of the multi-agent semi-Markov decision process are achieved by an event-driven mechanism; the time interval between two adjacent decisions is not fixed and is determined by the current state, the selected action, and the event evolution process; the events include task arrival, transmission completion, and computation completion.

9. The method according to claim 7, characterized in that, The specific process by which the Critic network receives the shared graph status includes: The graphs of all currently active tasks are stitched together into a shared graph. This shared graph is then processed uniformly using a graph structure encoder to obtain node-level, task-level, and global-level structural representations. Combining global location, link status, and system load information, the first... The value assessment input for each decision step is: in, and These represent the results of node embedding and graph embedding after pooling, respectively. This represents the global structural representation of the shared graph. This represents the numerical observations corresponding to the shared graph state.

10. A graph-enhanced multi-UAV assisted mission graph calculation and unloading device, characterized in that, The device includes: The system construction module is used to build a mobile edge computing system assisted by multiple drones; the mobile edge computing system assisted by multiple drones includes multiple drones providing computing offloading services and multiple ground users; the ground users generate task graphs and tasks in real time; The modeling and network construction module is used to model the multi-UAV assisted task graph computation unloading problem as a multi-agent semi-Markov decision process. For this multi-agent semi-Markov decision process, a graph-enhanced MAPPO algorithm model is constructed. This graph-enhanced MAPPO algorithm model includes: a local graph observation construction module, used to construct local graph observations based on the current active task graph and communication computation status of each ground user agent; a graph-enhanced Actor network module, used to take the local graph observations as input and output hybrid actions, including sub-task selection, execution location decision, and power control; and a centralized graph Critic network module, used to receive the shared graph state composed of all active task graphs during the training phase and output a system state value assessment. The reward calculation module is used to calculate the total system cost function based on the system state change after the execution of the hybrid action, and to calculate the regret-type shared reward based on the difference in the total system cost of adjacent decision steps. The iterative update module is used to estimate the advantage by utilizing regret-type shared reward calculation and to iteratively update the Actor network and Critic network by using PPO pruning objective and value function loss to obtain an optimized computational offloading strategy.

Images