Multi-unmanned aerial vehicle cooperative assisted mobile edge computing task scheduling method and system

Abstract

The invention discloses a multi-unmanned aerial vehicle cooperative auxiliary mobile edge computing task scheduling method and system, and belongs to the technical field of edge computing and intelligent scheduling, and the mobile edge computing task scheduling method comprises the following steps: obtaining the position information and computing task characteristics of a ground mobile terminal in a current time slot, performing preliminary task shunting according to the task data size and a preset threshold value; constructing a joint optimization model, and performing joint optimization on a two-dimensional track, a computing resource allocation strategy and a task migration decision of the unmanned aerial vehicle by taking minimization of total processing time delay of system tasks as a target; the optimization problem is solved by adopting a mixed action multi-agent deep reinforcement learning algorithm, the problem of complexity of multi-unmanned aerial vehicle cooperative scheduling in a dynamic environment is effectively solved, the task processing time delay is remarkably reduced, and the edge resource utilization rate and the system service continuity are improved.

Description

Technical Field

[0001] This invention relates to the field of edge computing and intelligent scheduling technology, specifically to a method and system for scheduling mobile edge computing tasks with multi-UAV collaborative assistance. Background Technology

[0002] With the rapid growth in the number of mobile smart terminals and the increasing demands for real-time large-scale computing services, traditional computing models relying on fixed infrastructure are no longer sufficient to meet the needs of terminal devices in edge environments for low-latency, high-reliability computing services. Mobile edge computing (MEC), as a technology that pushes computing resources down to the network edge, has become an important development direction to support the offloading of computing and service assurance for smart terminals. However, in actual deployment scenarios, fixed MEC servers are limited by site layout, infrastructure coverage, and complex terrain environments, especially in scenarios such as emergency communications, patrolling uninhabited areas, and temporary high-density activities, making it difficult to achieve continuous service coverage for all terminals. At this time, unmanned aerial vehicles (UAVs), with their high mobility, rapid deployment, and wide coverage, are widely used as aerial edge nodes to assist in the offloading and processing of computing tasks, becoming an important support method for next-generation edge intelligent services. However, limited by the energy and computing resources of a single UAV, its service capacity is limited and cannot meet the continuous and efficient task processing needs of a large number of terminals in dynamic environments. Therefore, using multiple UAVs to collaboratively provide edge computing services has become a research hotspot. Multi-UAV collaborative systems not only need to dynamically decide the service range and flight trajectory of each UAV, but also need to coordinate resource allocation and task unloading paths, thereby achieving full-process optimization of task calculation, transmission and return.

[0003] Existing research mostly employs fixed strategies, phased optimization, or heuristic methods for trajectory control and task scheduling of multiple UAVs, lacking end-to-end joint modeling capabilities for dynamic task environments, making it difficult to achieve efficient scheduling in complex scenarios. Furthermore, the task unloading process may involve discrete decisions such as task migration between UAVs and forwarding of computation results, further increasing the difficulty of solving the joint scheduling problem.

[0004] Therefore, there is an urgent need for an intelligent optimization method that can jointly consider trajectory control, computing resource allocation and task migration decisions in multi-UAV systems and is applicable to dynamic environments, so as to improve the overall service capability and response efficiency of the system. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for scheduling mobile edge computing tasks with multi-UAV collaborative assistance, so as to solve the problems mentioned in the background art.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a mobile edge computing task scheduling method with multi-UAV collaborative assistance, comprising the following steps:

[0007] Step A: Obtain the location information and computing task characteristics of the ground mobile terminal in the current time slot, perform preliminary task diversion based on the task data size and preset threshold, and select the local computing path when the task data is not higher than the preset threshold and the terminal's local computing resources meet the conditions; further dynamically select the task offloading path based on the predicted trajectory of the mobile terminal, the computing resource status of the UAV and the service coverage.

[0008] Step B: Construct a joint optimization model to jointly optimize the UAV's two-dimensional trajectory, computing resource allocation strategy, and task migration decision with the goal of minimizing the total processing latency of the system tasks;

[0009] Step C: The above optimization problem is solved using a hybrid action multi-agent deep reinforcement learning algorithm. Each UAV acts as an agent, jointly learning trajectory control, resource allocation, and task migration action strategies. Discrete task migration actions are modeled as differentiable using the Gumbel-Softmax mechanism. Based on the optimization results, the flight trajectory and resource scheduling behavior of each UAV are controlled, and corresponding task unloading and migration operations are executed.

[0010] A mobile edge computing task scheduling system with multi-UAV collaborative assistance, the system being an infrastructure-independent multi-UAV-assisted air-ground integrated computing system, comprising U U UAVs equipped with MEC servers, K mobile terminals, and a remote MEC server m;

[0011] The drone ensemble and the terminal ensemble are respectively used Let K = {1, 2, ..., K} represent this.

[0012] In a further embodiment, in step A, the task unloading path includes the following:

[0013] (1) Unload the task to the drone within the current service range for edge computing and return the results;

[0014] (2) When the original associated drone resources are insufficient, or the terminal will leave its service range in the next time slot, the task or calculation results are transferred to the target drone covering the predicted location of the terminal for processing or forwarding.

[0015] (3) If the terminal is not currently covered by any drone or the task cannot be processed locally, the task will be offloaded to a remote MEC server for execution.

[0016] In a further embodiment, in step A, the location information q of the ground mobile terminal k in the current time slot t is obtained.k (t)=(x k (t),y k (t) and task data size D k (t), when D k If (t) is not greater than the preset task threshold δ, the task is calculated locally on the terminal; otherwise, D k (t)>δ, the task is offloaded to the drone closest to the mobile terminal k within time slot t for processing;

[0017] When terminal k unloads the task to drone u, considering that mobile terminals usually have predictable trajectories, it is assumed that drone u can perceive the path evolution information of the mobile terminal in advance, thereby determining whether the position of terminal k in the next time slot is still within the service range of drone u, and combining the resource status of drone u itself to decide whether to keep the current position or move to a better position, so as to more effectively complete the joint decision of task processing and migration strategy.

[0018] In a further embodiment, the trajectory of terminal k within each time slot with a time slot width of τ adopts a perturbation model. Specifically, terminal k moves at its current speed v k (t)∈[v min ,v max Along the current direction θ k (t) moves forward, and a small normal perturbation is introduced at the direction angle of each time slot to simulate the direction fluctuations and path deviations common in real-world environments, as shown below:

[0019]

[0020] The simulated random disturbance is expressed as follows:

[0021] θ k (t+1)=θ k (t)+Δθ k (t),Δθ k (t)~N(0,σ 2 );

[0022] Where, Δθ k (t) represents the random disturbance in the motion direction at terminal k in time slot t, which follows a mean of 0 and a variance of σ. 2 The Gaussian distribution is used; in addition, to ensure that terminal k always stays within the L×L target area, a boundary constraint model is introduced to restrict its movement; based on the set safety margin ε, if the position of terminal k in the next time slot exceeds the boundary of this area, its movement direction is reflected; specifically, when the position is close to the left / right boundary, i.e., x k (t+1)<εor x k (t+1)>L-ε, then θ k(t+1)=π-θ k (t); if the position is close to the upper / lower boundary, i.e., y k (t+1)<εor y k (t+1)>L-ε, then θ k (t+1)=-θ k (t).

[0023] In a further embodiment, it is assumed that drone u serves multiple mobile terminals in the current time slot t, and the service set is represented as K. u (t), based on the predicted trajectory information of the mobile terminal and the resource status of the UAV itself, the task unloading processing path can be divided into the following cases:

[0024] (1) If the location of terminal k in the next time slot is still within the service range of the currently associated drone u, and drone u has been allocated computing resources to other mobile terminals, the computing resources f to be allocated to terminal k are as follows: k,u If (t) is sufficient, the task will be calculated and processed by the original associated UAV u and the result will be returned directly to the terminal k in the next time slot;

[0025] (2) If the location of terminal k in the next time slot is not within the service range of the currently associated drone u, but drone u has already been allocated computing resources to other terminal tasks, the computing resources f that need to be allocated to terminal k are... k,u (t) is still sufficient; to reduce the delay caused by task migration, the task can first be calculated by the original associated UAV u, and then the best target UAV can be selected for result migration, with the best target UAV j returning the calculation result of UAV u to the terminal k; let the migration decision variable between UAVs be represented as ψ u,j (t), if ψ u,j (t) = 1 indicates that UAV u will transfer the original task data or processing results to the target UAV j; considering the overlapping coverage areas between UAVs, the optimal target UAV j should have sufficient remaining computing resources in the next time slot and be compatible with q. k (t+1) The drone with the shortest distance;

[0026] (3) If terminal k is located at position q in the next time slot k If (t+1) is not within the service range of the currently associated drone u, and after drone u has been allocated computing resources to other terminal tasks, the remaining computing resources are insufficient to be allocated to terminal k, then the currently associated drone u cannot perform the unload computing task, but can only migrate the original task data to the best target drone j, and the target drone j will perform the task calculation and return the result.

[0027] If none of the above conditions are met, i.e. the mobile terminal cannot complete the task locally and is not within the coverage area of ​​any drone, the system will offload the task to the remote MEC server m as a fallback mechanism to ensure reliable task execution and maintain service continuity.

[0028] In a further embodiment, in step B, the constructed joint optimization model includes multiple decision variables: the two-dimensional trajectory of the UAV. Computing resource allocation With task migration decision The optimization objective is to minimize the total processing latency of all ground terminal tasks over the entire time T. The corresponding optimization problem is described as follows:

[0029]

[0030] C3: 0 ≤ x u (t)≤L,0≤y u (t)≤L,

[0031]

[0032] C1 indicates that the computational task of a single time slot migration can only be migrated from the original associated UAV to another target UAV; C2 ensures that the allocated CPU computing frequency does not exceed the maximum computing power of the UAV. C3 restricts the drone's flight range to a defined two-dimensional area; C4 represents the drone's movement vector Δq within a single time slot. u (t) cannot exceed the maximum movement distance. C5 ensures that the minimum safe distance is maintained between any two drones. min To avoid collisions.

[0033] In a further embodiment, the hybrid action multi-agent deep reinforcement learning algorithm in step C adopts a centralized training and distributed execution architecture for the joint optimization problem. Each UAV acts as an independent agent and makes trajectory control, computing resource allocation and task migration decisions based only on its own local observations during the execution phase. During the training phase, it shares the global state and all agent action information to jointly update the policy network, and improves system stability and convergence performance through a policy soft update mechanism.

[0034] Set the local observation state as follows:

[0035] The information that the UAV u can observe in time slot t includes its own position information q. u (t)=(x u (t),y u(t) and location information of mobile terminals within the service range q k (t)=(x k (t),y k (t) and its task information D k (t), and the remaining computing resources after the UAV u service computing task.

[0036] Set the global environment state to

[0037] The system status describes the state of location and service information for all drones and mobile terminals; therefore, it is a collection of observations from all drones.

[0038] Let the motion space of the drone u be a u (t)={q u (t),f k,u (t),ψ u,j (t)};

[0039] Based on the observed information, the current UAV will select a corresponding hybrid action, consisting of two parts. The continuous action part includes the UAV's two-dimensional trajectory adjustment vector. And the computing resources allocated to multiple mobile terminals within its service area. The migration decision variable for a discrete action corresponding to the original associated UAV u is to decide whether to migrate the task or task processing result to the target UAV j based on the remaining computing resources and the terminal's next location information.

[0040] Therefore, the set of actions of all agents is represented as:

[0041] Set the reward function of drone u as follows

[0042] The UAV u obtains the observation value o in time slot t. u (t) and take action a u (t), and receive a reward r u (t); where -p u It is an additional penalty when a drone flies out of the designated area, or when computing resources are insufficient, or when drones collide with each other;

[0043] Therefore, the set of rewards for all agents is represented as:

[0044] Based on the environmental state, the agent's actions in the environment, and the reward function, the maximum state-action value function Q is used. π (s i ,a iThe corresponding optimal strategy can be obtained:

[0045]

[0046] Where, π * γ represents the optimal strategy, which refers to the agent's actions in the environment; E represents the expectation; γ∈[0,1] represents the reward discount factor.

[0047] In a further embodiment, the discrete task transfer action in step C is modeled as differentiable using the Gumbel-Softmax mechanism. Specifically, the Gumbel-Softmax reparameterization technique is introduced into the Actor network to convert the non-differentiable discrete action representation into a differentiable probability representation, thereby supporting end-to-end joint training. Specifically, after receiving its own observations, each agent u∈U's policy network outputs a logits vector of length U-1, represented as:

[0048] κ u =[κ u,1 ,κ u,2 ,…,κ u,j ,…,κ u,U ],j≠u;

[0049] Among them κ u,j This indicates the strength of drone u's preference to migrate the task to drone j;

[0050] To simulate the randomness of the sampling process and maintain differentiability, a noise g sampled from the Gumbel(0,1) distribution is added to each logits, forming a perturbed logits vector. Next By normalizing using the Softmax function, the probability distribution vector of discrete actions is obtained:

[0051]

[0052] Where λ > 0 is the temperature parameter; p u,j ∈[0,1] represents the probability that drone u will transfer the task to drone j, and satisfies the normalization constraint Σ. j≠u p u,j =1; During the training phase, the probability output by Gumbel-Softmax can be used as a continuously differentiable representation in backpropagation; while during the execution phase, from the distribution... The process involves sampling a drone j based on the maximum probability value as the target drone for mission migration, and determining the final discrete migration action using the following strategy:

[0053]

[0054] In a further embodiment, in step C, the joint optimization problem is trained based on the hybrid action multi-agent deep deterministic policy gradient algorithm, and a converged near-optimal policy is obtained through multiple rounds of interactive learning. This allows control over the flight trajectory adjustment, computational resource scheduling, and task migration behavior of each UAV in each time slot. Based on the generated optimal scheduling policy, local processing, edge offloading, inter-UAV migration, or remote server offloading of tasks are executed respectively to achieve dynamic optimization scheduling of system-level task processing.

[0055] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: This invention provides a method and system for scheduling mobile edge computing tasks with multi-UAV collaborative assistance. In multi-UAV assisted MEC scenarios, addressing the issues of latency sensitivity, high mobility, and large fluctuations in computing demands of ground terminal tasks, a joint optimization scheduling mechanism is designed. Combining UAV trajectory control, computing resource allocation, and task migration strategies, it effectively alleviates the problems of limited edge node resources, uneven scheduling, and service interruption. This invention constructs a joint optimization model with the goal of minimizing the total system task processing latency, comprehensively considering terminal task characteristics, location prediction results, and UAV resource status, to achieve tasks locally, autonomous UAV service, and target migration. Dynamic offloading and scheduling between UAVs or remote MECs; simultaneously, a hybrid action multi-agent deep reinforcement learning algorithm is introduced to jointly optimize the continuous actions of UAV flight trajectory and computing resource allocation, and the discrete action decisions of task migration. After multiple rounds of training, the optimal scheduling strategy is obtained and deployed and executed in actual system operation, driving each UAV to make autonomous decisions in each time slot. Through this method, the system can dynamically adapt to the distribution changes of tasks and resources in mobile environments. While meeting the latency constraints of terminal tasks and the energy consumption and service range constraints of UAVs, it significantly reduces the overall task processing latency, improves the service efficiency and stability of the edge computing system, and has good intelligence, robustness and practical deployment value. Attached Figure Description

[0056] Figure 1 This is a flowchart of the mobile edge computing task scheduling method with multi-UAV collaborative assistance of the present invention;

[0057] Figure 2 This is a schematic diagram of the overall process of the mobile edge computing task scheduling method and system with multi-UAV collaborative assistance of the present invention;

[0058] Figure 3 This is a schematic diagram of a multi-UAV assisted mobile terminal task unloading scenario according to the present invention. Detailed Implementation

[0059] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0060] Please see Figures 1-3 This invention provides a technical solution: a method for scheduling mobile edge computing tasks with multi-UAV collaborative assistance, comprising the following steps:

[0061] Step A: Obtain the location information and computational task characteristics of the ground mobile terminal in the current time slot. Perform initial task distribution based on the task data size and a preset threshold. When the task data is not higher than the preset threshold and the terminal's local computing resources meet the conditions, select the local computing path. Based on the mobile terminal's predicted trajectory, the UAV's computing resource status, and service coverage, further dynamically select the task offloading path. The task offloading paths include the following:

[0062] (1) Unload the task to the drone within the current service range for edge computing and return the results;

[0063] (2) When the original associated drone resources are insufficient, or the terminal will leave its service range in the next time slot, the task or calculation results are transferred to the target drone covering the predicted location of the terminal for processing or forwarding.

[0064] (3) If the terminal is not currently covered by any drone or the task cannot be processed locally, the task will be offloaded to a remote MEC server for execution.

[0065] Step B: Construct a joint optimization model to jointly optimize the UAV's two-dimensional trajectory, computing resource allocation strategy, and task migration decision with the goal of minimizing the total processing latency of the system tasks;

[0066] Step C: The above optimization problem is solved using a hybrid action multi-agent deep reinforcement learning algorithm. Each UAV acts as an agent, jointly learning trajectory control, resource allocation, and task migration action strategies. Discrete task migration actions are modeled as differentiable using the Gumbel-Softmax mechanism. Based on the optimization results, the flight trajectory and resource scheduling behavior of each UAV are controlled, and corresponding task unloading and migration operations are executed.

[0067] A mobile edge computing task scheduling system with multi-UAV collaborative assistance is a multi-UAV-assisted air-ground integrated computing system that does not rely on infrastructure. The system includes U UAVs equipped with MEC servers, K mobile terminals and a remote MEC server m.

[0068] The drone ensemble and the terminal ensemble are respectively used The system, represented by K = {1,2,...,K}, provides continuous, low-latency computing services to dynamically distributed mobile terminals. It aims to achieve joint optimization and scheduling of task offloading paths, computing resource allocation, and drone trajectories through the collaborative work of multiple drones equipped with MEC servers.

[0069] Considering a multi-UAV-assisted air-to-ground converged computing system that does not rely on infrastructure, its mobile terminal task offloading network architecture is as follows: Figure 3 As shown.

[0070] In a further embodiment, in step A, the computational task size of mobile terminal k in time slot t is represented as D. k (t), the unloading mechanism is controlled by a preset task threshold δ. Specifically, if D k If (t)≤δ, the task is computed locally on terminal k, and its total computation latency is:

[0071]

[0072] Among them, f k This indicates the local computing power of terminal k.

[0073] Otherwise D k If (t) > δ, the task is offloaded to the UAV closest to the terminal k position in the current time slot for processing. Assume the service coverage radius of each UAV is R. u The index of the UAV u associated with terminal k in time slot t is defined as follows:

[0074]

[0075] When terminal k unloads the task to drone u, considering that mobile terminals usually have predictable trajectories, it is assumed that drone u can perceive the path evolution information of the mobile terminal in advance, thereby determining whether the position of terminal k in the next time slot is still within the service range of drone u, and combining the resource status of drone u itself to decide whether to keep the current position or move to a better position, so as to more effectively complete the joint decision of task processing and migration strategy.

[0076] In a further embodiment, the trajectory of terminal k within each time slot with a time slot width of τ adopts a perturbation model. Specifically, terminal k moves at its current speed v k (t)∈[v min ,v max Along the current direction θ k (t) moves forward, and a small normal perturbation is introduced at the direction angle of each time slot to simulate the direction fluctuations and path deviations common in real-world environments, as shown below:

[0077]

[0078] The simulated random disturbance is expressed as follows:

[0079] θ k (t+1)=θ k (t)+Δθ k (t),Δθ k (t)~N(0,σ 2 );

[0080] Where, Δθ k (t) represents the random disturbance in the motion direction at terminal k in time slot t, which follows a mean of 0 and a variance of σ. 2 The Gaussian distribution is used; in addition, to ensure that terminal k always stays within the L×L target area, a boundary constraint model is introduced to restrict its movement; based on the set safety margin ε, if the position of terminal k in the next time slot exceeds the boundary of this area, its movement direction is reflected; specifically, when the position is close to the left / right boundary, i.e., x k (t+1)<εor x k (t+1)>L-ε, then θ k (t+1)=π-θ k (t); if the position is close to the upper / lower boundary, i.e., y k (t+1)<εor y k (t+1)>L-ε, then θ k (t+1)=-θ k (t).

[0081] In a further embodiment, it is assumed that drone u serves multiple mobile terminals in the current time slot t, and the service set is represented as K. u (t), based on the predicted trajectory information of the mobile terminal and the resource status of the UAV itself, the task unloading processing path can be divided into the following cases:

[0082] Case 1: If the position of the next time slot of terminal k is q k (t+1) is still within the service range of the currently associated drone u, and the computing resources f that drone u needs to allocate to terminal k after being allocated to other mobile terminal task computing resources. k,u (t) is sufficient, that is, the following conditions are met:

[0083] Condition 1:

[0084] in This represents the maximum computing power of the drone u; This indicates that the drone u has been allocated computing resources to multiple other mobile terminal tasks within its service range; furthermore, the task is processed by the originally associated drone u and the result is directly returned to terminal k in the next time slot; where the offloading computation latency required for drone u to execute the task of terminal k in the current time slot t is:

[0085]

[0086] in This represents the number of CPU cycles required for the drone u to compute one unit of data.

[0087] To differentiate the task processing flow under different scenarios, the negligible task result feedback time is also included in the calculation. In scenario 1, the uplink transmission delay of the task data from terminal k to drone u is... The drone will process the mission results. The downlink communication transmission time returned to terminal k is expressed as follows: In summary, the total processing time for task 1 in scenario 1 can be expressed as follows:

[0088] Scenario 2: If the location of terminal k in the next time slot is not within the service range of the currently associated drone u, but drone u has already been allocated computing resources to other terminal tasks, the computing resources f that need to be allocated to terminal k are... k,u (t) is still sufficient; in order to reduce the delay caused by task migration, the task can first be calculated by the original associated UAV u and then the best target UAV can be selected for result migration, and the best target UAV j can return the calculation result of UAV u to the terminal k.

[0089] Let the migration decision variable between drones be represented as ψ u,j (t), if ψ u,j (t) = 1 indicates that UAV u will transfer the original task data or processing results to the target UAV j; considering the overlapping coverage areas between UAVs, the optimal target UAV j should have sufficient remaining computing resources in the next time slot, and be located at position q of terminal k in the next time slot. k The drone with the shortest distance (t+1), i.e., case 2, satisfies the following condition:

[0090] Condition 2:

[0091] The process of transferring the results of the computational task already processed by UAV u to the target UAV j is expressed as:

[0092]

[0093] in This represents the uplink data transmission rate between terminal k and drone u;

[0094] The target drone j then transmits the processing results of drone u to the terminal k, with a result feedback time of [time missing]. In summary, the total processing time for task 2 is expressed as follows:

[0095] Case 3: If terminal k is at position q in the next time slot k If (t+1) is not within the service range of the currently associated drone u, and after drone u has been allocated computing resources to other terminal tasks, the remaining computing resources are insufficient to allocate to terminal k, then the currently associated drone u cannot perform the offload computing task. Instead, it can only migrate the original task data to the best target drone j, which will then perform the task calculation and return the results. This satisfies the following conditions:

[0096] Condition 3:

[0097] The drone u transfers the original task to the target drone j. The transfer time is expressed as:

[0098]

[0099] Furthermore, the unloading calculation latency for drone j is:

[0100]

[0101] Where f k,j (t) represents the computing resources allocated to terminal k by drone j. This represents the number of CPU cycles required for drone j to compute one unit of data.

[0102] The drone j performs task processing and results feedback, with a result transmission time of [time missing]. Therefore, the total processing time for task 3 is expressed as follows:

[0103] If none of the above conditions are met, i.e. the mobile terminal cannot complete the task locally and is not within the coverage area of ​​any drone, the system will offload the task to the remote MEC server m as a fallback mechanism to ensure reliable task execution and maintain service continuity.

[0104] In a further embodiment, in step B, multiple UAV trajectory plans are jointly optimized based on a predefined adaptive unloading mechanism. Computing resource allocation and task migration decisions To minimize the total task processing latency of K mobile terminals within time T, the corresponding optimization problem is formulated as follows:

[0105]

[0106] C3: 0 ≤ x u (t)≤L,0≤y u (t)≤L,

[0107]

[0108] C1 indicates that the computational task of a single time slot migration can only be migrated from the original associated UAV to another target UAV; C2 ensures that the allocated CPU computing frequency does not exceed the maximum computing power of the UAV. C3 restricts the drone's flight range to a defined two-dimensional area; C4 represents the drone's movement vector Δq within a single time slot. u (t) cannot exceed the maximum movement distance. C5 ensures that the minimum safe distance is maintained between any two drones. min To avoid collisions.

[0109] In a further embodiment, in step C, the multi-agent deep deterministic policy gradient (HA-MADDPG) algorithm based on hybrid action space adopts a centralized training and distributed execution architecture for the joint optimization problem. Each UAV acts as an independent agent and makes trajectory control, computing resource allocation and task migration decisions based only on its own local observations during the execution phase. During the training phase, it shares the global state and all agent action information to jointly update the policy network, and improves the system stability and convergence performance through a policy soft update mechanism.

[0110] Each UAV has a limited observation range, therefore only partial state information can be observed, while the global state information and the actions of other UAVs are unknown. The local observation state is defined as follows:

[0111]

[0112] The information that the UAV u can observe in time slot t includes its own position information q. u (t)=(x u (t),y u (t) and location information of mobile terminals within the service range q k (t)=(x k (t),y k (t) and its task information D k (t), and the remaining computing resources after the UAV u service computing task.

[0113] The system state describes the location and service information of all UAVs and vehicles; therefore, it is a collection of observations from all UAVs. The global environment state is defined as follows:

[0114] Let the motion space of the drone u be a u (t)={q u (t),f k,u (t),ψ u,j (t)};

[0115] Based on the observed information, the current UAV will select a corresponding hybrid action, consisting of two parts. The continuous action part includes the UAV's two-dimensional trajectory adjustment vector. And the computing resources allocated to multiple mobile terminals within its service area. The migration decision variable for a discrete action corresponding to the original associated UAV u is to decide whether to migrate the task or task processing result to the target UAV j based on the remaining computing resources and the terminal's next location information.

[0116] Therefore, the set of actions of all agents is represented as:

[0117] Set the reward function of drone u as follows

[0118] The UAV u obtains the observation value o in time slot t. u (t) and take action a u (t), and receive a reward r u (t); where -p u It is an additional penalty when a drone flies out of the designated area, or when computing resources are insufficient, or when drones collide with each other;

[0119] Therefore, the set of rewards for all agents is represented as:

[0120] Based on the environmental state, the agent's actions in the environment, and the reward function, the maximum state-action value function Q is used. π (s i ,a i The corresponding optimal strategy can be obtained:

[0121]

[0122] Where, π * γ represents the optimal strategy, which refers to the agent's actions in the environment; E represents the expectation; γ∈[0,1] represents the reward discount factor.

[0123] In a further embodiment, in step C, the discrete task transfer action is modeled as differentiable using the Gumbel-Softmax mechanism. Specifically, the Gumbel-Softmax reparameterization technique is introduced into the Actor network to convert the non-differentiable discrete action representation into a differentiable probability representation, thereby supporting end-to-end joint training. Specifically, after receiving its own observations, each agent u∈U's policy network outputs a logits vector of length U-1, represented as:

[0124] κ u =[κ u,1 ,κ u,2 ,…,κ u,j ,…,κ u,U ],j≠u;

[0125] Among them κ u,j This indicates the strength of drone u's preference to migrate the task to drone j;

[0126] To simulate the randomness of the sampling process and maintain differentiability, a noise g sampled from the Gumbel(0,1) distribution is added to each logits, forming a perturbed logits vector. Next By normalizing using the Softmax function, the probability distribution vector of discrete actions is obtained:

[0127]

[0128] Where λ > 0 is the temperature parameter; p u,j ∈[0,1] represents the probability that drone u will transfer the task to drone j, and satisfies the normalization constraint ∑ j≠u p u,j =1; During the training phase, the probability output by Gumbel-Softmax can be used as a continuously differentiable representation in backpropagation; while during the execution phase, from the distribution... The process involves sampling a drone j based on the maximum probability value as the target drone for mission migration, and determining the final discrete migration action using the following strategy:

[0129]

[0130] In a further embodiment, in step C, at each time step, the UAV u, based on the current environmental state s(t), will change the local observation o. u (t) is input into the Actor network to obtain the immediate action a. u (t), after all drones have completed their actions, the environment returns a reward value r to each drone. u(t) and enters the next state s(t+1) with a certain probability, and UAV u obtains the local observation value o of the next time slot. u (t+1), and {o u (t),a u (t),r u (t),o u (t+1)} is stored as an experience tuple in an experience pool of size D, for use in network parameter training and updates;

[0131] Once the experience pool is full, N data is randomly drawn from each agent's experience pool. B The neural network is trained using a set of records. The Critic network of the UAV u is then updated by minimizing the mean squared error loss, defined as:

[0132]

[0133] in Indicates the target Q value;

[0134] The Actor network for the drone u uses the gradient of the strategy determining the Q-value to update the network parameters via gradient ascent to maximize the action value, expressed as:

[0135]

[0136] Finally, the two target networks update at a rate of τ. c <<1 and τ a <<1 performs a soft parameter update, as shown below:

[0137]

[0138] The HA-MADDPG algorithm is used to obtain optimization results, including the optimized UAV trajectory variables, computing resource allocation variables, and task migration decisions for each time slot. This allows for adjustments to the allocation of task computing resources and whether a task should be migrated to another UAV. The algorithm then determines the location the UAV will reach in the next time slot, maximizing vehicle communication rates in the next time slot and minimizing the latency of all mobile terminal task processing within the total time slots. This further enhances the multi-UAV collaborative service capability and the scheduling efficiency of computing tasks.

[0139] In step C, the joint optimization problem is trained based on the hybrid action multi-agent deep deterministic policy gradient algorithm, and a converged near-optimal policy is obtained through multiple rounds of interactive learning. Then, the flight trajectory adjustment, computing resource scheduling and task migration behavior of each UAV in each time slot are controlled. Based on the generated optimal scheduling policy, the local processing, edge offloading, inter-UAV migration or remote server offloading of tasks are executed respectively to achieve dynamic optimization scheduling of system-level task processing.

[0140] Against the backdrop of the rapid development of 6G mobile internet and smart IoT, MEC (Multi-access Edge Computing), as a key technology to alleviate the pressure on central computing and reduce terminal computing latency, has been widely applied in scenarios such as vehicle-to-everything (V2X), drone networks, and smart manufacturing. However, in highly dynamic environments with weak infrastructure support, the deployment of ground-based edge servers is limited, and a single fixed base station cannot meet the service demands of large-scale, multi-user, and low-latency operations. Therefore, multiple drones, with their flexible deployment, strong line-of-sight communication, and on-demand scheduling capabilities, have been widely studied for carrying MEC servers to provide aerial computing services to ground mobile terminals, forming an air-ground integrated edge computing system. In this system, the selection of offloading paths for terminal tasks, the flight trajectories of drones, the allocation of computing resources, and task migration strategies are coupled, constituting a high-dimensional, highly complex joint optimization problem. Traditional greedy or heuristic algorithms struggle to make globally optimal scheduling decisions in real time in dynamic environments. In recent years, deep reinforcement learning methods, due to their powerful policy approximation and online decision-making capabilities, have been widely used to solve high-dimensional control problems. In particular, for the hybrid action space, the hybrid action multi-agent reinforcement learning model enables multiple UAVs to acquire convergence strategies during training and achieve autonomous and efficient task scheduling and trajectory coordination during deployment. This effectively reduces system task processing latency and improves the overall performance of the aerial edge computing system while ensuring service continuity and resource utilization.

[0141] Therefore, this invention designs a mobile edge computing task scheduling method and system with multi-UAV collaborative assistance, aiming to address the problems of limited computing resources and high task processing latency of mobile terminals in highly dynamic scenarios. In this system, multiple UAVs equipped with MEC servers act as aerial computing nodes, jointly executing scheduling decisions such as task unloading, resource allocation, and task migration. Considering the high-dimensional hybrid action space composed of three types of optimization variables—UAV trajectory, computing resources, and migration behavior—the system design employs the Hybrid Action Multi-Agent Deep Deterministic Policy Gradient Algorithm (H-MADDPG) for solving the problem, combined with a centralized training and distributed execution architecture. During the training phase, each UAV shares global information to optimize the joint strategy; during the execution phase, each UAV autonomously decides its flight path, resource allocation scheme, and task migration actions based only on local observations. To achieve differentiable optimization of discrete migration actions, a Gumbel-Softmax mechanism is introduced to model migration actions as a continuously differentiable distribution, enabling the algorithm to converge effectively in end-to-end training. After multiple rounds of policy iteration training, the policy networks of each UAV converge, forming a deployable scheduling strategy. This strategy can dynamically optimize the trajectory control and scheduling behavior of each UAV in each time slot based on the predicted location of the mobile terminal, the workload, and the system status during actual operation, thereby effectively reducing the system task processing latency and improving the intelligence, efficiency, and reliability of edge computing services.

[0142] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and all such equivalent transformations fall within the protection scope of the present invention.

Claims

1. A method for scheduling mobile edge computing tasks with multi-UAV collaborative assistance, characterized in that, Includes the following steps: Step A: Obtain the location information and computing task characteristics of the ground mobile terminal in the current time slot, perform preliminary task diversion based on the task data size and preset threshold, and select the local computing path when the task data is not higher than the preset threshold and the terminal's local computing resources meet the conditions; further dynamically select the task offloading path based on the predicted trajectory of the mobile terminal, the computing resource status of the UAV and the service coverage. Step B: Construct a joint optimization model to jointly optimize the UAV's two-dimensional trajectory, computing resource allocation strategy, and task migration decision with the goal of minimizing the total processing latency of the system tasks; Step C: The above optimization problem is solved using a hybrid action multi-agent deep reinforcement learning algorithm. Each UAV acts as an agent, jointly learning trajectory control, resource allocation, and task migration action strategies. Discrete task migration actions are modeled as differentiable using the Gumbel-Softmax mechanism. Based on the optimization results, the flight trajectory and resource scheduling behavior of each UAV are controlled, and corresponding task unloading and migration operations are executed.

2. The mobile edge computing task scheduling system with multi-UAV collaborative assistance according to claim 1, characterized in that: The system is a multi-UAV-assisted air-ground integrated computing system that does not rely on infrastructure. The system includes U UAVs equipped with MEC servers, K mobile terminals, and a remote MEC server m. The set of drones and the set of terminals are represented by Uu={1,2,...,U} and K={1,2,...,K}, respectively.

3. The mobile edge computing task scheduling method with multi-UAV collaborative assistance according to claim 1, characterized in that: In step A, the task uninstallation path includes the following: (1) Unload the task to the drone within the current service range for edge computing and return the results; (2) When the original associated drone resources are insufficient, or the terminal will leave its service range in the next time slot, the task or calculation results are transferred to the target drone covering the predicted location of the terminal for processing or forwarding. (3) If the terminal is not currently covered by any drone or the task cannot be processed locally, the task will be offloaded to a remote MEC server for execution.

4. The mobile edge computing task scheduling method with multi-UAV collaborative assistance according to claim 1, characterized in that: In step A, the location information q of the ground mobile terminal k in the current time slot t is obtained. k (t)=(x k (t),y k (t) and task data size D k (t), when D k If (t) is not greater than the preset task threshold δ, the task is calculated locally on the terminal; otherwise, D k (t)>δ, the task is offloaded to the drone closest to the mobile terminal k within time slot t for processing; When terminal k unloads the task to drone u, considering that mobile terminals usually have predictable trajectories, it is assumed that drone u can perceive the path evolution information of the mobile terminal in advance, thereby determining whether the position of terminal k in the next time slot is still within the service range of drone u, and combining the resource status of drone u itself to decide whether to keep the current position or move to a better position, so as to more effectively complete the joint decision of task processing and migration strategy.

5. The mobile edge computing task scheduling method with multi-UAV collaborative assistance according to claim 4, characterized in that: The trajectory of terminal k within each time slot with a time slot width of τ adopts a perturbation model. Specifically, terminal k moves at its current speed v k (t)∈[v min ,v max Along the current direction θ k (t) moves forward, and a small normal perturbation is introduced at the direction angle of each time slot to simulate the direction fluctuations and path deviations common in real-world environments, as shown below: The simulated random disturbance is expressed as follows: i k (t+1)=θ k (t)+Δθ k (t),Δθ k (t)~N(0,σ 2 ); Where, Δθ k (t) represents the random disturbance in the motion direction at terminal k in time slot t, which follows a mean of 0 and a variance of σ. 2 The Gaussian distribution is used; in addition, to ensure that terminal k always stays within the L×L target area, a boundary constraint model is introduced to restrict its movement; based on the set safety margin ε, if the position of terminal k in the next time slot exceeds the boundary of this area, its movement direction is reflected; specifically, when the position is close to the left / right boundary, i.e., x k (t+1)<εor x k (t+1)>L-ε, then θ k (t+1)=π-θ k (t); if the position is close to the upper / lower boundary, i.e., y k (t+1)<εor y k (t+1)>L-ε, then θ k (t+1)=-θ k (t).

6. The mobile edge computing task scheduling method with multi-UAV collaborative assistance according to claim 3, characterized in that: Suppose that drone u serves multiple mobile terminals in the current time slot t, and the service set is represented as K. u (t), based on the predicted trajectory information of the mobile terminal and the resource status of the UAV itself, the task unloading processing path can be divided into the following cases: (1) If the location of terminal k in the next time slot is still within the service range of the currently associated drone u, and drone u has been allocated computing resources to other mobile terminals, the computing resources f to be allocated to terminal k are as follows: k,u If (t) is sufficient, the task will be calculated and processed by the original associated UAV u and the result will be returned directly to the terminal k in the next time slot; (2) If the location of terminal k in the next time slot is not within the service range of the currently associated drone u, but drone u has already been allocated computing resources to other terminal tasks, the computing resources f that need to be allocated to terminal k are... k,u (t) is still sufficient; to reduce the delay caused by task migration, the task can first be calculated by the original associated UAV u, and then the best target UAV can be selected for result migration, with the best target UAV j returning the calculation result of UAV u to the terminal k; let the migration decision variable between UAVs be represented as ψ u,j (t), if ψ u,j (t) = 1 indicates that UAV u will transfer the original task data or processing results to the target UAV j; considering the overlapping coverage areas between UAVs, the optimal target UAV j should have sufficient remaining computing resources in the next time slot and be compatible with q. k (t+1) The drone with the shortest distance; (3) If terminal k is located at position q in the next time slot k If (t+1) is not within the service range of the currently associated drone u, and after drone u has been allocated computing resources to other terminal tasks, the remaining computing resources are insufficient to be allocated to terminal k, then the currently associated drone u cannot perform the unload computing task, but can only migrate the original task data to the best target drone j, and the target drone j will perform the task calculation and return the result. If none of the above conditions are met, i.e. the mobile terminal cannot complete the task locally and is not within the coverage area of ​​any drone, the system will offload the task to the remote MEC server m as a fallback mechanism to ensure reliable task execution and maintain service continuity.

7. The mobile edge computing task scheduling method with multi-UAV collaborative assistance according to claim 1, characterized in that: In step B, the constructed joint optimization model includes multiple decision variables: the two-dimensional trajectory of the UAV. Computing resource allocation With task migration decision The optimization objective is to minimize the total processing latency of all ground terminal tasks over the entire time T. The corresponding optimization problem is described as follows: C1 indicates that the computational task of a single time slot migration can only be migrated from the original associated UAV to another target UAV; C2 ensures that the allocated CPU computing frequency does not exceed the maximum computing power of the UAV. C3 restricts the drone's flight range to a defined two-dimensional area; C4 represents the drone's movement vector Δq within a single time slot. u (t) cannot exceed the maximum movement distance. C5 ensures that the minimum safe distance is maintained between any two drones. min To avoid collisions.

8. A mobile edge computing task scheduling method with multi-UAV collaborative assistance according to claim 1, characterized in that: The hybrid action multi-agent deep reinforcement learning algorithm in step C adopts a centralized training and distributed execution architecture for the joint optimization problem. Each UAV acts as an independent agent and makes trajectory control, computing resource allocation and task migration decisions based only on its own local observations during the execution phase. During the training phase, it shares the global state and all agent action information to jointly update the policy network, and improves system stability and convergence performance through a policy soft update mechanism. Set the local observation state as follows: The information that the UAV u can observe in time slot t includes its own position information q. u (t)=(x u (t),y u (t) and location information of mobile terminals within the service range q k (t)=(x k (t),y k (t) and its task information D k (t), and the remaining computing resources after the UAV u service computing task. Set the global environment state to The system status describes the state of location and service information for all drones and mobile terminals; therefore, it is a collection of observations from all drones. Let the motion space of the drone u be a u (t)={q u (t),f k,u (t),ψ u,j (t)}; Based on the observed information, the current UAV will select a corresponding hybrid action, consisting of two parts. The continuous action part includes the UAV's two-dimensional trajectory adjustment vector. And the computing resources allocated to multiple mobile terminals within its service area. The migration decision variable for a discrete action corresponding to the original associated UAV u is to decide whether to migrate the task or task processing result to the target UAV j based on the remaining computing resources and the terminal's next location information. Therefore, the set of actions of all agents is represented as: Set the reward function of drone u as follows The UAV u obtains the observation value o in time slot t. u (t) and take action a u (t), and receive a reward r u (t); where -p u It is an additional penalty when a drone flies out of the designated area, or when computing resources are insufficient, or when drones collide with each other; Therefore, the set of rewards for all agents is represented as: Based on the environmental state, the agent's actions in the environment, and the reward function, the maximum state-action value function Q is used. π (s i ,a i The corresponding optimal strategy can be obtained: Where, π * γ represents the optimal strategy, which refers to the agent's actions in the environment; E represents the expectation; γ∈[0,1] represents the reward discount factor.

9. A mobile edge computing task scheduling method with multi-UAV collaborative assistance according to claim 1, characterized in that: The discrete task transfer actions in step C are modeled as differentiable using the Gumbel-Softmax mechanism. This involves introducing Gumbel-Softmax reparameterization into the Actor network to transform non-differentiable discrete action representations into differentiable probability representations, thereby supporting end-to-end joint training. Specifically, after receiving its own observations, each agent u∈U's policy network outputs a logits vector of length U-1, represented as: Among them κ u,j This indicates the strength of drone u's preference to migrate the task to drone j; To simulate the randomness of the sampling process and maintain differentiability, a noise g sampled from the Gumbel(0,1) distribution is added to each logits, forming a perturbed logits vector. Next By normalizing using the Softmax function, the probability distribution vector of discrete actions is obtained: Where λ > 0 is the temperature parameter; p u,j ∈[0,1] represents the probability that drone u will transfer the task to drone j, and satisfies the normalization constraint ∑ j≠u p u,j =1; During the training phase, the probability output by Gumbel-Softmax can be used as a continuously differentiable representation in backpropagation; During the execution phase, from the distribution The process involves sampling a drone j based on the maximum probability value as the target drone for mission migration, and determining the final discrete migration action using the following strategy:

10. A mobile edge computing task scheduling method with multi-UAV collaborative assistance according to claim 1, characterized in that: In step C, the joint optimization problem is trained based on the hybrid action multi-agent deep deterministic policy gradient algorithm, and a converged near-optimal policy is obtained through multiple rounds of interactive learning. Then, the flight trajectory adjustment, computing resource scheduling and task migration behavior of each UAV in each time slot are controlled. Based on the generated optimal scheduling policy, the local processing, edge offloading, inter-UAV migration or remote server offloading of tasks are executed respectively to achieve dynamic optimization scheduling of system-level task processing.

Images