Multi-unmanned aerial vehicle calculation unloading method and system based on height perception

Abstract

The invention discloses a multi-unmanned aerial vehicle calculation unloading method and system based on height perception. The method comprises the following steps: constructing a multi-node auxiliary mobile edge calculation system model; designing a state space containing a dynamic flight height; a highly-perceived multi-agent reinforcement learning algorithm is realized; the task unloading proportion, the user scheduling strategy and the flight height and angle of multiple nodes are jointly optimized; and dynamically adjusting the flight height according to the node energy state. By introducing a dynamic height perception and joint optimization mechanism, three beneficial effects are realized: firstly, the system can intelligently adjust the height of the unmanned aerial vehicle according to a real-time channel condition and an energy state, the flight energy consumption is remarkably reduced while the communication quality is ensured, and the total cost of the system is reduced by 15-25%; and secondly, through cooperative unloading and trajectory planning based on multi-agent reinforcement learning, the cooperative efficiency of multiple unmanned aerial vehicles is improved, and the task processing rate is improved by 20-30%.

Description

A method and system for computational unloading of multiple unmanned aerial vehicles based on altitude perception Technical Field

[0001] This invention relates to the field of mobile edge computing technology, and more specifically, to a multi-UAV computing offloading method and system based on altitude perception. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) and sixth-generation (6G) wireless networks, the number of smart mobile devices has seen unprecedented growth. These devices require substantial computing resources to support emerging innovative applications such as autonomous navigation, image processing, and facial recognition, which place stringent demands on low-latency services. Traditional cloud computing technologies, due to their bandwidth and propagation latency limitations, often fail to meet these requirements. To address this issue, mobile edge computing (MEC) has emerged, forming a distributed computing framework by deploying computing resources closer to end users to reduce end-to-end latency and improve service quality.

[0003] However, existing mobile edge computing networks face numerous challenges. First, traditional fixed-altitude base station deployment suffers from poor mobility and limited coverage, making it difficult to adapt to complex and ever-changing environments. Second, individual nodes also face insufficient computing power and poor battery life when handling a large number of tasks. To overcome these limitations, deploying nodes as aerial edge nodes, leveraging their high mobility and flexibility, has become an effective solution.

[0004] Existing research has the following shortcomings: 1) Most studies assume that nodes maintain a fixed altitude (e.g., 50 meters, 60 meters or 100 meters), ignoring the need for dynamic altitude adjustment in real flight environments; 2) In multi-node collaborative scenarios, there is a lack of comprehensive consideration of the impact of altitude changes on channel quality, energy consumption and latency; 3) Existing algorithms are insufficient in the joint optimization of altitude adjustment and task offloading.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] To address the aforementioned technical problems in related technologies, this invention proposes a multi-UAV computational unloading method and system based on altitude perception, which can jointly optimize flight altitude, trajectory planning, and task unloading decisions, thus overcoming the aforementioned shortcomings of existing technologies.

[0007] To achieve the above technical objectives, the technical solution of the present invention is implemented as follows: A multi-UAV computational offloading method based on altitude perception, comprising the following steps: S1 Constructing a system model containing multiple user devices and multiple UAVs equipped with mobile edge computing servers, wherein the flight altitude of the UAVs is configured to be dynamically adjusted within a preset continuous or discrete range; S2 Based on the system model, constructing a multi-dimensional state space including UAV dynamic flight altitude information, and defining a joint action space including UAV altitude adjustment actions, wherein the joint action space includes at least the UAV's selection of user devices, flight direction, flight distance, task offloading ratio, and altitude adjustment level; S3 Designing a multi-objective optimization reward function, wherein the reward function takes minimizing the total system latency and total energy consumption as its core objective, and integrates a virtual queue mechanism based on Lyapunov optimization theory to coordinately constrain long-term average energy consumption and task processing latency; S4 Based on the state space, joint action space, and reward function, using a multi-agent reinforcement learning algorithm for distributed training to generate an optimization strategy; the optimization strategy is used to control each UAV to autonomously and collaboratively determine its flight altitude, flight trajectory, and task offloading decision for user devices in each decision time slot.

[0008] Furthermore, the system model specifically characterizes the system behavior by integrating the following dynamic mathematical model: a node movement model, used to determine the position q(j+1) of the UAV in time slot j+1 based on the UAV's actions in time slot j, expressed as: Where v(j) is the flight speed, φ(j) is the flight direction angle, and Δt is the time slot length; the energy consumption model is used for the computer's energy consumption, and its total energy consumption E total At least including flight energy consumption E related to changes in flight distance and altitude. fly Energy consumption E related to edge computing comp And hovering energy consumption proportional to the current flight altitude h; a communication model to determine the transmission rate R(h) between the UAV and the user equipment at altitude h, whose channel power gain g(h) is expressed as a function related to the horizontal distance d and altitude h: Where α0 is the reference channel gain; the delay model is used to calculate the total delay T of task processing. total It includes at least the task transmission delay T. trans Edge computing latency T edge and user equipment local computing latency T local The aforementioned models collectively serve as the quantitative basis for the multi-agent reinforcement learning algorithm to interact with the environment, evaluate the effects of actions, and drive policy optimization.

[0009] Furthermore, the state space includes: the UAV's own state vector, the state vectors of all user devices, the state vectors of neighboring UAVs within communication range, and the system virtual queue state vector; wherein, the UAV's own state vector includes at least: current flight altitude, remaining battery energy, and horizontal plane position coordinates; the user device state vector for each user device includes at least: position coordinates, the amount of data for the computational task to be processed, and a binary flag indicating whether the communication link with the UAV is blocked; the neighboring UAV state vector includes at least: relative position coordinates and flight altitude; the system virtual queue state vector includes: an energy virtual queue reflecting the deviation between historical energy consumption and a preset average energy consumption budget, and a delay virtual queue reflecting the deviation between historical task processing delay and a preset delay threshold.

[0010] Furthermore, the altitude adjustment level corresponds to multiple discrete altitude values ​​within the preset flight altitude range.

[0011] Furthermore, the calculation expression for the multi-objective optimization reward function r(t) in time slot t is: Where V is an adjustable Lyapunov control parameter; r base (t) represents the total delay T actually generated by the system within time slot t. total (t) and total energy consumption E total (t) calculates the basic reward, and satisfies r base (t) = -[w1·T total (t) + w2·E total [(t)], where w1 and w2 are weighting coefficients; With Q D (t) represent the state values ​​of the energy virtual queue and the delay virtual queue in time slot t, respectively; E(t) and D(t) represent the actual energy consumption and task processing delay in time slot t, respectively; E avg D is the set average energy consumption budget; thresh The set delay threshold.

[0012] Furthermore, the multi-objective optimization reward function also includes a highly perceptive reward term r. height When the drone's flight altitude h is within the preset optimal altitude range, r height It is positive; otherwise, it is negative or zero.

[0013] Furthermore, the generation and execution of the optimization strategy depend on the dynamic sensitivity of the communication model and energy consumption model to flight altitude h; the channel power gain g(h) and the derived transmission rate R(h) in the communication model change dynamically with altitude h, directly affecting the transmission delay T in the delay model. trans The total energy consumption E in the energy consumption model totalIt includes a hovering energy consumption component β*h that is proportional to altitude h, and a flight energy consumption component E_fly that is affected by the altitude change Δh; by jointly optimizing altitude h through the multi-agent reinforcement learning algorithm, a dynamic balance is achieved between the transmission performance determined by the communication model and the energy cost determined by the energy consumption model, thereby optimizing the multi-objective reward function.

[0014] Furthermore, the multi-agent reinforcement learning algorithm is the SoftActor-Critic algorithm using the maximum entropy reinforcement learning framework, and its optimization objective function J(π) is: Where π is the strategy, ρ π Let r(s) be the state-action access distribution under policy π, γ be the discount factor, and r(s) be the state-action access distribution. t ,a t ) represents the reward, α represents the temperature parameter, and ℋ(π(·|s) t )) represents the policy π in state s t The entropy below is used to encourage exploration.

[0015] Furthermore, it also includes a multi-drone collaboration step: each drone periodically broadcasts its own status information within the communication range, the status information including at least: real-time location, flight altitude, remaining energy, and the identifier of the user equipment currently being served; collision avoidance is achieved by adding a collision penalty term to the multi-objective optimization reward function, and when the three-dimensional Euclidean distance between any two drones is less than a preset safe distance, a negative reward is applied to that drone.

[0016] According to another aspect of the present invention, a multi-UAV computational offloading system based on altitude perception is provided for implementing the method described above. The system includes: an environment modeling unit for constructing and maintaining a model of the multi-UAV assisted mobile edge computing system and acquiring system state information in real time; a decision generation unit, which includes a policy model pre-trained through multi-agent reinforcement learning, for generating joint action commands, including altitude adjustment, for each UAV based on the current state information provided by the environment modeling unit; a policy training unit for executing the training process of the multi-agent reinforcement learning algorithm, including constructing the state space, joint action space, and multi-objective optimization reward function, and iteratively updating the policy model parameters using interactive data until convergence; and a collaborative management unit for handling information exchange between multiple UAVs and performing conflict detection and coordination according to preset rules.

[0017] The beneficial effects of this invention are as follows: By introducing a dynamic altitude perception and joint optimization mechanism, this invention achieves three major benefits: First, the system can intelligently adjust the UAV altitude according to real-time channel conditions and energy status, significantly reducing flight energy consumption while ensuring communication quality, thus reducing the total system cost by 15-25%. Second, cooperative offloading and trajectory planning based on multi-agent reinforcement learning improve the efficiency of multi-UAV collaboration, increasing task processing rate by 20-30%. Finally, the proposed algorithm exhibits strong adaptability in real dynamic environments, with fast convergence speed and good robustness, making it particularly suitable for complex terrains such as urban areas and mountains. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 is a multi-node network offloading architecture diagram of the multi-UAV computation offloading method based on altitude perception according to an embodiment of the present invention; Figure 2 is the performance of altitude-aware SAC under a single node of the multi-UAV computation offloading method based on altitude perception according to an embodiment of the present invention; Figure 3 is the latency performance of AAMASAC under a multi-node multi-UAV computation offloading method based on altitude perception according to an embodiment of the present invention; Figure 4 is the performance of altitude-aware TD3 under a single node of the multi-UAV computation offloading method based on altitude perception according to an embodiment of the present invention; Figure 5 is the performance of altitude-aware PPO under a single node of the multi-UAV computation offloading method based on altitude perception according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0021] As shown in Figure 1, a multi-UAV computational offloading method based on altitude perception according to an embodiment of the present invention includes the following steps: S1 Constructing a system model containing multiple user devices and multiple UAVs equipped with mobile edge computing servers, wherein the flight altitude of the UAVs is configured to be dynamically adjusted within a preset continuous or discrete range; S2 Based on the system model, constructing a multi-dimensional state space including UAV dynamic flight altitude information, and defining a joint action space including UAV altitude adjustment actions, wherein the joint action space includes at least the UAV's selection of user devices, flight direction, flight distance, task offloading ratio, and altitude adjustment level; S3 Designing a multi-objective optimization reward function, wherein the reward function takes minimizing the total system latency and total energy consumption as its core objective, and integrates a virtual queue mechanism based on Lyapunov optimization theory to coordinately constrain long-term average energy consumption and task processing latency; S4 Based on the state space, joint action space, and reward function, using a multi-agent reinforcement learning algorithm for distributed training to generate an optimization strategy; the optimization strategy is used to control each UAV to autonomously and collaboratively determine its flight altitude, flight trajectory, and task offloading decision for user devices in each decision time slot.

[0022] Preferably, the system model specifically characterizes the system behavior by integrating the following dynamic mathematical model: a node movement model, used to determine the position q(j+1) of the UAV in time slot j+1 based on the UAV's action in time slot j, expressed as: Where v(j) is the flight speed, φ(j) is the flight direction angle, and Δt is the time slot length; the energy consumption model is used for the computer's energy consumption, and its total energy consumption E total At least including flight energy consumption E related to changes in flight distance and altitude. fly Energy consumption E related to edge computing comp And hovering energy consumption proportional to the current flight altitude h; a communication model to determine the transmission rate R(h) between the UAV and the user equipment at altitude h, whose channel power gain g(h) is expressed as a function related to the horizontal distance d and altitude h: Where α0 is the reference channel gain; the delay model is used to calculate the total delay T of task processing. total It includes at least the task transmission delay T. trans Edge computing latency T edge and user equipment local computing latency T local The aforementioned models collectively serve as the quantitative basis for the multi-agent reinforcement learning algorithm to interact with the environment, evaluate the effects of actions, and drive policy optimization.

[0023] Preferably, the state space includes: the UAV's own state vector, the state vectors of all user devices, the state vectors of neighboring UAVs within communication range, and the system virtual queue state vector; wherein, the UAV's own state vector includes at least: current flight altitude, remaining battery energy, and horizontal plane position coordinates; the user device state vector for each user device includes at least: position coordinates, the amount of data for the computational task to be processed, and a binary flag indicating whether the communication link with the UAV is blocked; the neighboring UAV state vectors include at least: relative position coordinates and flight altitude; the system virtual queue state vector includes: an energy virtual queue reflecting the deviation between historical energy consumption and a preset average energy consumption budget, and a delay virtual queue reflecting the deviation between historical task processing delay and a preset delay threshold.

[0024] Preferably, the altitude adjustment level corresponds to multiple discrete altitude values ​​within the preset flight altitude range.

[0025] Preferably, the calculation expression of the multi-objective optimization reward function r(t) in time slot t is: Where V is an adjustable Lyapunov control parameter; r base (t) represents the total delay T actually generated by the system within time slot t. total (t) and total energy consumption E total (t) calculates the basic reward, and satisfies r base (t) = -[w1·T total (t) + w2·E total [(t)], where w1 and w2 are weighting coefficients; With Q D (t) represent the state values ​​of the energy virtual queue and the delay virtual queue in time slot t, respectively; E(t) and D(t) represent the actual energy consumption and task processing delay in time slot t, respectively; E avg D is the set average energy consumption budget; thresh The set delay threshold.

[0026] Preferably, the multi-objective optimization reward function further includes a highly perceptive reward term r. height When the drone's flight altitude h is within the preset optimal altitude range, r height It is positive; otherwise, it is negative or zero.

[0027] Preferably, the generation and execution of the optimization strategy depend on the dynamic sensitivity of the communication model and energy consumption model to flight altitude h; the channel power gain g(h) and the derived transmission rate R(h) in the communication model change dynamically with altitude h, directly affecting the transmission delay T in the delay model. trans The total energy consumption E in the energy consumption model totalIt includes a hovering energy consumption component β*h that is proportional to altitude h, and a flight energy consumption component E_fly that is affected by the altitude change Δh; by jointly optimizing altitude h through the multi-agent reinforcement learning algorithm, a dynamic balance is achieved between the transmission performance determined by the communication model and the energy cost determined by the energy consumption model, thereby optimizing the multi-objective reward function.

[0028] Preferably, the multi-agent reinforcement learning algorithm is the Soft Actor-Critic algorithm using the maximum entropy reinforcement learning framework, and its optimization objective function J(π) is: Where π is the strategy, ρ π Let r(s) be the state-action access distribution under policy π, γ be the discount factor, and r(s) be the state-action access distribution. t ,a t ) represents the reward, α represents the temperature parameter, and ℋ(π(·|s) t )) represents the policy π in state s t The entropy below is used to encourage exploration.

[0029] Preferably, the method further includes a multi-UAV collaboration step: each UAV periodically broadcasts its own status information within the communication range, the status information including at least: real-time location, flight altitude, remaining energy, and the identifier of the user equipment currently being served; collision avoidance is achieved by adding a collision penalty term to the multi-objective optimization reward function, and when the three-dimensional Euclidean distance between any two UAVs is less than a preset safe distance, a negative reward is applied to that UAV.

[0030] According to another aspect of the present invention, a multi-UAV computational offloading system based on altitude perception is provided for implementing the method described above. The system includes: an environment modeling unit for constructing and maintaining a model of the multi-UAV assisted mobile edge computing system and acquiring system state information in real time; a decision generation unit, which includes a policy model pre-trained through multi-agent reinforcement learning, for generating joint action commands, including altitude adjustment, for each UAV based on the current state information provided by the environment modeling unit; a policy training unit for executing the training process of the multi-agent reinforcement learning algorithm, including constructing the state space, joint action space, and multi-objective optimization reward function, and iteratively updating the policy model parameters using interactive data until convergence; and a collaborative management unit for handling information exchange between multiple UAVs and performing conflict detection and coordination according to preset rules.

[0031] The specific usage of the above-mentioned technical solution of the present invention will be described in detail.

[0032] As shown in Figure 1, the system consists of M user equipments (UEs) and N nodes equipped with MEC servers. Nodes can provide computing services to all UEs and communicate with them. We consider a multi-node assisted mobile edge computing network. Multiple nodes act as mobile edge computing servers, providing computing offloading services to distributed edge devices. The entire communication cycle T is divided into I time intervals. In each time interval j, a node selects a UE, hovers over it, and then provides computing services. The tasks generated by the device are decomposed into two parts: tasks computed locally and tasks offloaded to the node.

[0033] We assume the node first flies at low speed to a random location, then hovers and communicates with one of the user equipment (UEs). The node continues to fly at a dynamic altitude H, ranging from 25 to 150 meters. The node's starting coordinate in time slot j is q(j), and its ending coordinate is q(j+1).

[0034] 1. Mathematical Model 1.1 Task Offloading Model In this mobile edge computing (MEC) system, we implement an offloading strategy for user equipment (UE) offloading tasks within each time slot. We define Rm(i) ∈ [0, 1] as the proportion of tasks offloaded to the node server, and (1 - Rm(i)) represents the remaining amount of tasks to be executed locally by the UE. The local execution delay of UE m in time slot i is: Where Am(i) represents the total number of computing tasks (in bits) of user equipment m in time slot i, S represents the number of CPU cycles required to process each unit byte, and fUE represents the computing power of the user equipment. In specific implementations, S should be explicitly defined as S = 1000 cycles / bit.

[0035] 1.2 Node Movement Model: When a node moves from position q(j) to a new hovering position: The maximum value of v(j) is v. max The flight direction angle is φ(j), and Δt = t fly + t com = 8 seconds is the time interval.

[0036] 1.3 Flight energy consumption of energy consumption model nodes considering altitude variation: , where m UAV For node quality, d horizontal t represents the horizontal flight distance, Δh represents the change in altitude, and t represents the horizontal flight distance. fly For flight time.

[0037] The computing power consumption of the MEC server is: Where κ is the chip structure coefficient, κ = 10^(-27) J·s³, t fly= 1 second, f UAV For server CPU frequency, t server Calculate the time for the server.

[0038] Total energy consumption also includes hovering energy consumption (proportional to altitude) and communication energy consumption: Where β is the hovering energy consumption coefficient (β = 0.002 W / m, unit: W / m), h is the current altitude, and E comm Communication energy consumption. Communication energy consumption E comm = P tx × t tr Among them, P tx Uplink transmit power, valued at 0.1W, t tr Transmission time, the calculation formula is shown in Section 1.4.

[0039] 1.4 Communication Model Considering Channel Power Gain under Dynamic Altitude Influence : Where α0 is the reference channel gain, d is the horizontal distance, and h is the node height. α0 = 0.00001, N 0LOS = 10^(-13) W, N 0NLOS = 10^(-11) W, This represents the channel power gain at height h.

[0040] The transmission rate is: Where B is the bandwidth, P tx Where N is the transmit power and N0 is the noise power. Based on the user equipment obstruction flag B... block Select noise power: N0 = { N 0LOS if B block = 0{ N 0NLOS if B block = 11.5 The total delay in the delay model includes transmission delay and computation delay: ,in: , , f UE User equipment calculation frequency, taken as 3 × 10^8 Hz; f UAV The node server's calculation frequency is set to 1.2 × 10^9 Hz; T total T represents the total delay. trans T represents transmission delay. edge T represents the edge computing latency. local Indicates local computation latency, A m R represents the total number of computing tasks for user device m. m This represents the proportion of tasks that are offloaded to the node server.

[0041] 1.6 Detailed Definition of System Parameters 1) Basic Environmental Parameters Site size: 100m × 100m; Initial node position: [50, 50] (site center); User position range: [0, 100]m randomly distributed; Task size range: 1.5-2.0 Mbits randomly generated; Initial total task volume: 60 Mbits (1048576 bits = 1Mbit); User movement speed: 1 m / s; Node mass muav: 9.65 kg; Node battery energy: 500 kJ.

[0042] 2) Time parameters: flight time t_fly = 1 s; calculation time t_com = 7 s; time slot length Δt = t_fly + t_com = 8 s; total number of time slots slot_num = int(T / Δt) = 40.

[0043] 3) Communication parameters: Reference channel gain α0 = 10⁻ 5 Line-of-sight link noise power N0_LOS = 10⁻¹³ W (-100dBm); non-line-of-sight link noise power N0_NLOS = 10⁻¹¹ W (-80dBm); communication bandwidth B = 1 MHz; uplink transmit power P_tx = 0.1 W.

[0044] 4) Calculation parameters: CPU cycles required per unit bit processing: S = 1000 cycles / bit; User equipment computing frequency f_UE = 3 × 10 8 Hz; Node server calculation frequency f_UAV = 1.2×10⁻⁶ 9 Hz; Chip structure coefficient κ = 10⁻² 7 J·s³; hovering energy consumption coefficient β = 0.002 W / m.

[0045] 5) Lyapunov optimization parameters: average energy budget E_avg = 200 J; delay threshold D_thresh = 20 s; Lyapunov parameter V = 100.0.

[0046] 6) Reward function parameters: Delay normalization reference value: 100 s; Energy consumption normalization reference value: 200 J; Delay weight: 0.7; Energy consumption weight: 0.3; Optimal altitude range: 60-90 m (based on channel quality and energy consumption balance); Altitude reward value: 0.5; Redo step penalty: -10.0; Node battery energy: 500 kJ.

[0047] 7) Algorithm hyperparameters; Maximum number of training rounds: 1000; Maximum number of steps per round: 100; Actor network learning rate: 0.0003; Critic network learning rate: 0.001; Discount factor: 0.95; Soft update parameter: 0.005; Entropy regularization coefficient: 0.1; Experience replay buffer capacity: 10000; Batch size: 256; Number of nodes: 2; Communication range: 50 m.

[0048] 8) Supplementary notes on parameter settings: Time parameter settings: The communication period T is set to 400 seconds. Each time slot contains 1 second of flight time and 7 seconds of calculation time, and there are a total of 40 time slots.

[0049] Network scale settings: The system includes 2 nodes and 4 user devices, with a communication range of 50 meters between nodes.

[0050] Action space settings: The discrete action space includes: UE selection: 4 options; flight direction: 11 discrete angles (uniformly distributed from 0 to 2π); flight distance: 11 levels (0 to 50m); unloading ratio: 11 levels (0 to 1); altitude adjustment: 5 levels (corresponding to 5 uniform altitudes from 25 to 150m); the total action space size is: 4 × 11 × 11 × 11 × 5 = 26,620 discrete actions.

[0051] The state space dimensions (system state vector) include: Node self-state: height (1) + remaining energy (1) + position (2) = 4 dimensions; User information: 4 users × (position 2 + task 1 + occlusion 1) = 16 dimensions; Neighbor information: (2-1) neighbors × (relative position 2 + height 1) = 3 dimensions; Virtual queue state: energy queue (1) + delay queue (1) = 2 dimensions; Total state dimensions: 4 + 16 + 3 + 2 = 25 dimensions.

[0052] The optimal height range is determined based on the following considerations: below 60 meters, the signal is easily blocked by ground obstacles, and the line-of-sight probability decreases; above 90 meters, the transmission distance increases, path loss increases, and energy consumption increases; 60-90 meters balances the line-of-sight probability and transmission distance, providing the best channel conditions in urban environments.

[0053] 2. Multi-objective reward function based on Lyapunov optimization. This invention adopts the Lyapunov optimization framework to balance delay and energy consumption constraints: 2.1 Virtual queue design. Energy consumption virtual queue: Delayed virtual queue: Among them, E avg For the average energy consumption budget, D thresh This is the delay threshold.

[0054] 2.2 Lyapunov functions: .

[0055] 2.3 Basic Design of Reward Function The reward function rbase(t) is defined as: r base (t) = -[w1·T total (t) + w2·E total [t] where; T total (t): Total delay (s) in time slot t; E total (t): Total energy consumption (J) in time slot t; w1, w2: Weighting coefficients, satisfying w1 + w2 = 1; default settings w1 = 0.7, w2 = 0.3.

[0056] The multi-objective reward function is: Where V is a Lyapunov parameter used to balance the trade-off between the objective function and constraint violations. E avg = 200 J, D thresh = 20 s, V = 100.0.

[0057] Highly optimized rewards: 3. Highly Aware Multi-Agent SAC Algorithm (AAMASAC) 3.1 State Space Design The system state includes: s = [h, Ebattery, x, y, xUE1, yUE1, Atask1, Bblock1, xUE2, yUE2, Atask2, Bblock2, xUE3, yUE3, Atask3, Bblock3, xUE4, yUE4, Atask4, Bblock4, Δxneighbor, Δyneighbor, hneighbor, QE, QD] where; h: current flight altitude (m); E battery Remaining battery energy (J); x, y: Current node position coordinates (m); x UEi , y UEi : Location coordinates (m) of the i-th user equipment; A taski : Task size (bits) of the i-th user device; B blocki : The occlusion flag for the i-th user device, where 0 indicates no occlusion and 1 indicates occlusion; Δx neighbor , Δy neighbor : Relative position (m) of neighboring nodes; h neighbor : Height of neighboring nodes (m); Q E Energy virtual queue state; Q DDelayed virtual queue status.

[0058] 3.2 Network Architecture Actor Network: Input state, output action probability distribution: Input layer: state dimension; Hidden layer: 400 neurons (ReLU activation); Hidden layer: 300 neurons (ReLU activation); Output layer: action dimension (Softmax activation); Critic Network (Double Q Network): Input layer: state + action (one-hot encoding); Hidden layer: 400 neurons (ReLU activation); Hidden layer: 300 neurons (ReLU activation); Output layer: Q-value (linear activation).

[0059] 3.3 Training Algorithm The algorithm is based on the Soft Actor-Critic (SAC) framework, with the addition of maximum entropy regularization: Where α is a temperature parameter that controls the balance between exploration and utilization, ρ π Let be the state-action distribution under policy π, and γ be the discount factor, ℋ (π(·|s t Let π be the policy in state s. t Entropy below.

[0060] Training parameter settings: discount factor γ = 0.95; temperature parameter α = 0.1; target network soft update parameter τ = 0.005; experience replay buffer size = 10000; batch size = 256.

[0061] Training process: 1) Initialize network parameters and experience replay buffer; 2) For each training stage: dynamically adjust the target height according to the training progress (staged training); reset the environment and obtain the initial state; for each time step: each agent selects an action according to the current policy, executes the joint action, observes the reward and the next state, stores the experience in the replay buffer, and when the buffer is sufficient, samples a small batch of data to update the Critic network to minimize the TD error, updates the Actor network to maximize the expected reward + entropy, and softly updates the target network; 3) Return the trained policy network. 4. Multi-node collaboration mechanism 4.1 Information sharing Nodes share the following within the communication range: current position and height, remaining energy, and current service user task queue status.

[0062] 4.2 Conflict Avoidance: Collisions are avoided through reward function design. , where d ij d represents the distance between nodes. safe This is the safe distance. The safe distance is d. safe Set to 10 meters, and apply a collision penalty when the distance between nodes is less than 10 meters.

[0063] 4.3 Load balancing: Load balancing is achieved through a virtual queue mechanism to prevent some nodes from being overloaded while other nodes are idle.

[0064] The execution steps are illustrated below: In a two-dimensional square area of ​​100×100 square meters, four user devices are randomly distributed, and two nodes are deployed to provide computing services.

[0065] 1. System parameters are set as follows: Node weight: 9.65kg; Time period: 400 seconds, divided into 40 time slots; User equipment computing power: 3×10^8 Hz; UAV server computing power: 1.2×10^9 Hz; Maximum flight speed: 50 m / s; Node battery energy: 500 kJ; Altitude range: 25-150 meters; Bandwidth: 1 MHz; Transmit power: 0.1 W; Noise power (LOS): 10^{-13} W; Noise power (NLOS): 10^{-11} W.

[0066] 2. Implementation steps 1) System initialization: Set the initial flight altitude to 100 meters; initialize the virtual queue Q_E=0, Q_D=0; initialize the experience replay buffer; 2) Status acquisition: acquire node height, remaining energy, and location information; acquire user equipment location, workload, and occlusion status; acquire neighbor node information (if any).

[0067] 3) Decision generation: Generate joint actions based on the AAMASAC algorithm, including: UE selection, flight direction, flight distance, unloading ratio, and altitude adjustment.

[0068] 4) Action execution: The node flies according to the decision and adjusts its altitude; performs task unloading and calculation; and calculates the actual delay and energy consumption.

[0069] 5) Learning and updating: Calculate Lyapunov rewards; store experience in the replay buffer; periodically update Actor and Critic network parameters; softly update the target network. 6) Height policy adjustment: Dynamically adjust the target height according to the training stage: Stage 1 (first 500 rounds): target height 100 meters; Stage 2 (500-800 rounds): target height 50 meters; Stage 3 (after 800 rounds): target height 25 meters; gradually learn the optimal policy at different heights. 3. Experimental results 1) Single-node scenario: Compared with the traditional fixed-height method, latency is reduced by 18.5%, energy consumption is reduced by 22.3%, and total system cost is reduced by 20.4%.

[0070] 2) Multi-node scenario: Although the latency increased slightly (+5.2%) due to the need for collaborative communication, the overall system energy consumption was reduced by 25.1%, the task completion rate was increased by 15.8%, and the overall system cost was still better than the existing baseline method by 12.7%.

[0071] 3) Altitude adaptability test: In a dynamic altitude change environment, the algorithm shows good adaptability and can intelligently adjust the altitude strategy according to the energy status. When energy is sufficient, it chooses a lower altitude to reduce transmission delay, and when energy is insufficient, it chooses a higher altitude to reduce flight energy consumption.

[0072] 4) Convergence performance: The algorithm basically converges after about 300 rounds, with good training stability and no violent oscillations, and is robust to different initial conditions.

[0073] In summary, by employing the technical solution described above, and through the introduction of a dynamic altitude perception and joint optimization mechanism, three major beneficial effects are achieved: First, the system can intelligently adjust the UAV altitude based on real-time channel conditions and energy status, significantly reducing flight energy consumption while ensuring communication quality, thereby reducing the total system cost by 15-25%. Second, the collaborative offloading and trajectory planning based on multi-agent reinforcement learning improves the efficiency of multi-UAV collaboration, increasing task processing rate by 20-30%. Finally, the proposed algorithm exhibits strong adaptability in real dynamic environments, with fast convergence speed and good robustness, making it particularly suitable for complex terrains such as urban areas and mountains.

[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-UAV computational unloading method based on altitude perception, characterized in that, Includes the following steps: S1 Constructs a system model comprising multiple user devices and multiple drones equipped with mobile edge computing servers, wherein the flight altitude of the drones is configured to be dynamically adjusted within a preset continuous or discrete range; S2 Based on the system model, constructs a multi-dimensional state space including the dynamic flight altitude information of the drones, and defines a joint action space including the drone altitude adjustment action, wherein the joint action space includes at least the drone's selection of user devices, flight direction, flight distance, task offloading ratio, and altitude adjustment level; S3 Designs a multi-objective optimization reward function, wherein the reward function takes minimizing the total system latency and total energy consumption as its core objective, and integrates a virtual queue mechanism based on Lyapunov optimization theory to coordinately constrain long-term average energy consumption and task processing latency; Based on the state space, joint action space, and reward function, S4 uses a multi-agent reinforcement learning algorithm for distributed training to generate an optimization strategy. The optimization strategy is used to control each UAV to autonomously and collaboratively determine its flight altitude, flight trajectory, and task offloading decision for user equipment in each decision time slot.

2. The multi-UAV computational unloading method based on altitude perception according to claim 1, characterized in that, The system model specifically characterizes the system behavior by integrating the following dynamic mathematical model: a node movement model, used to determine the position q(j+1) of the UAV in time slot j+1 based on the UAV's actions in time slot j, expressed as: Where v(j) is the flight speed, φ(j) is the flight direction angle, and Δt is the time slot length; the energy consumption model is used for the computer's energy consumption, and its total energy consumption E total At least including flight energy consumption E related to changes in flight distance and altitude. fly Energy consumption E related to edge computing comp And hovering energy consumption proportional to the current flight altitude h; a communication model to determine the transmission rate R(h) between the UAV and the user equipment at altitude h, whose channel power gain g(h) is expressed as a function related to the horizontal distance d and altitude h: Where α0 is the reference channel gain; the delay model is used to calculate the total delay T of task processing. total It includes at least the task transmission delay T. trans Edge computing latency T edge and user equipment local computing latency T local The aforementioned models collectively serve as the quantitative basis for the multi-agent reinforcement learning algorithm to interact with the environment, evaluate the effects of actions, and drive policy optimization.

3. The multi-UAV computational unloading method based on altitude perception according to claim 1, characterized in that, The state space includes: the UAV's own state vector, the state vectors of all user devices, the state vectors of neighboring UAVs within communication range, and the system virtual queue state vector; wherein, the UAV's own state vector includes at least: current flight altitude, remaining battery energy, and horizontal plane position coordinates; the user device state vector for each user device includes at least: position coordinates, the amount of data for the computational task to be processed, and a binary flag indicating whether the communication link with the UAV is blocked; the neighboring UAV state vectors include at least: relative position coordinates and flight altitude; the system virtual queue state vector includes: an energy virtual queue reflecting the deviation between historical energy consumption and a preset average energy consumption budget, and a delay virtual queue reflecting the deviation between historical task processing delay and a preset delay threshold.

4. The multi-UAV computational unloading method based on altitude perception according to claim 1, characterized in that, The altitude adjustment level corresponds to multiple discrete altitude values ​​within the preset flight altitude range.

5. The multi-UAV computational unloading method based on altitude perception according to claim 1, characterized in that, The calculation expression for the multi-objective optimization reward function r(t) in time slot t is as follows: Where V is an adjustable Lyapunov control parameter; r base (t) represents the total delay T actually generated by the system within time slot t. total (t) and total energy consumption E total (t) calculates the basic reward, and satisfies r base (t) = -[w1·T total (t) + w2·E total [(t)], where w1 and w2 are weighting coefficients; With Q D (t) represent the state values ​​of the energy virtual queue and the delay virtual queue in time slot t, respectively; E(t) and D(t) represent the actual energy consumption and task processing delay in time slot t, respectively; E avg D is the set average energy consumption budget; thresh The set delay threshold.

6. The multi-UAV computational offloading method based on altitude perception according to claim 5, characterized in that, The multi-objective optimization reward function also includes a highly perceptive reward term r. height When the drone's flight altitude h is within the preset optimal altitude range, r height It is positive; otherwise, it is negative or zero.

7. The multi-UAV computational unloading method based on altitude perception as described in claim 1 or 5, characterized in that, The generation and execution of the optimization strategy depend on the dynamic sensitivity of the communication model and energy consumption model to flight altitude h; the channel power gain g(h) and the derived transmission rate R(h) in the communication model change dynamically with altitude h, directly affecting the transmission delay T in the delay model. trans The total energy consumption E in the energy consumption model total It includes a hovering energy consumption component β*h that is proportional to altitude h, and a flight energy consumption component E_fly that is affected by the altitude change Δh; by jointly optimizing altitude h through the multi-agent reinforcement learning algorithm, a dynamic balance is achieved between the transmission performance determined by the communication model and the energy cost determined by the energy consumption model, thereby optimizing the multi-objective reward function.

8. The multi-UAV computational unloading method based on altitude perception according to claim 1, characterized in that, The multi-agent reinforcement learning algorithm is the Soft Actor-Critic algorithm using the maximum entropy reinforcement learning framework, and its optimization objective function J(π) is: Where π is the strategy, ρ π Let r(s) be the state-action access distribution under policy π, γ be the discount factor, and r(s) be the state-action access distribution. t ,a t ) represents the reward, α represents the temperature parameter, and ℋ(π(·|s) t )) represents the policy π in state s t The entropy below is used to encourage exploration.

9. The multi-UAV computational unloading method based on altitude perception according to claim 1, characterized in that, It also includes a multi-drone collaboration step: each drone periodically broadcasts its own status information within the communication range, the status information including at least: real-time location, flight altitude, remaining energy, and the identifier of the user equipment currently being served; collision avoidance is achieved by adding a collision penalty term to the multi-objective optimization reward function, and when the three-dimensional Euclidean distance between any two drones is less than a preset safe distance, a negative reward is applied to that drone.

10. A multi-UAV computational unloading system based on altitude perception, characterized in that, To implement the method as described in any one of claims 1 to 9, the system comprises: an environment modeling unit, configured to construct and maintain a model of the multi-UAV assisted mobile edge computing system and acquire system state information in real time; a decision generation unit, comprising a policy model pre-trained through multi-agent reinforcement learning, configured to generate joint action commands, including altitude adjustment, for each UAV based on the current state information provided by the environment modeling unit; a policy training unit, configured to execute the training process of the multi-agent reinforcement learning algorithm, including constructing the state space, joint action space, and multi-objective optimization reward function, and iteratively updating the policy model parameters using interactive data until convergence; and a collaborative management unit, configured to handle information exchange between multiple UAVs and perform conflict detection and coordination according to preset rules.

Images