A distributed computing offloading method in a multi-unmanned aerial vehicle space-air-ground integrated network

Abstract

The application relates to the technical field of multi-unmanned aerial vehicle cooperation computing offloading, and discloses a distributed computing offloading method in a multi-unmanned aerial vehicle space-air-ground integrated network. Each unmanned aerial vehicle acquires local observation information and inputs an Actor network to generate a mixed action containing mode selection, satellite selection and bandwidth allocation. According to the mode selection, local processing or offloading to a target satellite is carried out, and uplink resources are occupied according to the bandwidth allocation. In the centralized training stage, a Critic network estimates state values according to a global state, and network parameters are updated in combination with shared rewards. In the distributed execution stage, each unmanned aerial vehicle only deploys an Actor network to make an independent decision. Through the joint optimization of the centralized training and distributed execution architecture, the problem that joint coordination optimization is difficult to realize under dynamic satellite visibility and resource competition is solved, the consistency of offloading decisions and the feasibility of resource allocation are ensured, and the dynamic topology change of the space-air-ground integrated network is adapted.

Description

Technical Field

[0001] This invention relates to the field of multi-UAV collaborative computing offloading technology, and in particular to a distributed computing offloading method in a multi-UAV integrated air-ground network. Background Technology

[0002] With the widespread application of drone technology in areas such as wide-area perception, environmental monitoring, infrastructure inspection, emergency response, and data collection, mission-driven drones continuously generate computationally intensive and latency-sensitive workloads when performing tasks such as airborne sensing, imaging, and video analysis. However, drones are limited by their limited onboard computing resources and battery capacity, making it difficult to meet the stringent requirements for processing latency and computing power for these tasks solely through local execution. Mobile edge computing (MEC) can effectively reduce the computational burden on drones by deploying communication and computing resources at the network edge, providing a feasible technical path for task offloading. However, traditional terrestrial MEC architectures heavily rely on fixed ground infrastructure and stable ground connections. In scenarios such as at sea, in mountainous areas, in remote areas, or in disaster-stricken areas, ground communication networks are often sparse or even completely unavailable, making it impossible for terrestrial MEC to provide reliable service guarantees for drones.

[0003] To overcome the computing service bottlenecks in scenarios with limited ground infrastructure, Space-Air-Ground Integrated Networks (SAGINs) have emerged. SAGINs integrate three layers of network resources—drones, low Earth Orbit (LEO) satellites, and ground base stations—enabling drone-generated computing tasks to access distributed communication and computing resources beyond ground coverage. This provides a new solution for drone mission processing in remote and infrastructure-deficient areas. Under this architecture, drones can offload tasks to satellite edge nodes or ground base stations via satellite relay. Current research has explored computation offloading and resource management in drone-assisted MEC systems. For example, some studies have constructed a three-layer air-ground computing architecture for disaster relief scenarios, using game theory and evolutionary computation methods to optimize UAV task offloading and resource allocation; other studies have adopted deep reinforcement learning frameworks to jointly optimize task offloading, resource allocation, and UAV trajectories for scenarios with heterogeneous task priorities; still other studies have explored satellite-assisted offloading strategies, achieving air-space collaborative edge computing through Lyapunov optimization and delayed learning, or introducing collaborative multi-agent reinforcement learning for digital twin satellite edge networks to optimize satellite-layer task offloading. Furthermore, the computational offloading and resource allocation problems in LEO satellite-ground integrated networks have also attracted attention, with related research developing a delayed-aware deep reinforcement learning framework and a multi-level feedback queue mechanism for solving these problems.

[0004] However, existing solutions still have the following shortcomings. First, the low orbital altitude and high speed of LEO satellites cause the visible satellite set of each UAV to change constantly over time, resulting in time-varying connections and frequent satellite switching. Existing offloading strategies fail to effectively coordinate computational offloading and satellite association decisions under dynamic visibility constraints, and also lack modeling and optimization of switching signaling overhead. Second, multiple UAVs compete for limited satellite communication bandwidth and computing resources when performing task offloading. The offloading decisions of each UAV are strongly coupled through resource sharing. Most existing solutions generate decisions independently for each UAV, failing to fully characterize the interaction relationships and resource competition dynamics among multiple UAVs. In resource-constrained scenarios, this can easily lead to inconsistent or infeasible decisions, failing to guarantee the global effectiveness of the offloading strategy. Third, efficient task orchestration in SAGIN requires joint coordination of three decision dimensions: execution mode selection, satellite association, and bandwidth allocation. Furthermore, bandwidth allocation is a continuous variable, making the problem to be solved a hybrid decision space containing discrete and continuous decisions, further increasing the optimization difficulty. Therefore, there is an urgent need for a solution that can achieve distributed collaborative computational offloading of multiple UAVs in a dynamic SAGIN environment while ensuring decision feasibility and resource allocation effectiveness. Summary of the Invention

[0005] The purpose of this invention is to provide a distributed computing offloading method in a multi-UAV integrated air-space-ground network, which solves the problem that existing technologies, under dynamic satellite visibility and multi-UAV resource competition conditions, are unable to jointly coordinate and optimize the selection of task execution mode, satellite association and bandwidth allocation, resulting in inconsistent offloading decisions, infeasible resource allocation and inability to adapt to dynamic network topology changes.

[0006] To achieve the above objectives, this invention provides a distributed computing offloading method in a multi-UAV integrated air-ground network, comprising the following steps: Each UAV acquires local observation information, which includes the private state of each UAV and the side information broadcast by the visible satellite in the current time slot; Each UAV inputs the local observation information into the Actor network, which generates a hybrid action that includes mode selection, satellite selection, and bandwidth allocation. If the mode is selected as local execution, the UAV processes the computing task of the current time slot locally; if the mode is selected as offload execution, the UAV offloads the computing task of the current time slot to the target satellite determined by the satellite selection, and occupies the uplink bandwidth resources of the target satellite according to the bandwidth allocation. During the centralized training phase, the centralized Critic network estimates the state value based on the global state, and updates the parameters of the Actor network and the centralized Critic network based on the shared reward obtained by each UAV after performing the hybrid action and the state value. During the distributed execution phase, each UAV deploys only the trained Actor network and independently generates hybrid actions based on its own local observation information.

[0007] The Actor network includes a mode selection branch, a satellite selection branch, and a bandwidth allocation branch; after each UAV inputs the local observation information into the Actor network, it sequentially performs the following steps: The mode selection branch calculates the unnormalized score of each execution mode based on the local observation information, and then samples the score after normalization by the Softmax function to determine the mode selection. When the mode is selected as unload execution, the satellite selection branch generates an unnormalized fractional vector on each candidate satellite based on the local observation information, and then samples it after normalization by the Softmax function to determine the satellite selection. The bandwidth allocation branch generates the shape parameters of the Beta distribution based on the local observation information, and samples the bandwidth allocation from the Beta distribution.

[0008] The execution modes include local execution, satellite edge execution, and ground base station execution. Local execution means that the UAV processes the computing task on its onboard computing resources. Satellite edge execution means that the UAV offloads the computing task to a target satellite selected by the satellite selection. Ground base station execution means that the UAV forwards the computing task to a ground base station sequentially via the target satellite selected by the satellite selection. Each UAV selects one of the three execution modes to execute its computing task in the current time slot.

[0009] Specifically, after generating the unnormalized fractional vector and before performing Softmax normalization, the satellite selection branch also performs the following steps: Obtain the set of visible satellites for each UAV in the current time slot. The set of visible satellites is a collection of satellites whose elevation angle relative to each UAV in the current time slot is greater than a preset threshold. For satellites in the set of visible satellites, the corresponding unnormalized scores are kept unchanged; for satellites not in the set of visible satellites, the corresponding unnormalized scores are set to negative infinity, so that the selection probability of invisible satellites after normalization by the Softmax function is zero.

[0010] Before each UAV occupies the uplink bandwidth resources of the target satellite according to the bandwidth allocation, a bandwidth projection step is also performed: For each satellite, obtain the raw bandwidth requests of all drones associated with that satellite in the current time slot; The original bandwidth request is projected into a final bandwidth allocation ratio in a normalized manner. The final bandwidth allocation ratio is the ratio of the original bandwidth request to the sum of the original bandwidth requests of all UAVs, such that the sum of the final bandwidth allocation ratios of all associated UAVs on the satellite is not greater than one.

[0011] The bandwidth allocation branch uses a Beta strategy to generate the probability distribution of the bandwidth allocation, and the sampled values ​​of the bandwidth allocation are located in the interval [0,1].

[0012] The centralized Critic network is an attention-based centralized Critic network, and the attention-based centralized Critic network estimates the state values, specifically including: Extract the features of each UAV as the query vector, and extract the features of each satellite as the key vector and value vector; For each UAV, the attention weight between the query vector and the key vector of the UAV's currently visible satellites is calculated. The value vectors of each visible satellite are then weighted and summed according to the attention weight to obtain the attention-enhanced representation of the UAV. The attention-enhanced representations of all drones are concatenated and input into the value network, which outputs the state value estimate.

[0013] The private state includes the drone's current location, remaining battery power, data volume and workload of the current time slot calculation task, and the satellite identifier associated with the previous time slot; the side information includes the normalized number of visible satellites, the maximum visible channel gain, and the minimum visible satellite load indication.

[0014] The global state includes the location of all drones, the location of all satellites, the remaining battery power of each drone, the current mission status of each drone, and the drone-satellite association status of the previous time slot.

[0015] Specifically, updating the parameters of the Actor network and the centralized Critic network includes: Calculate the generalized advantage estimate based on the shared reward and the state value; The importance ratio is calculated based on the ratio of the joint probability density of each UAV performing the hybrid action to the joint probability density under the old strategy. The Actor network is updated by maximizing the pruned policy ratio objective function, and the centralized Critic network is updated by minimizing the mean squared error between the state value and the reward objective.

[0016] This invention discloses a distributed computing offloading method in a multi-UAV integrated air-space-ground network. By constructing a distributed decision-making framework based on local observation information, each UAV inputs its private state and visible satellite edge information into the Actor network, generating an integrated hybrid action that includes execution mode selection, target satellite association, and bandwidth allocation. During execution, the action is processed locally or offloaded to the associated satellite based on the selected mode, and bandwidth projection ensures the feasibility of resource allocation. Simultaneously, an architecture separating centralized training and distributed execution is adopted. During the training phase, a centralized Critic network combines global state and shared reward evaluation to update the policy network parameters. During the execution phase, each UAV independently completes its decision-making based solely on its own local observation information. This invention jointly optimizes the three decision dimensions of execution mode selection, satellite association, and bandwidth allocation, and leverages the modeling of inter-UAV interactions and resource competition during the centralized training phase to ensure consistency in UAV decisions and global feasibility of resource allocation in a distributed execution environment. It effectively solves the problem of difficulty in achieving joint coordinated optimization under dynamic satellite visibility constraints and resource competition conditions in existing technologies. It can adapt to dynamic topology changes in integrated air-space-ground networks and maintain stable latency-energy performance even under high UAV loads and limited satellite resources. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0018] Figure 1 This is a schematic diagram of the overall architecture of the distributed computing offloading method in the multi-UAV air-ground integrated network of the present invention.

[0019] Figure 2 This is a comparison chart of the training reward convergence curves based on the learning scheme under the default simulation settings of this invention.

[0020] Figure 3 This is a visualization of the decision-making results of the collaborative learning strategy of this invention over a long evaluation range.

[0021] Figure 4 This is a comparison diagram of the time delay-energy consumption trade-off under different time delay weights in this invention.

[0022] Figure 5 This is a comparison chart of the scalability of average task latency and average drone energy consumption for different schemes under different numbers of drones according to the present invention.

[0023] Figure 6 This is a comparison chart of the scalability of average mission latency and average UAV energy consumption for different schemes under different numbers of satellites according to the present invention.

[0024] Figure 7This is a flowchart of the steps of the distributed computing offloading method in the multi-UAV air-ground integrated network of the present invention. Detailed Implementation

[0025] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0026] Please refer to Figures 1 to 7 This invention provides a distributed computing offloading method in a multi-UAV air-ground integrated network, comprising the following steps: S101: Each UAV acquires local observation information, which includes the private state of each UAV and the side information of the visible satellite broadcast in the current time slot; S102: Each UAV inputs the local observation information into the Actor network, and the Actor network generates a hybrid action that includes mode selection, satellite selection, and bandwidth allocation; S103: If the mode is selected as local execution, the UAV processes the computing task of the current time slot locally; if the mode is selected as offload execution, the UAV offloads the computing task of the current time slot to the target satellite determined by the satellite selection, and occupies the uplink bandwidth resources of the target satellite according to the bandwidth allocation. S104: During the centralized training phase, the centralized Critic network estimates the state value based on the global state, and updates the parameters of the Actor network and the centralized Critic network based on the shared reward obtained by each UAV after performing the hybrid action and the state value. S105: During the distributed execution phase, each UAV deploys only the trained Actor network and independently generates hybrid actions based on its own local observation information.

[0027] Specifically, a space-air-ground integrated network (SAGIN) will be deployed in remote or infrastructure-constrained areas to provide computing and sensing services. In these areas, terrestrial communication networks are sparse or unreliable. This network will consist of… One drone , LEO satellites It consists of a ground station (GS) located at a fixed position. To maintain geometric consistency across the space, air, and ground domains, all 3D locations are represented in a geocentric-earth-fixed (ECEF) coordinate system. The system is in Evolution of discrete time slots The duration of each time slot is Drones In the time slot The instantaneous 3D position is And it evolves according to the Gauss-Markov movement model, i.e. ,in, It is the drone's velocity vector. It is the long-term average speed. It is a memory factor. This represents random perturbation. Each satellite Follow a predefined trajectory At the start of the time slot, each drone generates a computational task. ,in Indicates the size of the input data (such as sensor or video data). This represents the computational workload. These tasks originate from applications performed by the drone, such as environmental monitoring, video surveillance, or target tracking. Each task can be executed locally on the drone, offloaded to an associated LEO satellite, or routed to a ground station via satellite. The drone-satellite association is represented by a binary variable. The constraint is that each drone can be associated with at most one satellite per time slot: The unloading decision is represented by three mutually exclusive binary variables. in Indicates local execution. Indicates satellite edge execution, This indicates that the task is executed by the ground base station. Each task can only select one execution method: (1) Due to the rapid movement of LEO satellites, drones With satellite Reliable communication between them requires the satellite elevation angle observed from the UAV to exceed a minimum threshold. .set up Represents the Earth's radius. This indicates the satellite's orbital altitude. Because the drone's altitude is relative to... and Negligible, the satellite's coverage area can be approximated by the geocentric angle. Geometric analysis within the orbital plane yields the maximum geocentric angle corresponding to the elevation angle threshold. (2) In the time slot drones With satellite The instantaneous geocentric angle between them is calculated from their 3D positions. .when At that time, satellite In the time slot For drones It is evident that drones In the time slot The set of all visible satellites is defined as Frequent handovers between rapidly moving LEO satellites introduce non-negligible signaling overhead. The handover event is defined as a UAV... Associated with a different satellite than the previous time slot. The switching indicator is shown as follows. When a switch occurs (i.e. When applying a constant synchronization delay .

[0028] In SAGIN, mission offloading involves the drone-to-satellite (A2S) uplink and the satellite-to-ground (S2G) backlink.

[0029] UAV-Satellite Uplink: Setting Indicates time slot drones With satellite The channel gain between them is determined by Given, among which and These represent the transmit and receive antenna gains, respectively, and their values ​​depend on the specific transmitter-receiver pair. It is the signal wavelength. It is the aggregated atmospheric attenuation factor. To support multiple UAVs with heterogeneous flow requirements, an Orthogonal Frequency Division Multiple Access (OFDMA) scheme is adopted. Let... Indicates satellite Total uplink bandwidth, Indicates allocation to drones The ratio. Bandwidth allocation satisfies The corresponding uplink transmission rate is ,in It is a drone The transmission power, It is the noise power spectral density.

[0030] Satellite-to-Ground Base Station Backhaul: For tasks offloaded to ground base stations, data is further forwarded via the satellite-to-ground base station (S2G) backhaul link. The corresponding channel gain is determined by... Given. Assume dedicated backhaul bandwidth. Following the access-backhaul resource modeling in the terrestrial-satellite integrated network, a rate of [data missing] can be achieved. ,in It is a satellite The transmission power.

[0031] The processing cost of the computational task is determined by the offloading decision. This invention considers both latency and UAV energy consumption. The energy consumption indicator represents the UAV-side energy consumption related to the offloading decision, mainly including local computing energy consumption and uplink transmission energy consumption from the UAV to the satellite. Since satellites and ground base stations typically have more abundant energy supplies, and this invention focuses on offloading decisions under resource-constrained conditions at the UAV end, satellite computing energy consumption, ground base station computing energy consumption, and satellite-to-ground base station backhaul energy consumption are not included.

[0032] Local execution: When the drone Execute tasks locally ( When the latency depends on the onboard computing power, let's assume... This represents the CPU frequency (cycles / s). The local execution latency is... Following the CMOS dynamic power model, the corresponding energy consumption is... ,in It is the effective switching capacitor coefficient.

[0033] Satellite edge execution: When a mission is offloaded to a satellite ( The process includes A2S transmission and satellite-side calculations. The uplink transmission delay is... The corresponding transmission energy consumption is .set up Indicates satellite Total computing capacity. Due to competition among multiple drones, available resources are shared among all unloading drones, resulting in a load of [missing information]. The resulting execution delay is .

[0034] Ground base station execution: When the task is further offloaded to the ground base station ( When it occurs, it undergoes a two-hop transmission. After the A2S uplink, the task is forwarded via the S2G backlink, with a latency of [missing information]. .set up This indicates the computing capacity of the ground base station. The workload is shared among all ground-based tasks. The corresponding execution delay is .

[0035] Combining the above model and handover latency The total delay is expressed as Since the drone trajectory is mission-driven and not optimized by the offloading strategy, propulsion energy consumption is independent of the offloading decision and is therefore omitted. Thus, the drone energy consumption related to the offloading decision only includes local computing energy consumption and A2S uplink transmission energy consumption, i.e. .

[0036] Offloading tasks to ground base stations provides greater computing power, but may also introduce additional multi-hop transmission latency and higher uplink power consumption. Conversely, local execution reduces communication overhead, but is limited by onboard computing resources. Therefore, an efficient orchestration mechanism is needed to balance task latency and power consumption. To this end, this invention constructs a long-term joint optimization problem, which coordinates offloading decisions... Satellite-related variables and bandwidth allocation variables The optimization objective is to minimize the weighted sum of long-term average latency and UAV energy consumption. The corresponding optimization problem is formulated as follows:

[0037] . (3) (4) (5) (6) (7) (8) (9) (10) (11) here, and These represent the weighting factors related to latency and energy consumption, respectively. Maximum tolerable delay and predefined energy consumption scale This is used to normalize the time delay and energy consumption terms, making the two objectives comparable in magnitude. Furthermore, Indicates drone The maximum available energy for task computation and communication, Used only as an energy consumption normalization constant. Constraint (3) ensures that the mission delay does not exceed the maximum tolerable delay, and constraint (4) limits the UAV's energy consumption to within the available battery capacity. Constraint (5) guarantees that each mission is executed at exactly one computational layer. Constraint (6) limits each UAV to associating with at most one visible satellite per time slot, where the visibility condition is enforced by constraint (7). Constraint (8) ensures that mission offloading can only occur when an A2S communication link is established. Finally, constraint (9) limits the total bandwidth allocated to each satellite. Due to the strong spatial and temporal coupling between the UAV, satellites, and communication resources, the problem... This is difficult to solve. Specifically, the rapid movement of LEO satellites constantly alters the visible satellite set for each UAV, resulting in highly dynamic network connectivity and partial observability. Simultaneously, each UAV must jointly determine its mission execution mode, satellite associations, and bandwidth allocation, leading to a hybrid decision space containing both discrete and continuous variables. Furthermore, the decisions of different UAVs are inherently coupled through shared satellite bandwidth, shared computing resources, and latency caused by switching.

[0038] To address the above challenges, the present invention addresses the problem. The model is a cooperative distributed partially observable Markov decision process (Dec-POMDP), solved within a centralized training and distributed execution framework. During execution, each UAV makes decisions based solely on local observations via a lightweight BetaActor, avoiding excessive signaling overhead and improving the scalability of large-scale SAGINs. During training, an attention-based centralized Critic is introduced to capture interactions between UAVs and stabilize cooperative policy learning. To further improve practicality and training stability, a feasibility-aware action generation mechanism is explicitly introduced into the framework. Specifically, this invention employs visibility masking to eliminate infeasible satellite-associated actions, while using bandwidth projection to satisfy bandwidth allocation constraints without introducing additional penalty terms. Based on this, this invention develops a Multi-Agent Proximal Policy Optimization (MAPPO) algorithm to learn cooperative offloading policies in dynamic SAGIN environments.

[0039] This invention models the hierarchical SAGIN unloading process as having global state, local observation, mixed actions, controlled state transitions, shared rewards, and discount factors. The collaboration is based on Dec-POMDP. State transitions follow the principles of mobility, visibility, communication, computation, and battery dynamics outlined in Section II.

[0040] Global state: in time slot The global state is defined as follows:

[0041] in and Collect drone and satellite locations separately. It is the vector of remaining batteries. It is a set of task states. Record the drone-satellite correlation from the previous time slot. (Includes...) It is necessary because the switching delay depends on the previous serving satellite.

[0042] Local observation: Under distributed execution, drones Making decisions based on fixed-dimensional local observations

[0043] in It is in private state. Indicates the previously selected satellite index. Corresponds to local execution. Vector. Capture current visibility and recent load conditions, where It is a drone Normalized number of visible satellites, It is the maximum visible A2S channel gain. It is the minimum visible satellite payload indicator. Here, It is a normalized single-slot load indicator derived from the previous slot correlation matrix, and is considered as low-rate side information from visible satellite broadcasts. When When, set .

[0044] Hybrid Actions: Drones in each time slot Select Action ,in These represent local execution, satellite edge execution, and ground base station execution, respectively. It is the selected satellite index. This is the original bandwidth request. The offloading and associated variables in Section II are set... (for )and To restore. If you choose to execute locally, set deterministically. and To meet the per-satellite bandwidth constraint, the original bandwidth request is projected onto the feasible allocation as follows: (14) This ensures that for each satellite have .

[0045] Reward: To guide the policy learning towards latency-energy consumption targets and avoid timeout violations, the normalized timeout penalty is first defined as follows:

[0046] in Then the shared instantaneous reward is defined as follows:

[0047] here, This is the weight of the timeout violation penalty. When the task latency is satisfied, the penalty term is zero; otherwise, it increases linearly with the normalized timeout. Since bandwidth feasibility is handled by (14), no additional bandwidth violation penalty is introduced into the reward.

[0048] Based on the above Dec-POMDP modeling, this invention develops the MAPPO framework for SAGIN offloading, and its overall architecture is as follows: Figure 1 As shown. During distributed execution, each UAV feeds its local observations into a shared BetaActor to generate hybrid actions that include mode selection, satellite selection, and bandwidth allocation. During centralized training, the attention-based Critic takes the global state as input to estimate state values, while the collected trajectories are stored in an unfolded buffer and used for MAPPO-based policy and value updates.

[0049] The Actor is executed independently by each drone, while the Critic is only used to evaluate the global system state during training. Accordingly, this invention designs a lightweight BetaActor for distributed execution and an attention-based Critic for centralized value estimation.

[0050] BetaActor of the Actor Network: Let Represents the set of currently visible satellites The context vector obtained by aggregating satellite information summarizes the link quality, satellite load, and backhaul conditions of feasible offloading candidates. If ,set up Ensure that the Actor input is well defined. For drones. Actor will use a lightweight shared encoder to Compact local observations in (13) Fusion, generating potential features in Indicates a shared Actor encoder. Based on The Actor generates hybrid actions sequentially through three coordinating branches. This refers to execution mode selection, satellite selection, and bandwidth allocation.

[0051] For execution mode selection, the Actor first calculates the unnormalized score (logits) of the mode. ,in and These are the learnable weight matrix and bias vector of the mode selection head, respectively. The corresponding softmax policy is...

[0052] When the offloading mode is selected, the satellite selection branch is based on... Generate unnormalized fractional vectors on all candidate satellites To eliminate infeasible associations, a visibility mask is introduced before the softmax operation:

[0053] Masking score is defined as The satellite selection strategy is as follows:

[0054] This mask makes the probability of invisible satellites zero after softmax normalization, thus eliminating infeasible associations before action sampling. If In this case, the actor bypasses the unload branch and definitively falls back to local execution, i.e. , and .

[0055] For continuous bandwidth allocation branches, this invention employs a Beta strategy to model normalized bandwidth requests. (This is based on shared features.) Given the condition, the branch outputs two shape parameters of the Beta distribution:

[0056] in and It is a learnable weight vector. and This is the corresponding bias term for the Beta header. Use Ensure the shape parameters are positive and the conditions are good, and the finite support of the Beta distribution will naturally occur. No additional trimming or compression is required. In this way, BetaActor first determines whether to offload, then selects serving satellites if offloading is feasible, and finally allocates bounded bandwidth ratios.

[0057] Attention-Based Critic Networks: As the number of drones and satellites increases, fully connected centralized Critic networks can become inefficient because the value function must capture visibility constraints, bandwidth contention, and shared computational resources. To improve representation capabilities, this invention incorporates an attention mechanism into the Critic network. Centralized states are encoded as drone features. Satellite characteristics and global features For each drone , Used as a query, while the characteristics of its visible satellites are used as keys and values. When At that time, the attention weights and the satellite context after attention aggregation are calculated as follows:

[0058] in , and It is a trainable projection matrix. This indicates the dimension of the Critic feature used in scaling the dot product attention. If Then the context after attention aggregation is set as the zero vector, i.e. .therefore, Summary and Drones The most relevant satellite side information is retained, while remaining well-defined even when no visible satellites are currently present. Using attention-enhanced UAV representations, the centralized state-value function is expressed as follows: (twenty two) in All attention-enhanced drone features and global features are concatenated according to a fixed drone index order. Indicates by A parameterized value network. In this way, Critic can utilize centralized interactive information during the training phase, while the deployed Actors only need local observations and candidate satellite information.

[0059] Based on the aforementioned Dec-POMDP and Actor-Critic structures, this invention employs MAPPO for cooperative policy learning. In each iteration, all UAVs interact with the environment using a shared Actor, while a centralized Critic provides state value estimates for the collected deployment data. Let... This represents the Critic parameter vector used in the unfolded collection phase. The time difference residual estimate is... Generalized advantage estimation (GAE) is...

[0060] in It is the GAE coefficient.

[0061] set up and These represent the current and behavior Actor parameter vectors, respectively. For drones... time slot The PPO importance ratio of the mixed actions performed is defined as follows: For mixed action tuples The joint probability density is calculated by multiplying the mode probability in (17), the masked satellite selection probability in (19), and the Beta density in (20), while deterministic inactive components are omitted. The Actor objective function is...

[0062] in This is the PPO clipping factor. Critic is trained by minimizing the following objective function.

[0063] in This represents the reward target estimated from the collected unfolded data. In the implementation, entropy regularization is added to the Actor update to maintain sufficient exploration in the mixed action space. The overall training process is summarized as follows: 1: Initialize the Actor parameter vector and Critic parameter vector ; 2:for episode do; 3:for time slot do; 4: for each drone do; 5: Construction and and extract ; 6:if then; 7: Settings , and ; 8:else; 9: Sampling via (17) ; 10:if then; 11: Settings and ; 12:else; 13: Sampling via (19) and (20) and ; 14:end if; 15:end if; 16: Calculation in The joint logarithmic probability of the mixed actions executed below; 17:end for; 18: Evaluate the current state value ; 19: Request and perform joint actions via (14) projection bandwidth; 20: Observation and and in The storage includes the state, the action performed, the joint log probability, the reward, the value estimate, and the next state. 21:end for; 22: Use from estimate , and ; 23:for PPO epoch do; 24: In The action probabilities and state values ​​are recalculated. 25: Update by maximizing (24) Update by minimizing (25) ; 26:end for; 27:end for; After training, the Actor is deployed only on each drone. (In time slots) drones Constructed from its private state and visible satellite information and Then, the execution mode, service satellite, and original bandwidth request are generated sequentially according to (17), (19), and (20). The visibility mask ensures that only feasible satellites can be selected, and (14) ensures that bandwidth allocation is feasible when multiple UAVs request the same satellite. Therefore, the final strategy can be executed in a fully distributed manner while remaining consistent with the constraints. The computational complexity mainly comes from satellite scoring in the Actor and attention calculation in the Critic. Let and These represent the hidden dimensions of the Actor and Critic, respectively. Each episode and The training complexity on each PPO epoch is approximately Among them when In practice, the actual cost may be lower. During distributed execution, Critic is discarded, and each drone runs only one operation per time slot with a complexity of O(log n). The Actor.

[0064] Set a by Mission-driven drones The SAGIN-assisted aerial sensing system comprises one LEO satellite and one ground base station. Unmanned aerial vehicles (UAVs) operate at high altitudes. The LEO satellite constellation flies at km altitude and continuously generates sensing and computing tasks during flight. These tasks can be executed locally, offloaded to the satellite for edge processing, or further relayed to ground stations for execution. The LEO satellite constellation is generated using a Satellite Toolkit (STK), in which satellites are deployed at orbital altitudes of [missing information]. km. The UAV moves according to the Gauss-Markov movement model. The SAGIN system in The evolution of time slots, the duration of time slots During each time slot, each UAV generates one computational task. To simulate the demands of heterogeneous applications, both data-intensive and computationally intensive tasks were considered. Specifically, data-intensive tasks follow... Mbits and Gcycles, while computationally intensive tasks follow Mbits and Gcycles. The computing capacity settings for drones, satellites, and ground base stations are... GHz. Unless otherwise specified, the remaining parameters are configured as follows: visibility threshold is , , Joule Joule , The satellite uplink bandwidth and S2G backhaul bandwidth are set to... MHz, while the transmit power is W. Noise power spectral density is dBm / Hz, antenna gain is dBi, switching delay is For MAPPO training, the learning rate is set to... The discount factor is The generalized dominance estimation parameters are: The PPO cutting factor is PPO updates the epoch count to .

[0065] For performance evaluation, the proposed MAPPO framework is compared with several representative benchmarks. Two ablation variants are considered to evaluate the contribution of each component. MAPPO w / o Attention replaces the proposed attention-based centralized Critic with mean-pooled Critic while retaining the same Actor architecture, while MAPPO-Gaussian employs a sigmoid-compressed Gaussian strategy for continuous bandwidth allocation instead of the proposed BetaActor. Representative value-based DRL benchmarks include DQN, which follows the standard DQN architecture and its application in SAGIN computation offloading. The hybrid action space is discretized, and infeasible satellite selection is removed through visibility masking. Further comparisons are made with GA, which performs a budgeted online genetic search in the action space, following GA's use as a benchmark for joint user association and radio resource management in over-the-air networks; Greedy-Link, which performs selection of visible satellites with the strongest A2S channel gain at the satellite edge and falls back to local execution when no satellites are available; and Local-Only, which performs all tasks locally without offloading.

[0066] Figure 2 The training convergence of the learning-based scheme under default settings is illustrated. Since the reward is defined in (16) as a negative weighted latency-energy cost, a larger reward corresponds to a lower system cost. Solid lines represent smoothed training rewards, while semi-transparent trajectories show the corresponding unsmoothed rewards. All four schemes eventually improve the reward, but their learning speed and stability differ significantly. The proposed MAPPO enters the rapid improvement phase earlier than the other schemes, approaches a stable reward after approximately 1,500 episodes, and maintains the highest final reward. MAPPO with Attention improves later and converges to a slightly lower reward level. Since both schemes use the same BetaActor and differ mainly in the Critic structure, this comparison shows that the attention-based Critic improves learning efficiency by more effectively representing UAV-satellite interactions and shared resource competition during centralized training. The comparison between MAPPO with Attention and MAPPO-Gaussian provides a more controlled evaluation of the continuous action distribution, as both employ the same mean-pooled Critic. MAPPO-Gaussian exhibits a similar learning trend but converges to a lower reward, supporting bandwidth allocation using BetaActor. Specifically, the finite support of the Beta distribution directly matches the bounded bandwidth ratio, while the Gaussian policy requires additional action compression, and its parameterization does not perfectly match the bounded action space. DQN shows a longer initial learning phase: its reward remains almost constant and highly variable for approximately the first 2,000 episodes, only beginning to improve significantly afterward, and eventually stabilizing below all MAPPO variants. This is consistent with the difficulty of using value-based learning and ε-greedy exploration in a discretized mixed action space, where fine-grained bandwidth control increases the number of candidate action combinations. Overall, the convergence results show the complementary effects of the two proposed components: the attention-based Critic primarily improves the learning speed, while the BetaActor improves the quality of the convergence policy. Their combination enables the proposed MAPPO to achieve the fastest convergence and the highest final reward.

[0067] Figure 3 The study demonstrates collaborative strategies learned over a long evaluation period, including execution mode selection, satellite association, and bandwidth allocation. Overall, the learned strategies exhibit clear coupling among the three decision-making components and dynamically adapt to satellite visibility and resource availability, rather than relying on static rules. Figure 3(a) shows the temporal distribution of execution modes for all UAVs. Due to the discrete nature of mode selection, the stacking ratio fluctuates over time, with small changes in individual UAV decisions resulting in visible variations in the aggregate statistics. On average, satellite edge execution dominates with 69% of decisions, while local execution and terrestrial base station-based execution account for 20% and 11%, respectively. This demonstrates that satellite edge computing provides a favorable balance between communication overhead and computing power, while local execution is chosen when offloading is inefficient, and terrestrial base station execution is primarily triggered when satellite resources are congested or additional computing power is required despite high backhaul costs. Figure 3 (b) further illustrates the satellite association behavior of representative UAVs under time-varying visibility. The feasible satellite set evolves dynamically with LEO, and all association decisions remain within the visible area, confirming the effectiveness of visibility-aware action masking. Although multiple satellites may be visible simultaneously, serving satellites are adaptively selected based on channel quality and resource availability. The observed association variations are due to changes in A2S link conditions caused by UAV-satellite relative motion, while time-varying mission requirements and multi-UAV competition continuously affect the available bandwidth and computational load of each satellite. Therefore, the satellite providing the most favorable joint communication-computation conditions may vary between time slots. Satellites with longer visibility durations are selected more frequently, while satellites with short visibility windows are used only intermittently. Thus, visibility determines the action space, while actual association is jointly controlled by connectivity and resource conditions, not just visibility. Figure 3 (c) Describes the corresponding bandwidth allocation behavior. The allocated bandwidth ratio varies consistently with satellite association and execution mode. Specifically, local execution corresponds to zero A2S bandwidth allocation, while satellite edge and ground base station execution require adaptive and continuous bandwidth allocation over time. Furthermore, when a UAV associates with a less congested satellite, it receives a relatively higher bandwidth share compared to periods when satellite services compete more with UAVs, reflecting the impact of shared resource competition on allocation decisions.

[0068] Figure 4The latency-energy tradeoff relative to latency weights is illustrated. Overall, the results show a clear and consistent trend: increasing the learned policy from energy-oriented operations to latency-oriented operations leads to reduced latency but increased UAV energy consumption. This shows that the proposed framework can effectively adjust its offloading decisions according to different optimization preferences. When the value is small, the policy prioritizes energy saving and therefore tends to adopt more conservative offloading decisions, resulting in higher end-to-end latency. As the value increases, the framework gradually biases towards satellite edge and ground base station-assisted execution, which reduces latency but introduces additional communication and computational energy consumption. This behavior reflects the inherent coupling between communication and computation in multi-UAV systems supporting SAGIN. At a value of 0, the system achieves the lowest energy consumption while producing the highest latency, slightly exceeding 0 and leading to occasional timeout violations. As the value increases to a moderate value, latency decreases significantly while energy consumption remains at a reasonable level, resulting in a more balanced operating point. At a value of 0, the system achieves a stable tradeoff between latency and energy consumption with sufficient margin to meet latency constraints under varying workloads. Therefore, this setting is adopted as the default configuration in the remaining experiments.

[0069] Figure 5 The scalability of different schemes was evaluated as the number of UAVs increased, while the satellite constellation remained fixed. Overall, latency and energy consumption increased with the number of UAVs due to intensified competition for limited satellite communication and computing resources. However, the proposed MAPPO framework consistently achieved best performance across all system scales, demonstrating strong robustness under increasing network congestion. Among learning-based methods, the proposed MAPPO consistently outperformed all benchmarks in terms of latency and energy consumption. Its performance degradation was relatively mild with increasing UAV density, indicating that the proposed framework can effectively adapt to more congested network conditions. In contrast, MAPPO without Attention exhibited a more significant performance degradation under high-load scenarios, demonstrating that modeling UAV-satellite interactions becomes increasingly important as system scale grows. Compared to the Beta-based design, MAPPO-Gaussian showed a further degradation, highlighting the advantage of bounded bandwidth modeling for stable resource allocation. DQN suffered more severe scalability limitations due to the discretization of the hybrid action space. With increasing UAV numbers, the expanded combinatorial decision space led to inefficient exploration and suboptimal bandwidth allocation, resulting in consistently poor performance compared to MAPPO-based methods. In non-learning benchmarks, GA is limited by its finite search budget and becomes less efficient as the decision space expands. Greedy-Link relies only on instantaneous channel conditions and cannot coordinate long-term resource allocation across UAVs, while Local-Only avoids transmission overhead but consistently produces the highest latency across all system scales.

[0070] Figure 6The impact of varying satellite numbers on system performance was further evaluated. As the number of LEO satellites increased from 6 to 14, the average latency and average energy consumption of all schemes, except for Local-Only, decreased overall. This is because more satellites expand the visible candidate set for the UAVs, while providing more ample uplink bandwidth and satellite-side computing resources, thus alleviating communication and computing resource competition among multiple UAVs. Furthermore, it can be observed that as the number of satellites increases, the performance gap between different schemes generally shows a narrowing trend, especially among learning-based schemes. This is because with increased resource supply, bandwidth and computing resource competition is no longer as intense as in the few-satellite scenario, and ablation schemes and simple decision-making strategies are more likely to obtain feasible and superior offloading options. Therefore, the few-satellite scenario better reflects the algorithm's collaborative optimization capability under strong resource competition, while the performance gap between methods tends to narrow in the many-satellite scenario, but a significant gap still exists between the non-learning benchmark and the learning-based method. The proposed MAPPO maintains the lowest or near-lowest system cost across all satellite scales, demonstrating a significant advantage even with fewer satellites and stronger resource competition. This indicates that the attention-based Critic can more effectively model UAV-satellite interactions, while the BetaActor helps generate stable bounded bandwidth allocation. In contrast, MAPPO with / o Attention suffers a slight performance degradation in resource-constrained scenarios due to a lack of interaction modeling capabilities. MAPPO-Gaussian, affected by continuous action boundary modeling, exhibits higher latency and energy consumption than the Beta-based MAPPO scheme. DQN, due to its discretized mixed action space, struggles with fine-grained bandwidth control, further degrading its performance. For non-learning benchmarks, Greedy-Link relies solely on instantaneous link quality for association, easily neglecting long-term resource competition, resulting in higher energy consumption and latency. GA, constrained by a limited search budget, improves with increasing satellite count but still falls short of the performance of learning-based strategies. Local-Only does not utilize satellite resources, thus its results remain almost unchanged with the number of satellites, exhibiting lower communication energy consumption but the highest mission latency. The results validate the adaptability and scalability of the proposed framework under different satellite constellation scales.

[0071] This invention develops a joint task offloading, satellite association, and bandwidth allocation framework, and models the offloading process as a collaborative Dec-POMDP. To address the dynamic topology and distributed decision-making characteristics of SAGINs, an attention-enhanced MAPPO framework is proposed under a centralized training and distributed execution paradigm. The proposed framework jointly coordinates communication and computational decisions in a dynamic SAGIN environment, while ensuring the feasibility of satellite association and bandwidth allocation. Simulation results show that, compared with representative benchmark schemes, the proposed framework achieves faster convergence and a better latency-energy tradeoff, and the performance gain becomes more significant with increasing UAV density.

[0072] The above-disclosed embodiments are merely one or more preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments and equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. A distributed computing offloading method in a multi-UAV integrated air-space-ground network, characterized in that, Includes the following steps: Each UAV acquires local observation information, which includes the private state of each UAV and the side information broadcast by the visible satellite in the current time slot; Each UAV inputs the local observation information into the Actor network, which generates a hybrid action that includes mode selection, satellite selection, and bandwidth allocation. If the mode is selected as local execution, the UAV will process the computational task of the current time slot locally; If the mode is selected as unload execution, the UAV will unload the computing task of the current time slot to the target satellite determined by the satellite selection, and occupy the uplink bandwidth resources of the target satellite according to the bandwidth allocation. During the centralized training phase, the centralized Critic network estimates the state value based on the global state, and updates the parameters of the Actor network and the centralized Critic network based on the shared reward obtained by each UAV after performing the hybrid action and the state value. During the distributed execution phase, each UAV deploys only the trained Actor network and independently generates hybrid actions based on its own local observation information.

2. The distributed computing offloading method in a multi-UAV integrated air-ground network as described in claim 1, characterized in that, The Actor network includes a mode selection branch, a satellite selection branch, and a bandwidth allocation branch; after each UAV inputs the local observation information into the Actor network, it performs the following steps in sequence: The mode selection branch calculates the unnormalized score of each execution mode based on the local observation information, and then samples the score after normalization by the Softmax function to determine the mode selection. When the mode is selected as unload execution, the satellite selection branch generates an unnormalized fractional vector on each candidate satellite based on the local observation information, and then samples it after normalization by the Softmax function to determine the satellite selection. The bandwidth allocation branch generates the shape parameters of the Beta distribution based on the local observation information, and samples the bandwidth allocation from the Beta distribution.

3. The distributed computing offloading method in a multi-UAV integrated air-ground network as described in claim 2, characterized in that, The execution modes include local execution, satellite edge execution, and ground base station execution. Local execution means that the UAV processes the computing task on its onboard computing resources. Satellite edge execution means that the UAV offloads the computing task to the target satellite selected by the satellite selection. Ground base station execution means that the UAV forwards the computing task to the ground base station sequentially via the target satellite selected by the satellite selection. Each UAV selects one of the three execution modes to execute the computing task in its current time slot.

4. The distributed computing offloading method in a multi-UAV integrated air-ground network as described in claim 2, characterized in that, After generating the unnormalized fractional vector and before performing Softmax normalization, the satellite selection branch also performs the following steps: Obtain the set of visible satellites for each UAV in the current time slot. The set of visible satellites is a collection of satellites whose elevation angle relative to each UAV in the current time slot is greater than a preset threshold. For satellites in the set of visible satellites, the corresponding unnormalized scores are kept unchanged; for satellites not in the set of visible satellites, the corresponding unnormalized scores are set to negative infinity, so that the selection probability of invisible satellites after normalization by the Softmax function is zero.

5. The distributed computing offloading method in a multi-UAV integrated air-ground network as described in claim 1, characterized in that, Before each UAV occupies the uplink bandwidth resources of the target satellite according to the bandwidth allocation, a bandwidth projection step is also performed: For each satellite, obtain the raw bandwidth requests of all drones associated with that satellite in the current time slot; The original bandwidth request is projected into a final bandwidth allocation ratio in a normalized manner. The final bandwidth allocation ratio is the ratio of the original bandwidth request to the sum of the original bandwidth requests of all UAVs, such that the sum of the final bandwidth allocation ratios of all associated UAVs on the satellite is not greater than one.

6. The distributed computing offloading method in a multi-UAV integrated air-ground network as described in claim 2, characterized in that, The bandwidth allocation branch uses a Beta strategy to generate the probability distribution of the bandwidth allocation, and the sampled values ​​of the bandwidth allocation are located in the interval [0,1].

7. The distributed computing offloading method in a multi-UAV integrated air-space-ground network as described in claim 1, characterized in that, The centralized Critic network is an attention-based centralized Critic network. The attention-based centralized Critic network estimates the state values, specifically including: Extract the features of each UAV as the query vector, and extract the features of each satellite as the key vector and value vector; For each UAV, the attention weight between the query vector and the key vector of the UAV's currently visible satellites is calculated. The value vectors of each visible satellite are then weighted and summed according to the attention weight to obtain the attention-enhanced representation of the UAV. The attention-enhanced representations of all drones are concatenated and input into the value network, which outputs the state value estimate.

8. The distributed computing offloading method in a multi-UAV integrated air-space-ground network as described in claim 1, characterized in that, The private state includes the drone's current location, remaining battery power, data volume and workload of the current time slot calculation task, and the satellite identifier associated with the previous time slot; the side information includes the normalized number of visible satellites, the maximum visible channel gain, and the minimum visible satellite load indication.

9. The distributed computing offloading method in a multi-UAV integrated air-ground network as described in claim 1, characterized in that, The global status includes the location of all drones, the location of all satellites, the remaining battery power of each drone, the current mission status of each drone, and the drone-satellite association status of the previous time slot.

10. The distributed computing offloading method in a multi-UAV integrated air-ground network as described in claim 1, characterized in that, Updating the parameters of the Actor network and the centralized Critic network specifically includes: Calculate the generalized advantage estimate based on the shared reward and the state value; The importance ratio is calculated based on the ratio of the joint probability density of each UAV performing the hybrid action to the joint probability density under the old strategy. The Actor network is updated by maximizing the pruned policy ratio objective function, and the centralized Critic network is updated by minimizing the mean squared error between the state value and the reward objective.

Images