A method and system for optimizing resources of an air-space-ground integrated network in a post-disaster rescue scenario

Abstract

This invention discloses a resource collaborative optimization method and system for an integrated air-space-ground network in post-disaster relief scenarios. The method constructs a heterogeneous network architecture integrating space-based satellites, space-based UAVs, and ground-based base stations, along with a hierarchical priority-aware dynamic multi-agent near-end strategy optimization model, employing a centralized training and distributed execution architecture. It embeds a multi-branch attention mechanism, a priority-aware mechanism, and a dynamic weight adjustment mechanism into the Actor network, adaptively scaling feature weights according to real-time network conditions to adapt to dynamic changes in the post-disaster network. Each agent makes independent decisions based on local observation information, allocating relief tasks to satellites, UAVs, or ground base stations. This invention effectively improves task success rate and cache hit rate in post-disaster relief scenarios, significantly reduces system energy consumption and task processing latency, and possesses good self-organizing characteristics and scalability.

Description

Technical Field

[0001] This invention belongs to the technical fields of multi-agent deep reinforcement learning, edge computing and emergency communication, and specifically relates to a resource collaborative optimization method and system for an integrated air-space-ground network in a disaster relief scenario. Background Technology

[0002] In major disasters, communication infrastructure is often severely damaged, disrupting communication between the disaster area and the outside world, which seriously affects the efficiency of rescue command and disaster reporting. Post-disaster relief is characterized by time urgency, environmental complexity, and limited resources, requiring the rapid completion of tasks such as personnel search and rescue, medical treatment, and material distribution within the golden rescue time. The Space-Air-Ground Integrated Network (SAGIN) integrates three types of heterogeneous nodes: satellites (SAT), unmanned aerial vehicles (UAVs), and ground base stations (BS), enabling the rapid construction of an emergency communication system and providing communication assurance and computing support for post-disaster relief. This layered architecture achieves comprehensive communication assurance and optimized allocation of computing resources through a collaborative mechanism of wide-area coverage in the sky, flexible deployment in the air, and fixed support in the ground. In SAGIN, Multi-Agent Resource Collaborative Optimization (MARCO) is the core issue for improving network resource utilization efficiency. It requires the reasonable allocation of rescue tasks to different network nodes to reduce task processing latency, reduce system energy consumption, and improve task completion success rate.

[0003] However, due to the unique nature of post-disaster relief scenarios, multi-agent resource collaborative optimization faces significant challenges:

[0004] 1. The network status changes drastically after the disaster. Damage to communication infrastructure leads to network topology instability. Aftershocks, secondary disasters and other factors further exacerbate the network dynamics. Traditional methods are difficult to perceive and adapt to changes in network status in real time.

[0005] 2. The priorities of rescue missions differ significantly. Life-saving missions need to be completed in a very short time, while tasks such as disaster assessment can be appropriately postponed. Traditional methods are difficult to effectively distinguish and handle missions with different priorities.

[0006] 3. The complexity of multi-agent collaboration is high. Rescue drone swarms need to coordinate to perform multiple tasks such as search, rescue, and material delivery. The collaborative relationship between agents changes dynamically with the task type and network status. Traditional methods are difficult to efficiently model such complex collaborative relationships.

[0007] Traditional multi-agent deep reinforcement learning (MADRL) methods have significant limitations when handling such complex scenarios. Existing methods often employ fixed network architectures, making it difficult to adapt to drastic changes in network state after a disaster. Value function-based methods (such as the multi-agent deep deterministic policy gradient algorithm MADDPG) perform well in continuous action spaces, but suffer from poor training stability and are prone to getting trapped in local optima. Policy gradient-based methods (such as the multi-agent proximal policy optimization algorithm MAPPO) offer more stable training but lack the ability to perceive dynamic changes in task priority and network state. Existing research integrating attention mechanisms with MADRL often uses a single attention branch, making it difficult to comprehensively perceive the network state from multiple perspectives and failing to fully consider the impact of task priority differences on decision-making.

[0008] Therefore, there is an urgent need for a resource collaborative optimization method that can adapt to the dynamic changes in the network after a disaster, effectively distinguish task priorities, and achieve efficient collaboration among multiple agents.

[0009] A search revealed that Chinese invention patent CN120952386A discloses an asynchronous collaboration method in a distributed air-space-ground multi-agent system. This method is applied to task collaboration and allocation among heterogeneous air-space-ground multi-agent systems in disaster emergency scenarios, enabling distributed collaboration among various types of agents such as drones, satellites, and ground robots. It employs a hierarchical distributed architecture with regional autonomy, dividing the disaster site into multiple autonomous sub-regions, each configured with a regional command node. The core process includes: 1. Weighted Voronoi algorithm / manual division of autonomous sub-regions; 2. Boundary consistency handling mechanism for cross-region tasks; 3. Task allocation strategy based on asynchronous potential game theory, with agents asynchronously executing the best response and updating the task set; 4. Real-time regional state detection mechanism to dynamically respond to changes in agent and task states.

[0010] A search revealed that Chinese invention patent CN119676767A discloses a task offloading method for emergency rescue scenarios in an integrated air-space-ground network. This method is applied to task offloading and resource allocation in an integrated air-space-ground network for emergency rescue scenarios, including task offloading decisions for ground servers, drones, and low-Earth orbit satellites, and constructs a three-layer architecture (device layer, control layer, and management layer) for the integrated air-space-ground network. The core process is as follows: 1. Unify the description of task offloading features and establish offloading latency and node power consumption models for ground, drones, and satellites; 2. Establish a joint optimization problem model for latency and energy consumption with the goal of minimizing offloading latency and power consumption; 3. Determine task offloading priority based on the firefly algorithm, considering the maximum tolerable latency and task size; 4. Construct a reward mechanism for a deep reinforcement learning (TPODRL) model based on task priority, and solve the joint optimization problem using the DRL algorithm to obtain the task offloading scheme; 5. Simulate and verify the corrected model parameters to obtain the optimal model and the final offloading scheme.

[0011] Upon comparison, the present application differs from the aforementioned prior art documents in the following ways:

[0012] 1. Existing solutions are not adaptable to dynamic changes in network status. Although patent CN120952386A achieves distributed asynchronous collaboration, its region partitioning and task allocation strategies rely on a preset topology, making it difficult to respond in real time to drastic fluctuations in network links after a disaster; the optimization process of patent CN119676767A relies on the static priority pre-calculated by the firefly algorithm, lacking a feedback adjustment mechanism for real-time changes in network status.

[0013] 2. Existing solutions lack adequate characterization and protection mechanisms for task priorities. Although patent CN119676767A introduces the Firefly algorithm to quantify task priorities, its priorities are determined solely by two factors: task volume and maximum tolerable latency, without fully incorporating task types (such as the essential differences between life-saving and routine tasks); patent CN120952386A completely fails to address differentiated handling of task priorities, treating all tasks equally in collaborative allocation.

[0014] 3. Existing solutions have limitations in the information perception dimension of multi-agent collaborative decision-making. Existing methods mostly adopt a single-branch information extraction architecture, which makes it difficult to perceive environmental information from multiple perspectives such as global network state, collaborative interaction relationships, and access decision conditions simultaneously, thus limiting the decision-making quality of agents in complex rescue scenarios;

[0015] 4. Existing solutions have limited integration capabilities for heterogeneous network resources. Some studies focus on resource optimization for a single type of network, failing to fully integrate the complementary advantages of satellite wide-area coverage, flexible deployment of UAVs, and fixed computing at ground base stations, making it difficult to achieve globally optimal resource scheduling in post-disaster relief scenarios. Summary of the Invention

[0016] To address the aforementioned shortcomings in existing technologies, this invention provides a resource collaborative optimization method and system for an integrated air-space-ground network in post-disaster relief scenarios, aiming to solve the following technical problems:

[0017] 1. How to achieve real-time perception and adaptive adaptation to drastic changes in network status after a disaster;

[0018] 2. How to effectively differentiate between rescue missions of different urgency levels and ensure that high-priority missions are handled first;

[0019] 3. How to achieve efficient collaboration and resource scheduling among multiple agents in a dynamic network environment.

[0020] To achieve the above objectives, the present invention adopts the following technical solution:

[0021] A resource collaborative optimization method for an integrated air-space-ground network in a disaster relief scenario includes the following steps:

[0022] S1. Construct a heterogeneous network architecture that integrates space satellites, airborne UAVs, and ground base stations. Space satellites are used to provide wide-area coverage support, airborne UAVs are used for local task processing and multi-agent collaborative decision-making, and ground base stations are used to provide fixed computing support.

[0023] S2. Construct a hierarchical priority-aware dynamic multi-agent proximal policy optimization model based on a multi-agent proximal policy optimization framework; the hierarchical priority-aware dynamic multi-agent proximal policy optimization model adopts a centralized training and distributed execution architecture, where each agent has an independent Actor network and a shared Critic network;

[0024] S3. Embed a multi-branch attention mechanism in the Actor network, extract multi-view features of the network state in parallel through the sky layer attention branch, the air layer attention branch and the ground layer attention branch, and generate multi-view fused features through cross-layer attention fusion.

[0025] S4. Embed a priority-aware mechanism in the Actor network, generate priority embeddings according to the priority level of the rescue mission, and adjust the attention weights of the multi-view fusion features based on the priority embeddings.

[0026] S5. Embed a dynamic weight adjustment mechanism in the Actor network, calculate dynamic weight factors based on the real-time network state, and use dynamic weight factors to adaptively scale the multi-view fusion features.

[0027] S6. Based on local observation information and multi-view fusion features processed by multi-branch attention mechanism, priority perception mechanism and dynamic weight adjustment mechanism, each agent outputs task allocation decision through the Actor network, allocating rescue tasks to space satellites, airborne drones or ground base stations.

[0028] Furthermore, in step S1, the communication links between the space-level satellites, the space-level UAVs, and the ground-level base stations are allocated spectrum resources using the orthogonal frequency division multiple access method, specifically as follows:

[0029] Transmission latency of task data between heterogeneous nodes Determined by the characteristics of the physical link, as shown in the following formula:

[0030] ;

[0031] in, For the task The amount of data, For channel bandwidth, Signal-to-noise ratio;

[0032] Signal-to-noise ratio Calculate using the following formula:

[0033] ;

[0034] in, For transmission power, For antenna gain, For noise power spectral density, This refers to path loss.

[0035] The path loss Calculate using the following formula:

[0036] ;

[0037] in, For communication distance, and These are the additional loss factors in LoS and NLoS scenarios, respectively. and These are the path loss indices for LoS and NLoS scenarios, respectively. The probability of a Loss of Position (LoS) link occurring depends on the agent's real-time 3D pose and elevation angle. As shown in the following formula:

[0038] ;

[0039] in, , and To reflect the attenuation parameters of the disaster area's environmental characteristics, The angle of elevation.

[0040] Furthermore, in step S6, with the optimization objective of minimizing task processing latency and system energy consumption, a task allocation decision is output.

[0041] The aforementioned task processing delay Calculate using the following formula:

[0042] ;

[0043] in, For transmission delay, To calculate the delay, To delay queuing;

[0044] The calculation delay Based on the CPU cycle model, it is represented as follows:

[0045] ;

[0046] in, For the task Number of CPU cycles required For nodes CPU frequency;

[0047] The queuing delay Based on the classic queuing theory model, it is expressed as follows:

[0048] ;

[0049] in, For nodes Task arrival rate For nodes Service rate;

[0050] The system energy consumption Calculate using the following formula:

[0051] ;

[0052] in, For transmission energy consumption, To calculate energy consumption;

[0053] The transmission energy consumption Determined by the transmit power and transmission delay, as shown in the following formula:

[0054] ;

[0055] in, For transmission power, For transmission delay;

[0056] The calculated energy consumption Quantization is performed based on the processor power consumption model, as shown in the following formula:

[0057] ;

[0058] in, This is the CPU energy consumption coefficient, expressed in J / (cycles·Hz²).

[0059] Furthermore, each node in the heterogeneous network architecture adopts a task-type-based caching model, and a cache hit is triggered when the task type matches the cached content of the node.

[0060] Cache hit rate Calculate using the following formula:

[0061] ;

[0062] in, To cache the hit count, This represents the total number of tasks.

[0063] When the cache is hit, the computation latency is reduced to the original level. The multiple, as shown in the following formula:

[0064] ;

[0065] in, The computation latency when a cache hit occurs. This is a factor that reduces cache latency.

[0066] Furthermore, the local observation information includes its own status information, task status information, and network status information; the own status information includes the UAV's spatial location coordinates and remaining battery power; the task status information includes the amount of data to be processed, computational requirements, priority level, and task type; the network status information includes the normalized remaining computing power, normalized queue length, cache hit rate, and normalized distance from the UAV to each node for each heterogeneous node.

[0067] Furthermore, in the multi-branch attention mechanism described in step S3:

[0068] The top-level attention branch is used to capture global network state features, and its calculation is shown in the following formula:

[0069] ;

[0070] in, For the input feature vector, Here, MultiHeadAttn is the sky projection matrix, and MultiHeadAttn is the multi-head attention function.

[0071] The empty layer attention branch is used to capture collaborative interaction features, and its calculation is shown in the following formula:

[0072] ;

[0073] in, This is the projection matrix of the empty layer.

[0074] The formation attention branch is used to capture access decision features, and its calculation is shown in the following formula:

[0075] ;

[0076] in, This is the stratigraphic projection matrix.

[0077] Cross-layer attention is used to achieve information interaction and collaborative decision-making among the three branches, and its calculation is shown in the following formula.

[0078] ;

[0079] ;

[0080] in, , , To fuse the projection matrix, This is the fused feature vector.

[0081] Furthermore, in the priority perception mechanism described in step S4, the priority weight coefficient is determined according to the task priority level; the discrete task priority information is converted into a continuous feature vector as a priority embedding; as shown in equation (28).

[0082] ;

[0083] in, As a task priority, For the embedded dimension;

[0084] The priority embedding is combined with the multi-view fusion feature, and the feature weights are adjusted through an attention mechanism to obtain the priority-enhanced feature, as shown in the following formula;

[0085] ;

[0086] ;

[0087] in, To prioritize and enhance features, This is the feature vector adjusted for attention.

[0088] Furthermore, in the dynamic weight adjustment mechanism described in step S5:

[0089] The network state features are extracted using a state encoder, as shown in the following equation:

[0090] ;

[0091] in, For network state characteristics, This is the encoder weight matrix;

[0092] The extracted network state features are mapped using the Sigmoid function to generate dynamic weight factors between 0 and 1, as shown in the following formula:

[0093] ;

[0094] in, Here is the encoder weight matrix. For the Sigmoid function, For dynamic weighting factors;

[0095] The features are adaptively scaled according to the dynamic weighting factor to obtain the final output feature vector, as shown in the following formula:

[0096] ;

[0097] in, This is the final output feature vector.

[0098] Furthermore, the Actor network is trained according to the following steps:

[0099] a. Initialize the parameters of each agent's Actor network and the shared Critic network, and initialize the global shared experience replay buffer;

[0100] b. Each agent observes the local environment to obtain local observations and task priorities, and selects the action to be executed after processing by a multi-branch attention mechanism, a priority perception mechanism, and a dynamic weight adjustment mechanism.

[0101] c. After the joint action is executed, the environmental state transitions, all agents receive immediate rewards and joint observations at the next moment, and the joint experience tuple is stored in the experience replay buffer.

[0102] d. Randomly sample experience samples from the experience replay buffer, calculate the advantage function estimate using the generalized advantage estimation method, and calculate the discounted cumulative reward;

[0103] e. Update the Actor network parameters by maximizing the objective function, as shown in the following equation:

[0104] ;

[0105] in, For the trimming parameters, The advantage function is estimated as follows:

[0106] ;

[0107] In the formula, For GAE parameters, This refers to timing difference error.

[0108] d. Update the Critic network parameters by minimizing the value function error, as shown in the following equation:

[0109] ;

[0110] in, Accumulate rewards for discounts.

[0111] Furthermore, the method also includes a decentralized self-healing mechanism:

[0112] When any drone in the airspace becomes unable to provide service due to energy depletion or failure, the neighboring agent senses the failure event through the perception network state information and automatically redirects the tasks originally assigned to the failed drone to other available nodes based on the trained Actor network.

[0113] This invention also provides a resource collaborative optimization system for an integrated air-space-ground network in a disaster relief scenario, the system comprising:

[0114] A heterogeneous network architecture consisting of space satellites, space drones, and ground-based base stations;

[0115] And intelligent agents deployed on each drone, each intelligent agent making task allocation decisions using the method described in any one of claims 1 to 9.

[0116] The technical solution of the present invention has the following beneficial technical effects:

[0117] 1. This invention achieves multi-view perception of network state through a multi-branch attention mechanism.

[0118] The three parallel attention branches of the sky layer, the air layer, and the ground layer extract features from the three perspectives of global state, collaborative interaction, and access decision, respectively. The cross-layer fusion module realizes the interaction and integration of information from different perspectives, providing comprehensive environmental perception for decision-making and effectively solving the technical problem that the existing single attention branch is difficult to fully perceive the network state.

[0119] 2. This invention achieves differentiated protection for high-priority tasks through a priority-aware mechanism.

[0120] Dynamically adjusting attention and reward weights based on task priority ensures that high-priority tasks such as life rescue receive greater attention in feature extraction and decision optimization, effectively solving the technical problem that existing methods have difficulty in distinguishing and processing tasks of different priorities.

[0121] 3. This invention enhances the algorithm's adaptability to harsh network environments through a dynamic weight adjustment mechanism.

[0122] The feature weights are adaptively adjusted according to the current network state. When the network state is poor, the feature weights are reduced to avoid overfitting, and when the network state is good, the feature weights are increased to enhance decision accuracy. This effectively solves the technical problem that existing methods are difficult to adapt to drastic changes in network state after disasters.

[0123] 4. This invention achieves efficient distributed collaboration through the CTDE architecture.

[0124] During the centralized training phase, global information is used to learn collaborative strategies. During the distributed execution phase, each agent makes independent decisions based solely on local observations. This approach combines optimal collaboration with self-organizing execution, enabling it to adapt to extreme situations such as dynamic changes in network topology and node failures after disasters.

[0125] 5. This invention enhances system robustness through a decentralized self-healing mechanism.

[0126] When a drone fails, nearby intelligent agents automatically detect and redirect the task, without the need for intervention from a central node, which significantly improves the system's survivability and continuous operation capability in extreme environments. Attached Figure Description

[0127] Figure 1 This is a diagram of the SAGIN resource collaborative optimization architecture for disaster relief in an embodiment of the present invention;

[0128] Figure 2 This is a diagram of the HPD-MAPPO algorithm framework in an embodiment of the present invention;

[0129] Figure 3 This is a pseudocode diagram of the HPD-MAPPO algorithm in an embodiment of the present invention;

[0130] Figure 4 This is a comparison chart of training curves for seven unloading schemes in an embodiment of the present invention;

[0131] Figure 5 This is a performance comparison chart of nine unloading schemes in an embodiment of the present invention;

[0132] Figure 6This is a comparison chart of ablation experiment training curves in an embodiment of the present invention;

[0133] Figure 7 This is a comparison chart of ablation experiment performance in the embodiments of the present invention;

[0134] Figure 8 This is a comparison chart of training curves for the multi-branch attention mechanism in an embodiment of the present invention;

[0135] Figure 9 This is a performance comparison chart of the multi-branch attention mechanism in the embodiments of the present invention;

[0136] Figure 10 This is a comparison chart of training curves for the dynamic weight adjustment mechanism in this embodiment of the invention.

[0137] Figure 11 This is a comparison chart of the steady-state performance of the dynamic weight adjustment mechanism in this embodiment of the invention;

[0138] Figure 12 This is a comparison chart of training curves under different drone swarm sizes in an embodiment of the present invention;

[0139] Figure 13 This is a comparison chart of the final performance of different drone swarm sizes in the embodiments of the present invention;

[0140] Figure 14 This is a comparison chart of training curves under different learning rate configurations in an embodiment of the present invention;

[0141] Figure 15 This is a performance comparison chart under different learning rate configurations in an embodiment of the present invention. Detailed Implementation

[0142] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0143] Example 1:

[0144] This embodiment uses an earthquake disaster as an example. Post-disaster rescue scenarios are characterized by their suddenness, high timeliness, and complex environment. Communication infrastructure is severely damaged after an earthquake, and the rescue command center needs to quickly obtain information such as disaster area video, the location of the injured, and building damage to support rescue decisions. Traditional terrestrial communication networks are insufficient to meet the emergency communication needs of such scenarios. The post-disaster rescue SAGIN architecture constructed in this embodiment consists of a top-layer... Empty layer With strata Heterogeneous structures were combined to form a three-dimensional communication network covering the disaster area. (Sky Layer) Composed of low-Earth orbit SATs, it is responsible for providing wide-area coverage support for disaster areas, enabling remote communication between the rescue command center and the disaster area. (Air layer) Depend on The system comprises UAVs equipped with Mobile Edge Computing (MEC) capabilities, responsible for local task processing and multi-agent collaborative decision-making. It can be rapidly deployed to disaster areas to perform tasks such as personnel search and rescue and material delivery. (Stratigraphy) It encompasses both remaining base stations (BS) in the disaster area and temporarily deployed emergency nodes, providing fixed computing support. Under this architecture, rescue missions are categorized... Composed of attribute quadruples The descriptions represent the data scale, computational requirements, maximum latency tolerance, and priority level of each task. Task types encompass life-saving tasks (such as search and rescue of trapped personnel and vital sign monitoring) and routine processing tasks (such as disaster assessment and data collection), with different tasks having varying latency requirements and priority levels. The intelligent agent, by sensing the real-time network and resource status, allocates tasks to the aforementioned heterogeneous nodes, as shown in the specific architecture layout below. Figure 1 As shown.

[0145] Based on the above architecture, this embodiment includes the following model:

[0146] 1.1. Communication model

[0147] To address the challenges of damaged communication infrastructure and drastic network changes in post-disaster relief scenarios, this embodiment designs a SAGIN communication architecture based on orthogonal frequency division multiple access (OFDMA). This architecture effectively avoids co-channel interference caused by frequent signaling interactions during multi-agent collaboration by allocating orthogonal spectrum resources to satellite-to-ground, air-to-ground, and air-to-air links. In multi-agent collaboration scenarios, task data may need to be forwarded to the target node via multi-hop routing; this embodiment incorporates the accumulated transmission overhead of multi-hop paths into a unified latency model.

[0148] Transmission latency of task data between heterogeneous nodes It is determined by the characteristics of the physical link, as shown in the following formula.

[0149] ;

[0150] in, For the task The amount of data, For channel bandwidth, This refers to the signal-to-noise ratio.

[0151] Signal-to-noise ratio The calculation formula is as follows:

[0152] ;

[0153] in, For transmission power, For antenna gain, For noise power spectral density, This refers to path loss.

[0154] Considering the multipath effect caused by the complex environment of the disaster area on signal propagation, this embodiment uses line-of-sight (LoS) and non-line-of-sight (NLoS) probability models to characterize the propagation characteristics of air-to-ground links. The probability of LoS links occurring... According to the ITU-R P.1411 standard, its value depends on the agent's real-time three-dimensional pose and elevation angle. As shown in the following formula:

[0155] ;

[0156] in, , and To reflect the attenuation parameters of the disaster area's environmental characteristics, The angle of elevation.

[0157] System overall path loss It is represented as follows:

[0158] ;

[0159] in, For communication distance, and These are the additional loss factors in LoS and NLoS scenarios, respectively. and These represent the path loss indices for LoS and NLoS scenarios, respectively. This hybrid model can dynamically reflect the changes in link quality caused by obstruction during UAV maneuvers in disaster areas.

[0160] 1.2 Delay Model

[0161] Disaster relief missions are highly time-sensitive, with significant processing delays. It consists of three stages: data transmission, local computation, and queue waiting, as shown in the following formula:

[0162] ;

[0163] in, For transmission delay, To calculate the delay, This is to delay queuing time.

[0164] Calculate latency Depending on the remaining computing power resources of the target node, the model is based on the CPU cycle model, as shown in the following equation:

[0165] ;

[0166] in, For the task Number of CPU cycles required For nodes CPU frequency.

[0167] Given that a surge in tasks during disaster relief operations may cause node congestion, this embodiment uses a classic queuing theory model to analyze node congestion delay, as shown in the following equation:

[0168] ;

[0169] in, For nodes Task arrival rate For nodes Service rate.

[0170] Considering the dynamic characteristics of the post-disaster relief environment, this embodiment adopts a discrete-time stepping mechanism within the reinforcement learning framework, assuming that at each decision time... The system state remains stable within the instantaneous window, thus ensuring the applicability of the queuing theory model within that time slice. When the following conditions are met... hour( For the task The deadline is used to determine whether the mission was completed within the effective rescue time.

[0171] 1.3 Energy Consumption Model

[0172] In disaster relief scenarios, UAVs face limited power and difficulties in resupply, making energy consumption optimization crucial for the continuous operation of UAV clusters. (Task processing energy consumption) Defined as follows:

[0173] ;

[0174] in, For transmission energy consumption, To calculate energy consumption.

[0175] Transmission power consumption is determined by transmission power and transmission delay, as shown in the following formula:

[0176] ;

[0177] in, For transmission power, This refers to transmission delay.

[0178] Calculate energy consumption Quantization is performed based on the processor power consumption model, as shown in the following formula:

[0179] ;

[0180] in, This is the CPU energy consumption coefficient, expressed in J / (cycles·Hz²).

[0181] 1.4 Caching Model

[0182] In disaster relief scenarios, some task data (such as disaster area maps, medical guides, and rescue plans) can be cached by multiple nodes to reduce latency and energy consumption caused by repeated transmissions. This embodiment adopts a task-type-based caching strategy, where each node can cache data of several task types. When the task type matches the cached content of a node, computation latency can be significantly reduced. Cache hit rate. Defined as follows:

[0183] ;

[0184] in, To cache the hit count, This represents the total number of tasks.

[0185] When the cache is hit, the computation latency is reduced to the original level. The multiple, as shown in the following formula:

[0186] ;

[0187] in, The computation latency when a cache hit occurs. This is a factor that reduces cache latency.

[0188] 1.5 Problem Model

[0189] This embodiment models the multi-agent resource collaborative optimization problem in post-disaster relief as a constrained Markov decision process (CMDP), and learns the optimal task allocation strategy through reinforcement learning. To maximize cumulative reward benefits, as shown in the following formula:

[0190] ;

[0191] in, To accumulate reward earnings, This represents the total number of time steps. For a moment The reward.

[0192] The model consists of six tuples. Full definition. Global state space. Describe the network topology and resource distribution in the disaster area, and the local observation space. Characterizing intelligent agents The ability to perceive the surrounding network status and task priorities, and the action space. A discrete set of decisions containing the selection of task unloading targets. Let be the environmental state transition probability function. For the instantaneous reward function of each agent, This is a reward discount factor.

[0193] The state transition probability function is defined as follows:

[0194] ;

[0195] in, Representing the state space probability distribution on, For all intelligent agents at time Joint actions were taken. and Representing time respectively and The system's global state.

[0196] Based on the above modeling, the system needs to solve for the optimal cooperative strategy under multiple constraints such as latency, energy consumption, and success rate, as shown in the following equation:

[0197] ;

[0198] in, For the number of agents, , , These are the latency threshold, energy consumption threshold, and success rate threshold, respectively. This is an indicator function.

[0199] Example 2:

[0200] Based on the system architecture and model constructed in Example 1, this example provides a hierarchical priority-aware dynamic multi-agent proximal policy optimization algorithm (HPD-MAPPO). This algorithm embeds a multi-branch attention mechanism, a priority-aware mechanism, and a dynamic weight adjustment mechanism into the Actor network, achieving multi-view feature extraction of network states, high-priority task assurance, and adaptive adjustment to harsh post-disaster environments. The details are as follows:

[0201] 2.1 State Space

[0202] State space design must fully characterize the environmental features of post-disaster relief scenarios. For intelligent agents... Its local observation It includes three dimensions: self-state information, task state information, and network state information. Self-state information includes the UAV's location coordinates. With remaining power It is used to characterize the spatial location and endurance of an intelligent agent; task state information covers the amount of task data. Computational requirements Priority and task type This characterizes the resource requirements and urgency of the tasks to be processed; network state information includes the remaining computing power, queue length, cache status, and distance to the agent of each node, used to perceive the resource availability and access conditions of heterogeneous network nodes. The state vector is constructed as follows:

[0203] ;

[0204] in, For intelligent agents The local observation vector, As an observation dimension, For nodes Normalized residual computing power To normalize the queue length, For cache hit rate, For intelligent agents To the node Normalized distance, This represents the total number of nodes.

[0205] 2.2 Action Space

[0206] The action space is defined as a discrete task allocation decision. At any moment Choose to assign tasks to nodes ,action ,in This represents the total number of nodes (including local processing). The action space design is shown in the following formula:

[0207] ;

[0208] in, For intelligent agents The action.

[0209] 2.3 Reward Function

[0210] The reward function design needs to comprehensively consider four optimization objectives: latency, energy consumption, success rate, and caching. For intelligent agents... Its reward function Defined as follows:

[0211] ;

[0212] in, This is the priority weight coefficient. , , , These are the weighting coefficients for latency, energy consumption, success rate, and cache hit rate, respectively.

[0213] Priority weighting coefficient Determined based on task priority, as shown in the following formula:

[0214] ;

[0215] in, This represents the task priority level, with a possible value. These correspond to four priorities: emergency, high, medium, and low.

[0216] This design ensures that high-priority tasks (such as life-saving) receive higher weight in the reward function, guiding the algorithm to prioritize optimizing the performance of high-priority tasks.

[0217] 2.4 MAPPO Algorithm

[0218] Multi-Agent Proximal Policy Optimization (MAPPO) is a multi-agent reinforcement learning algorithm based on policy gradients, employing a centralized training and distributed execution architecture. Each agent... It has an independent Actor network and shared Critic network ,in For joint observation.

[0219] The Actor network updates its parameters by maximizing the objective function, as shown in the following equation:

[0220] ;

[0221] in, For trimming parameters, The advantage function is estimated using the generalized advantage estimation (GAE) method, as shown in the following equation:

[0222] ;

[0223] in, For GAE parameters, This refers to timing difference error.

[0224] The Critic network updates its parameters by minimizing the error of a value function, as shown in the following equation:

[0225] ;

[0226] in, Accumulate rewards for discounts.

[0227] MAPPO limits the policy update magnitude by pruning the objective function, ensuring training stability and making it suitable for multi-agent collaborative decision-making scenarios in both continuous and discrete action spaces.

[0228] 2.5 HPD-MAPPO Algorithm Framework

[0229] The HPD-MAPPO algorithm, based on MAPPO, introduces three core mechanisms: a multi-branch attention mechanism, a priority-aware mechanism, and a dynamic weight adjustment mechanism, enhancing the Actor network's perception and adaptability to post-disaster relief scenarios. The multi-branch attention mechanism extracts global network state, collaborative interaction, and access decision features through three parallel attention branches (sky layer, air layer, and ground layer), and integrates multi-perspective information through cross-layer attention fusion. The priority-aware mechanism dynamically adjusts attention weights according to task priority, ensuring that high-priority tasks such as life-saving are processed first. The dynamic weight adjustment mechanism adaptively adjusts feature weights according to network state, enhancing the algorithm's adaptability to harsh post-disaster network environments. The overall framework of HPD-MAPPO is as follows: Figure 2 As shown.

[0230] HPD-MAPPO deploys three mechanisms in the Actor network, achieving decentralized distributed decision-making based on the CTDE architecture. During the execution phase, each agent can independently make task allocation decisions based solely on local observations, without the need for central node coordination. This demonstrates strong self-organization and adaptability to dynamic changes in post-disaster network topology. Simultaneously, centralized training enables collaborative learning, allowing agents to maintain globally optimal collaborative strategies even during distributed execution. MAPPO's pruning objective function effectively limits the policy update magnitude, avoiding drastic fluctuations during training, making it suitable for the complex environments of post-disaster relief scenarios.

[0231] 2.5.1 Multi-branch attention mechanism

[0232] The multi-branch attention mechanism extracts features from different perspectives through three parallel attention branches and integrates information through cross-layer attention fusion. This mechanism comprises three independent attention branches and a cross-layer fusion module, with each branch learning its specific feature representation through an independent learnable projection layer.

[0233] SkyAttention is responsible for capturing global network state features, focusing on information such as satellite coverage and global resource distribution. Its calculation is shown in the following formula:

[0234] ;

[0235] in, For the input feature vector, is the sky projection matrix, and MultiHeadAttn is the multi-head attention function.

[0236] AirAttention is responsible for capturing collaborative interaction features, focusing on the collaborative relationships and task allocation coordination between agents. Its calculation is shown in the following formula:

[0237] ;

[0238] in, This is the projection matrix of the empty layer.

[0239] Ground Attention is responsible for capturing access decision features, focusing on information such as access selection by ground base stations and drones, and link quality. Its calculation is shown in the following formula:

[0240] ;

[0241] in, This is the stratigraphic projection matrix.

[0242] Cross-Layer Attention enables information exchange and collaborative decision-making among the three branches, and its calculation is shown in the following formula:

[0243] ;

[0244] ;

[0245] in, , , To fuse the projection matrix, This is the fused feature vector.

[0246] The multi-branch attention mechanism extracts features from three perspectives—global state, collaborative interaction, and access decision—through three parallel branches. The cross-layer fusion module enables the interaction and integration of information from different perspectives, providing comprehensive environmental perception for subsequent decision-making.

[0247] 2.5.2 Priority Awareness Mechanism

[0248] The priority-aware mechanism dynamically adjusts the decision-making strategy based on task priority through task priority embedding and priority attention weight adjustment. Task priority embedding transforms discrete priority information into a continuous feature vector, as shown in the following equation:

[0249] ;

[0250] in, As a task priority, For the embedded dimension.

[0251] Priority attention weight adjustment combines priority embedding with fused features and adjusts feature weights through an attention mechanism, as shown in the following equation:

[0252] ;

[0253] ;

[0254] in, To prioritize and enhance features, This is the feature vector adjusted for attention.

[0255] The priority-aware mechanism ensures that high-priority tasks (such as life-saving) receive more attention during feature extraction, guiding the algorithm to prioritize optimizing the performance metrics of high-priority tasks.

[0256] 2.5.3 Dynamic Weight Adjustment Mechanism

[0257] The dynamic weight adjustment mechanism adaptively adjusts feature weights based on network state, enhancing the algorithm's adaptability to harsh post-disaster network environments. Network state encoding extracts network state features through a state encoder, as shown in the following equation:

[0258] ;

[0259] in, For network state characteristics, This is the encoder weight matrix.

[0260] The dynamic weighting factor is calculated through the state encoder, as shown in the following formula:

[0261] ;

[0262] in, Here is the encoder weight matrix. For the Sigmoid function, This is a dynamic weighting factor.

[0263] Adaptive feature scaling adaptively scales features based on dynamic weight factors, as shown in the following equation:

[0264] ;

[0265] in, This is the final output feature vector. The dynamic weight adjustment mechanism can adaptively adjust the feature weights according to the current network state (such as link quality, node load, and network congestion). When the network state is poor, the feature weights are reduced to avoid overfitting; when the network state is good, the feature weights are increased to enhance decision accuracy.

[0266] 2.6 Training and Optimization of Resource Coordination Strategies

[0267] like Figure 3 As shown, the Actor network in this embodiment is trained according to the following steps:

[0268] a. Initialize the parameters of each agent's Actor network and the shared Critic network, and initialize the global shared experience replay buffer;

[0269] b. Each agent observes the local environment to obtain local observations and task priorities, and selects the action to be executed after processing by a multi-branch attention mechanism, a priority perception mechanism, and a dynamic weight adjustment mechanism.

[0270] c. After the joint action is executed, the environmental state transitions, all agents receive immediate rewards and joint observations at the next moment, and the joint experience tuple is stored in the experience replay buffer.

[0271] d. Randomly sample experience samples from the experience replay buffer, calculate the advantage function estimate using the generalized advantage estimation method, and calculate the discounted cumulative reward;

[0272] e. Update the Actor network parameters by maximizing the objective function;

[0273] d. Update the Critic network parameters by minimizing the error of the value function.

[0274] The HPD-MAPPO resource collaborative optimization scheme proposed in this embodiment integrates a multi-branch attention mechanism, a priority-aware mechanism, and a dynamic weight adjustment mechanism into the Actor network. During the training and execution phases, the multi-branch attention mechanism extracts features from the sky, airspace, and ground layers and fuses them into multi-view features. The priority-aware mechanism adjusts attention weights according to task priority, and the dynamic weight adjustment mechanism adaptively adjusts feature weights based on network state. For abnormal situations such as node failure, HPD-MAPPO implements a decentralized self-healing mechanism. When a UAV runs out of energy or fails, neighboring agents perceive the failure event by observing network state information and automatically redirect the task to other available nodes based on the trained policy network. The entire process requires no intervention from a central node. The overall learnable parameters of this algorithm cover the weights of the three mechanisms and the Actor-Critic network parameters. By leveraging the CTDE architecture, the complex feature processing is placed in the offline training phase. During online execution, the agent only needs to run a lightweight Actor network, thus ensuring low-latency decision-making for post-disaster relief resources. The time complexity of the training process is O(n log n). The total complexity is .

[0275] Example 3:

[0276] This embodiment is based on the experiments and analysis conducted in Embodiment 1 and Embodiment 2.

[0277] 3.1 Experimental Setup

[0278] To verify the effectiveness of the HPD-MAPPO algorithm in disaster relief scenarios, this embodiment designed a series of comparative experiments, ablation experiments, parameter sensitivity analysis, and scalability analysis.

[0279] The experimental environment configuration for this embodiment is shown in Table 1, and the simulation parameter settings are shown in Table 2.

[0280] Table 1 Experimental Environment Configuration

[0281]

[0282] Table 2: Simulation Parameter Settings

[0283]

[0284]

[0285] 3.2 Baseline Scheme

[0286] To comprehensively evaluate the performance of the HPD-MAPPO algorithm, this embodiment selects the following baseline schemes for comparative analysis.

[0287] The Random approach randomly selects and assigns tasks to targets at each time step, without environmental awareness or strategy optimization.

[0288] The Greedy approach selects the optimal task allocation target at each time step, making decisions based on a greedy strategy.

[0289] The DDPG scheme is a single-agent deep deterministic policy gradient algorithm, in which each agent independently learns the optimal policy.

[0290] SAC scheme: Soft actor-critic algorithm, based on the maximum entropy reinforcement learning framework, suitable for continuous action spaces.

[0291] The MADDPG scheme is a multi-agent deep deterministic policy gradient algorithm that adopts the CTDE architecture and is suitable for multi-agent collaborative decision-making scenarios in continuous action spaces.

[0292] MATD3 scheme: Multi-agent dual-delay deep deterministic policy gradient algorithm, which improves training stability through dual-Q network and delayed updates.

[0293] PPO scheme: a near-end policy optimization algorithm that limits the policy update magnitude based on a pruning objective function.

[0294] The MAPPO scheme: a multi-agent proximal policy optimization algorithm, adopts the CTDE architecture and is the basic framework of this embodiment.

[0295] 3.3 Comparative Experiment

[0296] This section comprehensively evaluates the superiority of the HPD-MAPPO algorithm compared to various mainstream baseline schemes through two dimensions: training curves and steady-state performance, and verifies its effectiveness and advancement in the SAGIN resource collaborative optimization scenario for post-disaster relief.

[0297] 3.3.1 Comparative Analysis of Training Curves

[0298] like Figure 4 As shown in (a), the average latency of each scheme exhibits different convergence trends with each training epoch. The latency convergence process of the single-agent schemes (DDPG, SAC, PPO) is slow and fluctuates drastically. HPD-MAPPO, by fusing attention mechanisms and dynamic weight adjustments, shows a faster descent slope in the early stages of training, ultimately converging the average latency to a lower level. Figure 4 As shown in (b), the average energy consumption convergence curves of each scheme are basically consistent with the latency trend. The energy consumption fluctuation of the single-agent scheme is relatively large, and the energy consumption curve of HPD-MAPPO enters a stable low-level operating range in the middle of training, with the final convergence value lower than all baseline schemes. Figure 4As shown in (c), the task success rates of each scheme gradually increase and stabilize with each training round. The success rate convergence values ​​of the single-agent schemes are generally low. HPD-MAPPO, through the synergistic effect of attention mechanism and dynamic weight adjustment, ultimately leads in the convergence value of the task success rate index. Figure 4 As shown in (d), HPD-MAPPO also has advantages in cache optimization. The cache hit rate of the multi-agent scheme is generally higher than that of the single-agent scheme, and HPD-MAPPO further improves the convergence level of the cache hit rate.

[0299] 3.3.2 Performance Comparison Analysis

[0300] like Figure 5 As shown in (a), HPD-MAPPO reduces the average latency to 0.73s, a 73.55% reduction compared to MAPPO (2.76s), and its latency advantage is even more pronounced compared to MADDPG (5.81s) and MATD3 (10.43s). Figure 5 As shown in (b), HPD-MAPPO reduced the average energy consumption from 6.84 J of MAPPO to 3.01 J, a reduction of 55.99%, which is more significant compared to the energy consumption reduction of MADDPG (12.33 J) and MATD3 (15.43 J). Figure 5 As shown in (c), HPD-MAPPO achieved an average task success rate of 83.14%, a 17.28% improvement compared to the basic framework MAPPO (70.89%), and improvements of 55.28% and 61.28% compared to MADDPG (53.54%) and MATD3 (51.55%), respectively. Figure 5 As shown in (d), the cache hit rate of HPD-MAPPO increased to 69.72%, which is 15.08% higher than that of MAPPO (60.58%). It also has a significant advantage over the cache hit rates of MADDPG (53.73%) and MATD3 (55.47%).

[0301] Under the four evaluation dimensions, the steady-state performance of the HPD-MAPPO algorithm is superior to all baseline schemes, which fully demonstrates that the proposed scheme can greatly optimize the system resource allocation efficiency while ensuring the quality of rescue missions.

[0302] 3.4 Ablation Test

[0303] This section quantitatively analyzes the independent contributions and synergistic effects of the multi-branch attention mechanism and the dynamic weight adjustment mechanism on the overall performance, revealing the mechanism by which the synergistic effect of the mechanisms improves the algorithm's performance.

[0304] 3.4.1 Comparative Analysis of Training Curves

[0305] like Figure 6As shown in (a), the delay curve of HPD-MAPPO converges the fastest and eventually stabilizes at the lowest level. The steady-state delay value of the configuration that only retains the dynamic weight adjustment mechanism is higher than that of the full version, and the convergence speed of the configuration that only retains the multi-branch attention mechanism is relatively slower. Figure 6 As shown in (b), the energy consumption curve of HPD-MAPPO enters a stable low-level operating range in the middle of training, and the curve oscillation amplitude is minimal. Removing the dynamic weight mechanism will lead to a higher convergence value of the energy consumption curve. Figure 6 As shown in (c), the success rate curve of HPD-MAPPO enters a stable high-level operating range around 100 rounds, eventually reaching its highest convergence value. The configuration retaining only dynamic weights shows rapid initial growth, but its steady-state success rate struggles to break through the high level. For example... Figure 6 As shown in (d), HPD-MAPPO also has advantages in cache optimization, with its cache hit rate convergence value being higher than that of each ablation version. HPD-MAPPO outperforms each ablation version in training stability and convergence upper limit, proving the necessity of the multi-branch attention mechanism and dynamic weight adjustment mechanism, and verifying the complementary role of each module in the policy optimization process.

[0306] 3.4.2 Analysis of the Contribution of Each Mechanism

[0307] like Figure 7 As shown in (a), the lack of a multi-branch attention mechanism increases the system latency from 0.73s to 1.73s, while the lack of a dynamic weight adjustment mechanism increases the system latency to 2.33s. Both mechanisms play an important role in latency optimization. Figure 7 As shown in (b), the average energy consumption of HPD-MAPPO is 3.01 J, lower than 5.05 J for retaining only the dynamic weight configuration and 6.01 J for retaining only the hierarchical attention configuration. Figure 7 As shown in (c), the lack of dynamic weights causes the task success rate to drop significantly from 83.14% to 72.21%, while the lack of a multi-branch attention mechanism causes the success rate to decrease to 78.63%. The dynamic weight adjustment mechanism plays a dominant role in balancing the overall system performance. Figure 7 As shown in (d), the cache hit rate of HPD-MAPPO is 69.72%, which is higher than 64.90% for retaining only the dynamic weight configuration and 61.43% for retaining only the hierarchical attention configuration.

[0308] The dynamic weight adjustment mechanism's more significant impact on success rate improvement stems from the unique characteristics of post-disaster relief scenarios. In extreme dynamic environments, fluctuations in channel state and node computing power occur instantly. The dynamic weight mechanism can adjust feature weights in real time based on network conditions, directly intervening in the policy update direction and thus playing a decisive role in the early stages of training. In contrast, the multi-branch attention mechanism primarily enhances information perception capabilities through feature extraction, its role leaning more towards information-level optimization, with a relatively indirect direct impact on policy updates. Furthermore, in post-disaster relief scenarios, task priorities differ significantly. The dynamic weight mechanism can adaptively balance the optimization objectives of tasks with different priorities; this global coordination capability is more crucial for improving final performance.

[0309] 3.5 Research on Multi-branch Attention Mechanisms

[0310] This section explores the impact of multi-branch attention mechanisms on network state perception capabilities by comparing training curves and steady-state performance under different attention configurations, and verifies their superiority in handling high-dimensional heterogeneous features.

[0311] 3.5.1 Comparative Analysis of Training Curves

[0312] like Figure 8 As shown in (a), the latency curve under the Full Attention configuration converges the fastest and eventually stabilizes at the lowest level. The No Attention and Random Attention configurations exhibit drastic performance jumps and longer convergence lags, while the Single Attention configuration shows intermediate convergence performance. Figure 8 As shown in (b), the energy consumption curve of the Full Attention configuration is the most compact and steadily decreases, while the energy consumption of the No Attention and Random Attention configurations fluctuates more significantly. Figure 8 As shown in (c), the success rate curve of the Full Attention configuration has the highest convergence value, while the success rate convergence values ​​of the No Attention and Random Attention configurations are generally low, and the success rate of the Single Attention configuration is in the middle. Figure 8 As shown in (d), the Full Attention configuration also has advantages in cache optimization, with its cache hit rate convergence value being higher than other configurations.

[0313] 3.5.2 Performance Comparison Analysis

[0314] like Figure 9As shown in (a), the Full Attention scheme reduces system latency from 6.64s to 0.73s compared to the non-attention scheme, a reduction of 89.0%. Its latency advantage is also significant compared to the Random Attention scheme (4.14s) and the Single Attention scheme (2.00s). Figure 9 As shown in (b), the average energy consumption of the Full Attention scheme is 3.01 J, lower than that of the No Attention scheme (10.97 J) and the Random Attention scheme (9.28 J), and also maintains a significant lead over the Single Attention scheme (5.37 J). Figure 9 As shown in (c), the success rate of the Full Attention scheme increased from 60.18% to 83.14%, a significant gain. Compared to the Single Attention scheme (73.94%) and the Random Attention scheme (61.24%), the Full Attention scheme also demonstrates a clear advantage in success rate. Figure 9 As shown in (d), the cache hit rate of the Full Attention scheme is 69.72%, which is higher than that of the Single Attention scheme (61.99%) and the NoAttention scheme (59.23%).

[0315] 3.6 Research on Dynamic Weight Adjustment Mechanism

[0316] This section verifies the adaptive adjustment capability and policy robustness of the dynamic weight adjustment mechanism in multi-objective optimization scenarios by comparing the training curves and steady-state performance under dynamic weights and various fixed weight configurations.

[0317] 3.6.1 Comparative Analysis of Training Curves

[0318] like Figure 10 As shown in (a), the latency curve under the Dynamic configuration converges the fastest and eventually stabilizes at the lowest level. The Fixed(1.0) scheme, due to its lack of flexibility, still exhibits significant fluctuations in its latency curve during the later stages of training. Figure 10 As shown in (b), the energy consumption curve of the Dynamic configuration enters a stable low-level operating range in the middle of training, while the energy consumption convergence value of the fixed weight configuration is generally higher. As the fixed weight value increases (from 0.5 to 1.0), the energy consumption convergence value of each configuration gradually increases. Figure 10 As shown in (c), the success rate curve of the Dynamic configuration has the highest convergence value, while the success rate of the fixed-weight configuration gradually decreases as the weight value increases. Figure 10As shown in (d), the Dynamic configuration also has advantages in cache optimization, with its cache hit rate convergence value being higher than all fixed weight configurations.

[0319] 3.6.2 Performance Comparison Analysis

[0320] like Figure 11 As shown in (a), the average latency of the Dynamic scheme is 0.73s, which is lower than that of the Fixed(1.0) scheme (4.36s), and also significantly lower than that of the Fixed(0.5) scheme (3.01s) and the Fixed(0.8) scheme (3.80s). Figure 11 As shown in (b), the average energy consumption of the Dynamic scheme is 3.01 J, which is lower than that of the Fixed (0.5) scheme (7.19 J), Fixed (0.8) scheme (8.22 J), and Fixed (1.0) scheme (9.66 J). Figure 11 As shown in (c), the average success rate of the Dynamic scheme (83.14%) is significantly higher than that of the optimal fixed-weight scheme Fixed(0.5) (67.25%), and its success rate advantage is even more pronounced compared to the Fixed(0.8) scheme (63.86%) and the Fixed(1.0) scheme (57.77%). Figure 11 As shown in (d), the cache hit rate of the Dynamic scheme is 69.72%, which is higher than that of the Fixed(0.5) scheme (59.69%), Fixed(0.8) scheme (57.26%), and Fixed(1.0) scheme (54.22%). In summary, compared to any fixed-weight scheme, the dynamic weight adjustment mechanism demonstrates absolute superiority in achieving multi-objective balance.

[0321] 3.7 Scalability Analysis

[0322] This section evaluates the adaptability of the HPD-MAPPO algorithm to changes in network size by comparing training curves and steady-state performance under different drone swarm sizes, and verifies its feasibility in large-scale deployment scenarios in actual disaster relief.

[0323] 3.7.1 Comparative Analysis of Training Curves

[0324] like Figure 12 As shown in (a), the HPD-MAPPO algorithm exhibits good convergence resilience under different UAV swarm sizes, adapting to dynamic scale changes from 5 to 20 nodes. The convergence speed is slightly slower in the UAV=20 configuration, but all configurations eventually reach a steady state within 150 rounds. Figure 12 As shown in (b), with the increase in network size, the convergence value of the energy consumption curve increases slightly, but the overall fluctuation range is small. Figure 12As shown in (c), the success rate curves converge more strongly for small-scale configurations (UAV=5 and UAV=10), while the success rate convergence value for large-scale configurations (UAV=20) decreases slightly, but remains within an acceptable range overall. Figure 12 As shown in (d), HPD-MAPPO can maintain good cache optimization performance under different network sizes. The convergence value of cache hit rate decreases slightly, but the overall stability is good.

[0325] 3.7.2 Performance Comparison Analysis

[0326] like Figure 13 As shown in (a), the average latency is 0.73s at a UAV=10 scale. When the cluster size is doubled to UAV=20, the average latency (0.84s) only fluctuates slightly. Figure 13 As shown in (b), the average energy consumption at the UAV=10 scale is 3.01 J, and the average energy consumption at the UAV=20 scale is 3.20 J, showing a relatively small increase. Figure 13 As shown in (c), the highest success rate of 83.14% was achieved with a UAV=10 cluster size. When the cluster size was doubled to UAV=20, the success rate still remained at a high level of 77.25%. Figure 13 As shown in (d), the cache hit rate is 69.72% for UAV=10 and 67.49% for UAV=20, with little difference. HPD-MAPPO maintains stable performance across different network sizes and exhibits excellent scalability.

[0327] 3.8 Learning Rate Sensitivity Analysis

[0328] This section explores the impact of learning rate on the algorithm's convergence characteristics and final performance by comparing training curves and steady-state performance under different learning rate configurations, providing guidance for hyperparameter tuning in practical applications of HPD-MAPPO.

[0329] 3.8.1 Comparative Analysis of Training Curves

[0330] like Figure 14 As shown in (a), the latency curve is smoothest and the convergence height is highest when the learning rate is set to 1e-4. An excessively high learning rate (5e-4) leads to severe policy oscillations, while an excessively low learning rate (5e-5) results in extremely slow convergence. Figure 14 As shown in (b), the energy consumption curve of the 1e-4 configuration has the lowest convergence value, while the energy consumption curve of the 5e-4 configuration still exhibits significant fluctuations in the later stages of training, demonstrating obvious parameter sensitivity. Figure 14 As shown in (c), the success rate curve of the 1e-4 configuration has the highest convergence value, while the success rate convergence value of the 5e-4 configuration is significantly lower. Figure 14As shown in (d), the 1e-4 configuration also has advantages in cache optimization, as an excessively high learning rate can lead to instability in the caching strategy.

[0331] 3.8.2 Performance Comparison Analysis

[0332] like Figure 15 As shown in (a), the latency in the 1e-4 configuration is 0.73s, a reduction of 73.74% compared to the 5e-4 scheme (2.78s), and also significantly better than the 5e-5 scheme (0.99s) and the 3e-4 scheme (1.12s). Figure 15 As shown in (b), the average energy consumption of the 1e-4 configuration is 3.01 J, lower than that of the 5e-4 scheme (6.77 J), and also maintains a leading advantage compared to the 5e-5 scheme (3.44 J) and the 3e-4 scheme (3.74 J). Figure 15 As shown in (c), the success rate of the 1e-4 configuration (83.14%) is approximately 20.56% higher than that of the 5e-4 configuration (68.96%), and also significantly higher than that of the 5e-5 configuration (79.07%) and the 3e-4 configuration (80.48%). Figure 15 As shown in (d), the cache hit rate of the 1e-4 configuration is 69.72%, which is higher than that of the 5e-4 scheme (61.63%), the 5e-5 scheme (66.50%), and the 3e-4 scheme (67.42%). When the learning rate is around 1e-4, the system achieves optimal performance on all evaluation metrics, providing important guidance for hyperparameter tuning of HPD-MAPPO in practical applications.

[0333] In summary, communication infrastructure is severely damaged in post-disaster relief scenarios, and traditional network architectures are insufficient to meet the real-time and reliability requirements of emergency rescue. This embodiment focuses on the collaborative optimization of integrated air-space-ground network resources, constructing a heterogeneous network architecture that integrates satellites, UAVs, and ground base stations, and designing an HPD-MAPPO resource collaborative optimization scheme based on the MAPPO framework. This scheme achieves multi-perspective perception of network status through a multi-branch attention mechanism, ensures timely processing of high-priority rescue tasks through a priority perception mechanism, and enhances the algorithm's adaptability to harsh post-disaster environments through a dynamic weight adjustment mechanism. Simulation experiments verify the superiority of the proposed scheme in terms of task success rate, latency, and energy consumption, while ablation experiments further reveal the contribution and synergistic effect of each mechanism to performance improvement.

[0334] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A resource collaborative optimization method for an integrated air-space-ground network in a post-disaster relief scenario, characterized in that, Includes the following steps: S1. Construct a heterogeneous network architecture that integrates space satellites, airborne UAVs, and ground base stations. Space satellites are used to provide wide-area coverage support, airborne UAVs are used for local task processing and multi-agent collaborative decision-making, and ground base stations are used to provide fixed computing support. S2. Construct a hierarchical priority-aware dynamic multi-agent proximal policy optimization model based on a multi-agent proximal policy optimization framework; the hierarchical priority-aware dynamic multi-agent proximal policy optimization model adopts a centralized training and distributed execution architecture, where each agent has an independent Actor network and a shared Critic network; S3. Embed a multi-branch attention mechanism in the Actor network, extract multi-view features of the network state in parallel through the sky layer attention branch, the air layer attention branch and the ground layer attention branch, and generate multi-view fused features through cross-layer attention fusion. S4. Embed a priority-aware mechanism in the Actor network, generate priority embeddings according to the priority level of the rescue mission, and adjust the attention weights of the multi-view fusion features based on the priority embeddings. S5. Embed a dynamic weight adjustment mechanism in the Actor network, calculate dynamic weight factors based on the real-time network state, and use dynamic weight factors to adaptively scale the multi-view fusion features. S6. Based on local observation information and multi-view fusion features processed by multi-branch attention mechanism, priority perception mechanism and dynamic weight adjustment mechanism, each agent outputs task allocation decision through the Actor network, allocating rescue tasks to space satellites, airborne drones or ground base stations.

2. The resource collaborative optimization method for an integrated air-space-ground network in a disaster relief scenario according to claim 1, characterized in that, In step S1, the communication links between the space satellite, the space drone, and the ground base station are allocated spectrum resources using the orthogonal frequency division multiple access method, specifically as follows: Transmission latency of task data between heterogeneous nodes Determined by the characteristics of the physical link, as shown in the following formula: ； in, For the task The amount of data, For channel bandwidth, Signal-to-noise ratio; Signal-to-noise ratio Calculate using the following formula: ； in, For transmission power, For antenna gain, For noise power spectral density, This refers to path loss. The path loss Calculate using the following formula: ； in, For communication distance, and These are the additional loss factors in LoS and NLoS scenarios, respectively. and These are the path loss indices for LoS and NLoS scenarios, respectively. The probability of a Loss of Position (LoS) link occurring depends on the agent's real-time 3D pose and elevation angle. As shown in the following formula: ； in, , and To reflect the attenuation parameters of the disaster area's environmental characteristics, The angle of elevation.

3. The resource collaborative optimization method for an integrated air-space-ground network in a disaster relief scenario according to claim 1, characterized in that, In step S6, the task allocation decision is output with the optimization objective of minimizing task processing latency and system energy consumption. The aforementioned task processing delay Calculate using the following formula: ； in, For transmission delay, To calculate the delay, To delay queuing; The calculation delay Based on the CPU cycle model, it is represented as follows: ； in, For the task Required number of CPU cycles For nodes CPU frequency; The queuing delay Based on the classic queuing theory model, it is expressed as follows: ； in, For nodes Task arrival rate For nodes Service rate; The system energy consumption Calculate using the following formula: ； in, For transmission energy consumption, To calculate energy consumption; The transmission energy consumption Determined by the transmit power and transmission delay, as shown in the following formula: ； in, For transmission power, For transmission delay; The calculated energy consumption Quantization is performed based on the processor power consumption model, as shown in the following formula: ； in, This is the CPU energy consumption coefficient, expressed in J / (cycles·Hz²).

4. The resource collaborative optimization method for an integrated air-space-ground network in a disaster relief scenario according to claim 3, characterized in that, Each node in the heterogeneous network architecture adopts a task-type-based caching model, and a cache hit is triggered when the task type matches the cached content of the node. Cache hit rate Calculate using the following formula: ； in, To cache the hit count, This represents the total number of tasks. When the cache is hit, the computation latency is reduced to the original level. The multiple, as shown in the following formula: ； in, The computation latency when a cache hit occurs. This is a factor that reduces cache latency.

5. The resource collaborative optimization method for an integrated air-space-ground network in a disaster relief scenario according to claim 1, characterized in that, The local observation information includes its own status information, task status information, and network status information; the own status information includes the UAV's spatial location coordinates and remaining battery power; the task status information includes the amount of data to be processed, computational requirements, priority level, and task type; the network status information includes the normalized remaining computing power, normalized queue length, cache hit rate, and normalized distance from the UAV to each node for each heterogeneous node.

6. The resource collaborative optimization method for an integrated air-space-ground network in a disaster relief scenario according to claim 1, characterized in that, In the multi-branch attention mechanism described in step S3: The top-level attention branch is used to capture global network state features, and its calculation is shown in the following formula: ； in, For the input feature vector, Here, MultiHeadAttn is the sky projection matrix, and MultiHeadAttn is the multi-head attention function. The empty layer attention branch is used to capture collaborative interaction features, and its calculation is shown in the following formula: ； in, The empty layer projection matrix; The formation attention branch is used to capture access decision features, and its calculation is shown in the following formula: ； in, This is the stratigraphic projection matrix; Cross-layer attention is used to achieve information exchange and collaborative decision-making among the three branches, and its calculation is shown in the following formula: ； ； in, , , To fuse the projection matrix, This is the fused feature vector.

7. The resource collaborative optimization method for an integrated air-space-ground network in a disaster relief scenario according to claim 6, characterized in that, In the priority-aware mechanism described in step S4, priority weight coefficients are determined based on task priority levels; discrete task priority information is converted into continuous feature vectors as priority embeddings, as shown in the following equation: ； in, As a task priority, For the embedded dimension; The priority embedding is combined with the multi-view fusion feature, and the feature weights are adjusted through an attention mechanism to obtain the priority-enhanced feature, as shown in the following formula; ； ； in, To prioritize and enhance features, This is the feature vector adjusted for attention.

8. The resource collaborative optimization method for an integrated air-space-ground network in a disaster relief scenario according to claim 7, characterized in that, In the dynamic weight adjustment mechanism described in step S5: The network state features are extracted using a state encoder, as shown in the following equation: ； in, For network state characteristics, This is the encoder weight matrix; The extracted network state features are mapped using the Sigmoid function to generate dynamic weight factors between 0 and 1, as shown in the following formula: ； in, Here is the encoder weight matrix. For the Sigmoid function, For dynamic weighting factors; The features are adaptively scaled according to the dynamic weighting factor to obtain the final output feature vector, as shown in the following formula: ； in, This is the final output feature vector.

9. The resource collaborative optimization method for an integrated air-space-ground network in a disaster relief scenario according to claim 1, characterized in that, The Actor network is trained according to the following steps: a. Initialize the parameters of each agent's Actor network and the shared Critic network, and initialize the global shared experience replay buffer; b. Each agent observes the local environment to obtain local observations and task priorities, and selects the action to be executed after processing by a multi-branch attention mechanism, a priority perception mechanism, and a dynamic weight adjustment mechanism. c. After the joint action is executed, the environmental state transitions, all agents receive immediate rewards and joint observations at the next moment, and the joint experience tuple is stored in the experience replay buffer. d. Randomly sample experience samples from the experience replay buffer, calculate the advantage function estimate using the generalized advantage estimation method, and calculate the discounted cumulative reward; e. Update the Actor network parameters by maximizing the objective function, as shown in the following equation: ； in, For the trimming parameters, The advantage function is estimated as follows: ； In the formula, For GAE parameters, This refers to timing difference error; d. Update the Critic network parameters by minimizing the value function error, as shown in the following equation: ； in, Accumulate rewards for discounts.

10. A resource collaborative optimization system for an integrated air-space-ground network in a disaster relief scenario, characterized in that, The system includes: A heterogeneous network architecture consisting of space satellites, space drones, and ground-based base stations; And intelligent agents deployed on each drone, each intelligent agent making task allocation decisions using the method described in any one of claims 1 to 9.

Images