Low-altitude unmanned aerial vehicle cluster resource scheduling method and system

Abstract

The application provides a low-altitude unmanned aerial vehicle cluster resource scheduling method and system, constructs a low-altitude unmanned aerial vehicle cluster emergency communication system, and takes the maximum energy efficiency of the cluster as an optimization target of resource collaborative optimization; meanwhile, a dynamic feature map is constructed based on the low-altitude unmanned aerial vehicle and the user, the node state and the relationship edge state information situation of the dynamic feature map are enhanced and fused, and the low-altitude unmanned aerial vehicle situation feature and the ground user situation feature are obtained respectively; a decision strategy is constructed based on the low-altitude unmanned aerial vehicle situation feature and the ground user situation feature; the resource collaborative optimization problem is converted into a Markov decision process, the cluster energy efficiency is used as a reward to train a deep reinforcement learning model, and the deep reinforcement learning model is deployed to each low-altitude unmanned aerial vehicle and ground user to perform distributed reasoning. The application can reduce the online decision complexity in the post-disaster dynamic emergency communication environment, improve the cluster energy efficiency, prolong the low-altitude unmanned aerial vehicle cluster communication service time and enhance the user coverage capability.

Description

Technical Field

[0001] This invention relates to the fields of wireless communication, low-altitude unmanned aerial vehicle (UAV) communication, and artificial intelligence, and in particular to a method and system for scheduling low-altitude UAV swarm resources. Background Technology

[0002] Following natural disasters such as earthquakes, floods, and typhoons, as well as other emergencies, ground communication infrastructure is often severely damaged, rendering ground communication networks unable to operate normally. Low-altitude drones, with their advantages of rapid deployment, maneuverability, and wide aerial coverage, can establish temporary communication networks quickly by carrying base stations or relay equipment, providing wireless access services to disaster victims and rescue workers, thereby improving rescue efficiency.

[0003] In low-altitude unmanned aerial vehicle (UAV) emergency communication, efficient utilization of limited energy resources is one of the key challenges. Low-altitude UAV swarms typically rely on battery power, and their flight and communication durations are strictly limited. Therefore, improving the energy efficiency of low-altitude UAV swarms is crucial for extending communication duration and expanding service range. To achieve this goal, the design of low-altitude UAV swarm trajectories, user association strategies, and power allocation schemes all affect service quality, interference intensity, and energy utilization efficiency.

[0004] Existing resource coordination methods mostly rely on optimization theory, heuristic algorithms, or ordinary deep neural networks. These methods require relatively complete channel information, have high computational complexity, and are difficult to implement real-time decision-making in dynamic post-disaster environments. As the user scale increases and the topological structure between low-altitude UAV swarms and users becomes more complex, ordinary deep neural networks struggle to effectively characterize the structural relationships between nodes. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for scheduling low-altitude unmanned aerial vehicle (UAV) swarm resources, thereby solving the aforementioned technical problems.

[0006] In a first aspect, the present invention provides a method for resource scheduling of low-altitude unmanned aerial vehicle (UAV) swarms, comprising:

[0007] Construct an emergency communication system for low-altitude UAV swarms and obtain information on the channel gain between low-altitude UAVs and users, the achievable communication rate for users, the system's transmission energy, and the swarm's energy efficiency.

[0008] Construct a resource coordination optimization problem with the goal of maximizing the energy efficiency of the cluster;

[0009] A dynamic feature graph carrying node features and edge features is constructed using low-altitude UAVs and users as graph nodes and the communication link between low-altitude UAVs and users as graph edges. The node states and relation edge states of the dynamic feature graph are defined.

[0010] The situational enhancement and fusion of the node state and relation edge state information of the dynamic feature graph are used to obtain the situational features of low-altitude UAVs and ground users, respectively.

[0011] Decision-making strategies are constructed based on the situational characteristics of the low-altitude UAV and the situational characteristics of ground users.

[0012] The resource collaborative optimization problem is transformed into a Markov decision process. A deep reinforcement learning model containing an Actor network and a Critic network is constructed, and the deep reinforcement learning model is trained using cluster energy efficiency as a reward.

[0013] The trained and converged deep reinforcement learning model is deployed to various low-altitude drones and ground users to perform distributed inference.

[0014] Preferably, the definition of node states and relation edge states includes:

[0015] Based on the sum of achievable communication rates and interference for users served by the low-altitude UAV at the previous moment, the observation information of the low-altitude UAV at the beginning of the current moment, and the low-altitude UAV's... The distances from the four boundaries (right, left, top, and bottom) at any given moment are used to construct the node state of the low-altitude UAV at the current moment;

[0016] The node state of the user at the current moment is defined by the sum of the achievable communication rate and the sum of the interference when the user connected to the low-altitude UAV that provides services at the previous moment, the achievable communication rate and interference estimated by the user at the current moment, and the distance of the user from the four boundaries of right, left, top and bottom at the current moment.

[0017] The relationship between the low-altitude UAV and the user is defined based on the transmit power, channel gain, two-dimensional horizontal Euclidean distance, and three-dimensional spatial Euclidean distance between the low-altitude UAV and the user at the previous moment.

[0018] Preferably, the dynamic feature graph's node state and relation edge state information are enhanced and fused to obtain low-altitude UAV situational features and ground user situational features, respectively, including:

[0019] The node state of the low-altitude UAV at the current moment is mapped to high-dimensional features through a linear layer, and the high-dimensional features are transformed into UAV multi-head features through a reshaping operation.

[0020] The node state of the user at the current moment is mapped to a high-dimensional feature through a linear layer, and the high-dimensional feature is transformed into a user multi-head feature through a reshaping operation;

[0021] The relationship edge state between the low-altitude UAV and the user is copied multiple times to obtain multi-head edge features. The multi-head edge features, UAV multi-head features, and user multi-head features are then concatenated and mapped through a Leaky ReLU nonlinear layer to obtain the attention weight from the low-altitude UAV to the user.

[0022] The attention weights from the low-altitude UAV to the user are normalized using Softmax to obtain the normalized attention weights from the low-altitude UAV to the user and the normalized attention weights from the user to the low-altitude UAV.

[0023] Based on the normalized attention weights from users to low-altitude drones and the multi-head features of drones and users, the low-altitude drones use a ReLU nonlinear layer to aggregate from all their neighboring users to obtain the low-altitude drone situation features.

[0024] Based on the normalized attention weights from the low-altitude UAV to the user, and the multi-head features of the UAV and the user, the user's situational features are aggregated from the low-altitude UAV connected to it.

[0025] Secondly, the present invention provides a low-altitude unmanned aerial vehicle (UAV) swarm resource scheduling system, comprising:

[0026] An emergency communication system construction module is used to build an emergency communication system for low-altitude UAV swarms and to obtain the channel gain between low-altitude UAVs and users, the achievable communication rate of users, the system transmission energy, and the swarm energy efficiency.

[0027] An optimization problem construction module is used to construct a resource collaborative optimization problem with the goal of maximizing the energy efficiency of the cluster.

[0028] The dynamic feature graph construction module is used to construct a dynamic feature graph carrying node features and edge features with low-altitude UAVs and users as graph nodes and the communication link between low-altitude UAVs and users as graph edges, and to define the node state and relation edge state of the dynamic feature graph.

[0029] The situation feature construction module is used to enhance and fuse the situation features of the node state and relation edge state information of the dynamic feature graph to obtain the situation features of the low-altitude UAV and the situation features of the ground user, respectively.

[0030] The decision strategy construction module is used to construct a decision strategy based on the situational characteristics of the low-altitude UAV and the situational characteristics of the ground user.

[0031] The reinforcement learning module is used to transform the resource collaborative optimization problem into a Markov decision process, construct a deep reinforcement learning model containing an Actor network and a Critic network, and train the deep reinforcement learning model using cluster energy efficiency as a reward.

[0032] The inference module is used to deploy the converged deep reinforcement learning model to various low-altitude drones and ground users to perform distributed inference.

[0033] Thirdly, the present invention provides an electronic device, comprising:

[0034] At least one processor;

[0035] and a memory communicatively connected to the at least one processor;

[0036] The memory stores a computer program that can be executed by at least one processor, wherein the computer program, when executed by at least one processor, implements the low-altitude unmanned aerial vehicle (UAV) swarm resource scheduling method.

[0037] The beneficial effects of this invention are as follows:

[0038] 1. This invention can unify the low-altitude UAV location, ground user distribution, air-to-ground link quality, user service status, interference status, energy consumption and motion constraints in post-disaster emergency communication scenarios into a situation map, thereby improving the ability to express the status in a dynamic emergency communication environment.

[0039] 2. This invention can utilize the state of relational edges to participate in node feature fusion, so that the situational features of low-altitude UAVs and ground users simultaneously include the node's own state, neighbor service state, and air-to-ground communication link relationship, thereby improving the ability to characterize complex air-to-ground cooperative relationships and realizing enhanced fusion of low-altitude information situational awareness.

[0040] 3. This invention can integrate low-altitude UAV trajectory adjustment, user access selection, and transmission power allocation into resource action decision-making, enabling discrete trajectory actions, discrete access actions, and continuous power actions to be generated collaboratively under the same cluster energy efficiency target;

[0041] 4. This invention can improve cluster energy efficiency, extend the continuous service time of low-altitude UAV clusters, and enhance coverage and communication support capabilities for dynamic ground users while ensuring the needs of emergency communication services after disasters. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the method of an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the framework of the method in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram comparing the cluster energy efficiency of the embodiments of the present invention and the comparative method during the training phase;

[0045] Figure 4This is a schematic diagram comparing the cumulative distribution function of cluster energy efficiency during the inference phase of the comparative methods in this embodiment of the invention;

[0046] Figure 5 This is a schematic diagram comparing the cluster energy efficiency under different numbers of users according to an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram comparing the swarm energy efficiency of different low-altitude UAV speeds according to an embodiment of the present invention;

[0048] Figure 7 This is a structural framework diagram of the system according to an embodiment of the present invention;

[0049] In the diagram, 10 is the emergency communication system construction module; 20 is the optimization problem construction module; 30 is the dynamic feature map construction module; 40 is the situation feature construction module; 50 is the decision strategy construction module; 60 is the reinforcement learning module; and 70 is the reasoning module. Detailed Implementation

[0050] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0051] like Figure 1 and 2 As shown, this embodiment provides a method for resource scheduling of low-altitude unmanned aerial vehicle (UAV) swarms, including:

[0052] S1: Construct an emergency communication system for low-altitude UAV swarms and obtain the channel gain between low-altitude UAVs and users, the achievable communication rate of users, the system's transmission energy, and the swarm's energy efficiency;

[0053] S2: Construct a resource coordination optimization problem with the goal of maximizing the energy efficiency of the cluster;

[0054] S3: Construct a dynamic feature graph carrying node features and edge features using low-altitude UAVs and users as graph nodes and the communication link between low-altitude UAVs and users as graph edges, and define the node state and relation edge state of the dynamic feature graph.

[0055] S4: Enhance and fuse the node state and relation edge state information of the dynamic feature graph to obtain the low-altitude UAV situational features and ground user situational features respectively;

[0056] S5: Construct a decision-making strategy based on the situational characteristics of the low-altitude UAV and the ground user;

[0057] S6: Transform the resource collaborative optimization problem into a Markov decision process, construct a deep reinforcement learning model containing an Actor network and a Critic network, and train the deep reinforcement learning model using cluster energy efficiency as a reward.

[0058] S7: Deploy the converged deep reinforcement learning model to various low-altitude drones and ground users to perform distributed inference.

[0059] In this embodiment, step S1 involves constructing a low-altitude UAV swarm emergency communication system and obtaining the channel gain between the low-altitude UAV and the user, the achievable communication rate of the user, the system transmission energy, and the swarm energy efficiency, including:

[0060] S11: The aforementioned low-altitude unmanned aerial vehicle (UAV) swarm emergency communication system includes... A low-altitude unmanned aerial vehicle (UAV) and K dynamic ground users are provided, with the UAV and the ground users sharing the same communication frequency band.

[0061] S12: Based on the aforementioned low-altitude UAV With users exist The channel gain is calculated from the path loss at time step and the small-scale fading, and is expressed as:

[0062] ;

[0063] In the formula, For low-altitude drones With users exist Channel gain at time; For low-altitude drones and users exist Path loss at any given moment; For low-altitude drones To users Small-scale decay.

[0064] In this embodiment, the low-altitude drone and users exist Path loss at time step Represented as:

[0065] ;

[0066] In the formula, , These are the path losses for line-of-sight (LoS) and non-line-of-sight (NLoS), respectively. , These are the probabilities of line-of-sight path (LoS) and non-line-of-sight path (NLoS), respectively.

[0067] In this embodiment, the path loss of the line of sight. Represented as:

[0068] ;

[0069] In the formula, For low-altitude drones With users The three-dimensional Euclidean distance; For carrier frequency; This refers to the shadow fading of the line-of-sight distance (LoS).

[0070] In this embodiment, the low-altitude drone With users Three-dimensional Euclidean distance Represented as:

[0071] ;

[0072] In the formula, ( For low-altitude drones The coordinates; For users The coordinates.

[0073] In this embodiment, the low-altitude drone and users exist Path loss of non-line-of-sight (NLoS) at time step Represented as:

[0074] ;

[0075] In the formula, This refers to shadow fading at non-view distances.

[0076] In this embodiment, the probability of the line-of-sight path (LoS) Represented as:

[0077] ;

[0078] In the formula, For low-altitude drones With users The angle of elevation.

[0079] In this embodiment, the low-altitude drone To users exist Small-scale decay of time The model is a time-varying complex Rayleigh process, i.e.:

[0080] ;

[0081] In the formula, The time correlation coefficient, , It is a zero-order Bessel function. For Doppler frequency shift, For time slot interval; For low-altitude drones To users exist Small-scale decay of time; Let be a complex Gaussian random variable.

[0082] S13: Introduce a 0-1 user association indicator variable to represent the service relationship between the UAV and the user, and calculate the signal-to-interference-plus-noise ratio (SINR) by combining channel gain, transmit power, and co-channel interference. And based on signal-to-interference-plus-noise ratio Calculate the achievable communication rate for each served user. ,Right now:

[0083] ;

[0084] ;

[0085] In the formula, Associate indicator variables with users. Represents low-altitude drones exist Always serve users ; For low-altitude drones For users The transmission power; This is due to co-frequency interference between clusters. Let Variance be the variance of the Gaussian noise; For low-altitude drones With users exist Channel gain at time; This refers to the communication bandwidth.

[0086] In this embodiment, the cluster co-frequency interference Including interference within the community Inter-cell interference ,Right now:

[0087] ;

[0088] ;

[0089] In the formula, For the number of users; It is a low-altitude drone.

[0090] S14: Calculate the system's transmit energy based on the transmit power and user-related indicator variables, and the power amplifier's energy consumption coefficient, i.e.:

[0091] ;

[0092] In the formula, In order to be in The system emits energy at any given moment; For low-altitude drones The power consumption coefficient of the power amplifier; Associate indicator variables with users. Represents low-altitude drones exist Always serve users ; For low-altitude drones For users The transmission power.

[0093] S15: Based on the achievable communication rate of the served user System emission energy And calculate the cluster energy efficiency by considering the energy consumed in establishing connections for all low-altitude UAVs and the circuit energy consumed by the system.

[0094]

[0095] In the formula, For cluster energy efficiency; For low-altitude drones The handover power is used to establish a connection for each user during a cell handover. For low-altitude drones exist The number of times users switch regions while being served at any given time; Power consumption for hardware.

[0096] In this embodiment, the low-altitude drone exist The number of times users switch regions while in real-time service Represented as:

[0097] ;

[0098] In the formula, Associate indicator variables for users.

[0099] In this embodiment, step S2 involves constructing a resource collaborative optimization problem with the goal of maximizing the energy efficiency of the cluster, including:

[0100] ;

[0101] In the formula, , , Low-altitude drones The upper and lower limits of the coordinates; For low-altitude drones Maximum transmission power.

[0102] In this embodiment, in step S3, a dynamic feature graph carrying node features and edge features is constructed using the low-altitude UAV and the user as graph nodes and the communication link between the low-altitude UAV and the user as graph edges. Node states and relational edge states are defined, including:

[0103] S31: The dynamic feature map is represented as follows: ;in, These are the sets of low-altitude unmanned aerial vehicle (UAV) nodes and the sets of user nodes, respectively. For the edge set; when a low-altitude drone Service users At that time, the edge of the graph Otherwise, the graph edges .

[0104] S32: Based on the previous moment's low-altitude drone The sum of the achievable communication rates of the served users and the sum of interference, low-altitude drones in Initial observation information, low-altitude drones exist The distances from the four boundaries (right, left, top, and bottom) at any given time are used to construct the low-altitude drone. exist Node state at time step ,Right now:

[0105]

[0106] In the formula, The low-altitude drones at the previous moment The sum of the achievable communication rates of the served users and the sum of interference; Low-altitude drones exist Initial observation information; Low-altitude drones exist The distance from the four boundaries (right, left, top, and bottom) at any given time.

[0107] S33: For user nodes, this embodiment defines users. exist Node state at time step for:

[0108] In the formula, users at the previous moment The sum of achievable communication rates when connecting to and serving low-altitude drones, plus the sum of interference. users respectively exist Estimated achievable communication rate and interference at any given time. users respectively exist The distance from the four boundaries (right, left, top, and bottom) at any given time.

[0109] S34: Defining Low-Altitude Unmanned Aerial Vehicles With users Relationship edge state Represented as:

[0110] ;

[0111] In the formula, For low-altitude drones With users The transmit power and channel gain at the previous moment; For low-altitude drones With users The two-dimensional horizontal Euclidean distance; For low-altitude drones With users The three-dimensional Euclidean distance.

[0112] In this embodiment, in step S4, the node state and relation edge state information of the dynamic feature graph are enhanced and fused to obtain the low-altitude UAV situational features and ground user situational features, including:

[0113] S41: The low-altitude drone is connected via a linear layer. exist The node state at any given time is mapped to a high-dimensional feature, and the high-dimensional feature is transformed into UAV multi-head features through a reshaping operation, i.e.:

[0114]

[0115] In the formula, For low-altitude drones exist The multi-headed characteristics of drones at any given moment; To reshape the operation; This is the weight matrix of the linear layer; For low-altitude drones exist The node state at any given moment.

[0116] S42: The user is connected through a linear layer exist The node state at each time step is mapped to a high-dimensional feature, and the high-dimensional feature is transformed into a user multi-head feature through a reshaping operation, i.e.:

[0117] ;

[0118] In the formula, For users exist Multi-headed user characteristics at any given moment; To reshape the operation; This is the weight matrix of the linear layer; For users exist The node state at any given moment.

[0119] S43: Targeting low-altitude drones With users The relational edge state is copied multiple times to obtain multi-head edge features. These multi-head edge features, UAV multi-head features, and user multi-head features are then concatenated and mapped using a Leaky ReLU nonlinear layer to obtain the low-altitude UAV features. To users The attention weights, i.e.:

[0120] ;

[0121] In the formula, For low-altitude drones To users Attention weights; For Leaky ReLU nonlinear layers; It has multiple edge features.

[0122] The low-altitude UAV was analyzed using Softmax. To users The attention weights are normalized to obtain the low-altitude UAV. To users The normalized attention weights, i.e.:

[0123] ;

[0124] In the formula, For low-altitude drones To users Normalized attention weights; This refers to the number of low-altitude drones.

[0125] Accordingly, the user To low-altitude drones The normalized attention weights are represented as follows:

[0126] ;

[0127] In the formula, For users To low-altitude drones Normalized attention weights; For the number of users.

[0128] S44: Based on the user To low-altitude drones The normalized attention weights and multi-head features of the UAV and users are used to aggregate the low-altitude UAV situational features from all its neighboring users using a ReLU nonlinear layer, i.e.:

[0129] ;

[0130] In the formula, Situational characteristics of low-altitude unmanned aerial vehicles (UAVs); It is a ReLU nonlinear function; It is the Hadamard product.

[0131] S45: Based on the aforementioned low-altitude UAV To users The normalized attention weights and drone multi-head features, user multi-head features, the user The user situation characteristics are aggregated from the low-altitude drones connected to it, namely:

[0132] ;

[0133] In the formula, User behavior characteristics.

[0134] In this embodiment, step S5, constructing a decision-making strategy based on the low-altitude UAV situational characteristics and ground user situational characteristics, includes:

[0135] S51: Construct a trajectory-action mapping network for low-altitude unmanned aerial vehicles (UAVs). This network generates discrete decision probabilities for trajectory actions based on the UAV's situational characteristics, including:

[0136] The horizontal motion trajectory of the low-altitude UAV is discretized into K different discretized trajectory actions. In this embodiment, K is 5. Therefore, the low-altitude UAV in The horizontal position at a given time is represented as:

[0137] ;

[0138] In the formula, For low-altitude drones Flight speed; For time slot interval;

[0139] Low-altitude UAV situational characteristics are analyzed using a trajectory and motion mapping network. Mapped to low-altitude drones of The probability vector of each discretized trajectory action, namely:

[0140] ;

[0141] In the formula, For low-altitude drones of A probability vector of discrete trajectory actions; For low-altitude drones Take the first The probability of a discrete trajectory action; , These are the weight matrix and bias term of the trajectory-action mapping network, respectively; It is a ReLU nonlinear function; It is a fully connected layer; For reshaping operations.

[0142] Based on the probability of each discretized trajectory action, Gumbel-Softmax is introduced to calculate the decision probability of each discretized trajectory action, i.e.:

[0143] ;

[0144] In the formula, For low-altitude drones Select the The decision probability of a discrete trajectory action; For low-altitude drones Select the Gumbel noise for a discrete trajectory action; For low-altitude drones Temperature coefficient.

[0145] In some embodiments, the low-altitude drone Select the Gumbel noise for a discretized trajectory action Represented as:

[0146] ;

[0147] In the formula, It is a constant close to zero, used to prevent numerical collapse of the log function; The noise is uniformly distributed in (0,1).

[0148] S52: Construct a user access mapping network and a power regulation mapping network for each user. These networks generate user access decisions and power allocation decisions based on user situational characteristics, including:

[0149] The user access mapping network is used to map the situational characteristics of ground users to users. with low-altitude drones The probability of association, i.e.:

[0150] ;

[0151]

[0152] In the formula, Indicates the user access mapping network output user with low-altitude drones Direct association probability; This is temperature noise; Temperature coefficient; For ground user situation characteristics; , These are the weight matrix and bias term of the user access mapping network, respectively; For users with low-altitude drones The normalized probability of association.

[0153] A power adjustment mapping network is used to map ground user situational characteristics into power allocation factors, namely:

[0154] ;

[0155] In the formula, This is a non-linear output layer based on the Sigmoid activation function; For users with low-altitude drones The power allocation factor.

[0156] In this embodiment, step S6 transforms the resource collaborative optimization problem into a Markov decision process, constructs a deep reinforcement learning model containing an Actor network and a Critic network, and trains the deep reinforcement learning model using cluster energy efficiency as a reward, including:

[0157] S61: Low-altitude unmanned aerial vehicle exist Node state at time step ,user exist Node state at time step Low-altitude drones With users Relationship edge state Define the system state space ,Right now:

[0158] ;

[0159] Discretized trajectory action decision probability, user with low-altitude drones The probability of association and the power allocation factor define the system action space. ,Right now:

[0160] ;

[0161] Cluster energy efficiency As a system reward .

[0162] The resource coordination optimization problem is transformed into a long-term optimization goal of reinforcement learning, namely, maximizing the cumulative energy efficiency reward from the current time slot to the end of the cycle. This has transformed the resource coordination optimization problem into a Markov decision process.

[0163] S62: Construct an Actor policy network and a Critic value network based on an edge-enhanced graph attention network. The Actor policy network uses the system state space... As input, the output system action space 𝑎(𝑡) is used; the Critic value network is a spliced ​​system state space. and system action space As input, the state-action value is output after graph attention aggregation and global pooling.

[0164] In this embodiment, the discrete trajectory action of each low-altitude UAV is ultimately decided. The highest probability ,Right now:

[0165] .

[0166] For each user Decision making and low-altitude drones The association details are as follows:

[0167] ;

[0168] Each user Decision to connect to low-altitude drones Its transmission power for:

[0169] .

[0170] In this embodiment, during the training of the Actor policy network, the policy gradients of the low-altitude UAV trajectory, user association, and power allocation policy are calculated separately to achieve joint updates of the Actor policy network. The gradient update of the Actor policy network is expressed as follows:

[0171] ;

[0172] ;

[0173] ;

[0174] In the formula, This represents the gradient of the trajectory decision objective with respect to the Actor parameters; This represents the gradient of the associated decision objective with respect to the Actor parameters; This represents the gradient of the power decision objective with respect to the Actor parameters; The mathematical expectation of the state at the previous moment; Indicates trajectory action; Associate actions with users; For power distribution action; For the deterministic strategy of the Actor; These are the parameters of the Actor policy network; This is the output of the Critic value network.

[0175] In this embodiment, the Critic value network enhances the fusion of low-altitude UAV situational awareness features. and enhanced user situation characteristics Perform global pooling, then concatenate the results to establish a value function. The mapping relationship is as follows:

[0176] ;

[0177] In the formula, This is a global pooling operation; The weights of the linear layer; It is a ReLU nonlinear function; For reshaping operation; For several feedforward neural network operations; For reshaping operation;

[0178] In this embodiment, during the training of the Critic value network, the loss function is minimized. To update the Critic value network parameters ,Right now:

[0179]

[0180] In the formula, express The mean of the empirical values; Indicates batch size; Indicates the first One experience, Indicates the discount factor; To use the target Actor parameter The action obtained; The parameters are for the target Critic.

[0181] In this embodiment, the target Actor parameter Parameters of the target Critic Update according to the soft update policy, that is:

[0182] ;

[0183] ;

[0184] In the formula, Parameters used to control the speed of soft updates.

[0185] This embodiment uses simulation experiments to model an emergency communication scenario for low-altitude unmanned aerial vehicles (UAVs), considering the coverage area of ​​the communication scenario as a circular area with a radius of 500m. Within this area, [the following is a description of a UAV deployment]. A cluster of low-altitude drones, simultaneously for Provide communication services to ground users. The initial positions of the low-altitude UAV swarm are shown in the table: (0,0,50), (0,0,40), (0,0,30), (0,0,35), and (0,0,40), in meters (m). The initial positions of users are randomly distributed within the communication area. Furthermore, the movement speed of the low-altitude UAV swarm is 20 m / s, and users employ a random walk movement mode with a speed of 5 m / s. Other system simulation parameters are shown in Table 1, and network parameters are shown in Table 2.

[0186] Table 1 System Simulation Parameters

[0187]

[0188] Table 2 Network Parameters

[0189]

[0190] To verify the effectiveness of the method in this embodiment, the following comparative methods were selected for comparative experiments: (1) The trajectory of the low-altitude UAV cluster was randomly generated, and user association and power allocation were obtained by the method of this invention (random location - cooperative resource scheduling); (2) The trajectory of the low-altitude UAV cluster was fixed, and user association and power allocation were obtained by the method of this invention (fixed location - cooperative resource scheduling); (3) The trajectory of the low-altitude UAV cluster, user association and power allocation were obtained by deep reinforcement learning (deep reinforcement learning method); (4) The trajectory of the low-altitude UAV cluster was obtained by the method of this embodiment, and user association and power allocation were randomly generated (trajectory scheduling - random resource scheduling).

[0191] Figure 3 This embodiment demonstrates the ability of the present invention to improve the cluster energy efficiency of an emergency communication system during the training phase. Results show that although the method described in this invention has a slower convergence speed during training, it ultimately significantly improves the system's energy efficiency, thereby extending the endurance of the low-altitude UAV swarm and enhancing the reliability and efficiency of emergency communication. In contrast, while methods based on deep reinforcement learning achieve high energy efficiency after approximately 6000 training slots, their performance subsequently fluctuates and declines, eventually converging after approximately 8000 iterations. The method in this embodiment achieves performance comparable to the deep reinforcement learning method within 6000 training slots and tends to converge after 12000 training slots. Compared to methods based on deep reinforcement learning, the method in this embodiment can effectively utilize the system's topological information, improving information utilization and thus enhancing the system's cluster energy efficiency. Compared to collaborative resource scheduling methods based on random or fixed locations, the method in this embodiment can effectively optimize the trajectory of the low-altitude UAV swarm, achieving higher cluster energy efficiency.

[0192] Figure 4 This paper demonstrates the cluster energy efficiency performance of the method in this embodiment during the inference phase. During inference, each low-altitude UAV and user only needs to aggregate information from neighboring nodes through their locally deployed network to complete optimal emergency resource coordination decisions. Compared to other comparative methods, the method of this invention has significant advantages in both minimum cluster energy efficiency and overall performance. As shown in the figure, the comparative methods achieve a cluster energy efficiency of 20 Mbit / J with only a 20-30% probability, while the method of this embodiment achieves the same performance in approximately 80% of cases. Furthermore, the comparative methods achieve 40 Mbit / J with less than a 10% probability, while the method of this embodiment achieves this level with approximately a 30% probability. These results indicate that the method of this embodiment possesses higher reliability and stability during the inference phase and can better adapt to complex and ever-changing post-disaster emergency communication scenarios.

[0193] Figure 5This paper demonstrates the cluster energy efficiency performance of the proposed method under different user numbers. The results show that as the number of users increases, the interference within the system becomes more complex and the interference intensity increases, leading to a gradual decrease in the cluster energy efficiency of most methods. However, the proposed method maintains a significant performance advantage regardless of the number of users. In contrast, deep reinforcement learning methods exhibit large performance fluctuations and poor stability when the number of users is 15; while the proposed method demonstrates more stable cluster energy efficiency across different user scales. This result further verifies the robustness and superiority of the proposed method in complex and ever-changing post-disaster emergency communication scenarios.

[0194] Figure 6 This paper presents a comparison of the swarm energy efficiency of the proposed method under different low-altitude UAV speed settings. The results show that the swarm energy efficiency decreases overall as the swarm speed increases. This is because high-speed moving low-altitude UAVs have larger displacements within adjacent time intervals, resulting in sparser environmental information collected, thus limiting accurate perception of the global state and affecting energy efficiency. Specifically, the swarm energy efficiency is relatively poor when the UAV speed is 10 m / s, as the slow movement speed makes it difficult for the UAVs to effectively track dynamically changing user distribution, leading to decreased resource allocation efficiency. Overall, the proposed method significantly outperforms the comparative algorithm under different low-altitude UAV speed settings, verifying its robustness and wide applicability in post-disaster emergency communication scenarios.

[0195] like Figure 7 As shown, this embodiment provides a low-altitude unmanned aerial vehicle (UAV) swarm resource scheduling system, including:

[0196] The emergency communication system construction module 10 is used to construct a low-altitude UAV swarm emergency communication system and obtain the channel gain between the low-altitude UAV and the user, the user's achievable communication rate, the system's transmission energy, and the swarm's energy efficiency.

[0197] In this embodiment, the low-altitude unmanned aerial vehicle (UAV) swarm emergency communication system includes: The system consists of a low-altitude drone and K dynamic ground users, with the low-altitude drone and dynamic ground users sharing the same communication frequency band.

[0198] The optimization problem construction module 20 is used to construct a resource collaborative optimization problem with the goal of maximizing the energy efficiency of the cluster.

[0199] The dynamic feature graph construction module 30 is used to construct a dynamic feature graph carrying node features and edge features with low-altitude UAVs and users as graph nodes and the communication link between low-altitude UAVs and users as graph edges, and to define the node state and relation edge state of the dynamic feature graph.

[0200] The situation feature construction module 40 is used to enhance and fuse the situation features of the node state and relation edge state information of the dynamic feature graph to obtain the situation features of the low-altitude UAV and the situation features of the ground user, respectively.

[0201] The decision strategy construction module 50 is used to construct a decision strategy based on the situational characteristics of the low-altitude UAV and the situational characteristics of the ground user.

[0202] In this embodiment, a trajectory-action mapping network is constructed for low-altitude UAVs. The trajectory-action mapping network generates discrete trajectory-action decision probabilities based on the situational characteristics of the low-altitude UAVs.

[0203] In this embodiment, a user access mapping network and a power regulation mapping network are constructed for users. The user access mapping network and the power regulation mapping network generate user access decisions and power allocation decisions based on user situation characteristics.

[0204] The reinforcement learning module 60 is used to transform the resource collaborative optimization problem into a Markov decision process, construct a deep reinforcement learning model containing an Actor network and a Critic network, and train the deep reinforcement learning model using cluster energy efficiency as a reward.

[0205] In this embodiment, a low-altitude unmanned aerial vehicle (UAV) is used. exist Node state at time step ,user exist Node state at time step Low-altitude drones With users Relationship edge state Define the system state space The decision probability of discretized trajectory actions and user... with low-altitude drones The probability of association and the power allocation factor define the system action space. Cluster energy efficiency As a system reward, the resource coordination optimization problem is transformed into a long-term optimization objective of reinforcement learning, namely, maximizing the discounted cumulative energy efficiency reward from the current time slot to the end of the cycle. This has transformed the resource coordination optimization problem into a Markov decision process.

[0206] An Actor policy network and a Critic value network are constructed based on an edge-enhanced graph attention network. The Actor policy network is based on the system state space. As input, the output system action space 𝑎(𝑡) is used; the Critic value network is a spliced ​​system state space. and system action space As input, the state-action value is output after graph attention aggregation and global pooling.

[0207] The inference module 70 is used to deploy the converged deep reinforcement learning model to various low-altitude drones and ground users to perform distributed inference.

[0208] Thirdly, embodiments of this application also provide an electronic device, including:

[0209] At least one processor;

[0210] and a memory communicatively connected to the at least one processor;

[0211] The memory stores a computer program that can be executed by at least one processor. When the computer program is executed by at least one processor, it implements the low-altitude UAV swarm resource scheduling method described in any of the above embodiments.

[0212] The embodiments of this application also provide a computer storage medium storing a computer program, which, when executed by a processor, implements the low-altitude unmanned aerial vehicle (UAV) swarm resource scheduling method described in any of the above embodiments.

[0213] The embodiments and descriptions above are merely illustrative of the principles and preferred embodiments of the present invention. Various changes and modifications may be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. A method for resource scheduling of low-altitude unmanned aerial vehicle (UAV) swarms, characterized in that, include: Construct an emergency communication system for low-altitude UAV swarms and obtain information on the channel gain between low-altitude UAVs and users, the achievable communication rate for users, the system's transmission energy, and the swarm's energy efficiency. Construct a resource coordination optimization problem with the goal of maximizing the energy efficiency of the cluster; A dynamic feature graph carrying node features and edge features is constructed using low-altitude UAVs and users as graph nodes and the communication link between low-altitude UAVs and users as graph edges. The node states and relation edge states of the dynamic feature graph are defined. The situational enhancement and fusion of the node state and relation edge state information of the dynamic feature graph are used to obtain the situational features of low-altitude UAVs and ground users, respectively. Decision-making strategies are constructed based on the situational characteristics of the low-altitude UAV and the situational characteristics of ground users. The resource collaborative optimization problem is transformed into a Markov decision process. A deep reinforcement learning model containing an Actor network and a Critic network is constructed, and the deep reinforcement learning model is trained using cluster energy efficiency as a reward. The trained and converged deep reinforcement learning model is deployed to various low-altitude drones and ground users to perform distributed inference.

2. The scheduling method according to claim 1, characterized in that: Construct an emergency communication system for low-altitude unmanned aerial vehicle (UAV) swarms, and obtain the channel gain between the UAVs and the user, the achievable communication rate for the user, the system's transmission energy, and the swarm's energy efficiency, including: The aforementioned low-altitude unmanned aerial vehicle (UAV) swarm emergency communication system includes A low-altitude unmanned aerial vehicle (UAV) and K dynamic ground users are provided, with the UAV and the ground users sharing the same communication frequency band. The channel gain is calculated based on the path loss and small-scale fading of the low-altitude UAV and the user at the current moment; A 0-1 user association indicator variable is introduced to characterize the service relationship between the UAV and the user. The signal-to-interference-plus-noise ratio (SINR) is calculated by combining channel gain, transmit power, and co-channel interference. The achievable communication rate of each served user is calculated based on the SINR. The system's transmit energy is calculated based on the transmit power and the user-related indicator variable, as well as the power amplifier's energy consumption coefficient. The cluster energy efficiency is calculated based on the achievable communication rate of the served users, the energy consumed by the system's transmission, the energy consumed by establishing connections for all low-altitude UAVs, and the circuit energy consumed by the system.

3. The scheduling method according to claim 2, characterized in that: The definition of node states and relation edge states includes: Based on the sum of achievable communication rates and interference for users served by the low-altitude UAV at the previous moment, the observation information of the low-altitude UAV at the beginning of the current moment, and the low-altitude UAV's... The distances from the four boundaries (right, left, top, and bottom) at any given moment are used to construct the node state of the low-altitude UAV at the current moment; The node state of the user at the current moment is defined by the sum of the achievable communication rate and the sum of the interference when the user connected to the low-altitude UAV that provides services at the previous moment, the achievable communication rate and interference estimated by the user at the current moment, and the distance of the user from the four boundaries of right, left, top and bottom at the current moment. The relationship between the low-altitude UAV and the user is defined based on the transmit power, channel gain, two-dimensional horizontal Euclidean distance, and three-dimensional spatial Euclidean distance between the low-altitude UAV and the user at the previous moment.

4. The scheduling method according to claim 3, characterized in that: The situational enhancement fusion of node state and relation edge state information in the dynamic feature graph yields situational features of low-altitude UAVs and ground users, including: The node state of the low-altitude UAV at the current moment is mapped to high-dimensional features through a linear layer, and the high-dimensional features are transformed into UAV multi-head features through a reshaping operation. The node state of the user at the current moment is mapped to a high-dimensional feature through a linear layer, and the high-dimensional feature is transformed into a user multi-head feature through a reshaping operation; The relationship edge state between the low-altitude UAV and the user is copied multiple times to obtain multi-head edge features. The multi-head edge features, UAV multi-head features, and user multi-head features are then concatenated and mapped through a Leaky ReLU nonlinear layer to obtain the attention weight from the low-altitude UAV to the user. The attention weights from the low-altitude UAV to the user are normalized using Softmax to obtain the normalized attention weights from the low-altitude UAV to the user and the normalized attention weights from the user to the low-altitude UAV. Based on the normalized attention weights from users to low-altitude drones and the multi-head features of drones and users, the low-altitude drones use a ReLU nonlinear layer to aggregate from all their neighboring users to obtain the low-altitude drone situation features. Based on the normalized attention weights from the low-altitude UAV to the user, and the multi-head features of the UAV and the user, the user's situational features are aggregated from the low-altitude UAV connected to it.

5. The scheduling method according to claim 4, characterized in that: A decision-making strategy is constructed based on the situational characteristics of the low-altitude UAV and the ground user, including: A trajectory-action mapping network is constructed for low-altitude unmanned aerial vehicles (UAVs). The trajectory-action mapping network generates discrete decision probabilities of trajectory actions based on the situational characteristics of the low-altitude UAVs. For each user, a user access mapping network and a power regulation mapping network are constructed. The user access mapping network and the power regulation mapping network generate user access decisions and power allocation decisions based on user situation characteristics.

6. The scheduling method according to claim 5, characterized in that: A trajectory-action mapping network is constructed for low-altitude unmanned aerial vehicles (UAVs). This network generates discretized decision probabilities for trajectory actions based on the UAV's situational characteristics, including: The horizontal motion trajectory of the low-altitude UAV is discretized into K different discretized trajectory actions; A trajectory-motion mapping network is used to map the situational characteristics of low-altitude UAVs to the UAV's own trajectory. A probability vector of discrete trajectory actions; Based on the probability of each discretized trajectory action, Gumbel-Softmax is introduced to calculate the decision probability of each discretized trajectory action.

7. The scheduling method according to claim 6, characterized in that: The resource collaborative optimization problem is transformed into a Markov decision process. A deep reinforcement learning model containing an Actor network and a Critic network is constructed, and the deep reinforcement learning model is trained using cluster energy efficiency as a reward, including: The system state space is defined by the node state of the low-altitude UAV at the current moment, the node state of the user at the current moment, and the relationship edge state between the low-altitude UAV and the user. The system action space is defined by the decision probability of discretized trajectory actions, the probability of user association with low-altitude UAV, and the power allocation factor; Cluster energy efficiency is used as a system reward; The resource coordination optimization problem is transformed into a long-term optimization goal of reinforcement learning, namely, maximizing the cumulative energy efficiency reward from the current time slot to the end of the cycle. This has transformed the resource coordination optimization problem into a Markov decision process. An Actor policy network and a Critic value network are constructed based on an edge-enhanced graph attention network. The Actor policy network takes the system state space as input and outputs the system action space. The Critic value network takes the concatenated system state space and system action space as input, and outputs the state-action value after graph attention aggregation and global pooling.

8. The scheduling method according to claim 7, characterized in that: During the training of the Actor policy network, the policy gradients of the low-altitude UAV trajectory, user association, and power allocation policy are calculated separately to achieve joint updates of the Actor policy network. During the training of the Critic value network, the parameters of the Critic value network are updated by minimizing the loss function.

9. A low-altitude unmanned aerial vehicle (UAV) swarm resource scheduling system, characterized in that, include: An emergency communication system construction module is used to build an emergency communication system for low-altitude UAV swarms and to obtain the channel gain between low-altitude UAVs and users, the achievable communication rate of users, the system transmission energy, and the swarm energy efficiency. An optimization problem construction module is used to construct a resource collaborative optimization problem with the goal of maximizing the energy efficiency of the cluster. The dynamic feature graph construction module is used to construct a dynamic feature graph carrying node features and edge features with low-altitude UAVs and users as graph nodes and the communication link between low-altitude UAVs and users as graph edges, and to define the node state and relation edge state of the dynamic feature graph. The situation feature construction module is used to enhance and fuse the situation features of the node state and relation edge state information of the dynamic feature graph to obtain the situation features of the low-altitude UAV and the situation features of the ground user, respectively. The decision strategy construction module is used to construct a decision strategy based on the situational characteristics of the low-altitude UAV and the situational characteristics of the ground user. The reinforcement learning module is used to transform the resource collaborative optimization problem into a Markov decision process, construct a deep reinforcement learning model containing an Actor network and a Critic network, and train the deep reinforcement learning model using cluster energy efficiency as a reward. The inference module is used to deploy the converged deep reinforcement learning model to various low-altitude drones and ground users to perform distributed inference.

10. An electronic device, comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by at least one processor, wherein the computer program, when executed by at least one processor, implements the low-altitude unmanned aerial vehicle (UAV) swarm resource scheduling method as described in any one of claims 1-8.

Images