A Multi-Resource Joint Scheduling Method for Integrated Sensing and Communication Networks for Unmanned Aerial Vehicle Swarms

Abstract

This invention relates to a multi-resource joint scheduling method for an integrated perception and communication network for UAV swarms, belonging to the field of wireless communication technology. The method includes: establishing a system model of multiple UAV-ISAC tasks and defining a joint optimization problem; extracting spatiotemporal features from a dynamic heterogeneous graph that abstracts UAVs, users, and perception targets as nodes and relationships as edges; using the features as input, solving the joint optimization problem online using a hierarchical multi-agent reinforcement learning architecture; generating resource allocation and trajectory planning actions through the collaboration of a central actor and individual UAV actors, with a central critic evaluating system performance; constructing a multi-objective weighted reward function, stabilizing the training process by combining experience replay and mini-batch sampling mechanisms, and updating network parameters in parallel; and obtaining the optimal resource allocation and UAV trajectory strategy through training. This invention improves perception performance and ensures communication quality, overcoming the shortcomings of existing technologies in dynamic resource scheduling.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and in particular to a multi-resource joint scheduling method for an integrated sensing and communication network for unmanned aerial vehicle (UAV) swarms. Background Technology

[0002] Integrated Sensing and Communications (ISAC), a key technology of 6th Generation Mobile Communication Systems (6G), integrates communication and sensing functions into a single system architecture by sharing spectrum resources and hardware facilities. This enables collaborative design and resource reuse of communication and sensing, effectively improving overall system performance and resource utilization efficiency. Existing ISAC systems mainly include terrestrial ISAC and UAV ISAC systems. Terrestrial ISAC systems rely primarily on fixed ground base stations, which present significant limitations in dynamic and complex scenarios compared to UAV ISAC systems: terrain and building obstructions can disrupt line-of-sight (LoS) links, leading to a double decrease in sensing accuracy and communication reliability; fixed base station deployment locations make it difficult to dynamically adjust network topology according to actual service needs, resulting in a lack of rapid response capabilities when facing sudden or wide-area service demands. To overcome these bottlenecks, unmanned aerial vehicles (UAVs) with three-dimensional maneuverability are considered ideal carriers for building dynamic ISAC systems due to their advantages such as rapid deployment, flexible networking, and controllable costs. UAV aerial platforms can avoid signal blockage caused by ground obstacles, achieving multi-angle sensing coverage and extending communication range; multi-UAV collaborative networking can construct a distributed MIMO architecture, further improving system capacity and sensing resolution. Based on these technological advantages, UAV-assisted ISAC systems have become a current research hotspot.

[0003] Existing UAV-ISAC technology solutions can be mainly divided into two categories:

[0004] 1) Single UAV-ISAC System: This type of solution focuses on a single UAV platform, improving the communication and sensing performance of the node itself by jointly optimizing resources such as UAV trajectory, spectrum, and power. However, limited by the coverage capacity and resource constraints of a single node, this system is difficult to adapt to the combined requirements of wide-area multi-target sensing and high-throughput communication.

[0005] 2) Multi-UAV Cooperative ISAC System: This type of solution utilizes multi-node cooperation mechanisms (such as cooperative multi-point transmission and distributed sensing) to improve the overall system performance. However, existing methods usually fail to fully consider the dynamic heterogeneous relationships between UAVs, users, and sensing targets, and also lack the ability to respond in real time to changes in network status, resulting in resource allocation strategies tending to be static and making it difficult to effectively coordinate resource competition between communication and sensing tasks.

[0006] Therefore, existing UAV-ISAC systems struggle to achieve efficient collaborative allocation of communication and sensing resources in dynamic environments. Especially in scenarios with rapid node movement and frequent changes in service requirements, traditional static resource scheduling strategies are poorly adaptable, severely hindering further improvements in system performance. Summary of the Invention

[0007] Based on the above analysis, this invention aims to disclose a multi-resource joint scheduling method for an integrated perception and communication network for UAV swarms. By constructing UAVs, communication users, and perception targets into a heterogeneous graph structure and using reinforcement learning to achieve adaptive decision-making, this method effectively alleviates communication and perception resource conflicts in dynamic environments, improves perception performance, and ensures communication quality, thus overcoming the shortcomings of existing technologies in dynamic resource scheduling.

[0008] This invention discloses a multi-resource joint scheduling method for an integrated perception and communication network for unmanned aerial vehicle (UAV) swarms, comprising:

[0009] S1. Establish a system model for multiple UAV-ISAC tasks and define a joint optimization problem with the goal of maximizing the average communication rate and minimizing the cooperative sensing Cramerlow lower bound.

[0010] S2. The drone, user and sensing target are abstracted as heterogeneous graph nodes, and the relationships are abstracted as edges to construct a dynamic heterogeneous graph. A spatiotemporal bimodal graph encoder is used to extract spatiotemporal features from the dynamic heterogeneous graph.

[0011] S3. Using the aforementioned features as input, a hierarchical multi-agent reinforcement learning architecture is employed to solve the joint optimization problem online. In this architecture, resource allocation and trajectory planning actions are generated through the collaboration of a central actor and various drone actors, and the system performance is evaluated by a central critic.

[0012] S4. Construct a multi-objective weighted reward function that includes factors such as communication rate, perception accuracy, resource constraints, and security. Combine experience replay and mini-batch sampling mechanism to stabilize the training process and update network parameters in parallel. The optimal resource allocation and UAV trajectory strategy are obtained through training.

[0013] Further, S1 includes:

[0014] S1-1. Establish a system model for multiple UAV-ISAC tasks, where each UAV can serve multiple users, but each user can be served by at most one UAV.

[0015] S1-2. The weighted sum of communication users is used as the communication performance index, and the lower bound of the cooperative perception Cramer-Rao is used as the perception performance index.

[0016] S1-3. Based on the system model, communication performance and perception performance indicators, construct a unified multi-objective optimization problem that maximizes the user communication rate and minimizes the Cramer-Rao lower bound, as well as constraints including UAV launch power limits and mobility and safety constraints.

[0017] Furthermore, the constraints include user association constraints, UAV mobility constraints, collision avoidance safety constraints and flight area constraints, subcarrier allocation constraints, and UAV transmit power limits; among which,

[0018] The user association constraint is: each user can be served by at most one drone;

[0019] The mobility constraint for drones is: the displacement of a drone in an adjacent time slot shall not exceed the distance corresponding to its maximum flight speed;

[0020] Collision avoidance safety constraints: The distance between any two drones must be maintained above the minimum safe distance;

[0021] Flight area constraints are: drones must remain within the designated flight area;

[0022] Subcarrier allocation constraints are as follows: each subcarrier can be allocated to at most one UAV on each OFDM symbol; the total number of subcarriers used by all UAVs cannot exceed the total system bandwidth.

[0023] The maximum transmit power of a drone is: the transmit power of each drone cannot exceed the hardware limit.

[0024] Further, S2 includes:

[0025] S2-1. Abstract the multiple drones, multiple targets, and multiple users in the system into a dynamic heterogeneous graph; the graph includes three types of nodes that evolve over time: drones, users, and targets, as well as three types of edges that evolve over time: services, perception, and collision avoidance.

[0026] S2-2. A spatiotemporal bimodal graph encoder composed of LSTM and GAT is used to extract spatiotemporal features from dynamic heterogeneous graphs.

[0027] During the extraction process, the node state is first updated through the LSTM network, then the neighbor node information is aggregated through the GAT network to obtain the spatiotemporal fusion features of each node, and finally the full-map context features are extracted from the features of all UAV nodes.

[0028] Furthermore, in the dynamic heterogeneous graph,

[0029] The feature vector of a UAV node includes a position vector, a velocity vector, power allocation, and subcarrier allocation results;

[0030] The feature vector of a user node includes the user location vector and the real-time communication rate.

[0031] The feature vector of the target node includes the position vector and the perception error of the target;

[0032] The dynamic heterogeneous graph.

[0033] The service edge is established when the drone serves the user, and its weight is defined by SINR to characterize the quality of the communication link;

[0034] The perception edge is established when the target is within the detection range of the UAV, and its weight is defined by the Cramerlow lower bound, representing the perception accuracy.

[0035] Collision avoidance edges are established when the distance between drones is less than the safe distance. The weights are based on a negative exponential function and are used to avoid collisions.

[0036] Further, S2-2 includes:

[0037] 1) For each node, use an LSTM network to fuse the original features of the current time step with the historical hidden state of the previous time step to update the current state;

[0038] 2) For each edge, the graph attention mechanism of the GAT network is used to aggregate the information of neighboring nodes, calculate the attention weights of the source node and target node features, and sum them up to obtain the spatiotemporal fusion features of each node.

[0039] 3) Perform max pooling on the spatiotemporal fusion features of all UAV nodes to extract the context features of the entire graph.

[0040] Further, S3 includes:

[0041] S3-1. Define the state space, which includes the global state and local observations.

[0042] The global state is the fused feature vector of the dynamic heterogeneous graph; the local state is the local observation of each UAV.

[0043] S3-2, Design includes a hierarchical Actor network comprising an upper-level control center and lower-level unmanned aerial vehicle (UAV) intelligent agents;

[0044] The upper-level control center Actor network inputs the global state and outputs system-level collaborative instructions containing subcarrier allocation vectors and user association matrices; the lower-level UAV intelligent agent Actor networks input self-locking local observations and system-level collaborative instructions and output local actions including motion control quantities.

[0045] S3-3. Generate the final execution instruction by weighted fusion of global instructions and local actions.

[0046] Further, S4 includes:

[0047] S4-1. Design a multi-objective weighted reward function that integrates communication rate, sensing accuracy, and multi-constraint penalties;

[0048] S4-2. Use experience replay and Mini-batch TD-error to alternately update the parameters of the hierarchical Actor-Critic network;

[0049] S4-3 drives end-to-end training and outputs a joint decision-making strategy for resource allocation and trajectory planning.

[0050] Furthermore, the reward function comprises two parts: a basic reward and a constraint penalty.

[0051] The basic reward consists of: a weighted sum of user rates and perceived utility based on Cramerlow's lower bound; where,

[0052] The weighted sum of user rates is used to quantify the benefits of communication tasks. The core is to sum the communication rates of each user in a weighted manner. The weights are set according to the differences in service priority of users, with important users being assigned higher weights.

[0053] Perception utility based on the Cramer-Rao lower bound is a negative indicator representing perception accuracy and is used to quantify perception accuracy. The core is to process the Cramer-Rao lower bound of each perception target so that the smaller the Cramer-Rao lower bound, the larger the corresponding reward value; the weights are set according to the sensitivity requirements of different targets to perception accuracy.

[0054] The constraint penalties include: power constraints, subcarrier constraints, and physical collision / boundary violation penalties; among which,

[0055] Power constraints are used to penalize actions that exceed a preset threshold of transmit power.

[0056] Subcarrier constraints are used to penalize subcarrier allocation conflicts or actions that exceed the available resource range.

[0057] Physical collision / boundary violation penalties are used to punish physical safety violations, including the risk of drone collisions and the risk of flying out of bounds.

[0058] Further, S4-2 includes:

[0059] 1) During the strategy optimization phase, the complete experience data consisting of the global state of the storage system, the actions executed, the total reward obtained, and the subsequent global state is collected into the experience pool through the experience replay mechanism. During training, a small number of samples are randomly selected from the experience pool to form a batch sample set.

[0060] 2) Calculate the target value of the sample by combining the total reward of the sample, the discount factor, and the value estimate of the subsequent state by the target Critic network; compare the current value estimate of the Critic network with the target value to obtain the error, and construct the mean squared loss function; use the Adam optimizer to adjust the parameters to reduce the loss and improve the accuracy of value assessment;

[0061] 3) Calculate the log probability of the policy network selecting the corresponding action in the current state, multiply it by the temporal difference error output by the Critic network, and use it as the core basis for the policy gradient; take the average of the calculation results of the batch samples to determine the gradient direction; use the RMSProp optimizer to update the parameters along the gradient direction, so that the policy is biased towards actions with high long-term returns.

[0062] 4) Alternately execute the parameter update process of the Critic and Actor networks to achieve stable training and policy optimization of the architecture.

[0063] This invention can achieve one of the following beneficial effects:

[0064] This invention discloses a multi-resource joint scheduling method for integrated perception and communication networks for UAV swarms. It accurately represents the system state by constructing a dynamic heterogeneous information graph and employs a hierarchical multi-agent reinforcement learning architecture to achieve distributed decision-making and centralized optimization. This method innovatively incorporates maximizing communication rate and minimizing the perception Cramero lower bound into the optimization model, utilizes graph neural networks to extract spatiotemporal features, and solves the core challenge of balancing communication quality and perception accuracy in dynamic and complex environments through a hierarchical mechanism combining central decision-making and local execution. Attached Figure Description

[0065] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0066] Figure 1 This is a flowchart of a multi-resource joint scheduling method for an integrated perception and communication network for unmanned aerial vehicle (UAV) swarms, as described in this embodiment of the invention. Detailed Implementation

[0067] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and, together with the embodiments of the present invention, serve to illustrate the principles of the present invention.

[0068] One embodiment of the present invention discloses a multi-resource joint scheduling method for an integrated perception and communication network for unmanned aerial vehicle (UAV) swarms, such as... Figure 1 As shown, it includes:

[0069] S1. Establish a system model for multiple UAV-ISAC tasks and define a joint optimization problem with the goal of maximizing the average communication rate and minimizing the cooperative sensing Cramerlow lower bound.

[0070] S2. The drone, user and sensing target are abstracted as heterogeneous graph nodes, and the relationships are abstracted as edges to construct a dynamic heterogeneous graph. A spatiotemporal bimodal graph encoder is used to extract spatiotemporal features from the dynamic heterogeneous graph.

[0071] S3. Using the aforementioned features as input, a hierarchical multi-agent reinforcement learning architecture is employed to solve the joint optimization problem online. In this architecture, resource allocation and trajectory planning actions are generated through the collaboration of a central actor and various drone actors, and the system performance is evaluated by a central critic.

[0072] S4. Construct a joint reward function that includes factors such as communication rate, perception accuracy, resource constraints, and security. Combine experience replay and mini-batch sampling mechanism to stabilize the training process and update network parameters in parallel. The optimal resource allocation and UAV trajectory strategy are obtained through training.

[0073] Specifically, S1 includes:

[0074] S1-1. Establish a system model for multiple UAV-ISAC tasks, where each UAV can serve multiple users, but each user can be served by at most one UAV.

[0075] The system includes A fleet of drones, assembled , A single-antenna user, grouped as ,as well as One target. Drone The user set of the service is And satisfy , Therefore, drones The mathematical expression for the transmitted ISAC signal is:

[0076] ;

[0077] In the formula, To be allocated to drones Number of subcarriers, For subcarrier indexing; Number of OFDM symbols per pulse For OFDM symbol index; For drones The center frequency, Indicates time; Indicates drone In the The first OFDM symbol The amplitude of each subcarrier, Indicates drone In the The first OFDM symbol Modulation symbols on each subcarrier, UAV The symbol matrix is ​​defined as:

[0078] ;

[0079] S1-2. The weighted sum of communication users is used as the communication performance index, and the lower bound of the Cramer-Rao threshold (CRLB) for cooperative sensing is used as the sensing performance index.

[0080] The weighted communication rate of communication users is expressed as follows:

[0081] ;

[0082] In the formula, For users Weighting factors For users In The achievable communication rate at any given time,

[0083] Represented as:

[0084] ;

[0085] For users exist Signal-to-interference-to-noise ratio at any given moment.

[0086] The lower bound of CRLB is given by the inverse of the Fisher information matrix, expressed as:

[0087] ;

[0088] In the formula, For drones The Fisher submatrix contains second-order derivative information of parameters including target distance and velocity. Represents the trace of a matrix.

[0089] S1-3. Based on the system model, communication performance and perception performance indicators, construct a unified multi-objective optimization problem that maximizes the user communication rate and minimizes the Cramer-Rao lower bound, as well as constraints including UAV launch power limits and mobility and safety constraints.

[0090] The constraints include user association constraints, UAV mobility constraints, collision avoidance safety constraints, flight area constraints, subcarrier allocation constraints, and the upper limit of UAV transmit power; among which...

[0091] The user association constraint is: each user can be served by at most one drone;

[0092] The drone mobility constraint is: the displacement of the drone in adjacent time slots cannot exceed the maximum speed;

[0093] Collision avoidance safety constraints: The distance between any two drones must be maintained above the minimum safe distance;

[0094] Flight area constraints are: drones must remain within the designated flight area;

[0095] Subcarrier allocation constraints are as follows: each subcarrier can be allocated to at most one UAV per OFDM symbol; the total number of subcarriers obtained by each UAV per OFDM symbol must be a set constant.

[0096] The drone's transmit power is constrained to the following: the transmit power of each drone cannot exceed the hardware limit.

[0097] The formula for the optimization problem is:

[0098] ;

[0099] in, Representing a moment, For the drone-user association matrix, Indicates drone Service users Otherwise, it is 0; Assign matrices to subcarriers. This is a binary variable representing a drone. In the The first OFDM symbol Subcarrier allocation indicator; For the power allocation matrix, Indicates drone In the The first OFDM symbol Transmit power on each subcarrier; It is a three-dimensional position coordinate vector; The total communication rate for all users; For the lower bound of Clameros in cooperative perception; As a weighting factor; Maximum speed; The number of subcarriers obtained on each OFDM symbol; This is the upper limit of the total transmission power; The time discretization step size; This is a collection of flyable areas for drones.

[0100] S2 aims to transform the high-dimensional dynamic state of the system into graph-structured data and extract feature representations with spatiotemporal correlations; specifically, it includes the following sub-steps:

[0101] S2-1. Abstract the multiple drones, multiple targets, and multiple users in the system into a dynamic heterogeneous graph; the graph includes three types of nodes that evolve over time: drones, users, and targets, as well as three types of edges that evolve over time: services, perception, and collision avoidance.

[0102] The dynamic heterogeneity graph is represented as: ; For a set of graph nodes, It is a dynamic set of edges;

[0103] Graph node set Defined as:

[0104] ;

[0105] in,

[0106] For drone node set; ; For at any time The Heterogeneous graph nodes corresponding to each drone;

[0107] The feature vector of the drone node is: Including the drone's position vector velocity vector Power distribution and subcarrier allocation result;

[0108] For user node set, , For at any time The Heterogeneous graph nodes corresponding to each user;

[0109] The user node feature vector is: Including user location vectors and real-time communication rate ;

[0110] For the target node set; , For at any time The Each target corresponds to a heterogeneous graph node;

[0111] The feature vector of the target node is: Including the target position vector and the perception error of the target .

[0112] Among the three types of edges—service, perception, and collision avoidance—service edges are established when the drone serves the user, with weights defined by SINR to characterize the quality of the communication link; perception edges are established when the target is within the drone's detection range, with weights defined by CRLB constraints to characterize perception accuracy; and collision avoidance edges are established when the distance between drones is less than the safe distance, with weights that are negative exponential functions used to avoid collisions.

[0113] Then, the dynamic edge set of a graph is defined as:

[0114] ;

[0115] in,

[0116] To serve the relationship, when drones Service users Time Construction Edge, weight Defined by SINR:

[0117] ; This is a communication link quality function that attenuates with increasing distance;

[0118] To perceive the coverage edge, when the target In drones When the detection range is within the construction Edge, weight Defined by CRLB constraints:

[0119] ; The perception accuracy function deteriorates with increasing distance;

[0120] For collaborative / collision avoidance: the distance between drones is less than the safe distance. Time Construction Edge, weight for:

[0121] ; The penalty intensity coefficient, This is the attenuation factor.

[0122] S2-2. A spatiotemporal bimodal graph encoder composed of LSTM and GAT (Graph Attention Network) is used to extract spatiotemporal features from dynamic heterogeneous graphs.

[0123] During the extraction process, the node state is first updated through the LSTM network, then the neighbor node information is aggregated through the GAT network to obtain the spatiotemporal fusion features of each node, and finally the full-map context features are extracted from the features of all UAV nodes.

[0124] include:

[0125] 1) For each node The current state is updated by fusing the original features of the current time step with the historical hidden state of the previous time step using an LSTM network. ;

[0126] ;

[0127] node At any moment The original input features include real-time state information such as position and velocity; For point At any moment The historical state;

[0128] 2) For each edge The graph attention mechanism of the GAT network is used to aggregate neighbor node information and calculate the source node. With the target node The attention weights of the features are calculated and then summed to obtain the spatiotemporal fusion features of the nodes.

[0129] ;

[0130] in Represents a node The set of neighboring nodes, This represents the Sigmoid function. Represents the learnable parameter matrix, Represents a node At any moment The current state, For nodes At any moment The high-dimensional embedding vector.

[0131] 3) Perform max pooling on the spatiotemporal fusion features of all UAV nodes to extract the context features of the entire image. ;

[0132] .

[0133] Specifically, S3 includes:

[0134] S3-1. Define the state space, which includes the global state and local observations.

[0135] The global state is the fused feature vector of the dynamic heterogeneous graph; the local state is the local observation of each UAV.

[0136] The global state is used by the upper-level control center and is characterized by the global graph. The system is composed of the average embedding features of the nodes and represents the overall state of the system, defined as:

[0137] ;

[0138] Among them, average embedding features ;

[0139] Local observations provide support for lower-level unmanned aerial vehicle (UAV) intelligent agents. Use, directly using drone nodes spatiotemporal fusion characteristics ,Right now:

[0140] ;

[0141] It includes interaction information with local neighbors (other drones, service users, and detection targets).

[0142] S3-2, Design includes a hierarchical Actor network comprising an upper-level control center and lower-level unmanned aerial vehicle (UAV) intelligent agents;

[0143] The upper-level control center Actor network inputs the global state and outputs system-level collaborative instructions containing subcarrier allocation vectors and user association matrices; the lower-level UAV intelligent agent Actor networks input self-locking local observations and system-level collaborative instructions and output local actions including motion control quantities.

[0144] Specifically, the upper-level control center Actor network receives global features as input. Output system-level coordination instructions :

[0145] ;

[0146] in, This represents the upper-layer policy network; To assign vectors to the subcarriers of each UAV, the following conditions must be met: , This represents the total number of subcarriers available in the system. This is a user association matrix.

[0147] Lower-level drones The Actor network receives central instructions and its own observations, and outputs local actions:

[0148] ;

[0149] in, This represents the lower-level policy network. Controlling the movement of the drone, including acceleration With steering angle , For subcarrier power allocation, and The number of subcarriers internally allocated to sensing and communication respectively, satisfying... .

[0150] S3-3. Generate the final execution instruction by weighted fusion of global instructions and local actions;

[0151] By weighted fusion of global commands and local actions, a drone is generated. Final execution instructions :

[0152] ;

[0153] in, To integrate the weighting coefficients, For drones Received central instructions.

[0154] Specifically, S4 includes:

[0155] S4-1. Design a multi-objective weighted reward function that integrates communication rate, sensing accuracy, and multi-constraint penalties;

[0156] The reward function consists of two parts: a base reward and a constraint penalty. The base reward is a weighted sum of user rates. Perceived utility based on CRLB The constraint penalty term is composed of power constraints. Subcarrier constraints Physical collision / boundary crossing penalty It consists of three parts, and the total reward function is expressed as follows:

[0157] ;

[0158] , For performance weighting coefficients, , , These are the penalty intensity coefficients;

[0159] Among them, the weighted sum of user rates It is used to quantify the benefits of communication tasks. The core is to sum the communication rates of each user with weights. The weights are set according to the differences in the service priorities of users, with important users being assigned higher weights.

[0160] Perceived Utility Based on CRLB A negative indicator representing the accuracy of perception ( This is used to quantify perception accuracy. The core is to process the CRLB of each perceived target, so that the smaller the CRLB, the larger the corresponding reward value; the weights are set according to the sensitivity requirements of different targets to perception accuracy.

[0161] Power constraints This is used to punish behaviors that exceed a preset threshold for transmission power.

[0162] Subcarrier constraints This is used to penalize subcarrier allocation conflicts or actions that exceed the available resource range;

[0163] Physical collision / boundary crossing penalty It is used to punish physical safety violations, including the risk of drone collisions and flight overboarding.

[0164] S4-2. Use experience replay and Mini-batch TD-error to alternately update the parameters of the hierarchical Actor-Critic network;

[0165] Specifically, including:

[0166] 1) During the strategy optimization phase, the complete experience data consisting of the global state of the storage system, the actions executed, the total reward obtained, and the subsequent global state is collected into the experience pool through the experience replay mechanism. During training, a small number of samples are randomly selected from the experience pool to form a batch sample set.

[0167] More specifically, storing experience tuples To the experience pool And randomly sample mini-batch during training. ; From A small number of empirical samples were randomly sampled from the data.

[0168] 2) Calculate the target value of the sample by combining the total reward of the sample, the discount factor, and the value estimate of the subsequent state by the target Critic network; compare the current value estimate of the Critic network with the target value to obtain the error, and construct the mean squared loss function; use the Adam optimizer to adjust the parameters to reduce the loss and improve the accuracy of value assessment;

[0169] More specifically, for updating the Critic network, the target value is first calculated. :

[0170] ;

[0171] in, As a discount factor, For the target Critic network, the value estimate of the next state; For target network parameters;

[0172] Then, calculate the loss function. :

[0173] ;

[0174] in, For the current Critic network's value estimate of the current state, the parameters are... The main network parameters to be optimized; To calculate the batch expectation, the average error of all samples in the Mini-batch is taken.

[0175] And update the parameters using the Adam optimizer;

[0176] 3) Calculate the log probability of the policy network selecting the corresponding action in the current state, multiply it by the temporal difference error output by the Critic network, and use it as the core basis for the policy gradient; take the average of the calculation results of the batch samples to determine the gradient direction; use the RMSProp optimizer to update the parameters along the gradient direction, so that the policy is biased towards actions with high long-term returns.

[0177] More specifically, Actor network policy gradient The calculation formula is:

[0178] ;

[0179] in, For policy networks, the output is in the state. Select action The probability distribution; Let be the logarithmic probability of the policy network.

[0180] The RMSProp optimizer is used to update parameters along the gradient direction. This causes the strategy to favor actions with high long-term returns;

[0181] S4-3 drives end-to-end training and outputs a joint decision-making strategy for resource allocation and trajectory planning.

[0182] In summary, the multi-resource joint scheduling method for integrated perception and communication networks in this embodiment accurately represents the system state by constructing a dynamic heterogeneous information graph and employs a hierarchical multi-agent reinforcement learning architecture to achieve distributed decision-making and centralized optimization. This method innovatively incorporates maximizing communication rate and minimizing the perception Cramero lower bound into the optimization model, utilizes graph neural networks to extract spatiotemporal features, and solves the core challenge of balancing communication quality and perception accuracy in dynamic and complex environments through a hierarchical mechanism combining central decision-making and local execution.

[0183] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for multi-resource joint scheduling of perception communication integrated network for UAV swarm, characterized in that, include: S1. Establish a system model for multiple UAV-ISAC tasks and define a joint optimization problem with the goal of maximizing the average communication rate and minimizing the cooperative sensing Cramerlow lower bound. S2. The drone, user and sensing target are abstracted as heterogeneous graph nodes, and the relationships are abstracted as edges to construct a dynamic heterogeneous graph. A spatiotemporal bimodal graph encoder is used to extract spatiotemporal features from the dynamic heterogeneous graph. S3. Using the aforementioned features as input, a hierarchical multi-agent reinforcement learning architecture is employed to solve the joint optimization problem online. In this architecture, resource allocation and trajectory planning actions are generated through the collaboration of a central actor and various drone actors, and the system performance is evaluated by a central critic. S4. Construct a multi-objective weighted reward function that includes factors such as communication rate, perception accuracy, resource constraints, and security. Combine experience replay and Mini-batch sampling mechanism to stabilize the training process and update network parameters in parallel. The optimal resource allocation and UAV trajectory strategy are obtained through training. S2 includes: S2-1. Abstract the multiple drones, multiple targets, and multiple users in the system into a dynamic heterogeneous graph; the graph includes three types of nodes that evolve over time: drones, users, and targets, as well as three types of edges that evolve over time: services, perception, and collision avoidance. S2-2. A spatiotemporal bimodal graph encoder composed of LSTM and GAT is used to extract spatiotemporal features from dynamic heterogeneous graphs. During the extraction process, the node state is first updated through the LSTM network, then the neighbor node information is aggregated through the GAT network to obtain the spatiotemporal fusion features of each node, and finally the full-map context features are extracted from the features of all UAV nodes. S2-2 includes: 1) For each node, use an LSTM network to fuse the original features of the current time step with the historical hidden state of the previous time step to update the current state; 2) For each edge, the graph attention mechanism of the GAT network is used to aggregate the information of neighboring nodes, calculate the attention weights of the source node and target node features, and sum them up to obtain the spatiotemporal fusion features of each node. 3) Perform max pooling on the spatiotemporal fusion features of all UAV nodes to extract the context features of the entire graph.

2. The multi-resource joint scheduling method for an integrated perception and communication network for unmanned aerial vehicle (UAV) swarms according to claim 1, characterized in that, S1 includes: S1-1. Establish a system model for multiple UAV-ISAC tasks, where each UAV can serve multiple users, but each user can be served by at most one UAV. S1-2. The weighted sum of communication users is used as the communication performance index, and the lower bound of the cooperative perception Cramer-Rao is used as the perception performance index. S1-3. Based on the system model, communication performance and perception performance indicators, construct a unified multi-objective optimization problem that maximizes the user communication rate and minimizes the Cramer-Rao lower bound, as well as constraints including UAV launch power limits and mobility and safety constraints.

3. The multi-resource joint scheduling method for an integrated perception and communication network for unmanned aerial vehicle (UAV) swarms according to claim 2, characterized in that, The constraints include user association constraints, UAV mobility constraints, collision avoidance safety constraints and flight area constraints, subcarrier allocation constraints, and the upper limit of UAV transmit power; among which... The user association constraint is: each user can be served by at most one drone; The mobility constraint for drones is: the displacement of a drone in an adjacent time slot shall not exceed the distance corresponding to its maximum flight speed; Collision avoidance safety constraints: The distance between any two drones must be maintained above the minimum safe distance; Flight area constraints are: drones must remain within the designated flight area; Subcarrier allocation constraints are as follows: each subcarrier can be allocated to at most one UAV on each OFDM symbol; the total number of subcarriers used by all UAVs cannot exceed the total system bandwidth. The maximum transmit power of a drone is: the total transmit power of each drone cannot exceed the hardware limit.

4. The multi-resource joint scheduling method for an integrated perception and communication network for unmanned aerial vehicle (UAV) swarms according to claim 1, characterized in that, The dynamic heterogeneous graph. The feature vector of a UAV node includes a position vector, a velocity vector, power allocation, and subcarrier allocation results; The feature vector of a user node includes the user location vector and the real-time communication rate. The feature vector of the target node includes the position vector and the perception error of the target; The dynamic heterogeneous graph. The service edge is established when the drone serves the user, and its weight is defined by SINR to characterize the quality of the communication link; The perception edge is established when the target is within the detection range of the UAV, and its weight is defined by the Cramerlow lower bound, representing the perception accuracy. Collision avoidance edges are established when the distance between drones is less than the safe distance. The weights are based on a negative exponential function and are used to avoid collisions.

5. The method for multi-resource joint scheduling in an integrated perception and communication network for unmanned aerial vehicle (UAV) swarms according to claim 2, characterized in that, The S3 includes: S3-1. Define the state space, which includes the global state and local observations. The global state is the fused feature vector of the dynamic heterogeneous graph; the local state is the local observation of each UAV. S3-2, Design includes a hierarchical Actor network comprising an upper-level control center and lower-level unmanned aerial vehicle (UAV) intelligent agents; The upper-level control center Actor network inputs the global state and outputs system-level collaborative instructions containing subcarrier allocation vectors and user association matrices; the lower-level UAV intelligent agent Actor networks input self-locking local observations and system-level collaborative instructions and output local actions including motion control quantities. S3-3. Generate the final execution instruction by weighted fusion of global instructions and local actions.

6. The multi-resource joint scheduling method for an integrated perception and communication network for unmanned aerial vehicle (UAV) swarms according to claim 5, characterized in that, The S4 includes: S4-1. Design a multi-objective weighted reward function that integrates communication rate, sensing accuracy, and multi-constraint penalties; S4-2. Use experience replay and Mini-batch TD-error to alternately update the parameters of the hierarchical Actor-Critic network; S4-3 drives end-to-end training and outputs a joint decision-making strategy for resource allocation and trajectory planning.

7. The method for multi-resource joint scheduling in an integrated perception and communication network for unmanned aerial vehicle (UAV) swarms according to claim 6, characterized in that, The reward function consists of two parts: a basic reward and a constraint penalty. The basic reward consists of: a weighted sum of user rates and perceived utility based on Cramerlow's lower bound; where, The weighted sum of user rates is used to quantify the benefits of communication tasks. The core is to sum the communication rates of each user in a weighted manner. The weights are set according to the differences in service priority of users, with important users being assigned higher weights. Perception utility based on the Cramer-Rao lower bound is a negative indicator representing perception accuracy and is used to quantify perception accuracy. The core is to process the Cramer-Rao lower bound of each perception target so that the smaller the Cramer-Rao lower bound, the larger the corresponding reward value; the weights are set according to the sensitivity requirements of different targets to perception accuracy. The constraint penalties include: power constraints, subcarrier constraints, and physical collision / boundary violation penalties; among which, Power constraints are used to penalize actions that exceed a preset threshold of transmit power. Subcarrier constraints are used to penalize subcarrier allocation conflicts or actions that exceed the available resource range. Physical collision / boundary violation penalties are used to punish physical safety violations, including the risk of drone collisions and the risk of flying out of bounds.

8. The multi-resource joint scheduling method for an integrated perception and communication network for unmanned aerial vehicle (UAV) swarms according to claim 6, characterized in that, S4-2 includes: 1) During the strategy optimization phase, the complete experience data consisting of the global state of the storage system, the actions executed, the total reward obtained, and the subsequent global state is collected into the experience pool through the experience replay mechanism. During training, a small number of samples are randomly selected from the experience pool to form a batch sample set. 2) Calculate the target value of the sample by combining the total reward of the sample, the discount factor, and the value estimate of the subsequent state by the target Critic network; compare the current value estimate of the Critic network with the target value to obtain the error, and construct the mean squared loss function; use the Adam optimizer to adjust the parameters to reduce the loss and improve the accuracy of value assessment; 3) Calculate the log probability of the policy network selecting the corresponding action in the current state, multiply it by the temporal difference error output by the Critic network, and use it as the core basis for the policy gradient; take the average of the calculation results of the batch samples to determine the gradient direction; use the RMSProp optimizer to update the parameters along the gradient direction, so that the policy is biased towards actions with high long-term returns. 4) Alternately execute the parameter update process of the Critic and Actor networks to achieve stable training and policy optimization of the architecture.

Images