Multi-uav task planning method and device, equipment and storage medium

Abstract

The application discloses a multi-unmanned aerial vehicle task planning method and device, equipment and a storage medium, relates to the technical field of unmanned aerial vehicle task planning, and comprises the following steps: acquiring unmanned aerial vehicle attribute data and task attribute data, and constructing a heterogeneous graph based on the unmanned aerial vehicle attribute data and the task attribute data; performing node-level attention and semantic-level attention on the heterogeneous graph along a preset meta path to obtain a unified graph representation; splicing the unified graph representation and a selected node at a previous moment into a context vector; inputting the context vector into a decoder based on a transformer for autoregressive processing to obtain a task node sequence; determining an instant reward based on the task node sequence, updating a deep reinforcement learning strategy network based on the instant reward until the strategy converges, and obtaining a target task planning result. The application can effectively utilize the attribute data of unmanned aerial vehicles and tasks, fully consider the complex relationship between the unmanned aerial vehicles and the tasks and various constraint conditions, and generate a scientific and reasonable task planning scheme.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) mission planning technology, and in particular to a multi-UAV mission planning method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of drone technology, its application in logistics, disaster relief, and other fields is becoming increasingly widespread. Multi-drone collaborative operations can significantly improve mission efficiency, meeting the urgent needs of modern society for efficient and precise mission execution. For example, in disaster relief, multiple drones can quickly reach the scene to perform tasks such as search and rescue and material delivery, saving valuable time for rescue efforts; in logistics, drones can avoid ground traffic congestion and achieve rapid delivery of goods.

[0003] However, existing technologies have significant shortcomings in multi-UAV mission planning. Traditional methods often employ manually designed rules or rely on a single algorithm, making them ill-suited for complex mission scenarios. For example, some methods only consider the UAV's flight distance, neglecting mission benefits and time constraints; others suffer from low computational efficiency when dealing with large-scale problems, failing to meet the requirements of real-time planning. These deficiencies severely limit the effectiveness and scope of multi-UAV systems in practical applications.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a method, apparatus, device, and storage medium for multi-UAV mission planning, aiming to solve the technical problem of how to balance mission benefits, flight distance, and flight time in large-scale heterogeneous multi-UAV mission planning and achieve efficient solution.

[0006] To achieve the above objectives, the present invention provides a multi-UAV mission planning method, which includes the following steps: Acquire drone attribute data and mission attribute data, and construct a heterogeneous graph based on the drone attribute data and mission attribute data; Node-level attention and semantic-level attention are performed on the heterogeneous graph along a preset meta-path to obtain a unified graph representation; The unified graph representation is concatenated with the selected nodes from the previous time step to form a context vector. The context vector is then input into a transformer-based decoder for autoregressive processing to obtain the task node sequence. Based on the task node sequence, an immediate reward is determined, and the deep reinforcement learning policy network is updated based on the immediate reward until the policy converges, thereby obtaining the target task planning result. The target task planning result includes the task execution order, take-off and landing base, and flight path of each UAV.

[0007] In one embodiment, the step of acquiring UAV attribute data and mission attribute data, and constructing a heterogeneous graph based on the UAV attribute data and mission attribute data, includes: Acquire drone location information, resource information, and battery life information as drone attribute data; Obtain task location information, required resource information, and task revenue information as task attribute data; The distance information between nodes is determined based on the UAV location information and the mission location information; The flight time information between nodes is determined based on the distance information between the nodes and the preset flight speed. By combining the UAV attribute data, the mission attribute data, the distance information between nodes, and the flight time information between nodes, a heterogeneous graph is constructed.

[0008] In one embodiment, the step of performing node-level attention and semantic-level attention on the heterogeneous graph along a preset meta-path to obtain a unified graph representation includes: The characteristics of the drone node are determined based on the drone attribute data, and the characteristics of the task node are determined based on the task attribute data. Type conversion matrices are applied to the UAV node features and the task node features respectively to map them to a unified dimension, resulting in unified UAV features and unified task features; The neighbor nodes are traversed along five preset meta-paths. Node-level attention weights are calculated in each meta-path, and neighbor features are aggregated in a weighted manner to obtain node-level embeddings. Semantic attention processing is performed on the node-level embeddings of the same node under different meta-paths to obtain the final node embedding; Global average pooling is applied to all final node embeddings to obtain graph-level embeddings. By concatenating the graph-level embeddings with the final node embeddings, a unified graph representation is obtained.

[0009] In this embodiment, the step of traversing neighbor nodes along five preset meta-paths, calculating node-level attention weights within each meta-path, and weighted aggregating neighbor features to obtain node-level embeddings includes: For each preset meta-path, determine the set of neighboring nodes of the current node; Extract the features of the current node and the features of each of the neighbor node sets; Calculate the similarity between the current node and each of its neighboring nodes, and use it as the initial attention weight; The initial attention weights are normalized to obtain normalized attention weights; The features of each set of neighbor nodes are weighted and summed using the normalized attention weights to obtain the weighted aggregated neighbor features. The weighted aggregated neighbor features are concatenated with the features of the current node to obtain the node-level embedding of the current node under the current metapath.

[0010] In one embodiment, the step of concatenating the unified graph representation with the previously selected node embedding to form a context vector, and inputting the context vector into the transformer decoder for autoregressive processing to obtain the task node sequence includes: The unified graph representation is embedded and concatenated with the selected task node from the previous time step to obtain the current context vector; Based on the current context vector, the probability distribution of unselected task nodes is determined through a multi-head attention mechanism; The current task node is selected based on the probability distribution; The current task node is added to the selected list, and the next autoregressive iteration continues until all task nodes are added to the selected list, thus obtaining the task node sequence.

[0011] In one embodiment, the step of determining the immediate reward based on the task node sequence and updating the deep reinforcement learning policy network based on the immediate reward until the policy converges to obtain the target task planning result includes: The total mission revenue, total flight distance, and target single-aircraft flight time are determined based on the mission node sequence. The total mission revenue, the total flight distance, and the target single-aircraft flight time are weighted and summed based on preset revenue weights, distance weights, and time weights to obtain the instant reward. The immediate reward is input into the value network and the advantage function is obtained by using a generalized advantage estimation algorithm. The policy loss in truncated form is calculated based on the advantage function using a preset near-end policy optimization algorithm, and the policy network parameters are updated based on the policy loss. Synchronize the old policy network parameters to the policy network parameters, iterate until the policy converges, and obtain the target mission planning result including the mission execution order, take-off and landing bases, and flight path.

[0012] In one embodiment, the step of performing a weighted summation of the total mission revenue, the total flight distance, and the target single-aircraft flight time based on preset revenue weights, distance weights, and time weights to obtain an immediate reward includes: Multiply the total task revenue by the preset revenue weight to obtain the first weighted value; Multiply the total flight distance by the preset distance weight to obtain the second weighted value; The target single-aircraft flight time is multiplied by a preset time weight to obtain a third weighted value; The first weighted value, the second weighted value, and the third weighted value are added together to obtain the instant reward.

[0013] Furthermore, to achieve the above objectives, the present invention also proposes a multi-UAV mission planning device, the device comprising: The data acquisition module is used to acquire UAV attribute data and mission attribute data, and construct a heterogeneous graph based on the UAV attribute data and the mission attribute data; The attention processing module is used to perform node-level attention and semantic-level attention on the heterogeneous graph along a preset meta-path to obtain a unified graph representation; The decoding module is used to concatenate the unified graph representation with the selected nodes from the previous time step into a context vector, and input the context vector into the transformer-based decoder for autoregressive processing to obtain the task node sequence. The policy update module is used to determine the immediate reward based on the task node sequence and update the deep reinforcement learning policy network based on the immediate reward until the policy converges, thereby obtaining the target task planning result. The target task planning result includes the task execution order, take-off and landing base, and flight path of each UAV.

[0014] Furthermore, to achieve the above objectives, the present invention also proposes a multi-UAV mission planning device, the device comprising: a memory, a processor, and a multi-UAV mission planning program stored in the memory and executable on the processor, the multi-UAV mission planning program being configured to implement the steps of the multi-UAV mission planning method described above.

[0015] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a multi-UAV mission planning program, which, when executed by a processor, implements the steps of the multi-UAV mission planning method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the multi-UAV mission planning method described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: By constructing a heterogeneous graph containing UAV and mission attribute data and processing it using a dual attention mechanism at the node and semantic levels, the feature information of complex relationships between UAV and mission nodes is effectively extracted. A stepwise decision-making process for mission planning is achieved by concatenating a unified graph representation with selected node embeddings into a context vector, which is then input into a transformer decoder for autoregressive processing. The introduction of immediate rewards and continuous updates to the deep reinforcement learning policy network ensure the efficiency and dynamic adaptability of mission planning. This method can fully consider various constraints between UAVs and missions, generating scientifically sound mission planning schemes, significantly improving the accuracy and reliability of mission planning. It is particularly suitable for large-scale heterogeneous multi-UAV scenarios, possessing significant technical advantages and broad application prospects. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the multi-UAV mission planning method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the multi-UAV mission planning method of this application. Figure 3 The flowchart of the reinforcement learning training algorithm based on PPO provided in Embodiment 2 of the multi-UAV mission planning method of this application is shown. Figure 4 This is a schematic diagram of the module structure of the multi-UAV mission planning device according to an embodiment of this application; Figure 5 This is a schematic diagram of the hardware operating environment involved in the multi-UAV mission planning method in the embodiments of this application.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or multi-UAV mission planning device capable of performing the above functions. The following description uses a multi-UAV mission planning device as an example to illustrate this embodiment and the subsequent embodiments.

[0025] Based on this, embodiments of this application provide a multi-UAV mission planning method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the multi-UAV mission planning method of this application.

[0026] In this embodiment, the multi-UAV mission planning method includes steps S10 to S40: Step S10: Obtain UAV attribute data and mission attribute data, and construct a heterogeneous graph based on the UAV attribute data and mission attribute data; It's important to note that we collect relevant attribute data for drones and missions, and then construct a heterogeneous graph based on this data. A heterogeneous graph is a graph structure that can represent different types of nodes (such as drone nodes and mission nodes) and their relationships. Specifically, we need to obtain the drone's location, remaining resources, and flight endurance information as drone attribute data, and simultaneously collect the mission's location, required resources, and mission rewards as mission attribute data. Using the location information of the drone and mission, we can calculate the distance between them, and thus determine the drone's flight time between different nodes. Integrating this data, including drone attributes, mission attributes, distances between nodes, and flight times, we ultimately construct a complete heterogeneous graph, providing a foundation for subsequent mission planning.

[0027] It should be noted that, therefore, in order to use deep reinforcement learning algorithms combined with heterogeneous graph neural networks to solve this problem, this paper models the multi-UAV mission planning problem as a Markov decision process, including the definitions of state, action, and reward: State space S: First, the multi-UAV mission planning problem is defined as having... ( , It is the number of drones. (This refers to a graph with a number of task nodes), where each node represents a drone or a task. Features are determined by their position x-coordinate y-axis Destroying missile requirements (drone) strike mission Required number of missiles), strike benefits (drone) strike mission The five sets of characteristics (income, etc.) represent the characteristics of the drone node. Features are defined by the x-coordinate of the position. y-axis Number of missiles carried Remaining flight time Total flight time Five features are used to represent the edge features. For the design of edge features, features for all edge types are designed as follows: , Two features, among which Indicates that the drone is generated by a node fly to node The time spent For drones from nodes fly to node The required costs (in this paper, flight distance is considered the cost), where the dimensions are normalized and the graph is fully connected. In the time steps... ,state A subgraph representation consisting of drone nodes and visited task nodes.

[0028] Action space: The actions performed by the agent at each time step. To select unvisited mission nodes to attack or to select drones to perform missions.

[0029] Reward Function: The optimization objective of this paper is to maximize task rewards while minimizing total flight distance and total flight time, provided that all tasks are completed. Therefore, the reward function is designed as follows: ,in = ( ) represents the permutation of all nodes, which serves as the solution to this problem, where , and .

[0030] Immediate benefits for intelligent agents (drone) From node fly to node The reward obtained is ,in , represents the weighting coefficient. Based on the current state drones Is the flight time the maximum flight time? If so, then it is... Otherwise, it is 0.

[0031] Based on the Markov decision process model described above, this paper designs a deep reinforcement learning algorithm that combines heterogeneous graph attention networks to solve the multi-UAV mission planning problem.

[0032] This paper designs a heterogeneous graph neural network model for multi-UAV mission planning and trains it using the Proximal Policy Optimization (PPO) deep reinforcement learning algorithm based on the Actor-Critic architecture. The Actor network employs an encoder-decoder structure, forming a graph of all UAVs and mission nodes. The encoder utilizes a heterogeneous graph attention network to capture the complex dependencies between mission nodes and UAVs. Each node interacts with its neighbors through its feature vectors, extracting important features from the local environment. The decoder, using a Transformer structure, helps to autoregressively generate mission planning sequences for the UAVs. The Critic network estimates the state-value function through a fully connected layer connected to the heterogeneous graph attention network, enabling accurate evaluation of the quality of the current policy based on the current state.

[0033] In one feasible implementation, step S10 includes steps A11 to A15: A11: Acquire drone location information, resource information, and battery life information as drone attribute data; It should be noted that the task location information refers to the coordinates of the task target; the resource requirement information refers to the amount of resources required to complete the task; and the task benefit information refers to the benefit value that can be obtained by completing the task.

[0034] A12: Obtain task location information, required resource information, and task revenue information as task attribute data; It should be noted that the task location information refers to the coordinates of the task target; the resource requirement information refers to the amount of resources required to complete the task; and the task benefit information refers to the benefit value that can be obtained by completing the task.

[0035] A13: Determine the distance information between nodes based on UAV location information and mission location information; It should be noted that calculating the spatial distance between the drone and the mission provides a basis for calculating the subsequent flight time.

[0036] A14: Determine the flight time information between nodes based on the distance information between nodes and the preset flight speed; It should be noted that the preset flight speed refers to the average flight speed of the drone when performing a mission. The time required for the drone to fly from one node to another is calculated by dividing the distance by the speed.

[0037] A15: Combine UAV attribute data, mission attribute data, distance information between nodes, and flight time information between nodes to construct a heterogeneous graph.

[0038] It should be noted that by integrating the above information, a complete heterogeneous graph structure is formed, in which drones and missions are treated as nodes, and the distance between nodes and flight time are treated as attributes of the edges.

[0039] Step S20: Perform node-level attention and semantic-level attention on the heterogeneous graph along the preset meta-path to obtain a unified graph representation; It should be noted that a dual attention mechanism at the node and semantic levels is used to process heterogeneous graphs, extracting feature information and generating a unified graph representation. Node-level attention focuses on the relationships between individual nodes and their neighbors, while semantic-level attention focuses on feature representations at different semantic levels. Specifically, UAV node features are determined based on UAV attribute data, and task node features are determined based on task attribute data. These features are then transformed and mapped to a unified feature space. Neighboring nodes are traversed along five predefined meta-paths, node-level attention weights are calculated, and the neighboring features are weighted and aggregated to obtain node-level embeddings. Semantic-level attention processing is applied to the node-level embeddings of the same node under different meta-paths to obtain the final node embeddings. Global average pooling is applied to all final node embeddings to obtain graph-level embeddings. Finally, the graph-level embeddings and the final node embeddings are concatenated to form a unified graph representation, providing a comprehensive feature representation for task planning.

[0040] In the specific implementation, the encoder network structure input is a heterogeneous graph containing UAV nodes and task nodes. For each type of node, an initial node embedding vector is first obtained using a separate fully connected layer. Then, the data is normalized using a BatchNormalization (BN) layer, which reduces the network's sensitivity to parameter initialization, making network training easier and more stable. The calculation formula before inputting HAN is as follows: (1) (2) (3) in This represents the initial node embedding vector before inputting HAN, where U and T represent the UAV node and the mission node, respectively. It is the initial edge embedding vector. and These are learnable weight parameters for fully connected layers.

[0041] The input is then processed through HAN, a two-layer attention architecture consisting of Node-Level Attention and Semantic-Level Attention. This paper's heterogeneous graph attention network considers edge features. In HAN, edge features are not merely bridges connecting nodes, but also participate as key information in the updating and aggregation of node features, enhancing the network's ability to understand node relationships and thus improving model performance and decision quality. Specifically, when aggregating neighbor node information in the Node-Level Attention layer, edge features are combined with node features, as shown in the attention weight calculation formula. This paper adds edge features as an additional input, allowing the attention weights to reflect the influence of edge features. Then, in the meta-path-based aggregation process, edge features are passed along with the meta-paths. Each iteration of node feature updates considers the feature information from the previous edge. First, the initial node embedding vector and edge embedding vector are input into Node-Level Attention, defining a series of meta-paths. (This can be understood as a path connecting different types of nodes. For example, UTU represents the path from the drone to the mission and back to the drone. This article defines the following 5 types of meta-paths:) UAV-Task-UAV (UTU): Describes an indirect connection between two drones through a single task, which can be used to capture opportunities for collaboration between different drones.

[0042] Task-UAV-Task (TUT): Describes an indirect connection between two tasks via a single UAV. It can be used to capture competition between tasks and to balance the load on the UAV.

[0043] UAV-Task (UT): Describes the direct connection between a drone performing a task. It can be used to capture the direct match between the drone and the task.

[0044] Task-Task (TT): Describes the direct relationship between tasks (e.g., the spatial distance between tasks). It can be used to optimize task allocation strategies and improve overall task completion efficiency.

[0045] UAV-UAV (UU): Describes the direct communication or collaboration between drones (e.g., communication or collaboration between drones). It can be used to optimize collaboration between drones and improve the overall efficiency of mission completion.

[0046] These metapaths help the model understand and learn the complex relationships and interactions between drones and tasks, thereby optimizing task allocation and path planning strategies. HAN and Transformer structures can capture the node and relationship features inherent in these metapaths, while the Critic is used to evaluate the state-value function of the entire system, guiding the Actor network to generate optimal action strategies and capturing the relationships between different types of nodes. Due to the heterogeneity of nodes, different types of nodes have different feature spaces. Therefore, for each type of node, a type transformation matrix is ​​first applied... Features of different node types are projected into the same feature space. The type-specific transformation matrix is ​​based on the node type, not the node edge type. The projection process is shown in Equation 4: (4) in and and are the original features and projected features of the node, respectively. Node-level attention can handle nodes of any type through type-specific projection operations. Then, for each central node (whether it's a drone or a mission node), its neighboring nodes are traversed along a specified meta-path, and the attention weight of each neighboring node's contribution to the central node is calculated, as shown in Equation 5: (5) in Metapath Connected node pairs ( ) nodes For nodes Importance coefficient This represents a deep neural network that computes node-level attention.

[0047] After obtaining the importance between node pairs based on metapaths, they are normalized using the softmax function to obtain the metapaths. Node-level attention weight coefficients: (6) in Represents a node Meta-path based The set of neighbors (including itself). || represents the activation function, and || represents the join operation. The weight matrix is ​​a learnable matrix. This indicates transpose.

[0048] Next, based on the attention weights, the features and attention coefficients of neighboring nodes are aggregated to generate a feature vector based on the center node of each meta-path: (7) To make the training process more stable, this paper extends node-level attention to multi-head attention. Specifically, node-level attention is repeated K times, and the learned embeddings are concatenated into semantically specific embeddings: (8) Then, the feature vectors of each type of node output from the Node-Level Attention layer, based on different meta-paths, are input into the Semantic-Level Attention layer. First, the weights of each meta-path are calculated: (9) This represents a semantic-level deep neural network. It involves weighted summation of feature vectors based on different meta-paths to obtain the final node embeddings. : (10) After HAN, each node (drone or mission) will receive a final feature vector that combines information from all its relevant neighbor nodes and aggregates it through a dual attention mechanism. Finally, these final feature vectors are used to calculate the final graph embedding: (11) The aggregation process of a single-layer HAN is as follows: the final feature vectors and graph embedding vectors of all nodes output by the encoder are input into the Transformer-based decoder. This application improves upon the Transformer based on the Pointer Network (PtrNet) proposed by Vinyals et al. Because the output dimension of the Transformer is fixed in advance and cannot change according to the input dimension, it cannot be directly applied to solve path planning problems. The Pointer Network, however, uses an attention mechanism to select a member from the input sequence based on the softmax probability distribution as the output of each decoding step, which perfectly solves this problem. The length of the output sequence is determined by the source sequence. Based on this, the decoder in this paper follows the PtrNet approach to output nodes, where each node uses the softmax probability distribution and its associated probability value as a "pointer" at each decoding time step; that is, each node corresponds to a probability value. Unlike the original Transformer proposed by Google, the decoder model in this paper mainly adopts the multi-head attention mechanism of the Transformer architecture decoder part. The decoder's workflow is roughly as follows: first, the decoder network generates the probability distribution of nodes that have not yet been selected based on Multi-head Attention; then, it uses Mask nodes to handle the constraints of the problem, filtering out nodes that do not meet the constraints. Finally, a greedy search decoding or sampling search decoding strategy is adopted to select nodes for output based on the probability distribution corresponding to the nodes output by the second layer.

[0049] At each step of the decoding process, the decoder first uses Multi-Head Attention to compute the context vector of the current state. This vector integrates the position, state, and relevant information of all drones and unassigned task nodes. The first input is the context vector of the Multi-Head Attention. Graph embedding vectors The node selected at the previous moment The embedding vector and the number of remaining missiles for each drone and remaining flight time The calculation yielded: in Indicates decoding step t (drone) The remaining number of missiles, Indicates decoding step t (drone) The remaining flight time, It is a learnable weight matrix, and and The update formula is as follows: Then, calculations are performed using Multi-Head Attention. , where the query vector From context vector The calculation yielded: The key vector and value vector of a node are both calculated from the node embedding vector output by the encoder: Among them W Q W k W v All are learnable weight matrices.

[0050] By query vector and key vector Calculate the attention coefficients of the Multi-Head Attention layer If there are no restrictions Then default Dividing by This helps maintain attention scores at different locations within a relatively focused and stable dynamic range, thereby improving the model's learning efficiency and performance. Indicates drone The number of times it was selected This indicates the previously selected drone.

[0051] This paper uses a masking mechanism to control the feasibility of the network output solution, specifically reflected in the attention coefficient calculation formula. The attention coefficients of task nodes visited before time t, drones with less remaining resources than the required resources for the task to be visited, and drones with insufficient remaining flight time (insufficient flight time to fly to the task node to be visited and support the drone's return to the starting point) are set to -∞ to mask these scenarios. Furthermore, if no drone is selected at the current time or the current drone returns to the starting point to replenish its flight time, i.e., when all k are traversed... Then it is necessary to decide which drone to use for the mission. Therefore, when All nodes except the currently executing drone node are disabled, including the drone node that last performed the task. Furthermore, to ensure that drones must return to the starting point to replenish resources or sustain insufficient power, all other drone nodes except the one currently performing the task are disabled after a task is selected. These measures ensure the feasibility of the solution output by the model.

[0052] Then, normalize the attention coefficients: The attention coefficient calculated for the h-th head is expressed as: Then, the attention coefficients of each head are merged and concatenated into a fully connected layer to obtain a new context vector. : In one feasible implementation, step S20 includes steps A21 to A26: A21: Determine UAV node characteristics based on UAV attribute data, and determine task node characteristics based on task attribute data; It should be noted that node features refer to key information extracted from attribute data that can characterize the features of a drone or mission.

[0053] A22: Apply type conversion matrices to the UAV node features and task node features respectively to map them to a unified dimension to obtain unified UAV features and unified task features; It should be noted that a type conversion matrix is ​​a mathematical tool used to convert node features of different types to the same dimensional space for subsequent feature aggregation operations.

[0054] A23: Traverse neighbor nodes along five preset meta-paths, calculate node-level attention weights within each meta-path, and aggregate neighbor features in a weighted manner to obtain node-level embeddings; It should be noted that the five preset meta-paths refer to five pre-defined path patterns based on the relationship between the drone and the mission, used to guide path selection during traversal of heterogeneous graphs. Node-level attention weights reflect the correlation between the current node and its neighboring nodes; a higher weight indicates a stronger correlation. Weighted aggregation of neighbor features refers to the weighted summation of the features of neighboring nodes based on the attention weights, highlighting the influence of important neighboring nodes. Node-level embeddings are feature vectors obtained after weighted aggregation, representing the relationship between the current node and its neighboring nodes.

[0055] Further, step A23 includes the following steps: For each preset meta-path, determine the set of neighboring nodes of the current node; Extract the features of the current node and the features of each set of neighboring nodes; Calculate the similarity between the current node and each of its neighboring nodes, and use it as the initial attention weight; The initial attention weights are normalized to obtain the normalized attention weights. The features of each set of neighbor nodes are weighted and summed using the normalized attention weights to obtain the weighted aggregated neighbor features. The weighted aggregated neighbor features are concatenated with the current node's features to obtain the node-level embedding of the current node under the current metapath.

[0056] It should be noted that the preset meta-path refers to a path pattern pre-defined based on the relationship between the drone and the mission, used to guide path selection during traversal of the heterogeneous graph. The set of neighboring nodes of the current node refers to all nodes directly connected to the current node under this meta-path. For example, if the meta-path is "drone-mission-drone", then the set of neighboring nodes of the current drone node is all the mission nodes related to it and other drone nodes connected through the mission nodes.

[0057] A node's features are vectors that characterize the node's properties, containing various attribute information. Feature extraction involves representing this attribute information as vectors for subsequent calculations.

[0058] Similarity calculation is used to measure the relevance between the current node and its neighboring nodes. The initial attention weight reflects the importance of neighboring nodes to the current node; the larger the weight, the greater the influence of the neighboring node on the current node.

[0059] Normalization adjusts the initial attention weights to a fixed range (usually between 0 and 1), ensuring that the sum of the weights is 1. This step helps to reasonably distribute the influence of each neighbor node's features in the subsequent weighted summation process.

[0060] Weighted summation involves multiplying the feature vector of each neighbor node by its corresponding normalized attention weight, and then summing all the results. This step highlights the features of important neighbors while suppressing the features of unimportant neighbors.

[0061] The concatenation process involves joining the weighted aggregated neighbor feature vectors with the current node's feature vector to form a new vector. This new vector is the node-level embedding of the current node within the current meta-path. It integrates the current node's own features with those of its neighboring nodes, providing a more comprehensive representation of the current node's characteristics within a specific meta-path.

[0062] A24: Perform semantic attention processing on the node-level embeddings of the same node under different meta-paths to obtain the final node embedding; It should be noted that semantic attention processing refers to further integrating multiple node-level embeddings obtained from the same node under different meta-paths. By calculating the correlation between embeddings under different meta-paths, assigning different weights, and then performing weighted summation, a final embedding vector that can comprehensively reflect the features of the node at different semantic levels is obtained.

[0063] A25: Perform global average pooling on all final node embeddings to obtain graph-level embeddings; It should be noted that global average pooling is a method for statistically summarizing features. By averaging the final embedding vectors of all nodes, a global feature vector that can represent the entire graph is obtained.

[0064] A26: By combining the graph-level embedding with the embedding of each final node, a unified graph representation is obtained.

[0065] It should be noted that by combining global features with the local features of each node, a complete unified graph representation is formed, which includes information about the overall graph while retaining the detailed features of each node.

[0066] Step S30: The unified graph representation and the selected nodes from the previous time step are concatenated into a context vector. The context vector is then input into the transformer-based decoder for autoregressive processing to obtain the task node sequence. It should be noted that the task node sequence is generated through autoregressive processing. Autoregressive processing is a method that progressively constructs the task node sequence based on selected nodes. Specifically, the unified graph representation and the embedded nodes of the selected nodes are concatenated into a context vector, which is then input into the transformer decoder. The transformer decoder uses a multi-head attention mechanism to determine the probability distribution of the unselected task nodes, selects the current task node based on this distribution, and adds it to the selected list. This process is repeated until all task nodes have been added to the list, ultimately resulting in a complete task node sequence.

[0067] In the specific implementation, the context vector is then... The second layer of the input decoder is a self-attention mechanism that generates a probability distribution for each incomplete task node. This distribution represents the probability that each node will be selected as the next task to be executed. This probability distribution is pruned by the tanh activation function. Attention coefficient at any moment Obtain and crop the result. Inside: Finally, based on the probability distribution The next node to be visited is selected using either a sampling or greedy decoding strategy. Specifically, in this study, to train a better model performance, a random sampling strategy is first used to explore the environment. At each decoding time step, the node to be visited is sampled according to the probability distribution to construct an effective solution. A temperature hyperparameter is introduced. To ensure diversity, samples are used: This strategy helps the model learn to explore new states and actions, avoiding fixation on a specific behavioral strategy. By continuously sampling randomly and updating the strategy, the model gradually improves its performance during exploration. During testing, a greedy strategy is employed to achieve better results. The greedy algorithm selects the optimal action based solely on the current state, choosing the action with the highest probability from all possible actions to maximize cumulative reward. This strategy works very well in deterministic environments but performs poorly when exploring unknown states. Therefore, a random sampling strategy is used for exploration during training, while a greedy strategy is used during testing to achieve better results. In summary, different strategies are employed during the training and testing of this model to achieve better performance, while comprehensively considering the balance between exploration and exploitation, making the model more robust and reliable in real-world environments.

[0068] In one feasible implementation, step S30 includes steps A31 to A34: A31: Embed and concatenate the unified graph representation with the selected task node from the previous time step to obtain the current context vector; It should be noted that embedding concatenation refers to connecting the unified graph representation vector with the embedding vector of the selected task node to form a new vector, which serves as the context information at the current moment.

[0069] A32: Determine the probability distribution of unselected task nodes based on the current context vector using a multi-head attention mechanism; It should be noted that multi-head attention is an attention model that can simultaneously focus on information from multiple different locations. By calculating the correlation between the current context vector and the unselected task nodes, the probability distribution of each unselected task node being selected is obtained.

[0070] A33: Select the current task node based on probability distribution; It should be noted that, based on the magnitude of the probability distribution, the task node with the highest probability is selected as the task to be executed.

[0071] A34: Add the current task node to the selected list and continue the next autoregressive iteration until all task nodes have been added to the selected list, thus obtaining the task node sequence.

[0072] It should be noted that autoregressive iteration refers to updating the selected list based on the selection results at the current moment, and then making the next selection based on the updated list, until all tasks are selected, ultimately forming a complete task execution sequence.

[0073] Step S40: Determine the immediate reward based on the task node sequence and update the deep reinforcement learning policy network based on the immediate reward until the policy converges to obtain the target task planning result. The target task planning result includes the task execution order, take-off and landing base, and flight path of each UAV.

[0074] It should be noted that immediate reward refers to an indicator calculated based on the current task node sequence, used to measure the quality of the current task planning. Deep reinforcement learning policy networks are neural network models based on reinforcement learning. By continuously interacting with the environment and adjusting network parameters based on immediate rewards, they learn the optimal task planning policy. Policy convergence refers to the point where, after multiple iterations, the output of the policy network tends to stabilize and no longer changes significantly. The target task planning result refers to the final determined UAV mission execution plan, including the task sequence to be performed by each UAV, the take-off and landing base locations, and the flight path planning.

[0075] Immediate rewards are determined based on the sequence of task nodes, and these immediate rewards are used to update the deep reinforcement learning policy network until the policy converges. Immediate reward is a metric for measuring the performance of the current task; the deep reinforcement learning policy network adjusts its policy based on this reward to optimize long-term task planning. Ultimately, we obtain a target task planning result that includes the task execution order, takeoff and landing base, and flight path for each UAV.

[0076] This embodiment provides a multi-UAV mission planning method, proposing a DRL framework based on a heterogeneous graph attention network model to solve the multi-UAV mission planning problem. This framework is not constrained by the problem size; once the model is trained, it can be applied to problems of different scales (different numbers of UAVs and missions). This paper addresses the UAV mission planning problem where different UAVs have different rewards for accessing the same mission. Mixed-integer programming and Markov decision process modeling are used to model the multi-UAV mission planning problem as a heterogeneous graph. A heterogeneous graph attention network model incorporating edge features is designed as an encoder to encode the heterogeneous graph state information during the solution process. A PPO-based DRL training algorithm is designed to train the proposed HAN-based model. To verify the feasibility and effectiveness of the proposed model, numerical experiments are conducted using randomly generated examples, and the results are compared with current mainstream DRL models for solving combinatorial optimization problems (based on GAT). The computational results show that the proposed model framework is feasible and efficient for solving UAV mission planning problems of different scales.

[0077] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S40 includes steps S401 to S405: Step S401: Determine the total mission revenue, total flight distance, and target single-aircraft flight time based on the mission node sequence; It should be noted that total task revenue refers to the total revenue obtained after completing all tasks, reflecting the economic or strategic value of the tasks.

[0078] Total flight distance refers to the total flight distance required for all UAVs to complete the mission, reflecting the geographical scope and flight cost of the mission.

[0079] The target single-drone flight time refers to the time required for a single drone to perform a mission, reflecting the time cost of the mission and the efficiency of drone utilization.

[0080] Step S402: Based on preset profit weight, distance weight, and time weight, perform weighted summation of total mission profit, total flight distance, and target single-aircraft flight time to obtain the instant reward; It should be noted that the preset reward weight, distance weight, and time weight reflect the relative importance of total mission reward, total flight distance, and target single-aircraft flight time in the immediate reward, respectively. By multiplying these three parameters by their corresponding weights and then summing them, an immediate reward value that comprehensively considers reward, distance, and time is obtained, which is used to measure the quality of the current mission planning.

[0081] In one feasible implementation, step S402 includes steps A41 to A44: A41: Multiply the total task revenue by the preset revenue weight to obtain the first weighted value; It should be noted that this step quantifies the contribution of total task rewards to immediate rewards.

[0082] A42: Multiply the total flight distance by the preset distance weight to obtain the second weighted value; It should be noted that this step quantifies the contribution of total flight distance to the instant rewards.

[0083] A43: Multiply the target single-aircraft flight time by the preset time weight to obtain the third weighted value; It should be noted that this step quantifies the contribution of the target's single-machine flight time to the immediate rewards.

[0084] A44: Add the first weighted value, the second weighted value, and the third weighted value to get the instant reward.

[0085] It should be noted that by adding the three weighted values ​​together, a comprehensive immediate reward value is obtained, which is used to evaluate the overall effectiveness of the current task planning.

[0086] Step S403: Input the immediate reward into the value network and calculate the advantage function using the generalized advantage estimation algorithm; It should be noted that a value network is a neural network model used to estimate the expected value of the cumulative reward obtainable in the future under the current task planning strategy. The generalized advantage estimation algorithm calculates the advantage function of the current task planning action by comparing the current immediate reward with the cumulative reward estimated by the value network, that is, the superiority or inferiority of the current action relative to the average action.

[0087] Step S404: Calculate the truncated policy loss based on the advantage function using a preset near-end policy optimization algorithm, and update the policy network parameters based on the policy loss. It should be noted that the proximal policy optimization algorithm is a reinforcement learning algorithm that stabilizes the policy learning process by limiting the magnitude of policy updates. The truncated policy loss is a loss function used to measure the difference between the current policy and the old policy, preventing drastic fluctuations during policy updates. Updating the policy network parameters based on the policy loss allows for gradual optimization of the task planning policy, improving its stability and convergence speed.

[0088] Step S405: Synchronize the old policy network parameters to the policy network parameters, iterate until the policy converges, and obtain the target mission planning results including the mission execution order, take-off and landing bases, and flight path.

[0089] It should be noted that synchronizing the old policy network parameters to the new policy network parameters is to use the updated parameters in the next iteration. Iterating until policy convergence means repeating the above steps until the output of the policy network stabilizes and no longer changes significantly. The final target mission planning result includes the mission execution sequence, take-off and landing base, and flight path for each UAV, providing a detailed guidance for mission execution of multi-UAV systems.

[0090] In specific implementations, such as Figure 3 The flowchart shown is for the PPO-based reinforcement learning training algorithm. This paper designs a PPO-based deep reinforcement learning algorithm to train the proposed HAN-Transformer model for the multi-UAV mission planning problem. Let the Actor network parameter set be... The Critic network parameter set is The Actor network in this paper adopts an encoder-decoder structure. The encoder is mainly based on a heterogeneous graph attention network structure, and the decoder mainly adopts a Transformer structure, which outputs the policy in the current state. The Critic network's main structure is a heterogeneous graph attention network followed by fully connected layers, and its output is the value function in the current state. The Actor and Critic heterogeneous graph attention networks do not share parameters. The objective function of the Actor network is defined using the PPO-truncation form: in Indicates state, Indicates an action, Indicates the current strategy. This is the set of parameters for the current policy network. This represents the set of network parameters for the new policy round. This represents the set of network parameters for the previous policy round. Indicates to Limited to Inside, `<clipping>` is a hyperparameter representing the range of clipping. The advantage function... , The discount factor has a value range of [0,1). A value closer to 1 indicates a greater focus on long-term cumulative rewards, while a value closer to 0 indicates a greater focus on short-term rewards.

[0091] In the programming implementation, the generalized advantage estimation (GAE) advantage function is used. : in , It is an additional hyperparameter introduced in GAE. When "When" indicates that only one step of difference is considered; when This represents the perfect average of the advantages obtained from each difference step. Then, the parameters of the Actor network are updated according to the policy gradient principle. Similarly, the loss function of the Critic network is first defined: This paper adopts a method similar to that of target networks, and incorporates the above formula into the... As a temporal difference objective, it does not generate gradients to update the value function. Therefore, the gradient of the value function is: Then, gradient descent is used to update the Critic value network parameters.

[0092] The PPO overall training framework is as follows: In the training phase, after inputting training data, the HAN encoder first encodes the heterogeneous graph composed of the UAV and the mission into unified graph embeddings and node embeddings. Then, the Transformer decoder autoregressively generates a sequence of mission nodes based on this representation. The generated policy (Actor) is parameterized by the HAN-Transformer, and the value network (Critic) estimates the state value function based on the HAN with a fully connected layer. Both are updated synchronously through the PPO optimizer until the policy converges. In the testing phase, the trained model is directly deployed. Inputting new test data quickly outputs a planning scheme containing the mission allocation order and flight path, without the need for retraining.

[0093] This embodiment provides a multi-UAV mission planning method. Based on constructing a heterogeneous graph and extracting feature information, it utilizes a deep reinforcement learning mechanism to continuously optimize the mission planning strategy, ultimately generating a scientifically sound mission planning scheme. Specifically, this method fully considers various constraints between UAVs and missions, ensuring the accuracy and reliability of mission planning through real-time reward calculation and policy network updates. The final target mission planning result includes not only the mission execution order of each UAV but also the takeoff and landing bases and detailed flight paths, demonstrating good application prospects and development potential.

[0094] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the multi-UAV mission planning method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0095] This application also provides a multi-UAV mission planning device; please refer to... Figure 4 The multi-UAV mission planning device includes: Data acquisition module 10 is used to acquire UAV attribute data and mission attribute data, and construct a heterogeneous graph based on the UAV attribute data and mission attribute data; Attention processing module 20 is used to perform node-level attention and semantic-level attention on heterogeneous graphs along preset meta-paths to obtain a unified graph representation; Decoding module 30 is used to concatenate the unified graph representation with the selected nodes from the previous time step into a context vector, and input the context vector into the transformer-based decoder for autoregressive processing to obtain the task node sequence. The policy update module 40 is used to determine the immediate reward based on the task node sequence and update the deep reinforcement learning policy network based on the immediate reward until the policy converges, thereby obtaining the target task planning result. The target task planning result includes the task execution order, take-off and landing base, and flight path of each UAV.

[0096] The multi-UAV mission planning device provided in this application, employing the multi-UAV mission planning method described in the above embodiments, can solve the technical problem of balancing mission benefit flight distance and flight time while achieving efficient solution in large-scale heterogeneous multi-UAV mission planning. Compared with the prior art, the beneficial effects of the multi-UAV mission planning device provided in this application are the same as those of the multi-UAV mission planning method provided in the above embodiments, and other technical features in the multi-UAV mission planning device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0097] In one embodiment, the data acquisition module 10 is further configured to acquire UAV location information, resource information, and battery life information as UAV attribute data; Obtain task location information, required resource information, and task revenue information as task attribute data; Determine the distance information between nodes based on the location information of the UAV and the location information of the mission; The flight time information between nodes is determined based on the distance information between nodes and the preset flight speed. By stitching together drone attribute data, mission attribute data, distance information between nodes, and flight time information between nodes, a heterogeneous graph is constructed.

[0098] In one embodiment, the attention processing module 20 is further configured to determine drone node features based on drone attribute data and to determine task node features based on task attribute data. Type conversion matrices are applied to the UAV node features and task node features respectively to map them to a unified dimension, resulting in unified UAV features and unified task features. The neighbor nodes are traversed along five preset meta-paths. Node-level attention weights are calculated in each meta-path, and neighbor features are aggregated in a weighted manner to obtain node-level embeddings. Semantic attention processing is performed on the node-level embeddings of the same node under different meta-paths to obtain the final node embedding; Global average pooling is applied to all final node embeddings to obtain graph-level embeddings. By combining the graph-level embeddings with the embeddings of each final node, a unified graph representation is obtained.

[0099] In one embodiment, the attention processing module 20 is further configured to determine the set of neighboring nodes of the current node for each preset meta-path; Extract the features of the current node and the features of each set of neighboring nodes; Calculate the similarity between the current node and each of its neighboring nodes, and use it as the initial attention weight; The initial attention weights are normalized to obtain the normalized attention weights. The features of each set of neighbor nodes are weighted and summed using the normalized attention weights to obtain the weighted aggregated neighbor features. The weighted aggregated neighbor features are concatenated with the current node's features to obtain the node-level embedding of the current node under the current metapath.

[0100] In one embodiment, the decoding module 30 is further configured to embed and concatenate the unified graph representation with the previously selected task node to obtain the current context vector; The probability distribution of unselected task nodes is determined based on the current context vector using a multi-head attention mechanism; Select the current task node based on probability distribution; Add the current task node to the selected list and continue the next autoregressive iteration until all task nodes have been added to the selected list, thus obtaining the task node sequence.

[0101] In one embodiment, the strategy update module 40 is further configured to determine the total task revenue, total flight distance, and target single-aircraft flight time based on the task node sequence; The total mission revenue, total flight distance, and target single-aircraft flight time are weighted and summed based on preset revenue weights, distance weights, and time weights to obtain the instant reward. The immediate reward is input into the value network and the advantage function is obtained by using a generalized advantage estimation algorithm. The policy loss in truncated form is calculated based on the advantage function using a pre-defined near-end policy optimization algorithm, and the policy network parameters are updated based on the policy loss. Synchronize the old policy network parameters to the new policy network parameters, iterate until the policy converges, and obtain the target mission planning results, which include the mission execution order, take-off and landing bases, and flight paths.

[0102] In one embodiment, the strategy update module 40 is further configured to multiply the total task revenue by a preset revenue weight to obtain a first weighted value; Multiply the total flight distance by the preset distance weight to obtain the second weighted value; The third weighted value is obtained by multiplying the target single-aircraft flight time by a preset time weight; The first weighted value, the second weighted value, and the third weighted value are added together to obtain the instant reward.

[0103] This application provides a multi-UAV mission planning device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the multi-UAV mission planning method in the above embodiment 1.

[0104] The following is for reference. Figure 5 The diagram illustrates a structural schematic of a multi-UAV mission planning device suitable for implementing embodiments of this application. The multi-UAV mission planning device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The multi-UAV mission planning device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0105] like Figure 5 As shown, the multi-UAV mission planning device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the multi-UAV mission planning device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the multi-UAV mission planning device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows a multi-UAV mission planning device with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0106] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0107] The multi-UAV mission planning device provided in this application, employing the multi-UAV mission planning method described in the above embodiments, can solve the technical problem of balancing mission benefit flight distance and flight time while achieving efficient solution in large-scale heterogeneous multi-UAV mission planning. Compared with the prior art, the beneficial effects of the multi-UAV mission planning device provided in this application are the same as those of the multi-UAV mission planning method provided in the above embodiments, and other technical features of this multi-UAV mission planning device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0108] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0109] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0110] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the multi-UAV mission planning method described in the above embodiments.

[0111] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0112] The aforementioned computer-readable storage medium may be included in the multi-UAV mission planning device; or it may exist independently and not be assembled into the multi-UAV mission planning device.

[0113] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a multi-UAV mission planning device, the multi-UAV mission planning device: acquires UAV attribute data and mission attribute data; constructs a heterogeneous graph based on the UAV attribute data and mission attribute data; performs node-level attention and semantic-level attention on the heterogeneous graph along a preset meta-path to obtain a unified graph representation; concatenates the unified graph representation with the previously selected nodes to form a context vector; inputs the context vector into a transformer-based decoder for autoregressive processing to obtain a task node sequence; determines an immediate reward based on the task node sequence and updates the deep reinforcement learning policy network based on the immediate reward until the policy converges to obtain the target task planning result.

[0114] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0116] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0117] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described multi-UAV mission planning method. This solves the technical problem of balancing mission benefits (flight distance and flight time) and achieving efficient solution in large-scale heterogeneous multi-UAV mission planning. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the multi-UAV mission planning method provided in the above embodiments, and will not be elaborated upon here.

[0118] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the multi-UAV mission planning method described above.

[0119] The computer program product provided in this application can solve the technical problem of how to balance mission benefits, flight distance, and flight time in large-scale heterogeneous multi-UAV mission planning and achieve efficient solution. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the multi-UAV mission planning method provided in the above embodiments, and will not be repeated here.

[0120] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A multi-UAV mission planning method, characterized in that, The method includes: Acquire drone attribute data and mission attribute data, and construct a heterogeneous graph based on the drone attribute data and mission attribute data; Node-level attention and semantic-level attention are performed on the heterogeneous graph along a preset meta-path to obtain a unified graph representation; The unified graph representation is concatenated with the selected nodes from the previous time step to form a context vector. The context vector is then input into a transformer-based decoder for autoregressive processing to obtain the task node sequence. Based on the task node sequence, an immediate reward is determined, and the deep reinforcement learning policy network is updated based on the immediate reward until the policy converges, thereby obtaining the target task planning result. The target task planning result includes the task execution order, take-off and landing base, and flight path of each UAV.

2. The method as described in claim 1, characterized in that, The steps of acquiring UAV attribute data and mission attribute data, and constructing a heterogeneous graph based on the UAV attribute data and mission attribute data, include: Acquire drone location information, resource information, and battery life information as drone attribute data; Obtain task location information, required resource information, and task revenue information as task attribute data; The distance information between nodes is determined based on the UAV location information and the mission location information; The flight time information between nodes is determined based on the distance information between the nodes and the preset flight speed. By combining the UAV attribute data, the mission attribute data, the distance information between nodes, and the flight time information between nodes, a heterogeneous graph is constructed.

3. The method as described in claim 1, characterized in that, The step of performing node-level attention and semantic-level attention on the heterogeneous graph along a preset meta-path to obtain a unified graph representation includes: The characteristics of the drone node are determined based on the drone attribute data, and the characteristics of the task node are determined based on the task attribute data. Type conversion matrices are applied to the UAV node features and the task node features respectively to map them to a unified dimension, resulting in unified UAV features and unified task features. The neighbor nodes are traversed along five preset meta-paths. Node-level attention weights are calculated in each meta-path, and neighbor features are aggregated in a weighted manner to obtain node-level embeddings. Semantic attention processing is performed on the node-level embeddings of the same node under different meta-paths to obtain the final node embedding; Global average pooling is applied to all final node embeddings to obtain graph-level embeddings. By concatenating the graph-level embeddings with the final node embeddings, a unified graph representation is obtained.

4. The method as described in claim 3, characterized in that, The step of traversing neighbor nodes along five preset meta-paths, calculating node-level attention weights within each meta-path, and weighted aggregating neighbor features to obtain node-level embeddings includes: For each preset meta-path, determine the set of neighboring nodes of the current node; Extract the features of the current node and the features of each of the neighbor node sets; Calculate the similarity between the current node and each of its neighboring nodes, and use it as the initial attention weight; The initial attention weights are normalized to obtain normalized attention weights; The features of each set of neighbor nodes are weighted and summed using the normalized attention weights to obtain the weighted aggregated neighbor features. The weighted aggregated neighbor features are concatenated with the features of the current node to obtain the node-level embedding of the current node under the current metapath.

5. The method as described in claim 1, characterized in that, The step of concatenating the unified graph representation with the previously selected node embeddings to form a context vector, and inputting the context vector into the transformer decoder for autoregressive processing to obtain the task node sequence includes: The unified graph representation is embedded and concatenated with the selected task node from the previous time step to obtain the current context vector; Based on the current context vector, the probability distribution of unselected task nodes is determined through a multi-head attention mechanism; The current task node is selected based on the probability distribution; The current task node is added to the selected list, and the next autoregressive iteration continues until all task nodes are added to the selected list, thus obtaining the task node sequence.

6. The method as described in claim 1, characterized in that, The steps of determining the immediate reward based on the task node sequence and updating the deep reinforcement learning policy network based on the immediate reward until the policy converges to obtain the target task planning result include: The total mission revenue, total flight distance, and target single-aircraft flight time are determined based on the mission node sequence. The total mission revenue, the total flight distance, and the target single-aircraft flight time are weighted and summed based on preset revenue weights, distance weights, and time weights to obtain the instant reward. The immediate reward is input into the value network and the advantage function is obtained by using a generalized advantage estimation algorithm. The policy loss in truncated form is calculated based on the advantage function using a preset near-end policy optimization algorithm, and the policy network parameters are updated based on the policy loss. Synchronize the old policy network parameters to the policy network parameters, iterate until the policy converges, and obtain the target mission planning result including the mission execution order, take-off and landing bases, and flight path.

7. The method as described in claim 6, characterized in that, The step of performing a weighted summation of the total mission revenue, the total flight distance, and the target single-aircraft flight time based on preset revenue weights, distance weights, and time weights to obtain the instant reward includes: Multiply the total task revenue by the preset revenue weight to obtain the first weighted value; Multiply the total flight distance by the preset distance weight to obtain the second weighted value; The target single-aircraft flight time is multiplied by a preset time weight to obtain a third weighted value; The first weighted value, the second weighted value, and the third weighted value are added together to obtain the instant reward.

8. A multi-UAV mission planning device, characterized in that, The device includes: The data acquisition module is used to acquire UAV attribute data and mission attribute data, and construct a heterogeneous graph based on the UAV attribute data and mission attribute data; The attention processing module is used to perform node-level attention and semantic-level attention on the heterogeneous graph along a preset meta-path to obtain a unified graph representation; The decoding module is used to concatenate the unified graph representation with the selected nodes from the previous time step into a context vector, and input the context vector into the transformer-based decoder for autoregressive processing to obtain the task node sequence. The policy update module is used to determine the immediate reward based on the task node sequence and update the deep reinforcement learning policy network based on the immediate reward until the policy converges, thereby obtaining the target task planning result. The target task planning result includes the task execution order, take-off and landing base, and flight path of each UAV.

9. A multi-UAV mission planning device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the multi-UAV mission planning method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the multi-UAV mission planning method as described in any one of claims 1 to 7.

Images