Path planning method, device and equipment for unmanned aerial vehicle cluster and storage medium

Abstract

The invention provides a path planning method, device and equipment for an unmanned aerial vehicle cluster and a storage medium, and the method comprises the steps: carrying out the preprocessing of motion data of the unmanned aerial vehicle cluster with timestamp information, and determining a motion vector of each unmanned aerial vehicle node in the unmanned aerial vehicle cluster; inputting the motion vector of the current unmanned aerial vehicle node, the one-hop neighbor node information of the current unmanned aerial vehicle node and the two-hop neighbor node information of the current unmanned aerial vehicle node into an unmanned aerial vehicle cluster path planning model for spatial convolutional coding processing, time domain cyclic coding processing and decision action selection processing to determine an optimal decision action; and continuing to process the next target node selected by the decision action until the next target node is a termination node, and outputting the optimal path formed by all the selected target nodes, thereby remarkably improving the accuracy of path selection and the routing convergence speed.

Description

Technical Field

[0001] The present application relates to the field of path planning technology, and in particular to a path planning method, apparatus, device and storage medium for a drone cluster. Background Art

[0002] The existing routing optimization process is generally divided into node motion prediction and routing message distribution. The first step is node motion prediction. Each node initializes its velocity, position, and other motion data and maintains a local routing knowledge base to store the routing information of its local neighbors (including velocity, position, and link quality). If the current node has past trajectory information, an iterative method is generally used to predict future motion data; otherwise, other non-iterative methods are used to calculate and estimate future node positions. However, existing routing optimization methods fail to fully understand the real-time evolution of the network topology, resulting in inaccurate path determination for drone swarms. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a path planning method, device, equipment and storage medium for a drone cluster, which uses spatial convolutional coding processing to enable each node to obtain richer network topology information, significantly improve the accuracy of path selection and routing convergence speed, and time domain cyclic coding processing can capture time series characteristics in real time, thereby adjusting path planning in advance.

[0004] The present invention provides a method for planning a path for a drone swarm. The method includes: Preprocess the motion data of the drone cluster with timestamp information to determine the motion vectors of the drone nodes in the drone cluster; The motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information of the current drone node are input into the drone cluster path planning model for spatial convolution coding, time domain cyclic coding, and decision action selection processing to determine the optimal decision action. The next target node selected by the decision action is processed until the next target node is the terminal node, and the processing is stopped. The optimal path consisting of all selected target nodes is output; Among them, the one-hop neighbor node is a node that directly communicates with the current drone node, and the two-hop neighbor node is a node that directly communicates with the one-hop neighbor node. The drone cluster path planning model is obtained by iterative training of the deep learning model.

[0005] In one possible implementation, the motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information are input into the drone cluster path planning model to perform spatial convolution coding processing, time domain cyclic coding processing, and action space decision processing to determine the optimal decision action, including: The spatial convolutional coding network layer based on the UAV cluster path planning model performs spatial convolution coding on the motion vector of the current UAV node, the one-hop neighbor node information of the current UAV node, and the two-hop neighbor node information, and outputs the environmental spatial features of the current UAV node; Based on the time-domain recurrent network layer of the UAV cluster path planning model, the environmental spatial features are captured and processed with the preceding temporal and spatial features to determine the local observation vector of the current UAV node; wherein the local observation vector includes the motion vector of the current UAV node, one-hop neighbor node information, two-hop neighbor node information, and environmental information; The value network layer of the UAV cluster path planning model performs action space decision processing on the local observation vector and outputs the optimal decision action.

[0006] In one possible implementation, the spatial convolution coding network layer based on the UAV cluster path planning model performs spatial convolution coding processing on the motion vector of the current UAV node, the one-hop neighbor node information of the current UAV node, and the two-hop neighbor node information, and outputs the environmental spatial features of the current UAV node, including: Aggregate the one-hop neighbor node information and the two-hop neighbor node information of the current drone node to determine the aggregated one-hop neighbor node information and the aggregated two-hop neighbor node information; The aggregated one-hop neighbor node information, the aggregated two-hop neighbor node information, and the motion vector of the UAV are subjected to spatial convolution coding processing to determine the environmental spatial features.

[0007] In one possible implementation, the value network layer based on the UAV cluster path planning model performs action space decision processing on the local observation vector and outputs the optimal decision action, including: Processing the local observation vector and action space based on the action-value function to output a Q value corresponding to each decision action; wherein the action space is a combination of the next hop node and the power allocation; Based on the greedy strategy, a target Q value is screened out from the Q values corresponding to multiple decision actions, and the decision action corresponding to the target Q value is taken as the optimal decision action.

[0008] In one possible implementation, the UAV cluster path planning model is determined by the following steps: Input the motion vector of the current sample drone node, the one-hop neighbor node information of the current sample drone node, and the two-hop neighbor node information of the current sample drone node into the deep learning model for processing to determine the sample local observation vector, perform action space decision processing on the local observation vector, and output the sample decision action; Determine the reward value of the next-hop sample drone node selected by the sample decision action based on the reward mechanism; Determining a loss value of the deep learning model based on the reward value, the sample local observation vector, and the sample decision action; Based on the loss value, the network parameters of the spatial convolutional coding network layer, the time domain recurrent network layer and the value network layer of the deep learning model are updated until the loss value is less than or equal to a preset threshold, and the updating of the network parameters is stopped to determine the drone cluster path planning model.

[0009] In one possible implementation, determining the reward value of the next-hop sample drone node selected by the sample decision action based on a reward mechanism includes: Determine the path progress reward value of the next-hop sample drone node based on the three-dimensional coordinate information of the next-hop sample drone node, the three-dimensional coordinate information of the end node, the three-dimensional coordinate information of the start node, the forward bias parameter, and the three-dimensional speed information of the next-hop sample drone node; Determine the channel quality reward value based on the link signal interference and noise ratio between the current sample drone node and the next hop sample drone node and the preset link signal interference and noise ratio threshold; Based on the sum of the path progress reward value and the channel quality reward value, a reward value of the next-hop sample drone node is determined.

[0010] In a possible implementation manner, the link signal to interference and noise ratio is determined in the following manner: Based on the transmission power of the current sample drone node, the channel gain between the current sample drone node and the next-hop sample drone node, the Gaussian white noise value, the transmission power of other sample drone nodes, and the channel gain between other sample drone nodes and the next-hop sample drone node, the link signal interference noise ratio between the current sample drone node and the next-hop sample drone node is determined.

[0011] The present application also provides a path planning device for a drone swarm, the path planning device for the drone swarm comprising: A data processing module is used to pre-process the motion data of the drone cluster with timestamp information and determine the motion vectors of the drone nodes in the drone cluster; The path determination module is used to input the motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information into the drone cluster path planning model for spatial convolution coding processing, time domain cyclic coding processing, and decision action selection processing to determine the optimal decision action, continue to process the next target node selected by the decision action until the next target node is the termination node, and stop processing, and output the optimal path composed of all selected target nodes; wherein, the one-hop neighbor node is a node that directly communicates with the current drone node, and the two-hop neighbor node is a node that directly communicates with the one-hop neighbor node. The drone cluster path planning model is obtained by iterative training of the deep reinforcement model.

[0012] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the path planning method for the drone cluster as described above are performed.

[0013] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the path planning method for the drone cluster as described above are executed.

[0014] Embodiments of the present application provide a path planning method, apparatus, device, and storage medium for a drone swarm. The path planning method includes: preprocessing motion data of a drone swarm with timestamp information to determine motion vectors of drone nodes in the drone swarm; inputting the motion vector of the current drone node, information about the current drone node's one-hop neighbor nodes, and information about the current drone node's two-hop neighbor nodes into a drone swarm path planning model for spatial convolution coding, temporal cyclic coding, and decision action selection to determine the optimal decision action; continuing to process the next target node selected by the decision action until the next target node is a termination node; and outputting an optimal path consisting of all selected target nodes. A one-hop neighbor node is a node that directly communicates with the current drone node, and a two-hop neighbor node is a node that directly communicates with a one-hop neighbor node. The drone swarm path planning model is obtained by iteratively training a deep learning model. Spatial convolution coding enables each node to obtain richer network topology information, significantly improving the accuracy of path selection and the speed of routing convergence. Temporal cyclic coding captures time series features in real time, thereby adjusting path planning in advance.

[0015] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flow chart of a path planning method for a drone cluster provided in an embodiment of the present application; Figure 2 A schematic diagram of the training of the drone cluster path planning model provided in an embodiment of the present application; Figure 3 This is one of the structural schematic diagrams of a path planning device for a drone swarm provided in an embodiment of the present application; Figure 4 This is a second structural diagram of a path planning device for a drone swarm provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0019] First, the application scenarios to which this application is applicable are introduced. This application can be applied in the field of path planning technology.

[0020] Research has found that the existing routing optimization process is generally divided into node motion prediction and routing message distribution. The first step is node motion prediction. Each node initializes its velocity, position, and other motion data and maintains a local routing knowledge base to store the routing information of its local neighbors (including speed, position, and link quality). If the current node has past trajectory information, an iterative method is generally used to predict future motion data; otherwise, other non-iterative methods are used to calculate and estimate future node positions. However, existing routing optimization methods fail to fully understand the real-time evolution of the network topology, resulting in inaccurate path determination for drone swarms.

[0021] Based on this, an embodiment of the present application provides a path planning method for a drone cluster, which uses spatial convolutional coding processing to enable each node to obtain richer network topology information, significantly improving the accuracy of path selection and the speed of routing convergence. The time domain cyclic coding processing can capture time series characteristics in real time, thereby adjusting the path planning in advance.

[0022] See also Figure 1 , Figure 1 This is a flow chart of a path planning method for a drone cluster provided in an embodiment of the present application. Figure 1 As shown in , the path planning method provided by the embodiment of the present application includes: S101: Preprocessing the motion data of the drone cluster with timestamp information to determine the motion vectors of the drone nodes in the drone cluster.

[0023] In this step, the motion data set of the drone cluster with timestamp information is processed to obtain the number of drones: , the flight time is , the motion scene size is , each node in Motion vectors within a time period . is the length of the time interval. In time The motion vector is represented as , Represents nodes respectively In time The three-dimensional velocity, Represents nodes respectively In time The three-dimensional coordinates of .

[0024] S102: Input the motion vector of the current drone node, the one-hop neighbor node information and the two-hop neighbor node information of the current drone node into the drone cluster path planning model for spatial convolution coding processing, time domain cyclic coding processing and decision action selection processing to determine the optimal decision action, continue to process the next target node selected by the decision action until the next target node is the termination node, and stop processing, and output the optimal path composed of all selected target nodes.

[0025] In this step, the motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information are input into the drone cluster path planning model for spatial convolution coding processing, time domain cyclic coding processing, and decision action selection processing to determine the optimal decision action. The next target node selected by the decision action is processed until the next target node is the termination node, and the processing is stopped according to the optimal path formed by the determined multiple target nodes.

[0026] Among them, the one-hop neighbor node is a node that directly communicates with the current drone node, and the two-hop neighbor node is a node that directly communicates with the one-hop neighbor node. The drone cluster path planning model is obtained by iterative training of the deep learning model.

[0027] Here, the communication distance between drones is set to ,Establish Adjacency matrix of drone swarm network within time period .in, Indicates time The adjacency matrix of the cluster network, The dimension is N x N Matrix, if the drone node With drone nodes distance Less than , indicating that a communication link can be established between the two, and ;otherwise , where the adjacency matrix includes one-hop neighbor nodes and two-hop neighbor nodes.

[0028] In one possible implementation, the motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information are input into the drone cluster path planning model to perform spatial convolution coding processing, time domain cyclic coding processing, and action space decision processing to determine the optimal decision action, including: A: The spatial convolutional coding network layer based on the UAV cluster path planning model performs spatial convolution coding on the motion vector of the current UAV node, the one-hop neighbor node information of the current UAV node, and the two-hop neighbor node information, and outputs the environmental spatial features of the current UAV node.

[0029] Here, according to the spatial convolutional coding network layer of the UAV cluster path planning model, the motion vector of the current UAV node, the one-hop neighbor node information of the current UAV node, and the two-hop neighbor node information are spatially convolutionally coded to output the environmental spatial features of the current UAV node.

[0030] In one possible implementation, the spatial convolution coding network layer based on the UAV cluster path planning model performs spatial convolution coding processing on the motion vector of the current UAV node, the one-hop neighbor node information of the current UAV node, and the two-hop neighbor node information, and outputs the environmental spatial features of the current UAV node, including: a: Aggregate the one-hop neighbor node information and the two-hop neighbor node information of the current drone node respectively to determine the aggregated one-hop neighbor node information and the aggregated two-hop neighbor node information.

[0031] b: Perform spatial convolution coding on the aggregated one-hop neighbor node information, the aggregated two-hop neighbor node information, and the motion vector of the drone to determine the environmental spatial features.

[0032] Here, due to the current drone node , one-hop neighbors, and two-hop neighbors play different roles in subsequent intelligent decision-making, so these three types of nodes can be considered heterogeneous. In order to make full use of the one-hop and two-hop spatial neighbor relationships of nodes, a graph convolution strategy is used for spatial encoding: ; ; ; ; in, Indicates that the node is from the drone To the drone node Aggregate messages, Represents a drone node Aggregated node information. and Respectively represent the drone nodes of The graph convolutional network and mean processing used by the jump neighbor. Moment, drone node iThe one-hop neighbor of (excluding its own node), the two-hop neighbor is represented as , Represents the rest of the environment nodes and the drone node Try to avoid signal interference caused by communication. Represents a drone node and drone nodes The channel gain between For drone nodes i Information, For drone nodes i Aggregated information, for t Time to download drone nodes information.

[0033] B: Based on the time-domain recurrent network layer of the drone cluster path planning model, the environmental spatial features are captured and processed with the preceding temporal and spatial features to determine the local observation vector of the current drone node; wherein the local observation vector includes the motion vector of the current drone node, one-hop neighbor node information, two-hop neighbor node information and environmental information.

[0034] Here, the environment space features are captured and processed based on the time domain recurrent network layer of the UAV cluster path planning model to determine the local observation vector of the current UAV node.

[0035] The observations of the agent are encoded as feature vectors, and the feature vectors encoded by the observations in the neighborhood are aggregated using spatiotemporal graph convolution, so that the agent can learn the temporal and spatial characteristics of the environment.

[0036] here, t The local observation vector includes the current UAV node that plays the role of the agent information ,Neighbor information , end node location As well as environmental information. Therefore, the current drone node In time Obtain some observation information , defined as follows:

[0037] , Represents drone nodes respectively and drone nodes velocity and position vectors; Representation node One-hop neighbor; Represents a drone node and drone nodes The channel gain between Represents the rest of the environment nodes l With drone nodes Try to avoid signal interference caused by communication.

[0038] To fully reflect the network performance of the self-organizing flying network composed of drones and efficiently capture the temporal dependencies of the network topology, a recurrent control module is added. This step receives the spatially encoded hidden state vector and maintains relevant memory for subsequent decision steps, thereby reducing some environmental observation constraints and preserving the association between local decisions and the final bottleneck throughput.

[0039]

[0040]

[0041]

[0042]

[0043] in, and They are the update gate and reset gate outputs of the loop control module, and Separate moments Candidate hidden vectors and cyclic encoded output vectors; represents the activation function, represents the hyperbolic tangent process, represents the Hadamard product; , , are all learnable parameters of the network layer. for The candidate hidden vector at time t.

[0044] C: Based on the value network layer of the UAV cluster path planning model, the local observation vector is processed in action space decision-making to output the optimal decision action.

[0045] In one possible implementation, the value network layer based on the UAV cluster path planning model performs action space decision processing on the local observation vector and outputs the optimal decision action, including: The local observation vector and action space are processed based on the action-value function, and the Q value corresponding to each decision action is output; wherein the action space is a combination of the next-hop node and the power allocation; based on the greedy strategy, a target Q value is screened out from the Q values corresponding to multiple decision actions, and the decision action corresponding to the target Q value is used as the optimal decision action.

[0046] Here, the local observation vector and action space are processed according to the action-value function, and the output is the Q value corresponding to each decision action. According to the greedy strategy, the target Q value is screened out from the Q values corresponding to multiple decision actions, and the decision action corresponding to the target Q value is taken as the optimal decision action.

[0047] To achieve joint optimization of the network layer and the physical layer, the action space is represented as a combination of the next hop node and the power allocation. The power allocation is represented hierarchically: ,in is the preset maximum communication power of the drone, The number of levels is preset. The next hop node is the current node One-hop neighbor node Therefore, the agent node exist Action space Defined as:

[0048] Among them, ⊙ represents the Cartesian product operation, is the allocated power.

[0049] In one possible implementation, the UAV cluster path planning model is determined by the following steps: I: The motion vector of the current sample drone node, the one-hop neighbor node information of the current sample drone node, and the two-hop neighbor node information are input into the deep learning model for processing to determine the sample local observation vector, perform action space decision processing on the local observation vector, and output the sample decision action.

[0050] Here, the process of determining the sample decision action is consistent with the method of determining the optimal decision action described above, and will not be described in detail in this section.

[0051] II: Determine the reward value of the next-hop sample drone node selected by the sample decision action based on the reward mechanism.

[0052] In one possible implementation, determining the reward value of the next-hop sample drone node selected by the sample decision action based on a reward mechanism includes: i: Determine the path progress reward value of the next-hop sample drone node based on the 3D coordinate information of the next-hop sample drone node, the 3D coordinate information of the ending node, the 3D coordinate information of the starting node, the forward bias parameter, and the 3D speed information of the next-hop sample drone node.

[0053] Here, the path progress reward value of the next-hop sample drone node is determined by the following formula:

[0054] in, is the path progress reward value, which encourages the agent to move closer to the terminal node. for Current next-hop sample drone node The three-dimensional coordinate representation of is the three-dimensional coordinate information of the termination node, is the three-dimensional coordinate information of the starting node, Indicates forward bias, The next hop sample drone node j Three-dimensional velocity information.

[0055] ii: Determine the channel quality reward value based on the link signal interference and noise ratio between the current sample drone node and the next hop sample drone node and the preset link signal interference and noise ratio threshold.

[0056] Here, the channel quality reward value of the next-hop sample drone node is determined by the following formula:

[0057] in, For channel quality rewards, for Current sample drone node and next hop node The link signal-to-interference-noise ratio between The preset link acceptable minimum threshold, if If the value is less than the threshold, a negative reward will be given; is the weight coefficient of the three intermediate rewards.

[0058] iii: Determine the reward value of the next-hop sample drone node based on the sum of the path progress reward value and the channel quality reward value.

[0059] Here, the reward value .in, Indicates execution of Rewards after the jump. This means executing the next hop from the starting node.

[0060] Here, the total reward formula of the path is expressed as follows:

[0061] in, Indicates the total reward at the end of the route; Indicates the current path. This means the current path contains the number of nodes, and the routing is successful. ; Represents the end point reward, Indicates execution of Rewards after the jump, Represents the discount factor.

[0062] In this application, the route is not terminated and the termination flag is set ; The routing termination trigger successfully reaches the destination node , set the termination flag ; The current node is isolated and there is no valid neighbor node as the next hop node, so the termination flag is set ; The route reaches the maximum number of hops and does not reach the destination node , set the termination flag .

[0063] In a possible implementation manner, the link signal to interference and noise ratio is determined in the following manner: Based on the transmission power of the current sample drone node, the channel gain between the current sample drone node and the next-hop sample drone node, the Gaussian white noise value, the transmission power of other sample drone nodes, and the channel gain between other sample drone nodes and the next-hop sample drone node, the link signal interference noise ratio between the current sample drone node and the next-hop sample drone node is determined.

[0064] Here, the link signal-to-interference-and-noise ratio is determined by the following formula:

[0065] Among them, in the region Inside, time Each drone node Set the transmit power to , the shared bandwidth is spectrum resources. For any two nodes and , Shi Cong arrive The channel gain is , the distance between nodes Negative correlation. Assume that the environment contains Gaussian white noise , k is other nodes.

[0066] The communication capacity of the link is For multi-hop network communication, a path The starting node is , the ending node is , its bottleneck throughput is defined as the capacity of the link with the smallest capacity in the path.

[0067] III: Determine a loss value of the deep learning model based on the reward value, the sample local observation vector, and the sample decision action.

[0068] Here, the loss value of the deep learning model is determined by the following formula:

[0069]

[0070] in, Expressing expectation, is the length of the data sequence, for The agent's observation vector of the environment, Take the action vector for the agent, are trainable network parameters, is the Q value estimation function. for The time difference target value at time , is the control factor of Q value, for The total reward value when , using the parameter The Q learning network is used to calculate the target Q value. In actual training, the network parameters The training updates will be performed on the mini-batch samples by stochastic gradient descent. It will also The ratio is updated synchronously.

[0071] IV: Based on the loss value, the network parameters of the spatial convolutional coding network layer, the time domain recurrent network layer and the value network layer of the deep learning model are updated until the loss value is less than or equal to the preset threshold, and the updating of the network parameters is stopped to determine the UAV cluster path planning model.

[0072] Here, the network parameters of the spatial convolutional coding network layer, the time domain recurrent network layer and the value network layer of the deep learning model are updated according to the loss value until the loss value is less than or equal to the preset threshold, and the updating of the network parameters is stopped to determine the UAV cluster path planning model.

[0073] See also Figure 2 , Figure 2 This is a training diagram of the drone cluster path planning model provided in the embodiment of this application. Figure 2 As shown in , the motion vector of the current sample drone node, the one-hop neighbor node information of the current sample drone node, and the two-hop neighbor node information are spatially convolutionally encoded to obtain the environmental spatial features. The update gate and reset gate of the time domain recurrent network layer are used to capture the preceding spatiotemporal features of the environmental spatial features to obtain the environmental spatiotemporal features, and a sample local observation vector is constructed. The value network is used to perform action space decision processing on the local observation vector, output the decision action, and obtain the reward value corresponding to the decision action. The training sample is an observation-action-reward-state update vector set of length L in the experience replay pool. The loss value is determined according to the vector set, and the network parameters of the spatial convolution coding network layer, the time domain recurrent network layer, and the value network layer are updated according to the loss value. The training is repeated to finally obtain a trained drone cluster path planning model.

[0074] In a specific embodiment, when any node detects a communication request, route optimization starts from the designated source node src. First, the source node acts as an intelligent agent based on its learned strategy, evaluates the current state of the flight self-organizing network, determines the optimal transmission power, and selects the next hop node for communication. Next, the intelligent agent role is transferred to the selected next hop node, which makes a decision based on its own state and learning strategy, selects the appropriate transmission power and next hop node, and passes the intelligent agent role to the next node selected by the node. This step is continued, and each node makes local decisions based on the strategy at its own position until a termination event is triggered. Ultimately, the communication path and power allocation plan are jointly determined by the local selection results continuously optimized by these nodes in their respective decision-making processes, ensuring the high adaptability and fast convergence requirements of communication in the flight self-organizing network.

[0075] Addressing the shortcomings of existing technologies in routing optimization for drone swarm networks (FANETs), this application achieves efficient capture and adaptation to dynamic network changes, significantly improving routing optimization and overall system performance. First, existing technologies fail to effectively consider timestamp information about network status, resulting in an inability to fully understand the real-time evolution of network topology, making it difficult to adapt to dynamically changing network environments. This application introduces a time-domain cyclic coding module and utilizes a time-domain cyclic network layer to process historical data on drone node motion states and channel changes, enabling real-time capture and memorization of the time series characteristics of network status. This time series modeling approach not only considers the current network state but also incorporates historical information, enabling accurate capture of future network topology changes. Second, existing routing decision schemes typically rely solely on link information between the current node and its immediate neighbors, ignoring the status of more distant neighbors. In large-scale drone swarm networks, nodes may have a large number of hops. Relying solely on information from immediate neighbors cannot fully reflect the entire network state, resulting in suboptimal routing and slow routing convergence. This application constructs a drone swarm network topology map using a spatial convolutional coding network and leverages multi-hop neighbor relationships for path optimization. Specifically, the spatial graph convolutional network not only considers the information of direct neighbor nodes, but also transmits and fuses the status of multi-hop neighbors through aggregation functions and edge update functions. This multi-hop perception capability based on the spatial graph enables each node to obtain richer network topology information, so that it can make better decisions when selecting the next hop node. Finally, the agent decision-making step only requires the distance between nodes and channel interference information to make decisions, without the need for global information support of the entire flying self-organizing network. Therefore, the learned strategy can be extended to other wireless self-organizing networks; and even if the high-speed movement of drone nodes significantly changes the network topology layout, routing optimization can be performed without retraining.

[0076] An embodiment of the present application provides a path planning method for a drone swarm. The method comprises: preprocessing motion data of a drone swarm with timestamp information to determine motion vectors of drone nodes in the drone swarm; inputting the motion vector of the current drone node, information about the current drone node's one-hop neighbor nodes, and information about the current drone node's two-hop neighbor nodes into a drone swarm path planning model for spatial convolution coding, temporal cyclic coding, and decision action selection to determine the optimal decision action; continuing to process the next target node selected by the decision action until the next target node is a termination node; and outputting an optimal path consisting of all selected target nodes. A one-hop neighbor node is a node that directly communicates with the current drone node, and a two-hop neighbor node is a node that directly communicates with a one-hop neighbor node. The drone swarm path planning model is obtained by iteratively training a deep learning model. Spatial convolution coding enables each node to obtain richer network topology information, significantly improving the accuracy of path selection and the speed of routing convergence. Temporal cyclic coding captures time series features in real time, thereby adjusting path planning in advance.

[0077] See also Figure 3 、 Figure 4 , Figure 3 This is one of the structural schematic diagrams of a path planning device for a drone swarm provided in an embodiment of the present application; Figure 4 This is a second structural diagram of a path planning device for a drone cluster provided in an embodiment of the present application. Figure 3 As shown in , the path planning device 300 of the drone cluster includes: The data processing module 310 is used to pre-process the motion data of the drone cluster with timestamp information to determine the motion vectors of the drone nodes in the drone cluster; The path determination module 320 is used to input the motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information into the drone cluster path planning model for spatial convolution coding processing, time domain cyclic coding processing, and decision action selection processing to determine the optimal decision action, continue to process the next target node selected by the decision action until the next target node is the termination node, and stop processing, and output the optimal path composed of all selected target nodes; wherein, the one-hop neighbor node is a node that directly communicates with the current drone node, and the two-hop neighbor node is a node that directly communicates with the one-hop neighbor node. The drone cluster path planning model is obtained by iterative training of the deep reinforcement model.

[0078] Furthermore, when the path determination module 320 is used to input the motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information into the drone cluster path planning model for spatial convolution coding, time domain cyclic coding, and action space decision processing to determine the optimal decision action, the path determination module 320 is specifically used to: The spatial convolutional coding network layer based on the UAV cluster path planning model performs spatial convolution coding on the motion vector of the current UAV node, the one-hop neighbor node information of the current UAV node, and the two-hop neighbor node information, and outputs the environmental spatial features of the current UAV node; Based on the time-domain recurrent network layer of the UAV cluster path planning model, the environmental spatial features are captured and processed with the preceding temporal and spatial features to determine the local observation vector of the current UAV node; wherein the local observation vector includes the motion vector of the current UAV node, one-hop neighbor node information, two-hop neighbor node information, and environmental information; The value network layer of the UAV cluster path planning model performs action space decision processing on the local observation vector and outputs the optimal decision action.

[0079] Furthermore, when the path determination module 320 is used to perform spatial convolution coding processing on the motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information based on the spatial convolution coding network layer of the drone cluster path planning model, and output the environmental spatial features of the current drone node, the path determination module 320 is specifically used to: Aggregate the one-hop neighbor node information and the two-hop neighbor node information of the current drone node to determine the aggregated one-hop neighbor node information and the aggregated two-hop neighbor node information; The aggregated one-hop neighbor node information, the aggregated two-hop neighbor node information, and the motion vector of the current drone node are subjected to spatial convolution coding processing to determine the environmental spatial features.

[0080] Furthermore, when the path determination module 320 performs action space decision processing on the local observation vector at the value network layer based on the UAV cluster path planning model and outputs the optimal decision action, the path determination module 320 is specifically used to: Processing the local observation vector and action space based on the action-value function to output a Q value corresponding to each decision action; wherein the action space is a combination of the next hop node and the power allocation; Based on the greedy strategy, a target Q value is screened out from the Q values corresponding to multiple decision actions, and the decision action corresponding to the target Q value is taken as the optimal decision action.

[0081] Further, such as Figure 4 As shown, the path planning device 300 for the drone cluster further includes a model training module 330, which is used to: Input the motion vector of the current sample drone node, the one-hop neighbor node information of the current sample drone node, and the two-hop neighbor node information of the current sample drone node into the deep learning model for processing to determine the sample local observation vector, perform action space decision processing on the local observation vector, and output the sample decision action; Determine the reward value of the next-hop sample drone node selected by the sample decision action based on the reward mechanism; Determining a loss value of the deep learning model based on the reward value, the sample local observation vector, and the sample decision action; Based on the loss value, the network parameters of the spatial convolutional coding network layer, the time domain recurrent network layer and the value network layer of the deep learning model are updated until the loss value is less than or equal to a preset threshold, and the updating of the network parameters is stopped to determine the drone cluster path planning model.

[0082] Furthermore, when the model training module 330 is used to determine the reward value of the next-hop sample drone node selected by the sample decision action based on the reward mechanism, the model training module 330 is specifically used to: Determine the path progress reward value of the next-hop sample drone node based on the three-dimensional coordinate information of the next-hop sample drone node, the three-dimensional coordinate information of the end node, the three-dimensional coordinate information of the start node, the forward bias parameter, and the three-dimensional speed information of the next-hop sample drone node; Determine the channel quality reward value based on the link signal interference and noise ratio between the current sample drone node and the next hop sample drone node and the preset link signal interference and noise ratio threshold; Based on the sum of the path progress reward value and the channel quality reward value, a reward value of the next-hop sample drone node is determined.

[0083] Furthermore, the model training module 330 determines the link signal-to-interference-and-noise ratio by: Based on the transmission power of the current sample drone node, the channel gain between the current sample drone node and the next-hop sample drone node, the Gaussian white noise value, the transmission power of other sample drone nodes, and the channel gain between other sample drone nodes and the next-hop sample drone node, the link signal interference noise ratio between the current sample drone node and the next-hop sample drone node is determined.

[0084] An embodiment of the present application provides a path planning device for a drone swarm. The device comprises: a data processing module for preprocessing motion data of a drone swarm with timestamp information to determine motion vectors of drone nodes in the drone swarm; a path determination module for inputting the motion vector of the current drone node, information about the current drone node's one-hop neighbor nodes, and information about the current drone node's two-hop neighbor nodes into a drone swarm path planning model for spatial convolution coding, temporal cyclic coding, and decision action selection to determine the optimal decision action. The device then continues processing the next target node selected by the decision action until the next target node is a termination node, and outputs an optimal path consisting of all selected target nodes. A one-hop neighbor node is a node that directly communicates with the current drone node, and a two-hop neighbor node is a node that directly communicates with the one-hop neighbor node. The drone swarm path planning model is obtained by iteratively training a deep reinforcement model. Spatial convolution coding enables each node to obtain richer network topology information, significantly improving the accuracy of path selection and the speed of routing convergence. Temporal cyclic coding captures time series features in real time, thereby adjusting path planning in advance.

[0085] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown in FIG, the electronic device 500 includes a processor 510, a memory 520 and a bus 530.

[0086] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 communicates with the memory 520 via the bus 530. When the machine-readable instructions are executed by the processor 510, the above-mentioned Figure 1 The steps of the path planning method for the drone cluster in the method embodiment shown are specifically implemented in accordance with the method embodiment and will not be described in detail here.

[0087] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of the path planning method for the drone cluster in the method embodiment shown are specifically implemented in accordance with the method embodiment and will not be described in detail here.

[0088] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0089] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. There may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0090] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0091] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0092] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0093] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A path planning method for a drone swarm, characterized in that: The path planning method comprises: Preprocess the motion data of the drone cluster with timestamp information to determine the motion vector of each drone node in the drone cluster; The motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information of the current drone node are input into the drone cluster path planning model for spatial convolution coding, time domain cyclic coding, and decision action selection processing to determine the optimal decision action. The next target node selected by the decision action is processed until the next target node is the terminal node, and the processing is stopped. The optimal path consisting of all selected target nodes is output; Among them, the one-hop neighbor node is a node that directly communicates with the current drone node, and the two-hop neighbor node is a node that directly communicates with the one-hop neighbor node. The drone cluster path planning model is obtained by iterative training of the deep learning model.

2. The path planning method according to claim 1, characterized in that: The motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information are input into the drone cluster path planning model for spatial convolution coding, time domain cyclic coding, and action space decision processing to determine the optimal decision action, including: The spatial convolutional coding network layer based on the UAV cluster path planning model performs spatial convolution coding on the motion vector of the current UAV node, the one-hop neighbor node information of the current UAV node, and the two-hop neighbor node information, and outputs the environmental spatial features of the current UAV node; Based on the time-domain recurrent network layer of the UAV cluster path planning model, the environmental spatial features are captured and processed with the preceding temporal and spatial features to determine the local observation vector of the current UAV node; wherein the local observation vector includes the motion vector of the current UAV node, one-hop neighbor node information, two-hop neighbor node information, and environmental information; The value network layer of the UAV cluster path planning model performs action space decision processing on the local observation vector and outputs the optimal decision action.

3. The path planning method according to claim 2, characterized in that: The spatial convolution coding network layer based on the UAV cluster path planning model performs spatial convolution coding processing on the motion vector of the current UAV node, the one-hop neighbor node information of the current UAV node, and the two-hop neighbor node information, and outputs the environmental spatial features of the current UAV node, including: Aggregate the one-hop neighbor node information and the two-hop neighbor node information of the current drone node to determine the aggregated one-hop neighbor node information and the aggregated two-hop neighbor node information; The aggregated one-hop neighbor node information, the aggregated two-hop neighbor node information, and the motion vector of the current drone node are subjected to spatial convolution coding processing to determine the environmental spatial features.

4. The path planning method according to claim 2, characterized in that: The value network layer based on the UAV cluster path planning model performs action space decision processing on the local observation vector and outputs the optimal decision action, including: Processing the local observation vector and action space based on the action-value function to output a Q value corresponding to each decision action; wherein the action space is a combination of the next hop node and the power allocation; Based on the greedy strategy, a target Q value is screened out from the Q values corresponding to multiple decision actions, and the decision action corresponding to the target Q value is taken as the optimal decision action.

5. The path planning method according to claim 1, wherein: The UAV cluster path planning model is determined by the following steps: Input the motion vector of the current sample drone node, the one-hop neighbor node information of the current sample drone node, and the two-hop neighbor node information of the current sample drone node into the deep learning model for processing to determine the sample local observation vector, perform action space decision processing on the local observation vector, and output the sample decision action; Determine the reward value of the next-hop sample drone node selected by the sample decision action based on the reward mechanism; Determining a loss value of the deep learning model based on the reward value, the sample local observation vector, and the sample decision action; Based on the loss value, the network parameters of the spatial convolutional coding network layer, the time domain recurrent network layer and the value network layer of the deep learning model are updated until the loss value is less than or equal to a preset threshold, and the updating of the network parameters is stopped to determine the drone cluster path planning model.

6. The path planning method according to claim 5, characterized in that: The reward value of the next-hop sample drone node selected by the sample decision action is determined based on the reward mechanism, including: Determine the path progress reward value of the next-hop sample drone node based on the three-dimensional coordinate information of the next-hop sample drone node, the three-dimensional coordinate information of the end node, the three-dimensional coordinate information of the start node, the forward bias parameter, and the three-dimensional speed information of the next-hop sample drone node; Determine the channel quality reward value based on the link signal interference and noise ratio between the current sample drone node and the next hop sample drone node and the preset link signal interference and noise ratio threshold; Based on the sum of the path progress reward value and the channel quality reward value, a reward value of the next-hop sample drone node is determined.

7. The path planning method according to claim 6, characterized in that: The link signal to interference and noise ratio is determined by: Based on the transmission power of the current sample drone node, the channel gain between the current sample drone node and the next-hop sample drone node, the Gaussian white noise value, the transmission power of other sample drone nodes, and the channel gain between other sample drone nodes and the next-hop sample drone node, the link signal interference noise ratio between the current sample drone node and the next-hop sample drone node is determined.

8. A path planning device for a drone swarm, characterized in that: The path planning device of the drone cluster includes: A data processing module is used to pre-process the motion data of the drone cluster with timestamp information and determine the motion vectors of the drone nodes in the drone cluster; The path determination module is used to input the motion vector of the current drone node, the one-hop neighbor node information of the current drone node, and the two-hop neighbor node information into the drone cluster path planning model for spatial convolution coding processing, time domain cyclic coding processing, and decision action selection processing to determine the optimal decision action, continue to process the next target node selected by the decision action until the next target node is the termination node, and stop processing, and output the optimal path composed of all selected target nodes; wherein, the one-hop neighbor node is a node that directly communicates with the current drone node, and the two-hop neighbor node is a node that directly communicates with the one-hop neighbor node. The drone cluster path planning model is obtained by iterative training of the deep learning model.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the path planning method for a drone cluster as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the path planning method for a drone cluster as described in any one of claims 1 to 7 are executed.

Images