Unmanned aerial vehicle inspection trajectory planning method and system based on multi-decision branch parallelism

Abstract

The application discloses a kind of unmanned aerial vehicle inspection trajectory planning method and system based on multiple decision branch parallel, belong to path planning technical field.The method includes: obtaining inspection task instance and initializing unmanned aerial vehicle inspection state;Inspection task instance is input into strategy network, and the selection probability of each node is obtained;According to the selection probability, the node with the maximum probability is selected as the next inspection node, the unmanned aerial vehicle inspection state is updated, and the complete inspection trajectory is output until all inspection nodes are accessed.The application realizes parallel exploration by constructing multiple decision branches, and in the strategy network, the aggregator parameters are shared, the decoder parameters are independent, which effectively expands the exploration space of the strategy and improves the quality and efficiency of trajectory planning.

Description

Technical Field

[0001] This invention belongs to the field of path planning technology, specifically relating to a method and system for planning UAV inspection trajectories based on parallel multi-decision-branching. Background Technology

[0002] With the continuous expansion of power systems and the advancement of artificial intelligence technology, drone inspections are gradually replacing traditional power inspection methods due to their significant advantages in reducing human risks and improving operational efficiency. The key to achieving efficient drone inspections lies in planning an inspection trajectory that can cover all nodes to be inspected at the lowest possible cost.

[0003] However, the currently widely used UAV inspection trajectory planning methods generally have the following limitations: First, most existing methods adopt a single path generation strategy, which has a limited search space and is prone to getting trapped in local optima, making it difficult to guarantee the quality of the solution; Second, existing learning-based methods often use global attention when encoding the relationships between nodes, resulting in a quadratic increase in computational complexity with the number of nodes, leading to low efficiency and difficulty in real-time response when facing large-scale tasks; Third, existing technologies often fail to fully utilize the spatial information of the inspection scene and the state information of the UAV, which restricts further improvement in planning quality.

[0004] Therefore, there is an urgent need for an intelligent planning method that can automatically learn from inspection task data and explore multiple feasible trajectories in parallel to make end-to-end decisions, so as to effectively utilize spatial information while limiting computational complexity, thereby comprehensively improving the operational efficiency and economy of UAV inspection. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a UAV inspection trajectory planning method and system based on multi-decision branch parallelism, so as to solve the problems of single strategy being prone to local optima, high computational complexity of global attention, and insufficient utilization of spatial information and UAV state in the existing UAV inspection trajectory planning methods.

[0006] To achieve the above objectives, the present invention employs the following technical solution: The first aspect of this invention discloses a method for planning UAV inspection trajectory based on multi-decision branch parallelism, comprising the following steps: S1, Obtain an inspection task instance and initialize the UAV inspection status. The task instance consists of multiple nodes. S2, the inspection task instance is input into a policy network, which includes a single encoder and multiple decision branches; the encoder outputs a node embedding based on the inspection task instance; one of the multiple decision branches is selected as the current decision branch, and the current UAV inspection status, unvisited mask, and the node embedding are input into the aggregator of the current decision branch; the aggregator performs dynamic mask enhancement on the node embedding and incorporates the UAV inspection status, outputting a multi-source information aggregation vector; the decoder of the current decision branch calculates the selection probability of each node based on the multi-source information aggregation vector and the node embedding; S3, based on the selection probability, select the node with the highest probability as the next inspection node, and update the UAV inspection status and unvisited mask; S4, repeat S2 and S3 until all inspection nodes are visited, and output the complete inspection trajectory.

[0007] Preferably, in S2, the aggregator parameters of the multiple decision branches are shared, while the decoder parameters are independent; the multiple decision branches work in parallel during the training of the policy network.

[0008] Preferably, in S2, the encoder takes an undirected graph corresponding to the inspection task instance as input, the undirected graph including node data and raw spatial data; the node data is the static two-dimensional node coordinates of each node, and the raw spatial data is the spatial distance between each node.

[0009] Preferably, in S2, the encoder includes multiple encoding modules connected in sequence, each encoding module including a neighborhood feature aggregation layer and a feature refinement layer; the neighborhood feature aggregation layer calculates aggregated features through neighborhood space embedding attention operation, and fuses the aggregated features with the input node representation through an adaptive gating mechanism to obtain a neighborhood fusion representation; the feature refinement layer performs a nonlinear transformation on the neighborhood fusion representation and obtains the output features using the gating residual; the neighborhood space embedding attention operation restricts the attention calculation to the neighborhood range of each node.

[0010] Preferably, the neighborhood spatial embedding attention operation includes: determining a set of neighboring nodes for each node based on spatial distance; calculating a similarity score between a node and its neighboring nodes that includes spatial representation; calculating attention weights within the neighborhood using the similarity scores; weighting and aggregating the features of the neighboring nodes using the attention weights, and adding them to the input features to obtain the node feature update result.

[0011] Preferably, in S2, the aggregator performs dynamic masking enhancement on the node embedding to obtain a dynamic inspection task vector, integrates the UAV inspection state into the dynamic inspection task vector to obtain a dynamic feature vector, and calculates a multi-source information aggregation vector based on the dynamic feature vector; the decoder concatenates the node embedding with the multi-source information aggregation vector to obtain a context embedding, calculates the attention compatibility value of each node based on the context embedding, and normalizes the attention compatibility value to obtain the selection probability.

[0012] Preferably, in S1, the drone inspection status includes the drone's remaining battery power and the current inspection node index; in S3, the update process is as follows: set the element corresponding to the selected node in the unvisited mask to be visited, subtract the distance from the current node to the next inspection node from the drone's remaining battery power, and update the current inspection node index to the selected node.

[0013] Preferably, in S2, the policy network is a network trained offline; the training process includes: constructing a policy network containing the encoder and multiple decision branches, where the aggregator parameters of each decision branch are shared and the decoder parameters are independent; for an inspection task instance, using the policy network to generate multiple complete inspection trajectories in parallel and recording experience data; iteratively optimizing the policy network parameters based on batch experience data; wherein, the iterative optimization uses a total loss function composed of policy loss, evaluation loss and diversity regularization loss for gradient descent update, and the diversity regularization loss is calculated based on the probability difference of different decision branches selecting actions in the same state.

[0014] Preferably, during the training process, an iterative optimization mechanism is constructed by introducing a policy evaluation network, an asynchronous policy network, and an asynchronous policy evaluation network; each decision branch records experience data during training.

[0015] The second aspect of this invention discloses a UAV inspection trajectory planning system based on multi-decision branch parallelism, comprising: The task acquisition module is used to acquire inspection task instances and initialize the UAV inspection status. The task instance consists of multiple nodes. A policy network module is used to input the inspection task instance into a policy network, which includes a single encoder and multiple decision branches. The encoder outputs node embeddings based on the inspection task instance. One of the multiple decision branches is selected as the current decision branch, and the current UAV inspection status, unvisited mask, and the node embeddings are input into the aggregator of the current decision branch. The aggregator performs dynamic mask enhancement on the node embeddings and incorporates the UAV inspection status, outputting a multi-source information aggregation vector. The decoder of the current decision branch calculates the selection probability of each node based on the multi-source information aggregation vector and the node embeddings. The node update module is used to select the node with the highest probability as the next inspection node based on the selection probability, and update the UAV inspection status and unvisited mask. The trajectory generation module is used to repeatedly execute the strategy network module and the node update module until all inspection nodes have been visited, and output the complete inspection trajectory.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention discloses a UAV inspection trajectory planning method based on multi-decision-branch parallelism. This method constructs a policy network consisting of a single encoder and multiple decision branches, where the aggregator parameters of each decision branch are shared, while the decoder parameters are independent. During training, this invention designs a multi-decision-branch parallel decision-making and optimization approach to achieve end-to-end inspection trajectory planning. While controlling computational complexity, it effectively expands the policy exploration space, improves trajectory planning efficiency and trajectory quality, and can effectively support UAV inspection tasks. It overcomes the defect of single policies easily getting trapped in local optima, significantly improving the planning quality of inspection trajectories.

[0017] Furthermore, attention operations are embedded in the neighborhood space, restricting attention computation to the neighborhood of each node. In each encoding module of the encoder, the nearest K nodes are selected to form a neighborhood based solely on spatial distance. Similarity scores, including spatial representations, are calculated only between a node and its neighboring nodes, and attention weights are calculated only within the neighborhood. By fusing spatial information with node representations, deep spatial relationships between nodes are mined. Simultaneously, restricting attention computation to the neighborhood of each node controls computational complexity, ensuring efficient solution even when facing large-scale inspection tasks.

[0018] Furthermore, the aggregator integrates UAV status information with node embeddings. First, the aggregator performs dynamic masking enhancement on the node embeddings to distinguish between visited and unvisited nodes, obtaining a dynamic inspection task vector. Then, it integrates real-time status information such as the UAV's remaining battery power and the current inspection node index into the dynamic inspection task vector, outputting a multi-source information aggregation vector. This process enables the policy network to continuously perceive the inspection progress and the UAV's remaining battery power, allowing for more rational node selection during dynamic planning. This prevents the UAV from being forced to interrupt inspections due to insufficient power, further improving the overall economy of the inspection trajectory and the task completion rate.

[0019] Furthermore, this invention introduces diversity regularization loss during training, calculating it based on the probability differences in action selection by different decision branches under the same state. This loss term encourages each decision branch to develop differentiated policy preferences during learning, rather than converging on the same optimal policy, thereby enhancing the exploration diversity of the policy network. Simultaneously, the multiple trajectories generated by parallel sampling from multiple decision branches during training constitute richer empirical data, improving the quality of training data and the accuracy of policy evaluation. In practical applications, the trained policy network can select multiple decision branches to generate candidate trajectories, and then select the optimal trajectory based on the actual state of the UAV and the characteristics of the inspection scenario, enhancing the generalization ability and robustness of the method. Attached Figure Description

[0020] Figure 1 A flowchart of a UAV inspection trajectory planning method based on multi-decision branch parallelism; Figure 2 This is a schematic diagram of the policy network structure; Figure 3 Example diagram of autonomous planning of inspection trajectory for drones. Detailed Implementation

[0021] Hereinafter, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first," "second," "third," or "fourth" may explicitly or implicitly include one or more of that feature.

[0022] It should be noted that the terms "first," "second," etc., used in the specification and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] The first aspect of this invention discloses a method for planning UAV inspection trajectory based on multi-decision branch parallelism, comprising the following steps: S1 generates an inspection task instance based on the base station nodes and inspection nodes in the inspection area in each task instance, and initializes the UAV inspection status for each task instance.

[0024] In S1, each task instance consists of N nodes, including 1 base station node and N-1 inspection nodes. The coordinates of each node are set within a unit area of ​​[0,1]×[0,1].

[0025] Extracting undirected graphs from task instances ,in For node data, Representing the The static two-dimensional node coordinates of each node; For raw spatial data, Representative node With nodes Spatial distance between them and These are the node numbers; they are obtained by calculating the Euclidean distance between the coordinates of the two points.

[0026] In S1, the drone inspection status is as follows:

[0027] in, For time step The drone inspection status below, superscript UAV It is an abbreviation for Unmanned Aerial Vehicle, meaning drone. For time steps The remaining battery power of the drone is set during initialization. For maximum battery capacity ; For time step The current inspection node index of the drone is set during initialization. For base station nodes, loc "Location" is an abbreviation for "location".

[0028] S2, The inspection task instance is input into the policy network, and the policy network calculates the selection probability of each node; S201, the encoder has A series of sequentially connected encoding modules, where the input of each encoding module is the output of the previous encoding module. The first encoding module is an undirected graph. For input, After each encoding module encodes in sequence, the final node embedding is obtained. , and output.

[0029] During encoder processing, the static two-dimensional node coordinates are first processed. and spatial distance Perform feature encoding to generate an initial node representation vector for each node. Generate a spatial representation vector for each spatial distance. .

[0030] Subsequently, the initial node representation vectors are aggregated into the initial node representation. Represent all spatial representation vectors as spatial representations ,Will , and Send in The encoding modules are connected sequentially, starting with the first encoding module and ending with the last encoding module. Output the final node embedding In the encoder, the first The input of the encoding module is the first... The node representation output by each encoding module Spatial representation and raw spatial data The output is the updated node representation. , No. The output of each encoding module is , due to the The last encoding module is the last encoding module of the encoder, therefore This refers to the node embedding at the final output of the encoder.

[0031] In some embodiments of the present invention, each encoding module consists of a neighborhood feature aggregation layer and a feature refinement layer. In the neighborhood feature aggregation layer, aggregated features are first calculated using a neighborhood spatial embedding attention operation:

[0032] in, Embed attention operations in the neighborhood space. It is an abbreviation for Neighborhood Embedded Spatial Attention; For the first The aggregated features of each encoding module, where the subscript agg It is an abbreviation for aggregation, meaning aggregation; For the first The node representation output by the encoding module is the th . l The node representation input to each encoding module.

[0033] Then, an adaptive gating mechanism is used to fuse the aggregated features with the input node representation to obtain the first... Aggregate gating vector of each coding module:

[0034]

[0035] in, For the first The aggregated gating vector of each encoding module, It is the Sigmoid activation function. For element-wise multiplication, For the first The gating weight parameter matrix of each encoding module, For the first The gating bias parameter matrix of each encoding module; For the first The neighborhood fusion representation of each encoding module is the fused feature, where the subscript... fuse It is an abbreviation for fusion, meaning integration; Layer Normalization is an abbreviation for Layer Normalization.

[0036] In the feature refinement layer, the fused features—the first... The neighborhood fusion representation of the first encoding module is subjected to a nonlinear transformation, and the gated residual is used again to obtain the second... The final output of each module :

[0037]

[0038]

[0039] in, For the first Feedforward mapping features of each encoding module ff It is an abbreviation for feed-forward, meaning feedforward. ReLU is the activation function, an abbreviation for Rectified Linear Unit. For the first The feedforward weight parameter matrix of each encoding module For the first Feedforward bias parameter matrix of each encoding module; For the first Refined gating vectors for each encoding module For the first The refined gating weight parameter matrix of each encoding module, For the first The refined gating bias parameter matrix of each encoding module.

[0040] In a specific example, the neighborhood space embedding attention operation Specifically: For any nth in the encoder l Each encoding module, input Spatial representation and raw spatial data Firstly, based on spatial distance For each node Determine its set of neighboring nodes Take the node The K nodes with the smallest distance form a neighborhood, where K is a preset constant and 1 ≤ K. <N。

[0041] Next, only node i and its neighboring nodes are considered. Calculate the similarity score including the spatial representation:

[0042] in, For the first Nodes in each encoding module With nodes Similarity score, LeakyReLU is a nonlinear activation function, short for Leaky Rectified Linear Unit, and is an improved activation function. This is a horizontal splicing operation. For the first Similarity scaling parameters for each encoding module For the first The spatial weight parameter matrix of each encoding module, where the subscripts are... spa It is an abbreviation for spatial, meaning spatial. For the first Spatial bias parameter matrix of each encoding module The input for the l-th encoding module The first in One portion, The input for the l-th encoding module The first in One portion, Indicates any, It indicates that it belongs to.

[0043] Then, attention weights are calculated only within the neighborhood using similarity scores:

[0044] Where exp(·) is an exponential function, For the first Nodes in each encoding module Corresponding node Attention weights For the first Nodes in each encoding module With nodes Similarity score, This indicates that it does not belong to this category.

[0045] Finally, attention weights are used to weight and aggregate the features of neighboring nodes, and then added to the input features to obtain the updated result. The input features are... The calculation formula is as follows:

[0046] in, For the first Nodes in each encoding module The node feature update results.

[0047] Thus, the result of the neighborhood space embedding attention operation is: .

[0048] S202, for M decision branches with the same structure, each decision branch contains an aggregator and a decoder; the aggregator parameters of all decision branches are shared, and the decoder parameters of all decision branches are independent.

[0049] For decision branches In time step Its aggregator uses the drone inspection status of the m-th decision branch. Unaccessed mask and shared node embedding As input, the output is a multi-source information aggregation vector. The specific steps are as follows: First, dynamic masking enhancement is applied to the node embedding to obtain the dynamic inspection task vector:

[0050] in, For decision branches At time step The unvisited mask is a 1×N vector that corresponds to N nodes in the inspection task, where the elements corresponding to visited inspection nodes are 0 and the elements corresponding to unvisited inspection nodes are 1. For decision branches At time step The dynamic inspection task vector is defined below. This is a mask enhancement weight parameter matrix, where the subscript mask represents the mask. The bias parameter matrix is ​​used to enhance the mask.

[0051] Subsequently, the drone inspection status is incorporated to obtain dynamic feature vectors:

[0052]

[0053] in, For decision branches At time step Find the maximum and minimum values ​​of the dynamic inspection task vector. This is an operation to find the maximum value of the elements in a vector; For decision branches At time step The dynamic feature vectors under, This is the state weight parameter matrix for the UAV. Here is the UAV state bias parameter matrix, where, FFN It is an abbreviation for Feed-Forward Network, which means feedforward network.

[0054] Finally, the multi-source information aggregation vector output by the aggregator is calculated:

[0055]

[0056] in, For decision branches At time step The local mean of the dynamic inspection task vector. For decision branches At time step Dynamic inspection task vector The One element, For decision branches At time step The multi-source information aggregation vector output by the aggregator.

[0057] For decision branches Its decoder uses multi-source information from this decision branch to aggregate vectors. and node embedding The specific steps for calculating the node selection probability are as follows: First, the node embeddings are concatenated with the multi-source information aggregation vector:

[0058] in, For decision branches At time step The obtained context embedding, This is the node embedding vector for the base station node. For decision branches At time step The node embedding vector of the selected node. and All are node embeddings The components in the equation are determined by their corresponding nodes.

[0059] Then, calculate the attention compatibility value:

[0060] in, For decision branches At time step For nodes Attention compatibility value, A constant set by humans. It is the hyperbolic tangent function. For nodes The node embedding vector is the node embedding vector. The first in One portion, For decision branches The attention query value weight parameter matrix, where the superscript T represents the matrix transpose. For decision branches Attention key-value weight parameter matrix, for and The vertical dimension size. For nodes that have already been inspected, their attention compatibility value is set to... To block it.

[0061] Finally, based on the obtained attention compatibility value, the decoder selects the node. The probability value of the current time step inspection node:

[0062] in, For decision branches The decoder at time step Select node As the probability value of the inspection node at the current time step, For decision branches At time step For nodes Attention compatibility value.

[0063] S3, for each task instance: at time step Decision branch The probability distribution of N nodes is obtained based on their current state vector. Based on this probability distribution, sampling is performed to obtain the action. This action serves as a node index to point to the next inspection node, followed by a decision branch. Update its own status; repeat until all inspection nodes have been visited, obtaining a complete trajectory. ,in, The total number of time steps and , For decision branches The complete trajectory, The initial action of branch m points to the base station node. The action chosen by decision branch m at time step T-1 points to the inspection node corresponding to the index. The inspection process needs to traverse all N nodes, with time steps... From 1 to T, these correspond to the selection time for each inspection node.

[0064] Among them, decision branches At time step Current state vector Defined as:

[0065] in, For decision branches At time step The status of drone inspections For decision branches At time step The unvisited mask below.

[0066] Execution time step Decision Branch action Then, the unaccessed mask will be... The element corresponding to this node is updated from 1 to 0, and the drone inspection status is also updated. The remaining battery power of the drone is Updated to And the current inspection node of the drone Update to this node, where For decision branches At time step The remaining battery power of the drone For decision branches At time step The index of the current inspection node.

[0067] Therefore, the current state vector is updated as follows:

[0068] in, For decision branches At time step The current state vector is as follows. For time step Next decision branch Unaccessed mask, For time step Next decision branch The drone inspection status.

[0069] S4, repeat S2-S3 until all inspection nodes are visited, and output the complete inspection trajectory.

[0070] In a specific embodiment, this invention discloses the training process of the aforementioned UAV inspection trajectory planning method based on multi-decision branch parallelism. The training process includes the following steps: Step 1: Generate inspection task instances online and initialize the drone inspection status for each task instance; In step 1, each task instance consists of N nodes, including 1 base station node and N-1 inspection nodes. The coordinates of each node are randomly generated within a unit area of ​​[0,1]×[0,1]. Extracting undirected graphs from task instances ,in For node data, Representing the The static two-dimensional node coordinates of each node; For raw spatial data, Representative node With nodes The spatial distance between two points is obtained by calculating the Euclidean distance between their coordinates.

[0071] In step 1, the drone inspection status is as follows:

[0072] in, For time step The current status of drone inspection For time step The remaining battery power of the drone is set during initialization. For maximum battery capacity ; For time step The current inspection node index of the drone is set during initialization. For base station nodes.

[0073] Step 2, see Figure 2 Construct a policy network that includes a single encoder and multiple decision branches; In step 2, the policy network includes a shared encoder that has Each encoding module, in an undirected graph As input, follow these steps to output. The node embedding obtained by sequential encoding of each encoding module .

[0074] First, the static two-dimensional node coordinates... and spatial distance Perform feature encoding to generate an initial node representation vector for each node. Generate a spatial representation vector for each spatial distance. .

[0075] Subsequently, the initial node representation vectors are aggregated into the initial node representation. Represent all spatial representation vectors as spatial representations ,Will , and Send in The encoding modules are connected sequentially, where the first... The input of the encoding module is the first... The node representation output by each encoding module Spatial representation and raw spatial data The output is the updated node representation. The output of the last encoding module is a node embedding. .

[0076] Each encoding module consists of a neighborhood feature aggregation layer and a feature refinement layer. In the neighborhood feature aggregation layer, aggregated features are first calculated using a neighborhood spatial embedding attention operation:

[0077] in, Embed attention operations in the neighborhood space; For the first Aggregation features of each coding module; For the first The node representation output by each encoding module.

[0078] Then, an adaptive gating mechanism is used to fuse the aggregated features with the input node representation to obtain the first... Aggregate gating vector of each coding module:

[0079]

[0080] in, For the first The aggregated gating vector of each encoding module, It is the Sigmoid activation function. For element-wise multiplication, For the first The gating weight parameter matrix of each encoding module, For the first The gating bias parameter matrix of each encoding module; For the first The neighborhood fusion representation of each coding module is the fused feature; Layer Normalization is an abbreviation for Layer Normalization.

[0081] In the feature refinement layer, the fused features are subjected to a nonlinear transformation, and the gated residual is used again to obtain the first... The final output of each module:

[0082]

[0083]

[0084] in, For the first Feedforward mapping features of each encoding module It is the ReLU activation function; For the first The feedforward weight parameter matrix of each encoding module For the first Feedforward bias parameter matrix of each encoding module; For the first Refined gating vectors for each encoding module For the first The refined gating weight parameter matrix of each encoding module, For the first The refined gating bias parameter matrix of each encoding module.

[0085] In a specific example, the neighborhood space embedding attention operation Specifically: For input Spatial representation and raw spatial data Firstly, based on spatial distance For each node Determine its set of neighboring nodes Take the node The K nodes with the smallest distance form a neighborhood, where K is a preset constant and 1 ≤ K. <N。

[0086] Next, only node i and its neighboring nodes are considered. Calculate the similarity score including the spatial representation:

[0087] in, For the first Nodes in each encoding module With nodes Similarity score, Use LeakyReLU nonlinear activation; This is a horizontal splicing operation. For the first Similarity scaling parameters for each encoding module For the first Spatial weight parameter matrix of each encoding module For the first Spatial bias parameter matrix of each encoding module The input for the l-th encoding module The first in One portion, The input for the l-th encoding module The first in One portion, Indicates any, It indicates that it belongs to.

[0088] Then, attention weights are calculated only within the neighborhood using similarity scores:

[0089] Where exp(·) is an exponential function, For the first Nodes in each encoding module Corresponding node Attention weights For the first Nodes in each encoding module With nodes The similarity score.

[0090] Finally, attention weights are used to weight and aggregate the features of neighboring nodes, and then added to the input features to obtain the updated result. The input features are... The calculation formula is as follows:

[0091] in, For the first Nodes in each encoding module The node feature update results.

[0092] Thus, the result of the neighborhood space embedding attention operation is: .

[0093] The policy network also contains M decision branches with identical structures, each decision branch containing an aggregator and a decoder; the aggregator parameters of all decision branches are shared, and the decoder parameters of all decision branches are independent.

[0094] For decision branches In time step Its aggregator uses the drone inspection status of this decision branch. Unaccessed mask and shared node embedding As input, the output is a multi-source information aggregation vector. The specific steps are as follows: First, dynamic masking enhancement is applied to the node embedding to obtain the dynamic inspection task vector:

[0095] in, For decision branches At time step The unvisited mask is a 1×N vector that corresponds to N nodes in the inspection task, where the elements corresponding to visited inspection nodes are 0 and the elements corresponding to unvisited inspection nodes are 1. For decision branches At time step The dynamic inspection task vector is defined below. To enhance the weight parameter matrix of the mask, The bias parameter matrix is ​​used to enhance the mask.

[0096] Subsequently, the drone inspection status is incorporated to obtain dynamic feature vectors:

[0097]

[0098] in, For decision branches At time step Find the maximum and minimum values ​​of the dynamic inspection task vector. This is an operation to find the maximum value of the elements in a vector; For decision branches At time step The dynamic feature vectors under, This is the state weight parameter matrix for the UAV. This is the state bias parameter matrix for the UAV.

[0099] Finally, the multi-source information aggregation vector output by the aggregator is calculated:

[0100]

[0101] in, For decision branches At time step The local mean of the dynamic inspection task vector. For decision branches At time step Dynamic inspection task vector The One element, For decision branches At time step The multi-source information aggregation vector output by the aggregator.

[0102] For decision branches Its decoder uses multi-source information from this decision branch to aggregate vectors. and node embedding The specific steps for calculating the node selection probability are as follows: First, the node embeddings are concatenated with the multi-source information aggregation vector:

[0103] in, For decision branches At time step The obtained context embedding, This is the node embedding vector for the base station node. For decision branches At time step The node embedding vector of the selected node. and All are node embeddings The components in the equation are determined by their corresponding nodes.

[0104] Then, calculate the attention compatibility value:

[0105] in, For decision branches At time step For nodes Attention compatibility value, A constant set by humans. It is the hyperbolic tangent function. For nodes The node embedding vector is the node embedding vector. The first in One portion, For decision branches The attention query value weight parameter matrix, where the superscript T represents the matrix transpose. For decision branches Attention key-value weight parameter matrix, for and The vertical dimension size. For nodes that have already been inspected, their attention compatibility value is set to... To block it.

[0106] Finally, based on the obtained attention compatibility value, the decoder selects the node. The probability value of the current time step inspection node:

[0107] in, For decision branches The decoder at time step Select node As the probability value of the inspection node at the current time step, For decision branches At time step For nodes Attention compatibility value.

[0108] Step 3: For a single task instance, multiple complete inspection trajectories are generated in parallel using a policy network, and experience data is formed based on the time-step records. Each training round processes a batch of task instances to form batch experience data. In step 3, for each task instance, Each decision branch works in parallel: at time step Decision branch The probability distribution of N nodes is obtained based on their current state vector. Based on this probability distribution, sampling is performed to obtain the action. This action serves as a node index to point to the next inspection node, followed by a decision branch. Update its own status; repeat until all inspection nodes have been visited, obtaining a complete trajectory. ,in, The total number of time steps and , For decision branches The complete trajectory, The initial action of branch m points to the base station node. The action chosen by decision branch m at time step T-1 points to the inspection node corresponding to the index. The inspection process needs to traverse all N nodes, with time steps... From 1 to T, these correspond to the selection time for each inspection node.

[0109] Among them, decision branches At time step Current state vector Defined as:

[0110] in, For decision branches At time step The drone inspection status.

[0111] Execution time step Decision Branch action Then, the unaccessed mask will be... The element corresponding to this node is updated from 1 to 0, and the drone inspection status is also updated. The remaining battery power of the drone is Updated to And the current inspection node of the drone Update to this node, where For decision branches At time step The remaining battery power of the drone For decision branches At time step The index of the current inspection node.

[0112] Therefore, the current state vector is updated as follows:

[0113] in, For decision branches At time step The current state vector is as follows. and These are the corresponding time steps. Next decision branch Unaccessible mask and drone inspection status.

[0114] Furthermore, a policy evaluation network, an asynchronous policy network, and an asynchronous policy evaluation network are introduced in this process to construct a stable iterative optimization mechanism: The policy network parameters are denoted as Its asynchronous strategy network parameters are denoted as The subscript "asy" is an abbreviation for "asynchronous," indicating that the two networks have the same structure but independent parameters. The policy evaluation network is a feedforward neural network that takes the state vector as input and outputs a scalar evaluation value for that state. Its parameters are denoted as The asynchronous strategy evaluates network parameters as follows: They have the same structure but independent parameters.

[0115] For a single task instance, its decision branches In time step Record empirical data during the decision-making process. :

[0116] in, For decision branches At time step empirical data; For asynchronous policy networks (parameters are) ) decision branches The decoder outputs the current state. Select action The probability value, For decision branches At time step The immediate cost, Indicates the drone's actions The node pointed to flies to the action The straight-line distance to the node being pointed to. For execution time step Decision Branch The action; Evaluate the network for asynchronous policies (parameters are) )right The evaluation value.

[0117] Based on decision branches Based on the empirical data accumulated at the current time step and all previous time steps, further calculate the cumulative return and decision bias:

[0118]

[0119] in, for At time step The cumulative return For decision branches At time step The immediate cost, for At time step Decision-making bias.

[0120] Step 4: Based on the batch experience data, perform multiple iterations to optimize the strategy network parameters, and clear the batch data after the update is completed; In step 4, after collecting data containing... After processing the batch experience data for each task instance, first configure the policy network parameters. Strategy evaluation of network parameters Copy to the asynchronous strategy network parameters respectively Asynchronous strategy for evaluating network parameters ; then proceed Secondary parameter updates, where epoch represents the training epoch, and each update starts from... Randomly selected from each task instance There are task instances (1 ≤ b < B), using this... All empirical data from each task instance are used as sub-batches to calculate the total loss and apply gradient descent to the policy network parameters. Strategy evaluation of network parameters Perform an update; This batch of experience data will be cleared after the next update.

[0121] Define the policy loss for a single task instance. , policy Express strategy and assess loss subscript eval This is an abbreviation for evaluation, referring to the assessment of diversity regularization loss. , div , short for divergence, represents the degree of divergence, and the formula is as follows:

[0122]

[0123]

[0124] Where M is the number of strategy decision branches, and T is the total number of time steps. To perform the minimum value operation, For policy network (parameters are) ) decision branches The decoder in the current state Select action The probability value, For policy network (parameters are) ) decision branches The decoder in the current state Select action The probability value, For policy network (parameters are) ) decision branches The decoder in the current state Select action The probability value, For execution time step Decision Branch The action, For execution time step Decision Branch The action, For decision branches At time step The current state vector, For decision branches At time step The current state vector. For policy evaluation network (parameters are) )right The evaluation values ​​for both are generated online in step 4; hyperparameters For the cropping range parameter, the cropping operation Limit the ratio to Interval.

[0125] Therefore, for a single task instance, its total loss subscript inst "Instance" is an abbreviation for task instance. The specific calculation formula is as follows:

[0126] in, , , These are the preset strategy loss weights, evaluation loss weights, and diversity regularization loss weights, respectively.

[0127] Each time the parameters are updated, the total loss of the sub-batch is taken from the sampled data. The arithmetic mean of the loss values ​​of each task instance is used to update the policy network parameters using gradient descent. Evaluation of network parameters with strategy .

[0128] Step 5: Repeat steps 3 and 4 until the preset training rounds are completed. In actual application, select a decision branch from the policy network and concatenate it with the encoder. Select inspection nodes in sequence according to the highest probability to generate inspection trajectories.

[0129] In step 5, in practical applications, a decision branch is selected from the trained policy network. The undirected graph of the task to be planned is extracted and the static two-dimensional node coordinates are normalized to [0,1]×[0,1] as in step 1. The UAV inspection state and unvisited mask are initialized. At each time step, the selected decision branch calculates the node selection probability and selects the node with the highest probability as the action until all nodes are visited, and a complete inspection trajectory is output. If multiple decision branches are selected, multiple complete inspection trajectories are output to be selected according to the actual situation.

[0130] The UAV inspection trajectory planning method based on multi-decision branch parallelism provided by this invention constructs a policy network consisting of a single encoder and multiple decision branches. By exploring in parallel through multiple decision branches, the exploration space of the policy is expanded, overcoming the defect of single policy being prone to getting trapped in local optima, and significantly improving the planning quality of inspection trajectories.

[0131] A second aspect of the present invention provides a UAV inspection trajectory planning system based on multi-decision branch parallelism, comprising: The task acquisition module is used to acquire inspection task instances and initialize the UAV inspection status. The task instance consists of multiple nodes. A policy network module is used to input the inspection task instance into a policy network, which includes a single encoder and multiple decision branches. The encoder outputs node embeddings based on the inspection task instance. One of the multiple decision branches is selected as the current decision branch, and the current UAV inspection status, unvisited mask, and the node embeddings are input into the aggregator of the current decision branch. The aggregator performs dynamic mask enhancement on the node embeddings and incorporates the UAV inspection status, outputting a multi-source information aggregation vector. The decoder of the current decision branch calculates the selection probability of each node based on the multi-source information aggregation vector and the node embeddings. The node update module is used to select the node with the highest probability as the next inspection node based on the selection probability, and update the UAV inspection status and unvisited mask. The trajectory generation module is used to repeatedly execute the strategy network module and the node update module until all inspection nodes have been visited, and output the complete inspection trajectory.

[0132] The following is a further explanation using a specific embodiment.

[0133] Example Step 1: Generate inspection task instances online and initialize the drone inspection status for each task instance; Each task instance consists of 1 base station node and 49 inspection nodes, i.e., N=50; the coordinates of each node are randomly generated within a unit area of ​​[0,1]×[0,1]. In this embodiment, each task instance is generated online during training, and the specific information of the generated task instances is shown in Table 1.

[0134] Table 1 Task Instance Information

[0135] Step 2: Construct a policy network containing a single encoder and multiple decision branches; The policy network includes a shared encoder, which uses an undirected graph. For input, output node embedding .

[0136] In this embodiment, the nodes are initially represented. Raw spatial data Spatial representation Send in In this embodiment, a series of sequentially connected encoding modules are... The value is 5.

[0137] Embed attention operations in neighborhood space During the process, take nodes The 20 nodes with the smallest distances form a neighborhood.

[0138] The policy network also includes four structurally identical decision branches, which serve as four parallel decision branches; the aggregator parameters are shared across all decision branches, while the decoder parameters are independent across all decision branches. The formula for calculating the attention compatibility value is:

[0139] in, For decision branches At time step For nodes Attention compatibility value, This is a constant set by the user; in this embodiment, it is set to 10. It is the hyperbolic tangent function. For nodes The node embedding vector, For decision branches Attention query value weight parameter matrix, For decision branches Attention key-value weight parameter matrix, for and The vertical dimension is set to 128 in this embodiment. For nodes that have already been inspected, their attention compatibility value is set to... To block it.

[0140] Finally, based on the obtained attention compatibility value, the decoder selects the node. The probability value of the current time step inspection node .

[0141] Step 3: For a single task instance, multiple complete inspection trajectories are generated in parallel using a policy network, and experience data is formed based on the time-step records. Each training round processes a batch of task instances to form batch experience data.

[0142] Step 4: Based on the batch experience data, perform multiple iterations to optimize the strategy network parameters, and clear the batch data after the update is completed; In this embodiment, each training round is processed Each task instance, after collecting data containing... After processing the batch experience data for each task instance, first configure the policy network parameters. Strategy evaluation of network parameters Copy to the asynchronous strategy network parameters respectively Asynchronous strategy for evaluating network parameters ; then proceed Secondary parameter update (in this embodiment) (Value is 4), each update starts from... Randomly selected from each task instance One task instance (in this embodiment) (The value is 32), using this All empirical data from each task instance are used as sub-batches to calculate the total loss and apply gradient descent to the policy network parameters. Strategy evaluation of network parameters Perform an update; This batch of experience data will be cleared after the next update.

[0143] Define the policy loss for a single task instance. , assess the loss Diversity regularization loss as follows:

[0144]

[0145]

[0146] Among them, hyperparameters The trimming range parameter is set to 0.2 in this embodiment for the trimming operation. Limit the ratio to Interval.

[0147] Therefore, for a single task instance, its total loss for:

[0148] in, , , These are the preset strategy loss weight, evaluation loss weight, and diversity regularization loss weight, which are set to 0.4, 0.4, and 0.2 respectively in this embodiment.

[0149] Each time the parameters are updated, the total loss of the sub-batch is taken from the sampled data. The arithmetic mean of the loss values ​​of each task instance is used to update the policy network parameters using gradient descent. Evaluation of network parameters with strategy .

[0150] Step 5: Repeat steps 3 and 4 until the preset training rounds are completed. In practical applications, select a decision branch from the policy network, select inspection nodes in sequence according to the highest probability, and generate inspection trajectories.

[0151] For the planning task, extract the undirected graph as in step 1 and normalize the static two-dimensional node coordinates to [0,1]×[0,1]. Initialize the UAV inspection state and unvisited mask. At each time step, calculate the node selection probability by the selected decision branch and select the node with the highest probability as the action until all nodes are visited, and output a complete inspection trajectory. If multiple decision branches are selected, output multiple complete inspection trajectories and select one for inspection based on the actual situation.

[0152] In this embodiment, a total of 100 rounds of training are performed. After training, a decision branch of the policy network is randomly selected and concatenated with the encoder of the policy network, such as... Figure 1 As shown, it is used for actual UAV inspection trajectory planning tasks.

[0153] In this embodiment, a drone inspection trajectory planning task in a real-world scenario is selected. This task includes 1 base station node and 39 inspection nodes, that is, the total number of nodes is N=40. The coordinate information of the 40 nodes is normalized to a unit area of ​​[0,1]×[0,1], and the normalized node information is shown in Table 2.

[0154] Table 2. Normalized node information

[0155] The trajectory planning results using the proposed method are shown in Table 3. Figure 3The drone starts from the base station node, visits all inspection nodes sequentially along its planned trajectory, and then returns to the base station node. The results show that the method proposed in this invention can efficiently and reliably plan drone inspection trajectories, and also possesses reliable generalization ability when facing drone inspection trajectory planning tasks in real-world scenarios.

[0156] Table 3. Results of UAV trajectory planning

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for planning UAV inspection trajectory based on multi-decision branch parallelism, characterized in that, Includes the following steps: S1, Obtain an inspection task instance and initialize the UAV inspection status. The task instance consists of multiple nodes. S2, input the inspection task instance into the policy network, the policy network including a single encoder and multiple decision branches; The encoder outputs node embeddings based on the inspection task instance; Select one of the multiple decision branches as the current decision branch, and input the current UAV inspection status, unaccessed mask, and the node embedding into the aggregator of the current decision branch; The aggregator performs dynamic masking enhancement on the node embedding and incorporates the UAV inspection status, outputting a multi-source information aggregation vector; the decoder of the current decision branch calculates the selection probability of each node based on the multi-source information aggregation vector and the node embedding. In S2, the encoder includes multiple encoding modules connected in sequence. Each encoding module includes a neighborhood feature aggregation layer and a feature refinement layer. The neighborhood feature aggregation layer calculates aggregated features through neighborhood space embedding attention operations and fuses the aggregated features with the input node representation through an adaptive gating mechanism to obtain a neighborhood fusion representation. The feature refinement layer performs a nonlinear transformation on the neighborhood fusion representation and uses the gating residual to obtain the output features. The neighborhood space embedding attention operation restricts attention computation to the neighborhood range of each node; The neighborhood spatial embedding attention operation includes: determining the set of neighboring nodes for each node based on spatial distance; calculating the similarity score between the node and its neighboring nodes, which contains spatial representations; calculating attention weights within the neighborhood using the similarity scores; weighting and aggregating the features of the neighboring nodes using the attention weights, and adding them to the input features to obtain the node feature update result; S3, based on the selection probability, select the node with the highest probability as the next inspection node, and update the UAV inspection status and unvisited mask; S4, repeat S2 and S3 until all inspection nodes are visited, and output the complete inspection trajectory.

2. The UAV inspection trajectory planning method based on multi-decision branch parallelism according to claim 1, characterized in that, In S2, the aggregator parameters of the multiple decision branches are shared, while the decoder parameters are independent; the multiple decision branches work in parallel during the training of the policy network.

3. The UAV inspection trajectory planning method based on multi-decision branch parallelism according to claim 1, characterized in that, In S2, the encoder takes the undirected graph corresponding to the inspection task instance as input. The undirected graph includes node data and raw spatial data. The node data is the static two-dimensional node coordinates of each node, and the raw spatial data is the spatial distance between each node.

4. The UAV inspection trajectory planning method based on multi-decision branch parallelism according to claim 1, characterized in that, In S2, the aggregator performs dynamic masking enhancement on the node embedding to obtain a dynamic inspection task vector, integrates the UAV inspection state into the dynamic inspection task vector to obtain a dynamic feature vector, and calculates a multi-source information aggregation vector based on the dynamic feature vector; the decoder concatenates the node embedding with the multi-source information aggregation vector to obtain a context embedding, calculates the attention compatibility value of each node based on the context embedding, and normalizes the attention compatibility value to obtain the selection probability.

5. The UAV inspection trajectory planning method based on multi-decision branch parallelism according to claim 1, characterized in that, In S1, the drone inspection status includes the drone's remaining battery power and the current inspection node index; in S3, the update process is as follows: set the element corresponding to the selected node in the unvisited mask to be visited, deduct the distance from the current node to the next inspection node from the drone's remaining battery power, and update the current inspection node index to the selected node.

6. The UAV inspection trajectory planning method based on multi-decision branch parallelism according to claim 1, characterized in that, In S2, the policy network is a network obtained through offline training; The training process includes: constructing a policy network containing the encoder and multiple decision branches, where the aggregator parameters of each decision branch are shared and the decoder parameters are independent; for each inspection task instance, using the policy network to generate multiple complete inspection trajectories in parallel and recording empirical data; iteratively optimizing the policy network parameters based on batch empirical data; wherein, the iterative optimization uses a total loss function composed of policy loss, evaluation loss, and diversity regularization loss for gradient descent updates, and the diversity regularization loss is calculated based on the probability difference of different decision branches selecting actions in the same state.

7. The UAV inspection trajectory planning method based on multi-decision branch parallelism according to claim 6, characterized in that, During the training process, an iterative optimization mechanism is constructed by introducing a policy evaluation network, an asynchronous policy network, and an asynchronous policy evaluation network; each decision branch records experience data during training.

8. A UAV inspection trajectory planning system based on multi-decision branch parallelism for implementing the UAV inspection trajectory planning method of claim 1, characterized in that, include: The task acquisition module is used to acquire inspection task instances and initialize the UAV inspection status. The task instance consists of multiple nodes. The policy network module is used to input the inspection task instance into the policy network, which includes a single encoder and multiple decision branches. The encoder outputs node embeddings based on the inspection task instance; Select one of the multiple decision branches as the current decision branch, and input the current UAV inspection status, unaccessed mask, and the node embedding into the aggregator of the current decision branch; The aggregator performs dynamic masking enhancement on the node embedding and incorporates the UAV inspection status, outputting a multi-source information aggregation vector; the decoder of the current decision branch calculates the selection probability of each node based on the multi-source information aggregation vector and the node embedding. The node update module is used to select the node with the highest probability as the next inspection node based on the selection probability, and update the UAV inspection status and unvisited mask. The trajectory generation module is used to repeatedly execute the strategy network module and the node update module until all inspection nodes have been visited, and output the complete inspection trajectory.

Images