An unmanned aerial vehicle search path planning method and system suitable for emergency rescue

Abstract

The application belongs to the technical field of aviation automatic control, and discloses a search path planning method and system of a UAV suitable for emergency rescue. The method comprises the following steps: facing a search area, generating a random training sample, restoring discrete nodes in a graph and resetting an environment, obtaining multiple pieces of trajectory data through multi-trajectory sampling, and inputting data features into a network model; based on an encoder, extracting problem features from the data features through an embedding layer, encoding the problem features through an attention layer, obtaining the relationship of nodes in the graph, and outputting graph encoding information; based on a decoder, obtaining the transition probability distribution of remaining nodes through a multi-head attention layer, a residual layer and a single-head attention layer, and updating network model parameters through the probability distribution; selecting a next search path node; obtaining an optimal search path node in multiple trajectories, and the like. The application is suitable for path planning of a UAV for implementing rescue search, and shortens the planning time of a search path.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of aviation automatic control, and particularly relates to a search path planning method and system of a UAV suitable for emergency rescue. BACKGROUND

[0002] Emergency rescue refers to that search personnel use artificial and auxiliary tool equipment to search for the position of the person in distress and provide preliminary medical treatment and other rescue needs for the person in distress, wherein the search methods mainly include artificial search, search dog search and instrument search. With the increasing breadth and depth of general aviation participating in emergency rescue, the UAV plays an important role in the aviation emergency rescue system. The UAV has the characteristics of low flight cost and easy operation and maintenance, and can meet various rescue needs.

[0003] Unlike manned aircraft, the UAV has small size, flexible take-off and landing, low flight height and strong maneuverability, and is more suitable for close-range reconnaissance of complex ground, especially in the case of limited resources and time after a disaster, the search personnel set the search task sub-target according to the personnel density of the reconnaissance area, reasonably plan the search path of multiple task sub-targets of the UAV, use the camera loaded on the UAV to collect ground information from different angles, quickly determine the target position and timely carry out rescue actions.

[0004] The UAV path planning method mainly includes a traditional method based on knowledge guidance, which plans the path by learning heuristic rules. The heuristic rules mainly include simulated annealing algorithm, genetic algorithm, list search algorithm, evolutionary programming, evolutionary strategy, ant colony algorithm, artificial neural network and the like, but this kind of method can only quickly obtain the optimal solution of a small-scale problem, and cannot meet the task demand of quickly solving a certain search scale, therefore, a new technical scheme is needed to solve the above problems and reduce the time cost of the UAV in executing rescue search under certain search scale conditions.

[0005] Through the above analysis, the problems and defects of the prior art are that in the actual application environment of the prior art, the search efficiency of the rotor UAV in the emergency rescue process is low, the speed of determining the target position of the object for different terrain environments is slow, and the accuracy of the object image processing is poor, which cannot provide effective information for timely rescue actions. SUMMARY

[0006] To overcome the problems in the related art, the embodiment of the present application provides a search path planning method and system of a UAV suitable for emergency rescue. The mainstream of the prior art is a swarm intelligence algorithm, such as a particle swarm algorithm and an ant colony algorithm, but such algorithms are limited to the calculation of a node number within 100, and the calculation speed is getting slower and slower, but compared with the model of the present application, the speed advantage of the prediction process is very obvious. The present application aims to realize the search path planning of a UAV. Before planning, a certain number of discrete nodes must be passed through, and the planning process is to obtain a reasonable order for accessing these discrete nodes, and the path is to traverse the nodes in order. These nodes are called path nodes, and this path is called trajectory data.

[0007] The technical solution is as follows: a search path planning method of a UAV suitable for emergency rescue, comprising the following steps:

[0008] S1, when a rotor UAV performs a rescue search task, random training samples are generated for a search area, and sample pre-processing node coordinate data is obtained by completing random training sample data preprocessing through data enhancement according to the emergency rescue demand;

[0009] S2, restore the discrete nodes in the sample and reset the environment, obtain trajectory data of multiple path nodes by multi-trajectory sampling, and input the features of the multiple trajectory data into a network model;

[0010] S3, based on the encoder, extract problem features from the multiple trajectory data features through the embedding layer, encode the problem features through the attention layer, obtain the relationship of the discrete path nodes in the graph, and output the graph encoding information;

[0011] S4, based on the decoder, obtain the transition probability distribution of the target node according to the graph encoding information of the multiple trajectories, and update the network model parameters by using the probability distribution;

[0012] S5, according to the target node probability distribution and the result of the learning agent interacting with the environment, select the next search path target node;

[0013] S6, judge whether a loop is formed, if not, feed back the existing information to the network model, otherwise, calculate the search path length and obtain the optimal search path node in the multiple trajectories.

[0014] In step S3, the encoder has one embedding layer and multiple attention layers; the embedding layer transforms the two-dimensional features of node information into high-dimensional features through linear mapping; each attention layer is composed of one multi-head attention sublayer and one feedforward network sublayer, the multi-head attention sublayer calculates the weight of each attention head, and the node information of all attention heads in the sublayer is merged and output to the feedforward network sublayer for processing; the data between each attention layer needs to be normalized, and finally the encoder outputs the graph encoding information.

[0015] In one embodiment, the transformation formula of linear mapping is:

[0016]

[0017] In the formula, represents the initial problem features output by the embedding layer, x i represents the data features of node i, W represents the coefficient, and b represents the bias, W x and b x jointly represent the network parameters of the embedding layer;

[0018] The normalization data processing formula of the attention layer is:

[0019]

[0020] In the formula, represents the output of the first L-1 layers, represents the multi-head attention network of the Lth layer for node i, FF L represents the feedforward network layer, Norm L represents the batch normalization data processing of the network layer, represents the intermediate information combined with the node information of the previous layer, represents the node information of the Lth layer after path processing;

[0021] The formula for the way the encoder outputs the graph encoding information is:

[0022]

[0023] In the formula, represents the graph encoding information of the Lth layer averaged N node information .

[0024] In step S4, the decoder only has one attention layer, which includes a multi-head attention sublayer, a multi-layer perceptron-based residual sublayer, and a single-head attention sublayer; the decoder extracts the query information of the visited nodes in the graph at the current time based on the multi-head attention sublayer by inputting the graph encoding information and the mask information, and assigns the pointing information and the node information of the unvisited nodes to different attention heads for parallel calculation; the residual sublayer is used to strengthen the multi-head attention sublayer output to obtain the graph information containing the graph encoding information and the mask information; and the single-head attention sublayer is used to calculate the probability distribution of the target node.

[0025] In one embodiment, the calculation formula of the similarity of each single-head attention sublayer is as follows:

[0026]

[0027] In the formula, u j is the to-be-visited node, u j represents the similarity of the output node j in the multi-head attention sublayer, k j represents the pointing information of the node j, q represents the query information of the node j, i.e., the visited node adjacent to j, q T represents the transpose matrix of q; there are H attention heads in the multi-head attention sublayer, represents the dimension of the single-head attention sublayer; for the to-be-visited node, the numerical range of the similarity is processed by the hyperparameter C and the activation function tanh, and then the proportion of different to-be-visited nodes in the graph is calculated;

[0028] The calculation formula of the probability distribution of the target node is as follows:

[0029]

[0030] In the formula, u j represents the similarity of the final output of the single-head attention sublayer, represents the exponential value of the similarity of the target node i at time T, represents the sum of the exponential values of the similarities of the neighbors j of the node i, P i represents the probability of selecting the target node i, and is represented as P θ (a T =i|a 1：T-1 , s).

[0031] In step S5, the next search path target node is selected one by one, the total loss value is calculated after all the target nodes are visited, and the classical reinforcement learning algorithm is used to update the model, which uses the policy gradient optimization strategy of the loss value to learn and update the solution strategy; by collecting multiple trajectory data of instances, the average path length is used to calculate the total reward value, and the loss value of the information entropy is introduced and adjusted to improve the target loss function and improve the stability of the model.

[0032] In one embodiment, when selecting the next search path target node, multiple trajectory data use the same baseline value, which is formulated as follows:

[0033]

[0034] In the formula, R(π) j ) represents the reward value for the j-th trajectory, b i (s) represents the average reward value of M trajectories, which is shared by all M trajectories.

[0035] In one embodiment, the policy network output probability formula is:

[0036]

[0037] In the formula, s represents the N indexes of the node, and π i Let p represent the i-th complete trajectory. θ (π i |s) represents the trajectory π i The output probability value, This represents the sequence of actions before time t. Indicates the action output at time t. The single-step state transition probability, Πp θ This represents the cumulative product of the single-step probabilities.

[0038] In one embodiment, the gradient formula for the target loss value output by the policy network during backpropagation is:

[0039]

[0040] In the formula, J(θ) represents the average target loss value across M trajectories. R(π) represents the gradient value of the derivative of J(θ). i ) represents the trajectory π i The reward value, b i (s) represents the trajectory π i The baseline value, log pθ (π i |s) represents the logarithmic probability of the policy network, (R(π) i )-b i (s))log pθ (π i |s) represents the unbiased estimated trajectory π i The target loss value;

[0041] The formula for the loss value of adding information entropy to the target loss value is:

[0042]

[0043] In the formula, p θ (·|s) represents the probability value formed by any one trajectory under the same instance, H(p θ (·|s)) represents the information entropy of the probability distribution, and H(θ) represents the average value of the information entropy of the M trajectories;

[0044] The information entropy is controlled by adjusting two parameters λ and β, and the target loss function formula is:

[0045] L(θ)=J(θ)+λ·H(θ)·β epoch

[0046] In the formula, λ represents the proportional coefficient of the information entropy, the parameter β represents the decay rate of the information entropy, β epoch represents the convergence coefficient of the information entropy, the decay rate is a decimal number, and the convergence coefficient decreases with the increase of the training round epoch.

[0047] Another object of the present application is to provide an unmanned aerial vehicle search path planning system suitable for emergency rescue for implementing the unmanned aerial vehicle search path planning method, and the unmanned aerial vehicle search path planning system suitable for emergency rescue comprises:

[0048] A data preprocessing module is used for generating random training samples facing the search area, rotating original node information based on data enhancement, further enriching trajectory data features, and completing data preprocessing.

[0049] A plurality of trajectory data acquisition modules are used for restoring discrete nodes in the graph and resetting the environment, since different node sequences represent the same solution, different nodes are taken as sampling starting points based on multi-trajectory sampling, and a plurality of trajectory data are acquired, and data features are input into a network model.

[0050] An encoder is used for inputting T-time node information, aggregating node features by using an L-layer multi-head attention network, and outputting graph encoding information at T time.

[0051] A decoder is used for inputting graph encoding information and mask information, extracting query information of the visited nodes in the current time graph based on a multi-head attention sublayer, and distributing the pointing information and node information of the unvisited nodes to different attention heads for parallel calculation; the multi-head attention sublayer output is strengthened by a residual sublayer to obtain graph information containing graph encoding information and mask information; and the probability distribution of the target node is calculated based on a single-head attention sublayer.

[0052] The solving strategy updating module is configured to select a next target node according to the probability distribution of the target node and a result of interaction between the learning agent and the environment, calculate a total loss value after all the target nodes are accessed, optimize the strategy network by using a policy gradient of the loss value, and learn and update the solving strategy.

[0053] The trajectory optimal search path node acquisition module is configured to determine whether a loop is formed, feed back existing information to the network model if the loop is not formed, otherwise calculate a search path length, and acquire an optimal search path node in the multiple trajectories.

[0054] In combination with all the technical solutions described above, the advantages and positive effects of the present application are as follows: the search path planning method for the rotary-wing unmanned aerial vehicle suitable for emergency rescue provided by the present application includes the following steps: facing a search area, generating random training samples, and completing data preprocessing by data enhancement according to emergency rescue requirements; restoring discrete nodes in a graph and resetting an environment, acquiring multiple trajectory data by multi-trajectory sampling, and inputting data features into a network model; based on an encoder, extracting problem features from the data features by an embedding layer, encoding the problem features by an attention layer, acquiring relationships of nodes in the graph from the problem features, and outputting graph encoding information involving node mask information; based on a decoder, acquiring transition probability distribution of remaining nodes by using a multi-head attention layer, a residual layer and a single-head attention layer according to the graph encoding information of the multiple trajectories, and updating network model parameters by using the probability distribution; selecting a next search path node according to a node probability distribution and a result of interaction between a learning agent and an environment; determining whether a loop is formed, feeding back existing information to the network model if the loop is not formed, otherwise calculating a search path length, and acquiring an optimal search path node in the multiple trajectories.

[0055] Experiments show that: the training device is an RTX 3090 server with 24GB video memory, and the test device is a Quadro P5000 server with 16GB video memory; the deep learning development framework Pytorch1.10 under the Ubuntu system is used; three types of data sets TSP20, TSP50 and TSP100 are randomly generated, each containing 100,000 training instances and 10,000 test instances; the embedding layer dimension of the model is 128, the number of attention layers is 6, the number of attention heads is 8, the activation range is 10, the optimizer is Adam, the learning rate is 0.0001, the batch training amount is 64, the training round is 300, the residual layer loss rate is 0.05, the proportion coefficient of the improved target loss function is 0.01, and the decay rate is 0.9. The evaluation index in the test process includes running time, and the difference between path length and shortest path length, that is, "gap = path length difference / shortest path length", the results show that the time of the invention for planning 10,000 paths is 2.5, 5.1 and 19.6 seconds respectively, while the traditional method needs 96.4, 548.7 and 2382.5 seconds respectively, in addition, the average gap obtained by the invention is 0.01%, 0.03% and 0.16% respectively.

[0056] For the rescue search path planning undertaken by the unmanned aerial vehicle, the rescue personnel need to set the position points that must be passed through in the search task according to the actual scene, and then use the invention to obtain an optimal path to quickly provide the unmanned aerial vehicle with an execution route, so as to facilitate the unmanned aerial vehicle to quickly implement the rescue search task. The biggest advantage of the invention is that the model is trained by using a machine learning method, the influence of the node scale on the solving time of the model is small, and training plans can be developed for different orders of magnitude requirements. In practical application, not only an optimal path result can be obtained, but also the planning time of the search path is shortened.

[0057] For the rescue search path planning undertaken by the unmanned aerial vehicle, the rescue personnel need to set the position points that must be passed through in the search task according to the actual scene, and then use the invention to obtain an optimal path to quickly provide the unmanned aerial vehicle with an execution route, so as to facilitate the unmanned aerial vehicle to quickly implement the rescue search task. The network model of the invention has the following advantages: based on multi-trajectory sampling, sampling deviation of trajectory data is avoided; based on data enhancement, trajectory data features are further enriched; based on a target loss function combining information entropy, the model is guided to explore a more optimal feasible solution space; based on the network model of the coding-decoding framework, the learning ability of the encoder is improved by adjusting the attention mechanism network, and the generalization ability of the decoder is improved by adding a residual sublayer. The invention overcomes the problem that the prior art cannot be applied in the actual application environment. The model is trained by using a machine learning method, the influence of the node scale on the solving time of the model is small, and training plans can be developed for different orders of magnitude requirements. An optimal search path can be quickly obtained, and the search efficiency of the rotary-wing unmanned aerial vehicle in the emergency rescue process is improved.

[0058] The application is embedded in the software tool of the unmanned aerial vehicle, the problem to be solved is input interactively in the software, and the corresponding scheme is output. The reinforcement learning method of the application can assist the unmanned aerial vehicle in path planning and can be effectively applied to the unmanned aerial vehicle path planning scene. BRIEF DESCRIPTION OF DRAWINGS

[0059] The drawings incorporated into the specification and forming a part of the specification, show embodiments consistent with the present disclosure, and together with the specification serve to explain the principles of the present disclosure;

[0060] Figure 1 is the principle diagram of the unmanned aerial vehicle search path planning method suitable for emergency rescue provided by the embodiment of the application;

[0061] Figure 2 is the flowchart of the unmanned aerial vehicle search path planning method suitable for emergency rescue provided by the embodiment of the application;

[0062] Figure 3 is the schematic diagram of the unmanned aerial vehicle search path planning system suitable for emergency rescue provided by the embodiment of the application;

[0063] Figure 4 is the network model of the reinforcement learning method used in the path planning of the application;

[0064] Fig. 5(a) is a schematic diagram of the berlin52 nodes in the solution path provided by the embodiment of the application, which are in a generally dispersed and chaotic distribution;

[0065] Fig. 5(b) is a schematic diagram of the lin105 nodes in the solution path provided by the embodiment of the application, which are in a generally dispersed and semi-regular and chaotic distribution;

[0066] Fig. 5(c) is a schematic diagram of the pr136 nodes in the solution path provided by the embodiment of the application, which are in a generally dispersed and regular distribution;

[0067] Fig. 5(d) is a schematic diagram of the d198 nodes in the solution path provided by the embodiment of the application, which are in a partially concentrated and chaotic distribution;

[0068] In the figure: 1, data preprocessing module; 2, multiple trajectory data acquisition module; 3, encoder; 4, decoder; 5, solution strategy updating module; 6, trajectory optimal search path node acquisition module. DETAILED DESCRIPTION

[0069] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below with reference to the drawings. In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the spirit of the present application, so the present application is not limited to the specific implementations disclosed below.

[0070] The search path planning process of the rotor unmanned aerial vehicle suitable for emergency rescue provided by the embodiment of the present application is as shown in Figure 1 The search path planning process of the rotor unmanned aerial vehicle suitable for emergency rescue provided by the embodiment of the present application is as shown in

[0071] Illustratively, emergency rescue often uses unmanned aerial vehicles for rescue search. When carrying out a rescue search task, the unmanned aerial vehicle takes off from an initial point after receiving a task instruction. In order to effectively shorten the rescue search time, the unmanned aerial vehicle will choose a flight path with the shortest time as much as possible and each task sub-target will only be reached once. After all the task sub-targets are searched, the unmanned aerial vehicle will return to the initial point of the task.

[0072] It can be understood that the typical characteristics of the rescue search process are: the initial point is unique, the final target node is still the initial point, there are multiple sub-target nodes in the middle, the unmanned aerial vehicle is required to start from the initial point, traverse all the sub-target nodes, and finally return to the initial point.

[0073] Compared with manned aircraft, unmanned aerial vehicles are easy to operate and maintain, do not require professional runways and takeoff and landing conditions, have low construction and flight costs, and effectively reduce life and property losses caused by search work. According to the flight principle and mechanical structure, unmanned aerial vehicles mainly include fixed-wing unmanned aerial vehicles and rotary-wing unmanned aerial vehicles. Among them, the fixed-wing unmanned aerial vehicle changes the axial acceleration, heading angle, pitch angle and roll angle of the aircraft body by controlling the engine thrust, rudder, elevator and aileron, so as to generate different axial forces and moments, so that the fixed-wing unmanned aerial vehicle has multiple flight attitudes. However, in the flight process, the fixed-wing unmanned aerial vehicle must maintain continuous motion state and constantly adjust the flight attitude and turning radius according to environmental factors. Compared with the fixed-wing unmanned aerial vehicle, the power system of the rotary-wing unmanned aerial vehicle can offset the aerodynamic force and moment of adjusting the flight attitude, simplify the attitude control by generating vertical lift, and avoid the partial loss of flight coverage area caused by turning radius. In order to realize full coverage of the rescue search area, the rotary-wing unmanned aerial vehicle is developed for rescue search path planning.

[0074] When performing a rescue search task, the unmanned aerial vehicle takes off from an initial point according to a task instruction, selects a flight path with the shortest time as far as possible for rescue search, returns to the initial point after all sub-target information collection is completed, so as to ensure that emergency personnel can obtain the rescue position in time and ensure that the rescue action can be carried out in time and in order. The process of constructing the shortest path may output multiple different route trajectories, so based on multi-trajectory sampling, multiple trajectories are generated from the initial point with equal weight, the information entropy of the initial point is maximized, the decision mechanism of the model is enriched and optimized, and biased decision behavior is avoided.

[0075] The technical solutions of the present application will be further described below in combination with specific embodiments.

[0076] As shown in Figure 1, the embodiment of the present application provides a rotary-wing unmanned aerial vehicle search path planning method suitable for emergency rescue, which comprises the following steps: Figure 1

[0077] S1, when the rotary-wing unmanned aerial vehicle performs a rescue search task, random training samples are generated for the search area, and data preprocessing of the random training samples is completed through data enhancement according to the emergency rescue demand, so as to obtain node coordinate data after sample preprocessing;

[0078] It can be understood that random training samples can be generated by using the random function, and in data enhancement, the method similar to image enhancement, rotation and flipping, is used. The data preprocessing is the data enhancement;

[0079] The sample data is point coordinates, which are mapped in a unit coordinate. Before being put into the model, the sample data should be expressed as node coordinate data, which is used to form graph information between nodes (including point sequence and position relationship).

[0080] ​S2, reduce the discrete nodes in the sample and reset the environment, obtain trajectory data of multiple path nodes by multi-trajectory sampling, and input the trajectory data features of the multiple trajectories into the network model;

[0081] It can be understood that the multi-trajectory sampling is to simulate a plurality of feasible paths randomly, and then modify the neural network parameters by using the simulation probability (a certain sample size is required). The Markov decision chain of reinforcement learning needs a large number of samplings, and the probability calculation is similar to the Monte Carlo method. The algorithm used for training is a classical reinforcement learning method, which improves the target loss value by calculating the shared baseline and information entropy of the multiple trajectories, and updates the parameters of the policy network by using the gradient of the back-propagated target loss value, so as to optimize the solving strategy of the network model.

[0082] S3, based on the encoder, extracting problem features from the multiple trajectory data features through the embedding layer, encoding the problem features by using the attention layer, obtaining the relationship of the discrete path nodes in the graph, and outputting the graph encoding information, the graph encoding information including discrete node mask information;

[0083] For example, the encoder has one embedding layer and multiple attention layers; the embedding layer transforms the two-dimensional features of the discrete node information into high-dimensional features by linear mapping; the parameters between different attention layers are not shared, and each attention layer is composed of one multi-head attention sublayer and one feedforward network sublayer, the multi-head attention sublayer calculates the weight of each attention head, and the discrete node information of all attention heads in the sublayer is merged and output to the feedforward network sublayer for processing. Normalization data processing is needed between each attention layer, and the graph encoding information is finally output by the encoder; wherein all nodes are discrete, but will be divided into visited and unvisited discrete nodes in the calculation process;

[0084] S4, based on the decoder, obtaining the transition probability distribution of the target node by using the multi-head attention layer, the residual layer and the single-head attention layer according to the graph encoding information of the multiple trajectories, and updating the network model parameters by using the probability distribution;

[0085] For example, the decoder only has one attention layer, which includes a multi-head attention sublayer, a residual sublayer based on a multi-layer perceptron, and a single-head attention sublayer; the decoder inputs the graph encoding information and the mask information, extracts the query information of the visited discrete nodes in the graph at the current time based on the multi-head attention sublayer, and distributes the pointing information of the unvisited discrete nodes and the discrete node information to different attention heads for parallel calculation; the residual sublayer strengthens the multi-head attention sublayer output to obtain the graph information containing the graph encoding information and the mask information; the probability distribution of the target node is calculated based on the single-head attention sublayer;

[0086] S5. Based on the probability distribution of the target node and the results of learning the interaction between the agent and the environment, select the next target node for the search path;

[0087] The total loss value is calculated after all target nodes have been visited. The policy gradient of the loss value is used to optimize the policy network, learn and update the solution policy.

[0088] S6. Determine whether a loop has been formed. If no loop has been formed, feed the existing information back to the network model. Otherwise, calculate the search path length and obtain the optimal search path node among multiple trajectories.

[0089] It's understandable that in determining whether a loop has formed, the check is that all nodes are traversed only once.

[0090] To find the optimal search path node among multiple trajectories, calculate the Euclidean distance between the coordinates of each discrete node, thereby calculating the final distance length of each path, and finally selecting the path node trajectory with the shortest distance.

[0091] Example 2, taking a rotary-wing UAV as an example, the UAV search path planning method for emergency rescue provided in this embodiment of the invention includes the following steps:

[0092] Step 1: For the search area, generate random training samples, and based on data augmentation, rotate the original node information to further enrich the trajectory data features and complete the data preprocessing.

[0093] Step 2: Restore the discrete nodes in the graph and reset the environment. Since different node sequences can represent the same solution, use multi-trajectory sampling to traverse different nodes as the sampling starting point to obtain multiple trajectory data, avoid sampling bias in the trajectory data, and input the data features into the network model.

[0094] Step 3: The encoder takes the node information at time T as input, aggregates the node features using an L-layer multi-head attention network, and outputs the graph encoding information at time T. Increasing the number of layers in the multi-head attention network enhances the learning ability of the network model.

[0095] In this embodiment of the invention, the encoder has one embedding layer and multiple attention layers. The embedding layer transforms the two-dimensional features of the node information into high-dimensional features through linear mapping, as shown in Equation (1).

[0096]

[0097] In the formula, x represents the initial problem features output by the embedding layer. i Let W represent the data characteristics of node i, W represent the number of systems, and b represent the bias. x and b x The combined parameters represent the network parameters of the embedding layer.

[0098] Different attention layers do not share parameters, and each attention layer has 1 multi-head attention sublayer and 1 feedforward network sublayer. The multi-head attention sublayer calculates the weight of each attention head, and the node information of all attention heads of the sublayer is combined and output to the feedforward network sublayer for processing. Normalization processing is required between each attention layer, and the data processing of the Lth attention layer is shown in formula (2).

[0099]

[0100] In the formula, represents the output of the previous L-1 layers, represents the multi-head attention network of the Lth layer for node i, FF L represents the feedforward network layer, Norm L represents the batch normalization data processing of the network layer, represents the intermediate information combined with the node information of the previous layer, represents the node information of the Lth layer after path processing.

[0101] The final output graph encoding information is shown in formula (3).

[0102]

[0103] In the formula, represents the graph encoding information of the Lth layer average N node information .

[0104] Step four, the decoder only has 1 attention layer, which includes a multi-head attention sublayer, a residual sublayer based on a multi-layer perceptron, and a single-head attention sublayer. The decoder extracts the query information of the visited nodes in the current time graph based on the multi-head attention sublayer by inputting the graph encoding information and the mask information, and distributes the pointing information and node information of the unvisited nodes to different attention heads for parallel calculation. The residual sublayer strengthens the multi-head attention sublayer output to obtain graph information containing graph encoding information and mask information. The probability distribution of the target node is calculated based on the single-head attention sublayer. The residual sublayer added between the multi-head attention sublayer and the single-head attention sublayer has a multi-layer perceptron with a dropout rate of 0.05. The dropout rate is defined as the proportion of randomly lost network model parameters, which can avoid model overfitting.

[0105] In the embodiment of the application, the multi-head attention sublayer distributes the unvisited nodes to each attention head according to the mask information, and quickly calculates the similarity of each to-be-visited node j in combination with the query information in the graph. The calculation method of each single-head attention sublayer is shown in formula (4):

[0106]

[0107] wherein j is a node to be accessed, u j denotes the similarity of the single-head attention sub-layer of the output node j in the multi-head attention sub-layer, k j denotes the pointing information of the node j, q denotes the query information of the node j, i.e., the accessed node adjacent to j, q T denotes the transpose matrix of q; there are H attention heads in the multi-head attention sub-layer, denotes the dimension of the single-head attention sub-layer; the value range of the similarity is processed by the hyperparameter C and the activation function tanh again, and then the proportion of different nodes to be accessed in the graph is calculated.

[0108] The probability distribution of the target node i is shown in formula (5):

[0109]

[0110] wherein u j denotes the similarity of the final output of the single-head attention sub-layer, denotes the exponential value of the similarity of the target node i at the T time, denotes the sum of the exponential values of the similarities of the neighbors j of the node i, P i denotes the probability of selecting the target node i, and is denoted as P θ (a T = i | a 1:T-1 , s).

[0111] Step five, according to the probability distribution of the target node and the result of the interaction between the learning agent and the environment, the next target node is selected, the total loss value is calculated after all the target nodes are accessed, the policy gradient of the loss value is used to optimize the policy network, and the solving policy is learned and updated. By collecting multiple trajectory data of instances, the total reward value is counted by using the average path length, the loss value of the information entropy is introduced and adjusted, the target loss function is improved, and the model stability is improved.

[0112] wherein the policy network is similar to a neural network, and the essence is that the target function in the model is different from the target function of traditional classification / fitting, and a reinforcement learning algorithm based on policy gradient optimization is used, so the neural network for training the solving policy is called a “policy network”.

[0113] In the embodiment of the application, in order to reduce the variance of the sampling gradient and avoid the model from falling into a local minimum value, the same baseline value is used for the M trajectories, and the baseline value is shown in formula (6).

[0114]

[0115] wherein R(π jrepresents the reward value of the jth trajectory, b i (s) represents the average value of the reward values of the M trajectories, shared by the M trajectories.

[0116] The policy network outputs the probability as shown in equation (7).

[0117]

[0118] where s represents the N order numbers of the node, π i represents the ith complete trajectory, p θ (π i |s) represents the probability value output according to the trajectory π i , and b represents the action sequence before the t time, represents the single-step state transition probability of the action output at the t time , and Πp θ represents the cumulative product of the single-step probabilities.

[0119] During the back propagation process, the gradient of the target loss value output by the policy network is shown in equation (8).

[0120]

[0121] where J(θ) represents the average target loss value of the M trajectories, represents the gradient value of the derivation of J(θ), R(π i ) represents the reward value of the trajectory π i , b i (s) represents the baseline value of the trajectory π i , log pθ (π i |s) represents the logarithmic probability of the policy network, (R(π i )-b i (s))log pθ (π i |s) represents the unbiased estimation of the target loss value of the trajectory π i .

[0122] Since each trajectory will compete with the other M-1 trajectories, the initial information entropy can be maximized, the diversity of the trajectory sample data is improved, and it is helpful to continuously correct the shared baseline. However, the initial information entropy cannot be continuously retained in the subsequent nodes. In order to avoid the model converging to a suboptimal strategy too early, the loss value of the information entropy is added in the target loss value, as shown in equation (9).

[0123]

[0124] where p θP (·|s) represents the probability value formed by any one trajectory under the same instance, H (P (·|s) ) represents the information entropy of the probability distribution, and H (0) represents the average value of the information entropy of the M trajectories. θ (·|s)) represents the information entropy of the probability distribution, and H(0) represents the average value of the information entropy of the M trajectories.

[0125] In the embodiment of the present application, in order to give full play to the advantages of different loss functions, the information entropy is controlled by adjusting two parameters lambda and beta, and the target loss function formula is:

[0126] L (0) = J (0) + lambda H (0) beta epoch (10)

[0127] In the formula, lambda represents the proportion coefficient of the information entropy, parameter beta represents the decay rate of the information entropy, beta epoch represents the convergence coefficient of the information entropy, the decay rate is a decimal number, and the convergence coefficient decreases with the increase of the training round number epoch.

[0128] The target loss function can make the model fully explore the feasible solution space at the initial stage, and gradually converge as the training proceeds, avoiding the problem of unstable solution of the model, and ensuring that the unmanned aerial vehicle can successfully complete the rescue search task.

[0129] Step six, it is judged whether a loop is formed, if a loop is not formed, the existing information is fed back to the network model, otherwise the search path length is calculated, and the optimal search path node in the multi-trajectory is obtained.

[0130] As shown in Embodiment 3, Figure 3 The unmanned aerial vehicle search path planning system suitable for emergency rescue provided by the embodiment of the present application comprises:

[0131] The data preprocessing module 1 is used for generating random training samples facing the search area, rotating the original node information based on data enhancement, further enriching the trajectory data features, and completing data preprocessing.

[0132] The multi-trajectory data acquisition module 2 is used for restoring the discrete nodes in the graph and resetting the environment. Since different node sequences can represent the same solution, different nodes are traversed as sampling starting points based on multi-trajectory sampling, and multiple trajectory data are acquired, so as to avoid sampling deviation of the trajectory data, and the data features are input into the network model.

[0133] The encoder 3 is used for inputting the node information at T time, aggregating the node features by using the L-layer multi-head attention network, and outputting the graph encoding information at T time. The number of layers of the multi-head attention network is increased, and the learning ability of the network model is improved.

[0134] The decoder 4 is configured to extract query information of the visited nodes in the current time graph based on the multi-head attention sublayer by inputting the graph encoding information and the mask information, and assign the pointing information and the node information of the unvisited nodes to different attention heads for parallel calculation. The graph information containing the graph encoding information and the mask information is strengthened by the residual sublayer after the multi-head attention sublayer. The probability distribution of the target node is calculated based on the single-head attention sublayer. The residual sublayer added between the multi-head attention sublayer and the single-head attention sublayer adopts a multi-layer perceptron with a dropout rate of 0.05. The dropout rate is defined as the proportion of randomly lost network model parameters, which can avoid model overfitting.

[0135] The solving strategy updating module 5 is configured to select the next target node according to the probability distribution of the target node and the result of the interaction between the learning agent and the environment, calculate the total loss value after all the target nodes are visited, optimize the strategy network by using the policy gradient of the loss value, and learn and update the solving strategy. The total reward value is calculated by using the average path length of the collected trajectory data, the loss value of the information entropy is introduced and adjusted, the target loss function is improved, and the stability of the model is improved.

[0136] The trajectory optimal search path node acquisition module 6 is configured to judge whether a loop is formed. If the loop is not formed, the existing information is fed back to the network model, otherwise the search path length is calculated to obtain the optimal search path node in the multi-trajectory.

[0137] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0138] The information interaction, execution process and the like between the above devices / units are based on the same concept as the method embodiments of the present application, and the specific functions and the technical effects brought by the method embodiments can be referred to the method embodiments part, which will not be described here.

[0139] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is only for the convenience of mutual distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiment.

[0140] According to the embodiments of the present application, the present application further provides a computer device, comprising at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0141] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program, wherein the computer program is executable by a processor to implement the steps in the above method embodiments.

[0142] The embodiments of the present application further provide an information data processing terminal, which is used to provide a user input interface to implement the steps in the above method embodiments when executed on an electronic device, and the information data processing terminal is not limited to a mobile phone, a computer, or a switch.

[0143] The embodiments of the present application further provide a server, which is used to provide a user input interface to implement the steps in the above method embodiments when executed on an electronic device.

[0144] The embodiments of the present application further provide a computer program product, which, when executed on an electronic device, enables the electronic device to implement the steps in the above method embodiments.

[0145] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods by a computer program to instruct related hardware to complete, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium can at least include any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0146] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0147] To further prove the positive effect of the above-mentioned embodiments, the present application based on the above technical solutions carries out the following experiments.

[0148] According to the distribution characteristics of the rescue search survey area, four typical examples with different node distributions are selected for path planning. In the solution path, the node distribution characteristics of Fig. 5(a) berlin52 are generally messy and scattered, the node density is not uniform, and it is similar to the randomly generated sample data, so its planning effect is the best. The node distribution characteristics of Fig. 5(b) lin105 are generally semi-messy and scattered, the node density is similar, and part of it is regularly distributed. The node distribution characteristics of Fig. 5(c) pr136 are generally uniformly scattered and regularly, and the node density is similar. The node distribution characteristics of Fig. 5(d) d198 are that most nodes are concentrated in a certain place, resulting in large differences in node density, and the path in the dense situation is difficult to get a lot of improvement, but it can still meet the demand of path planning. The planning method meets the actual application demand of multi-scene rescue search.

[0149] The above describes only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any modification, equivalent replacement and improvement made by those skilled in the art within the technical range disclosed by the present application, as long as it is within the spirit and principle of the present application, should be covered within the protection scope of the present application.

Claims

1. A method for planning search paths for unmanned aerial vehicles (UAVs) suitable for emergency rescue, characterized in that, The method includes the following steps: S1. When the rotary-wing UAV is carrying out a rescue and search mission, it generates random training samples in the search area. According to the emergency rescue needs, it completes the data preprocessing of the random training samples through data augmentation to obtain the node coordinate data after sample preprocessing. S2, restore the discrete nodes in the sample and reset the environment, obtain the trajectory data of multiple path nodes through multi-trajectory sampling, and input the features of multiple trajectory data into the network model; S3, based on the encoder, extracts problem features from multiple trajectory data features through the embedding layer, encodes the problem features using the attention layer, obtains the relationship between discrete path nodes in the graph, and outputs graph encoding information; S4, based on the decoder, uses the graph encoding information of multiple trajectories to obtain the transition probability distribution of the target node through a multi-head attention layer, a residual layer, and a single-head attention layer, and uses the probability distribution to update the network model parameters; S5. Based on the probability distribution of the target node and the results of learning the interaction between the agent and the environment, select the next target node for the search path; S6. Determine whether a loop has been formed. If no loop has been formed, feed the existing information back to the network model. Otherwise, calculate the search path length and obtain the optimal search path node among multiple trajectories.

2. The drone search path planning method for emergency rescue according to claim 1, characterized in that, In step S3, the encoder has one embedding layer and multiple attention layers; the embedding layer transforms the two-dimensional features of node information into high-dimensional features through linear mapping; Each attention layer consists of one multi-head attention sub-layer and one feedforward network sub-layer. The multi-head attention sub-layer calculates the weight of each attention head, and outputs it to the feedforward network sub-layer for processing by merging the node information of all attention heads in the sub-layer. Each attention layer requires normalized data processing, and the encoder ultimately outputs graph-encoded information.

3. The drone search path planning method for emergency rescue according to claim 2, characterized in that, The transformation formula for a linear mapping is: In the formula, x represents the initial problem features output by the embedding layer. i Let W represent the data characteristics of node i, W represent the number of systems, and b represent the bias. x and b x Together, they represent the network parameters of the embedding layer; The normalized data processing formula for the attention layer is: In the formula, This represents the output of the first L-1 layers. This represents a multi-head attention network at layer L for node i, where FF L Norm represents the feedforward network layer. L This indicates batch normalized data processing at the network layer. This represents intermediate information that combines information from the previous layer of nodes. This represents the node information of the Lth layer after path processing; The formula for encoding information in the encoder output image is: In the formula, This represents the average information of N nodes in the Lth layer. Image encoding information.

4. The UAV search path planning method for emergency rescue according to claim 1, characterized in that, In step S4, the decoder has only one attention layer, which includes a multi-head attention sub-layer, a residual sub-layer based on a multilayer perceptron, and a single-head attention sub-layer. The decoder takes the input graph encoding information and mask information, extracts the query information of the visited nodes in the graph at the current time step based on the multi-head attention sublayer, and assigns the pointing information and node information of the unvisited nodes to different attention heads for parallel computation; the multi-head attention sublayer is enhanced by the residual sublayer to output graph information containing graph encoding information and mask information; and the probability distribution of the target node is calculated based on the single-head attention sublayer.

5. The UAV search path planning method for emergency rescue according to claim 4, characterized in that, The formula for calculating the similarity of each single-head attention sublayer is as follows: In the formula, j represents the node to be visited, u j k represents the similarity of the output node j in the multi-head attention sub-layer to the single-head attention sub-layer. j q represents the pointer information of node j, and q represents the query information of node j, that is, the visited nodes adjacent to j. T This represents the transpose of q; there are H attention heads in the multi-head attention sublayer. This represents the dimension of the single-head attention sublayer; for the nodes to be visited, the similarity value range is further processed by the hyperparameter C and the activation function tanh, and then the proportion of different nodes to be visited in the graph is calculated. The formula for calculating the probability distribution of the target node is: In the formula, u j This indicates the similarity of the final outputs of a single-head attention sublayer. The index value representing the similarity of target node i at time T. P represents the sum of the exponential values ​​of the similarity between node i and its neighbors j. i Let P represent the probability of selecting target node i. θ (a T =i|a 1:T-1 ,s).

6. The drone search path planning method for emergency rescue according to claim 1, characterized in that, In step S5, after selecting the next target node for the search path, the total loss value is calculated after all target nodes have been visited. The policy gradient of the loss value is used to optimize the policy network, learn and update the solution policy. By collecting multiple trajectory data from instances, the total reward value is calculated using the average path length. At the same time, the loss value of information entropy is introduced and adjusted to improve the objective loss function and enhance the stability of the model.

7. The drone search path planning method for emergency rescue according to claim 6, characterized in that, When selecting the next target node for the search path, multiple trajectory data points use the same baseline value, which is calculated using the following formula: In the formula, R(π) j ) represents the reward value for the j-th trajectory, b i (s) represents the average reward value of M trajectories, which is shared by all M trajectories.

8. The drone search path planning method for emergency rescue according to claim 6, characterized in that, The formula for the output probability of the policy network is: In the formula, s represents the N indexes of the node, and π i Let p represent the i-th complete trajectory. θ (π i |s) represents the trajectory π i The output probability value, This represents the sequence of actions before time t. Indicates the action output at time t. The single-step state transition probability, ∏p θ This represents the cumulative product of the single-step probabilities.

9. The drone search path planning method for emergency rescue according to claim 6, characterized in that, The gradient formula for the target loss value output by the policy network during backpropagation is: In the formula, j(θ) represents the average target loss value across M trajectories. R(π) represents the gradient value of the derivative with respect to j(θ). i ) represents the trajectory π i The reward value, b i (s) represents the trajectory π i The baseline value, log pθ (π i |s) represents the logarithmic probability of the policy network, (R(π) i )-b i (s))log pθ (π i |s) represents the unbiased estimated trajectory π i The target loss value; The formula for the loss value of adding information entropy to the target loss value is: In the formula, p θ (·|s) represents the probability value of any trajectory being formed under the same instance, H(p) θ (·|s)) represents the information entropy of the probability distribution, and H(θ) represents the average information entropy of the M trajectories; By adjusting two parameters λ and β to control the information entropy, the target loss function formula is: L(θ)=J(θ)+λ·H(θ)·β epoch In the formula, λ represents the proportionality coefficient of information entropy, and the parameter β represents the decay rate of information entropy. epoch The convergence coefficient represents the information entropy, and the decay rate is a decimal. The convergence coefficient decreases as the number of training epochs increases.

10. A drone search path planning system suitable for emergency rescue, implementing the drone search path planning method according to any one of claims 1-9, characterized in that, The drone search path planning system suitable for emergency rescue includes: The data preprocessing module (1) is used to generate random training samples for the search area, and further enrich the trajectory data features by rotating the original node information based on data augmentation, thus completing the data preprocessing. The multi-trajectory data acquisition module (2) is used to restore the discrete nodes in the graph and reset the environment. Since different node sequences represent the same solution, the multi-trajectory sampling traverses different nodes as the sampling start point to acquire multiple trajectory data and input the data features into the network model. The encoder (3) is used to aggregate node features by using an L-layer multi-head attention network to input node information at time T and output graph coding information at time T. The decoder (4) is used to extract the query information of the visited nodes in the graph at the current time based on the multi-head attention sublayer by inputting graph encoding information and mask information, and to allocate the pointing information and node information of the unvisited nodes to different attention heads for parallel computation; the multi-head attention sublayer is strengthened by the residual sublayer to output graph information containing graph encoding information and mask information; the probability distribution of the target node is calculated based on the single-head attention sublayer. The solution strategy update module (5) is used to select the next target node based on the probability distribution of the target node and the result of learning the interaction between the agent and the environment. After all target nodes are visited, the total loss value is calculated, and the policy gradient of the loss value is used to optimize the policy network, learn and update the solution strategy. By collecting multiple trajectory data of the instance, the total reward value is calculated using the average path length, and the loss value of information entropy is introduced and adjusted to regulate the target loss function. The optimal search path node acquisition module (6) is used to determine whether a loop is formed. If no loop is formed, the existing information is fed back to the network model. Otherwise, the search path length is calculated and the optimal search path node among multiple trajectories is obtained.

Images