Unmanned aerial vehicle path planning method and device based on travel salesman problem, and medium

Abstract

The invention discloses an unmanned aerial vehicle path planning method and device based on a traveling salesman problem and a medium, belongs to the technical field of combinatorial optimization, and aims to solve the technical problems of low calculation efficiency, large memory overhead and weak generalization ability of the traveling salesman problem in unmanned aerial vehicle mapping. Constructing a deep reinforcement learning model based on an improved Transformer model and a symmetric regularization REINFORCE algorithm, and training the deep reinforcement learning model by adopting a training sample set to ensure that an approximately optimal path is found in an unmanned aerial vehicle map mapping instance; after the improved Transform model performs data enhancement on an unmanned aerial vehicle map mapping instance through a rotation-based random transformation data enhancement method, an encoder based on an orthogonal random feature fast attention mechanism encodes the enhanced data; and a complete feasible path is generated for each unmanned aerial vehicle mapping instance step by step by using an autoregression decoder combining an enhanced context embedding mechanism and a pointer network.

Description

Technical Field

[0001] The present invention relates to the field of combinatorial optimization technology, in particular to a method, device and medium for unmanned aerial vehicle path planning based on a traveling salesman problem. Background Art

[0002] The traveling salesman problem is a classic problem in the field of combinatorial optimization. It not only plays an important role in theoretical research, but also has been widely used in the fields of drone path planning, printed circuit board design, cargo distribution and DNA sequencing. The basic description of the traveling salesman problem is: given a set of cities and the distances between them, the traveling salesman is required to start from a certain city, visit each city only once, and finally return to the starting city, and find the travel route with the shortest total path distance.

[0003] When using drones for mapping, how to plan a path to efficiently capture terrain and buildings can be considered a classic traveling salesman problem, where each terrain and building that needs to be captured can be considered a city that needs to be reached, and the drone needs to capture images at each capture point and eventually return to the starting point. Since drones are subject to energy constraints and changes in the number of capture points during mapping, higher requirements are placed on the performance and generalization capabilities of the path planning algorithm. Therefore, an algorithm with accurate performance and strong generalization capabilities is urgently needed for drone path planning.

[0004] In recent years, more and more studies have tried to use machine learning methods to solve the traveling salesman problem in drone mapping. Compared with supervised learning that relies on the optimal solution, reinforcement learning has greater advantages in this field because it can obtain approximate optimal solutions without ground truth. In particular, deep reinforcement learning approximates the optimal solution by learning strategies, and has shown strong capabilities in dealing with complex decision space problems, so it has attracted much attention in solving the traveling salesman problem. According to the way of constructing the solution, deep reinforcement learning algorithms can be divided into two categories when solving the traveling salesman problem in drone mapping: end-to-end deep reinforcement learning and search-based deep reinforcement learning. Search-based deep reinforcement learning combines learning strategies with heuristic search, usually starting from an initial solution, and gradually approaches the optimal solution by continuously applying reinforcement learning strategies to improve the current solution. However, with the increase in the number of shooting points in drone mapping, the computational cost of iterative search increases significantly, and the performance of the solution is also very sensitive to the choice of search operators. In contrast, end-to-end deep reinforcement learning methods avoid the tedious search process by directly extracting features from raw data and optimizing path selection strategies without manually designing heuristic rules. In recent years, the Transformer model has performed well in solving the traveling salesman problem in drone mapping. However, as the number of shooting points increases, solving the traveling salesman problem in drone mapping based on the Transformer model faces challenges such as high computational complexity, large memory requirements, and limited generalization ability.

[0005] Traditional deep reinforcement learning methods based on the Transformer architecture usually use the dot product attention mechanism when processing UAV map mapping path planning, and its computational complexity is O(n 2 ), which makes it difficult to effectively apply them in large-scale drone mapping instances. In addition, existing methods often have difficulty in capturing both local and global information at the same time, resulting in insufficient solution performance and generalization ability. Summary of the invention

[0006] The technical task of the present invention is to provide a UAV path planning method, device and medium based on the traveling salesman problem to solve the problems of low computational efficiency, large memory overhead and weak generalization ability of the traveling salesman problem in UAV map surveying and mapping.

[0007] The technical task of the present invention is achieved in the following way: a UAV path planning method based on the traveling salesman problem, the method is as follows:

[0008] Obtaining a drone mapping instance; wherein the drone mapping instance includes a training sample set and a test sample set; the training sample set is composed of randomly generated shooting point location data; and the test sample set is the geographic location data of the actual drone mapping;

[0009] A deep reinforcement learning model is constructed based on the improved Transformer model and the REINFORCE algorithm based on symmetric regularization. The deep reinforcement learning model is trained using a training sample set to ensure that the drone finds a near-optimal path in the test sample set, thereby improving the generalization ability of the deep reinforcement learning model.

[0010] Among them, the improved Transformer model performs data augmentation on the UAV map surveying and mapping instance through a rotation-based random transformation (SR transformation) data augmentation method, and then encodes the enhanced data based on the encoder of the orthogonal random feature fast attention mechanism. Then, the autoregressive decoder combined with the enhanced context embedding mechanism and the pointer network is used to gradually generate a complete feasible path for each UAV map surveying and mapping instance. Finally, the deep reinforcement learning model is trained using the REINFORCE algorithm based on symmetric regularization, and a deep reinforcement learning model is obtained that can efficiently solve the traveling salesman problem of UAV map surveying and mapping.

[0011] Preferably, the rotation-based random transformation data enhancement method is to expand the original coordinate data of the drone shooting point to four perspectives by performing random rotation at the coordinate center, thereby increasing the diversity of drone map surveying and mapping instances; wherein the angle range of the random rotation is 0 to 2π.

[0012] Preferably, the encoder based on the orthogonal random feature fast attention mechanism encodes the enhanced data as follows:

[0013] The eight coordinate features of each node’s drone mapping instance after four times data augmentation are used as the input of the initial embedding layer;

[0014] The correlation between input features is calculated through a multi-head fast attention mechanism based on orthogonal random features, focusing on key nodes that have a significant impact on path selection;

[0015] The input features are added to the transformed features through residual connections to enhance the learning effect of the deep learning model; and feature standardization is performed in each layer to accelerate training and improve stability;

[0016] The normalized features are input into the feedforward network to extract high-level node features.

[0017] Preferably, an autoregressive decoder combined with an enhanced context embedding mechanism and a pointer network is used to gradually generate a complete feasible path for each UAV mapping instance as follows:

[0018] The high-level node features of the drone map shooting points after the encoder based on the orthogonal random feature fast attention mechanism are input into the autoregressive decoder, which combines the high-level node features with the enhanced context embedding information to generate a query vector;

[0019] The query vector is processed through a multi-head masked attention mechanism to capture the relationship between nodes and ensure that the generated path meets the constraints;

[0020] The probability distribution of visiting nodes is calculated through the pointer network, and then the next node to be visited is selected;

[0021] Among them, the complete feasible path is carried out step by step through the autoregressive process, and then the current state and query vector are summarized at each step to select the next drone map shooting point, and finally a sequence output is formed to represent the drone's shooting path.

[0022] Better yet, the orthogonal random feature-based fast attention mechanism uses orthogonal random features to approximate the kernel function in self-attention, thereby significantly reducing the computational complexity and memory consumption, as follows:

[0023] The kernel function K is defined by the random feature map φ(u), and the form of the kernel function K is: Among them, φ is a mapping from d dimension to r dimension;

[0024] for The rows of Q' and K' are and An efficient attention mechanism is obtained through mapping in the form of:

[0025] Among them, D b =diag(Q′((K′) T 1 L )); diag is to convert an input vector into a diagonal matrix; 1 L is a vector of length L containing all 1s; Att approx Represents the approximate attention mechanism; brackets indicate the order of calculation; the space complexity of the fast attention mechanism based on orthogonal random features is 0 (Lr+Ld+rd), and the time complexity is 0 (Lrd);

[0026] The φ processing functions f1, ..., f are taken as follows l and ω1,…,ω through Gram-Schmidt orthogonalization m :

[0027]

[0028] in, l=2, f1(u)=exp(u), f2(u)=exp(-u).

[0029] More preferably, the query vector is formulated as follows:

[0030]

[0031] Among them, q t represents the query vector; represents the static embedding of the first node on the path; represents the embedding of the last node on the path, which changes as the path is updated; l σ represents the sum of all node embeddings on the current encoding path; l dy represents the embedding updated based on the mean embedding of the most recently visited nodes to adapt to dynamic changes during decoding; l mean Represents the average embedding of the entire graph, focusing on global information;

[0032] The query vector is obtained through the context embedding mechanism. The context representation of the query vector not only guides the selection of the next node, but also allows the deep reinforcement learning model to adjust the embedding during the decoding process to capture dynamically changing contextual information and global information. At the same time, the autoregressive decoder calculates the probability distribution of access nodes through the pointer network to better capture the relationship between nodes, as follows:

[0033] Get the query vector q through the dot multiplication operation t Interaction with node j;

[0034] The interaction score between node i and node j is calculated by subtracting the cost cost(i, j) between the last node i and node j in the partial path. ij , which in turn encourages starting from an effective strategy, ensuring that the node closest to the current node is always selected as the next visit target;

[0035] The interaction score ij Through nonlinear transformation, it is converted to C·tanh(score ij );

[0036] By applying the Softmax function to generate the output probability distribution P, the selection of the next node to be visited is made random and reasonable, which not only improves the performance of the model but also effectively utilizes the spatial structure information between nodes.

[0037] Among them, the decoder adopts a dynamic selection mechanism based on coordinate distance. It does not need to calculate the distance from the current node to all nodes, but only selects the nearest neighbor nodes to participate in the attention calculation. It not only reduces the computational complexity, but also retains the most relevant local information, making the decoder more efficient when processing large-scale UAV map surveying and mapping path planning instances; at the same time, the decoder improves the model's comprehensive processing capabilities for local and global information, thereby improving the decoder's adaptability and efficiency in different path selections, while effectively reducing the time and memory overhead of attention calculations.

[0038] As a preference, the method also uses the Lion optimizer, which uses symbolic operations to calculate parameter updates and only tracks momentum, thereby significantly reducing memory overhead and achieving uniform update amplitudes in all dimensions. The formula is as follows:

[0039]

[0040] θ t =θ t-1 -αsign(m t );

[0041] Among them, m t represents the momentum vector; β1 represents the first momentum hyperparameter; The symbolic function representing the gradient of the loss function L with respect to the parameter θ; θ t Represents the parameters after the tth iteration; α represents the learning rate. The update rule based on symbolic function introduced by Lion optimizer not only simplifies the calculation process, but also shows good convergence and efficiency.

[0042] As a preferred method, the symmetric regularized REINFORCE algorithm introduces a symmetry-based baseline term in the process of training the deep reinforcement learning model. Specifically, multiple solutions are sampled from the drone map surveying instance after data enhancement, and the average return of the solution is used as a shared baseline. During the retraining process of the symmetric regularized REINFORCE algorithm, each solution sampled by the solver is pushed to exceed the shared baseline, thereby continuously optimizing the strategy so that the performance of all solutions tends to be consistent; wherein, the symmetric approximation is achieved by the symmetric regularized REINFORCE algorithm as follows:

[0043] Solution symmetry requires that solutions generated on the same TSP instance have the same target value. Solution symmetry is approximated by using the average reward of K sampled solutions to obtain the regularized loss L ss , the formula is as follows:

[0044]

[0045] Where log F θ Yes Fθ The log-likelihood of

[0046] The symmetry of the problem requires that the original problem and its rotation problem have the same solution. By sampling the rotation matrix to construct a symmetric shared baseline, the regularization loss L is obtained. ps , the formula is as follows;

[0047]

[0048] in, Q is the distribution of random orthogonal matrices;

[0049] Use the regularization loss term L inv Force Solver F θ With an invariant representation, the formula is as follows:

[0050] L inv =-Scos(g(h(x)),g(h(Q(x))));

[0051] where h(x) and h(Q(x)) denote the hidden representations of the original question and its rotation, respectively; Scos(a, b) is the cosine similarity between a and b; g is the projection head parameterized by a multilayer perceptron;

[0052] The solver F(θ) is learned by minimizing the total loss function, which is defined as follows:

[0053] L total =L ps +βL ss +αL inv ;

[0054] Among them, L ss represents the solution symmetry regularization loss; L ps represents the symmetry regularization loss of the problem; L inv represents the regularization loss that induces invariant representation; α and β∈[0,1] are weight coefficients;

[0055] The regularization schemes for solution symmetry and problem symmetry both improve the overall reward quality of the solution set by inducing competition between rewards, so that the solver can generate solutions with high rewards and small reward differences; the REINFORCE algorithm based on symmetry regularization can not only effectively utilize the symmetry in the UAV map surveying and mapping path planning problem, but also significantly improve the performance and generalization ability of the model.

[0056] An electronic device comprising: a memory and at least one processor;

[0057] Wherein, the memory stores a computer program;

[0058] The at least one processor executes the computer program stored in the memory, so that the at least one processor performs the drone path planning method based on the traveling salesman problem as described above.

[0059] A computer-readable storage medium having a computer program stored therein, wherein the computer program can be executed by a processor to implement the above-mentioned drone path planning method based on the traveling salesman problem.

[0060] The UAV path planning method, device and medium based on the traveling salesman problem of the present invention have the following advantages:

[0061] (i) The present invention inputs the training sample set into the deep reinforcement learning model, and performs data enhancement through random transformation based on rotation to improve the generalization ability of the model, and then uses the Transformer encoder with a fast attention mechanism based on orthogonal random features to obtain a high-level representation of the initial embedding, and combines a complex enhanced context embedding mechanism and an autoregressive encoder of a pointer network to gradually generate a complete feasible path for each drone map surveying sample, and continues training using the REINFORCE algorithm based on symmetric regularization. The deep reinforcement learning model finally generated can effectively solve the path problem of drone mapping in the test sample set, and then determine that the drone finds a nearly optimal path in the test sample set, which not only improves the performance and generalization ability of the traveling salesman problem for drone map surveying, but also effectively reduces the memory consumption of the model;

[0062] (ii) The symmetric regularized REINFORCE algorithm of the present invention takes the generated path as input, calculates the total distance of each point in the path, and designs a reward function (i.e., the negative value of the total distance) to encourage the generation of shorter paths; and through the policy gradient method, the REINFORCE algorithm adjusts the generation strategy during the training process to maximize the expected reward; samples multiple solutions from the data-enhanced drone map surveying and mapping instance, calculates the average return of these solutions as a shared baseline, pushes each sampled solution beyond the baseline, optimizes the strategy, and makes the performance of all solutions consistent. After hundreds of rounds of iterative training, the final deep reinforcement learning model can effectively solve the traveling salesman problem in drone map surveying and mapping with less than 1,000 city nodes, thereby helping drones find suboptimal shooting sequences among multiple shooting points, saving resources and improving work efficiency;

[0063] (III) The present invention provides a deep reinforcement learning method based on symmetric regularization to solve the traveling salesman problem in drone mapping. Specifically, the traditional dot product attention mechanism of the encoder part of the Tansformer model is replaced by an orthogonal random feature fast attention mechanism, which reduces the computational complexity from O(n 2) is reduced to O(n), which significantly improves the efficiency of processing large-scale UAV map surveying and mapping instances. Pointer networks and enhanced context embedding methods are also introduced, which not only improve the adaptability and decision-making accuracy of the model in dynamically changing environments, but also enable it to select the most relevant local information, thereby effectively reducing unnecessary calculations and memory usage. In addition, the Lion optimizer is used, and its symbolic operations and momentum tracking mechanisms further reduce memory overhead and enhance the scalability of the model in large-scale UAV map surveying and mapping path planning problems. Finally, the introduction of the REINFORCE algorithm based on symmetric regularization significantly enhances the solution performance and generalization ability of the model, better meeting the needs of large-scale UAV map surveying and mapping path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The present invention is further described below in conjunction with the accompanying drawings.

[0065] Attached Figure 1 This is a schematic diagram of the structure of the deep reinforcement learning model;

[0066] Attached Figure 2 Schematic diagram of the approximation of the conventional attention mechanism AV by random feature mapping (the dotted blocks indicate the calculation order, with the corresponding time complexity). DETAILED DESCRIPTION

[0067] The UAV path planning method, device and medium based on the traveling salesman problem of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments of the specification.

[0068] Embodiment 1:

[0069] Problem description: Two-dimensional Euclidean traveling salesman problem in drone mapping. In this problem, the points photographed by the drone are regarded as points on a two-dimensional plane, and the distance between the points is calculated by the Euclidean distance. The traveling salesman problem in drone mapping can be formalized as follows: Let G = (V, E) be a complete graph, where V = {V i |1≤i≤N} represents the set of drone shooting points, E={e ij |1≤i, j≤N} represents the set of edges connecting the drone shooting points, and N is the number of drone shooting points. The coordinates of each drone shooting point i are (x i ,y i ). The distance from drone shooting point i to drone shooting point j is: The goal of the traveling salesman problem in drone mapping is to find a better planning route σ so that the total travel distance L(σ) is minimized. It can be formalized as: Where σ is the arrangement of the drone’s shooting points.

[0070] The solution σ to the traveling salesman problem in drone mapping can be generated sequentially by selecting the next shooting point from all the shooting points to be visited until returning to the starting shooting point. This process can be regarded as a Markov decision process (MDP) with the following specific characteristics:

[0071] - Status: Status t It is an ordered sequence of the shooting points currently visited by the traveling salesman instance in the drone map survey. The initial state s0 is an empty sequence, and the final state s N It is a complete visit sequence of all drone shooting points.

[0072] -Action: Action a t is to select the next unvisited shooting point.

[0073] -Reward: The reward is the negative distance from the current shooting point to the next shooting point, that is, r(a i ,s i )=-d σ(i)σ(i+1) .

[0074] -Transfer: State transfer is to select a shooting point from a set of shooting points that have not been visited to visit.

[0075] -Strategy: Strategy π θ (σ|s) is a probability distribution represented by a neural network, and the parameters θ are trainable weights. Given a sequence of visited points, the policy function π θ (σ|s) returns the probability distribution of the next shooting point. The policy function is defined as:

[0076]

[0077] The goal is to maximize the expected cumulative reward, which is given by the following formula:

[0078]

[0079] By optimizing the strategy π θ The parameter θ can find the optimal visit sequence to minimize the total travel distance.

[0080] This embodiment provides a method for UAV path planning based on the traveling salesman problem, and the method is specifically as follows:

[0081] S1. Obtaining a drone mapping instance; wherein the drone mapping instance includes a training sample set and a test sample set; the training sample set is composed of randomly generated shooting point location data; and the test sample set is the geographic location data of the actual drone mapping;

[0082] S2. A deep reinforcement learning model is constructed based on the improved Transformer model and the REINFORCE algorithm based on symmetric regularization, and the deep reinforcement learning model is trained using a training sample set to ensure that a near-optimal path is found in the drone test sample set, thereby improving the generalization ability of the deep reinforcement learning model; the details are as follows:

[0083] S201. The improved Transformer model performs data enhancement on the UAV map surveying and mapping instance through the rotation-based random transformation (SR transformation) data enhancement method, and then encodes the enhanced data based on the encoder of the orthogonal random feature fast attention mechanism; wherein, as shown in the attached Figure 1 As shown in the figure, the improved Transformer model includes an encoder based on orthogonal random feature fast attention mechanism and an autoregressive decoder. In the encoder based on orthogonal random feature fast attention mechanism, the multi-head self-attention mechanism is optimized, a linear multi-head fast attention mechanism based on orthogonal random features is introduced, and the position encoding is removed. The other components remain unchanged. The autoregressive decoder innovates the context embedding, adds a dynamic selection mechanism based on coordinate distance and an average embedding mechanism, and adds a pointer network. The input of the encoder based on orthogonal random feature fast attention mechanism is the coordinate features of the drone map surveying and mapping shooting points after four times data enhancement, a total of eight features (the input of the encoder based on orthogonal random feature fast attention mechanism is the location of the drone shooting point, and the coordinate information of each position consists of abscissa and ordinate, i.e. (x, y); the coordinates of the drone shooting points are four times data enhanced based on random transformation before encoding (i.e. before inputting the encoder), so there are eight features). The improved Transformer model calculates the correlation between the input features through the multi-head fast attention mechanism based on orthogonal random features, focusing on the points that have a greater impact on path selection. Subsequently, the input features are added to the transformed features through residual connections to enhance the learning effect of the improved Transformer model. Feature normalization is performed in each layer to accelerate training and improve stability, and finally the processed features are input into the feedforward network to extract high-level features. After being processed by the encoder based on the fast attention mechanism of orthogonal random features, a high-level representation of the shooting point is obtained and input into the autoregressive decoder. The autoregressive decoder combines the encoded node features with the enhanced context embedding information to generate a query vector. Next, the query vector is processed by a multi-head mechanism to capture the relationship between nodes to ensure that the generated path satisfies the constraints, and finally the next node to be visited is selected by calculating the probability distribution of the visited nodes through the pointer network. The generation of the path is carried out step by step through the autoregressive process, and the current state and query vector are used in each step to select the next shooting point, and finally a sequence output is formed to represent the shooting path of the drone.

[0084] S202, using an autoregressive decoder combined with an enhanced context embedding mechanism and a pointer network to gradually generate a complete feasible path for each drone mapping instance;

[0085] S203. Use the REINFORCE algorithm based on symmetric regularization to train the deep reinforcement learning model to obtain a deep reinforcement learning model that can efficiently solve the traveling salesman problem of drone mapping. It can handle drone mapping instances with less than 1,000 city nodes, effectively improving the efficiency and accuracy of the solution.

[0086] In order to cope with the challenge of the sharp increase in resource consumption of the traditional Transformer model when processing long sequences, this embodiment designs an encoder that does not use position encoding and is based on the fast attention mechanism of orthogonal random random features, which can be effectively expanded to larger-scale drone map surveying instances. It is worth noting that the fast attention mechanism based on orthogonal random random features has not yet been applied to the path planning problem of drone map surveying. In addition, based on the symmetry of the drone map surveying instance, a rotation-based random transformation (SR transformation) data enhancement method is introduced.

[0087] The rotation-based random transformation data enhancement method in step S201 of this embodiment is to expand the original coordinate data of the drone shooting point to four perspectives by performing random rotation at the coordinate center, thereby increasing the diversity of drone map surveying and mapping instances; wherein the angle range of the random rotation is 0 to 2π.

[0088] In the Transformer model, the traditional dot product attention mechanism has the following form, where is the so-called attention matrix:

[0089]

[0090] Here, exp(·) is applied element-wise, 1 L is a vector of all 1s of length L, and diag(·) is a diagonal matrix with the input vector as the diagonal. The time complexity of the traditional dot product attention mechanism is O(L 2 d), the space complexity is O(L 2 +Ld). This quadratic complexity will result in extremely high computing resources and memory requirements when processing large-scale drone mapping path planning. In order to solve this problem, the encoder based on the orthogonal random feature fast attention mechanism in step S201 of this embodiment encodes the enhanced data as follows:

[0091] S20101, the eight coordinate features of the drone mapping instance after four times data enhancement for each node are used as the input of the initial embedding layer;

[0092] S20102, through the multi-head fast attention mechanism based on orthogonal random features, the correlation between input features is calculated, focusing on the key nodes that have a significant impact on path selection;

[0093] S20103. Add the input features to the transformed features through residual connections to enhance the learning effect of the deep learning model; and perform feature standardization in each layer to accelerate training and improve stability;

[0094] S20104. Input the normalized features into the feedforward network to extract high-level node features.

[0095] As attached Figure 2 As shown, in this embodiment, the orthogonal random feature based fast attention mechanism uses orthogonal random features to approximate the kernel function in the self-attention, thereby significantly reducing the computational complexity and memory consumption, as follows:

[0096] (1) The kernel function K is defined by the random feature map φ(u). The kernel function K is in the form of: Among them, φ is a mapping from d dimension to r dimension;

[0097] (2) For The rows of Q' and K' are and An efficient attention mechanism is obtained through mapping, in the form of:

[0098] Among them, D b =diag(Q′((K′) T 1 L )); diag is to convert an input vector into a diagonal matrix; 1 L is a vector of length L containing all 1s; Att approx Represents the approximate attention mechanism; brackets indicate the order of calculation; the spatial complexity of the fast attention mechanism based on orthogonal random features is 0 (Lr+Ld+rd), and the time complexity is 0 (Lrd);

[0099] (3) Using the following form of φ processing function f1, ..., f l and ω1,…,ω through Gram-Schmidt orthogonalization m :

[0100]

[0101] in, l=2, f1(u)=exp(u), f2(u)=exp(-u).

[0102] In step S202 of this embodiment, the autoregressive decoder combined with the enhanced context embedding mechanism and the pointer network is used to gradually generate a complete feasible path for each drone mapping instance as follows:

[0103] S20201. Input the high-level node features of the drone map shooting points after the encoder based on the orthogonal random feature fast attention mechanism into the autoregressive decoder, and the autoregressive decoder combines the high-level node features with the enhanced context embedding information to generate a query vector;

[0104] S20202. Process the query vector through the multi-head masked attention mechanism to capture the node-to-node relationship and ensure that the generated path meets the constraints.

[0105] S20203. Calculate the probability distribution of accessing nodes through the pointer network, and then select the next node to be accessed;

[0106] Among them, the complete feasible path is carried out step by step through the autoregressive process, and then the current state and query vector are summarized at each step to select the next drone map shooting point, and finally a sequence output is formed to represent the drone's shooting path.

[0107] The formula of the query vector in step S20202 of this embodiment is as follows:

[0108]

[0109] Where qt represents the query vector; represents the static embedding of the first node on the path; represents the embedding of the last node on the path, which changes as the path is updated; l σ represents the sum of all node embeddings on the current encoding path; l dy represents the embedding updated based on the mean embedding of the most recently visited nodes to adapt to dynamic changes during decoding; l mean Represents the average embedding of the entire graph, focusing on global information;

[0110] The query vector in step S20202 of this embodiment is obtained through a context embedding mechanism. The context representation of the query vector not only guides the selection of the next node, but also allows the deep reinforcement learning model to adjust the embedding during the decoding process to capture dynamically changing context information and global information. At the same time, the autoregressive decoder calculates the probability distribution of access nodes through a pointer network to better capture the relationship between nodes, as follows:

[0111] (1) Obtain the query vector q through dot multiplication t Interaction with node j;

[0112] (2) Calculate the interaction score between node i and node j by subtracting the cost cost(i, j) between the last node i and node j in the partial path. ij , which in turn encourages starting from an effective strategy, ensuring that the node closest to the current node is always selected as the next visit target;

[0113] (3) The interaction score ij Through nonlinear transformation, it is converted to C·tanh(score ij );

[0114] (4) By applying the Softmax function to generate the output probability distribution P, the selection of the next node to be visited is made random and reasonable, which not only improves the performance of the model but also effectively utilizes the spatial structure information between nodes;

[0115] Among them, the decoder adopts a dynamic selection mechanism based on coordinate distance. It does not need to calculate the distance from the current node to all nodes, but only selects the nearest neighbor nodes to participate in the attention calculation. It not only reduces the computational complexity, but also retains the most relevant local information, making the decoder more efficient when processing large-scale UAV map surveying and mapping path planning instances; at the same time, the decoder improves the model's comprehensive processing capabilities for local and global information, thereby improving the decoder's adaptability and efficiency in different path selections, while effectively reducing the time and memory overhead of attention calculations.

[0116] The Lion optimizer is also used in this embodiment. The Lion optimizer uses symbolic operations to calculate parameter updates and only tracks momentum, thereby significantly reducing memory overhead and achieving a uniform update amplitude in all dimensions. The formula is as follows:

[0117]

[0118] θ t =θ t-1 -αsign(m t );

[0119] Where mt represents the momentum vector; β1 represents the first momentum hyperparameter; The symbolic function representing the gradient of the loss function L with respect to the parameter θ; θ t Represents the parameters after the tth iteration; α represents the learning rate. The update rule based on symbolic function introduced by Lion optimizer not only simplifies the calculation process, but also shows good convergence and efficiency.

[0120] The symmetric regularized REINFORCE algorithm in step S203 of this embodiment introduces a symmetry-based baseline term in the process of training the deep reinforcement learning model, specifically: multiple solutions are sampled from the drone map surveying and mapping instance after data enhancement, and the average return of the solutions is used as a shared baseline. During the retraining process of the symmetric regularized REINFORCE algorithm, each solution sampled by the solver is pushed to exceed the shared baseline, thereby continuously optimizing the strategy so that the performance of all solutions tends to be consistent; wherein, the symmetric approximation is achieved by the symmetric regularized REINFORCE algorithm as follows:

[0121] S20301. Solution symmetry requires that the solutions generated on the same traveling salesman problem instance have the same target value. The solution symmetry is approximated by using the average reward of K sampled solutions to obtain the regularized loss L ss , the formula is as follows:

[0122]

[0123] Where log F θ Yes F θ The log-likelihood of

[0124] S20302. Problem symmetry requires that the original problem and its rotation problem have the same solution. By sampling the rotation matrix, a shared baseline of problem symmetry is constructed to obtain the regularization loss L ps , the formula is as follows;

[0125]

[0126] in, Q is the distribution of random orthogonal matrices;

[0127] S20303, use regularization loss term L inv Force Solver F θ With an invariant representation, the formula is as follows:

[0128] L inv =-Scos(g(h(x)),g(h(Q(x))));

[0129] where h(x) and h(Q(x)) denote the hidden representations of the original question and its rotation, respectively; Scos(a, b) is the cosine similarity between a and b; g is the projection head parameterized by a multilayer perceptron;

[0130] S20304. Minimize the total loss function to learn the solver F(θ). The total loss function is defined as follows:

[0131] L total =L ps+βL ss +αL inv ;

[0132] Among them, L ss represents the solution symmetry regularization loss; L ps represents the symmetry regularization loss of the problem; L inv represents the regularization loss that induces invariant representation; α and β∈[0,1] are weight coefficients;

[0133] The regularization schemes for solution symmetry and problem symmetry both improve the overall reward quality of the solution set by inducing competition between rewards, so that the solver can generate solutions with high rewards and small reward differences; the REINFORCE algorithm based on symmetry regularization can not only effectively utilize the symmetry in the UAV map surveying and mapping path planning problem, but also significantly improve the performance and generalization ability of the model.

[0134] Embodiment 2:

[0135] This embodiment also provides an electronic device, including: a memory and a processor;

[0136] Wherein, the memory stores computer-executable instructions;

[0137] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the drone path planning method based on the traveling salesman problem in any embodiment of the present invention.

[0138] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0139] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state storage devices.

[0140] Embodiment 3:

[0141] This embodiment also provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions are loaded by a processor, so that the processor executes the UAV path planning method based on the traveling salesman problem in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0142] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0143] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0144] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0145] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A UAV path planning method based on the traveling salesman problem, characterized in that: The method is as follows: Obtaining a drone mapping instance; wherein the drone mapping instance includes a training sample set and a test sample set; the training sample set is composed of randomly generated shooting point location data; and the test sample set is the geographic location data of the actual drone mapping; A deep reinforcement learning model is constructed based on the improved Transformer model and the REINFORCE algorithm based on symmetric regularization, and the deep reinforcement learning model is trained using a training sample set to ensure that a near-optimal path is found in the UAV mapping instance. Among them, the improved Transformer model performs data augmentation on the UAV map surveying and mapping instance through a rotation-based random transformation data augmentation method, and then encodes the enhanced data based on the encoder of the orthogonal random feature fast attention mechanism. Then, the autoregressive decoder combined with the enhanced context embedding mechanism and the pointer network is used to gradually generate a complete feasible path for each UAV map surveying and mapping instance. Finally, the deep reinforcement learning model is trained using the REINFORCE algorithm based on symmetric regularization, and a deep reinforcement learning model is obtained that can efficiently solve the traveling salesman problem of UAV map surveying and mapping.

2. The UAV path planning method based on the traveling salesman problem according to claim 1 is characterized in that: The rotation-based random transformation data enhancement method is to expand the original coordinate data of the drone shooting points to four perspectives by performing random rotation at the coordinate center, thereby increasing the diversity of drone map surveying and mapping instances; wherein the angle range of the random rotation is 0 to 2π.

3. The UAV path planning method based on the traveling salesman problem according to claim 1 or 2 is characterized in that: The encoder based on the orthogonal random feature fast attention mechanism encodes the enhanced data as follows: The eight coordinate features of each node’s drone mapping instance after four times data augmentation are used as the input of the initial embedding layer; The correlation between input features is calculated through a multi-head fast attention mechanism based on orthogonal random features, focusing on key nodes that have a significant impact on path selection; The input features are added to the transformed features through residual connections to enhance the learning effect of the deep learning model; and feature standardization is performed in each layer to accelerate training and improve stability; The normalized features are input into the feedforward network to extract high-level node features.

4. The UAV path planning method based on the traveling salesman problem according to claim 3 is characterized in that: The autoregressive decoder combined with the enhanced context embedding mechanism and the pointer network is used to gradually generate a complete feasible path for each UAV mapping instance as follows: The high-level node features of the drone map shooting points after the encoder based on the orthogonal random feature fast attention mechanism are input into the autoregressive decoder, which combines the high-level node features with the enhanced context embedding information to generate a query vector; The query vector is processed through a multi-head masked attention mechanism to capture the relationship between nodes and ensure that the generated path meets the constraints; The probability distribution of visiting nodes is calculated through the pointer network, and then the next node to be visited is selected; Among them, the complete feasible path is carried out step by step through the autoregressive process, and then the current state and query vector are summarized at each step to select the next drone map shooting point, and finally a sequence output is formed to represent the drone's shooting path.

5. The UAV path planning method based on the traveling salesman problem according to claim 3 is characterized in that: The fast attention mechanism based on orthogonal random features uses orthogonal random features to approximate the kernel function in self-attention, as follows: The kernel function K is defined by the random feature map φ(u), and the form of the kernel function K is: Among them, φ is a mapping from d dimension to r dimension; for The rows of Q' and K' are and An efficient attention mechanism is obtained through mapping in the form of: Among them, D b =diag(Q′((K′) T 1 L )); diag is to convert an input vector into a diagonal matrix; 1 L is a vector of length L containing all 1s; Att approx Represents the approximate attention mechanism; brackets indicate the order of calculation; the space complexity of the fast attention mechanism based on orthogonal random features is O(Lr+Ld+rd), and the time complexity is O(Lrd); The φ processing functions f1,…,f are taken as follows l and ω1,…,ω through Gram-Schmidt orthogonalization m : in, 6. The UAV path planning method based on the traveling salesman problem according to claim 4 is characterized in that: The formula for the query vector is as follows: Among them, q t represents the query vector; represents the static embedding of the first node on the path; represents the embedding of the last node on the path, which changes as the path is updated; l σ represents the sum of all node embeddings on the current encoding path; l dy represents the embedding updated based on the mean embedding of the most recently visited nodes to adapt to dynamic changes during decoding; l mean Represents the average embedding of the entire graph, focusing on global information; The query vector is obtained through the context embedding mechanism. The context representation of the query vector not only guides the selection of the next node, but also allows the deep reinforcement learning model to adjust the embedding during the decoding process to capture dynamically changing contextual information and global information. At the same time, the autoregressive decoder calculates the probability distribution of access nodes through the pointer network to better capture the relationship between nodes, as follows: Get the query vector q through the dot multiplication operation t Interaction with node j; The interaction score between node i and node j is calculated by subtracting the cost cost(i,j) between the last node i and node j in the partial path. ij , which in turn encourages starting from an effective strategy, ensuring that the node closest to the current node is always selected as the next visit target; The interaction score ij Through nonlinear transformation, it is converted to C·tanh(score ij ); By applying the Softmax function to generate the output probability distribution P, the selection of the next visited node is made random and reasonable; Among them, the decoder adopts a dynamic selection mechanism based on coordinate distance and only selects the nearest neighbor nodes to participate in the attention calculation.

7. The UAV path planning method based on the traveling salesman problem according to claim 1 is characterized in that: This method also uses the Lion optimizer, which uses symbolic operations to calculate parameter updates and only tracks momentum, thereby significantly reducing memory overhead and achieving uniform update amplitudes in all dimensions. The formula is as follows: i t =θ t-1 -asign(m t ); Among them, m t represents the momentum vector; β1 represents the first momentum hyperparameter; The symbolic function representing the gradient of the loss function L with respect to the parameter θ; θ t represents the parameters after the tth iteration; α represents the learning rate.

8. The UAV path planning method based on the traveling salesman problem according to claim 1 is characterized in that: The symmetric regularized REINFORCE algorithm introduces a symmetry-based baseline term in the process of training deep reinforcement learning models. Specifically, multiple solutions are sampled from the drone map surveying instance after data enhancement, and the average return of the solution is used as a shared baseline. During the retraining process of the symmetric regularized REINFORCE algorithm, each solution sampled by the solver is pushed to exceed the shared baseline, thereby continuously optimizing the strategy and making the performance of all solutions tend to be consistent. Among them, the symmetric approximation achieved by the symmetric regularized REINFORCE algorithm is as follows: The solution symmetry is approximated by using the average reward of K sampled solutions, giving a regularized loss L ss , the formula is as follows: Where log F θ Yes F θ The log-likelihood of By sampling the rotation matrix to construct a shared baseline with symmetry, we get the regularization loss L ps , the formula is as follows; in, Q is the distribution of random orthogonal matrices; Use the regularization loss term L inv Force Solver F θ With an invariant representation, the formula is as follows: IT inv =-Subtract(g(h(x)),g(h(Q(x)))); where h(x) and h(Q(x)) denote the hidden representations of the original question and its rotation, respectively; Scos(a,b) is the cosine similarity between a and b; g is the projection head parameterized by a multilayer perceptron; The solver F(θ) is learned by minimizing the total loss function, which is defined as follows: L total =L ps +βL ss +αL inv ; Among them, L ss represents the solution symmetry regularization loss; L ps represents the symmetry regularization loss of the problem; L inv represents the regularization loss that induces invariant representation; α and β∈[0,1] are weight coefficients.

9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores a computer program; The at least one processor executes the computer program stored in the memory, so that the at least one processor performs the drone path planning method based on the traveling salesman problem as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which can be executed by a processor to implement the UAV path planning method based on the traveling salesman problem as described in any one of claims 1 to 8.

Images