A method for UAV flight path planning based on sequence generative adversarial networks

Abstract

This invention discloses a method for UAV flight path planning based on sequence generative adversarial networks, comprising: Step 1: constructing a 3D mesh map; Step 2: generating several paths on the 3D mesh map using a path planning algorithm as a dataset and cleaning the data; Step 3: constructing the UAV's state space, action space, environmental wind field effect model, and UAV energy consumption model; Step 4: using the paths in the cleaned dataset from Step 2 as positive samples, and expanding the dataset to construct negative samples; converting the positive samples into state-action sequences and inputting them into the generator, and inputting the positive and negative sample path sequences into the discriminator; Step 5: performing adversarial training based on the generator and discriminator. Thus, the method enables the UAV to move towards the target and its corresponding action parameters when facing complex obstacle environments. Furthermore, this invention, combined with the guidance of the generator's prior knowledge, improves the performance of traditional algorithms in long path planning.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) route planning technology, and in particular to a method for UAV flight path planning based on sequence generative adversarial networks. Background Technology

[0002] With the rapid development of drone technology, drones have been widely used in various fields such as logistics delivery and power line inspection, significantly increasing the requirements for the accuracy, adaptability, and safety of drone flight path planning. The core of drone path planning algorithms is that, after determining the starting point and the target point, the drone can autonomously plan a high-quality path that bypasses obstacles in the environment, has a short distance, low energy consumption, and meets dynamic constraints, ensuring that drone missions are completed efficiently and safely.

[0003] Currently, existing UAV route planning methods typically involve direct application of algorithms such as sampling-based planning, graph-based search, and intelligent biomimetic algorithms, or further optimization based on different algorithms such as neural networks. These existing technologies do not address the path planning specific to the operational characteristics of UAVs, resulting in planned paths that fail to meet actual flight requirements. Furthermore, they exhibit low planning efficiency in complex urban environments and poor performance in long-path planning. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a UAV flight path planning method based on sequence generative adversarial networks, which can provide reliable UAV route planning and can perform reliable planning even in complex flight conditions and urban environments.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for UAV flight path planning based on sequence generative adversarial networks includes: Step 1: Converting a real OSM map into a 3D city model and constructing a no-fly zone using cylinders to build a 3D mesh map; wherein, the 3D mesh map stores building data; Step 2: Generating several paths on the 3D mesh map using a path planning algorithm as a dataset and performing data cleaning, including: checking point by point whether the path reaches the target and whether it collides with obstacles in the map, and filtering out collision-free paths that reach the target; Step 3: Constructing the UAV's state space, action space, environmental wind field effect model, and UAV energy consumption model; Step 4: Converting the paths in the dataset into corresponding state-action sequences, using the cleaned dataset from Step 2 as positive samples, and expanding the dataset to construct negative samples; inputting the positive sample state-action sequences into... The generator inputs positive and negative sample path sequences into the discriminator; positive samples are input into the generator, and positive and negative samples are input into the discriminator; Step 5: Adversarial training is performed based on the generator and discriminator; wherein, the generator obtains the current local map, state, target position, and random noise, and predicts the next action and action parameters of the UAV. After Monte Carlo search, it generates a complete path from the starting point to the end point, and the discriminator evaluates the quality score of the path. The generated path sequence and path information are stored in the experience replay pool. The experience replay pool stores the generated paths and samples according to the path quality priority to improve training efficiency. Each sample in the experience replay pool includes a path point sequence, state sequence, action sequence, map data, starting point, target point, and path quality score; when the number of paths in the experience replay pool reaches a certain value, the adversarial training between the generator and the discriminator begins.

[0007] The beneficial effects of this invention are as follows:

[0008] This application utilizes the decisions from traditional algorithms as the generator's dataset, enabling the generator to learn to choose actions and corresponding action parameters to avoid obstacles and move towards the target when facing complex obstacle environments. This is then used as prior knowledge for Monte Carlo search, reducing the random search range of Monte Carlo search and increasing its efficiency. In particular, traditional Monte Carlo algorithms perform poorly in long path planning, but this invention improves the performance of traditional algorithms in long path planning by combining the guidance of the generator's prior knowledge.

[0009] In this application, the map modeling is based on a real city map and a no-fly zone is added. At the same time, considering the impact of buildings on wind fields in the urban environment, wind fields that conform to the actual situation are simulated between buildings. During the flight of the UAV, the impact of wind fields on the trajectory and energy consumption of the UAV is dynamically considered, the trajectory deviation caused by wind fields is corrected in a timely manner, the energy consumption distribution is optimized reasonably, the feasibility of the UAV path is improved, and the stable and efficient flight of the UAV is ensured.

[0010] In this application, the path generation process combines a generator and Monte Carlo search. The generator provides prior knowledge, while the Monte Carlo search further expands upon it, making the flight path planning traceable and the decision-making logic of path generation more transparent. For path quality assessment, the discriminator employs a combination of a rule-based discriminator and a neural network discriminator. Clear and quantifiable rules constitute the rule-based discriminator, which performs a preliminary assessment of the path. The neural network discriminator obtains more implicit features of the path. The combination of these two methods not only improves the accuracy of path quality assessment but also provides clear rule-based evidence and auxiliary explanations for the assessment results, thereby enhancing the interpretability of path generation and discriminator judgments. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a UAV flight path planning method based on sequence generative adversarial networks according to some embodiments of this application;

[0012] Figure 2 This is a schematic diagram illustrating the process of generating positive samples, negative samples, and subsequent processing according to some embodiments of this application;

[0013] Figure 3 This is a schematic diagram of a generator network structure according to some embodiments of this application;

[0014] Figure 4 This is a schematic diagram of a discriminator network structure according to some embodiments of this application;

[0015] Figure 5 This is a schematic diagram illustrating the process of adversarial training of a generator and a discriminator based on Monte Carlo search according to some embodiments of this application;

[0016] Figure 6 This is a schematic diagram of a reward mechanism based on Monte Carlo search according to some embodiments of this application. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figures 1-6 This invention provides a method for UAV flight path planning based on sequence generative adversarial networks, referencing... Figure 1 Understanding includes the following steps:

[0019] Step 1: Use real-world OSM (Open Street Map) maps to convert them into 3D city models and use cylinders to construct no-fly zones. Combine these maps into various types of 3D mesh maps and save building data, including building coordinates, radius, size, and height information. Divide the map data into difficulty levels 1-10 based on the number of obstacles, with 30 maps for each difficulty level.

[0020] Step 2: Generate 1000 paths on the 3D grid map using a path planning algorithm and perform data cleaning. The path planning algorithm can be any traditional path planning algorithm such as RRT algorithm (incremental path planning algorithm based on random sampling) and A* algorithm (A* algorithm, a heuristic algorithm). The partitioning process specifically includes: checking whether the path reaches the target point by point and whether it collides with obstacles in the map, and filtering out paths that do not collide and reach the target.

[0021] Step 3: Construct the UAV's state space, motion space, environmental wind field effect model, and UAV energy consumption model; specifically, Step 3 includes the following steps:

[0022] Step 31: Constructing the UAV's state space includes: The UAV's spatial state contains the UAV's position coordinates in three-dimensional space. and the velocity components of the drone in the three coordinate axes The spatial state of the UAV is defined as a 6-dimensional vector: .

[0023] Step 32: Constructing the drone's motion space includes: The drone contains seven basic discrete motions: And three continuous control parameters: ,in Represents the speed scaling factor, Represents yaw angle, Represents the pitch angle.

[0024] Step 33: Constructing an environmental wind field effect model includes: storing wind field data in a three-dimensional mesh, with each mesh point containing a three-dimensional wind vector; wherein, the wind field is generated based on obstacles in the scene, and the wind field of the surrounding area is calculated for each obstacle, including both wind tunnel effect and turbulence effect.

[0025] Specifically, a wind tunnel effect influence zone is established on both sides of the building. This zone extends from outside the building's width to within 1.5 times its maximum size, with the wind direction parallel to the obstacle surface, forming an acceleration channel. The wind speed decreases with increasing distance, reaching a maximum empirical acceleration coefficient of 5.0 m / s². The wind tunnel effect is based on the Bernoulli equation, which is used to pre-calculate the wind field distribution on the constructed map. Bernoulli's principle describes the relationship between fluid velocity and pressure. In the flow scenario around the building, when airflow encounters the building, it accelerates from both sides, forming a high-speed airflow channel. Bernoulli's equation is defined as: ,in Let ρ be the air density, v be the wind speed, p be the pressure, g be the gravitational acceleration, and h be the height. The turbulence influence zone is located between the edge of the obstacle and 80% of the influence radius. Random disturbances will occur in the turbulence influence zone. The instability of airflow is simulated by a Gaussian distribution. The magnitude of the disturbance decreases with distance, with a maximum fluctuation amplitude of 3.0 m / s.

[0026] For wind field models, it is necessary to consider not only the wind field of obstacles in advance, but also the wind field interaction based on real-time position and obstacle relationship. In UAV motion simulation, the influence of wind field is calculated in real time according to the current position and flight direction.

[0027] The wind tunnel effect occurs when the UAV is in a narrow passage, such as when there are obstacles on both sides, and it accelerates the UAV in the flight direction. The turbulence effect adds random disturbances when the UAV is in the turbulence influence zone and close to the leeward side of the obstacle, that is, when the obstacle is behind the flight direction. In addition, as the altitude increases, a constant horizontal wind field is applied along the east-northeast direction to simulate the high-altitude wind field. The above interactions directly affect the UAV's velocity vector and affect its position update.

[0028] Therefore, by modeling the environmental wind field effect through data, a grid environment containing wind field information can be obtained. This allows for real-time adjustment of the drone's speed and position based on the wind field effect, taking energy consumption into account for tailwinds and headwinds, enabling the drone to actively avoid strong headwind areas and improving the actual feasibility of the path.

[0029] Compared to existing sampling-based planning algorithms, graph-based search algorithms, and intelligent biomimetic algorithms such as ant colony optimization, A* algorithm, RRT algorithm, and PSO (Particle Swarm Optimization), which only process the dataset directly through the algorithm, this approach lacks modeling of wind field effects and does not adjust the UAV's speed and position in real time to adapt to wind field changes. Furthermore, it does not fully consider the additional consumption caused by wind field effects in energy consumption calculations, which seriously affects the actual feasibility of the path. The embodiments of this application consider the significant impact of dynamic environmental factors such as wind field effects on the UAV path, which may lead to UAV deviation or a surge in energy consumption. It overcomes the limitations of existing path planning algorithms in scenarios with dynamic wind fields and complex 3D obstacles. In 3D environments, due to the dense distribution of obstacles and complex spatial structures, the algorithm has low search efficiency, is prone to getting trapped in local optima, has a single optimization objective, and does not meet dynamic constraints. Ultimately, the planned path can meet the actual flight requirements.

[0030] S34. Constructing an energy consumption model for the UAV includes: obtaining the basic displacement energy consumption, which consists of two parts: horizontal displacement and vertical displacement. Specifically, since the UAV needs to overcome gravity to do work when climbing or descending vertically, it typically consumes more energy than during horizontal flight. Therefore, a higher energy consumption coefficient is set for vertical displacement. The horizontal displacement energy consumption is obtained by multiplying the horizontal distance between the starting and ending points by the horizontal energy consumption coefficient.

[0031] Vertical displacement energy consumption is obtained by multiplying the absolute value of the height difference between the starting and ending points by the vertical energy consumption coefficient, as defined by the formula: ,in, and These are vertical displacement and horizontal displacement, respectively. and These are the energy consumption coefficients for horizontal and vertical displacement, respectively. When a drone accelerates or decelerates, additional energy is required to change its state of motion, which involves the energy consumption of acceleration, and acceleration-related energy consumption. The magnitude of acceleration and the flight distance are jointly determined, and the formula is defined as follows: ,in, This is the acceleration energy consumption coefficient.

[0032] Furthermore, when wind is present in the environment, it significantly impacts the drone's energy consumption, especially headwinds, which increase energy consumption. Pre-calculated wind field data, including wind speed vectors at the start and end points, is obtained from the occupied grid. The average of these vectors is used as an approximation of the wind speed for the entire flight segment. The relative wind speed is obtained by subtracting the drone's current speed from the average wind speed. The projection component of the relative wind speed onto the flight direction is calculated. A positive component indicates a headwind, while a negative component indicates a tailwind. The relative wind speed is obtained by multiplying the square of the headwind component by the flight distance and then by the drag coefficient, as defined in the following formula: ;

[0033] in, This represents the headwind component of the relative wind speed in the flight direction, i.e., the projected component opposite to the flight direction. This is the wind resistance energy consumption coefficient.

[0034] Understandably, by modeling energy consumption to calculate the total energy consumption of the drone during its flight from the starting point to the destination, and by combining the basic energy consumption related to displacement, the energy consumption affected by the wind field, and the energy consumption related to acceleration, the results are made more realistic.

[0035] Step 4: Use the paths in the dataset from Step 2 as positive samples, and expand the dataset to construct negative samples; convert the positive samples into state-action sequences and input them into the generator, and input the positive and negative sample path sequences into the discriminator. Specifically, combined with... Figure 2 Understanding step 4, we have: S41, data cleaning based on step 2, that is, checking the path data in the dataset point by point to determine whether the path reaches the target and whether it collides with obstacles in the map, thereby filtering out paths that do not collide and reach the target. The cleaned dataset is then used as positive samples, and more negative samples are generated by adding collision points, extending path lengths, and adding random noise to the path sequences in the dataset, thus generating a balanced dataset of positive and negative samples.

[0036] S42. Process the cleaned dataset (positive samples) by sampling the path point sequence at equal intervals. Then, calculate the velocity using the displacement of adjacent path points and dynamically adjusted time intervals. When the distance between two points is too far, adjust the time interval to make the velocity reasonable. Calculate the 6-dimensional vector of the spatial state at each moment in sequence. Then, determine the action sequence by matching the velocity direction with the dot product of preset actions, thus transforming the path data into the corresponding action sequence.

[0037]

[0038] in, Let A be the action at time t, A be the action space, and e be the action direction component, obtained based on the action space. Let t be the position at time t.

[0039] S43. Input the processed state-action sequence into the generator. The generator receives the local map, current state, target point position and noise vector from the current data, and outputs the action type probability prediction and action parameter prediction.

[0040] In some examples, the generator network structure is as follows: Figure 3As shown, a bidirectional LSTM is used to capture long temporal dependencies in the path, and a three-dimensional spatial attention mechanism and multimodal feature encoding are integrated, covering noise encoding and four feature encoding branches: state encoding, target encoding, geometric relationship encoding and three-dimensional map encoding. Among them, the noise encoder encodes 32-dimensional random noise into 128-dimensional features through two fully connected layers, introducing randomness into the generator and enhancing the diversity of output.

[0041] In detail, the state encoder employs a three-layer fully connected network to encode the 6-dimensional state vector (including a 3-dimensional position vector and a 3-dimensional velocity vector) into a 128-dimensional feature vector. Leaky ReLU activation and batch normalization are used to enhance the non-linear representation of the features. The target encoder also employs a three-layer fully connected network to encode the 3-dimensional target position (coordinates) obtained in step 31 into a 128-dimensional feature vector, ensuring that the target features and state features are in the same feature space. A multi-head attention mechanism is used to enhance the target feature representation (i.e., target attention in the figure), enabling the generator to focus on the state information most relevant to the target. The geometric relationship encoder explicitly models multi-dimensional spatial geometric features, encompassing kinematic features such as position difference, distance, and orientation angle, capturing the spatial geometric relationship between the state and the target. The input 9-dimensional geometric features include a 3-dimensional position difference vector, a 1-dimensional horizontal distance vector, a 3-dimensional orientation vector, and a 2-dimensional angle vector, which are encoded into a 128-dimensional feature vector through the three-layer fully connected network. The map encoder uses 3D convolutions to compute attention in parallel across the width, height, and depth dimensions. To obtain multi-scale spatial correlations in complex environments and reduce computational complexity, a feature enhancement module is added to decompose the 3D convolutions into 2D convolutions in three orthogonal planes: XY, YZ, and XZ. The first layer is a 3D convolution containing the 3D spatial attention module, the second layer is an XY plane convolution and a Z-axis convolution, the third layer is a 3D convolution containing the feature enhancement module, followed by a fully connected layer that maps the flattened features to 128-dimensional feature vectors.

[0042] The feature fusion layer receives the above five features and passes them through a bidirectional LSTM layer to obtain more features of the path sequence context. Then, it passes through two fully connected neural networks and outputs seven discrete action probabilities after using Softmax activation. At the same time, it passes through two fully connected neural networks and uses the Tanh activation function to output continuous action parameters.

[0043] As mentioned above, the path is transformed into a sequence of state actions and fed into the generator. Supervised learning is used during pre-training to teach the generator how to choose actions when facing obstacles. There are a total of seven actions, and a probability is predicted for each action. The action type loss function is defined as:

[0044]

[0045] Where N is the batch size and C is the number of action categories. Let c be the true label of the i-th sample in the c-th category. It is the predicted probability of the c-th class for the i-th sample.

[0046] In the method of this application embodiment, it is necessary not only to learn to select actions, but also to learn how to adjust action parameters when facing current obstacles. The action parameter loss function is defined as:

[0047]

[0048] Where N is the batch size. It is the vector of action parameters predicted by the generator for the current sample i. It is the target parameter vector of the current sample i.

[0049] Therefore, the pre-trained generator can initially learn to choose to avoid obstacles and move towards the target when facing the current obstacle, as well as the action parameters, thus avoiding the generation of useless paths at the beginning of adversarial training.

[0050] S44. Input positive and negative samples into the discriminator. The discriminator integrates explicit evaluation based on physical rules with implicit feature extraction from deep learning networks, which improves the interpretability of the discriminator's evaluation. The discriminator receives path sequences, target locations, and 3D maps, and outputs a total score for path quality as well as detailed scores in multiple dimensions, including six dimensions: collision, compliance with dynamic constraints, target achievement, safety, smoothness, and energy consumption. The discriminator integrates explicit evaluation based on physical rules with implicit feature extraction from deep learning networks, which improves the interpretability of the discriminator's evaluation.

[0051] It is worth noting that S43 and S44 can be performed simultaneously.

[0052] Understandably, through the aforementioned steps, the method, based on explicit evaluation of physical rules, assesses six dimensions from the perspectives of kinematic constraints and environmental physical characteristics. Collision detection checks whether a path point is within an obstacle grid; if so, it's considered a collision. It also checks for obstacles within a safe distance around the path point; if present, the score is lowered. Goal achievement checks the distance between the endpoint and the target point; a distance less than 1 meter indicates task completion, with the score decreasing as the distance increases, and a negative score occurring if the distance exceeds a certain threshold. Dynamic constraints include checking whether the speed exceeds the maximum speed, whether the height is within the allowable range, and calculating the curvature of the path point through the angle change between two vectors formed by three consecutive points. Smoothness is assessed by checking changes in continuous curvature and the direction of speed changes. Safety checks the distance between each point on the path and the nearest obstacle. This explicit rule-based evaluation provides a transparent decision-making basis for judging path quality.

[0053] Furthermore, the discriminator network receives the path sequence, target location, and 3D map, and outputs a 7-dimensional vector, where the first dimension is the total score and the last 6 dimensions are the specific scores. The scores are constrained to the interval [0,1] by Sigmoid activation.

[0054] In some embodiments, the discriminator network is a multimodal fusion deep learning model, with the following architecture: Figure 4 As shown, the multimodal fusion includes three feature encoding branches: path sequence encoding, environment map encoding (i.e., the map in the figure), and target encoding (i.e., the target location in the figure). Temporal features are obtained through a bidirectional LSTM. The path encoder encodes each state vector in the path sequence into a high-dimensional feature representation through a three-layer fully connected neural network structure, enhancing the model's ability to represent local states of the path. Then, the sequence features output by the path encoder are input into a bidirectional long short-term memory network to integrate the contextual information of each state in the path. The output of the last time step in the LSTM is selected as the overall representation of the sequence. This allows simultaneous capture of the temporal information of the path sequence from the starting point to the current step and from the ending point to the current step. The map encoder effectively captures local correlation features in three-dimensional space. After passing through a 3D convolutional layer and batch normalization, the map is input into a max pooling layer, followed by another 3D convolutional layer. The convolutional layer is fed into an adaptive max-pooling layer and then flattened into a one-dimensional vector. After passing through the LeakyReLU activation function, the output is obtained. The target encoder maps the three-dimensional target coordinates to a feature space of the same dimension as the path features through two linear transformations, which facilitates the subsequent fusion of multimodal features. Then, the path encoding features, map encoding features, and target encoding features are concatenated and fused, and then passed through a three-layer fully connected network to output a six-dimensional score value. The above network can capture the implicit features and complex patterns of the path.

[0055] The paths in the dataset are processed by a rule-based discriminator to generate detailed scores encompassing the six dimensions mentioned above, and these scores are used as supervision signals. The discriminator network is pre-trained using a multi-task loss function with two supervision objectives. When using the six-dimensional detailed scores as supervision signals, the loss function calculates the mean squared error between the detailed scores output by the discriminator and the rule-based discriminator scores. When using binary classification labels as supervision signals, the loss function calculates the mean squared error between the total score output across the six dimensions and the true label. The specific definition of the loss function is as follows:

[0056]

[0057]

[0058]

[0059] Where, represents the batch sample size. Let be the score for the j-th dimension of the current sample i output by the discriminator. To provide the rule discriminator with a score for the j-th dimension of the current sample i, The total score of the current sample i is output by the discriminator. For binary classification labels, This represents the task weighting coefficient. It is a loss function for six-dimensional scoring. The loss function is the total path score. The trained discriminator network can initially determine the path quality and score the path.

[0060] Step 5: Perform adversarial training based on the generator and discriminator.

[0061] Specifically, such as Figure 5 As shown, during training, maps are progressively loaded, gradually increasing from difficulty level 1 to difficulty level 10, with 30 maps at each difficulty level. A path is generated for each map. The generator receives the current local map, state, target location, and random noise from the starting point and predicts the drone's next action and action parameters. After Monte Carlo search, a complete path from the starting point to the destination is generated. The discriminator evaluates the path's quality score, and the generated path sequence and path information are stored in the experience replay pool. The experience replay pool stores the generated paths and samples them according to path quality priority to improve training efficiency. Each sample in the experience replay pool includes a path point sequence, state sequence, action sequence, map data, starting point, target point, and path quality score. When the number of paths in the experience replay pool reaches a certain value, training of the generator and discriminator begins. The ratio of training steps for the generator and discriminator is dynamically adjusted based on changes in the generator's average reward and the discriminator's loss.

[0062] In detail, the discriminator is first trained, the generator is fixed, the paths in the dataset are obtained and samples are sampled from the experience replay pool. Then, the paths are divided into three groups: high, medium and low, based on their quality. The labels of high-quality samples are close to 0.9, while the labels of low-quality samples are close to 0, which encourages the discriminator to evaluate the quality of the generated samples more carefully.

[0063] The discriminator's adversarial training is based on quality perception. For example, the labels of generated samples are dynamically adjusted according to the quality of the generation path. High-quality samples have labels close to those of real samples, while low-quality samples have labels close to 0. The discriminator's complete loss function consists of four parts: focus loss for real samples, focus loss for generated samples, contrastive loss, and L2 regularization term. Among them, focus loss reduces the weight of easily classified samples, making the model pay more attention to difficult-to-classify samples. Contrastive loss forces a certain difference in scores between high-quality and low-quality samples, thereby enhancing the discriminator's ability to distinguish between samples of different quality.

[0064] The loss function of the discriminator is defined as follows:

[0065]

[0066] in It is the focus loss of the real sample. It is the focus loss of the generated samples. These are the weight parameters for the contrastive loss. These are the coefficients of L2 regularization. It is a comparison of losses. It is the L2 regularization loss.

[0067] The formula for the focus loss of real samples is as follows:

[0068] in The binary cross-entropy loss BCEWithLogitsLoss for the real samples. Increase the weight of difficult samples as a focal loss parameter.

[0069] The formula for the focus loss of the generated samples is as follows: .

[0070] in is the binary cross-entropy loss BCEWithLogitsLoss for generating the path, and is the focus loss parameter.

[0071] BCEWithLogitsLoss is a combination of binary cross-entropy loss and the sigmoid function, and its formula is as follows:

[0072]

[0073] Where is the discriminator's output before passing through the Sigmoid function, and is the target label. It is the Sigmoid function, where N is the current batch size and i is the current sample.

[0074] Contrast loss requires that the score of high-quality samples be at least 0.6 higher than the score of low-quality samples, as shown in the following formula:

[0075]

[0076] Where H is the set of high-quality samples and L is the set of low-quality samples. It is the discriminator output of high-quality samples. This is the discriminator output for low-quality samples, where h represents the current high-quality samples and l represents the current low-quality samples.

[0077] The L2 regularization loss formula is as follows: ,in It is the k-th parameter matrix of the discriminator. It is the regularization strength.

[0078] The generator predicts the drone's next action and action parameters. After the Monte Carlo search in step 6, it generates a complete path from the starting point to the end point. The generator's goal is to generate actions and parameters that avoid obstacles and move toward the target when facing the current obstacle. When training the generator, the discriminator's parameters are fixed, and the generator's parameters are updated through the policy gradient method, so that the generator can generate more appropriate actions and action parameters.

[0079] The generator samples a batch of generated paths from the experience replay pool and uses a discriminator to evaluate the quality of the paths as a reward signal. This quality is then used to calculate the policy gradient to update the generator. The generator's updated loss function consists of three parts: the policy gradient of the action probabilities, the policy gradient loss of the action parameters, and an entropy regularization term. The policy gradient loss of the action probabilities encourages the generator to choose actions that lead to high rewards. The policy gradient loss of the action parameters aims to make the predicted parameters closer to the actual optimal parameters. The entropy regularization term encourages exploration and prevents the policy from prematurely converging to a local optimum. The specific formulas are defined as follows: ,in The weights for the loss of action parameters.

[0080] Policy gradient of action probability ,in Is the generator in state? Select Action The probability, It is the advantage function of the current sample i, and N is the batch size.

[0081] Policy gradient loss of action parameters ,in The action parameters of the current sample i predicted by the time generator. is the actual action parameter used by the current sample i, and N is the batch size.

[0082] Entropy regularization term ,in It is the entropy regularization coefficient. For the action of the current sample i, This represents the current state of sample i.

[0083] In addition to the loss function, policy updates also use an advantage function. The advantage function is the difference between the reward gained from the current action and the baseline reward, and is defined by the formula: ,in, It is the path quality score given by the discriminator. It is the moving average of the rewards over a recent period.

[0084] The generator samples a batch of paths from the experience replay pool, uses a discriminator to evaluate the quality of each path, calculates the advantage value of each path, and recalculates the log probability of the action and the log probability of the parameter by replaying the forward process of the generator for each state-action pair. The generator is trained using the policy gradient method to maximize the reward signal given by the discriminator and minimize the path length and energy consumption objectives.

[0085] Continue to combine Figure 5 and Figure 6 Understanding is that the node attributes of the Monte Carlo search in step 5 include environmental state information such as obstacle information, position, speed, target, path length, decision history of the action, node relationships with parent and child node mappings and access counts, and cumulative reward statistics. It also introduces action prior probability and parameterized action space, and provides action and action parameter sampling guidance through the generator.

[0086] After each node is expanded, the node validity will be verified. If the current node or the next node collides with an obstacle, exceeds the longest path limit, violates dynamics, the current position is less than the safe distance from the obstacle, or exceeds the map boundary, it will be judged as invalid and no further node expansion will be performed.

[0087] Based on the UAV path planning model, the generator progressively predicts the UAV's next action and velocity components at each location, continuously updating the UAV's position. Monte Carlo tree search utilizes the generator's output to guide the iterative search process. Figure 6 As shown, a search tree is constructed through node initialization and four iterative stages: selection, expansion, simulation, and backtracking, ultimately selecting the optimal obstacle avoidance path from the starting point to the end point.

[0088] Specifically, MCTS nodes store the environment state (including obstacle information, location, speed, target, path length), node relationships (parent node, child node mapping), decision history (execution action sequence), and statistical information (access count, cumulative reward), as well as the action space (prior probability distribution, parameterized actions). The selection process involves recursively selecting child nodes starting from the root node until a partially expanded, scalable node is reached. An improved UCB formula is used to balance exploration and exploitation. The improved UCB formula is as follows:

[0089]

[0090] in, It is a child node The cumulative reward value, It represents the number of visits to child nodes. This is the number of times the current node has been accessed. It is the generator that gives the state. Select action The prior probability of is used to guide the search in directions that the generator deems promising, and c is the exploration weight. Represents child nodes The average reward value is considered, and this option tends to select nodes with high historical average rewards. Used for balance exploration The number of visits to the parent node increases, giving child nodes with fewer visits a greater chance to be explored. This makes child nodes with fewer visits have higher exploration value, and the exploration weight c controls the intensity of the exploration;

[0091] In the child node selection phase, the prior knowledge of the generator is introduced into the exploration items. The generator provides the prior probability of the action, thereby reducing random exploration and accelerating convergence. When the prior probability of an action is high, it may be explored even if its current average reward is not high. When the search gets stuck in a local optimum or cannot find an effective action, multiple recovery strategies such as random actions, goal-oriented actions, and backtracking to checkpoints are used to ensure the robustness of the algorithm.

[0092] The expansion process includes: when a node that is not fully expanded is selected, adding one or more child nodes to it; using a generator to generate the action probabilities and parameters of the current node; to balance efficiency and diversity, only the top K actions with the highest probabilities are expanded, and the value of K is dynamically adjusted according to the current state, for example, K is smaller when close to the target to improve efficiency, and larger when far away or close to obstacles to increase exploration; at the same time, the actions are sorted according to the target direction and safe distance to ensure that the expanded node is both safe and efficient; before or during expansion, the node validity must be verified (including collision detection, path length limit, dynamic constraints, safe distance check, and map boundary check). If a node is determined to be invalid, the expansion of that node is stopped (node ​​failure).

[0093] The simulation includes: starting from a valid expanded node, simulating the execution of a series of actions until the termination state, and calculating the simulation reward value. Node termination states include reaching the target, collision, dynamic violation, and exceeding the boundary. The simulation employs... Strategy, with The probability of randomly selecting an action, in order to The probability selection generator recommends the action with the highest probability. During the simulation, dynamic factors such as wind field effects are considered, and path length and energy consumption are calculated. The number of simulation iterations is also dynamically adjusted based on the distance from the current position to the nearest obstacle and the current position to the target.

[0094] The backtracking process includes: updating the simulated reward values ​​to all nodes along the path, starting from the node where the simulation ended and backtracking upwards to the root node, updating the visit count and accumulated reward for each node, incrementally updating the node value, avoiding the storage of all historical data, and using the following update formula:

[0095]

[0096] in, It is the expected cumulative reward of node s. This is the reward value obtained in this simulation. It is the number of times node s is visited; and the number of visits increases; in addition, the backtracking phase also includes path value propagation and incremental update strategies.

[0097] After the above four stages, the optimal obstacle avoidance path is finally selected. In this process, the generator acts as a provider of prior knowledge. In the selection stage, it improves the UCB formula through prior probabilities and conducts balanced exploration and utilization, including introducing the generator's prior probability P and dynamically adjusting the exploration weight C. In the expansion stage, it provides action prior probabilities and parameters, including the generator providing action prior expansion Top-K, high-probability actions combined with target direction sorting, safety distance priority, and dynamic K value adjustment strategy. Then, in the simulation stage, it guides action selection, reduces ineffective exploration actions, and improves search efficiency.

[0098] In some embodiments, the reward settings in the Monte Carlo search process are also as follows: In Monte Carlo tree search, the reward function is used to evaluate the quality of the current state, thereby guiding the overall search towards a better direction. A multi-objective optimization design is adopted. Under the premise of ensuring safety constraints, the reward for guiding the goal constitutes the driving force for goal approach. At the same time, path quality penalties are introduced to reduce redundant paths, height change penalties optimize vertical efficiency, energy penalties reflect physical constraints, and smoothing rewards improve trajectory continuity. This reward structure effectively balances the optimization needs of goal achievement, path quality, and motion constraints in Monte Carlo search.

[0099] Setting a target reward: When the drone reaches the vicinity of the target point and is less than 1 unit away, a high positive reward is given. Simultaneously, to encourage the generation of shorter paths, a path length penalty is introduced into this positive reward. Therefore, the actual target reward is:

[0100]

[0101] Set a target-guided reward: the closer the drone gets to the target, the higher the reward.

[0102]

[0103] Set a path smoothness reward: To encourage the generation of smooth paths, calculate the cosine of the angle between the two direction vectors formed by three consecutive positions: the current state, the parent state, and the grandparent state. The reward is maximized when the path direction changes smoothly;

[0104]

[0105] Set collision and boundary constraint penalties: If the drone collides with an obstacle, a penalty of -500 will be imposed; if the drone exceeds the preset boundary, a penalty of -500 will also be imposed.

[0106] Set a path quality penalty: To avoid unnecessary vertical movement, calculate the absolute value of the height change between adjacent states and multiply it by a penalty coefficient.

[0107]

[0108] The height value at time t. The height value at the previous time step t

[0109] Calculate the energy consumption between adjacent states and multiply it by a penalty factor:

[0110]

[0111] When the target is not reached, the path length penalty coefficient is dynamically adjusted according to the distance to the target; the closer the distance, the larger the penalty coefficient.

[0112]

[0113] If the target is not reached and no collision / boundary crossing occurs, the total reward is:

[0114]

[0115] The reward function uses target distance reward to guide the drone closer to the target, while using penalty terms to optimize path length, energy consumption, smoothness and height stability to achieve multi-target balance, and using collision and boundary penalties to ensure path safety, thus meeting the needs of path planning.

[0116] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for UAV flight path planning based on sequence generative adversarial networks, characterized in that, include: Step 1: Convert a real-world open street map into a 3D city model, and use cylinders to construct a no-fly zone to build a 3D mesh map; the 3D mesh map stores building data. Step 2: Use a path planning algorithm to generate several paths on a 3D grid map as a dataset and perform data cleaning, including: checking point by point whether the path reaches the target and whether it collides with obstacles in the map, and selecting the path that does not collide with the target and reaches the target. Step 3: Construct the state space, motion space, environmental wind field effect model, and UAV energy consumption model of the UAV; Step 4: Use the paths in the cleaned dataset from Step 2 as positive samples, and expand the dataset to construct negative samples; convert the positive samples into state-action sequences and input them into the generator, and input the positive and negative sample path sequences into the discriminator; Step 5: Perform adversarial training based on the generator and discriminator; The generator obtains the current local map, state, target location, and random noise, and predicts the next action and action parameters of the UAV. After Monte Carlo search, it generates a complete path from the starting point to the ending point. The generation of the complete path from the starting point to the ending point after Monte Carlo search includes: The Monte Carlo search utilizes the generator's output to guide search iterations. Through node initialization and four iterative stages—selection, expansion, simulation, and backtracking—a search tree is constructed, ultimately selecting the optimal obstacle avoidance path from the starting point to the end point. Node storage includes environmental states such as obstacle information, position, speed, target, and path length; node relationships include parent-child node mappings; decision history includes the sequence of executed actions; statistical information includes the number of visits and cumulative rewards; and the action space includes prior probability distributions and parameterized actions. The selection process involves: starting from the root node, recursively selecting child nodes until a partially expanded scalable node is reached, balancing exploration and exploitation using the improved UCB formula for the upper bound of the confidence interval. in, It is a child node The cumulative reward value, It is a child node Number of visits, This is the number of times the current node has been accessed. It is the generator that gives the state. Select action The prior probability, c is the exploration weight; The expansion includes: when a node that is not fully expanded is selected, adding one or more child nodes to it; using a generator to generate the action probability and parameters of the current node; expanding the top K actions with the highest probabilities; simultaneously, sorting the actions based on the target direction and safe distance; after each node expansion, a node validity verification is performed. If the current node or the next node collides with an obstacle, exceeds the longest path limit, violates dynamics, the current position is less than the safe distance from the obstacle, or exceeds the map boundary, it will be judged as invalid and no further node expansion will be performed. The simulation includes: starting from a valid extended node, simulating the execution of a series of actions until the termination state, and calculating the simulation reward value. The termination states of the node include reaching the target, collision, dynamic violation, and exceeding the boundary. The backtracking process includes: updating the simulated reward value backtracked to all nodes along the path, starting from the node where the simulation ended, backtracking upwards to the root node, updating the visit count and accumulated reward of each node, incrementally updating the node value, avoiding the storage of all historical data, and using the following update formula: in, It is the expected cumulative reward of node s. This is the reward value obtained in this simulation. It represents the number of times node s has been visited; and the number of visits is increasing. After the discriminator evaluates the quality score of the path, the generated path sequence and path information are stored in the experience replay pool. The experience replay pool stores the generated paths and samples them according to the path quality priority to improve training efficiency. Each sample in the experience replay pool includes a path point sequence, state sequence, action sequence, map data, starting point, target point, and path quality score. When the number of paths in the experience replay pool reaches a certain value, the adversarial training between the generator and the discriminator begins.

2. The UAV flight path planning method based on sequence generative adversarial networks according to claim 1, characterized in that, Step 3 includes the following steps: Step 31: Constructing the UAV's state space includes: The UAV's spatial state contains the UAV's position coordinates in three-dimensional space. and the velocity components of the drone in the three coordinate axes The spatial state of the UAV is defined as a 6-dimensional vector: ; Step 32: Constructing the drone's motion space includes: The drone comprises seven basic discrete actions: And three continuous control parameters: ,in Represents the speed scaling factor, Represents yaw angle, Represents pitch angle; Step 33: Constructing an environmental wind field effect model includes: Wind field data is stored in a 3D grid, with each grid point containing a 3D wind vector. The wind field is generated based on obstacles in the scene. For each obstacle, the wind field around it is calculated, including both wind tunnel effect and turbulence effect. S34. Constructing the UAV energy consumption model includes: obtaining the basic displacement energy consumption, which includes two parts: horizontal displacement and vertical displacement. The horizontal displacement energy consumption is obtained by multiplying the horizontal distance between the starting point and the ending point by the horizontal energy consumption coefficient; Vertical displacement energy consumption is obtained by multiplying the absolute value of the height difference between the starting and ending points by the vertical energy consumption coefficient. The calculation formula is defined as follows: ,in, and These are vertical displacement and horizontal displacement, respectively. and These are the energy consumption coefficients for horizontal and vertical displacements, respectively.

3. The UAV flight path planning method based on sequence generative adversarial networks according to claim 2, characterized in that, When wind fields are present in the environment, incorporating wind field data into the drone energy consumption model includes: The pre-calculated wind field data, including the wind speed vectors at the start and end points, is obtained from the occupied grid. The average of these vectors is taken as an approximation of the wind speed for the entire flight segment. The relative wind speed is obtained by subtracting the current speed of the UAV from the average wind speed. Calculate the projected component of relative wind speed in the flight direction. If the component is positive, it indicates a headwind; if it is negative, it indicates a tailwind. It is obtained by multiplying the square of the headwind component by the flight distance, and then multiplying by the drag coefficient. The formula is defined as follows: ; in, This represents the headwind component of the relative wind speed in the flight direction, i.e., the projected component opposite to the flight direction. The wind resistance energy consumption coefficient, This refers to the flight distance.

4. The UAV flight path planning method based on sequence generative adversarial networks according to claim 2, characterized in that, Step 4 includes: S41. Using the cleaned dataset as positive samples, the path sequences in the dataset are extended by adding collision points, and negative samples are obtained by adding random noise after adding collision points, thus generating a dataset with balanced positive and negative samples. S42. Process the positive samples by sampling the path point sequence at equal intervals. Then, calculate the velocity using the displacement of adjacent path points and the dynamically adjusted time interval. When the distance between two points is too far, adjust the time interval to make the velocity reasonable. Calculate the 6-dimensional vector of the spatial state at each moment in sequence. Then, determine the action sequence by matching the velocity direction with the dot product of preset actions, thereby converting the path data into the corresponding action sequence. in, Let A be the action at time t, and let A be the action space. For the direction component of the motion, It is obtained over time, based on the action space. The position at time t; S43. Input the processed state-action sequence into the generator. The generator receives the local map, current state, target point position and noise vector in the current data, and outputs the action type probability prediction and action parameter prediction. S44. Input positive and negative samples into the discriminator. The discriminator receives the path sequence, target location, and 3D map, and outputs a total score for path quality and a multi-dimensional detailed score, including six dimensions: collision, dynamic constraint compliance, target achievement, safety, smoothness, and energy consumption.

5. The UAV flight path planning method based on sequence generative adversarial networks according to claim 4, characterized in that, The generator network uses a bidirectional long short-term memory network and integrates a 3D spatial attention mechanism with multimodal feature encoding, covering noise encoding and four feature encoding branches including: state encoding, target encoding, geometric relationship encoding, and 3D map encoding; wherein: The noise encoder encodes 32-dimensional random noise into 128-dimensional features through a two-layer fully connected network; The state encoder uses a three-layer fully connected network to encode the 6-dimensional state vector of the spatial state into a 128-dimensional feature vector; The target encoder uses a three-layer fully connected network to encode the three-dimensional vector corresponding to the target location coordinates into a 128-dimensional feature vector, and uses a multi-head attention mechanism to enhance the target feature representation; The geometric relationship encoder inputs 9-dimensional geometric features, which include a 3-dimensional position difference vector, a 1-dimensional horizontal distance vector, a 3-dimensional direction vector, and a 2-dimensional angle vector, and is encoded into 128-dimensional features through a three-layer fully connected network. The map encoder uses 3D convolution to compute attention in parallel across the width, height, and depth dimensions. It also adds a feature enhancement module to decompose the 3D convolution into 2D convolutions in three orthogonal planes: XY, YZ, and XZ. The first layer is a 3D convolution containing a 3D spatial attention module, the second layer is an XY plane convolution and a Z-axis convolution, the third layer is a 3D convolution and also contains a feature enhancement module, followed by a fully connected layer that maps the flattened features into 128-dimensional feature vectors. After the path is transformed into a state-action sequence, it is fed into the generator. Supervised learning is used during pre-training to teach the generator how to choose actions when facing the current obstacle. This includes seven actions, and a probability is predicted for each action. The action type loss function is defined as follows: Where N is the batch size and C is the number of action categories. Let c be the true label of the i-th sample in the c-th category. It is the predicted probability of the c-th class for the i-th sample; The action parameter loss function is defined as: Where N is the batch size. It is the vector of action parameters predicted by the generator for the current sample i. It is the target parameter vector of the current sample i.

6. The UAV flight path planning method based on sequence generative adversarial networks according to claim 5, characterized in that, The discriminator network receives path sequences, target locations, and 3D maps, and outputs a 7-dimensional vector. The first dimension of the vector is the overall score, and the last 6 dimensions are the specific scores. The discriminator network is a multimodal fusion deep learning model, which includes three feature encoding branches: path sequence encoding, environment map encoding, and target encoding, and obtains temporal features through a bidirectional long short-term memory network. The path encoder encodes each state vector in the path sequence into a high-dimensional feature representation through a three-layer fully connected neural network structure. The sequence features output by the path encoder are input into a bidirectional long short-term memory network to integrate the contextual information of each state in the path. The output of the last time step in the long short-term memory network is selected as the overall representation of the sequence. The target encoder maps the 3D target coordinates to a feature space of the same dimension as the path features through two linear transformations. Then, the path encoding features, map encoding features and target encoding features are concatenated and fused, and then output a six-dimensional score value through a three-layer fully connected network. The paths in the dataset are processed by a rule-based discriminator to generate detailed scores encompassing the six dimensions mentioned above, and these scores are used as supervision signals. The discriminator network is pre-trained using a multi-task loss function with two supervision objectives. When using the six-dimensional detailed scores as supervision signals, the loss function calculates the mean squared error between the detailed scores output by the discriminator and the rule-based discriminator scores. When using binary classification labels as supervision signals, the loss function calculates the mean squared error between the total score output across the six dimensions and the true label. The loss function L is specifically defined as follows: in, This refers to the batch sample size. Let be the score for the j-th dimension of the current sample i output by the discriminator. To provide the rule discriminator with a score for the j-th dimension of the current sample i, The total score of the current sample i is output by the discriminator. For binary classification labels, This represents the task weighting coefficient. It is a loss function for six-dimensional scoring. The loss function is the total path score. The trained discriminator network can initially determine the path quality and score the path.

7. The UAV flight path planning method based on sequence generative adversarial networks according to claim 6, characterized in that, In step 5, the adversarial training of the discriminator is based on quality perception. The complete loss function of the discriminator includes the focus loss of real samples, the focus loss of generated samples, the contrastive loss, and the L2 regularization term. The loss function of the discriminator is defined as follows: in, It is the focus loss of the real sample. It is the focus loss of the generated samples. These are the weight parameters for the contrastive loss. These are the coefficients of L2 regularization. It is a comparison of losses. It is the L2 regularization loss; The formula for the focus loss of real samples is as follows: in The binary cross-entropy loss is for real samples. Here, e is the focus loss parameter, and e is the natural constant. The formula for the focus loss of the generated samples is as follows: ， in The binary cross-entropy loss for generating the path; Contrast loss requires that the score of high-quality samples be at least 0.6 higher than the score of low-quality samples, as shown in the following formula: Where H is the set of high-quality samples and L is the set of low-quality samples. It is the discriminator output of high-quality samples. This is the discriminator output for low-quality samples, where h represents the current high-quality samples and l represents the current low-quality samples. The L2 regularization loss formula is as follows: ,in It is the k-th parameter matrix of the discriminator. It is the regularization strength.

8. The UAV flight path planning method based on sequence generative adversarial networks according to claim 7, characterized in that, Step 5 further includes: The generator samples a batch of generated paths from the experience replay pool and uses the discriminator to evaluate the quality of the paths as a reward signal. This reward signal is then used to calculate the policy gradient to update the generator. The loss function updated by the generator includes the policy gradient of the action probabilities, the policy gradient loss of the action parameters, and an entropy regularization term; the specific formula is defined as follows: ,in The weights for the loss of action parameters; The policy gradient loss for the action probability is: ,in Is the generator in state? Select Action The probability, It is the advantage function of the current sample i, and N is the batch size; The policy gradient loss for the action parameters is: ,in These are the action parameters predicted by the generator for the current sample i. These are the actual action parameters used by the current sample i, and N is the batch size; The entropy regularization term is: ,in It is the entropy regularization coefficient. For the action of the current sample i, This represents the current state of sample i. Among them, the dominant function The difference between the reward obtained from the current action and the baseline reward.

9. The UAV flight path planning method based on sequence generative adversarial networks according to claim 8, characterized in that, The node attributes of the Monte Carlo search include environmental state information such as obstacle information, location, speed, target, path length, decision history of the executed actions, node relationships with parent and child node mappings and access counts, and cumulative reward statistics. It also introduces action prior probability and parameterized action space, and provides action and action parameter sampling guidance through the generator. Furthermore, based on the drone path planning model, the generator progressively predicts the next action and velocity components of the drone at each location, continuously updating the drone's position.

10. The UAV flight path planning method based on sequence generative adversarial networks according to claim 9, characterized in that, This also includes the reward settings during the Monte Carlo search process: Setting a target reward: When the drone reaches the vicinity of the target point and is less than 1 unit away, a high positive reward is given. Simultaneously, to encourage the generation of shorter paths, a path length penalty is introduced into this positive reward. Therefore, the actual target reward is: in, Penalty for path length; Set a target-guided reward; the closer the drone gets to the target, the higher the reward. The distance between the drone and the target; A path smoothness reward is set to encourage the generation of smooth paths. The cosine of the angle between the two direction vectors formed by three consecutive positions (current, parent, and grandparent states) is calculated. The reward reaches its maximum when the path direction changes smoothly, and is represented as... : Set collision and boundary constraint penalties. If the drone collides with an obstacle, a penalty of -500 will be imposed. If the drone goes beyond the preset boundary, a penalty of -500 will also be imposed. To set a path quality penalty and avoid unnecessary vertical movement, calculate the absolute value of the height change between adjacent states and multiply it by the penalty coefficient. The height value at time t. The height value at the previous time step t; Calculate the energy consumption between adjacent states Multiply by the penalty coefficient: Energy consumption; When the target is not reached, the path length penalty coefficient is dynamically adjusted according to the distance to the target; the closer the distance, the larger the penalty coefficient. If the target is not reached and no collision / boundary crossing occurs, the total reward is: 。

Images