Unmanned aerial vehicle flight path planning method based on sequence generative adversarial network

Abstract

The invention discloses an unmanned aerial vehicle flight path planning method based on a sequence generative adversarial network. The method comprises the following steps: step 1, establishing a three-dimensional grid map; 2, a plurality of paths are generated on the three-dimensional grid map through a path planning algorithm to serve as a data set, and data cleaning is conducted; 3, constructing a state space, an action space, an environment wind field effect model and an unmanned aerial vehicle energy consumption model of the unmanned aerial vehicle; 4, taking a path in the cleaned data set in the step 2 as a positive sample, and expanding the data set to construct a negative sample; positive samples are converted into state-action sequences to be input into a generator, and positive and negative sample path sequences are input into a discriminator; and 5, performing confrontation training based on the generator and the discriminator. Therefore, the motion of advancing towards the target and corresponding motion parameters can be achieved in the face of a complex obstacle environment, and the performance of a traditional algorithm in long path planning is improved in combination with guidance of priori knowledge of a generator.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) route planning technology, and in particular to a method for UAV flight path planning based on sequence generative adversarial networks. Background Technology

[0002] With the rapid development of drone technology, drones have been widely used in various fields such as logistics delivery and power line inspection, significantly increasing the requirements for the accuracy, adaptability, and safety of drone flight path planning. The core of drone path planning algorithms is that, after determining the starting point and the target point, the drone can autonomously plan a high-quality path that bypasses obstacles in the environment, has a short distance, low energy consumption, and meets dynamic constraints, ensuring that drone missions are completed efficiently and safely.

[0003] Currently, existing UAV route planning methods typically involve direct application of algorithms such as sampling-based planning, graph-based search, and intelligent biomimetic algorithms, or further optimization based on different algorithms such as neural networks. These existing technologies do not address the path planning specific to the operational characteristics of UAVs, resulting in planned paths that fail to meet actual flight requirements. Furthermore, they exhibit low planning efficiency in complex urban environments and poor performance in long-path planning. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a UAV flight path planning method based on sequence generative adversarial networks, which can provide reliable UAV route planning and can perform reliable planning even in complex flight conditions and urban environments.

[0005] The objective of this invention is achieved through the following technical solution: A method for UAV flight path planning based on sequence generative adversarial networks includes: Step 1: Converting a real OSM map into a 3D city model and constructing a no-fly zone using cylinders to build a 3D mesh map; wherein, the 3D mesh map stores building data; Step 2: Generating several paths on the 3D mesh map using a path planning algorithm as a dataset and performing data cleaning, including: checking point by point whether the path reaches the target and whether it collides with obstacles in the map, and filtering out collision-free paths that reach the target; Step 3: Constructing the UAV's state space, action space, environmental wind field effect model, and UAV energy consumption model; Step 4: Converting the paths in the dataset into corresponding state-action sequences, using the cleaned dataset from Step 2 as positive samples, and expanding the dataset to construct negative samples; inputting the positive sample state-action sequences into... The generator inputs positive and negative sample path sequences into the discriminator; positive samples are input into the generator, and positive and negative samples are input into the discriminator; Step 5: Adversarial training is performed based on the generator and discriminator; wherein, the generator obtains the current local map, state, target position, and random noise, and predicts the next action and action parameters of the UAV. After Monte Carlo search, it generates a complete path from the starting point to the end point, and the discriminator evaluates the quality score of the path. The generated path sequence and path information are stored in the experience replay pool. The experience replay pool stores the generated paths and samples according to the path quality priority to improve training efficiency. Each sample in the experience replay pool includes a path point sequence, state sequence, action sequence, map data, starting point, target point, and path quality score; when the number of paths in the experience replay pool reaches a certain value, the adversarial training between the generator and the discriminator begins.

[0006] The beneficial effects of this invention are as follows: This application utilizes the decisions from traditional algorithms as the generator's dataset, enabling the generator to learn to choose actions and corresponding action parameters to avoid obstacles and move towards the target when facing complex obstacle environments. This is then used as prior knowledge for Monte Carlo search, reducing the random search range of Monte Carlo search and increasing its efficiency. In particular, traditional Monte Carlo algorithms perform poorly in long path planning, but this invention improves the performance of traditional algorithms in long path planning by combining the guidance of the generator's prior knowledge.

[0007] In this application, the map modeling is based on a real city map and a no-fly zone is added. At the same time, considering the impact of buildings on wind fields in the urban environment, wind fields that conform to the actual situation are simulated between buildings. During the flight of the UAV, the impact of wind fields on the trajectory and energy consumption of the UAV is dynamically considered, the trajectory deviation caused by wind fields is corrected in a timely manner, the energy consumption distribution is optimized reasonably, the feasibility of the UAV path is improved, and the stable and efficient flight of the UAV is ensured.

[0008] In this application, the path generation process combines a generator and Monte Carlo search. The generator provides prior knowledge, while the Monte Carlo search further expands upon it, making the flight path planning traceable and the decision-making logic of path generation more transparent. For path quality assessment, the discriminator employs a combination of a rule-based discriminator and a neural network discriminator. Clear and quantifiable rules constitute the rule-based discriminator, which performs a preliminary assessment of the path. The neural network discriminator obtains more implicit features of the path. The combination of these two methods not only improves the accuracy of path quality assessment but also provides clear rule-based evidence and auxiliary explanations for the assessment results, thereby enhancing the interpretability of path generation and discriminator judgments. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating a UAV flight path planning method based on sequence generative adversarial networks according to some embodiments of this application; Figure 2 This is a schematic diagram illustrating the process of generating positive samples, negative samples, and subsequent processing according to some embodiments of this application; Figure 3 This is a schematic diagram of a generator network structure according to some embodiments of this application; Figure 4 This is a schematic diagram of a discriminator network structure according to some embodiments of this application; Figure 5 This is a schematic diagram illustrating the process of adversarial training of a generator and a discriminator based on Monte Carlo search according to some embodiments of this application; Figure 6 This is a schematic diagram of a reward mechanism based on Monte Carlo search according to some embodiments of this application. Detailed Implementation

[0010] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] See Figures 1-6 This invention provides a method for UAV flight path planning based on sequence generative adversarial networks, with reference to... Figure 1 Understanding includes the following steps: Step 1: Use real-world OSM (Open Street Map) maps to convert them into 3D city models and use cylinders to construct no-fly zones. Combine these maps into various types of 3D mesh maps and save building data, including building coordinates, radius, size, and height information. Divide the map data into difficulty levels 1-10 based on the number of obstacles, with 30 maps for each difficulty level.

[0012] Step 2: Generate 1000 paths on the 3D grid map using a path planning algorithm and perform data cleaning. The path planning algorithm can be any traditional path planning algorithm such as RRT algorithm (incremental path planning algorithm based on random sampling) and A* algorithm (A* algorithm, a heuristic algorithm). The partitioning process specifically includes: checking whether the path reaches the target point by point and whether it collides with obstacles in the map, and filtering out paths that do not collide and reach the target.

[0013] Step 3: Construct the UAV's state space, motion space, environmental wind field effect model, and UAV energy consumption model; specifically, Step 3 includes the following steps: Step 31: Constructing the UAV's state space includes: The UAV's spatial state contains the UAV's position coordinates in three-dimensional space. and the velocity components of the drone in the three coordinate axes The spatial state of the UAV is defined as a 6-dimensional vector: .

[0014] Step 32: Constructing the drone's motion space includes: The drone contains seven basic discrete motions: And three continuous control parameters: ,in Represents the speed scaling factor, Represents yaw angle, Represents the pitch angle.

[0015] Step 33: Constructing an environmental wind field effect model includes: storing wind field data in a three-dimensional mesh, with each mesh point containing a three-dimensional wind vector; wherein, the wind field is generated based on obstacles in the scene, and the wind field of the surrounding area is calculated for each obstacle, including both wind tunnel effect and turbulence effect.

[0016] Specifically, a wind tunnel effect influence zone is established on both sides of the building. This zone extends from outside the building's width to within 1.5 times its maximum size, with the wind direction parallel to the obstacle surface, forming an acceleration channel. The wind speed decreases with increasing distance, reaching a maximum empirical acceleration coefficient of 5.0 m / s². The wind tunnel effect is based on the Bernoulli equation, which is used to pre-calculate the wind field distribution on the constructed map. Bernoulli's principle describes the relationship between fluid velocity and pressure. In the flow scenario around the building, when airflow encounters the building, it accelerates from both sides, forming a high-speed airflow channel. Bernoulli's equation is defined as: ,in Let ρ be the air density, v be the wind speed, p be the pressure, g be the gravitational acceleration, and h be the height. The turbulence influence zone is located between the edge of the obstacle and 80% of the influence radius. Random disturbances will occur in the turbulence influence zone. The instability of airflow is simulated by a Gaussian distribution. The magnitude of the disturbance decreases with distance, with a maximum fluctuation amplitude of 3.0 m / s.

[0017] For wind field models, it is necessary to consider not only the wind field of obstacles in advance, but also the wind field interaction based on real-time position and obstacle relationship. In UAV motion simulation, the influence of wind field is calculated in real time according to the current position and flight direction. The wind tunnel effect occurs when the UAV is in a narrow passage, such as when there are obstacles on both sides, and it accelerates the UAV in the flight direction. The turbulence effect adds random disturbances when the UAV is in the turbulence influence zone and close to the leeward side of the obstacle, that is, when the obstacle is behind the flight direction. In addition, as the altitude increases, a constant horizontal wind field is applied along the east-northeast direction to simulate the high-altitude wind field. The above interactions directly affect the UAV's velocity vector and affect its position update.

[0018] Therefore, by modeling the environmental wind field effect through data, a grid environment containing wind field information can be obtained. This allows for real-time adjustment of the drone's speed and position based on the wind field effect, taking energy consumption into account for tailwinds and headwinds, enabling the drone to actively avoid strong headwind areas and improving the actual feasibility of the path.

[0019] Compared to existing sampling-based planning algorithms, graph-based search algorithms, and intelligent biomimetic algorithms such as ant colony optimization, A* algorithm, RRT algorithm, and PSO (Particle Swarm Optimization), which only process the dataset directly through the algorithm, this approach lacks modeling of wind field effects and does not adjust the UAV's speed and position in real time to adapt to wind field changes. Furthermore, it does not fully consider the additional consumption caused by wind field effects in energy consumption calculations, which seriously affects the actual feasibility of the path. The embodiments of this application consider the significant impact of dynamic environmental factors such as wind field effects on the UAV path, which may lead to UAV deviation or a surge in energy consumption. It overcomes the limitations of existing path planning algorithms in scenarios with dynamic wind fields and complex 3D obstacles. In 3D environments, due to the dense distribution of obstacles and complex spatial structures, the algorithm has low search efficiency, is prone to getting trapped in local optima, has a single optimization objective, and does not meet dynamic constraints. Ultimately, the planned path can meet the actual flight requirements.

[0020] S34. Constructing an energy consumption model for the UAV includes: obtaining the basic displacement energy consumption, which consists of two parts: horizontal displacement and vertical displacement. Specifically, since the UAV needs to overcome gravity to do work when climbing or descending vertically, it typically consumes more energy than during horizontal flight. Therefore, a higher energy consumption coefficient is set for vertical displacement. The horizontal displacement energy consumption is obtained by multiplying the horizontal distance between the starting and ending points by the horizontal energy consumption coefficient.

[0021] Vertical displacement energy consumption is obtained by multiplying the absolute value of the height difference between the starting and ending points by the vertical energy consumption coefficient, as defined by the formula: ,in, and These are vertical displacement and horizontal displacement, respectively. and These are the energy consumption coefficients for horizontal and vertical displacement, respectively. When a drone accelerates or decelerates, additional energy is required to change its state of motion, which involves the energy consumption of acceleration, and acceleration-related energy consumption. The magnitude of acceleration and the flight distance are jointly determined, and the formula is defined as follows: ,in, This is the acceleration energy consumption coefficient.

[0022] Furthermore, when wind is present in the environment, it significantly impacts the drone's energy consumption, especially headwinds, which increase energy consumption. Pre-calculated wind field data, including wind speed vectors at the start and end points, is obtained from the occupied grid. The average of these vectors is used as an approximation of the wind speed for the entire flight segment. The relative wind speed is obtained by subtracting the drone's current speed from the average wind speed. The projection component of the relative wind speed onto the flight direction is calculated. A positive component indicates a headwind, while a negative component indicates a tailwind. The relative wind speed is obtained by multiplying the square of the headwind component by the flight distance and then by the drag coefficient, as defined in the following formula: ; in, This represents the headwind component of the relative wind speed in the flight direction, i.e., the projected component opposite to the flight direction. This is the wind resistance energy consumption coefficient.

[0023] Understandably, by modeling energy consumption to calculate the total energy consumption of the drone during its flight from the starting point to the destination, and by combining the basic energy consumption related to displacement, the energy consumption affected by the wind field, and the energy consumption related to acceleration, the results are made more realistic.

[0024] Step 4: Use the paths in the dataset from Step 2 as positive samples, and expand the dataset to construct negative samples; convert the positive samples into state-action sequences and input them into the generator, and input the positive and negative sample path sequences into the discriminator. Specifically, combined with... Figure 2Understanding step 4, we have: S41, data cleaning based on step 2, that is, checking the path data in the dataset point by point to determine whether the path reaches the target and whether it collides with obstacles in the map, thereby filtering out paths that do not collide and reach the target. The cleaned dataset is then used as positive samples, and more negative samples are generated by adding collision points, extending path lengths, and adding random noise to the path sequences in the dataset, thus generating a balanced dataset of positive and negative samples.

[0025] S42. Process the cleaned dataset (positive samples) by sampling the path point sequence at equal intervals. Then, calculate the velocity using the displacement of adjacent path points and dynamically adjusted time intervals. When the distance between two points is too far, adjust the time interval to make the velocity reasonable. Calculate the 6-dimensional vector of the spatial state at each moment in sequence. Then, determine the action sequence by matching the velocity direction with the dot product of preset actions, thus transforming the path data into the corresponding action sequence. in, Let A be the action at time t, A be the action space, and e be the action direction component, obtained based on the action space. Let t be the position at time t.

[0026] S43. Input the processed state-action sequence into the generator. The generator receives the local map, current state, target point position and noise vector from the current data, and outputs the action type probability prediction and action parameter prediction.

[0027] In some examples, the generator network structure is as follows: Figure 3 As shown, a bidirectional LSTM is used to capture long temporal dependencies in the path, and a three-dimensional spatial attention mechanism and multimodal feature encoding are integrated, covering noise encoding and four feature encoding branches: state encoding, target encoding, geometric relationship encoding and three-dimensional map encoding. Among them, the noise encoder encodes 32-dimensional random noise into 128-dimensional features through two fully connected layers, introducing randomness into the generator and enhancing the diversity of output.

[0028] In detail, the state encoder employs a three-layer fully connected network to encode the 6-dimensional state vector (including a 3-dimensional position vector and a 3-dimensional velocity vector) into a 128-dimensional feature vector. Leaky ReLU activation and batch normalization are used to enhance the non-linear representation of the features. The target encoder also employs a three-layer fully connected network to encode the 3-dimensional target position (coordinates) obtained in step 31 into a 128-dimensional feature vector, ensuring that the target features and state features are in the same feature space. A multi-head attention mechanism is used to enhance the target feature representation (i.e., target attention in the figure), enabling the generator to focus on the state information most relevant to the target. The geometric relationship encoder explicitly models multi-dimensional spatial geometric features, encompassing kinematic features such as position difference, distance, and orientation angle, capturing the spatial geometric relationship between the state and the target. The input 9-dimensional geometric features include a 3-dimensional position difference vector, a 1-dimensional horizontal distance vector, a 3-dimensional orientation vector, and a 2-dimensional angle vector, which are encoded into a 128-dimensional feature vector through the three-layer fully connected network. The map encoder uses 3D convolutions to compute attention in parallel across the width, height, and depth dimensions. To obtain multi-scale spatial correlations in complex environments and reduce computational complexity, a feature enhancement module is added to decompose the 3D convolutions into 2D convolutions in three orthogonal planes: XY, YZ, and XZ. The first layer is a 3D convolution containing the 3D spatial attention module, the second layer is an XY plane convolution and a Z-axis convolution, the third layer is a 3D convolution containing the feature enhancement module, followed by a fully connected layer that maps the flattened features to 128-dimensional feature vectors.

[0029] The feature fusion layer receives the above five features and obtains more path sequence context features through a bidirectional LSTM layer. Then, it passes through two fully connected neural networks and outputs seven discrete action probabilities after Softmax activation. At the same time, it passes through two fully connected neural networks and outputs continuous action parameters using the Tanh activation function.

[0030] As mentioned above, the path is transformed into a sequence of states and actions and fed into the generator. Supervised learning is used during pre-training to teach the generator how to choose actions when facing obstacles. There are a total of seven actions, and a probability is predicted for each action. The action type loss function is defined as: Where N is the batch size and C is the number of action categories. Let c be the true label of the i-th sample in the c-th category. It is the predicted probability of the c-th class for the i-th sample.

[0031] In the method of this application embodiment, it is necessary not only to learn to select actions, but also to learn how to adjust action parameters when facing current obstacles. The action parameter loss function is defined as: Where N is the batch size. It is the vector of action parameters predicted by the generator for the current sample i. It is the target parameter vector of the current sample i.

[0032] Therefore, the pre-trained generator can initially learn to choose to avoid obstacles and move towards the target when facing the current obstacle, as well as the action parameters, thus avoiding the generation of useless paths at the beginning of adversarial training.

[0033] S44. Input positive and negative samples into the discriminator. The discriminator integrates explicit evaluation based on physical rules with implicit feature extraction from deep learning networks, which improves the interpretability of the discriminator's evaluation. The discriminator receives path sequences, target locations, and 3D maps, and outputs a total score for path quality as well as detailed scores in multiple dimensions, including six dimensions: collision, compliance with dynamic constraints, target achievement, safety, smoothness, and energy consumption. The discriminator integrates explicit evaluation based on physical rules with implicit feature extraction from deep learning networks, which improves the interpretability of the discriminator's evaluation.

[0034] It is worth noting that S43 and S44 can be performed simultaneously.

[0035] Understandably, through the aforementioned steps, the method, based on explicit evaluation of physical rules, assesses six dimensions from the perspectives of kinematic constraints and environmental physical characteristics. Collision detection checks whether a path point is within an obstacle grid; if so, it's considered a collision. It also checks for obstacles within a safe distance around the path point; if present, the score is lowered. Goal achievement checks the distance between the endpoint and the target point; a distance less than 1 meter indicates task completion, with the score decreasing as the distance increases, and a negative score occurring if the distance exceeds a certain threshold. Dynamic constraints include checking whether the speed exceeds the maximum speed, whether the height is within the allowable range, and calculating the curvature of the path point through the angle change between two vectors formed by three consecutive points. Smoothness is assessed by checking changes in continuous curvature and the direction of speed changes. Safety checks the distance between each point on the path and the nearest obstacle. This explicit rule-based evaluation provides a transparent decision-making basis for judging path quality.

[0036] Furthermore, the discriminator network receives the path sequence, target location, and 3D map, and outputs a 7-dimensional vector, where the first dimension is the total score and the last 6 dimensions are the specific scores. The scores are constrained to the interval [0,1] by Sigmoid activation.

[0037] In some embodiments, the discriminator network is a multimodal fusion deep learning model, with the following architecture: Figure 4As shown, the multimodal fusion includes three feature encoding branches: path sequence encoding, environment map encoding (i.e., the map in the figure), and target encoding (i.e., the target location in the figure). Temporal features are obtained through a bidirectional LSTM. The path encoder encodes each state vector in the path sequence into a high-dimensional feature representation through a three-layer fully connected neural network structure, enhancing the model's ability to represent local states of the path. Then, the sequence features output by the path encoder are input into a bidirectional long short-term memory network to integrate the contextual information of each state in the path. The output of the last time step in the LSTM is selected as the overall representation of the sequence. This allows simultaneous capture of the temporal information of the path sequence from the starting point to the current step and from the ending point to the current step. The map encoder effectively captures local correlation features in three-dimensional space. After passing through a 3D convolutional layer and batch normalization, the map is input into a max pooling layer, followed by another 3D convolutional layer. The convolutional layer is fed into an adaptive max-pooling layer and then flattened into a one-dimensional vector. After passing through the LeakyReLU activation function, the output is obtained. The target encoder maps the three-dimensional target coordinates to a feature space of the same dimension as the path features through two linear transformations, which facilitates the subsequent fusion of multimodal features. Then, the path encoding features, map encoding features, and target encoding features are concatenated and fused, and then passed through a three-layer fully connected network to output a six-dimensional score value. The above network can capture the implicit features and complex patterns of the path.

[0038] The paths in the dataset are processed by a rule-based discriminator to generate detailed scores encompassing the six dimensions mentioned above, and these scores are used as supervision signals. The discriminator network is pre-trained using a multi-task loss function with two supervision objectives. When using the six-dimensional detailed scores as supervision signals, the loss function calculates the mean squared error between the detailed scores output by the discriminator and the rule-based discriminator scores. When using binary classification labels as supervision signals, the loss function calculates the mean squared error between the total score output across the six dimensions and the true label. The specific definition of the loss function is as follows: Where, represents the batch sample size. Let be the score for the j-th dimension of the current sample i output by the discriminator. To provide the rule discriminator with a score for the j-th dimension of the current sample i, The total score of the current sample i is output by the discriminator. For binary classification labels, This represents the task weighting coefficient. It is a loss function for six-dimensional scoring. The loss function is the total path score. The trained discriminator network can initially determine the path quality and score the path.

[0039] Step 5: Perform adversarial training based on the generator and discriminator.

[0040] Specifically, such as Figure 5 As shown, during training, maps are progressively loaded, gradually increasing from difficulty level 1 to difficulty level 10, with 30 maps at each difficulty level. A path is generated for each map. The generator receives the current local map, state, target location, and random noise from the starting point and predicts the drone's next action and action parameters. After Monte Carlo search, a complete path from the starting point to the destination is generated. The discriminator evaluates the path's quality score, and the generated path sequence and path information are stored in the experience replay pool. The experience replay pool stores the generated paths and samples them according to path quality priority to improve training efficiency. Each sample in the experience replay pool includes a path point sequence, state sequence, action sequence, map data, starting point, target point, and path quality score. When the number of paths in the experience replay pool reaches a certain value, training of the generator and discriminator begins. The ratio of training steps for the generator and discriminator is dynamically adjusted based on changes in the generator's average reward and the discriminator's loss.

[0041] In detail, the discriminator is first trained, the generator is fixed, the paths in the dataset are obtained and samples are sampled from the experience replay pool. Then, the paths are divided into three groups: high, medium and low, based on their quality. The labels of high-quality samples are close to 0.9, while the labels of low-quality samples are close to 0, which encourages the discriminator to evaluate the quality of the generated samples more carefully.

[0042] The discriminator's adversarial training is based on quality perception. For example, the labels of generated samples are dynamically adjusted according to the quality of the generation path. High-quality samples have labels close to those of real samples, while low-quality samples have labels close to 0. The discriminator's complete loss function consists of four parts: focus loss for real samples, focus loss for generated samples, contrastive loss, and L2 regularization term. Among them, focus loss reduces the weight of easily classified samples, making the model pay more attention to difficult-to-classify samples. Contrastive loss forces a certain difference in scores between high-quality and low-quality samples, thereby enhancing the discriminator's ability to distinguish between samples of different quality. The loss function of the discriminator is defined as follows: in It is the focus loss of the real sample. It is the focus loss of the generated samples. These are the weight parameters for the contrastive loss. These are the coefficients of L2 regularization. It is a comparison of losses. It is the L2 regularization loss.

[0043] The formula for the focus loss of real samples is as follows: in The binary cross-entropy loss BCEWithLogitsLoss for the real samples. Increase the weight of difficult samples as a focal loss parameter.

[0044] The formula for the focus loss of the generated samples is as follows: .

[0045] in BCEWithLogitsLoss is the binary cross-entropy loss for the generated path, and is the focus loss parameter.

[0046] BCEWithLogitsLoss is a combination of binary cross-entropy loss and the sigmoid function, and its formula is as follows: Where is the discriminator's output before passing through the Sigmoid function, and is the target label. It is the Sigmoid function, where N is the current batch size and i is the current sample.

[0047] Contrast loss requires that the score of high-quality samples be at least 0.6 higher than the score of low-quality samples, as shown in the following formula: Where H is the set of high-quality samples and L is the set of low-quality samples. It is the discriminator output of high-quality samples. This is the discriminator output for low-quality samples, where h represents the current high-quality samples and l represents the current low-quality samples.

[0048] The L2 regularization loss formula is as follows: ,in It is the k-th parameter matrix of the discriminator. It is the regularization strength.

[0049] The generator predicts the drone's next action and action parameters. After the Monte Carlo search in step 6, it generates a complete path from the starting point to the end point. The generator's goal is to generate actions and parameters that avoid obstacles and move toward the target when facing the current obstacle. When training the generator, the discriminator's parameters are fixed, and the generator's parameters are updated through the policy gradient method, so that the generator can generate more appropriate actions and action parameters.

[0050] The generator samples a batch of generated paths from the experience replay pool and uses a discriminator to evaluate the quality of the paths as a reward signal. This quality is then used to calculate the policy gradient to update the generator. The generator's updated loss function consists of three parts: the policy gradient of the action probabilities, the policy gradient loss of the action parameters, and an entropy regularization term. The policy gradient loss of the action probabilities encourages the generator to choose actions that lead to high rewards. The policy gradient loss of the action parameters aims to make the predicted parameters closer to the actual optimal parameters. The entropy regularization term encourages exploration and prevents the policy from prematurely converging to a local optimum. The specific formulas are defined as follows: ,in The weights for the loss of action parameters.

[0051] Policy gradient of action probability ,in Is the generator in state? Select Action The probability, It is the advantage function of the current sample i, and N is the batch size.

[0052] Policy gradient loss of action parameters ,in The action parameters of the current sample i predicted by the time generator. is the actual action parameter used by the current sample i, and N is the batch size.

[0053] Entropy regularization term ,in It is the entropy regularization coefficient. For the action of the current sample i, This represents the current state of sample i.

[0054] In addition to the loss function, policy updates also use an advantage function. The advantage function is the difference between the reward gained from the current action and the baseline reward, and is defined by the formula: ,in, It is the path quality score given by the discriminator. It is the moving average of the rewards over a recent period.

[0055] The generator samples a batch of paths from the experience replay pool, uses a discriminator to evaluate the quality of each path, calculates the advantage value of each path, and recalculates the log probability of the action and the log probability of the parameter by replaying the forward process of the generator for each state-action pair. The generator is trained using the policy gradient method to maximize the reward signal given by the discriminator and minimize the path length and energy consumption objectives.

[0056] Continue to combine Figure 5 and Figure 6Understanding is that the node attributes of the Monte Carlo search in step 5 include environmental state information such as obstacle information, position, speed, target, path length, decision history of the action, node relationships with parent and child node mappings and access counts, and cumulative reward statistics. It also introduces action prior probability and parameterized action space, and provides action and action parameter sampling guidance through the generator. After each node is expanded, the node validity will be verified. If the current node or the next node collides with an obstacle, exceeds the longest path limit, violates dynamics, the current position is less than the safe distance from the obstacle, or exceeds the map boundary, it will be judged as invalid and no further node expansion will be performed. Based on the UAV path planning model, the generator progressively predicts the UAV's next action and velocity components at each location, continuously updating the UAV's position. Monte Carlo tree search utilizes the generator's output to guide the iterative search process. Figure 6 As shown, a search tree is constructed through node initialization and four iterative stages: selection, expansion, simulation, and backtracking, ultimately selecting the optimal obstacle avoidance path from the starting point to the end point.

[0057] Specifically, MCTS nodes store the environment state (including obstacle information, location, speed, target, path length), node relationships (parent node, child node mapping), decision history (execution action sequence), and statistical information (access count, cumulative reward), as well as the action space (prior probability distribution, parameterized actions). The selection process involves recursively selecting child nodes starting from the root node until a partially expanded, scalable node is reached. An improved UCB formula is used to balance exploration and exploitation. The improved UCB formula is as follows: in, It is a child node The cumulative reward value, It represents the number of visits to child nodes. This is the number of times the current node has been accessed. It is the generator that gives the state. Select action The prior probability of is used to guide the search in directions that the generator deems promising, and c is the exploration weight. Represents child nodes The average reward value is considered, and this option tends to select nodes with high historical average rewards. Used for balance exploration The number of visits to the parent node increases, giving child nodes with fewer visits a greater chance to be explored. This makes child nodes with fewer visits have higher exploration value, and the exploration weight c controls the intensity of the exploration; In the child node selection phase, the prior knowledge of the generator is introduced into the exploration items. The generator provides the prior probability of the action, thereby reducing random exploration and accelerating convergence. When the prior probability of an action is high, it may be explored even if its current average reward is not high. When the search gets stuck in a local optimum or cannot find an effective action, multiple recovery strategies such as random actions, goal-oriented actions, and backtracking to checkpoints are used to ensure the robustness of the algorithm. The expansion process includes: when a node that is not fully expanded is selected, adding one or more child nodes to it; using a generator to generate the action probabilities and parameters of the current node; to balance efficiency and diversity, only the top K actions with the highest probabilities are expanded, and the value of K is dynamically adjusted according to the current state, for example, K is smaller when close to the target to improve efficiency, and larger when far away or close to obstacles to increase exploration; at the same time, the actions are sorted according to the target direction and safe distance to ensure that the expanded node is both safe and efficient; before or during expansion, the node validity must be verified (including collision detection, path length limit, dynamic constraints, safe distance check, and map boundary check). If a node is determined to be invalid, the expansion of that node is stopped (node ​​failure).

[0058] The simulation includes: starting from a valid expanded node, simulating the execution of a series of actions until the termination state, and calculating the simulation reward value. Node termination states include reaching the target, collision, dynamic violation, and exceeding the boundary. The simulation employs... Strategy, with The probability of randomly selecting an action, in order to The probability selection generator recommends the action with the highest probability. During the simulation, dynamic factors such as wind field effects are considered, and path length and energy consumption are calculated. The number of simulation iterations is also dynamically adjusted based on the distance from the current position to the nearest obstacle and the current position to the target. The backtracking process includes: updating the simulated reward values ​​to all nodes along the path, starting from the node where the simulation ended and backtracking upwards to the root node, updating the visit count and accumulated reward for each node, incrementally updating the node value, avoiding the storage of all historical data, and using the following update formula: in, It is the expected cumulative reward of node s. This is the reward value obtained in this simulation. It is the number of times node s is visited; and the number of visits increases; in addition, the backtracking phase also includes path value propagation and incremental update strategies.

[0059] After the above four stages, the optimal obstacle avoidance path is finally selected. In this process, the generator acts as a provider of prior knowledge. In the selection stage, it improves the UCB formula through prior probabilities and conducts balanced exploration and utilization, including introducing the generator's prior probability P and dynamically adjusting the exploration weight C. In the expansion stage, it provides action prior probabilities and parameters, including the generator providing action prior expansion Top-K, high-probability actions combined with target direction sorting, safety distance priority, and dynamic K value adjustment strategy. Then, in the simulation stage, it guides action selection, reduces ineffective exploration actions, and improves search efficiency.

[0060] In some embodiments, the reward settings in the Monte Carlo search process are also as follows: In Monte Carlo tree search, the reward function is used to evaluate the quality of the current state, thereby guiding the overall search towards a better direction. A multi-objective optimization design is adopted. Under the premise of ensuring safety constraints, the reward for guiding the goal constitutes the driving force for goal approach. At the same time, path quality penalties are introduced to reduce redundant paths, height change penalties optimize vertical efficiency, energy penalties reflect physical constraints, and smoothing rewards improve trajectory continuity. This reward structure effectively balances the optimization needs of goal achievement, path quality, and motion constraints in Monte Carlo search.

[0061] Setting a target reward: When the drone reaches the vicinity of the target point and is less than 1 unit away, a high positive reward is given. Simultaneously, to encourage the generation of shorter paths, a path length penalty is introduced into this positive reward. Therefore, the actual target reward is: Set a target-guided reward: the closer the drone gets to the target, the higher the reward. Set a path smoothness reward: To encourage the generation of smooth paths, calculate the cosine of the angle between the two direction vectors formed by three consecutive positions: the current state, the parent state, and the grandparent state. The reward is maximized when the path direction changes smoothly; Set collision and boundary constraint penalties: If the drone collides with an obstacle, a penalty of -500 will be imposed; if the drone exceeds the preset boundary, a penalty of -500 will also be imposed. Set a path quality penalty: To avoid unnecessary vertical movement, calculate the absolute value of the height change between adjacent states and multiply it by a penalty coefficient. The height value at time t. The height value at the previous time step t Calculate the energy consumption between adjacent states and multiply it by a penalty factor: When the target is not reached, the path length penalty coefficient is dynamically adjusted according to the distance to the target; the closer the distance, the larger the penalty coefficient. If the target is not reached and no collision / boundary crossing occurs, the total reward is: The reward function uses target distance reward to guide the drone closer to the target, while using penalty terms to optimize path length, energy consumption, smoothness and height stability to achieve multi-target balance, and using collision and boundary penalties to ensure path safety, thus meeting the needs of path planning.

[0062] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for generating a UAV flight path plan based on a sequence generative adversarial network, characterized in that, The method comprises the following steps: Step 1: converting a real open street map into a three-dimensional city model, using a cylinder to build a no-fly zone to form a three-dimensional grid map, wherein building data is saved in the three-dimensional grid map; Step 2: using a path planning algorithm to generate a plurality of paths on the three-dimensional grid map as a data set and performing data cleaning, including: checking whether the path reaches the target and collides with the obstacles in the map point by point, and selecting the path that does not collide and reaches the target; Step 3: constructing a state space, an action space, an environmental wind field effect model and a UAV energy consumption model of the UAV; Step 4: taking the path in the data set cleaned in step 2 as a positive sample, and expanding the data set to construct a negative sample; converting the positive sample into a state-action sequence and inputting the positive and negative sample path sequences into a generator and a discriminator; Step 5: performing adversarial training based on the generator and the discriminator; Wherein, the generator obtains the current local map, state, target position and random noise and predicts the next action and action parameter of the UAV, generates a complete path from the starting point to the ending point after Monte Carlo search, and judges the quality score of the path through the discriminator, stores the path sequence and path information in the experience replay pool, the experience replay pool stores the generated path, and the training efficiency is improved according to the path quality priority sampling, each sample in the experience replay pool includes a path point sequence, a state sequence, an action sequence, map data, a starting point, a target point and a path quality score; when the number of paths in the experience replay area reaches a preset value, the adversarial training of the generator and the discriminator is started. 2.The method of claim 1, wherein, Step 3 comprises the following steps: Step 31: constructing a state space of the UAV; The spatial state of the UAV includes position coordinates of the UAV in three-dimensional space and velocity components of the UAV in three coordinate axis directions in a spatial state vector of the UAV: ; Step 32: constructing an action space of the UAV; The drone comprises seven basic discrete actions: and three continuous control parameters: , wherein represents a speed scaling factor, represents a yaw angle, represents a pitch angle; Step 33: constructing an environmental wind field effect model; Store the wind field data in the three-dimensional grid, each grid point contains a three-dimensional wind vector; wherein, the wind field data is generated based on the obstacles in the environment, and the wind field around each obstacle is calculated, including calculating the wind tunnel effect and the turbulence effect; S34, constructing a UAV energy consumption model, comprising: obtaining displacement-based energy consumption, including horizontal displacement and vertical displacement; The horizontal displacement energy consumption is obtained by calculating the horizontal distance between the starting point and the ending point and multiplying the horizontal energy consumption coefficient; Energy consumption of vertical displacement The calculation formula is defined as: wherein, and are the vertical displacement and the horizontal displacement, respectively, and are the energy consumption coefficients of the horizontal displacement and the vertical displacement, respectively. 3.The method of claim 2, wherein, When there is a wind field in the environment, the wind field data is introduced into the UAV energy consumption model, including: Obtaining the precomputed wind field data from the occupied grid, including the wind speed vector of the starting point and the ending point and taking the average to obtain the average wind speed as the approximate wind speed of the entire flight segment, and the average wind speed is subtracted from the current speed of the UAV to obtain the relative wind speed; Computing the projection component of the relative wind speed in the direction of flight If the projection component is positive, it indicates an adverse wind; if it is negative, it indicates a favorable wind; Computing the projection component The formula is defined as: ; wherein is the wind speed in the flight direction, is the wind resistance energy consumption coefficient, is the flight distance. 4.The method of claim 2, wherein, The step 4 comprises: S41, taking the cleaned data set as a positive sample, adding collision points to the path sequence in the data set, lengthening the path, adding random noise after adding the collision points to obtain a negative sample, and generating a balanced data set of positive and negative samples; S42, the positive sample is processed, the path point sequence is equally spaced sampled, then the speed is calculated through the displacement of adjacent path points and the dynamically adjusted time interval; the spatial state vector at each time is calculated in turn, then the action sequence is determined through the dot product matching of the speed direction and the preset action, so as to convert the path data into the corresponding action sequence: wherein, is the action at time t, A is the action space, is the action direction component, is the elapsed time, obtained based on the action space, is the position at time t; S43, the processed state action sequence is input into the generator, the generator receives the local map, the current state, the target point position and the noise vector in the current data, and outputs the action type probability prediction and the prediction of the action parameter; S44, the positive and negative samples are input into the discriminator, the discriminator receives the path sequence, the target position and the three-dimensional map, and outputs the total score of the path quality and the multi-dimensional detailed score, the multi-dimensional detailed score including six-dimensional scores of whether to collide, whether to meet the dynamic constraint, whether to reach the target, safety, smoothness and energy consumption. 5.The method of claim 4, wherein, The generator network uses a bidirectional long short-term memory network and fuses three-dimensional spatial attention mechanism and multi-modal feature encoding, including encoding and four-way feature encoding branches, respectively state encoding, target encoding, geometric relationship encoding and three-dimensional map encoding; wherein: The noise encoder encodes 32-dimensional random noise into 128-dimensional features through two fully connected networks; The state encoder adopts a three-layer fully connected network to encode the spatial state vector into a 128-dimensional feature vector; The target encoder uses a three-layer fully connected network to encode the three-dimensional vector corresponding to the coordinates of the target position into a 128-dimensional feature vector, and uses a multi-head attention mechanism to enhance the target feature representation; The geometric relationship encoder inputs 9-dimensional geometric features, including a three-dimensional position difference vector, a one-dimensional horizontal distance vector, a three-dimensional direction vector and a two-dimensional angle vector, and encodes them into a 128-dimensional feature through a three-layer fully connected network. After the path is converted into a state action sequence, it is input into the generator, and pre-training is performed using supervised learning to enable the generator to select actions for the current obstacles, and a probability is predicted for each action, wherein the action type loss function is defined as: where N is the batch size, C is the number of action classes, is the true label of the c-th class for the i-th sample, is the predicted probability of the c-th class for the i-th sample. The action parameter loss function is defined as: where N is the batch size, is the action parameter vector predicted by the current sample i generator, is the target parameter vector for the current sample i. 6.The method of claim 5, wherein, The discriminator network receives the path sequence, the target position and the three-dimensional map, and outputs a 7-dimensional vector, wherein the first dimension is the total score, and the last 6 dimensions are multi-dimensional detailed scores; The discriminator network is a multi-modal fusion deep learning model, including three-layer feature encoding branches, respectively path sequence encoding, environment map encoding and target encoding, and obtaining time sequence features through a bidirectional long short-term memory network; The path encoder encodes each state vector in the path sequence into a high-dimensional feature representation through a three-layer fully connected neural network structure, and the sequence feature output by the path encoder is input into the bidirectional long short-term memory network to integrate the context information of each state in the path, and the output of the last time step in the long short-term memory network is selected as the overall representation of the sequence. The target encoder maps the 3D target coordinates to the feature space with the same dimension as the path feature through two linear transformations, then splices and fuses the path encoding feature, the map encoding feature and the target encoding feature, and finally outputs a six-dimensional detailed score through a three-layer fully connected layer network; The scores generated by the rule discriminator in the data set include the six-dimensional detailed score, and the score of the rule discriminator is used as the supervision signal; wherein the pre-training of the discriminator network uses a multi-task loss function, including two supervision targets: When using the six-dimensional detailed score as the supervision signal, the loss function calculates the mean square error between the detailed score output by the discriminator and the rule-based discriminator score, and when using the binary classification label as the supervision signal, the loss function calculates the mean square error between the total score of the six-dimensional detailed score output and the real label, and the loss function L is defined as follows: In the formulae, ; ; wherein, is the number of batch samples, is the score of the jth dimension of the current sample i output by the discriminator, is the score of the jth dimension of the current sample i given by the rule discriminator, is the total score of the current sample i output by the discriminator, is the binary classification label truth value, is the task weight coefficient, is the loss function of the six-dimensional multi-dimensional detailed score, is the loss function of the path total score, and the trained discriminator network preliminarily judges the path quality as the path score.

7. The sequence-generative adversarial network based UAV flight path planning method of claim 6, wherein, In the step 5, the adversarial training of the discriminator is based on quality perception, and the complete loss function of the discriminator includes the focal loss of the real sample, the focal loss of the generated sample, the contrast loss and the L2 regularization term; The loss function of the discriminator is defined as follows: wherein, is the focal loss for real samples, is the focal loss for generated samples, is a weight parameter for the contrastive loss, is a coefficient for L2 regularization, is the contrastive loss, is the regularization loss; The focal loss formula of the real sample is as follows: wherein the binary cross-entropy loss for real samples, is the focal loss parameter and e is the natural constant; The focal loss formula of the generated sample is as follows: ， wherein is the binary cross-entropy loss for generating the path; The contrast loss requires that the score of the high-quality sample be at least 0.6 higher than that of the low-quality sample, and the formula is as follows: where H is a set of high-quality samples, L is a set of low-quality samples, is a discriminator output of a high-quality sample, is a discriminator output of a low-quality sample, h is a current high-quality sample, and l is a current low-quality sample. The regularization loss formula is as follows: wherein is the kth parameter matrix of the discriminator, is the regularization strength. 8.The method of claim 7, wherein, The step 5 further includes: The generator samples a batch of generated paths from the experience replay pool, and evaluates the quality of the paths using the discriminator as a reward signal, and updates the generator using the policy gradient of the reward signal calculation strategy, and the loss function of the generator update includes the policy gradient of the action probability, the policy gradient loss of the action parameter, and the entropy regularization term; the specific formula definition is as follows: wherein is the weight of the action parameter loss; The policy gradient loss for the action probability is: where is the probability of the generator selecting action at state , is the advantage function for the current sample i, and N is the batch size. The policy gradient loss of the action parameter is: wherein is the action parameter of the current sample i predicted by the generator, is the action parameter actually used by the current sample i, and N is the batch size. The entropy regularization term is: where is an entropy regularization coefficient, is the action for the current sample i, is the state for the current sample i; where the advantage function is the difference between the reward obtained for the current action and the baseline reward. 9.The method of claim 8, wherein, The node attribute of the Monte Carlo search sets the environment state containing obstacle information, position, velocity, target, path length, decision history of executing action, node relationship with parent node, child node mapping, access frequency and statistical information of cumulative reward, and introduces action prior probability and parameterized action space, and the generator provides action and action parameter sampling guidance; After each node is expanded, node validity verification is performed, and if the current node or the next node collides with the obstacle, exceeds the maximum path limit, violates the dynamics, the current position is less than the safety distance from the obstacle, or exceeds the map boundary, it is judged as invalid and no longer expanded; Furthermore, based on the unmanned aerial vehicle path planning model, the generator gradually predicts the next step action and velocity component of the unmanned aerial vehicle at each position, and constantly updates the position of the unmanned aerial vehicle, constructs a search tree through four iterative stages of node initialization, selection, expansion, simulation and backtracking, and finally selects the optimal obstacle avoidance path from the starting point to the end point. 10.The method of claim 9, wherein, It also includes the reward setting in the Monte Carlo search process: The target reward is set as: when the UAV reaches the vicinity of the target point, the distance is less than 1 unit, a high positive reward is given, at the same time, in order to encourage the generation of shorter path, the path length penalty is introduced in this positive reward, therefore, the actual target reward is: wherein, is a path length penalty; Setting target guidance rewards The closer the drone gets to the target, the higher the reward it gets: distance from the drone to the target; A path smoothness reward is set to encourage generating a smooth path, and a cosine value of two direction vectors formed by current, parent state, and grandparent state continuous three position points is calculated The reward reaches the maximum when the path direction changes smoothly, and is expressed as : Collision and boundary constraint penalties are set, and if the unmanned aerial vehicle collides with the obstacle, a penalty of-500 is given, and if the unmanned aerial vehicle exceeds the preset boundary, a penalty of-500 is also given; A path quality penalty is set to avoid unnecessary vertical motion, and the absolute value of the height change between adjacent states is calculated and multiplied by a penalty coefficient: is the height value at time t, is the height value at the previous time t. Computing the energy consumption between adjacent states , multiplied by a penalty factor: Energy expenditure; When the target is not reached, the path length penalty coefficient is dynamically adjusted according to the distance from the target, and the closer the distance, the larger the penalty coefficient: When the target is not reached and no collision / boundary crossing occurs, the total reward R is: 。

Images