Multi-vertical fixed-wing unmanned aerial vehicle flight path planning and intelligent obstacle avoidance method

Abstract

The invention discloses a multi-vertical fixed-wing unmanned aerial vehicle flight path planning and intelligent obstacle avoidance method, and relates to the technical field of unmanned aerial vehicle flight path planning. Comprising the following steps: according to the dynamic characteristics of the multi-vertical fixed-wing unmanned aerial vehicle in a vertical take-off and landing mode and a horizontal flight mode, constructing a unified kinematics model and initializing unmanned aerial vehicle parameters and an environment model; carrying out global pre-flight path planning by adopting an improved particle swarm optimization algorithm, and generating a global flight path of the multi-vertical fixed-wing unmanned aerial vehicle according with flight characteristics; in the actual flight process, in combination with a deep reinforcement learning DDPG and an improved interference fluid power system IIFDS algorithm, flight modes of the unmanned aerial vehicle are intelligently switched, and multiple dynamic obstacles are efficiently and automatically avoided; and after obstacle avoidance is completed, the unmanned aerial vehicle rapidly restores the fixed-wing flight mode and flies along the preset route again. According to the method, the problem of multi-vertical fixed-wing unmanned aerial vehicle cooperative flight path planning in multiple complex scenes can be solved, and the purpose that the cost of task execution of multiple unmanned aerial vehicles is minimum is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) trajectory planning, and in particular to a trajectory planning and intelligent obstacle avoidance method for multiple vertical take-off fixed-wing UAVs based on an improved particle swarm algorithm combined with deep reinforcement learning. Background Art

[0002] In recent years, drones have been widely used in various industries. However, as the tasks faced by drones become more and more complex, a single drone often cannot effectively perform complex tasks. Vertical fixed-wing drones combine the advantages of multi-rotor drones and fixed-wing drones and are low-cost, easy to use, and widely used. At the same time, trajectory planning is one of the key technologies in the research of multi-drone collaborative work. Therefore, the trajectory planning method of multiple vertical fixed-wing drones has important theoretical value and practical significance.

[0003] Particle swarm optimization is a global optimization method based on swarm intelligence, which has the characteristics of fast convergence, simple implementation and strong adaptability. It searches for the optimal solution in a complex high-dimensional space by simulating the collaborative behavior of groups in nature (such as bird migration or fish foraging). In the trajectory planning of unmanned aerial vehicles, the particle swarm optimization algorithm can be used to quickly generate the initial path and optimize multiple objectives such as energy consumption, smoothness and safety of the path.

[0004] However, traditional particle swarm optimization methods are not adaptable enough to sudden situations (such as changes in obstacles) in dynamic environments, and it is difficult to update path planning results in real time. Reinforcement learning is a machine learning method that interacts with the environment and continuously adjusts strategies to achieve goals. In UAV trajectory planning, reinforcement learning enables drones to learn optimal strategies in dynamic environments and complete tasks autonomously by designing a reasonable reward mechanism. Reinforcement learning based on deep neural networks (such as deep Q network DQN, deep deterministic policy gradient DDPG, etc.) further improves the performance of reinforcement learning in high-dimensional state and action space, providing UAVs with flexible real-time path planning capabilities. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to provide a drone trajectory planning and intelligent obstacle avoidance method that can solve the problem of collaborative trajectory planning of multiple vertical take-off fixed-wing drones in multiple complex scenarios, meet the purpose of minimizing the cost of multiple drones performing tasks, and ensure that all drones can smoothly reach their destinations.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a multi-vertical fixed-wing UAV trajectory planning and intelligent obstacle avoidance method, comprising the following steps: S1, based on the dynamic characteristics of multiple vertical take-off and landing fixed-wing UAVs in vertical take-off and landing mode and horizontal flight mode, a unified kinematic model is constructed and the UAV parameters and environmental model are initialized; S2, using the improved particle swarm optimization algorithm for global pre-trajectory planning to generate global trajectories that meet the flight characteristics of multiple vertical take-off fixed-wing UAVs; S3, in the actual flight process, combines the DDPG algorithm of deep reinforcement learning and the improved interferometric fluid dynamic system IIFDS algorithm to realize intelligent switching of the UAV's flight mode and achieve efficient automatic avoidance of multiple dynamic obstacles; S4, after obstacle avoidance is completed, the drone quickly resumes fixed-wing flight mode and flies along the predetermined route again.

[0007] The beneficial effects of adopting the above technical solution are: the method is based on the fusion of particle swarm optimization and reinforcement learning, and uses particle swarm optimization to quickly generate an initial path, providing a good initial strategy for reinforcement learning, thereby shortening the training time; at the same time, reinforcement learning continuously optimizes the strategy, intelligently switches the flight mode of vertical take-off fixed-wing UAVs, improves the path planning ability of UAVs in dynamic environments, improves the global track planning and intelligent dynamic obstacle avoidance capabilities of multiple vertical take-off fixed-wing UAVs, and improves the task execution and adaptability of multiple vertical take-off fixed-wing UAVs in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0009] Figure 1 is a flow chart of the method described in an embodiment of the present invention; Figure 2 is a schematic diagram of global trajectory planning using an improved particle swarm optimization algorithm in the method described in an embodiment of the present invention; Figure 3 It is a system architecture diagram of the DDPG-IIFDS algorithm for dynamic obstacle avoidance in the method described in an embodiment of the present invention. DETAILED DESCRIPTION

[0010] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0011] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0012] like Figure 1 As shown, the present invention discloses a method for trajectory planning and intelligent obstacle avoidance of multiple vertical take-off fixed-wing UAVs, and the method specifically comprises the following steps: S1, based on the dynamic characteristics of the vertical take-off and landing mode and the horizontal flight mode of the fixed-wing UAV, a unified kinematic model is constructed and the UAV parameters and environmental model are initialized.

[0013] Furthermore, in step S1, the fixed-wing UAV moves in the fixed-wing mode and the rotary-wing mode, and the kinematic equation can be obtained by calculation as follows: In the formula is the displacement offset of the drone on the x-axis, y-axis, and z-axis. , , Represent the linear velocity in the x-axis, y-axis, and z-axis directions respectively; is the velocity component of the UAV, is the pitch angle of the drone, and the tilt angle is positive; is the roll angle of the drone, and rolling to the right is defined as positive; The angle at which the drone rotates is called the yaw angle, and a leftward deflection is considered positive.

[0014] Furthermore, in step S1, during the initialization process, the basic parameters of the vertical fixed-wing UAV, the initial position of the UAV , Target location , Maximum pitch angle , Maximum roll angle , Maximum yaw angle , the maximum speed of unmanned flight in free mode V max Initialize the environment model.

[0015] S2, using the improved particle swarm optimization algorithm for global pre-trajectory planning, generates global trajectories that meet the flight characteristics of multiple vertical take-off fixed-wing UAVs, such as Figure 2 The figure shows the global pre-planned trajectory flow chart, which specifically includes the following steps: S2.1: Initialize the particle swarm N, each particle is used as a trajectory planning route, set the maximum number of iterations T, inertia weight A, displacement factors r1 and r2, the initial values ​​of the particle individual extreme value and the global extreme value; S2.2: Start iterative optimization, and calculate the fitness value each time according to the constraints and penalty values ​​set by the fitness function. The constraints are the track length cost, threat cost, and pitch angle constraint. The update formula of the fitness function is as follows: In the formula, G represents the fitness function, represents the path length cost, Indicates the threat cost, represents the pitch angle cost, Represents the weight value of each constraint indicator, and satisfies ; S2.3: Adopt the fusion dynamic classification processing strategy, and divide the sub-groups into elite sub-groups, exploration sub-groups and equilibrium sub-groups according to the fitness ranking.

[0016] Step 2.4: The elite subgroup combines the Ryan flight strategy with the greedy algorithm to effectively avoid falling into the local optimum and enhance its ability to jump out of the local optimum. The position update formula of its Ryan flight is as follows: In the formula, For the next flight position, is the step length reduction factor, is the Lévy step length. , A random variable that follows a normal distribution, is the shape parameter of the Lévy distribution.

[0017] S2.5: The exploration subgroup introduces the Gaussian mutation mechanism to improve the global search efficiency by increasing the search step size and gradually approaching the global optimal solution. The update formula of Gaussian mutation is as follows: In the formula, represents the position of the particle after mutation, (0, 1) is a random number of 0 and 1. is the current particle position.

[0018] S2.6: The balanced subgroup optimizes the position and speed of particles through dynamic inertia weights to balance the capabilities of local search and global survey. The update formula of dynamic inertia weights is as follows: In the formula, is the current inertia weight, , is the maximum and minimum value of the inertia weight, i is the current iteration number, is the maximum number of iterations.

[0019] Subsequently, each sub-cluster is merged to form a new particle swarm to further improve the search performance and optimization capability of the overall algorithm.

[0020] S2.7. After executing the set maximum number of iterations, multiple trajectory planning routes for each UAV are generated.

[0021] S2.8. According to spatial constraints, select collaborative trajectories planned by multiple UAVs without repeated paths to avoid collisions between multiple UAVs during flight. At the same time, according to time constraints, coordinate the flight times of multiple UAVs so that they arrive at the target area as synchronously as possible to achieve efficient collaborative task execution.

[0022] S2.9. Use cubic spline interpolation method to smooth the generated track, eliminate mutation points in the path, and improve the feasibility and safety of the track.

[0023] S3, in the actual flight process, combines DDPG and IIFDS of deep reinforcement learning to realize intelligent switching of drone flight modes and efficient automatic avoidance of multiple dynamic obstacles. The system framework diagram of its DDPG-IIFDS algorithm is shown in the figure below: Figure 3 As shown in Figure 1, the algorithm contains n vertical fixed-wing drones, each of which has an Actor network, a target Actor network, a Critic network, and a target Critic network. The drone interacts with the environment and obtains environmental information. After training, each drone will get a strategy. S3 specifically includes the following steps: S3.1: Initialize the Actor network parameters, target Actor network parameters, Critic network parameters and target Critic network parameters of the obstacle avoidance model of deep reinforcement learning for multiple vertical take-off fixed-wing UAV clusters; S3.2: The system creates an experience replay pool to store the experience data generated during the interaction between the drone and the environment, initializes the experience pool capacity D, the number of samples for training n, and the maximum number of training rounds Episode; S3.3: Each drone senses the environment, obtains the status data of the flight environment in real time, and updates the Drone status , the state space of the UAV is defined as: ; In the formula The linear velocity of the UAV at time t is expressed as , Respectively represent the linear velocity in the x-axis, y-axis, and z-axis directions, and the position in the world coordinate system , It is expressed as the displacement along the x-axis, y-axis, and z-axis in space. is the rolling angle, is the pitch angle, is the yaw angle, For obstacle status, is the current flight mode; The action space is defined as: ; Mode Indicates The output parameters of the IIFDS of the UAV at time t, , To exclude parameters and tangential parameters, is the tangential coefficient, Indicates actions in flight mode; S3.4: According to the current state and explore noise Generating Actions via Actor Networks , adding noise to increase the randomness of the training process, predicting the movement trajectory of obstacles through the improved interferometric fluid dynamic system (IIFDS) algorithm, dynamically adjusting the position and speed of the vertical fixed-wing drone, and updating the status and every Rewards for flying a robot ; Furthermore, in S3, the vertical fixed-wing UAV intelligent obstacle avoidance performs real-time obstacle avoidance by intelligently switching flight modes. The fixed-wing mode has a faster flight speed and is stable in avoiding sparse obstacles. In the vertical flight mode, the UAV can flexibly hover and ascend vertically to avoid collisions with dense horizontal obstacles. The mode switching of the vertical fixed-wing UAV is evaluated by evaluating the risk factor E. When E reaches the set threshold, the UAV enters emergency refuge and switches to the vertical flight mode; otherwise, it maintains the horizontal flight mode. The evaluation formula for the risk level is as follows: In the formula, To assess the risk level factor; The angle between the speed direction of the drone and the obstacle is used to reflect whether the obstacle is approaching the drone. Indicates that the obstacle is in the same direction as the drone and there is no obvious threat; Indicates that the obstacle is approaching and poses the greatest threat; is the relative speed between the drone and the obstacle is the obstacle speed; , , , is the risk assessment coefficient, the sum of which is 1, and is used to adjust the weight of the impact of various parameters on the risk level; is the safety radius of the obstacle; is the distance between the drone and the center of the obstacle; Represents the obstacle density.

[0024] Furthermore, by improving the interferometric fluid dynamic system IIFDS algorithm, when there are dynamic obstacles, the obstacles will affect the speed of the UAV. The speed adjustment and position update of a single vertical take-off fixed-wing UAV are expressed as follows: In the formula, is the interference modulation matrix of the kth obstacle, is the identity matrix, is the normal vector of the obstacle, is the tangent vector of the obstacle, is the obstacle equation, and To exclude parameters and tangential parameters, For the speed of modification, is the speed at time t, is the overall velocity vector of the obstacle, Calculate the next track point, is the sampling time.

[0025] Furthermore, the S3.4 updates the status by the IIFDS algorithm. and rewards The reward mechanism is aimed at The drones are represented as follows: In the formula, , , is the reward coefficient, which is always a positive number. The distance reward can be expressed as: Where d0 is the distance between the previous position of the drone and the target position, and d1 is the distance between the current position and the target position.

[0026] The collision reward can be expressed as: The time penalty can be expressed as: in, is the time step that the UAV has flown.

[0027] S3.5: Stored in the experience replay pool, if the number of samples in the experience pool is greater than the number of samples for training, then randomly extract N from the experience replay pool A indivual Mini-batch samples, update the target parameters, otherwise continue with step 3.4.

[0028] Furthermore, the steps of controlling and updating the target parameters in S3.5 are as follows: 1) For the extracted small batch samples, use the target network to calculate the target Q value ; In the formula No. The reward of a drone at time t, is the discount factor used to correct the online network, is the current state, For the next joint state, For the status Next action The target state-action value; 2) Update the Critic network parameters of the DDPG algorithm by minimizing the loss function L; the loss function is defined as the mean square error between the current Q value estimate and the calculated target Q value. The loss function is expressed as: ; In the formula For joint action, For joint rewards, Express expectations; 3) A delayed update strategy is adopted for the update of online Actor network parameters. The Actor network parameters are updated by the gradient ascent method. For each vertical fixed-wing UAV, the gradient formula is: ; In the formula is the Q-value function with respect to the action The gradient of is the policy function about The gradient of 4) DDPG adopts a soft update strategy, using the Actor objective function and the Critic objective function to calculate the target value. The update method of the Actor target network and the Critic target network is as follows: ; In the formula is the Actor target network parameter, is the Actor network parameter, is the Critic target network parameter, is the Critic network parameter, is the sliding average coefficient, and its value is less than 1.

[0029] S3.6: Increase the number of time steps and conduct the next round of training until it ends, and obtain the trained model.

[0030] S3.7: Input the optimized state features into the DDPG network to decide the next action of the multi-vertical fixed-wing UAV, including flight mode switching and trajectory control.

[0031] S4, after the obstacle avoidance is completed, the drone quickly resumes the fixed-wing flight mode and re-flies along the planned route to ensure the continuity and efficient execution of the mission. After the drone completes the obstacle avoidance, it will automatically switch back to the normal fixed-wing flight mode and re-execute the flight mission according to the pre-planned global route, thus achieving efficient trajectory planning for multi-drone collaboration.

[0032] This method improves the global trajectory planning and intelligent dynamic obstacle avoidance capabilities of multiple vertical take-off fixed-wing UAVs, and enhances the mission execution and adaptability of multiple vertical take-off fixed-wing UAVs in complex environments.

Claims

1. A method for trajectory planning and intelligent obstacle avoidance of multiple vertical take-off fixed-wing UAVs, characterized in that The steps include: S1, based on the dynamic characteristics of multiple vertical take-off and landing fixed-wing UAVs in vertical take-off and landing mode and horizontal flight mode, a unified kinematic model is constructed and the UAV parameters and environmental model are initialized; S2, using the improved particle swarm optimization algorithm for global pre-trajectory planning to generate global trajectories that meet the flight characteristics of multiple vertical take-off fixed-wing UAVs; S3, in the actual flight process, combines the DDPG algorithm of deep reinforcement learning and the improved interferometric fluid dynamic system IIFDS algorithm to realize intelligent switching of the UAV's flight mode and achieve efficient automatic avoidance of multiple dynamic obstacles; S4, after the obstacle avoidance is completed, the multi-vertical fixed-wing UAV quickly resumes the fixed-wing flight mode and flies along the predetermined route again.

2. The method for trajectory planning and intelligent obstacle avoidance of multiple vertical take-off fixed-wing UAVs as claimed in claim 1, characterized in that: In step S1, the multi-vertical fixed-wing UAV moves in fixed-wing mode and rotary-wing mode, and the speed In the body coordinate system, it is decomposed into , the kinematic equation is obtained by calculation as follows: ； In the formula is the displacement offset of the drone on the x-axis, y-axis, and z-axis. , , Represent the linear velocity in the x-axis, y-axis, and z-axis directions respectively; is the velocity component of the UAV, is the pitch angle of the drone, and the tilt angle is positive; is the roll angle of the drone, and rolling to the right is defined as positive; The angle at which the drone rotates is called the yaw angle, and a leftward deflection is considered positive.

3. The method for trajectory planning and intelligent obstacle avoidance of multiple vertical take-off fixed-wing UAVs as claimed in claim 1, characterized in that: The drone parameters initialized in S1 include: Initial position of the drone , Target location , Maximum pitch angle , Maximum roll angle , Maximum yaw angle And the maximum speed V of unmanned flight in free mode max .

4. The method for trajectory planning and intelligent obstacle avoidance of multiple vertical take-off fixed-wing UAVs as claimed in claim 1, characterized in that: The method for generating the global track in S2 comprises the following steps: S2.1, initialize the particle swarm N, each particle is used as a trajectory planning route, set the maximum number of iterations T, inertia weight A, displacement factors r1 and r2, the initial values ​​of the particle individual extreme value and the global extreme value; S2.2, start iterative optimization, calculate each fitness value according to the constraints and penalty values ​​set by the fitness function, the constraints are the track length cost, the threat cost, and the pitch angle constraint, and the update formula of the fitness function is as follows: ； In the formula, G represents the fitness function, represents the path length cost, Indicates the threat cost, represents the pitch angle cost, Represents the weight value of each constraint indicator, and satisfies ; S2.3, adopt the fusion dynamic classification processing strategy, and divide the sub-groups into elite sub-groups, exploration sub-groups and equilibrium sub-groups according to the fitness ranking; S2.4, the elite subgroup combines the Ryan flight strategy with the greedy algorithm and enhances its ability to jump out of the local optimum. The position update formula of its Ryan flight is as follows: ； ； In the formula, For the next flight position, is the step length reduction factor, is the Lévy step length; , A random variable that follows a normal distribution, is the shape parameter of the Lévy distribution; S2.5, the exploration subgroup introduces the Gaussian mutation mechanism, which increases the search step size to improve the global search efficiency and gradually approaches the global optimal solution. The update formula of Gaussian mutation is as follows: ； In the formula, represents the position of the particle after mutation, (0, 1) is a random number of 0 and 1. is the current particle position; S2.6, the balanced subgroup optimizes the position and speed of particles through dynamic inertia weights, balancing the capabilities of local search and global survey. The update formula of dynamic inertia weights is as follows: ； In the formula, is the current inertia weight, , is the maximum and minimum value of the inertia weight, i is the current iteration number, is the maximum number of iterations; Subsequently, each sub-cluster is merged to form a new particle cluster; S2.7, after executing the set maximum number of iterations, generating multiple trajectory planning routes for each UAV; S2.

8. According to the spatial constraints, select the collaborative trajectory planned by multiple UAVs without repeated paths, and avoid collisions between multiple UAVs during flight; at the same time, according to the time constraints, coordinate the flight time of multiple UAVs so that they arrive at the target area synchronously to achieve efficient collaborative task execution; S2.

9. Use cubic spline interpolation to smooth the generated track and eliminate mutation points in the path.

5. The method for trajectory planning and intelligent obstacle avoidance of multiple vertical take-off fixed-wing UAVs as claimed in claim 1, characterized in that: The S3 combines the DDPG algorithm and the IIFDS algorithm, and adopts a centralized training distributed execution method. The algorithm uses deep reinforcement learning to generate the parameters of the IIFDS algorithm, including n vertical fixed-wing drones. Each drone has an Actor network, a target Actor network, a Critic network, and a target Critic network. After training, each drone obtains a strategy. The specific steps are as follows: S3.

1. Initialize the Actor network parameters, target Actor network parameters, Critic network parameters and target Critic network parameters of the obstacle avoidance model of deep reinforcement learning for multiple vertical take-off fixed-wing UAV clusters; S3.2, the system creates an experience replay pool to store the experience data generated during the interaction between the drone and the environment. The initialization experience pool capacity is D, the number of samples for training is n, and the maximum number of training rounds is Episode; S3.

3. Each drone senses the environment, obtains the status data of the flight environment in real time, and updates the Drone status , the state space of the UAV is defined as: ； In the formula The linear velocity of the UAV at time t is expressed as , Respectively represent the linear velocity in the x-axis, y-axis, and z-axis directions, and the position in the world coordinate system , It is expressed as the displacement along the x-axis, y-axis, and z-axis in space. is the rolling angle, is the pitch angle, is the yaw angle, For obstacle status, is the current flight mode; The action space is defined as: ； Mode Indicates The output parameters of the IIFDS of the UAV at time t, , To exclude parameters and tangential parameters, is the tangential coefficient, Indicates actions in flight mode; S3.4, according to the current state and explore noise Generating Actions via Actor Networks , adding noise to increase the randomness of the training process, by improving the interferometric fluid dynamic system IIFDS algorithm, predicting the movement trajectory of obstacles, dynamically adjusting the position and speed of multiple vertical take-off fixed-wing drones, and updating the status and each Rewards for flying a robot ; S3.

5. Stored in the experience replay pool, if the number of samples in the experience pool is greater than the number of samples for training, then randomly extract N from the experience replay pool A indivual For small batch samples, update the target parameters, otherwise continue with step S3.4; S3.6, increase the number of time steps, conduct the next round of training until it is finished, and obtain the trained model; S3.

7. Input the optimized state features into the DDPG network to decide the next action of the multi-vertical fixed-wing UAV, including flight mode switching and trajectory control.

6. The method for trajectory planning and intelligent obstacle avoidance of multiple vertical take-off fixed-wing UAVs as claimed in claim 5, characterized in that: In S3, the mode switching of the vertical take-off fixed-wing UAV is evaluated by evaluating the risk factor E. When E reaches the set threshold, the UAV enters emergency evacuation and switches to the vertical flight mode for obstacle avoidance; otherwise, the horizontal flight mode is maintained. The formula for evaluating the risk level coefficient is as follows: ； ； In the formula To assess the risk level factor; is the angle between the speed direction of the drone and the obstacle, which is used to reflect whether the obstacle is approaching the drone. When the obstacle is in the same direction as the drone, there is no obvious threat; when When , it means the obstacle is approaching and poses the greatest threat; is the relative speed between the UAV and the obstacle; is the obstacle speed; , , , is the risk assessment coefficient, the sum of which is 1, and is used to adjust the weight of the impact of various parameters on the risk level; is the safety radius of the obstacle; is the distance between the drone and the center of the obstacle; Represents the obstacle density.

7. The method for trajectory planning and intelligent obstacle avoidance of multiple vertical take-off fixed-wing UAVs as claimed in claim 5, characterized in that: In S3.4, dynamic obstacle avoidance is performed by combining the improved interferometric fluid dynamic system IIFDS algorithm. When there are dynamic obstacles, the obstacles will affect the speed of the UAV. The speed adjustment and position update of a single vertical take-off fixed-wing UAV are expressed as follows: ； In the formula is the interference modulation matrix of the kth obstacle, is the identity matrix, is the normal vector of the obstacle, is the tangent vector of the obstacle, is the obstacle equation, and To exclude parameters and tangential parameters, For the speed of modification, is the speed at time t, is the overall velocity vector of the obstacle, Calculate the next track point, is the sampling time.

8. The method for trajectory planning and intelligent obstacle avoidance of multiple vertical take-off fixed-wing UAVs as claimed in claim 5, characterized in that: Update rewards in S3.4 The reward mechanism for The drones are represented as follows: ； In the formula , , is the reward coefficient, all of which are positive numbers; The distance reward can be expressed as: ； Among them, d0 is the distance between the previous position of the drone and the target position, and d1 is the distance between the current position and the target position; The collision reward can be expressed as: ； The time penalty can be expressed as: ； in is the time step that the UAV has flown.

9. The method for trajectory planning and intelligent obstacle avoidance of multiple vertical take-off fixed-wing UAVs as claimed in claim 5, characterized in that: The steps of controlling and updating the target parameters in S3.5 are as follows: 1) For the extracted small batch samples, use the target network to calculate the target Q value ； In the formula No. The reward of a drone at time t, is the discount factor used to correct the online network, is the current state, For the next joint state, For the status Next action The target state-action value; 2) Update the Critic network parameters of the DDPG algorithm by minimizing the loss function L; the loss function is defined as the mean square error between the current Q value estimate and the calculated target Q value. The loss function is expressed as: ； In the formula For joint action, For joint rewards, Express expectations; 3) A delayed update strategy is adopted for the update of online Actor network parameters. The Actor network parameters are updated by the gradient ascent method. For each vertical fixed-wing UAV, the gradient formula is: ； In the formula is the Q-value function with respect to the action The gradient of is the policy function about The gradient of 4) DDPG adopts a soft update strategy, using the Actor objective function and the Critic objective function to calculate the target value. The update method of the Actor target network and the Critic target network is as follows: ； In the formula is the Actor target network parameter, is the Actor network parameter, is the Critic target network parameter, is the Critic network parameter, is the sliding average coefficient, and its value is less than 1.

Images