A method and system for intelligent global and local path planning of multiple unmanned aerial vehicles

Through the improved two-way search A-star algorithm and deep deterministic strategy gradient algorithm, combined with millimeter wave radar data, the problem of path planning and obstacle avoidance of drones in complex environments is solved, and an efficient and real-time multi-UAV system is realized control.

CN115494866BActive Publication Date: 2024-11-19珠海安擎科技有限公司
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202211160137.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-22
Publication Date
2024-11-19
Estimated Expiration
2042-09-22

AI Technical Summary

Technical Problem

Traditional drone global path planning and local dynamic obstacle avoidance methods are not effective in complex and dynamic unknown environments, and deep learning and reinforcement learning methods have problems such as difficult convergence and poor real-time performance.

Method used

The improved two-way search A-star algorithm combined with the depth deterministic strategy gradient algorithm is used to collect obstacle motion state and drone flight state information through millimeter wave radar, design reward functions, generate local obstacle avoidance strategies, and realize the global and multi-drone drone Intelligent planning of local paths.

Benefits of technology

It improves the track planning and obstacle avoidance efficiency of multi-UAV systems in dynamic unknown complex environments, and realizes continuous action output control with strong real-time and high efficiency, which is suitable for multi-type obstacle environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115494866B_ABST
    Figure CN115494866B_ABST
Patent Text Reader

Abstract

The invention discloses a method and system for intelligent global and local path planning of multiple unmanned aerial vehicles, comprising constructing a grid map, obtaining a global path planning model and performing global path planning by using an improved two-way search A-star algorithm; collecting obstacle motion state information in an environment by using a millimeter wave radar, and obtaining unmanned aerial vehicle flight state information; judging whether there is a conflict, designing a reward function, and generating a corresponding obstacle avoidance strategy; obtaining a state space and an action space, performing obstacle avoidance training by using a deep deterministic policy gradient algorithm, realizing control of unmanned aerial vehicle actions in a continuous action space, avoiding obstacles, obtaining a local path planning model and tracking a global navigation path; each unmanned aerial vehicle performs path planning by using the global and local path planning models at the same time; the invention can improve the track planning and obstacle avoidance efficiency of a multi-unmanned aerial vehicle system, realize control of the continuous action output of multiple unmanned aerial vehicles in a dynamic unknown complex environment, and meet all-weather working conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle technology, and more specifically to a method and system for intelligent global and local path planning for multiple unmanned aerial vehicles. Background Art

[0002] With the continuous progress and development of science and technology, drones have been widely used in military, agriculture, transportation, public management and other fields due to their strong mobility, low cost and convenient operation. The above applications usually involve multiple drones to complete tasks together to improve efficiency. Due to the existence of static and dynamic obstacles in the air, drones need to have high global path planning capabilities and local dynamic obstacle avoidance capabilities to complete the specified tasks. The global path planning and local dynamic obstacle avoidance of multiple drones is one of the important technologies in the field of drones.

[0003] Traditional global path intelligent planning algorithms mainly include A-star search algorithm, particle swarm algorithm, rapidly expanding random tree algorithm and genetic algorithm. Local dynamic obstacle avoidance methods mainly include guidance rate-based obstacle avoidance algorithm, speed obstacle method-based obstacle avoidance algorithm and artificial potential field-based obstacle avoidance algorithm. However, traditional obstacle avoidance methods have great limitations and are not suitable for complex, dynamic and unknown environments. In recent years, one of the hot topics that has attracted much attention is the intelligent obstacle avoidance method combining deep learning and reinforcement learning. However, this method has problems such as high convergence difficulty and poor real-time performance.

[0004] Therefore, how to provide a multi-UAV global and local path intelligent planning method and system is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a global and local path intelligent planning method and system for multiple UAVs, which can improve the track planning and obstacle avoidance efficiency of the multi-UAV system, realize the control of the continuous action output of multiple UAVs in a dynamic unknown complex environment, and meet all-weather working conditions.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A method for intelligent global and local path planning of multiple UAVs, comprising the following steps:

[0008] S1. Construct a grid map based on the starting point and target point of each UAV navigation mission, use the improved two-way search A-star algorithm to obtain the global path planning model and perform global path planning to obtain a reference global navigation path for each UAV;

[0009] S2. Collect the obstacle motion status information in the drone's surrounding environment through the millimeter wave radar, and obtain the drone's flight status information;

[0010] S3. Determine whether to plan a local obstacle avoidance path based on the relative position relationship between the drone and the obstacle;

[0011] S4. According to the obstacle motion state information and the UAV flight state information, determine whether the UAV has a flight conflict with the obstacle, and design a reward function accordingly to generate a corresponding local obstacle avoidance strategy;

[0012] S5. Obtain the state space and action space, use the deep deterministic policy gradient algorithm for obstacle avoidance training, realize the control of the UAV action in the continuous action space, avoid obstacles, obtain the local path planning model and track the global navigation path;

[0013] S6. Perform path planning for each drone in the drone cluster using the global path planning model and the local path planning model at the same time.

[0014] Preferably, the improved two-way search A-star algorithm in S1 includes:

[0015] S11 optimizes path planning;

[0016] Specifically: By adjusting the heuristic function of the A-star algorithm, the shortest path to the destination is searched;

[0017] S12 smoothes the track;

[0018] Specifically: the navigation path is smoothed using a quasi-uniform B-spline curve, and the quasi-uniform B-spline curve adopts a piecewise continuous polynomial.

[0019] Preferably, the obstacle motion state information includes the distance between the obstacle and the UAV, the position information and motion speed information of the obstacle relative to the UAV; the UAV flight state information includes the UAV position information and flight speed information.

[0020] Preferably, when it is detected that the speed direction of the UAV relative to the obstacle and the position direction of the obstacle relative to the UAV are in the same quadrant, local path planning is performed;

[0021] Preferably, the specific method for determining whether a flight conflict occurs between the drone and an obstacle in S4 is:

[0022] Simplify the drone as a mass point A, and regard the obstacle as an obstacle circle ⊙O with a safety radius. The distance between points A and O is d i , line segment OA falls on line l, and the relative velocity vector between the drone and the obstacle is v uioi :

[0023] v uioi =v ui -voi

[0024] Among them, v ui is the velocity vector information of the UAV, v oi is the velocity vector information of the obstacle;

[0025] Get the tangent lines l1 and l2 that pass through point A and are tangent to the obstacle circle. The angle between the straight line l and the tangent line l1 or l2 is α i , relative velocity vector v uioi The angle between it and the straight line l is β i , the radius of the obstacle circle is R i , when β i ≤α i When , there is a flight conflict between the UAV and the obstacle; otherwise, when β i >α i There is no flight conflict.

[0026] Preferably, the state space after the drone interacts with the environment in S5 is:

[0027]

[0028] Among them, u i is the real-time location information of the ith UAV; v ui is the real-time velocity vector information of the i-th UAV; is the real-time heading angle information of the i-th UAV, γ i is the real-time pitch angle information of the drone, o is the real-time position vector information of the obstacle detected by the millimeter-wave radar, and v o is the real-time velocity vector information of the obstacle detected by the millimeter-wave radar, d is the real-time distance vector information between the obstacle and the millimeter-wave radar, p is the real-time position information of the sub-target point on the global path, and g is the position information of the end point of the global path.

[0029] Preferably, the action space after the drone interacts with the environment in S5 is:

[0030]

[0031] Among them, v ui is the real-time velocity vector information of the UAV, is the real-time heading angle information of the UAV, γ i Real-time pitch angle information of the drone;

[0032] v ui , and γ i The range of variation is:

[0033] v ui ∈[vmin , v max ]∩[vv a Δt,v+v b Δt]

[0034]

[0035] γ i ∈[0,π]∩[γ-γ a Δt,γ+γ b Δt]

[0036] Among them, v min and v max is the minimum and maximum speed of the drone, v a and v b is the maximum deceleration and maximum acceleration of the drone in the forward direction, and is the maximum deceleration and maximum acceleration of the UAV at the heading angle, γ a and γ b It is the maximum deceleration and maximum acceleration of the drone at the pitch angle.

[0037] Preferably, the reward function is:

[0038] R(s,a)=R1(t)+R2(t)+R3(t)+R4(t)

[0039] Among them, R1(t) is the reward generated by the change in the distance between the drone and the obstacle. When the drone is within the safe distance from the obstacle, the closer the distance to the obstacle, the smaller the reward. When the drone is outside the safe distance, the reward is a positive constant.

[0040] R2(t) is the reward generated by the speed change when the drone detects an obstacle that threatens the drone's flight. When there is a flight conflict between the drone and the obstacle, a negative reward is obtained; otherwise, there is no flight conflict and a positive reward is obtained.

[0041] R3(t) is the reward generated by the change in the distance between the drone and the navigation mission target point. The closer the distance, the greater the reward.

[0042] R4(t) represents the reward generated by the change in the distance between the drone and the temporary sub-target point. The closer the distance, the greater the reward.

[0043] Preferably, the specific contents of using the deep deterministic policy gradient algorithm for obstacle avoidance training in S5 include:

[0044] The algorithm includes four networks: a policy network based on strategy, a target policy network, a value network based on value, and a target value network;

[0045] The drone gets the current state s through the policy network t The next action a t , interact with the environment to get the next state s t+1 , where the values ​​of state and action satisfy the constraints of the state space and the action space, and the current reward r is calculated by the reward function t and the reward r in the next state t+1 ;

[0046] Save t ,a t ,r t+1 ,S t+1} to the experience pool. When the sample meets the conditions for starting training, N samples are randomly selected from the experience pool to train the network to complete parameter update.

[0047] A multi-UAV global and local path intelligent planning system, comprising a global path planning module, a collection module, a local path planning module and a training module;

[0048] The global path planning module is used to construct a grid map according to the starting point and target point of each UAV navigation mission, obtain a global path planning model and perform global path planning using an improved two-way search A-star algorithm, obtain a reference global navigation path for each UAV, and evenly select a number of sub-target points in the reference global navigation path;

[0049] The acquisition module is used to collect obstacle motion state information in the surrounding environment of the UAV through the millimeter wave radar, and is also used to obtain the UAV flight state information, wherein the obstacle motion state information includes the distance between the obstacle and the UAV, the position information of the obstacle relative to the UAV, and the motion speed information; the UAV flight state information includes the UAV position information and the flight speed information;

[0050] The local path planning module is used to determine whether to plan a local obstacle avoidance path according to the relative position relationship between the UAV and the obstacle, determine whether a flight conflict occurs between the UAV and the obstacle according to the obstacle motion state information and the UAV flight state information, and use the obstacle avoidance strategy of the local path planning model to control the UAV action in the continuous action space, avoid obstacles, and track the UAV to the global navigation path;

[0051] The training module is used to design a reward function, obtain a state space and an action space, use a deep deterministic policy gradient algorithm to perform obstacle avoidance training, realize control of the UAV action in a continuous action space, and obtain a local path planning model;

[0052] The global path planning model and the local path planning model are used to perform path planning for each drone in the drone cluster.

[0053] It can be seen from the above technical solutions that, compared with the prior art, the beneficial effects of the multi-UAV global and local path intelligent planning method and system disclosed in the present invention are:

[0054] (1) The global and local intelligent path planning of drones provided by the present invention for multi-drone multi-type obstacle environments is more flexible than single global or local path planning. Each drone in the drone cluster has the ability of global path planning and local dynamic obstacle avoidance at the same time.

[0055] (2) The present invention realizes the control of the continuous action output of the multi-UAV system through the deep deterministic policy gradient algorithm, and has the advantages of being applicable to dynamic unknown complex environments, strong real-time performance and high efficiency;

[0056] (3) The present invention considers both the UAV flight status information and the obstacle motion status information, determines whether to take obstacle avoidance measures according to the threat level of the obstacle, and designs a reward function accordingly, thereby improving the path planning and obstacle avoidance efficiency of the multi-UAV system;

[0057] (4) The present invention uses millimeter-wave radar to collect environmental data around each drone and obtain obstacle information. Compared with laser radar, it has a wider detection range. Its seeker has a strong ability to penetrate fog, smoke, dust, etc., can work in all weather conditions, has strong adaptability to the environment, and has a wider range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0059] Figure 1 The accompanying drawing is a flow chart of a method for intelligent global and local path planning for multiple UAVs provided by the present invention;

[0060] Figure 2 The accompanying drawing is a schematic diagram of a method for intelligent global and local path planning for multiple UAVs provided by the present invention;

[0061] Figure 3 The accompanying drawing is a schematic diagram of dividing the flight conflict range and the non-flight conflict range of a UAV relative to an obstacle provided by the present invention;

[0062] Figure 4 The accompanying figure is a flow chart of obstacle avoidance training based on a deep deterministic policy gradient algorithm provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The embodiment of the present invention discloses a method for intelligently planning global and local paths of multiple unmanned aerial vehicles, such as Figure 1 and Figure 2 , including the following steps:

[0065] S1. Construct a grid map based on the starting point and target point of each UAV navigation mission, use the improved two-way search A-star algorithm to obtain the global path planning model and perform global path planning to obtain a reference global navigation path for each UAV;

[0066] S2. Collect the obstacle motion status information in the drone's surrounding environment through the millimeter wave radar, and obtain the drone's flight status information;

[0067] S3. Determine whether to plan a local obstacle avoidance path based on the relative position relationship between the drone and the obstacle;

[0068] S4. According to the obstacle motion state information and the UAV flight state information, determine whether the UAV has a flight conflict with the obstacle, and design a reward function accordingly to generate a corresponding local obstacle avoidance strategy;

[0069] S5. Obtain the state space and action space, use the deep deterministic policy gradient algorithm for obstacle avoidance training, realize the control of the UAV action in the continuous action space, avoid obstacles, obtain the local path planning model and track the global navigation path;

[0070] S6. Perform path planning for each drone in the drone cluster using the global path planning model and the local path planning model at the same time.

[0071] In order to further implement the above technical solution, the improved two-way search A star algorithm in S1 includes:

[0072] S11 optimizes path planning;

[0073] Specifically: By adjusting the heuristic function of the A-star algorithm, it is more inclined to search for a path that reaches the destination quickly at the beginning of the search, and more inclined to search for the shortest path to the destination at the end of the search;

[0074] In this embodiment, the improved heuristic function is:

[0075] F(n)=G(n)+w*H(n)

[0076] Where n is the current node, F(n) is the estimated value from the starting point to the target point, G(n) is the estimated value from the starting point to the current node, H(n) is the estimated value from the current node to the target point, and ω is the dynamic coefficient; the value of ω decreases as the diagonal distance from the current node to the target point decreases;

[0077] S12 smoothes the track;

[0078] Specifically: the navigation path is smoothed using a quasi-uniform B-spline curve, which uses a piecewise continuous polynomial;

[0079] The whole curve uses a complete expression, but the intrinsic quantity is segmented. Compared with the Bezier curve at the same order, it has better optimization effect and facilitates the modification of local path planning.

[0080] In one embodiment, each curve segment is determined by four consecutive adjacent track control points to smooth the navigation path.

[0081] In this embodiment, the improved bidirectional search A-star algorithm can greatly shorten the path search time and improve the search efficiency.

[0082] In order to further implement the above technical solution, the obstacle motion state information includes the distance between the obstacle and the UAV, the position information and motion speed information of the obstacle relative to the UAV; the UAV flight state information includes the UAV position information and flight speed information.

[0083] In this embodiment, millimeter-wave radars are arranged on the left side, right side, front side and rear side of the UAV to collect information on obstacles around the UAV. The millimeter-wave radar lead has a strong ability to penetrate fog, smoke, dust, etc., and has high ranging accuracy and a long ranging range, which provides a strong guarantee for the safe flight of multi-UAV systems under complex climate and environmental conditions; the UAV flight status information is obtained through ground station monitoring.

[0084] In order to further implement the above technical solution, when it is detected that the speed direction of the UAV relative to the obstacle and the position direction of the obstacle relative to the UAV are in the same quadrant, local path planning is performed.

[0085] Specific:

[0086] 1. When the obstacle is in front of the right side of the drone, the drone will immediately avoid the obstacle if the following three conditions are met in the track coordinate system: a. The projection of the drone's speed direction relative to the obstacle on the xoy plane is in the first quadrant, b. The obstacle is approaching the drone's flight altitude, c. Within the preset time period, the obstacle will pose a certain threat to the drone's flight; if the above conditions are not met, the drone will fly according to the original track.

[0087] 2. When the obstacle is in front of the left side of the drone, the drone will immediately avoid the obstacle if the following three conditions are met in the track coordinate system: a. The projection of the drone's speed direction relative to the obstacle on the xoy plane is in the second quadrant, b. The obstacle is approaching the drone's flight altitude, c. Within the preset time period, the obstacle will pose a certain threat to the drone's flight; if the above conditions are not met, the drone will fly according to the original track.

[0088] 3. When the obstacle is behind the left side of the drone, the drone will immediately avoid the obstacle if the following three conditions are met in the track coordinate system: a. The projection of the drone's speed direction relative to the obstacle on the xoy plane is in the third quadrant, b. The obstacle is approaching the drone's flight altitude, c. Within the preset time period, the obstacle will pose a certain threat to the drone's flight; if the above conditions are not met, the drone will fly according to the original track.

[0089] 4. When the obstacle is to the right rear of the drone, the drone will immediately avoid the obstacle if the following three conditions are met in the track coordinate system: a. The projection of the drone's speed direction relative to the obstacle on the xoy plane is in the fourth quadrant, b. The obstacle is approaching the drone's flight altitude, c. Within the preset time period, the obstacle will pose a certain threat to the drone's flight; if the above conditions are not met, the drone will fly along the original track.

[0090] In order to further implement the above technical solution, Figure 3 The specific method for determining whether a flight conflict occurs between the UAV and an obstacle in S4 is:

[0091] Simplify the drone as a mass point A, and regard the obstacle as an obstacle circle ⊙O with a safety radius. The distance between points A and O is d i , line segment OA falls on line l, and the relative velocity vector between the drone and the obstacle is v uioi :

[0092] v uioi =v ui -v oi

[0093] Among them, v uiis the velocity vector information of the UAV, v oi is the velocity vector information of the obstacle;

[0094] Get the tangent lines l1 and l2 that pass through point A and are tangent to the obstacle circle. The angle between the straight line l and the tangent line l1 or l2 is α i , relative velocity vector v uioi The angle between it and the straight line l is β i , the radius of the obstacle circle is R i , when β i ≤α i When , there is a flight conflict between the UAV and the obstacle; otherwise, when β i >α i There is no flight conflict.

[0095] Preferably, the state space after the drone interacts with the environment in S5 is:

[0096]

[0097] Among them, u i is the real-time location information of the ith UAV; v ui is the real-time velocity vector information of the i-th UAV; is the real-time heading angle information of the i-th UAV, γ i is the real-time pitch angle information of the drone, o is the real-time position vector information of the obstacle detected by the millimeter-wave radar, and v o is the real-time velocity vector information of the obstacle detected by the millimeter-wave radar, d is the real-time distance vector information between the obstacle and the millimeter-wave radar, p is the real-time position information of the sub-target point on the global path, and g is the position information of the end point of the global path.

[0098] Preferably, the action space after the drone interacts with the environment in S5 is:

[0099]

[0100] Among them, v ui is the real-time velocity vector information of the UAV, is the real-time heading angle information of the UAV, γ i Real-time pitch angle information of the drone;

[0101] v ui , and γ i The range of variation is:

[0102] v ui ∈[v min , v max ]∩[vv a Δt,v+vb Δt]

[0103]

[0104] γ i ∈[0,π]∩[γ-γ a Δt,γ+γ b Δt]

[0105] Among them, v min and v max is the minimum and maximum speed of the drone, v a and v b is the maximum deceleration and maximum acceleration of the drone in the forward direction, and is the maximum deceleration and maximum acceleration of the UAV at the heading angle, γ a and γ b It is the maximum deceleration and maximum acceleration of the drone at the pitch angle.

[0106] In order to further implement the above technical solution, the reward function is:

[0107] R(s,a)=R1(t)+R2(t)+R3(t)+R4(t)

[0108] Among them, R1(t) is the reward generated by the change in the distance between the drone and the obstacle. Within the safe distance between the drone and the obstacle, the closer the distance to the obstacle, the smaller the reward. When the drone is outside the safe distance, the reward is a normal number.

[0109] R2(t) is the reward generated by the speed change when the drone detects an obstacle that threatens the drone’s flight. When there is a flight conflict between the drone and the obstacle, that is, β i ≤α i , a negative reward is obtained; otherwise, there is no flight conflict and a positive reward is obtained;

[0110] R3(t) is the reward generated by the change in the distance between the drone and the navigation mission target point. The closer the distance, the greater the reward.

[0111] R4(t) represents the reward generated by the change in the distance between the drone and the temporary sub-target point. The closer the distance, the greater the reward.

[0112] In this embodiment,

[0113]

[0114] Among them, R1(t) is the reward generated by the change in the distance between the drone and the obstacle. The farther the distance, the greater the reward. The purpose is to make the drone avoid obstacles. r1 is a negative constant, which represents the penalty obtained when the drone enters the obstacle threat area. d o is the distance between the drone and the edge of the obstacle without puffing at different times, D o is the width of the obstacle expansion, D u is the safe distance between the drone and the obstacle, k1 is a negative reward and penalty coefficient, v r is the navigation speed of the UAV relative to obstacles;

[0115]

[0116] Among them, R2(t) is the reward generated by the speed change when the drone detects an obstacle that threatens the drone's flight. The purpose is to make the drone navigate within the range of non-flight conflicts. r2 is a negative constant, which represents the penalty for the drone to navigate within the range of flight conflicts. r3 is a positive constant, which represents the reward for the drone to navigate within the range of non-flight conflicts.

[0117]

[0118] Among them, R3(t) is the reward generated by the change in the distance between the UAV and the navigation mission target point. The closer the distance, the greater the reward. The purpose is to make the UAV deviate from the navigation mission target point as little as possible during the obstacle avoidance process. r4 is a positive constant, which represents the reward obtained by reaching the navigation mission target point. k2 is a positive reward and punishment coefficient, and d g is the distance between the UAV and the navigation mission target point at different times. g <d m1 When , it means that the UAV has reached the navigation mission target point, v u is the navigation speed of the UAV;

[0119]

[0120] Among them, R4(t) represents the reward generated by the change in the distance between the drone and the temporary sub-target point. The closer the distance, the greater the reward. The purpose is to enable the drone to return to the global path planned by the improved two-way search A-star algorithm by tracking the temporary sub-target point on the global path after completing obstacle avoidance. r5 is a positive constant, which represents the reward obtained by reaching the temporary sub-target point. k3 is a positive reward and penalty coefficient, and d p Represents the distance between the UAV and the temporary sub-target point at different times. When d p <d m2 When , it means that the UAV has reached the temporary sub-target point, v u is the flight speed of the drone.

[0121] In order to further implement the above technical solution, the specific contents of obstacle avoidance training using the deep deterministic policy gradient algorithm in S5 include:

[0122] The algorithm includes four networks: a policy network based on strategy, a target policy network, a value network based on value, and a target value network;

[0123] The drone gets the current state s through the policy network t The next action a t , interact with the environment to get the next state s t+1 , where the values ​​of state and action satisfy the constraints of the state space and the action space, and the current reward r is calculated by the reward function t and the reward r in the next state t+1 .

[0124] Save t , a t , r t+1 , S t+1} to the experience pool. When the sample meets the conditions for starting training, N samples are randomly selected from the experience pool to train the network to complete parameter update. When the maximum number of parameter update rounds is reached or the network convergence condition is reached, the parameter update is stopped to obtain the planned local obstacle avoidance path.

[0125] In one embodiment, Figure 4 , the specific content of the deep deterministic policy gradient algorithm for obstacle avoidance training is:

[0126] S51 initializes the parameters θ and η of the policy network π(s; θ) and the value network q(s, a; η); initializes the parameters θ′ and η′ of the target policy network π′(s; θ′) and the target value network q′(s, a; η′); initializes the learning rate ε of the target network, the batch size of each learning sample is N, and the experience pool size is Z;

[0127] S52 uses the strategy network to calculate the current S according to the state space and action space. t Output action a in state t =π(s t ), where the selection of states and actions satisfies the constraints of the state space and action space;

[0128] S53 executes the action and obtains the reward r from the environment according to the reward function t+1 and the next state S t+1 ;

[0129] S54 Save {S t , a t , r t+1 , S t+1} to the experience pool. When the sample meets the conditions for starting training, N samples are randomly selected from the experience pool.

[0130] S55 updates the strategy network and value network:

[0131] y i =r i+1 +γq(s i+1 ,π(s i+1 ,θ′),η′)

[0132]

[0133]

[0134] S56 updates the target value network and target strategy network at intervals:

[0135] η′←εη+(1-ε)η′

[0136] θ′←εθ+(1-ε)θ′

[0137] S57 returns to S52 to loop until the maximum number of rounds is reached or the network convergence condition is met.

[0138] A multi-UAV global and local path intelligent planning system, comprising a global path planning module, a collection module, a local path planning module and a training module;

[0139] The global path planning module is used to construct a grid map according to the starting point and target point of each UAV navigation mission, obtain the global path planning model and perform global path planning using the improved two-way search A-star algorithm, obtain the reference global navigation path of each UAV, and evenly select a number of sub-target points in the reference global navigation path;

[0140] The acquisition module is used to collect the obstacle motion state information in the surrounding environment of the UAV through the millimeter wave radar, and is also used to obtain the UAV flight state information. The obstacle motion state information includes the distance between the obstacle and the UAV, the position information of the obstacle relative to the UAV, and the motion speed information; the UAV flight state information includes the UAV position information and flight speed information;

[0141] The local path planning module is used to divide the flight conflict range and non-flight conflict range of the drone relative to the obstacle according to the obstacle motion state information and the drone flight state information, generate the corresponding obstacle avoidance strategy, and evenly select a number of sub-target points in the reference global navigation path. After the drone completes the obstacle avoidance, the nearest sub-target point is used as a temporary sub-target point and tracked to the reference global navigation path;

[0142] The training module is used to design the reward function, obtain the state space and action space, use the deep deterministic policy gradient algorithm for obstacle avoidance training, realize the control of the UAV action in the continuous action space, and obtain the local path planning model;

[0143] The global path planning model and the local path planning model are used to perform path planning for each drone in the drone cluster.

[0144] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0145] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for intelligent global and local path planning of multiple UAVs, characterized in that: The following steps are involved: S1. Construct a grid map based on the starting point and target point of each UAV navigation mission, use the improved two-way search A-star algorithm to obtain the global path planning model and perform global path planning to obtain a reference global navigation path for each UAV; S2. Collect the obstacle motion status information in the drone's surrounding environment through the millimeter wave radar, and obtain the drone's flight status information; S3. Determine whether to plan a local obstacle avoidance path based on the relative position relationship between the drone and the obstacle; S4. According to the obstacle motion state information and the UAV flight state information, determine whether the UAV has a flight conflict with the obstacle, design a reward function, and generate a corresponding local obstacle avoidance strategy; S5. Obtain the state space and action space, use the deep deterministic policy gradient algorithm for obstacle avoidance training, realize the control of the UAV action in the continuous action space, avoid obstacles, obtain the local path planning model and track the global navigation path; S6. Perform path planning for each drone in the drone cluster using the global path planning model and the local path planning model at the same time; The state space after the drone interacts with the environment in S5 is: Among them, u i is the real-time location information of the ith UAV; v ui is the real-time velocity vector information of the i-th UAV; is the real-time heading angle information of the i-th UAV, γ i is the real-time pitch angle information of the drone, o is the real-time position vector information of the obstacle detected by the millimeter-wave radar, and v o is the real-time velocity vector information of the obstacle detected by the millimeter-wave radar, d is the real-time distance vector information between the obstacle and the millimeter-wave radar, p is the real-time position information of the sub-target point on the global path, and g is the position information of the end point of the global path; The action space after the drone interacts with the environment in S5 is: Among them, v ui is the real-time velocity vector information of the UAV, is the real-time heading angle information of the UAV, γ i Real-time pitch angle information of the drone; v ui , and γ i The range of variation is: v ui ∈[v min ,v max ]∩[vv a Δt,v+v b [Δt] c i ∈[0,π]∩[γ-γ a Δt,γ+γ b Δt] Among them, v min and v max is the minimum and maximum speed of the drone, v a and v b is the maximum deceleration and maximum acceleration of the drone in the forward direction, and is the maximum deceleration and maximum acceleration of the UAV in the heading angle, γ a and γ b is the maximum deceleration and maximum acceleration of the drone at the pitch angle; The specific contents of obstacle avoidance training using the deep deterministic policy gradient algorithm described in S5 include: The algorithm includes four networks: a policy network based on strategy, a target policy network, a value network based on value, and a target value network; The drone obtains the current state S through the strategy network t The next action a t , interact with the environment to get the next state S t+1 , where the values ​​of state and action satisfy the constraints of the state space and the action space, and the current reward r is calculated by the reward function t and the reward r in the next state t+1 ; Save t , a t , r t+1 , S t+1 } to the experience pool. When the sample meets the conditions for starting training, N samples are randomly selected from the experience pool to train the network to complete parameter update.

2. The method for intelligent global and local path planning of multiple UAVs according to claim 1, characterized in that: The improved two-way search A-star algorithm described in S1 includes: S11 optimizes path planning; Specifically: By adjusting the heuristic function of the A-star algorithm, the shortest path to the destination is searched; S12 smoothes the track; Specifically: the navigation path is smoothed using a quasi-uniform B-spline curve, and the quasi-uniform B-spline curve adopts a piecewise continuous polynomial.

3. The method for intelligent global and local path planning of multiple UAVs according to claim 1 is characterized in that: The obstacle motion state information includes the distance between the obstacle and the UAV, the position information of the obstacle relative to the UAV and the motion speed information; the UAV flight state information includes the UAV position information and flight speed information.

4. The method for intelligent global and local path planning of multiple UAVs according to claim 3 is characterized in that: When it is detected that the speed direction of the UAV relative to the obstacle and the position direction of the obstacle relative to the UAV are in the same quadrant, local path planning is performed.

5. The method for intelligent global and local path planning of multiple UAVs according to claim 1, characterized in that: The specific method for determining whether a flight conflict occurs between the drone and an obstacle in S4 is as follows: Simplify the drone as a mass point A, and regard the obstacle as an obstacle circle ⊙O with a safety radius. The distance between points A and O is d i , line segment OA falls on line l, and the relative velocity vector between the drone and the obstacle is v uioi : v uioi =v ui -v oi Among them, v ui is the velocity vector information of the UAV, v oi is the velocity vector information of the obstacle; Get the tangent lines l1 and l2 that pass through point A and are tangent to the obstacle circle. The angle between the straight line l and the tangent line l1 or l2 is α i , relative velocity vector v uioi The angle between it and the straight line l is β i , the radius of the obstacle circle is R i , when β i ≤α i When , there is a flight conflict between the UAV and the obstacle; otherwise, when β i >α i There is no flight conflict.

6. The method for intelligent global and local path planning of multiple UAVs according to claim 1, characterized in that: The state space after the drone interacts with the environment in S5 is: Among them, u i is the real-time location information of the ith UAV; v ui is the real-time velocity vector information of the i-th UAV; is the real-time heading angle information of the i-th UAV, γ i is the real-time pitch angle information of the drone, o is the real-time position vector information of the obstacle detected by the millimeter-wave radar, and v o is the real-time velocity vector information of the obstacle detected by the millimeter-wave radar, d is the real-time distance vector information between the obstacle and the millimeter-wave radar, p is the real-time position information of the sub-target point on the global path, and g is the position information of the end point of the global path.

7. A multi-UAV global and local path intelligent planning system, based on a multi-UAV global and local path intelligent planning method according to any one of claims 1 to 6, characterized in that: It includes global path planning module, acquisition module, local path planning module and training module; The global path planning module is used to construct a grid map according to the starting point and target point of each UAV navigation mission, obtain a global path planning model and perform global path planning using an improved two-way search A-star algorithm, and obtain a reference global navigation path for each UAV; The acquisition module is used to collect the obstacle motion state information in the surrounding environment of the UAV through the millimeter wave radar, and is also used to obtain the UAV flight state information; The local path planning module is used to determine whether to plan a local obstacle avoidance path according to the relative position relationship between the UAV and the obstacle, determine whether a flight conflict occurs between the UAV and the obstacle according to the obstacle motion state information and the UAV flight state information, and use the obstacle avoidance strategy of the local path planning model to control the UAV action in the continuous action space, avoid obstacles, and track the UAV to the global navigation path; The training module is used to design a reward function, obtain a state space and an action space, use a deep deterministic policy gradient algorithm to perform obstacle avoidance training, realize control of the UAV action in a continuous action space, and obtain a local path planning model; The global path planning model and the local path planning model are used to perform path planning for each drone in the drone cluster.

Citation Information

Patent Citations

  • Agricultural plant protection unmanned aerial vehicle and obstacle avoidance and spraying path planning method for same

    CN106774395A

  • Double-layer path planning method for overcoming limitations ofcalculation and storage capacity of UAV (unmanned aerial vehicle)

    CN110456823A

  • Unmanned aerial vehicle obstacle avoidance and path planning method

    CN113110592A

  • Unmanned aerial vehicle conflict detection and release method based on improved self-adaptive adjustment threshold potential field method

    CN113282103A

  • Target tracking-based unmanned aerial vehicle obstacle avoidance method and apparatus, and unmanned aerial vehicle

    WO2021013110A1